Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari
299
Springer Berlin Heidelberg NewYork Hong Kong London Milan Paris Tokyo
T.-J. Tarn S.-B. Chen C. Zhou (Eds.)
Robotic Welding, Intelligence and Automation With 234 Figures and 36 Tables
13
Series Advisory Board A. Bensoussan · P. Fleming · M.J. Grimble · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · J.N. Tsitsiklis
Editors Prof. Tzyh-Jong Tarn Washington University School of Engineering and Applied Sciences Ones Brookings Drive 63130 St. Louis, MO USA
Prof. Dr. Changjiu Zhou Singapore Politechnic School of Electrical and Electronic Engineering Dover Road 500 139651 Singapore Singapore
Prof. Shan-Ben Chen Shanghai Jiao Tong University Institute of Welding Engineering 200030 Shanghai People’s Republic of China
ISSN 0170-8643 ISBN 3-540-20804-6
Springer-Verlag Berlin Heidelberg New York
Cataloging-in-Publication Data applied for A catalog record for this book is available from the Library of Congress. Bibliographic information published by Die Deutsche Bibliothek Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in the Internet at
. This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under German Copyright Law. Springer-Verlag is a part of Springer Science+Business Media springeronline.com © Springer-Verlag Berlin Heidelberg 2004 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by the editors. Final processing by PTP-Berlin Protago-TeX-Production GmbH, Berlin Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper 62/3020Yu - 5 4 3 2 1 0
Preface
Robotic welding systems have been used in different types of manufacturing. They can provide several benefits in welding applications. The most prominent advantages of robotic welding are precision and productivity. Another benefit is that labor costs can be reduced. Robotic welding also reduces risk by moving the human welder/operator away from hazardous fumes and molten metal close to the welding arc. The robotic welding system usually involves measuring and identifying the component to be welded, welding it in position, controlling the welding parameters and documenting the produced welds. However, traditional robotic welding systems rely heavily upon human intervention. It does not seem that the traditional robotic welding techniques by themselves can cope well with uncertainties in the welding surroundings and conditions, e.g. variation of weld pool dynamics, fluxion, solid, weld torch, and etc. On the other hand, the advent of intelligent techniques provides us with a powerful tool for solving demanding realworld problems with uncertain and unpredictable environments. Therefore, it is interesting to gather current trends and to provide a high quality forum for engineers and researchers working in the filed of intelligent techniques for robotic welding systems. This volume brings together a broad range of invited and contributed papers that describe recent progress in this field. This volume is mainly based on the papers selected from the 2002 International Conference on Robotic Welding, Intelligence and Automation (RWIA’2002), December 9-12, 2002, Shanghai, China. We have also invited some known authors as well as announced a formal Call for Papers to several research groups related to welding robotics and intelligent systems to contribute the latest progress and recent trends and research results in this field. The primary aim of this volume is to provide researchers and engineers from both academic and industry with upto-date coverage of new results in the field of robotic welding, intelligent systems and automation. The volume is divided into four logical parts containing twenty-five papers. In Part 1 (6 papers), the authors introduce the recent development of welding automation. Various applications such as vision sensing and control of welding process are discussed. In Part 2 (4 papers), the authors deal with some intelligent techniques for robotic welding. In Part 3 (7 papers), the authors describe their work on intelligent robotic systems which may lead to new development of intelligent robotic welding systems. Finally, in Part 4 (8 papers), the authors introduce some emerging intelligent techniques and their applications, which may contribute significantly to the further development of intelligent robotic welding systems. We would like to thank Professors J.L. Pan, B.S. Xu, S.Y. Lin, H.G. Cai, S.W. Xie, T.H. Song, L. Wu, Y.X. Wu and T. Wang for their kind advice and support to
VI
Preface
the organization of the RWIA’2002 and the publication of this book; to Wenjie Chen, Bing Wang, Wenhang Lei and Surong Zhang for their precious time to devote all RWIA’2002 correspondences and to reformat the most final submissions into the required format of the book; to Jun Ni, research associate of Singapore Polytechnic and PhD candidate of Shanghai Jiaotong University, for his editorial assistance, last but not least to Dr. Thomas Ditzinger for his advice and help during the production phases of this book.
Tzyh-Jong Tarn, Washington University at St. Louis, USA Shan-Ben Chen, Shanghai Jiao Tong University, China Changjiu Zhou, Singapore Polytechnic, Singapore October 2003
Contents
Preface
V
T.-J. Tarn, S.-B. Chen, and C. Zhou
Part 1: Intelligent and Automatic Techniques for Welding Dynamic Process Process Stability of Automated Gas Metal Arc Welding of Aluminum S. Emil
1
Nd:YAG – Laser – GMA – Hybrid Welding in Shipbuilding and Steel Construction U. Jasnau, J. Hoffmann, and P. Seyffarth
14
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics S.B. Chen, D.B. Zhao, Y.J. Lou, and L. Wu
25
Surface Shape Reconstruction of Weld Pool during Pulsed GTAW from Its Single Image D.B. Zhao, J.Q. Yi, S.B. Chen, and L. Wu
56
A Novel Intelligent Monitor for Manufacturing Processes M. Ge and Y. Xu
63
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique Y.H. Liu, D. Ding, and M.L. Lam
80
Part 2: Intelligent Techniques for Robotic Welding Laser Visual Sensing and Process Control in Robotic Arc Welding of Titanium Alloys H. Luo and X.Q. Chen
110
VIII
Contents
Intelligent Technologies for Robotic Welding S.B. Chen, T. Qiu, T. Lin, L. Wu, J.S. Tian, W.X. Lv, and Y. Zhang
123
The Seam Tracking System for Submerged Arc Welding H. Zhang, X. Ding, M. Chen, B. Da, and C. Zou
144
Visual Measurement of Path Accuracy for Laser Welding Robots D. Du, B. Sui, Y.F. He, Q. Chen, and H. Zhang
152
Part 3: Intelligent Robotic Systems Map Building and Localization for Autonomous Mobile Robots Y.L. Ip, A.B. Rad, and Y.K. Wong
161
Control and Stabilization of the Inverted Pendulum via Vertical Forces D. Maravall
190
Vision Based Motion Planning of Humanoid Robots M.F. Ercan and C. Zhou
212
Evolution of Locomotion Controllers for Legged Robots A. Boeing and T. Bräunl
228
Gait Planning for Soccer-Playing Humanoid Robots Z. Tang, C. Zhou, and Z. Sun
241
Fuzzy-Neural Impedance Control for Robots Z. Xu and G. Fang
263
On Singular Perturbation Based Inverse Dynamics Control for a Two-Link Flexible Manipulator X. Dai, L. Sun, and H. Cai
276
Part 4: Intelligent Techniques and Its Applications Wavelets and Biorthogonal Wavelets for Image Compression D. Hong, M. Barrett, and P. Xiao
281
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay F.H. Ho, A.B. Rad, Y.K. Wong, and W.L. Lo
304
Contents
Adaptive Robust Fuzzy Tracking Control for a Class of Nonlinear Systems Y. Yang and C. Zhou A Complete Algorithm for Attribute Reduction B. Wang and S.B. Chen
IX
327 345
Preliminary Probe of Embedded Real-Time Operating System and Its Application in Welding Control W. Chen, S. Chen, and T. Lin
353
Estimation of Oxygen Concentration Using Dynamic Recurrent Neural Network Based on PCA D. Sun, F. Ye, and X. Gu
358
Robust Stability of Interval Lur'e Systems: A Bilinear Matrix Inequality Approach J.T. Sun
365
Thresholding Segmentation and Classification in Automated Ultrasonic Testing Image of Electrical Contact H.D. Chen, Z.Y. Cao, Y.W. Wang, and J. Xue
372
Author Index
381
Process Stability of Automated Gas Metal Arc Welding of Aluminium Emil Schubert Alexander Binzel Schweisstechnik GmbH & Co. KG, Gießen/Germany, Kiesacker 7-9, D-35418 Buseck, Germany [email protected]
Abstract. For modern transport systems Aluminum is getting of growing importance. To fully utilize this potential economic, effective and stable joining technologies have to be available. However, the special features of Aluminium materials like high heat conductivity, oxide formation, pore and hot cracking susceptibility require special measures for high quality welding especially for thin sheet metals. New developments in gas metal arc welding power sources, torches and wire feeders for Aluminum welding to overcome these problems will be described. Besides the technological factors system aspects and their influence an process stability and manufacturing up-time will also be discussed. Actual and potential future applications of these new developments will be demonstrated.
1 Introduction The advantages of the material aluminium (low density and high stability at the same time) are more and more used as a possibility for weight reduction, especially in the field of transportation. Fig. 1. shows the use of aluminium in automobiles during the period from 1975 until today and gives an outlook to the year 2005 showing that the employment of aluminium will become twice as large until then [8]. However, in order to produce automobiles out of sheets, profiles and cast parts suitable joining processes have to be available to enable an effective, efficient joining of the aluminium materials with sufficient process stability. Gas metal arc welding processes such as the GMA , TIG- and Plasma welding process can be used in this field for producing sealed and continuous welding seams [6]. In addition the combination of these processes with the Laser welding process can be used to realize new possibilities for welding structures [3]. Especially in the field of thin sheets the handling of the soft filler metals as well as the minimized heat input represent the most important challenge for the gas metal arc welding processes. Additionally, the system aspects (i.e. integration of the welding process into the production cells including torch change, cleaning stations, robot control etc.) play an important role in the field of production technology. T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 1−13, 2004. Springer-Verlag Berlin Heidelberg 2004
2 2 E. Schubert
80
74 68
70
63
60
55 steel and iron elastomers / others polymers Aluminium other non-ferrous metals
50 40 30 20 10
13 6
1413
13 10 3 4
5 4
6 4
1415 10
6
0 1975 Year of construction
1985
1995
2005
Fig. 1. Employment of aluminium alloys in the automotive industry
Therefore, possibilities to increase and maintain the process stability during gas metal arc welding will be described in the following considering the special characteristics of the aluminium materials, and examples for applications will be given for explanation. Starting point are new developments in the field of welding power sources and torch technology which will open the field of thin sheets for the inert-gas welding in the future. Another core component is the feeding of the filler metal as well as the system integration of all components into the industrial manufacturing cell as already mentioned above.
2 Potentials and Problems of Aluminium Welding The low density of 2,7 g/cm³ at stabilities of up to 300 N/mm² (depending from the type of alloy) are opening for the aluminium alloys a large potential for application in the field of light-weight construction. Due to other physical characteristics (high heat conductivity, hydrogen absorption, large interval of solidification) and without going too deep into detail regarding these special characteristics - problems such as pipe cavities, pores and hot cracks as well as thermal distortions are occurring very frequently when welding these materials. To minimize the distortion problem it is generally possible to weld with an energy per unit of length as low as possible and to distribute the heat input by suitable welding cycles. Especially in the field of pulsed arc welding the heat input can be minimized (see chapter 3). The problem of pore formation [2] is caused by the high affinity of aluminium towards hydrogen. The hydrogen is dissolved in the melt pool and will be
Process Stability of Automated Gas Metal Arc Welding of Aluminium
33
segregated due to the considerably lower solubility in the solid aluminium, accumulating in form of pores. To avoid this problem the hydrogen has to be prevented from dissolving on one side and there has to be sufficient time and place for the hydrogen to degas in case that it has dissolved into the pool. Hydrogen gets into the pool e.g. through the formation of hydroxide layers in case of oxidized surfaces in a wet atmosphere or by dirt or remainders from greases and oils. Another possibility is the direct entering through the humidity in the air in case of insufficient gas-shielding or sputtering wire feeding. Measures to prevent pores are the use of inert shielding gases, dry stocking of work pieces and wires and a removal of the oxide layer by brushing, pickling or milling of edges. Besides the interaction of oxides with oxygen, oxides have another negative characteristic: due to the considerably higher melting temperature compared to the base metal (2050 ° C resp. 66 °C) notches can occur at the upper and the root bead or oxides can sink into the pool [1]. Measures for degasification are especially an adapted welding speed to give the gas pores time to rise as well as the creation of degasification areas (bevel opening angles as large as possible) where ever the design allows. In additon, air gaps between the sheets, e.g. through distance sheets can improve the degasification. The problem of hot crack formation is especially occurring with alloys having a large solidification interval where the elements Mg, Si and Cu are critical [2]. Reason for this is a too small quantity of intergranular eutectic during the solidification process accompanied by shrinkage stress. Countermeasures are the use of filler metal to bring the composition of the weld metal out of the crackcritical area, a suitable design enabling a shrinkage and an adapted weld heat input (not too high solidification speeds).
3 Process Stability of Welding of Aluminium Materials
3.1 Welding power source- and welding torch technology The GMA-, TIG- and Plasma welding process are suitable for the gas metal arc welding of aluminium.
GMA-Welding While welding of thick plates is preferably done using spray arc, the pulsed-arc welding is recommended for the welding of thin sheets. Fig. 2. is a current – time diagram for a typical pulsed-arc process. With the growing application of the inverter technology new possibilities are arising for the inert-gas welding process. The GMA-process too, having so far
4 4 E. Schubert
only limited access into the industrial production of structural components out of aluminium, is profiting from these developments. pulsed current non-short-circuiting metal transfer
current
medium current
background current
pulse time actual welding time
time
Fig. 2. Current-time-characteristic for impulsed welding of aluminium alloys
pulse voltage background current
neg.current time negative background current
pulse time pulse period time
(pulse frequency)
Fig. 3a. MIG-AC-Pulsating of voltage
Process Stability of Automated Gas Metal Arc Welding of Aluminium
55
pulse voltage background current
off-time
pulse time pulse period time
(pulse frequency)
Fig. 4b. MIG-DC-Pulsating of voltage
These process variants enable a process-stable welding of structural components out of aluminium up to a thickness of 0,6 mm. Fig. 4. shows the crosssection of a fillet weld with the clearly visible minimum penetration. Fig. 5. Cross-section of a fillet weld of AIMgSi welded by using the MIG-AC process
But also regarding conventional GMA welding of aluminium materials special requirements on the torch technology are arising, especially when using pulsed welding. A torch with a double-cooling circuit enables stable welding processes even at high pulse currents due to an optimal cooling of the torch neck, contact tip and gas nozzle. Fig. 5. shows the sectional drawing of such a torch with two cooling circuits.
6 6 E. Schubert
Fig. 6. Sectional drawing and photo of a double cooling-circuit-GMA-welding torch for impulse welding
io n
el ec tro ns
s
electrode at positive pole
ga s
ga si
on s
electrode at negative pole
type of penetration
Fig. 7. TIG-welding with negative and positive polarity at the electrode
el ec tro ns
Process Stability of Automated Gas Metal Arc Welding of Aluminium
77
TIG Welding Process The TIG-AC welding can be recommended for the TIG welding of aluminium[10]. Fig. 6. shows the TIG welding with different polarities at the electrode. The electrode is cooling down during the minus phase enabling a deep penetration while during the plus phase a cleaning effect occurs as the oxide layer is torn open by gas ions. The penetration during this phase is more flat and wider and the thermal load of the electrode is very high. In practice therefore, ACwelding is applied in order to combine the advantages.
Plasma Welding The Plasma welding is offering possibilities similar to the TIG welding with the difference of a stronger arc-constriction by the so-called plasma gas [9]. Additional advantages are the low heat input, low quantity of spatter and the very smooth seams. Fig. 7. is a diagram showing an industrial plasma torch used for aluminium welding. The plasma plus-pole torch with a rating above 200 A is under development [5] and promises a large application potential.
back cap
Plasma gas
upper cooling circuit
cooling body tungsten electrode gas diffusor
shielding gas
lower cooling circuit ceramic shielding gas nozzle Plasma nozzle
Fig. 8. Industrial plasma welding torch The conventional inert-gas welding process stands in competition to the laserbeam welding and the hybrid processes. While the laser processes are superior in
8 8 E. Schubert
12 P a ra m e te rs : b u t t j o in t , 1 m m L a s e r: 2 ,5 k W c o n v e n t io n a l w e ld in g : s ta n d a rd p a ra m e te rs
welding speed m/min
10 8 6 4
s ta in le s s s te e l s tru c tu ra l s te e l
2
A lu m in iu m
0 HLDL
L a s e r (C O 2 , Y A G )
P la s m a
s tru c tu ra l s te e l
energy per unit length kJ/cm
A lu m in iu m
2 1 ,5 1 0 ,5 0 L a s e r (C O 2 , Y A G )
admissible tolerance
zul äs sig e To ler an z in m
W I IG G T
P a ra m e te rs : P a r a m e t e r : S tu m p fs to ß , 1 m m b u t t jo in t, 1 m m L a s e r: 2 ,5 k W L a s e r, 2 ,5 k W k co on nv v. eSncthiowneai lß wv ee rl df ai nh gr e: n : g sä tna gn idg ae r d A rpbaeriat smp a e rt ea rms e t e r
s t a i n le s s s t e e l
2 ,5
M IG /M A G
HLDL
P la s m a
M IG /M A G
W T I IGG
gap dim. tolerance Spaltmaßtoleranz
Positional tolerance ofder power source Positionstoleranz S 1,25 1,00 0,75 0,50 0,25 0,00 Laser (CO2, YAG)
HLDL
Plasma
WIG TIG
MIG/MAG
Fig. 9. Comparison of GMA-, TIG, Plasma- and Laser welding processes
Process Stability of Automated Gas Metal Arc Welding of Aluminium
99
welding speed and energy input, the inert-gas welding processes offer enormous advantages on the cost side. The fig. 8 a) to c) show a comparison of the possible welding speeds, energy inputs per unit of length and the gap bridging abilities. The diagrams show the especially large potential of the plasma welding technology as a bridge between GMA- and TIG-welding on one side and the laser technology on the other side. 3.2 Wire Feeding The feeding of soft, easy deformable wires with low stability and low resistance to wear and the risk of oxide formation and humidity absorption is a great challenge for the wire-feeding systems. One solution, already proven in industrial use, is a system consisting of a heated wire-spool housing, a wire spool with drive motor and a digitally controlled 4-roller drive inside the housing, a cable assembly and a second digitally controlled 4-roller drive directly mounted to the torch, as shown in fig. 9.
Fig. 10. Wire feeding equipment for Al-wires (Aluminium Power Drive: APD)
Both 4-roller drives are running absolutely synchronous as push-push drive and a certain wire quantity is always available in the cable assembly as reservoir. The rollers are preset and only require very small pressing forces as the wire roll is driven separately so that only minimum wear or deformation will occur at the wire. In addition, this is also minimizing the tensile forces acting on the wire. The heated housing keeps the temperature above dew point guaranteeing a dry wire with uniform oxide layer without humidity so that a welding with constant processing parameters and without the danger of pore formation due to oxide or hydrogen from the wire.
10 10 E. Schubert
3.3 System Aspects The integration into complete production cells plays an important role for the extensive use of the single components. The power source, the welding torch, the wire-feeding unit and the welding robot should produce trouble-free in a production cell with highest-possible equipment availability and stability. As especially at the welding torch and during the wire feeding wear and dirt accumulation is occurring – e.g. by weld spatter and worn products - which cannot be avoided, peripheral devices have to enable an automatic cleaning or an exchange of the components in questions, if possible. To avoid damages at the welding torch in case of collision, emergency shut-down devices have to be used. Modular systems are especially suitable for frequently changing welding jobs. Fig. 10. shows a concept of a wire-feeding system for different inert-gas welding processes, the so-called Master-Feeder-System. The user is selecting the suitable wire bunch (small coil, large coil, barrel, etc.) and welding process (GMA, TIG, Plasma, Laser or Hybrid) depending on the welding job. The required filler metal is fed by the first 4-roller drive after the separately driven wire bunch and from there transported through a cable assembly to the second 4-roller drive situated close to the process. The wire is then led into a GMA torch or is made available for other processes as cold or hot wire. Fig. 11. shows a processing head for LaserGMA welding.
Process -depending wire feeding :
MIG torch
Adjustment of wire packages :
barrel spool
Cold-/hot wire:
large spool
TIG
small spool
Plasma Laser
Fig. 11. Modular concept for wire feeding with different welding processes (MIG, TIG, Plasma, Laser)
Process Stability of Automated Gas Metal Arc Welding of Aluminium
11 11
Fig. 12. Laser-GMA processing head
4 Examples for Application Since 1999, the automatic inert-gas welding of structural components out of aluminium in combination with the wire-feeding concept (APD) described is used in the production of the AUDI A 8. Different seams at the wheel arch have been welded using in this process and equipment. The use of the APD system requires minimum intervention by the operator (change of spool once a week) and thus enables an economic production. The proven APD system could also be used for the GMA welding in the production of the AUDI A2. Here too, this system is used for welding jobs at the wheel arch with 17 welding units in total, see fig. 12.
12 12 E. Schubert
Original component
Process picture
GMA weld seam
Fig. 13. Al-wheel arch of the Audi A2 welded with APD
Besides this application, the wire feeding technology described above is successfully implemented as Laser cold-wire feeder at the successors of A2 and A8. Fig. 13. is showing the cold-wire feeding in detail. To observe the narrow process tolerances an additional positioning possibility is existing in x-, yand z direction. The cold-wire feeding can be adjusted to any focus length and interfering contours of laser processing heads.
Fig. 14. Laser cold-wire feeding system
Process Stability of Automated Gas Metal Arc Welding of Aluminium
13 13
5 Conclusions The material aluminium is making special demands to the welding technology. Due to new developments in the field of power sources, adapted welding torch designs and new wire-feeding units these demands can be met economically as well as regarding process stability.The integration of these components as a system into the production cells allows to carry out welding jobs with high process stability and equipment availability. The modular concept presented for wirefeeding has certainly a large future potential for feeding wires difficult to handle for the whole range of inert-gas joining processes. This does not only apply to the material Aluminium but also to the welding and soldering of other light metals as well as the proven steel materials and their developments, the high strength and super-high strength steels. In addition, applications in the field of Laser as well as hybrid processes become possible so that here too, improvements in process stability can be expected.
References [1] [2] [3]
[4]
[5] [6] [7] [8] [9] [10]
Baum L, Fichter V (1999) Der Schutzgasschweißer. DVS-Verlag, Düsseldorf Dilthey U (1995) Schweißtechnische Fertigungsverfahren. VDI-Verlag, Düsseldorf Dilthey U, Lüder F, Wieschemann A (1999) Erweiterte Möglichkeiten beim Schweißen von Aluminiumlegierungen durch den Laser-MIG-Hybridprozeß. Aluminium; Bd. 75, Nr. 1/2, pp 64-75 Goecke SF, Dorn L (2000) MAG-Chop-Arc-Schweißen für Feinstblech > 0,2 mm. Große Schweißtechnische Tagung, 2000, Nürnberg, DVS-Verlag, Düsseldorf, pp 163-1682. Kabatnik L (2000) Plasma-Pluspolschweißen. Vortragsmanuskript Große Schweißtechnische Tagung 2000, Nürnberg,DVS-Verlag, Düsseldorf Klock H, Schoer H (1977) Schweißen und Löten von Aluminiumwerkstoffen. DVS-Verlag, Düsseldorf Nickel HG (2000) MIG-MAG DC-O und AC-0-Spannungspulsen. Pers. Mitteilung, Nisterberg N.N. (2000) Werkstoffverteilung beim PKW in Deutschland. VDI-Nachrichten, Düsseldorf, pg 41 Radscheit C (1999) Plasmalöten - Grundlagen und Einsatz; Neue Löttechnologien für den Dünnblechbereich. SLV München Zahariev S, Herold H (1996) Besonderheiten der Anwendung des WIG-Wechselstromschweißens von Aluminium unter Nutzung modifizierter Halbwellen. DVS-Berichte 176, pp 34-37
14
Nd:YAG – Laser – GMA – Hybrid Welding in Shipbuilding and Steel Construction Ulf Jasnau, Jan Hoffmann, and Peter Seyffarth Welding Training and Research Institute Mecklenburg – Western Pomerania Ltd., Rostock, Germany [email protected]
Abstract. The paper deals with the Nd:YAG-laser – GMA – hybrid welding process for the use in shipbuilding and steel construction. The advantages of a laser – arc – hybrid welding process in comparison with a laser alone or an arc alone welding process are shown. The current possibilities of the Nd:YAG-laser – GMA – hybrid welding process are shown by some results from hybrid welding of steel and aluminium alloys. Furthermore the advantages of the Nd:YAG-laser technology, especially for a simple and flexible automation are described. The Nd:YAGlaser – GMA – hybrid welding is a process which demands a higher degree of automation in shipbuilding. Regarding to the advantages and related to the available Nd:YAG-laser sources a perspective for the use of the Nd:YAG-laser technology in shipbuilding and steel construction is pointed out. A current German research project under participation of the authors is presented in the paper. One aim of this project is to develop a system for the automatically programming of robots and other welding systems in shipbuilding. Therefore the new programming tool should use the real geometrical data of the component before welding, resulting from a image processing tool. Concluding, the first manufacturing system in shipbuilding (a welding gantry for 12 meter T-joints) outfitted with Nd:YAG-laser – GMA – hybrid welding equipment at a German shipyard is presented. The retrofitting of this welding gantry has been planned and accompanied by the company the authors are with.
1 Introduction The process of laser beam welding is today a well established joining process in industrial applications. The advantages, e.g. deep and narrow welds with high speed and low heat input are accompanied from some disadvantages like the necessary precise preparation of the edges of parts to be joined and the precise clamping of the work pieces. Especially in shipbuilding and steel construction the advantages of the laser technology are to be used only with high costs and efforts due to the boundary conditions in the production line. With the coupling of the laser beam and the arc T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 14−24, 2004. Springer-Verlag Berlin Heidelberg 2004
Nd:YAG - Laser - GMA - Hybrid Welding in Shipbuilding and Steel Construction
15 15
it is possible to extrude the range of application of the laser technology and their advantages. Since the fundamentals of a laser-arc-hybrid process have been described in the 70th [1] different options of this technology were studied, like the laser-GMA-, the laser-TIG- or the laser-plasma-process. In the most cases of this variations a CO2laser was used as a laser source because of the higher laser power and better beam quality. With the now available higher laser power up to 6 kW in case of Nd:YAG-laser this laser source will be more interesting for the laser-arc-hybrid welding in shipbuilding and steel construction. The gas-metal-arc welding is today the preferred welding process in shipbuilding and steel construction in Germany. Therefore and because of some important advantages of the Nd:YAG-Laser the coupling of this type of laser with the GMA-welding process will be contemplated.
2 Process Fundamentals The fundamentals of the coupled process are nearly the same for both, the CO2laser and the Nd:YAG-laser. In Fig. 1 the general principle of the process is shown. Laser process and arc process have a common process zone and weld pool. Only one shielding gas flow is necessary, mostly the gas flow is realized via the GMA-torch.
laser beam
laser induced metal vapor protective gas cover
welding electrode arc fusion zone
keyhole
workpiece
welding direction
Fig. 1. Principle of the laser-GMA-hybrid welding process
The superposition of the arc and the laser leads to different interaction between the two processes. The stabilization of the arc under the influence of laser radiation or a laser induced plasma is one of the main advantages. The wandering of the arc is prevented, because the laser beam evaporates material and ionizing the at-
16 16 U. Jasnau, J. Hoffmann, and P. Seyffarth
mosphere, producing good conditions for ignition and burning of the arc. So the transfer of the energy of the electric arc is more efficient and the thermal effect of the coupled process is in excess of a simple sum of the effects of the laser and arc heat sources taken separately [2]. Depending of the energy ratio of the two energy sources the character of the coupled process may be either more arc-like or more laser-like. The welding depths increases with the increase of laser energy whereas the weld width increases with the arc energy. For a given sheet thickness to be joined the proportion between the “laser part” and the “arc part” of the hybrid weld pool give us an estimation about the process character. The transverse sections in Fig. 2. are showing an example for such different process characters.
Fig. 2. Comparison between a more laser-like (left) and a more arc-like (right) laser-GMAhybrid weld (sheet thickness 5 mm, I-groove)
Depending on the welding task the one or the other process character may be preferable. For the control of the process character and the utilisation of the advantages of the coupled process (Fig. 3) the effects of a great amount of welding parameters have to be known. The effect of the laser beam in a hybrid weld is well illustrated by comparing transverse sections of GMA welds and hybrid welds with the same welding speed (Fig. 4. ). The welding depths increases from 1,5 mm to 5,1 mm using a 4 kW Nd:YAG-laser beam. Despite the high welding speed the quality of the top bead shape is good in case of hybrid welding, resulting from the laser-stabilized arc. The effect of the arc, resulting in a better gap bridging ability and a better weld geometry then in a pure laser weld is illustrated in Fig. 5. The addition of the arc power and the process stabilisation in hybrid welding leads to a significant higher welding speed. So the heat input per unit of length remains almost constant in comparison with the laser alone welding process.
Nd:YAG - Laser - GMA - Hybrid Welding in Shipbuilding and Steel Construction
arc
laser
• low cost energy source • gap bridging ability • filler material • influence of structure
• deep weld penetration • high welding speed • low heat input
17 17
hybrid process • better gap bridging ability, compared with laser welding • higher welding speeds, compared with arc and laser • deeper weld penetration, compared with arc and laser • lower heat input compared with arc-welding
Fig. 3. Coupling of the advantages of arc and laser welding in a hybrid process
sheet thickness: 8 mm laser type: Nd:YAG laser power: ~ 4 kW without laser
with laser
welding speed: 2 m/min shielding gas: 82% Ar / 18% CO2
Fig. 4. Transverse sections of a GMA-weld and a laser-GMA-hybrid weld
sheet thickness: 5 mm I-groove laser type: Nd:YAG laser power: ~ 4 kW shielding gas: 90% Ar / 10% CO2 laser weld,
laser-GMA-hybrid weld,
welding speed: 1,1 m/min
welding speed: 2,5 m/min
Fig. 5. Transverse sections of a laser weld and a laser-GMA-hybrid weld
18 18 U. Jasnau, J. Hoffmann, and P. Seyffarth
Besides the parameters laser power and arc power (arc voltage and arc current) and welding speed the geometrical formation of the laser beam and the torch and also the type of shielding gas are very important. The arbitrary geometrical formation of the laser beam and the torch (torch before or behind or besides the beam in welding direction) and the possibility to couple more than one GMA-torch with the laser beam offers a great variety of geometrical formations for the solution of welding tasks. The usable shielding gas is one of the biggest difference between the hybrid welding with CO2-laser and Nd:YAG-laser. For the work with the CO2-laser gas mixtures with a high Helium content are necessary. In case of Nd:YAG-laser also Argon and active gases, e.g. mixtures from Ar, CO2 and O2 are usable. So it is possible to influence the metallurgy of the weld pool like in case of pure GMA welding. Pure Argon and active shielding gases are not usable for hybrid welding with a CO2-laser, because of the risk of ignition of an protection plasma above the weld pool. This plasma interrupt the absorption of the laser radiation into the work piece and the hybrid welding process discontinues. This disadvantage of the CO2laser is caused in the different wave lengths of the radiation of the two laser types mentioned above. The CO2-laser with a wave length of 10,6 µm offers other optical conditions then the Nd:YAG-laser with a wave length of 1,06 µm. The practical qualification of the Nd:YAG-laser-GMA-hybrid welding for different shipbuilding applications is the main content of two research projects the authors company is involved. Besides the welding of constructional and shipbuilding steel also the welding of aluminium alloys for shipbuilding has to be done in one of the projects. In a lot of welding tests the hybrid welding of square butt joints and T-joints of the mentioned materials is carried out under laboratory conditions. Welding tests at a real welding gantry at the shipyard “Kvaerner Warnow Werft” in Rostock, Germany, for the qualification of the hybrid welding process also have been started. Despite the high welding speeds reachable with the coupled process the technical and economical effects of the new welding process are not only resulting from the higher welding speed compared with GMA-welding. The main aspect of the positive effects is the lower energy per unit of length, leading to lower distortions and less postweld straightening. Some results of the welding tests are shown in Fig. 6 and Fig. 7. For aluminium alloys the laser-GMA-hybrid welding offers an additional advantage over the pure laser welding compared with the welding of steel. During laser welding of aluminium alloys the keyhole is very instable due to the lower viscosity of the aluminium melt versus steel melt. This leads to numerous collapses of the keyhole and to a lot of big pores, so called process pores. The outgassing of these pores are very difficult due to the very small weld pool and short cycle time in case of laser welding. The phenomenon of process pores is limiting the welding depths in case of laser welding of aluminium. To solve this problem some strategies were developed, e.g. the welding with two laser spots for the enlargement of the weld pool. In principle all these strategies leads to a higher necessary laser power at constant welding depths.
Nd:YAG - Laser - GMA - Hybrid Welding in Shipbuilding and Steel Construction
19 19
The laser-GMA-hybrid welding is a good possibility to realise the higher power at the working piece. With the coupling of the laser beam and the arc high welding depths and high welding speeds are also possible on aluminium alloys without the danger of process pores. The bigger weld pool geometry and the modified flow conditions in the melt allows the outgassing of pores. Furthermore the override of the metallurgy of the weld with the melting filler wire is much better compared with pure laser welding.
root formation
transverse section Parameters: sheet thickness:
5 mm
current type/ polarity /arc type: DC / + / pulsed arc
weld type:
I-groove
arc current / voltage:
290 A / 29 V
welding position:
PA
wire feed rate:
12500 mm / min
laser type:
Nd:YAG
arc power:
~ 8,4 kW
laser focal length:
200 mm
welding speed:
3000 mm / min
laser power:
~ 4 kW
shielding gas:
12 l /min (Ar / O2)
Fig. 6. Laser-GMA-hybrid weld on constructional steel
20 20 U. Jasnau, J. Hoffmann, and P. Seyffarth
1 cm typical top bead shape
transverse section Parameters of the weld (transverse section): sheet thickness:
5 mm
current type/ polarity /arc type:
DC / + / pulsed arc
weld type:
I-groove
arc current / voltage:
162 A / 24 V
welding position:
PA
wire feed rate:
12.000 mm / min
laser type:
Nd:YAG
arc power:
~ 4,0 kW
laser focal length:
200 mm
welding speed:
2500 mm / min
laser power:
~ 4 kW
shielding gas:
45 l /min (Ar / He)
Fig. 7. Laser-GMA-welds on aluminium alloy 5083
3 The Equipments of Production Lines
3.1 Theoretical Aspects The possibility for transferring the Nd:YAG-laser beam via a flexible laser light cable is a second big advantage of the Nd:YAG-laser caused in the lower wave length. In contrary, the CO2-laser beam must be transferred via a mirror system to the working point. Laser light cables and laser working heads are manageable by industrial robots, a lot of such solutions in automobile manufacturing are making these solutions to state of the art.
Nd:YAG - Laser - GMA - Hybrid Welding in Shipbuilding and Steel Construction
21 21
But the flexible beam transfer in connection with the higher available laser power of modern Nd:YAG-laser systems up to 6 kW makes the use of this laser type also very interesting for production lines in shipbuilding and steel construction. In case of using a Nd:YAG-laser only the working head has to fulfil the high requirements on the dynamical behaviour and not the whole beam transfer way, the complete machine tool respectively, as in case of using a CO2-laser. A common welding gantry or another welding machine, respectively the point of the working head, should be fulfil the requirements according to [3] as a supposition for refitting with a Nd:YAG-laser source. In connection with the laser-GMA-hybrid welding process the advantages of laser technology are usable for conventional production lines. Nevertheless the Nd:YAG-laser-GMA-hybrid welding demands a higher degree of automation in the production line. A precise seam tracking system including measuring of the gap size and the control of laser- and / or arc-power is an elementary equipment. The control system of the welding gantry must be able to control higher welding speeds than in case of pure GMA-welding and a lot of new functions, connected with the use and control of the laser system. But the effort for refitting a conventional welding gantry with Nd:YAG-laser equipment is much lower than the purchase of a new special designed laser welding gantry. Another advantage of the Nd:YAG-laser sources is the possibility to connect one laser source with up to 6 handling systems and to add the radiation of two or three laser sources into one laser light cable. So also with solid state lasers a power of more than 12 kW at the work piece is possible. As already realised in the automobile industry so called “lasernetworks” with several laser sources and several handling systems are also thinkable for production lines in shipbuilding and steel construction. Such a “lasernetwork” can be installed “step by step”, regarding to the actual demands of the shipyard. Also a occasionally insert of laser technology for special manufacturing tasks is possible. This flexibility is only caused in the type of laser source, the Nd:YAG-laser and the possibility to transfer the laser beam via laser light cable. In the two already mentioned German research projects the predicting problems with the use of Nd:YAG-lasers in shipbuilding, e.g. the occupational safety and health and the necessary higher level of control systems and automation should be solved. One of these projects is going on in a regional network of the authors company and two other research companies, attended by several German shipyards and related firms. Besides the qualification of Nd:YAG-laser-GMA-hybrid welding and problems regarding to safety and control of the laser system the aim of this project is to develop a system for the automatically programming of robots and other welding systems in shipbuilding. Therefore the new programming tool should use the real geometrical data of the component before welding, resulting from a image processing tool. Thus the expensive manual work for the programming of welding machines and the dependence from not up-to-date CAD-data should be replaced in the future. Further on with such a automatically programming tool it is possible to react without delay on short-term changes in the manufacturing tasks, the flexibility and reactivity of the production line increases.
22 22 U. Jasnau, J. Hoffmann, and P. Seyffarth
For the automatically programming the data resulting from the image processing tool are independent from any kind of machine control system and should be therefore usable for all programmable welding machines in a production line. The further processing of the data should be realized with a general software solution, usable for the different programmable robots and welding machines in a production line. Despite this great variability of the planned system an adaptation to existing machines and manufacturing lines will be necessary. Additionally to the specific adaptation of programming tools and laser-GMAhybrid welding equipment the theoretical and practical training of the staff is a focus point in the project of the three incorporated research companies. 3.2 Practical Applications The first industrial application of the laser-GMA-hybrid welding process in a shipyard was realized with a CO2-laser (Fig. 8) at the German shipyard “MeyerWerft”. This production line was designed especially for the hybrid welding of plates and stiffeners with a CO2-laser. Panels with a geometry of 20 m *20 m are welded completely at this line. “Meyer-Werft” is specialised in great cruise liners for the international market. Therefore a lot of panels with lower sheet thickness are used for their products.
Fig. 8. CO2-Laser–GMA–hybrid welding from panels and stiffeners at shipyard “Meyer-Werft” in Papenburg, Germany [4] At the German shipyard “Kvaerner Warnow Werft” in Rostock (Fig. 9) a common welding gantry was refitted with equipment for the Nd:YAG-laser-GMAhybrid welding. During the above mentioned regional research project the prob-
Nd:YAG - Laser - GMA - Hybrid Welding in Shipbuilding and Steel Construction
23 23
lems regarding with such a refitting should are solved. The refitting was done after measuring the dynamical behaviour at the point, were the future hybrid working head shall be located. The results correlated well with the requirements regarding to [3]. The first test welds made with the refitted conventional welding gantry also verified the suitability of the gantry for use with laser technique. In the meantime also other German shipyards are interested in such a refitting solution.
Fig. 9. Welding gantry (right) and Nd:YAG-laser-GMA-hybrid welding head (left) at the shipyard “Kvaerner Warnow Werft” in Rostock, Germany
4 Conclusions The Nd:YAG-laser in connection with the gas metal arc welding offers great possibilities for the use of laser technology in shipbuilding and steel construction. The lower energy per unit of length resulting from laser based joining processes leads to lower distortions and less postweld straightening. Besides the higher welding speed compared with conventional welding methods the laser-GMAhybrid welding offers a better gap bridging ability compared with pure laser welding and makes the advantages of laser technology usable for manufacturing under the conditions in shipbuilding and steel construction. The high flexibility of the Nd:YAG-laser equipment, resulting from the wavelength of the Nd:YAG-laser radiation and their optical properties enabled an implementation of laser technology according to the actual demands of a company. In principle the refitting of conventional welding machines with Nd:YAG-laser equipment, such as robots and welding gantries is possible.
24 24 U. Jasnau, J. Hoffmann, and P. Seyffarth
The implementation of laser equipment and technology into shipbuilding and steel construction leads to a higher level of automation in this branches, connected with a higher level of product quality and reproducibility. Besides the technical requirements for the introduction of laser technology in the mentioned industrial areas the qualification level of the staff, workers as well as technicians and engineers must be increased.
References 1. 2. 3. 4.
Eboo, Steen, Clarke: Arc Augmented Laser Processing of Materials; Conf. Proc. Of Advances in Welding Processes, Harrogate, UK, 1978, p. 257 Gvosdetzky, Krivtsun, Chzhenko, Yarinich; Laser-Arc Discharge: Theorie and Applications; Harwood Academic Publishers, 1995 N.N.:IIW-Document IV-789-01: Acceptance test for Nd:YAG-laser beam welding machines, part 1: machines with optical fiber delivery; 2001 N.N.: Schiffe aus Licht; in “INFORM-Das Magazin für Umformtechnik”, 01/2002, Schuler AG, Göppingen
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics S.B. Chen,1 D.B. Zhao,2 Y.J. Lou,3 and L. Wu3 1. S. B. Chen is with Institute of Welding Technology, Shanghai Jiao Tong University, Shanghai, 200030, P. R. China. 2. D. B. Zhao is with Mechanical Depart., Qinghua Univ., Beijing, 100080, P. R. China. 3. Y. J. Lou and L. Wu are with and he was with Welding Division, HIT , Harbin, 150001, P. R. China. [email protected]
Abstract. This paper presents a successful application of intelligent technologies, such as computer vision, image processing fuzzy-neural networks, for sensing, characterization, modeling and control of the welding pool dynamic process in the pulsed Gas Tungsten Arc Welding (GTAW). The image processing algorithm is discussed in detail. The results of the 2-dimesion and 3-dimesion geometric characterization of the weld pool from the single-item pool images are presented. Neural models and self-learning neural controller for the weld pool dynamic process have been developed. The experiment results on the process indicate that the designed vision sensing and control systems are able to emulate a skilled welder’s intelligent behaviors: observing, estimating, decision-making and operating. The work in the paper shows great potential and promising prospect of artificial intelligent technologies in the welding manufacturing.
1 Introduction Artificial intelligence (AI) is an applied science and technology across multiple disciplines. Industrial requirements and promotion of AI techniques have resulted in considerable development in AI. Developing effective AI methodology for practical applications is all along attracted research efforts in scientific and technical fields. In practical engineering applications of AI, a challenging problem is how to control a complex object with uncertainty (Willis, et al,1992). In general, the means for T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 25−55, 2004. Springer-Verlag Berlin Heidelberg 2004
26 26 S.B. Chen et al.
resolving this problem includes the sensing techniques for simulating human sensing organ functions, modeling techniques for cognizing, control techniques for reasoning and decision-making, and administering of motion and operation. Computer vision technology has become a powerful tool for a wide variety of engineering applications (Hoffman ,1991; Marr,1982). Its use in the industry has accelerated advancement in quality control and automation systems. AI provides powerful approaches to control complex systems through synthesis of intelligent techniques. The non-linear behaviour and uncertainties of many practical systems make it difficult for conventional methods to model analytically and control these systems. Many real industrial process plants fall into this category, and hence intelligent techniques are required to model and control such systems. Among many different “intelligent” approaches, neural network and fuzzy methodologies have emerged as very powerful tools for designing intelligent control systems, **owing to their capabilities of emulating human learning processes (Willis et al ,1992). Artificial neural networks (ANN) shows that the method has been used in a modest scale to develop nonlinear models and control systems. For example, recent studies (Andersen et.al,1990; Lim, et al, 1993; Pham et.al, 1999) and their cited references have demonstrated the utility and flexibility of the concept within the domain of process engineering. Artificial neural network applications have great prospect and potential in various fields of industrial engineering. As is well known, arc welding processes are extremely nonlinear and multivariable-coupled. They involve many uncertainties, such as, influences of metallurgy, heat transfer, chemical reaction, arc physics, and magnetization (Saedi, et al, 1988). As a result, it is very difficult to obtain a practical and useful controllable model of an arc welding process through classical modeling approaches. Until now, control of weld quality, such as welding penetration, and fine formation of welding bead (i.e., control of the size and changes of the welding pool), is still a perplexed faced by control engineers or for weld technologists (Vilkas, 1966). The weld seam accuracy is difficult to control due to the non-linearity and uncertainties of the process. Intelligent control systems should be developed for modeling and control of the welding process, as they derive the control performance based on human experience, knowledge, and logic techniques, instead of mathematical process models. Andersen K (1990), Lim T.G. et al (1993) have studied neural network and fuzzy techniques for arc welding processes. Another critical difficulty of dynamic control of an arc welding process is how to detect weld pool geometrical features, such as weld bead width and penetration, especially from the topside of weld pool, conveniently and in real-time. Though various efforts have been made to sense weld pool sizes in real-time from the topside, such as ultrasonic detection, infrared sensing, pool image processing, and radiographic sensing (Brzakovic et al ,1991; Lee et al, 1996; Pietrzak el al, 1994; Richardson et al 1994), results of weld quality control have been achieved with limited successes. In order to achieve better control of weld quality, we should not only improve reliable measurement means, but also introduce advanced real-time control methodologies. The two technical fronts should be paid equal attentions, and developed simultaneously.
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
27 27
In this study, a computer vision system combining video camera and image processing, was used in the closed loop system to control a pulse GTAW in real-time. A novel sensing system, which captures the topside and backside vision images of a weld pool in the same frame simultaneously, was employed for monitoring and modeling of the welding process. The real-time image processing algorithm was developed to acquire topside and backside sizes of weld pool. Furthermore, neural network models of pulsed GTAW process were established to predict weld pool features from measured data. Based on the ANN models, experiments on intelligent control schemes for the weld pool dynamic process were conducted.
2 The Pulsed GTAW Process and Monitoring Systems 2.1 Description of the pulsed GTAW process The pulse GTAW is one of arc weld dominant technologies. It is widely used in the high-quality weld manufacturing, especially in high-precise thin joining. The major quality index in pulsed GTAW is precise penetration and fine formation of the weld seam. In order to obtain good quality of welding products, dynamics of the welding pool should be exactly dominated in real time, i.e., regulating accurate size of weld pool. Main influences on weld pool in welding process involve weld current, such as pulse duty ratio, peak current, base current, arc voltage, and welding speed; experiment conditions such as the root opening or geometry of the groove, material, thickness, work piece size, electrode tip angle, and rate of shielding gas flow; welding conditions such as heat transfer condition, arc emission and so on. In designing control systems, we should deal with following problems: sensing of weld pool features, modeling of weld pool dynamic process, and designing effective controllers for the process.
2.2 Monitoring systems of the pulsed GTAW The structure diagram of monitoring system for the pulse GTAW welding is shown in Fig.2-1. The experiment system includes a double-side visual sensing subsystem, which consists of CCD camera , recorder, frame grabber, and monitor, image processing sections, welding quality controller completing by the computer , welding power supply, and welding current setup was regulated by a computer. Single chip microcomputer (SCM) was used for control of travel speed. A wire feeder was applied for filling wire metal into the weld pool during pulsed GTAW.
28 28 S.B. Chen et al. SCM motion control board
Personal computer Interface
M anipulator
Frame grabber
Recorder
Power supply
Wire feeder
M onitor CCD camera
Torch Work piece Work plate
Backside Topside Composed filter system
Travel direction
Fig.2-1 The structure diagram of experimental system for pulsed GTAW
2.3 Double-side visual sensing system for weld pool dynamics In many practical cases, accessing to the backside of the weld piece is not possible. According to the experience of skilled welder, the geometry of weld pool can provide instantaneous information about welding penetration. Topside and backside images of weld pool should be captured and the geometry should be extracted for modeling and control of the dynamic process. The backside width and penetration could be predicted by the dynamic model and controlled by intelligent controller. Therefore, a double-side visual sensor was designed to detect the face and back of the weld pool for modeling welding penetration and formation process. The sensing system consists of topside and backside light path and composite filters. Both topside and backside images concentrate on a same target of the CCD camera through the above double-side imaging light path system.
3 Image Processing of Weld Pool in Pulsed GTAW 3 .1 Two dimensional character processing of the weld pool image without wire filler Bead-on-plate experiment was conducted on low carbon steel during pulsed GTAW using the double-side imaging system, the experiment conditions are omitted here. A complete weld pool image in a frame is shown in Fig.3-1, in which the left is backside image and the right is the topside image. In Fig.3-1, the nozzle, arc center, topside molten portion, and topside solidified portion could be clearly distinguished from the topside image of weld pool.
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics Backside image of weld pool
29 29
Topside image of weld pool Nozzle Arc centre partition
Backside molten partition
Topside molten partition Topside solidified partition
Backside solidified partition
Backside
Topside
Fig.3-1 A frame complete weld pool image of pulsed GTAW
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Fig.3-2 The visual images of different time's weld pool in a pulse cycle
Fig 3-2 is top/backside pool serial images in a pulsed cycle. Fig.3-2 (b),(c),(d) are pool images in pulsed peak current and (e),(f),(g) in the pulsed based current. Based on the above analysis, the time sequence of pulsed current for collecting images were determined. The taking image current was selected as 60A, taking image time as 80ms for a frame complete weld pool image. Used of image contrast between arc mirror reflect of the pool liquid surface and arc diffuse reflect of frozen metal on work-piece surface, the arc disturbance could be changed into an advantage for taking a clear image of weld pool. Based on arc spectral analysis, an appropriate filtering light system was designed in wavelength 600nm 700nm and the stable top/backside images was obtained as Fig 3-3.
Nozzle Arc centre partition
Backside of weld pool
Topside of weld pool Unmolten material
(a) Backside visual image
Solidified material
(b) Topside visual image
Fig.3-3 Canonical topside and backside visual image of weld pool
30 30 S.B. Chen et al.
3.1.1 Exponential base filter processing of weld pool images The first step of weld pool image processing was filter and wiping off noise disturbance. In order to reduce calculation, the recursion exponential base filter was used to smooth images. The response property of exponential base filter is similar to the Guass filter(Marr,1982).
One dimensional recursion exponential base filter The unit sample response of Exponential Base Smoothing (EBS) was defined as:
S ( n) = k (α n + 1) e−α n
3-1
In Eq.(3-1) n is a discrete variable, α is a constant of the filter for one-dimesional space range, k is a proportion factor defined as :
k=
(1 − e− α ) 1 + 2 α e − α − e −2 α
3-2
Decomposing S ( n ) as cause-effect and non-cause-effect parts and supposing ( ) is the output response of the EBS filtering S ( n ) we filter input as x ( n ) yn have
y ( n ) = yc ( n ) + ya ( n )
3-3 −α
yc ( n) = k x ( n) + e ( α − 1) x ( n − 1) + 2 e y c ( n − 1) − e −α
−2 α
y c (n − 2)
y a (n) = k [e−α (α + 1) x(n + 1) − e−2α x(n + 2)] + 2 e−α y a (n + 1) − e−2α y a (n + 2) =0
Initial conditions as: x( 0)
y c ( 0) = y c ( −1) = 0
n = 1, 2,"" M
x ( M + 1) = x ( M + 2 ) = 0 y a ( M + 1) = y a ( M + 2) = 0 n = M ,""2,1 In Eq.(3-3), yc ( n ) is the response for the filt cause-effect part, ya ( n ) is the
response for the filt non-cause-effect part. One dimensional recursion exponential base filtering algorithms could be realized by Eq.(3-3).
Two dimensional recursion exponential base filtering algorithms Based on one-dimensional exponential base filtering function, the separable two-dimensional exponential base filter was developed. For two-dimensional input x ( m, n ) , one-dimensional recursion filtering algorithms along one direction could be first completed due to separability of the filter, and then taking its output as input for next one-dimensional filtering algorithms. The top/back side images of the weld pool were smoothed by the above algorithms, i.e. Fig 3-4. The results shown that
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
31 31
noises in the topside image was filtered and the image margin was fully maintained while α =1.41( σ = 1. 0 ). Taking α =0.94( σ = 1.5 ), the noises in backside image was ideally sieved by the exponential base filter. Since the two-dimensional exponential base filter could be separated two one-dimensional recursion filters in two directions, the computing time and space costs of the algorithms was greatly reduced, e.g., the algorithm time cost on a PC-486 computer with main frequency 100MHz was not more than 8ms for an image window with 160×190 pixel. The time cost processing the same size image by the Guass filter was about 50ms. It is obvious that exponential base recursion filter is suitable to process images in real-time.
(a)
(b)
(c)
(d)
Fig.3-4 The smoothed image of weld pool with EBS algorithm (a) Original topside image
(b) Topside image smoothed with EBS algorithm
(c) Original backside image
(d) Backside image smoothed with EBS algorithm
3.1.2 Contrast Enhancement algorithms for the weld pool Based on the contrast enhancement algorithms presented by Gordon et al. (1984) and Beghdadi et.al.(1986), for supposed center pixel point ( x ,
Gxy
y ) with greyscale and window Wm and the each pixel point with greyscale Gij the following
contrast enhancement algorithms, named as CE, was adopted to enhance the contrast of the weld pool. The CE algorithms follows as: Step1 For the pixel point( x , adjacent region:
G =
y ) calculating average greyscale G in its
¦ Gij / ( m × m)
(i , j )∈Wm
Step2 For all pixel points in the window of the pool:
(3-4
Wm calculating the edge value ∆ ij
∆ ij = Gij − G
(3-5
32 32 S.B. Chen et al.
Step3 For the window the pool edge:
Wm , calculating weighted average greyscale Exy for
E xy = Step4
¦ ∆ ij ∗ Gij
(i , j ) ∈Wm
(3-6
(i , j ) ∈Wm
Cxy of the pixel point ( x , y ): Gxy + E xy
Calculating the contrast degree
Cxy = Gxy − E xy Step5
¦ ∆ ij
Changing
Cxy to contrast transfer function C xy ′ = F (C xy )
(3-7
Cxy′ = F ( Cxy ) = ( Cxy ) a b , b = 2 p , a < b (3-8 x , y G ′ and calculating Defining greyscale of the pixel point ( ) as xy
Step6 functions as following:
R xy = (1 − Cxy′ ) (1 + Cxy′ )
if Gxy ≤ E xy
= (1 + Cxy′ ) (1 − Cxy′ )
(3-9
if Gxy > E xy
Gxy′ = E xy ⋅ R xy Step7
(3-10
Repeat algorithms Step1 to Step6 for each pixel point.
Using the above CE algorithm for the topside images, the contrast enhancement processing results are shown as Fig. 3-5. Let
β = 0. 5
to
β = 0. 25
β=a/b
window size m, for
and m=5 to m=13, the edges and background of the weld
pool are learly distinguished in Fig. 3-5. Trading off the enhancement effect and calculating complexity,
β = 0. 5
and m=9 were determined in processing and
calculating time was not more than 16ms.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Fig.3-5 Contrast enhancement of the topside image of weld pool (a) EBS smoothed image (b) CE( β = 0.5 , m=5) (c) CE(β = 0.5 , m=9) (d) CE(β = 0.5 ,m=13)(e) CE(β = 0. 25 , m=5)
(g) CE(β = 0. 25 , m=13)
(f) CE( β = 0. 25 , m=9)
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
33 33
3.1.3 Extracting the geometry characters of the weld pool Defined the topside pool characters as: maximum width length
Wb max
W f max , maximum half
L f max , and the backside pool characters as: maximum width maximum length Lb max and area S b , shown as Fig. 3-6, these
parameters could be used to discribe the pool characters in full penetration. y
y Lf m ax Wf max
b
(a) Topside of weld pool
Lbm ax
x
o
c Wbmax
a
o
x
Sb
(b) Backside of weld pool
Fig.3-6 The definition of feature size parameters of fully penetrated weld pool
Algorithm for extracting the topside characters of the weld pool Based on the greyscale distribution of the image in Fig.3-7, the algorithm ETG for extracting top geometry of the weld pool was developed as the following: The algorithm ETG: extract-top-geometry (image window topw) { Step1: get-centre-point C (image window topw); Step2: for (i=-5; i<=5; i=+2) {get-line -grey (along weld pool width direction fkv, through point C, line-i) find-edge-point (line-i) ( find point A and B from greyscale distribution); calculate-width (line-i); } Step3: find-max-width (W f max from 11 lines); Step4: Step5: Step6: Step7: Step8:
}
find-max-centre( point C’); get-line-grey (along weld pool length direction fkw, from C’); find-peak-point((along fkw); find-max-peak(point D); calculate-mid-length( L f max from C’ to D);
34 34 S.B. Chen et al. x
o fkv B C C' A
D
fkw
y
Fig.3-7 Characteristic points of the topside image of weld pool x
o bkw
Lp
Ep
Cp Rp Sp
bkv
y
Fig.3-8 Characteristic points of the backside image of weld pool
Algorithm for extracting the backside characters of the weld pool The binaryzation of the back weld pool images , as Fig. 3-8, was obtained by the EBS filter and the threshold division. From Fig.3-8, the pool length was determined by the edge point Sp and Ep along bkw, and the width was determined by the edge point Lp and Rp along bkv. Based on weld pool vertical-square picture distribution by the EBS, the algorithm extracting back geometry (EBG) was developed as the following: The algorithm EBG extract-back-geometry(image window backw) { Step1: calculate-histm ( histm-histogram, for back pool image); Step2: histm-moving-smooth( averaging-moving algorithm for histogram); Step3: find-threshold Step4: binaryzation( for back pool image); Step5: get-centre-point(Cp in Fig.3-8); Step6: for(i=-5; i<=5; i=+2) { get-line-grey (line-i grey from Cp, along bkw,)
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
35 35
find-edge-point(Sp and Ep, along bkw); calculate-length (along line-i); } Step7: find-max-length ( Lb max from 11 lines); Step8: get-interval (determining 21 lines from Lb max along bkv); Step9: for(i=0; i<=20;i++) { get-line-grey (line-i along bkv) find-edge-point (Lp and Rp along bkv); calculate-width (along line-i); } Step10: find-max-width ( Wb max from 21 lines ); Step11: calculate-area( S b from 21 lines and Lbmax ); } The signal flowchart of processing images of weld pool by the above algorithms is shown as Fig.3-9, EBS, CE , ETG, EBG as the above, TD is the binary algorithm for finding threshold from the histogram distribution of the backside weld pool image.
Fig.3-9 Signal flowchart of processing images of weld pool
The above real-time processing algorithms for the top/backside weld pool images was running on the PC-486 computer, the calculating time for a frame image was not more than 50ms for the top image and 30ms for back image. Ii was sufficient for real-time closed loop control of welding dynamic process. 3.2 Three-dimensional character processing of the weld pool image with wire filler
Although many achievements of sensing, modeling and control of GTAW pool were proposed, studies on GTAW with wire filler was seldom carried out and reported in present. Since two-dimensional image processing extracted the weld pool edges, width, length and so on, which only reflected the plane shape variety, if there
36 36 S.B. Chen et al.
is a convex and concave changes on weld pool surface and a high precise requirements for weld seam height figuration, extracting three-dimensional characters of the weld pool would be necessary. In our study, three-dimension surface parameters of weld pool has been extracted from single-item image of the weld pool. One of evident advantages using single-item image is to made sensing instrument more simple and practical.
3.2.1 The topside images of the weld pool with wire filler Under the supposed experiment conditions, welding parameters were set as follows: pulse peak current 120A, base current 60A, pulse duty ratio 40%, and travel speed 2.5mm/s. According to imaging principle, image was determined by light source, camera and object shape. In this experiment, imaging current was set as 30A, and the light source of arc could be supposed as a point light source. The shutter of CCD camera was set as 1/1000s, and the iris diaphragm and filter ratio were mounted. The topside image of the weld pool with wire filler was taken as Fig.3-10. In Fig.3-10, when the weld pool was partially penetrating, the top shape of weld pool was approximate to a ellipse ,i.e., a convex model image; while on full penetrating, the pool image was similar to a peach shape, i.e. a concave model. The length and stem shape varieties of the weld pool were most obvious.
Fig.3-10 The shape variation of topside weld pool
3.2.2 The character definition of weld pool with wire filler 1) Defining the top shape characters of the weld pool with wire filler as follows: 2) Defining the top height characters of the weld pool with wire filler as follows: Due to the impact of arc plasma, the surface of weld pool was depressed in full penetration, while the surface of weld pool could be convex in partial penetration or with wire filler. The rear part of weld pool showed the different concave or convex shape distinguishably, as shown in Fig.3-12. Where Sw was defined as
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics yr
37 37
y L th
L tt
0
x
1
xr
Wt
R hl
Lt
Fig.3-11 The definition of shape parameters of topside weld pool
concave area in the pool width y axis, Sl as concave area in the pool length x axis, Wt as the top width, Lt as the top length, then the concave parameters Dl and Dw Along top length and width of the weld pool were defined as: Dl = S l / Lt , Dw = S w / Wt (3-11) The surface height Ht of the weld pool was defined as maximum height of the rear part, and it is similar to the seam surplus due to the frozen metal attached on the rear part of weld pool. 3) The character definitions of the back weld pool is similar to that of weld pool without wire filler in Fig.3-6(b) Welding direction
Welding torch Weld pool Solid pool Ht
Arc
Workpiece
a) Concave type Welding direction
Welding torch
Arc
Solid pool Ht
Weld pool
Wire Filler
Workpiece
b) convex type. Fig.3-12 The definition of height parameters of topside weld pool
38 38 S.B. Chen et al.
3.2.3 The image processing of the filling wire weld pool For short, calibration of the vision sensing systems is omitted and the processing algorithms for the top characters of weld pool images are briefly described as the following. 1) Image processing of the top weld pool The different surface heights of the filling wire weld pool lead to obvious difference between greyscale distributions of concave and convex type images, so that the different processing algorithms should be designed for them. The threshold division method is more suitable to convex type object images, and the edge extraction method is suitable to concave type images. In this paper, the image processing algorithm TSE(Topside Shape Extraction) was developed to extract top characters of the filling wire weld pool. It is consist of TI (Type Identification) algorithm for distinguishing concave or convex type images, EE(Edges Extraction) for extracting top edge points of the weld pool, and ER(Edges Regression) for imitating top edge curve of the weld pool.
i) Type identification of concave or convex images TI In this paper, Hough transformation [10] was used to distinguish the rear pool shapes and judge concave or convex type images. The calculating results are shown in Fig.3-13. ii) Edges Extraction of top edge points of the weld pool EE For the convex type image, usually, the image histogram indicates that the image greyscale distribution has typical twin apex property, binary threshold value could be determined by finding the vale point of twin apices, and the convex type binary image could be obtained in Fig.3-14(a). The edge points of the weld pool could be precisely extracted by the edge tracking method, as shown in Fig.3-14(b).
O
X
fkw
C {E'}
fkv
{E}
Y
a) Convex type
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics O
X
fkw
C {E'} {E}
fkv
Y
b) concave type Fig.3-13 Type identification of topside image
a) Thresholding of convex image
b) Edge tracing of the thresholding image
fkw U2 U1
c D2
fkv
D1
c) Edge extraction of concave image Fig.3-14 Extracting edge points of topside image
39 39
40 40 S.B. Chen et al.
a) Convex type
b) concave type
Fig. 3-15 The results of edges regression for topside pool
Topside image Camera
TI
EE
ER
Lt Wt R hl
Fig.3-16 Signal flowchart of image processing for topside pool image
The signal flowchart of processing images of weld pool by the above algorithms is shown as Fig.3-16, the algorithms TI, EE and ER are as the above. The whole processing algorithms for the top/backside weld pool images was running on the PC-486 computer with main frequency 100MHz, the calculating time for a frame image was not more than 50ms. It was sufficient for real-time sensing and control of welding dynamic process. 2) The backside image processing of the weld pool with filling wire is similar to that of no-filling wire pool, here is omitted.
3.2.4 The extraction of the surface height of the weld pool with wire filler In the image of the weld pool with wire filler, the arc inverted reflection in the weld pool image could be distinguished due to mirror reflection on molten metal surface of the weld pool. So a simple method for extracting the surface height of weld pool was presented from the reflection phenomenon: surface weight of weld pool could be indirectly calculated by the extracted distance between the tungsten tip and its inverted reflection in weld pool image. The pool images with different surface height are shown as Fig. 3-17. The tungsten pole nozzle and terminal (i.e., the arc initial terminal), arc shape in the images can be clearly distinguished in Fig3-17. The algorithm calculating top height from the weld pool images follows as:
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
41 41
Fig.3-17 Weld pool image with different surface height of weld pool
Step1: Image filtering (Guass filtering) Step2: Binaryzation of images Processing results shown as Fig.3-18(b) by threshold division of pool image histgram, Step3: Determining the position of tungsten tip, the result calculated by binaryzation images and Hough transformation is shown as Fig.19 and the length L1 in Fig.17, the processed image as 3-18(d). Step4: Calculating the surface height of the weld pool Because the tungsten pole was warded by welding torch nozzle in welding, the distance between the tungsten tip and its inverted reflection in weld pool image couldn’t be directly calculated. So we obtained distance between the tungsten pole and nozzle, L0 as Fig. 3-18(e), and let arc length as La, then the top surface height of the weld pool, Ht was calculated by
H t = La − fcal.z * ( L0 + L1 ) / 2.0
(3-11)
a) Initial image b) binary image c) width calculation d) height extraction e) distance from the tip to nozzle Fig.3-18 The height result from image processing topside weld pool W , P ix e ls
16 12 8 4
Arc
Tungsten
0 0
2
4
6
8
10
12
Num be r
Fig.3-19 Determination of the position of tungsten tip
42 42 S.B. Chen et al.
1
0.2
0.8
0
0.6 0.4
Calculated Measured
0.2 0
Ht, mm
Ht, mm
The concave and convex of the weld pool surface could be obtained by the above image processing and calculating. Comparing the calculated and measured results, shown as Fig.3-20, the average error of the concave and convex height was about 0.1mm. The errors were brought due to such reasons: a) the mirror assumption on the metal surface of weld pool; b) the height changes as solidifying metal in weld pool; c) the image quality and processing precision.
-0.2 -0.4
Calculated Measured
-0.6 -0.8
5
7
9 11 13 15 17 19 Pulse number
5
7
9 11 13 15 17 19 Pulse number
a) Convex surface b) Concave surface Fig.3-20 Comparing the calculated height of topside weld pool and the measured reinforcement
4 Modeling and Control of Weld Pool Characters Based on Two-Dimensional Image Processing Based on extracted two-dimensional characters of weld pool images, the relationship between the top shape parameters and back width of weld pool without wire filler could be established for predicting and control of the back width of weld pool. 4.1 The neural network modeling of weld pool dynamics
The traditional modeling method is usually unsuitable to complex welding dynamics. Since artificial neural networks (ANN) is able to preferably approach complex non-linearity and uncertainty of unknown object and non supposition on the object is necessary for modeling, it was used to weld pool dynamics.
4.1.1 Neural network modeling principle of the welding pool dynamics Generally, neural network modeling is directly learning the input/output datum from the system experiment, the index model convergence is so-called cost function to be minimum. The identification modeling principle by neural network for the welding pool dynamics is shown as Fig.4-1, where TDL (Tapped Delay Line) is time delay unit.
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
43 43
Fig.4-1 The principle diagram of dynamic process identification with neural network
As is well known, identification modeling process by neural network usually includes to design exciting persist signal determine model structure, choose identifying algorithms, train the model and verify model precise, here the details are omitted.
4.1.2 Neural network model for predicting the backside size of the weld pool In order to exactly control full penetration and weld seam forming, the neural network model for predicting the backside size of the weld pool, BNNM(Backside Neural Network Model), was established as Fig.4-2, the model includes 17 inputs, 24 hide layer units, the output of the model, Wb max , for the back maximum width of the weld pool. The examined results of the model is shown as Fig.4-3. The average relative error of the model output Wb max was 3.32%, mean-square error was 4.01%. The net model was converged and the model precise was desired. 4.2 The neuron self-learning control of the based-on weld pool in pulsed GTAW
As to the multivariable and time-delay non-linearity in welding process, the conventional PID closed-loop control was limited because its coefficients were hardly regulated on-line, and the simple fuzzy control didn’t contain the adaptive function with the variation of welding conditions. Aiming at the characteristics of changing structure and coefficients, a self-learning controller based on single neuron was proposed in this study. Only the desired output and the real output detected in on-line were needed to form the self-learning closed-loop control system, without the on-line identification of the process coefficients. The weight of neuron was corrected on-line with the improved BP algorithm, and the performance of error was minimized to optimize the output of the control system.
44 44 S.B. Chen et al. Hidden layer 18 19 20
Input layer
21 Ip(t) Ib(t) Vw(t-2) Vw(t-1) Vw(t) Ua(t-2) Ua(t-1) Ua(t) δ (t-2) δ (t-1) δ (t) W fmax(t-2) W fmax(t-1) W fmax(t) Lfmax(t-2) Lfmax(t-1) Lfmax(t)
1
22
2
23
3
24
4
25
5
26
6
27
7
28
8
29
9
30
10
31
11
32
12
33
13
34
14
35
15
36
16
37
17
38
Output layer
42
W bmax(t)
39 40 41
Fig. 4-2 The structure of BNNM neural network dynamic model 7
W bmax, mm
BNNM output 6 5 4
actual output
3 0
30
60
90
120
150
180
Sample number
Fig.4-3 The result of detecting BNNM model
4.2.1 Neuron self-learning control system for pulsed GTAW The schematic diagram of the neuron self-learning control system is shown in Fig.4-4, where the effect of signal convert is to combine the desired and predicted backside width, for generating the output variables X = {x1,x2,x3} as the input of neuron self-learning controller, which are summed and transferred nonlinear to derive the variation of pulse duty ratio (∆ δ ). The actual outputs are measured by MS and input in BNNM to attain the predicted backside width Wmb max , with which the set value R input in Signal Converter. At the same time, the input weights of neuron is adjusted in on-line with BP algorithm according to the error of backside width.
45 45
x1
w1 x 2 w2 ¦ x w3
F ( s)
∆δ +
z −1
δ
+
Wbmax
Pulsed GTAW Process
3
WP
R
Signal Converter
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
Wmbmax
Wfmax BNNM
MS
Lfmax
Fig.4-4 The schematic diagram of single neuron self-learning control syste mduring pulsed GTAW
4.2.2 Designs of self-learning neuron controller for based-on plate in pulsed GTAW In Fig.4-4, the input variables X = {x1,x2,x3} of the controller for based-on plate in pulsed GTAW are the error, error with first order, error with second order between desired backside width and BNNM output, shown as follows.
x1 = α1e( t ) = α1 ( R − Wmb max ( t )) x2 = α 2 ∆e( t ) = α 2 ( e( t ) − e( t − 1)) x3 = α 3 ∆2 e( t ) = α 3 ( e( t ) − 2e( t − 1) + e( t − 2 ))
4-1 4-2
4-3
Where α 1 , α 2 and α 3 are constant indicating the emphasis degree in the controller, selected as α 1 is 1.0, α 2 is 0.3, α 3 is 0.1. Weight normalization is to avoid the saturation of weight during learning process, shown as follows:
wi′ = wi
3
¦w
2 i
j =1
4-4
The sum of weighted neuron is: 3
s = ¦ wi′ ⋅ xi i =1
4-5
F ( s) is nonlinear transfer function, selected as hyperbolic tangent function.
∆δ = F ( s ) = γ (1 − e−2ξ S ) (1 + e−2ξ S )
4-6 Where γ and ξ are two constants, γ sets the saturated value of control
variable, and ξ sets the linear degree of control variable. The larger is γ , the more possible is to attain the desired value. The smaller is ξ , the wider is the linear work region to restrain the fluctuation of stable status. γ is selected as 300 and ξ is 0.135. Object function is minimized by the follow equation.
E( w) =
1 ( R − Wmb max ( k ))2 ¦ 2 k
The derived corrected formula of weight is.
(4-7
46 46 S.B. Chen et al.
∆wi = ηϕ x i wi ( k + 1) = wi ( k ) + ∆wi
(4-8
(4-9 Where i = 1, 2, 3, η is learning rate, equals to 1.0. ϕ is the equalized output error of neuron, resembled with the actual output error.
ϕ = ( R − Wmb max ( k )) ⋅ (1 − ∆δ ( k ) γ ) ⋅ (1 + ∆δ ( k ) γ )
(4-10
4.2.3 Experiments on self-learning neuron controller for based-on plate in pulsed GTAW To test the feasibility of the self-learning neuron control schematic for practical use, the experiment during bead-on-plate pulsed GTAW on the dumbbell specimen, 2mm thickness mild steel plate, was conducted, the desired seam width is 5.00 mm. The controlled and responsing curves are shown in Fig.4-5. The maximum error between BNNM output and desired value was 0.26mm. Tested data was compared with the desired value, the maximum error was 0.30mm, the average error was 0.10mm, and the root-mean-square deviation was 0.08mm. The perfect effect of topside and backside photographs are also shown as Fig.4-6. The top/backside images during welding process are shown as Fig.4-7, one could see that the welding pool images were maintained stable and controlled effect was fine. Moreover, PID and Fuzzy controller were also implemented for pulse GTAW process control. Each control scheme was limited to ensure each backside width. Fuzzy control could simulate the worker’s experience, but without the adaptive regulation with the varied conditions. While the neuron self-learning control could attain perfect control effect with different set values and conditions, and was suitable for the varied structure and coefficients of welding process specially. 50 40
6
30 4
pulse duty ratio actual output BNNM output
Vw=2.5mm/s; Ip =120A; Ib=60A; l=3.0mm; L=8l/min;
2 0 0
10
20
30
40
50
60
70
80
90
20
δ, %
W bmax, mm
8
10 0 100
Time, s
Fig.4-5 The neuron self-learning closed-loop control curves of dummy bell workpiece during pulsed GTAW
(a) Topside
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
47 47
(b) Backside Fig.4-6 A photograph of dummy bell workpiece with neuron self-learning control Welding time, s 30 35
15
20
25
60
65
70
75 80 85 Welding time, s
40
45
50
90
95
100
55
backside topside
backside topside
10
Fig.4-7 The topside and backside images of weld pool of dummy bell workpiece with neuron self-learning closed-loop control
5 Modeling and Control of Weld Pool Characters Based on Three-Dimensional Image Processing Based on extracting three-dimensional characters of the weld pool images, the relationship among the top shape and height parameters and back width of the weld pool with wire filler was modeled by the ANN, and predicting and control of characters of weld pool could be realized. 5.1 The neural network model for the top height and back width of the weld pool with filler
In order to reflect relationships between the top height and back width of the weld pool with filler, the neural network dynamic model for the top height and back
48 48 S.B. Chen et al.
width, BHDNNM(Backside and Height Dynamic Neural Network Model), was established as Fig. 5-1. The model includes the current values of the top pool shape and welding parameters, such as welding current or pulse duty ratio , filling rate, travel speed, and their former moment values, 21 input variables, 5 hide nodes. P(1) P(2) P(3)
Input layer
22
2
23
3
# # #
20
25
# # # # # #
P(20) P(21)
Hidden layer
1
21
P(1) =Ip (t) P(2) =δ(t) Output layer P(3) =Vw (t) P(4) =Vf(t) P(5) =Ip(t-1) W b(t) P(6) =δ(t-1) 27 P(7) =Vw(t-1) 28 Ht(t) P(8) =Vf(t-1) P(9) =Ip(t-2) P(10)=δ(t-2) P(11)=Vw(t-2) P(12)=Vf(t-2)
P(13)=Lt(t) P(14)=Wt(t) P(15)=Rhl(t) P(16)=Lt(t-1) P(17)=Wt(t-1) P(18)=Rhl(t-1) P(19)=Lt(t-2) P(20)=Wt(t-2) P(21)=Rhl(t-2)
26
Fig.5-1 Structure of BHDNNM
With the trained BHDNNM and inputs of the model, backside width and topside height could be predicted, and the predicted and measured values were shown in Fig.5-2. The results showed that mean errors of backside width and topside height were 0.004mm and 0.042mm respectively, and the relative mean square error were 5.54% and 7.83% respectively. The statistic results verified the validity of BHDNNM. Under the variation of heat-sinking conditions, the backside width and topside height varied irregularly, but the backside width and topside height could be predicted accurately with the developed model BHDNNM, then these predicted values could be input to the controller as feedback for control of bead shape. 5.2. Double-inputs and Double-outputs (DIDO) Self-learning Fuzzy-neural network Control of the backside width and topside height of the weld pool
From the step response of the weld pool process with wire filler, one can see that the pulsed duty ratio is a main impacting variable to the back pool width, and the filling rate to the top pool height. The backside width and topside height could be regulated respectively by pulse duty ratio and filling rate. While the both shape parameters could not be controlled at the desired value with the single-variable controller. The pulse duty ratio and filling rate should be regulated simultaneously for controlling the backside width and topside height of the filling wire pool.,i.e, The double-input and double-output (DIDO) controller should be developed.
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
49 49
8.0 Measured
Estimated
Wb,mm
6.0 4.0 2.0 0.0 1800
1820
1840 1860 1880 Sampling Number
1900
1.2 Measured
Estimated
Ht,mm
0.6 0.0
-0.6 -1.2 1800
1820
1840 1860 1880 Sampling Number
1900
Fig. 5-2 Testing results of BHDNNM
5.2.1 Design of the DIDO Self-learning Fuzzy-neural network Controller Fuzzy controller was capable of humanoid inferable function. On designing fuzzy controller, the decision of membership function and fuzzy rules was the hindrance. An improved DIDO self-learning fuzzy controller based on single layer neural network for real-time perfecting fuzzy rules was designed as Fig.5-3. Where the MS was the measuring system for detecting the topside shape parameters TSP Learning Algorithm Co W bg + Si Htg
gn +
e 1 ce 1 nv ert
e 2 ce 2
Multi-variable Self-learning Fuzzy Controller
∆δ
+
z-1
δ
+ + ∆ Vf +
Pulsed GTAW Process with Wire Filler
Wb Ht
Vf
z-1
Wbp Htp
WP & TSP BHNNM
MS
Fig.5-3 Double variable self-learning fuzzy controller for butt pulsed GTAW with wire filler
50 50 S.B. Chen et al.
(Lt ,Wt and Rhl) and the welding parameters WP, such as welding current, travel speed, etc. The predicted backside width (Wbp) and topside height (Htp) were generated by BHDNNM. The given backside width (Wbg) and topside height (Htg), Wbp and Htp were input to signal converter for calculating the errors ei(i=1,2) and change in errors ce i(i=1,2).
Fuzzy Rules and Inference Takagi-Sugeno rules (Takagi, Sugeno ,1985) were adopted as the fuzzy rules.
Rule1: if e1 is E11 and ce1 is CE11 ande 2 is E 21 andce2 is CE21 , then∆δ = u11 and ∆Vf = u 21 Rule2 : if e1 is E12 andce1 is CE12 ande 2 is E 22 andce2 is CE22 , then∆δ = u12 and ∆Vf = u 22 ""
Where
Rulen : if e1 is E1n andce1 is CE1n ande 2 is E 2n andce2 is CE2n , then∆δ = u1n and ∆Vf = u 2n Case : e1 is E10 andce1 is CE10 and e 2 is E 20 andce2 is CE20 Conclusion: ∆δ = u10 and ∆Vf = u 20 Eij and CEij are the fuzzy set of ei and cei, uij is the control variable, ei0 and cei0 are the actual sampling values, i=1,2 , j=1,2,…,n The matching degree of the fact in the rules is:
h j = µ E1 j (e10 ) ∧ µ CE1 j (ce10 ) ∧ µ E2 j (e 20 ) ∧ µ CE2 j (ce 20 ) , j = 1,2,..., n (5-1) Where µ Eij (ei 0 ) and µ CEij (cei 0 ) are the membership of ei0 cei0 to fuzzy sets Eij
CEij, and “∧” is for minimum calculation. The conclusion ui0 was derived as follows: n
¦h u j
ui0 =
j =1
n
¦ j =1
hj
ij
n
=
¦ j =1
hj n
¦
hj
n
u ij =
¦ h′ u j
ij ,
i = 1,2
j =1
(5-2)
j =1
Where h′j represents the normalized value of the membership hj. In the fuzzy control rules, the premise variables were described as fuzzy set and constant. The conclusion variable was described as constant, adapted according to control performance, and derived without de-fuzzier calculation.
Realization of Single-layer Fuzzy Neural Network The memberships of the premise variable in the rules were set as isosceles tri angle,
51 51
1
1
0.8
0.8
0.6
0.6
MF
MF
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
0.4 0.2
0.4 0.2
0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 Zone of E
1
0 -0.5 -0.25 0 0.25 Zone of CE
0.5
Fig.5-4 Fuzzy sets of premise variables
shown in Fig.5-4. The sets of e1 e2 were [-1,1], and 11 fuzzy sets were defined. The sets of ce1 ce2 were [-0.5,0.5], and 5 fuzzy sets were defined. The set of error and change in error were [-2,2] and [-1,1] for the backside width, and [-1,1] and [-0.5,0.5] for the topside height. The overlap of the neighboring fuzzy sets was 50%, so a known value of any premise variable could activate two fuzzy sets. There were four variables in the premise, so a known fact could activate 16 (24=16) fuzzy rules. There were maximum 16 non-zero values in the n normalized matching degrees, and they were numbered renewably as h′m, m=1,2,…,16. The above inference process could be realized with a single-layer neural network, shown in Fig.5-5. The single-layer neural network had 16 inputs, corresponding to the 16 normalized matching degree h′m. The network had two outputs, corresponding to the two control outputs of the fuzzy controller. The transferring function of the neuron was sigmoid function.
u i = u i ,max
1 − e − oi
16
1 + e −oi ,
oi =
¦ h′ u
m im
m =1
, i = 1, 2
(5-3)
Where ui,max was the largest scope of output variable, u1,max=∆δmax=10%, u2,max=∆Vf=5mm/s. uim denoted the weight m connecting to neuron i, as well as the conclusion constant to be regulated in the control rules. hm
ce2
Matching degree calculation
2 ...
ce1 e2
h'm
1
16
1
Normalization
u1 (∆δ)
ΣS
u1,max
ΣS
u2,max
2 ...
e1
16
u2 (∆ ∆ Vf )
Single layer neural network
Fig.5-5 Double variable fuzzy controller constructed by single-layer neural network
Learning Algorithm of the controller The performance cost of the neural network learning was the output errors of welding process.
52 52 S.B. Chen et al.
E (t ) =
1 2
2
¦ [r (t ) − y (t )] l
2
l
(5-4)
l =1
Where rl(t) represented the given backside width and topside height Wbg and Htg, and yl(t) represented the predicted Wbp and Htp. Then the weights of the neural network was corrected as follows:
u im (t + 1) = u im (t ) + ∆u im (t ) = u im (t ) − β = u im (t ) + β
2
¦ l =1
[rl (t ) − y l (t )] ⋅
∂E (t ) ∂u im (t )
∂y l (t ) ∂u i (t ) ∂o i (t ) (5-5) ⋅ ⋅ ∂u i (t ) ∂o i (t ) ∂u im (t )
Where β was learning coefficient, β=1.0. ∂y l (t ) was the partial differential of the output l to the input i at time t. ∂u i (t )
∂y l (t ) y l (t ) − y l (t − 1) ∆y l (t ) ≈ = ∂u i (t ) u i (t ) − u i (t − 1) ∆u i (t )
(5-6)
and, u (t ) ∂u i (t ) 1 = u i ,max ⋅ [1 − ( i ) 2 ] ∂oi (t ) 2 u i ,max
(5-7)
∂oi (t ) = hm′ (t ) (5-8) ∂u im (t ) then the final formula of weight correction was derived. 2 ∆y (t ) 1 u (t ) u im (t + 1) = u im (t ) + β el (t ) ⋅ l ⋅ u i ,max ⋅ [1 − ( i ) 2 ] ⋅ hm′ (t ) (5-9) ∆u i (t ) 2 u i ,max l =1
¦
5.2.2 Experiments on DIDO self-learning Control systems for weld pool with wire filler To verify the performance of the double-variable self-adaptive fuzzy controller, butt welding experiments with wire filler were conducted on dumbbell-shape specimens. The minimum regulating unit was 1% for pulse duty ratio, and 0.1mm/s for Vf. The backside width and topside height were given as 4.0mm and 0.2mm. The errors and changes in errors were calculated with the given and predicted value by the BHDNNM, and were looked up in the fuzzy control rules. If matched, the change of the control variable could be derived directly. If not, the self-learning was carried out. The control impacting was added at initial 7th pulse. The weld pool images were shown in Fig.5-6, and it was concluded that the double-sided shape of the weld pool controlled reliably. The variation of the shape parameters was shown in Fig.5-7. This indicated that the backside width and topside height accessed to the given value quickly when control affected. The controlled parameters were maintained around the given
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
53 53
values even in hard heat-sinking conditions. Statistic results showed that the maximum errors of the backside width and topside height were -0.44mm and 0.06mm, the mean errors were -0.09mm and 0.006mm. The results testified the feasibility and accuracy of the double-variable self-adaptive fuzzy controller. Photographs in Fig.5-8 also showed that backside width and topside height of bead were stable. Time, s 10
15
20
25
55
60
65
70
75
30
35
40
45
50
80
85
90
95
100
Backside
Topside
Backide
Topside
5
Time, s
0.5
4
0.4
3
0.3
2
0.2
1
60
8 6
50
4 40
2
0.1 Wb
Ht
0 0
20
40 60 Time, s
80
0.0 100
Vf, mm/s
5
Ht, mm ,%
Wb, mm
Fig.5-6 Double-side images of butt welding with wire filler by double variable fuzzy control during pulsed GTAW
δ
Vf
30 0
20
40 60 Time, s
80
0 100
a) Double-side shape parameters of weld pool b) pulse duty ratio and filler rate Fig.5-7 Curves of double variable fuzzy control
6 Conclusion Marks The works in this paper has shown that the artificial intelligence methodology, such as computer vision sensing, image processing, fuzzy logic, artificial neural networks, intelligent control scheme etc., would be greatly suitable to be applied in sensing, modeling and control of the welding pool dynamics in pulsed GTAW. Developments of AI technology applied in welding manufacturing would inaugurate an inspired approach of solving difficult problems for many years. We believe that the many puzzle problems in welding automation and robotic welding need to be processed by AI. And AI technology would tend towards to practicality in welding and manufacturing.
54 54 S.B. Chen et al.
a) Topside
b) Backside Fig.5-8 Photographs of dumbbell-shaped workpiece with double variable fuzzy control during pulsed gas tungsten arc butt welding with wire filler
Acknowledgment This work was supported by the National Natural Science Foundation of China and Science and Technology Committee of Shanghai, China, No.021111116.
References 1.
2.
3. 4.
Andersen, K., Cook, G.E., 1990. Artificial Neural Networks Applied to Arc Welding Process Modeling and Control. IEEE transaction Industry Application 26 (9), 824-830. Beghdadi, A., Negrate, A. L., 1986. Contrast Enhancement Technique based on Local Detection of Edges. 1Comput. Vision Graphics Image Process, 46 (9), 162-174. Brzakovic, D., Khani, D.T., 1991. Weld pool edge detection for automated control of welding. IEEE Transactions on Robotics and Automation 7 (3), 397-403. Canny, A., 1986. Computational Approach to Edge Detection. IEEE Trans. on Image Processing 28(2), 679-698.
Computer Vision Sensing and Intelligent Control of Welding Pool Dynamics
5. 6. 7.
8. 9.
10.
11.
12. 13. 14. 15.
16.
17. 18.
55 55
Gordon, R., Rangayan, R. M., 1984. Feature Enhancement of Film Mammograms using Fixed and Adaptive Neighborhood Appl. Opt. 23(5), 560-564. Hoffman, T., 1991. Real-time imaging for process control. Advanced Materials & Processes 140 (3), 37-40. Kim, E.W., Allem, C., .Eagar T.W., 1987. Visible Light Emissions during Gas Tungsten Arc Welding and Its Application to Weld Image Improvement. Welding Journal 66 (12), 369s - 377s. Lee, C.W., Na S.J., 1996. A Study on the Influence of Reflected Arc Light on Vision Sensors for Welding Automation. Welding Journal. 75 (12), 379s-387s. Lim, T.G., Cho, H. S., 1993. Estimation of Weld Pool Sizes in Gma Welding Process Using Neural Networks. Journal of Systems and Control Engineering 207(1), 15-26. Marr, D., Vision, A., 1982. Computational Investigation into the Human Representation and Processing of Visual Information. W. H. Freeman and Company, San Francisco. Pham, D.T., Karaboga, D., 1999. Self-tuning fuzzy controller design using genetic optimization and neural network modeling, Artificial Intelligence in Eng. 13(2), 119-130. Pietrzak, K.A., Packer, S.M., 1994. Vision-based weld pool width control. ASME Journal of Engineering for Industry 116 (2), 86-92. Richardson, R.W., Gutow, D.A., 1984. Coaxial Arc Weld Pool Viewing for Process Monitoring and Control. Welding Journal 63 (3), 43-50. Saedi, H.R., Unkel, W., 1988 Arc and weld pool behavior for pulsed current GTAW. Welding Journal 67(11), 247s - 255s. Salter, S., Deam, R.T., 1986. A Practical Front Face Penetration Control System for TIG Welding. Proceedings of an International Conference on Trends in Welding Research, Gatlinburg, Tennessee, USA, May 18-22, 381-392. Takagi,T., Sugeno, M., 1985. Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. on Systems, Man and Cybernetics 15(1), 116-132 . Vilkas, E. P., 1966. Automation of gas tungsten arc welding process. Welding Journal 45(5), 410s - 416s. Willis, M. J., et al., 1992. Artificial neural networks in process estimation and control. Automatica 28(.6), 1181-1187.
Surface Shape Reconstruction of Weld Pool during Pulsed GTAW from Its Single Image D.B. Zhao1, J.Q. Yi1, S.B. Chen2, and L. Wu3 1
Lab of Complex System, Institute of Automation, Chinese Academy of Sciences,
Beijing 100080, P.R. China 2
Institute of Welding Technology, Shanghai Jiao Tong University, Shanghai
200030, P.R. China 3
National Key Laboratory of Advanced Welding Production Technology, Harbin
Institute of Technology, Harbin 150001, P.R. China. [email protected]
Abstract. This paper addresses an improved algorithm of shape from shading with a single real image, and its application on three-dimensional surface reconstruction. The improved algorithm consists of smoothing constraint, boundary height constraint, and weighted light error. For consideration of actual visual sensing systems, the feasibility of the algorithm is verified with a single synthetic image. Finally, the surface shape of weld pool during pulsed GTAW is reconstructed successfully from its real single image.
1 Introduction The algorithm of shape from shading is based on the principle of imaging geometry reflection. Several algorithms were developed by Horn [1], etc, to extract the surface shape of an object, aiming at ideal reflection model, such as Lambertian surface, orthographic projection, and a distant light source. The reflection model becomes more complex in actual imaging conditions, such as non-Lambertian surface, prospective projection, and a nearby point light source. A generalized reflection model was established by Lee [2] to recover the object surface shape from multiple images taken by different illumination directions. But the multiple images are impossible in most cases, so shape from a single image under actual imaging conditions is crucial for many applications, such as aerial image, human face photos, and industrial vision applications. T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 56−62, 2004. Springer-Verlag Berlin Heidelberg 2004
Surface Shape Reconstruction of Weld Pool during Pulsed GTAW
57 57
Visual imaging is a key method to sense pulsed gas tungsten arc welding process (GTAW). Zhang [3] developed a laser structured light system to achieve the specular reflection of a pulsed laser projected on weld pool, to calculate the three-dimensional weld pool surface shape. The shape of weld pool is crucial for weld quality. Therefore, the more accuracy of the three-dimensional weld pool surface shape is measured, the higher quality of the weld can be achieved. This paper presents an improved algorithm of shape from shading with a single image and its application on weld pool surface reconstruction.
2 Basic Algorithm of Shape from Shading The generalized imaging reflection geometry model is shown in Fig.2.1. In the figure p is a point on object surface, and n is its normal line. S is a point light source, r is the distance from the light source S to the point p. i is the unit vector toward the light source S, and v is the unit vector toward camera C. The specula direction h is the bisector of i and v. θi, θv and α represent the angles between n and i, n and v, n and h respectively. Considering a nearby point light source, diffuse and specula reflection of object surface, and perspective project of a camera, a generalized reflection model (image intensity E) is derived * I0 T exp{−k r [cos −1 (hT n)]2 } T - (v n z ) 4 ( β d (i T n) + β s ), (i n) ≥ 0 (2.1) E = R ( z i , z j , z l ) R (i , n , v , r ) = + r 2 (i T n)(v T n) -0, otherwise , !
Where kr is the surface roughness parameter, βd and βs are the diffuse and specula reflective coefficients. A square image domain can be divided into a set of Mt non-overlapping triangles Tk, k=1, 2, …, Mt, with Mn nodal points Pk, k=1, 2,…, Mn. The smooth object surface can be approximated with the corresponding triangle surface patches Ti with nodal points pk (zi, zj, zl). The variable vectors i, n, and v can be described with the three vertices pk (zi, zj, zl). With Taylor series expansion, the generalized reflection model is linearized.
Rk ≈
Mn
# wkm z m + ξ k
Point light s ource S
θi i
(2.2)
m =1
C
n α
h
d ωi
θv
Camera
v
d ωv dA p
Fig. 2.1. Generalize reflection geometry model
58 58 D.B. Zhao et al.
Where Mn
ξ k = Rk ( zin −1,z nj −1,zln −1 ) - # wkm z mn −1 , m=1
* ∂Rk ( z i , z j , z l ) , if m ∈ Vk = {i, j , l} ( zin−1 , z nj−1 , zln−1 ) wkm = + , Vk is the set of each ∂z m -0 , otherwise , triangle vertices. Light error is taken as the cost function. 2
' $ Mn .1 eb = # ( E k − R k ) = # ( E k − %% # wkm z m + ξ k //2 k =1 ) k =1 03 & m =1 To minimize the cost quadratic function, a linear formula is derived. Mt
2
Mt
Mt
Mt
k =1
k =1
(2.3)
Az = d , [A ]m,n = 2 # wkm wkn , [d ]m,n = 2 # ( E k − ξ k ) wkm , 1 ≤ m, n ≤ M n (2.4)
Where A is a sparse and symmetric coefficient matrix, and d is a load vector. The height matrix of the vertices z can be solved by iterative method. Synthetic image is generated with the reflection model, and is often used to verify the feasibility of reconstruction algorithms. The object is supposed as a most half sphere embedded in a plane zp=-100 mm. The sphere diameter is 6.0 mm, and the diameter of the intersecting circle is 4.0 mm. The position of the camera is xc=0, yc=0, zc=0, the lens focal length is f=2.5 mm, and the target dimensions are t=4.8×4.8 mm2, corresponding to the maximum image sizes 512×512. The parameters of the object surface are βd=0.5, βs=0.5, and kr=5.0. The light intensity is supposed as rs2. Two synthetic images are formed with different illumination directions and a fixed camera, shown in Fig.2.2. The image sizes are 32×32. With the above basic algorithm, the surface height can be reconstructed accurately with the two images, shown in Fig.2.3 (a). But the surface height results from a single image have a large error, shown in Fig.2.3 (b).
(a) (b) Fig.2.2. Synthetic images with different illumination directions: (a) xs=60 mm, ys=0 mm, zs=0 mm; (b) xs=0 mm, ys=60 mm, zs=0 mm.
59 59
Surface Shape Reconstruction of Weld Pool during Pulsed GTAW
3 Improved Algorithm of Shape from a Single Image To improve the accuracy of the algorithm, smoothing constraint, some prior knowledge about the surface height, and weighted cost function are proposed. The object surface is supposed as smooth, then the cost function are renewed. (3.1) e = eb + λe s , Cz = b , C = A + λB Where es is the cost function of the smoothing constraint. λ is a smoothing factor.
j
j 2
# # w( E k ) ⋅ ( E k − R k )
k =1 j =1
z, mm
-99 -100 -101 8
24 y, 16 p ix el
8
0
8
24 l i xe p ,
16 x
(a)
-99 -100 -101
(3.2)
Where ω(Ek) is the grayness-weighted factor, *-e − ( Ek −150) 2 / 5000 , E ≥ 150 k and w( E k ) = + -,1 , otherwise When grayness is smaller than 150, the weight is the maximum 1.0, which indicates that image point with low grayness plays an important role in the surface height reconstruction. With the proposed smoothing constraint, boundary condition, and grayness-weighted regulation, the improved algorithm of shape from a single image is formed. Surface height is reconstructed accurately from a single image, shown in Fig2.3(c). Statistic results show that the average error is -0.04 mm, and the mean square error is 0.11 mm.
24 y, 16 pix el
-98
0
8
24 16 el pix , x
(b) -98 z, mm
eb =
Mt J
-98
z, mm
The algorithm of shape from shading is effective specially to reconstruct smooth surface, which can be considered as a curved surface embedded in a known background plane. We take the background plane or boundary as prior knowledge. Other information can also be considered, such as the maximum or minimum height, the position of the occluding boundary, etc. The illumination direction influences the accuracy of the calculation results greatly. Where the normal line of the surface patch is similar to the illumination direction, the image intensity is the highest and the calculation error is the largest. So grayness-weighted regulation is introduced
-99 -100 -101 24 y, 16 8 pix el
0
8
24 16 el ix p x,
(c) Fig.2.3. Reconstructed surface height results: (a) from two images with the basic algorithm; (b) from a single image with the basic algorithm; (c) from a single image with the improved algorithm.
60 60 D.B. Zhao et al.
4 Surface Height from a Single Weld Pool Image The visual sensing system for GTAW process is shown in Fig.4.1. The world coordinate system o-xyz with the center of weld pool as the original, and the camera coordinate system O-XYZ with the CCD camera as the original are both Cartesian coordinate systems. Two mirrors, M1 and M2, are used to reflect weld pool. O1A and O2B are the normal line of the mirrors, described by the angles to the axes. A point p(x,y,z) in the world coordinate system is reflected by M1 and M2, and is focused to a point Pf(Xf,Yf,-f) on the focus plane of the CCD camera. With the visual sensing system, weld pool images are captured, shown in Fig.4.2. The left corresponds to convex weld pool surface on local patches with large wire feed speed, and the right corresponds to concave weld pool surface with small wire feed speed. The image sizes are 128×128. After shape reconstruction from weld pool images, surface height results in the camera coordinate system should be transformed into the world coordinate system considering the real visual sensing system characteristics. The imaging conditions are: Point light source: In the visual sensing process for GTAW, electric arc can be thought as a point light source [4]. Its position is s(xs,ys,zs)=(0, 0, la)= (0, 0, 3), where la is the electric arc length. The light intensity is supposed as I0 = rs2 . Camera: The lens focal length f is 50 mm. The CCD target sizes are 4.8×3.6 mm2, corresponding to the maximum image sizes 500×592; Surface reflectance property: The parameters of weld pool surface are selected as βd=0.3, βs=0.7 and k=3.0. While the workpiece and solidified weld surface are thought as Lambertian surface βd=1.0, βs=0.0.
Fig.4.1. Visual sensing system for GTAW process
61 61
Surface Shape Reconstruction of Weld Pool during Pulsed GTAW
With the improved algorithm of shape from a single image and the coordinate system transformation, the surface height of weld pool is calculated from a single weld pool image, shown in Fig.4.3. The surface heights of weld pool are reconstructed successfully. The results indicate that low wire feed speed cause weld pool surface to depress and surface height to decrease, resulting in full penetration.
1
z, mm
0.5 0 -0.5
6 3
-1 3
(a)
-9
-6
y, m
-3
m
0 -1 x, m -5 m
(a)
1
z, mm
0.5 0 -0.5
6 3
-1 3
(b) Fig.4.2. Weld pool images with different surface shape during GTAW with wire filler: (a) local convex surface; (b) concave surface.
m
-5 -9
-6
y,
-3
mm
0 -1 x, m
(b) Fig.4.3. Surface height results reconstructed from single weld pool images: (a) local convex surface; (b) concave surface.
5 Conclusions The problems of shape from shading with a single image are analyzed. An improved algorithm is proposed incorporating smoothing constraint, boundary condition, and grayness-weighted light error. Finally, the surface height of weld pool is successfully reconstructed from its single image considering visual sensing
62 62 D.B. Zhao et al.
coordinate transformation. The surface shape of weld pool provides abundant information for modeling and control of pulsed GTAW process.
Acknowledgements This work is funded by National Science Foundation of China No.59575057 and No. 59635160.
References 1. 2. 3. 4.
Horn BKP (1990) Height and gradient form shading. International Journal of Compute Vision 5(1):37-75. Lee KM, Kuo CJ (1997) Shape from shading with a generalized reflectance map model. Computer Vision and Image Understanding 67(2): 143-160. Kovacevic R, Zhang YM (1995) Vision sensing of 3D weld pool surface. 4th International Conference on Trends in Welding Research, pp5-8. Zhao DB, Chen SB, Wu L, Chen Q. Extraction of three-dimensional parameters for weld pool surface in pulsed GTAW with wire filler. (To be published in ASME Journal of Manufacturing Science and Engineering)
63
A Novel Intelligent Monitor for Manufacturing Processes M. Ge* and Y. Xu Department of Automation & Computer-Aided Engineering, The Chinese University of Hong Kong, Hong Kong [email protected]
Abstract. The development of condition monitoring in continuous-time mass production is a traditional but critical topic owing to its role in increasing costeffectiveness. A new method for intelligent manufacturing processes monitor was presented using the newly developed learning approach, Support Vector Regression, and related noise density model for efficient and effective modeling. A new similarity measurement method was introduced resulting in a real-time system developed. The system was applied for certain manufacturing process condition monitoring. The experimental results showed that the proposed method is substantially more effective than others. In addition, the new monitor requires only normal condition samples for implementation, an attractive feature for shop floor applications. The novel system ensures the automatic and intelligent operation, and it has potential applications on the other manufacturing processes.
1 Introduction It is reported that more than half of the manufacturing cost is attributed to the production process; the demand of the cost reduction in production becomes curial [1] . Aiming to increase the economic benefit, the monitoring systems are mainly designed to increase the productivity and quality of production, resulting in the broad application of process monitors; however, it is difficult for some manufacturing operations, i.e., sheet metal stamping, laser welding. The detection of faults in manufacturing processes is of great practical significance. Most manufacturing processes involve a great number of process variables, many of which interact with one another. When any of these process variables deviate beyond specified limits, faults occur. The traditional approaches to fault detection, hence, are based on threshold (limit) checking. The more advanced methods involve the spectral analysis of signals emanating from the processes[2]. *
Please address the corresponds to this author at [email protected]
T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 63−79, 2004. Springer-Verlag Berlin Heidelberg 2004
64 64 M. Ge and Y. Xu
Modern methods use decision-making relying on machine learning (ML), and statistics[3], and they have improved the performance of monitoring systems. However, the requirement of collecting sample sets under different conditions for analysis often impedes their effectiveness in many real-world applications. It is worth noting that although sensor and computer technology have been greatly advanced, decision-making is still a subject of research. The statistical approach to condition monitoring is concerned with the analysis and interpretation of data, while ML deals primarily with learning rules or structures. The two approaches are mutually beneficial, and hence they are often used in combination. From a mathematical point of view, ML can be considered as an estimate of a function f using n sets of training samples, resulting in the construction of a classifier or an estimate. Ideally, f shall correctly describe both the training samples and the unseen examples from the population. Traditionally, classical ML methods are based on the Empirical Risk Minimization (ERM) principle[4]. Unfortunately, tests have shown that ERM cannot ensure the minimization of actual risk; moreover, several factors impact the complexity of the learning machine. For example, the cases of overfitting and the local minimum of Artificial Neural Networks (ANN) reveal the failure of the ERM principle. The unreasonable design structure might also lead to failure[4]. Statistical learning theory can precisely identify the factors that need to be taken into account for successful learning. Arising from statistical learning theory, the support vector (SV) technique develops theory into a tool for not only theoretical analysis but also creating practical algorithms for estimating multidimensional functions[5]. The SV technique has recently gained popularity for two reasons. First, it is satisfactory from a theoretical point of view: support vector learning is based on simple ideas that allow us to learn intuitively from examples. Second, it works well in many practical examples. For this reason alone, the method has been applied to many real-world problems, such as pattern identification, regression, and density estimation[6,7,8]. When the SV learning technique was applied to the function estimation, the support vector regression (SVR) approach emerged. Many approaches have been utilized to solve the function estimation. For example, radial basis function (RBF) is typically used in many applications[9,10], and demonstrates relatively better performance in regression estimation among the traditional neural network techniques. Since the format of the RBF network is similar to the SVR, there have been many comparisons in theoretical analysis and real-world application[7,11]. Evaluations show that SVR is the superior method because it is based on a unique idea and methodology. This paper presents a new condition monitoring system using the SVR approach and a noise density model. The remainder of the paper is structured as follows. Section 2 outlines the principles of SVR and introduces a new similarity measurement method. Section 3 proposes the new intelligent monitor, and Section 4 describes the experimental design and how to apply the newly developed system for monitoring the sheet metal stamping operation. Discussion of the experimental results is presented in the Section 5. Section 6 concludes the paper.
A Novel Intelligent Monitor for Manufacturing Processes
65 65
2 Modeling with SVR and Similarity Measurement 2.1 The principle of SVR estimation Although the SVR technique is relatively new, its basic theory is well understood [12] . Briefly, given a set of independent and identically distributed (iid) training samples Sf = {(x1, y1), (x2, y2), …, (xn, yn)}, where x ∈ RN denotes the input vector, and y ∈ R represents the output value, the goal is to find a function f that correlates input and output: N
y = f (x, w ) = w, x + b = % wi xi + b ,
(2.1)
i =1
where w ∈ RN and b ∈ R are the weighting factors, *,* denotes the dot products
in RN, and f : RN → R. Introducing a nonlinear function ϕ(x), a more generalized form is as follows: N
y = f (x, w ) = % wiϕ ( xi ) + b = w , ϕ (x) + b .
(2.2)
i =1
The nonlinear function ϕ maps the input vector x onto a feature space Ζ, ϕ(x) ∈ Ζ Note here, w ∈ Ζ is in a different feature space to the one in Eq. (2.1). The SVR method finds a function in the family F (x, w) = {f | f : RN → R} that minimizes the risk function: R(w ) = ' c (x, y, f (x, w )) | dp(x, y ) ,
(2.3)
where c(x, y, f(x, w)) denotes the loss function, which determines how estimation error based on the training samples will be penalized; p(x,y) is the probability distribution of (x,y). Vapnik designed the ε-insensitive loss function[4] as: c(x, y, f (x, w )) = y − f (x, w ) ε = max(0, y − f (x, w ) − ε ) .
(2.4)
Due to the lack of information on p(x, y), it is common to use the empirical risk function estimated from the training sample set Sf, This results in the following empirical risk function: Remp (w ) =
1 n % c(x, y, f (x, w )) . n i =1
(2.5)
In addition, to overcome overfitting and enhance the generalization, a capacity control term is often added leading to the following [13], Rreg (w ) = Remp (w ) +
λ 2
w
2
=
1 n λ 2 c(x, y, f (x, w )) + w ; % 2 n i =1
(2.6)
66 66 M. Ge and Y. Xu
where λ > 0 is a regularization constant. The last term controls the flatness of the function. In this context the use flatness denotes that one seeks small w. This has a significant effect on the generalization capability of the algorithm. Intuitively, the SVR maps the input vectors onto a high dimensional feature space, and subsequently locates an optimal function f that not only reduces the risk but also controls the generalization capability. It can be shown by use of the loss function as Eq. (2.4) that solving a SVR is equivalent to solving the following optimization problem:
min
w,b, , !
!
*
s.t.
n n 1 2 1 $w, w& + C%(ξi + ξi*) = w + C%(ξi +ξi* ) 2 2 i =1 i =1 yi −$w,ϕ(xi )& −b ≥ ε +ξi
(2.7)
$w,ϕ(xi )& +b − yi ≥ ε +ξi* ξi, ξi* ≥ 0, i = 1,2,…, n.
where slack variables ξi,ξi* cope with additive noise, and ε represents the expected level of precision. With the slack variables ξi and ξi* equal to zero, Eqs. (2.7) form the so-called ε-tube. C, a function of (λn), in Eq. (2.7) is a parameter corresponding to the value ||w||2. It provides a trade-off between two terms: the flatness of f and the amount to which deviations larger than ε. According to Eq. (2.7), a Lagrange function can be constructed as: n n 1 2 w + C% (ξi + ξi* ) − %αi (ε + ξi − yi + w,ϕ (xi ) + b) 2 i =1 i , j =1
L=
−
n
%αi* (ε + ξi* + yi −
i , j =1
n
(2.8)
w,ϕ (xi ) − b) − % (ηiξi + ηi*ξi* ), i =1
where αi, αi*, ηi, ηi* ≥ 0 are the Lagrange multipliers. A more generalized form of SVR involves the use of a kernel function, K. A kernel is a function, K, such that for all x, z ∈ Ω. K (x, z ) = $ϕ (x) ⋅ ϕ (z )& .
(2.9) [14]
The kernel function must satisfy the Mercer condition
.
'' K (x, z)ϕ (x)ϕ (z)dxdz > 0 ,
(2.10)
where K is a symmetric function, ϕ(x) is not always equal to zero, and 2 ' ϕ (x)dx < ∞ . Replacing the w ⋅ ϕ (x ) with K and by the partial derivatives of
L with respect to the variables (w, b, ξi, ξi*), it can be shown that solving the optimization problem in Eq. (2.8) is equivalent to solving the quadratic optimization problem: min* α ,α
n
− % (α i − α i* ) yi + i =1
n 1 n (α i − α i* )(α j − α *j ) K (xi , x j ) + ε % (α i + α i* ) % 2 i, j =1 i =1
(2.11)
A Novel Intelligent Monitor for Manufacturing Processes
s.t.
67 67
n
% (α i − α i* ) = 0 i =1
0 ≤ αi,αi* < C,
i = 1, 2, …, n.
This problem can be solved using various methods such as projected conjugate gradients and Quasi-Newton methods. The quadratic optimization process guarantees the solution must be the global minima. After the unique solutions α, α* are found, the weight factor can then be represented as: (2.12)
n
w = %(αi − αi* )ϕ (xi ) . i =1
Note that, according to Karush-Kuhn-Tucker (KKT) complementarity conditions , the product of the dual variables and constraints at the optimal solution must vanish; that is [15]
α i ( y i − (α i − α i* ) K (x i , x j ) − b − ε − ξ i ) = 0 α i* ((α i − α i* ) K (x i , x j ) + b − ε − ξ i* − y i ) = 0
.
(2.13)
Thus, a number of input vectors lies on the boundary of the ε-tube; and they are referred to as the support vector (SV). Thus, according to the SVs, ξi,ξi* = 0, b can be calculated as follows: n
b = y i − % (α i − α i* ) K (x i , x j ) − ε j =1
for α i ,α i* ∈ (0, C ) .
(2.14)
Finally, the SVR can be constructed for real-valued function estimation as
f ( x, , !
!
*
)=
% (α i − α i* ) K (x i , x) + b .
i∈SV
(2.15)
2.2 Similarity measurement
Let us examine Eq. (2.6) again. Given a set of training samples with additive noise, Sf, the goal of SVR is to find a function f that has at most ε deviation from the actually measured targets for all the training samples, and at the same time is as flat as possible. To do this, the definition of the loss function c(x, y, f(x, w)) is very important. Ideally, it should have a simple structure to avoid difficult optimization problems. In addition, it should be general enough so that the data can be fit. Assuming that the set Sf was generated by f plus additive noise ξ with density p(ξ), y i = f (x i ) + ξ i .
(2.16)
Then, the likelihood of an estimate Ff = {(x1, f(x1,w)), (x2, f(x2,w)), …, (xn, f(xn,w))} based on Sf is
68 68 M. Ge and Y. Xu n
n
P(F f | F) = ∏ P( f (x i , w ) | (x i , y i )) = ∏ P ( f (x i , w ) | y i ) i =1
i =1
n
n
i =1
i =1
= ∏ p ( y i − f (x i , w )) = ∏ p(ξ i )
.
(2.17)
To maximize the P(Ff | F) is equivalent to maximize logP(Ff | F): n
n
n
i =1
i =1
i =1
log P(F f | F) = log ∏ p(ξ i ) = % log p(ξ i ) = % log p ( y i − f (x i , w )) .
(2.18)
Therefore, in a maximum likelihood sense, by setting c(x, y, f (x, w )) = − log p( y − f (x, w )) = − log p(ξ ) ,
(2.19)
optimal loss function can be achieved. Therefore, the desired density minimizes the risk function described by Eq. (2.3) with the loss function described by Eq. (2.19). Eq. (2.18) is equivalent to n
n
i =1
i =1
log P (F f | F) = % log( p (ξ i )) = −% c(x i , y i , f (x i , w )) .
(2.20)
The importance of this result is twofold. On the one hand, it presents a principle that enables the construction of a loss function. Once the noise density of the system is defined, its related loss function can be obtained according to Eq. (2.19). As in many practical applications, the standard Normal density model, N(0,1), is used to describe noise. p( y − f (x, w )) = p (ξ ) =
1
1 exp(− ξ 2 ) . 2 2π
(2.21)
This results in the following loss function: c(x, y, f (x, w )) =
1 1 ( y − f (x, w )) 2 = ξ 2 . 2 2
(2.22)
On the other hand, the result provides a new similarity measurement method – a theoretical standard by which it can be determined whether two signals are generated in the same condition or not. Under normal conditions, the process demonstrates unique characteristics that are reflected in its related signals. The signals from abnormal conditions would exhibit deviations, referred to here as noise. The predicted values emerge when substituting the unknown signal as input into the regression modeling. With a certain noise density model, or the related loss function, the unknown input vectors’ likelihood of estimating the function can be calculated according to Eq. (2.20). As a result, it is possible to implement condition monitoring by comparing the likelihood of an unknown signal with the likelihood distribution of normal ones. Employing the loss function defined in Eq. (2.22), the related SVR estimation can be defined as
A Novel Intelligent Monitor for Manufacturing Processes
min*
w,b, , !
!
s.t.
n n 2 2 1 2 1 2 2 $w, w& + C%(ξi + ξi* ) = w + C%(ξi +ξi* ) 2 2 i =1 i =1 yi −$w,ϕ(xi )& −b ≥ ε +ξi $w,ϕ(xi )& +b − yi ≥ ε +ξi* i = 1,2,…, n. ξi, ξi* ≥ 0,
69 69
(2.23)
The problem can be derived using the method introduced in Section 2 and taking into account that ξiξi* = 0 and, therefore, that the same relation αiαi* = 0 holds for the corresponding Lagrange factors[12]. min
n
n
i =1
i =1
(2.24)
− % (α i − α i* ) yi + ε % (α i + α i* ) 1 n 1 + % (α i − α i* )(α j − α *j )(K (x i , x j ) + δ i, j ) 2 i, j =1 C
s.t.
n
% (α i − α i* ) = 0 i =1
αi≥0, αi*≥0,
i = 1, 2, …, n,
where δ is an n by n identity matrix. The precision error ε plays an important role in the SVR estimation. Aiming to model the signal for condition monitoring, ε was set to zero. This corresponds to a standard linear least squares regression with a weight decay factor controlled by the parameter C. As C → ∞, the problem tends to an unconstrained least squares, which is equivalent to leaving the dot product matrix diagonal unchanged. According to the KKT complementarity condition, b can be calculated using Eq. (2.25). n
b = y i − % (α i − α i* ) K (x i , x j ) − ε − j =1
(α i − α i* ) C
for α i , α i* > 0 .
(2.25)
The final function estimation shares the same expression as Eq. (2.15).
3 Novel Condition Monitoring System To implement condition monitoring systems requires the measurement of typical quantities and is related to process and quality data. In practice, the data are preferred to employ the sensor signals with high signal-to-noise ratio (SNR). Nevertheless, despite the methods employed, extraneous noise still remains amongst normal signals. Some are related to measurements, but the process causes others. Current thinking suggests that abnormal operations contribute to the deviations in signal patterns, which can also be regarded as a kind of noise in addition to normal signal. Thus, a state-of-the-art condition monitoring system can
70 70 M. Ge and Y. Xu
be developed based on the SVR and noise density models employing the architecture depicted in Figure 1.
Fig. 1. The architecture of the condition monitoring system : off-line training path : on-line monitoring path It consists of two paths: off-line path is used in training and on-line path is used for real-time condition monitoring. In off-line training mode, SVR is carried out based on the available training sample sets in normal condition. Consequently, the mean and variance of the likelihood values P will be calculated according to the loss function. Finally, this enables us to calculate the threshold of P. In on-line monitoring mode, by assessing the likelihood of real data, the system can detect faults or otherwise and issue appropriate and efficient commands to the actuator. The SVR-based process monitor demonstrates a number of interesting features. First, it has an exceptional ability to estimate the functions of signals due to the regression estimation in the kernel-induced feature space. By replacing the dot product $ϕ(x),ϕ(xi)& with the kernel K(x, xi), inputs are mapped onto a high dimensional feature space non-linearly. The dimension in the feature space is determined by the kernel function. For example, if the kernel function is a 2nd order polynomial for a d dimension vector, the dimension in the feature space is d(d+3)/2. This effectively captures more information to depict the input. In practice, various kernel functions can be employed as long as they comply with the Mercer condition defined in Eq. (2.10). Second, the solution for SVR is the globe optimal. The quadratic optimization process guarantees the globe minima solution. Moreover, as shown in[15], there are two parts associated with a learning machine designed according to ERM principles: the empirical risk of the training samples Remp(w) and the confidence interval Φ R(w ) ≤ Remp (w ) + Φ (h ) ,
(2.26)
where h is called the Vapnik-Chervonenkis (VC) dimension and Φ is inverse proportionate to the 1/h. The structural risk minimization (SRM) principle[13] provides a new way of designing learning machines by endeavoring to locate a hyperplane that can minimize the sum of the Remp(w) and Φ, and achieve the
A Novel Intelligent Monitor for Manufacturing Processes
71 71
goal of R(w) minimization. As opposed to the ERM approach commonly employed in machine learning, the formulation of the SVR embodies the SRM principle[15]; therefore, SVR not only constructs an optimal hyperplane that displays a strong generalization capacity, but also overcomes overfitting and other difficulties in the design structure of the learning machine. Third, the system does not require complicated calculation during monitoring and hence can operate in real-time. Although the dimensions of the feature space increases, the construction of the optimal hyperplane does not require extra computational load. The reason for this is the introduction of the kernel function. By replacing the dot product with an appropriately chosen kernel function, it is implicit that performing a non-linear mapping onto a high dimensional feature space does not increase the number of tunable parameters. This means that the kernel function computes the dot product of the feature vectors corresponding to the two input vectors. As a result, the exceptional ability does not bring computational complexity and load. Table 1. The procedure for training the SVR-based condition monitoring system Step 1.
Step 2. Step 3. Step 4. Step 5. Step 6.
Step 7. Step 8.
Step 9. Step 10.
Obtain training sample sets in normal condition
S i = (x ij , yij ) nj=1
i = 1, 2, …, M; where Si ∈ S = {Si | i : number of training sample sets} (Optional) Signal processing for the de-noising. Obtain a sample set derived from S using statistics to train the system. Define a kernel function K(x, z); C could be determined by cross-validation. Define the loss function as Eq. (2.22). Apply quadratic optimization, Eq. (24), to find out αi,αi* (αi,αi* ≠ 0) → xi (corresponding support vectors according to Eq. (2.13)) → acquiring b* (Eq. (2.25)). xi, αi, αi*, # Eq. (2.15) → f(x, α, α*). for i = 1:M Eq. (20) → log (Pi); end. Log (Pi) → N(m,σ) → the threshold of log (Pi) = m - 3σ. n Monitoring on-line: obtain the real-time data set, S new = ( x j , y j ) j =1 , and substitute them into the system to calculate log (Pnew); if log (Pnew) < m - 3σ, then condition = normal; otherwise condition = fault; end. Note: x → y: y can be acquired from x;
X # y: substitutes x into y.
Finally, the system requires only the training sample sets produced under normal operating conditions thus making it more feasible than other techniques. Given a number of training sample sets, the SVR technique will calculate
72 72 M. Ge and Y. Xu
dependency between input vectors and outputs; moreover, its loss function defined by the noise density model helps compute thresholds of the likelihood automatically. Thus, the system has the intelligence to cope with different processes. All of these features make the proposed process monitor very attractive for real-world applications. In summary, the procedures for training the monitor and carrying out the on-line monitoring are in Table 1. In practice, three major factors may affect the performance of the system. First, an appropriate kernel can make the system more efficient. The kernel function reflects the geometric relationship between the new input vector and a specific support vector. The Gram matrix K = {Ki,j}, where Ki,j = K(xi, xj), of the kernel is positive definite, which contains all necessary information for the learning algorithm. In condition monitoring, K is used to measure the similarities between normal features and faulty features. In this paper, the exponential RBF kernel function is employed. Second, the size of the input vector may pose a computational limitation. In high-speed stamping, a stroke is completed in a fraction of a second. Meanwhile, SVR requires significant amount of computation load. In fact, to protect precise moulds and ensure quality, real-time factors and the accuracy of detection must have first propriety in process monitoring. Therefore, the size of input vectors should be as small as possible. Third, with regard to the signal type, it should be noted that an appropriate signal with high SNR not only effectively represents physical characteristics, but can also make the monitoring system more robust.
4 Experimental Study 4.1 Experimental design
To verify the performance, the new monitor has been applied on a complicated process. Sheet metal stamping operation is one of the most commonly used manufacturing processes which involves the design and lubrication of the mould, the transient elastic and plastic deformation of the sheet metals, as well as the static and dynamic behavior of the press[16]. Nearly 40 process variables could affect the operation and the product. It is quite difficult in monitoring its processes. The experiments have been carried out on a C-frame double column press (SEYI: SN1-25), a typical machine used in practice. The maximum stamping force is 25 tons and the maximum speed is 110spm. It is reported that strain signals are most commonly used in the case of monitoring stamping operations [16], and rely on gathering meaningful information and robust physical properties. Here a strain signal was employed, with the result that signal processing it was not required. The raw signals are acquired from a strain sensor (Kistler 9232A)
A Novel Intelligent Monitor for Manufacturing Processes
73 73
mounted on the middle of the press frame. The signals were sampled using a NI DAQ (PCI 4452), installed in an Intel Pentium III 550MHz PC. Sheet metal stamping is a transient process in high speed, in order to capture the signal every time at the same instance, a trigger mechanism, a proximity sensor installed near the main shaft, was developed. Every time when the key way of the shaft rotates passing the sensor, a pulse is generated and sent to computer to trigger the start of the sampling. Three experiments were conducted covering a range of situations. The first and second experiments were conducted in a laboratory setting. In the third experiment, data from real-world application were acquired using a JG21-14 press (manufactured by Jieyang Machine Tool Works). Parts produced were manufactured under normal conditions. When working conditions changed, so-called faults developed and deviations occurred. Table 2 provides descriptions of conditions that occurred frequently in practice. Table 2. Fault types occurring frequently during stamping process Type -A B C D E F G
Description No detection: under normal conditions. Misfeed: workpiece is not aligned with the dies. Slug: slug(s) left on the surface of sheet metal. Too thick: workpiece thickness is thicker than normal. Too thin: workpiece thickness is thinner than normal. Material missing: the material is used up. Crack: occurred in the drawing operation due to bad lubrication, materials or design. Double stroke: part left in upper module.
4.2 Experiment conduction
The first experiment incorporated a blanking operation with a simple mould. The parts produced in the experiment are simple buckets used for mounting I/O boards in desktop computers, whose material is SPCC and the blank dimension is approximately 32 cm. The normal working condition is 85spm with workpiece 1.1mm. The signals in Figure 2 show that except the last two conditions, the other signals are all similar in appearance. The second experiment used a multi-station mould, and included the three stamping categories. It produces the covers of the rear part of a small type motor. The material is SPCE and the dimension of the parts is approximately 35mm x 6.6mm (diameter x height). Its normal working condition is 80spm with workpiece 1.0mm. Due to its quite long multi-station mould and operation procedure, two strain sensors were mounted on the left and right sides of the press to detect faults. The strain signals from both sensors merged and provided us with boosted information. Figure 3 presents its typical strain under different conditions.
74 74 M. Ge and Y. Xu
Fig. 2. Typical strain signals under different stamping conditions in the first experiment
Fig. 3. Typical strain signals under different stamping conditions in the second experiment The third experiment also used a multi-station mould; its products are the battery caps. The material is copper and the approximate dimension of the part is 12mm x 4.3mm (diameter x height), designed to work under 150spm with workpiece 0.7mm. It is noted the normal stamping force of the press is 140KN. There are numbers of presses working in the same area, which may cause disruption to the process, and the press is timeworn. Figure 4 consists of illustration strain signals under different conditions in the experiment. Material specifications and the design of the mould caused the battery caps to break easily. Moreover, due to the age and condition of the press, along with adverse working conditions, the SNR of the strain signals in the third experiment was low. As shown in Figure 4, only serious cracks were detected by traditional condition monitoring systems, i.e., peak value, RMS, kurtosis and skewness. In all
A Novel Intelligent Monitor for Manufacturing Processes
75 75
cases, operators are informed of the results too late to ensure reductions in the number of faulty parts manufactured. No obvious phenomenon can be detected to suggest that cracks will presently occur, and it is almost impossible to diagnose these faults.
Fig. 4. Typical strain signal under normal condition in the third experiment It was for just these circumstances that a condition monitoring system based on SVR was developed. In order to develop the system, three major factors need to be determined: the value of ε, the type of kernel function, and the training sample sets. In section 3, ε was set at 0, and the exponential RBF kernel function was chosen; meanwhile, its σ was set at 0.4 As to the third factor, training sample sets, a range of data was acquired from signal(s) according to the scan rate within each stroke. To eliminate meaningless data, and to ensure effective coverage of the entire stroke, specific points were drawn. In the first experiment, 512 points were selected for each stroke. In the second and third experiments, 1024 points and 640 points were selected respectively. Further, because the dimension of the input vector has an obvious impact on the computational load of the system, the number of points was reduced on the basis of maintaining crucial features of the processes whilst still aiming to guarantee the speed of the system. To this end, one in two points were chosen for each stroke from the selected data in the first experiment, and one in four from both the second and third experiments. In the SVR model, the regression step is T = 10. Hence, the model is as follows: y i = f (x i ) + ξ i = f ( xi −1 , xi − 2 ,..., xi −T ) + ξ i ,
i =1, 2, 3, …, n.
(2.27)
where, x is the point of the strain signal. A total of 130 sample sets were collected under normal and faulty conditions in each experiment. 30 training sample sets produced under normal conditions and selected randomly act as training sample sets for each experiment. The algorithm for training SVR is Sequential Minimal Optimization (SMO), which has been well defined by Platt[15]. The algorithm ensures the solution of quadratic programming optimization problem satisfies the time requirement of the system.
76 76 M. Ge and Y. Xu
4.3 Experiment results
Three experiments were carried out in progression according to the procedure defined above. The distributions of the likelihood for the three experiments are illustrated in Figure 5 (a), (b) and (c), respectively, where the solid line is the mean value of the likelihood, the broken line is the threshold, * represents the normal samples, and “•”represent abnormal samples
5 Discussion of Results In summary, the results of the three experiments are represented in Table 3. In the first experiment, the number of the sample sets produced under normal conditions numbered 60, and the remaining 70 sets were the result of misfeeds, slugs, and material either too thick or too thin. The second experiment consisted of 90 normal condition sample sets, with 40 sets resulting from misfeeds, slugs and double strokes. In the third experiment, suspending the flow of lubricating oil created faulty condition sample sets. Parts produced under such conditions are faulty and must to be discarded; however, there is no clear information with regard to strain signals. Table 3: Experimental results using an SVR-based system Experiment 1st 2nd 3rd
Training sample set 30 30 30
Testing sample set Normal sets Abnormal sets 30 70 60 40 70 30
Success rate 99% 94% 89%
Figures 5 (a), (b) and (c) show that the likelihood of the samples under normal conditions is quite centralized in the first and second experiments, and that samples in abnormal conditions have a wide range of distribution. Due to the poor SNR in the third experiment, signal patterns for different working conditions are similar, which was reflected in the distribution of likelihood. However, the experimental results show that the system has robust performance. The approach extracts the invariant features from the strain signal for normal conditions. Moreover, to compare the new approach, the authors also employed the backpropagation (BP) approach, a traditional ANN technique; a Hidden Markov Model approach, another modeling technique; and a support vector machine (SVM) approach. In each experiment the same sample was used. Table 4 presents the results[16, 17].
A Novel Intelligent Monitor for Manufacturing Processes
77 77
Fig. 5. Experimental Results (a) Top left: results for Experiment 1 (b) Top left: results for Experiment 2 (c) Bottom: results for Experiment 3 Table 4: Experimental results for fault classification using other approaches Source of training sample Samples in Experiment 1 Samples in Experiment 1 Samples in Experiment 1 Samples in Experiment 2 Samples in Experiment 2 Samples in Experiment 3 Samples in Experiment 3
Approach BP network (two layers) Hidden Markov Model Support Vector Machine BP network (two layers) Support Vector Machine BP network (two layers) Support Vector Machine
Success rate 93% 92% 99% 84% 96% Failed (<50%) 67%
In comparison, the SVM performed perfectly. The SVM and SVR techniques share a common basic technical background. The former requires samples from all conditions for training the system, which results in the acquisition of information
78 78 M. Ge and Y. Xu
richer than the latter; however, unstable signals in the third experiment make it difficult to choose the training samples. However, by modeling for normal condition signals and considering the characteristics of noise density, the proposed system shows a significant improvement in the third experiment. More appealing though, the proposed system only requires samples from normal operating conditions for training, which makes the approach more ideal for real-world application.
6 Conclusions A new intelligent manufacturing process monitor was developed based on the SVR and noise density model. Based on the experimental results, the following conclusions can be drawn: (a) Derived from statistical learning theory, SVR realizes the SRM principle in its design. Unlike other learning algorithms, it overcomes the drawbacks of local minima and overfitting, and it has a greater capacity for generalization. This makes the system intelligent when dealing with complicated processes. (b) SVR nonlinearly maps input vectors onto a higher dimensional feature space via simple kernel, where a linear regression is constructed so that accuracy is increased significantly. Simultaneously, kernel overcomes the problems of dimensionality in both computation and generalization. This guarantees the system is able carry out tasks accurately. (c) The perfect characteristics of sample sets derived under normal condition for training, and the robust performance within selection parameters, makes the condition monitoring system more practicable. (d) Together with the appropriate noise density model, the new similarity measurement according to the system could reveal the characteristics of the manufacturing process in different conditions. (e) A condition monitoring system using SVR and related density noise model has the potential for application in other manufacturing processes.
Acknowledgement The partial support of this work by the Innovation and Technology Commission of Hong Kong, under grant ITF AF/79/99, is gratefully acknowledged.
References 1.
S. G. Tzafestas Advances in Manufacturing: Decision, Control and Information Technology. London: Springer, 1999.
A Novel Intelligent Monitor for Manufacturing Processes
2. 3. 4. 5. 6. 7.
8. 9. 10. 11.
12. 13. 14. 15. 16. 17.
79 79
J. J. Gertler, Fault Detection and Diagnosis in Engineering Systems. New York: Marcel Dekker, 1998. W. G. Fenton, T. M. McGinnity, and L. P. Maguire, “Fault diagnosis of electronic systems using intelligent techniques: A review,” IEEE Trans. Syst., Man, Cybern. C, vol. 31, no.3, pp. 269-279, 2001. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer Verlag, 1995. V. N. Vapnik, “An Overview of Statistical Learning Theory,” IEEE Trans. Neural Networks, vol. 10, no.5, pp. 988–1000, 1999. S. Gutta, J. R. J. Huang, P. Jonathon, H. Wechsler, and L. R. Rabiner, “Mixture of Experts for Classification of Gender, Ethnic Origin, and Pose of Human Faces,” IEEE Trans. Neural Networks, vol. 11, pp. 948–960, 2000. T. B. Trafalis and H. Ince, “Support Vector Machine for Regression and Applications to Financial Forecasting, IJCNN 2000,” Proceedings of the IEEE-INNS-ENNS International Joint Conference on, vol. 6, pp. 348–353, 2000. D. J. Sebald and J. A. Bucklew, “Support Vector Machine Techniques for Nonlinear Equalization,” IEEE Trans. Signal Processing, vol. 48, pp. 3217– 3226, 2000. J. A. K. Suykens and J. Vandewalle, Nonlinear Modeling: Advanced Blackbox Techniques. Boston: Kluwer Academic Publishers, 1998. R. J. Schilling, J. J. Carroll, and A. F. Al-Ajlouni, “Approximation of Nonlinear Systems with Radial Basis Function Neural Networks,” IEEE Trans. Neural Networks, vol. 12, pp. 1-15, Jan. 2001. K.-R. Müller, A. J. Smola, G. Rätsch, B. Schölkopf, J. Kohlmorgen, and V. N. Vapnik, “Predicting Time Series with Support Vector Machines,” in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C.J.C. Burges, and A.J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 243–254. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods. Cambridge, UK: Cambridge University Press, 2000. V. N. Vapnik, Estimation of Dependencies Based on Empirical Data. Berlin: Springer Verlag, 1982. J. Mercer, “Function of Positive and Negative Type and Their Connection with Theory of Integral Equations,” Philos. Trans. Roy. Soc. A 209, pp. 415–446, 1909. B. Schölkopf, C. J. C. Burges, A. J. Smola, Eds. Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT Press, 1999. M. Ge, G. C. Zhang, R. Du, and Y. Xu, “Application of Support Vector Machine Based Fault Diagnosis,” in Proceedings of XV IFAC World Congress, Barcelona, Spain, 2002. M. Ge, R. Du, and Y. Xu, “Condition Monitoring Using Marginal Energy and Hidden Markov Model,” to be appeared in Journal of Control and Intelligent Systems.
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique Yun-Hui Liu, Dan Ding and Miu-Ling Lam Department of Automation and Computer Aided Engineering, Chinese University of Hong Kong, Shatin, NT, Hong Kong [email protected]
Abstract. The ray-shooting technique is a powerful tool for handling geometric information in Computational Geometry and Computer Graphics. This chapter demonstrates that this technique can be effectively applied to stability analysis and planning of 3-D form-closure grasps in continuous and discrete domains. In formclosure grasps, any external wrench applied at the grasped object can be always balanced by the grasping forces at the contact points. A ray-shooting based formulation, which leads to several simple and efficient algorithms, is presented for qualitative test and computation of 3-D form-closure grasps as well as for automatic fixture layout design of workpieces.
1 Introduction Multi-fingered robotic hands have been extensively studied in robotics and automation since the late 1980s for their great potential in performing dexterous and fine manipulation tasks that are carried out by human hands. To develop a robot hand that is as skillful as human hands, it is necessary to solve various technical issues including mechanical design [3], [25], [27], kinematics [9], [16], grasping analysis and control [6], [10], [21], [26], dexterous manipulation [33], etc. Grasp analysis and planning addressed here are problems of analyzing and judging stability of a grasp and finding a stable grasp on a given object, being the most fundamental issues in the field. The lack of real-time solutions to those problems has been widely recognized one of major factors that obstruct wide applications of multi-fingered robot hands in industry and other related sectors. The stability of a grasp is characterized by form-closure and force-closure properties [1], [25]. Under a form-closure grasp, any external wrench applied at the grasped object can be balanced by grasping forces of the robot hand. The difference of force-closure from form-closure lies in that kinematics of robot hands is taken into account in force-closure analysis [1], while the form-closure means geometric property only. Many research efforts have been directed to testing form-closure property of a given grasp in last two decades. Salisbury and Roth [25] have shown that a necessary and sufficient condition for form-closure grasp is that the primitive contact wrenches resulted by contact forces at the T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 80−109, 2004. Springer-Verlag Berlin Heidelberg 2004
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
81 81
the primitive contact wrenches resulted by contact forces at the contact points positively span the entire wrench space. This condition is further proven to be equivalent to that the origin of the wrench space lies strictly inside the convex hull of the primitive contact wrenches[15], [18]. Nguyen [19] proposed a simple test algorithm for 2-finger form-closure grasps. Ferrari and Canny [5] use the largest sphere inside the convex hull for the qualitative test and optimization algorithm for 2D grasps. Trinkle [26] provided a formulation of quantitative test using Linear Programming. In [7], Han et. al. formulized the problem as a linear matrix inequality problem. Liu [10] developed a qualitative test algorithm of formclosure grasps by transforming the problem to a ray-shooting problem of a convex hull. Another important issue concerning grasp stability is the sufficient and necessary conditions for the number of fingers needed to achieve form-closure grasp. Reulaux [23] has shown that in frictionless case at least 4 and 7 fingers are required to achieve 2-D and 3-D form-closure grasps, respectively. Mishra et al. [15] found that 6 (resp. 12) fingers are sufficient for 2-D (resp. 3-D) frictionless form-closure grasps. Markenscoff et al. [14] further tightened the results by proving that 4 and 7 fingers are sufficient to achieve 2-D and 3-D form-closure grasps, respectively. When Coulomb friction is taken into account, 2 and 3 fingers are, respectively, sufficient in 2-D and 3-D cases. Grasp synthesis is another important research aspect, concerning the problem of placing contacts on an object with given geometry so as to prevent object motions. In [20], Nguyen presented a geometric approach to find maximal contact regions where two fingers can be positioned independently while maintaining a stable grip on a polygon. Liu [11] proposed an algorithm for calculating all planar formclosure grasps of n fingers on polygonal objects based on a new sufficient and necessary condition for form-closure. Ponce et al. [21] presented an approach in computing 3-finger stable grasps on planar polygonal objects with a projective algorithm based on linear programming and variable elimination among linear constraints. In [22], they further extended the approach to 3-D 4-finger grasps. The authors [4], [8] have recently developed an algorithm for computing form-closure grasps by iteratively moving the convex hull of the primitive contact wrenches towards the origin of the wrench space. However, the algorithm is not complete as the search may trap at a local minimum point of the heuristic function. An importance application of grasping analysis and synthesis is automatic fixture layout design, which is a problem of finding a set of fixturing points called fixels firmly holding a workpiece. Since the fixturing points must form a formclosure so as to reject any motion tendency caused by forces generated during a fabrication process, automatic fixture design is essentially the same as the synthesis problem of form-closure grasps. Wolter and Trinkle [31] proposed an algorithm for selecting discrete fixture points for polygonal/polyhedral parts to achieve stability against the gravity forces. The stability considered here is different from the form-closure. Form-closure imposes more and stricter constraints so that the difficulty of the problem is increased. Their algorithm does not yield the minimum number of fixturing points and the number of contacts cannot be specified beforehand. A complete algorithm has been proposed by Brost and Goldberg [24] for de-
82 82 Y.-H. Liu, D. Ding, and M.-L. Lam
signing planar fixtures using modular components in a discrete domain. Wallack and Canny studied a similar problem in [28], [29]. However, those algorithms can be hardly applied to 3-D grasp planning problems. Wang [30] presented an algorithm for optimal design for 3-D fixture synthesis in a point-set domain, but the algorithm is not complete. It should be noted that the works mentioned above did not take frictions into account. Mattikalli et. al. [13] proposed an algorithm for finding all stable orientation of assemblies with frictions under application of gravitational force, but the algorithm is not suitable for computation of formclosure grasps. In this chapter, we present a new framework using the ray-shooting technique for stability analysis and planning of 3-D form-closure grasps. The ray-shooting technique is a powerful tool for extracting geometric information in Computational Geometry and Computer Graphics. Based on the duality between convex hulls and convex polytopes, a ray-shooting problem can be transformed to a Linear Programming (LP) problem [17], which can be solved in real-time with advanced computer technology when the size of the problem is not huge. It will be demonstrated that the qualitative test and computation of a form-closure grasp, and automatic fixture layout design of workpieces can all be formulized as rayshooting problems. In detail, the qualitative test can be transformed to a problem of checking the intersection of the convex hull of the primitive contact wrenches with a ray from an interior point of the convex hull to the origin of the wrench space. This transformation leads to the development of an efficient algorithm for qualitative test of form-closure grasps. On the basis of this qualitative test algorithm, a simple and efficient heuristic method can be developed for searching for a form-closure grasp on the surface of a 3-D object. The heuristic search is effectively guided by the distance between the convex hull and the origin of the wrench space, which is obtained in a ray-shooting process. To overcome the local minimum problem in the heuristic search, a recursive problem decomposition strategy is introduced. The combination of the heuristic search with the recursive problem decomposition strategy results in a complete and efficient algorithm for finding a form-closure grasp in a discrete domain. Compared to other heuristic methods, this algorithm can always find a solution or report no solution. Compared to the exhaustive search, this algorithm is more efficient. Finally, this chapter addresses automatic fixture layout design on polyhedral workpieces and demonstrates that this fixture design problem can be also transformed to a ray-shooting problem. To demonstrate the effectiveness of this new formulation and efficiency of the algorithms, several numerical examples are presented in this chapter. This chapter is organized as follows. Section II reviews the frictional and frictionless grasp models and the definition of form-closure grasps. Section III presents the ray-shooting based qualitative test algorithm. In Section IV, we present an efficient local search algorithm and a complete algorithm for 3-D grasp planning. Section V demonstrates the application of the ray-shooting based formulization in automatic fixture layout design of workpieces. Finally, we conclude this chapter in Section VI.
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
83 83
2 Grasp Model and Form-Closure This section reviews the grasp models used in this chapter and the definition of form-closure grasps. 2.1 Assumptions The work presented here is based on the following assumptions on robot fingers and contacts among the fingers and the object: 1) The fingers and the grasped object are rigid bodies. 2) All contacts are point-to-point contacts, i.e. the fingers are assumed to be hard fingers. 3) The model of the object to be grasped is available. The object is represented by either a continuous surface or discrete points. The corresponding surface normal of any point on the surface is well-defined and obtainable. Robot finger Object frame
ri
Fig. 1 The coordinate frame attached to the object It should be noted that 3) in the assumptions only assumes the availability of the model of the 3-D object. No assumption has been made on the geometric shape of the object. The object under consideration could be a polyhedron or one with complicated surface. Denote the contact point of a finger on the surface by rj = (xj, yj, zj)T, j = 1, 2, …, n, where n is the number of fingers. The contact points rj are defined with respect to the object coordinate frame with origin at the center of mass (Fig. 1). Denote the surface normal of the object at point rj by nj. 2.2 Frictionless Grasps It is well known that seven frictionless contacts are necessary [15] to hold a 3-D object in form-closure and sufficient [14] for a 3-D object without rotational symmetries. We assume that seven hard fingers are to grasp a 3-D rigid body without rotational geometries in the absence of friction at the contact points. We denote a grasp by G = { ri }, i = 1, 2, …, 7, where ri denotes the contact point on the surface of the object.
84 84 Y.-H. Liu, D. Ding, and M.-L. Lam
In the absence of friction, grasping forces fi are in the normal directions of the surface of the object at the contact points. The force and moment τi, corresponding to grasping force fi, applied at the center of mass of the object is given by § fi · § fi · § ni · ¸ ¸¸ = α i ¨¨ ¨¨ ¸¸ = ¨¨ ri × n i ¸¹ © © τ i ¹ © ri × f i ¹
(2.1)
wi
where αi is a non-negative constant representing the magnitude of the grasping force. The combination of the force and moment is called wrench. The vector wi is called the primitive contact wrench. W=(w1, w2, …, w7) ∈ R6x7 is the wrench matrix of a 7-finger frictionless grasp.
Fig. 2 Linearization of a friction cone
2.3 Frictional Grasps We consider multi-fingered grasps with Coulomb friction. It is known that three fingers are sufficient to hold an object in 3-D frictional cases. Suppose that n ≥ 3 hard fingers are to grasp a rigid object in a 3-D workspace. Assume that the Coulomb friction with friction coefficient µ exists at the contact points. To ensure non-slipping at the contact point, the grasp force fi must satisfy f ix2 + f iy2 ≤ µ 2 f iz2
(2.2)
where (fix, fiy, fiz) denotes x, y and z components of the grasp force fi w.r.t. the contact frame located at the contact point on the surface of the object. The nonlinear
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
85 85
constraints in (2.1) geometrically define a cone called friction cone (Fig. 2). The grasp forces can be exerted in any direction within the friction cone. To simplify the problem, the friction cone is linearized by a polyhedral convex cone with m sides. Under this approximation, the grasp force fi can be represented as fi =
m
¦α
α ij ≥ 0
s ij ,
ij
(2.3)
j =1
where sij represents the j-th edge vector of the polyhedral convex cone for the grasp force fi. Coefficients αij are nonnegative constants. The wrench, corresponding to the grasp force fi, applied at the center of mass of the object is given by § Rf i · ¸¸ w i = ¨¨ © ri × ( Rf i ) ¹
(2.4)
where R denotes the rotation matrix of the contact frame with respect to the object frame. Substituting (2.3) into (2.4) derives wi =
m
¦α
ij
w ij
(2.5)
j =1
where § Rs ij · ¸ w ij = ¨¨ ¸ © ri × ( Rs ij ) ¹
(2.6)
Vectors wij are called primitive contact wrenches. The net wrench applied at the object by the n fingers is w net =
n
m
¦ ¦α
ij
w ij = W α
(2.7)
i =1 j =1
where W and α are given by
W = (w11 , w12 , " , w1m , " , w n1 , w n 2 , " , w nm )
α = (α 11 , α 12 , " , α 1m , " , α n1 , α n 2 , " , α nm )
(2.8)
W is a 6 × nm wrench matrix and its column vectors are the primitive contact wrenches. For convenience, wi with a single subscript i, instead of wij, is used to denote the i-th column vector of grasp matrix W, and αi is used to represent the ith component of vector α. 2.4 Form-Closure Grasps Form-closure is a measure of the capability of a grasp to completely constrain motions of the grasped object through the contacts under the application of external forces and moments. This century old concept is defined as follows:
86 86 Y.-H. Liu, D. Ding, and M.-L. Lam
Definition 1: Form-closure grasp: Suppose that a n-finger frictional/frictionless grasp is given. If any external wrench wext applied at the object can be always balanced by the grasp forces fi of the fingers, the grasp is said to be form-closure. It should be noted that the form-closure grasp in Definition 1 is called forceclosure grasp in [9], [19], [21]. We here follow the definition given in [1], [25], [26]. Form-closure can be also defined as follows using wrench matrix W: Definition 2: Form-closure grasp: W is the wrench matrix of an n-finger frictionless or frictional grasp. For any external wrench wext ∈ R6 applied at the object, if it is always possible to find an α with all αi ≥ 0 such that W α + w ext = 0
(2.9)
the grasp is said to be form-closure. It is well-known that a form-closure grasp is equivalent to that the origin of the wrench space R6 lies exactly inside the convex hull H(W) of the primitive contact wrenches wi. Any point P strictly inside the convex hull H(W) can be represented by N
P=
¦
N
¦α
α i wi ,
i =1
i
= 1,
αi ≥ 0
(2.10)
i =1
Since in form-closure grasps the origin of the wrench space is strictly contained by the convex hull of the primitive contact wrenches, there exist coefficients αi such that N
¦ α i wi = 0 i =1
N
¦α
i
=1
all α i > 0
(2.11)
i =1
Note that in eq. (2.11) the coefficients αi cannot be equal to zero so as to guarantee that the origin is not on the facets (boundary) of the convex hull.
3 Qualitative Test of Form-Closure Grasps Qualitative test is a problem of checking whether a grasp is form-closure. The problem definition is given as follows: Problem 1: Given the contact positions of the n fingers on the surface of the grasped object and their corresponding surface normal vectors, check whether the given grasp forms a form-closure. One possible method for the qualitative test is to construct the convex hull of the primitive contact wrenches. However, construction of a convex hull in six dimensional wrench space is not a trivial task according to the results in Computational Geometry. For sake of computational efficiency, it is necessary to develop an ap-
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
87 87
proach without constructing the convex hull. In this section, a simple and efficient qualitative check algorithm is presented based on the following observation: Theorem 1: Suppose that an n-finger frictional/frictionless grasp is given. Denote the convex hull of the primitive contact wrenches by H(W). Assume that point P is an interior point of H(W). The ray from point P to the origin O of the wrench space intersects the convex hull H(W) in a point Q only. The grasp is formclosure if and only if the distance between points P and Q is strictly larger than the distance between points P and O. Proof: For a convex hull H(W), the definition of its interior point P means that there always exists a r such that the spherical neighborhood region of point P: B( P, r ) = {x : x − P < r } lies exactly inside the convex hull. Since H(W) is a convex region, any ray emanating from its interior point P intersects H(W) only in a point Q on the boundary. Denote the intersection of H(W) with the ray PO by Q. The point Q lies on a facet of the convex hull H(W), i.e., on the boundary. Any point on segment PQ except for the endpoint Q is an interior point of the convex hull. If the distance between point P and the origin O is strictly smaller than that between points P and Q, the origin O must be an interior point. Therefore, there always exists a spherical neighborhood B(O, r) of the origin O that is strictly contained by H(W).
This theorem is illustrated in Fig. 3. In Fig. 3(a), the origin O is on the segment PQ and hence the grasp is form-closure. In Fig. 3(b), the origin O does not lie on segment PQ and thus the grasp is not a form-closure grasp. It should be emphasized that one interior point P is sufficient for the check. The point P cannot lie on the boundary of H(W). Obviously, a strictly positive combination of the primitive contact wrenches is an interior point of the convex hull H(W). Therefore, we can choose points P as the centroid of the primitive contact wrenches: P=
1 N
N
¦w
i
(2.12)
i =1
where N denotes the number of the primitive contact wrenches. To locate the intersection point Q of H(W) with the ray from the interior point P to the origin O of the wrench space R6, we first detect the facet E of H(W) intersected by the ray PO, and then calculate the intersection of the facet with the ray PO . The problem of detecting the facet is closely related to the ray-shooting problem of a convex hull: Definition 3: Ray-shooting problem: Let M be a given set of points in Rd. Assume that the convex hull H(M) contains the origin. Given a ray emanating from the origin, find the facet of H(M) intersected by this ray. It is well-known in Computational Geometry that the ray-shooting problem can be transformed to a linear programming problem based on the duality between convex hull and convex polytope. Assume that the convex hull H(M) contains the
88 88 Y.-H. Liu, D. Ding, and M.-L. Lam
origin. According to well-known results in Computational Geometry [17], the convex hull H(M) can be dually transformed to a convex polytope CT(M) defined by
Fig. 3 A form-closure grasp (a) and a non-form-closure grasp (b)
vT x ≤ 1 ∀v ∈ M and vice versa by virtue of a dual transformation T that maps a point vT=(v1,v2,...,vd) in Rd to the hyperplane:
v1 x1 + v 2 x 2 + • • • + v d x d = 1
(2.13)
This mapping is illustrated in Fig. 4. By the transformation T, a point v of the convex hull H(M) is transformed to a hyperplane T(v). The transformation T is reversible and has the following important properties: • A vertex of the convex hull H(M) is transformed to a facet of the convex polytope CT(M) and vice versa. For instance, in Fig. 4 the vertex D is transformed to the facet T(D). • A facet of the convex hull H(M) is transformed to a vertex of the convex polytope CT(M) and vice versa. For example, in Fig. 4 the facet DC of the convex hull is transformed to the vertex corresponding to the intersection of the facets T(D) and T(C). • Points exactly inside the convex hull H(M) are transformed to redundant hyperplanes of the convex polytope CT(M) and vice versa. For example, in Fig. 4 the interior point G of the convex hull is transformed to the redundant hyperplane T(G). Note that a ray-shooting problem assumes that the convex hull contains the origin. By applying a coordinate translation -P of on all points in R6, we readily change the origin point to point P, which lies exactly inside the convex hull. After the coordinate translation, the convex hull H(W) is dual to the convex polytope defined by
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
89 89
T(G) C Convex hull H(M)
D
T(B)
G
T(D)
T(C)
B
A
Convex polytope CT(M)
T(A)
Fig. 4 The duality between a convex hull and a convex polytope.
(w i
− P) x ≤ 1 T
i = 1, 2 , " , N .
(2.14)
Denote the direction vector of the ray PO by t. Based on the duality between convex hull and convex polytope, the ray-shooting problem is equivalent to a problem of maximizing the objective function: z = tT x
(2.15)
subject to the constraints in (2.14). Suppose that the optimal solution of the linear programming problem is eG = (e1 , e 2 , " , e 6 )T . According to the duality, the facet E of H(W) intersected by the ray PO is e1 x 1 + e 2 x 2 + " + e 6 x 6 = 1
(2.16)
Then, the intersection point Q of H(W) with the ray PO is the intersection of the hyperplane defined by (2.16) with the ray PO. To summarize the discussions above, the qualitative test of a form-closure grasp can be performed by the following algorithm: Algorithm 1
1) Calculate all the primitive contact wrenches wi corresponding to the given grasp. 2) Calculate the interior point P using eq. (2.12). 3) By the linear programming, calculate the optimal point eT=(e1,e2,...,e6) that maximizes function z in eq. (2.15) subject to the constraints in eq.
90 90 Y.-H. Liu, D. Ding, and M.-L. Lam
(14). The hyperplane T(e) corresponding to the optimal solution: eT x = 1 is the facet of H(W) intersected by the ray PO. 4) Calculate the intersection Q of the ray PO with the hyperplane T(e). The point Q is the intersection of H(W) with the ray PO. 5) If the distance between points P and Q is larger than that between points P and O, the grasp is form-closure; otherwise, it is not. 6) The algorithm ends. Fig. 5(b) plots computation time of the algorithm with respect to the number m of segments linearizing a friction cone for a 4-finger frictional grasps on a rectangle box shown in Fig. 5(a). Note that the time is obtained on a Sun Ultra I170 using C programming language. The grasp is not form-closure when the frictional coefficient is 0.3; it becomes a form-closure grasp when the frictional coefficient is increased to 0.5. From the results, the qualitative check is performed efficiently. For example, when m is equal to 100, which implies a reasonably accurate linearization of the friction cone, the computation time was 0.45 seconds when the frictional coefficient is 0.5. Finger 3
12000
10000
Friction coefficient=0.3 8000
Time (ms)
Finger 4 Finger 2
6000
4000
Friction coefficient=0.5 2000
0
Finger 1
(a)
0
50
100
150
200 m
250
300
350
400
(b)
Fig. 5 A 4-finger grasp (a) and the results of its qualitative test (b)
4 Grasp Synthesis in Discrete Domain Compared to the qualitative test, a more challenging and important problem is how to find a form-closure grasp on a given object. This problem cannot be avoided in motion planning of dexterous tasks. In this section, we address the grasp synthesis problem in a discrete domain, i.e. we will solve the problem of finding an n-finger form-closure grasp on an object represented by a set of discrete
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
91 91
points. It is assumed that the number of discrete points is big enough to yield specified accuracy of approximation. Following is the problem defined: Problem 2: Given a set Ω of surface points representing a 3-D object of arbitrary geometry and their corresponding surface normals, find an n-finger frictionless or frictional form-closure grasp in the point set Ω. A simple, efficient and complete algorithm will be presented for solving Problem 2 based on the ray-shooting technique. To better explain the algorithm, the description will be focused on the problem of finding a 7-finger frictionless grasp. The extension of the algorithm to frictional grasps is discussed in the last subsection of this section. 4.1 Algorithm Outline
The basic idea of the algorithm to be presented is to combine a local heuristic search with a recursive problem decomposition strategy. First, randomly select seven points from the points set and check whether the selected points form a form-closure. If the selected points achieve a form-closure grasp, the algorithm finishes; otherwise, a local heuristic search is executed until a form-closure grasp is found or a local minimum is encountered. The local search algorithm searches for a form-closure grasp in the direction of reducing the distance between the convex hull of the primitive contact wrenches and the origin of the wrench space. In case the distance reaches a local minimum during the search process, the point set is divided into subsets by a separating hyperplane in the wrench space and a few sub-problems defined in the subsets, which have less points than the original set, are generated according to existing conditions of form-closure grasps. To effectively handle the process, we adopt a search tree whose root represents the original problem. The sub-problems are represented as child nodes of the root. This process is recursively carried out on every node of the search tree until a form-closure grasp is found or all the nodes have been processed. An inevitable step in the proposed algorithm is qualitative test of a formclosure grasp, for which the ray-shooting based algorithm presented in the previous section is used. In addition to the qualitative test, an important property is that the qualitative test algorithm detects the closest facet of the convex hull to the origin of the wrench space. The closest facet will used to guide the direction of the local search algorithm. 4.2 Local Heuristic Search
The local search algorithm iteratively searches for finger positions that yield formclosure under the guidance of a heuristics. The local search starts from a randomly selection of an initial grasp. It searches for a form-closure grasp in the direction of reducing the distance between the convex hull of the primitive wrench and the
92 92 Y.-H. Liu, D. Ding, and M.-L. Lam
origin of the wrench space. The key idea here is to move the convex hull gradually closer and closer to the origin of the wrench space through a iterative process. This idea is illustrated in Fig. 6.
Fig. 6 The local heuristic search process
We consider that each of the given points in set Ω is connected with four neighboring points on the surface of the object. The connection means four possible motions of a finger at an iteration of the search. To simplify the problem, we here assume that at a iteration only one finger can be moved to its neighboring position. Consequently, for a frictionless grasp with 7 contacts there are totally 28 candidate grasps to be selected at every iteration. Observed from the qualitative test, it is desirable to move the convex hull H(W) towards the origin O of the wrench space R6 until the origin O completely lies inside the convex hull. Therefore, we consider the distance ||QO|| or ||PO|| as the heuristics and thus select the candidate grasp with the minimum distance. The local search algorithm is described in detail as follows: 1) k = 1; arbitrarily choose an initial grasp Gk = { ri }, for i = 1, 2, …, 7, where all ri ∈ Ω. Calculate the primitive contact wrenches wik. 2) Check whether Gk is form-closure using the qualitative test algorithm; if so, return Gk. 3) For each of the grasping position ri, locate the four adjacent positions ζ i ∈ Ω, for l = 1, 2, 3, 4; generate all the 28 combinations of candidate l
grasps CG i . l 4) For each candidate grasp CG i , calculate the corresponding ||QO( CG i )||. l
l
5) Find the candidate grasp CG* with the minimum value ||QO||. If ||QO(CG*)|| is not strictly smaller than ||QOk||, report that local minimum
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
93 93
is encountered; return Gk. Otherwise, k = k + 1; update Gk = CG*, wik(CG*), ||QOk|| = ||QO(CG*)||; go back to 2). Note that the local search terminates under two situations: when a form-closure grasp is obtained in 2) or when a local minimum is encountered in 5). In case a local minimum is encountered, the algorithm needs to select another initial grasp and start another round of the local search. In the next subsection, we propose a recursive problem decomposition strategy to divide the problem into sub-problems with fewer points so as to overcome the local minimum problem and at the same time to improve the computational efficiency.
Fig. 7 The separating hyperplane used in the problem decomposition
4.3 Recursive Problem Decomposition
To cope with the local minimum problem, a recursive problem decomposition strategy is employed to divide the problem in the original point set Ω into subproblems in sub-sets. The division is based on an important observation that the primitive contact wrenches corresponding to the seven contact points must not lie one side of any hyperplane passing through O to achieve a form-closure grasp. Having this observation, we can drastically reduce the number of eligible candidate grasps by neglecting those grasps with all the wrenches on one side of a specific hyperplane called separating hyperplane. Recall, in the qualitative test, that the hyperplane detected is the facet E of H(W) intersecting with the ray PO. Here, we select a hyperplane Y that is parallel to the facet E obtained in the local minimum iteration as the separating hyperplane (Fig. 7):
94 94 Y.-H. Liu, D. Ding, and M.-L. Lam
e1 x 1 + e 2 x 2 + " + e 6 x 6 = 0
(2.17)
The hyperplane divides the wrench space into two half-spaces, which are denoted by Y + and Y − , respectively. Y + is the half-space: − ; and denotes the half-space: Y e1 x 1 + e 2 x 2 + " + e 6 x 6 ≥ 0 e1 x1 + e 2 x 2 + " + e 6 x 6 < 0 and As shown in Fig. 4, the separating hyperplane Y divides the original point set Ω into two subsets: Ω (Y − ) = {r j ∈ Ω | w j ∈ Y − }
(2.18)
Ω (Y + ) = {r j ∈ Ω | w j ∈ Y + }
(2.19)
and
Since in a form-closure grasp the seven points should not all belong to subset Ω (Y − ) or subset Ω (Y + ) , points of form-closure grasps must be selected from both of these subsets. This gives rise to a division of the problem in the original set into problems in the subsets based on existence conditions of form-closure grasps. Six problems in the subsets need to be considered, which are grasps consisting of • one grasp point from subset Ω (Y − ) and six grasp points from subset Ω (Y + ) , •
two grasp points from Ω (Y + ) ,
•
three points from subsect Ω (Y − ) and four points from subset Ω (Y + ) , four points from subset Ω (Y − ) and three points from subset Ω (Y + ) ,
• •
subset Ω (Y − ) and five points from subset
five points from subset Ω (Y − ) and two points from subset Ω (Y + ) , and • six points from subset Ω (Y − ) and one point from subset Ω (Y + ) . The sub-problems obtained are represented as child nodes of the search tree whose root represents the problem in the original set (Fig. 8). The (i,j) in the node denotes that i and j grasp points must be selected from subsets Ω (Y − ) and subset Ω (Y + ) , respectively, in order to obtain a form-closure grasp in the corresponding subsets of points. From the six child nodes, we select one and carry out the local search of a form-closure grasp within the subset of points. When a new local minimum is encountered, the separating hyperplane is employed to further divide the subset into three or four subsets. Based on the existence condition, a new set of sub-problems are generated and represented as child nodes of the node selected. This process is performed recursively until a form-closure grasp is found or all the nodes have been explored. It should be noted that no child will be generated for a node if none of the combination of the subsets divided by the separating hyperplane satisfies the existence condition of form-closure grasps.
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
95 95
To determine which child should be traversed first, we consider a node with eligible contact points more evenly distributed over the point sets has a higher chance to contain form-closure grasps. Therefore, we define a heuristic function as follows:
Original Problem
(1,6)
(2,5)
(3,4)
(4,3)
(5,2)
(6,1)
Fig. 8 The search tree generated 2
· §7 (2.20) ¨ − ai ¸ ¦ ¹ i =1 © s where s is the number of subsets obtained in the decomposition and ai denotes the number of fingers (contact points) in the subset i of the points. For example, for the left-most node in Fig. 8, the a1 and a2 are 1 and 6, respectively. A node with a smaller f value will be traversed prior to the one with a larger f value. s
f ( < a1 a 2 " a s > ) =
4.4 Extension to Frictional Grasps
The difference in frictional case from frictionless case lies that a contact point results in m primitive contact wrenches, so the primitive contact wrenches corresponding to a grasp point may lie on both sides of the separating hyperplane. This requests us to divide the original point set Ω into three subsets: Ω(Y − ) = {ri ∈ Ω | wij ∈ Y − , for all j = 1,2,", m} Ω(Y + ) = {ri ∈ Ω | wij ∈ Y + , for all j = 1,2,", m} −
+
(2.21) (2.22) (2.23)
and Ω (Y ) = {ri ∈ Ω | ri ∉ Ω (Y ), ri ∉ Ω (Y )} where wij denotes the primitive contact wrenches. Ω(Y − ) denotes the subset containing contact points with the m primitive contact wrenches fully located in the half-space Y − . Ω(Y + ) denotes the subset containing contact points with the cor0
responding m primitive contact wrenches fully located in the half-space Y + . Ω(Y 0 ) denotes the subset containing contact points with the corresponding m primitive contact wrenches located on the both sides of the separating hyperplane. Since in a form-closure grasp, the n grasp points should not all belong to subset Ω(Y − ) or subset Ω(Y + ) . This gives rise to a division of the problem in the origi-
96 96 Y.-H. Liu, D. Ding, and M.-L. Lam
nal set into problems in the subsets based on the existence conditions of formclosure grasps. Grasp points of form-closure grasps must be selected from the subset(s) as stated in any one of the following combinations: • Ω(Y − ) , Ω(Y + ) and Ω(Y 0 ) , • Ω(Y − ) and Ω(Y + ) only, • •
Ω(Y − ) and Ω(Y 0 ) only, Ω(Y + ) and Ω(Y 0 ) only, and • Ω(Y 0 ) only. To illustrate the idea, let’s consider a 3-finger frictional grasp. Eight problems in the subsets need to be considered, which are grasps consisting of • one grasp point from subset Ω(Y − ) , one grasp point from subset Ω(Y + ) and one grasp point from subset Ω(Y 0 ) . •
two grasp points from subset Ω(Y − ) and one grasp point from subset Ω(Y + ) ,
•
one grasp point from subset Ω(Y − ) and two grasp points from subset Ω(Y + ) ,
•
two grasp points from subset Ω(Y − ) and one grasp point from subset
Ω(Y 0 ) , •
one grasp point from subset Ω(Y − ) and two grasp points from subset
Ω(Y 0 ) , •
two grasp points from subset Ω(Y + ) and one grasp point from subset Ω(Y 0 ) ,
•
one grasp point from subset Ω(Y + ) and two grasp points from subset Ω(Y 0 ) , and
•
all the three grasp points from subset Ω(Y 0 ) .
Fig. 9 The search tree generated for a 3-finger frictional grasp
The sub-problems obtained are represented as child nodes of the search tree whose root represents the problem in the original set (Fig. 9). The (i,j,k) in the
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
97 97
node denotes that i, j and k grasp points must be selected from subsets Ω(Y − ) , Ω(Y + ) and Ω(Y 0 ) , respectively. From the eight child nodes, we select one by a similar heuristics and carry out the local search of a form-closure grasp within the subset of points. When a new local minimum is encountered, the separating hyperplane is employed to further divide the subset into subsets. Based on the existence condition, a new set of subproblems are generated and represented as child nodes of the node selected. This process is performed recursively until a form-closure grasp is found or all the nodes have been explored. It should be noted that no child would be generated for a node if none of the combination of the subsets divided by the separating hyperplane satisfies the existence condition of form-closure grasps. 4.5 Completeness and Complexity
Although the algorithm employs a heuristics to guide the local search and selection of the sub-problems, it always finds a form-closure grasp or reports no solution after all the nodes in the search tree have been visited. This can be explained in the following. The algorithm divides using the separating hyperplane the problem into problems in subsets of the points when the distance between the convex hull and the origin of the wrench space reaches a local minimum. The division makes the numbers of points in the subsets smaller than those in the original point sets. The search reaches a leave node when the division results in zero subset or all the points lie on one side of the separating hyperplane. In both cases, it is guaranteed that no form-closure grasp exists in the subsets. The algorithm will always terminate with reporting no solution after all the leave nodes are visited. Therefore, the proposed algorithm is complete as it always finds a solution if there is and otherwise reports no solution. Then, consider the computational complexity of the algorithm. The complexity depends on the number of nodes of the search tree. Proposition 1: Denote by n the number of fingers/contact points. In frictionless cases, the number of child nodes of each node is not more than 2 n − 2 ; In frictional case, the number of each node is not more than 3n-2. This can be simply proven. The separating hyperplane divides the subset for each finger into at most 2 subsets in frictionless case. According to the existence conditions, there are at most 2 n − 2 sub-problems to be considered. Therefore, there are not more than 2 n − 2 children for each node in the search tree. In frictional case, the separating hyperplane divides a subset into at most 3 subsets, so there are not more than 3n − 2 children for each node. Since the search tree generates new nodes whenever a local minimum is detected, the computational efficiency is strongly dependent of the number of local minimum points. Denote by K the total number of the local minima in the configuration space of grasps. Note that the number K is determined by the surface geometry of the object. Once the object is specified, the number K is a constant. For
98 98 Y.-H. Liu, D. Ding, and M.-L. Lam
simplicity, we assume that the local minima are uniformly distributed over the configuration space. The average number of attraction points for each local mini-
Nn , where N is the total number of the discrete geometric points on the K object surface. The corresponding number of points in the subset for each finger is N N . In other words, if there are less than points in subset set of each finn nK K mum is
ger, we can assume that there is only one minimum point (global minimum) in the subset. When ||QO|| is used as the heuristics, the local search process will definitely find a solution if there are form-closure grasps in the subsets. This is because the distance will be convergent to the minimum value, which is negative. Therefore, Proposition 2: Assume that the K local minimum points are uniformly distributed in the configuration space. In case there is a form-closure grasp in the given discrete point set, the number of nodes generated in the search tree is O(
K ln K ) n
if the search tree is expanded in the breadth-first strategy. Proof: For simplicity, assume that at each decomposition the separating hyperplane divides points in each subset equally, i.e., the numbers of points in the two new subsets obtained after the division are the same. In the frictionless case, the level l of the search tree leading to a smaller number than the average number N of attraction points from each local minimum in the subsets is calculated by nK
N N ln K =n l = l n ln 2 2 K
(2.24)
If the breadth-first search is adopted, the total number L of nodes generated in the search tree satisfies: 1
ln K ln K ln K n ( K − K n ) = O( K) ( 2 − 2) l = L≤ n n ln 2 n ln 2
(2.25)
A similar result can be obtained for frictional grasps. It should be noted that the number of nodes is independent of the number of points representing the object in case there exists a solution. The complexity of the tree is mainly determined by the surface geometry of the object and the number of fingers. The total computational time is equal to the product of the number of nodes with the times of calling the local search process. Note that the local search algorithm is efficient because it is essentially a gradient-based algorithm and the ray-shooting based qualitative test algorithm takes a time linear to the number of the primitive contact wrenches. The problem is more complicated when there exists no form-closure grasp in the given problem. Theoretically, if the corresponding grasp is not form-closure at the global minimum point in the subsets, it is not necessary to carry out any further investigation. However, since no method is available to check that the mini-
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
99 99
mum reached in the local search process is the global one in the subsets, it is necessary to further divide the subsets. From the nature of the strategy adopted in the algorithm, it is reasonable to consider that most of the points are on one side of the separating hyperplane obtained at the global minimum. This implies that the algorithm would be able to understand non-existence of a solution in the subsets with a few times of further decomposition. Therefore, a weaker result can be obtained that the number of nodes generated in the search tree when there is no solution in the problem is similar to that when there exists a solution. When the distance ||PO|| is used as the heuristics, the complexity is similar. However, a rigorous proof is difficult and necessary. Although the discussion above is based on assumptions, this algorithm works more efficiently than the exhaustive search algorithm whose time is O( N n ) . Compared to other heuristic approaches, the algorithm is complete. 4.6 Implementation
We have implemented the proposed algorithm using MATLAB. Two numerical examples are used to demonstrate the usefulness and efficiency of the algorithm. The first example concerns computation a 7-finger frictionless grasps and the second example addresses computation frictional grasps. In both examples, we used the model of a turbine airfoil (Fig. 10) with 1546 predetermined candidate locations and surface normals. Turbine airfoils are good examples of 3-D workpieces with complex geometry. The geometric shape of an airfoil is primarily defined by its aerodynamics. However, the geometric representation in a CAD system is usually approximated by parametric surfaces such as B-splines. Only a dense set of points of the surfaces are defined exactly as calculated in the aerodynamic analysis. In order to minimize the effects of the geometric approximation, the airfoil is required to be fixtured or grasped at some of these precise surface locations in its manufacture and inspection. Therefore, the point-set constraint is imposed by the practical conditions related to the functional and/or manufacturing requirements. In the first example, seven frictionless fingers are to grasp the airfoil. The algorithm takes 68 iterations in total to obtain a form-closure grasp. The initial grasp, selected randomly, shown in Fig. 11(a) is non-form-closure. After 62 iterations, the algorithm encounters a local minimum in the performance measure ||PO||. A separating hyperplane is defined to divide the candidate contact points into two subsets and the algorithm performs the local search process in the state < 3 4 >. Then, a form-closure final grasp is found after 6 more iterations (Fig. 11(b)). Fig. 12 plots the distance difference ||PO|| - ||PQ|| and the distance ||PO|| against the iteration number. In second example, we consider the problem of finding a frictional grasp on the airfoil Assume that the number of fingers is 4. The friction cone at each contact point is linearized with a polyhedral convex cone with 20 segments (i.e. m = 20). The algorithm is required to obtain a 4-finger form-closure grasp on the airfoil model in the presence of friction. A friction coefficient µ = 0.2 exists between the
100100Y.-H. Liu, D. Ding, and M.-L. Lam
object and each finger. The initial grasp upon random selection is non-formclosure (Fig. 13(a)). The algorithm performs the local search in obtaining trajectories of finger motions in the direction to decrease ||PO||. In the 62nd iteration, the distance between P and O reaches a local minimum value (Fig. 14). The search tree is then generated and the child (1,1,2) is adopted. Then, a new grasp is selected from points in the subsets corresponding to node (1,1,2) and another round of local search was executed. Finally, a form-closure grasp is obtained (Fig. 13(b)) at the 71st iteration. The results ascertained the efficiency of the proposed algorithm.
Fig. 10 An airfoil approximated by 1546 discrete points
Fig. 11 Frictionless grasp: the initial grasp (a) and the final grasp (b)
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
101 101
Fig. 12 Frictionless grasp: plots of the distances ||PO||-||PQ|| and ||PO||
Fig. 13 Frictional grasp: The initial grasp (a) and the final grasp (b)
5 Automatic Fixture Layout Design A closely related problem to mutli-fingered grasping is automatic fixture layout design, which is to find a set of fixturing points (locators and clamp) firmly holding a workpiece. A firm holding implies that the fixturing points must form a
102102Y.-H. Liu, D. Ding, and M.-L. Lam
form-closure. For sake of safety during fabrication process, in fixture layout design all contacts are conservatively considered frictionless, and thus the problem is similar to synthesis problem of frictionless form-closure grasps. Since seven frictionless contact points are sufficient and necessary to form a form-closure, we suppose that the number of fixturing points to be found is 7. This section will show that the ray-shooting problem can be also applied to automatic fixture layout design on polyhedral workpiece. The problem addressed here is defined as follows: Problem 3: Given a geometric model of a polyhedral workpiece with welldefined surface normal vectors and vertices information, (1) find an eligible set of fixturing surfaces on which seven fixturing locations satisfying form-closure property exist; (2) find seven fixturing points on the eligible set of surfaces which ensure the total constraint and an accurate localization of the workpiece.
Fig. 14 Frictional Grasps: Plots of the distances ||PO|| and ||QO||
5.1 Eligibility Set of Fixturing Surfaces
Fixturing surfaces are the surfaces of a workpiece used to locate and clamp the workpiece where functional fixturing elements including locators and clamps are in contact with these surfaces. Besides some trivial requirements like nonmachining surfaces and surfaces with accessible normal directions, the primary concern when selecting the fixturing surfaces is whether there exists possible fixturing points yielding form-closure on them. A set of fixturing surfaces are said eligible if there exists fixturing points on them yielding form-closure.
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
103 103
On a set of eligible surfaces, it is always possible to find seven fixturing points ensuring form-closure. A simple condition is presented here to check the eligibility of a set of surfaces. A polyhedral workpiece is comprised of convex as well as concave faces. Note that a concave surface can be always decomposed into convex ones. The position vector ri of a contact point on a convex surface can be represented as a convex combination of the vertices of the surface. Denote by vij the coordinates of vertex j on the i-th surface and by cij the corresponding multiplier. Then position ri of the i-th fixturing point on the surface is a convex combination of vij, i.e.,
ri =
ti
¦c v
(2.26)
ij ij
j =1
where ti is the number of the vertices of the i-th convex surface. The positive multipliers cij satisfy ti
¦c
ij
=1
(2.27)
cij > 0
j =1
Note that cij are chosen to be strictly positive to ensure that the fixturing points calculated are inside the surfaces and not at the vertices or on the edges. cij are the variables of interest, as they define the location of each fixturing contact. Then the wrench applied at the object by the i-th contact can be rewritten as:
· § ti ¸ ¨ cij ni ¸ § ni · ¨ i =1 ¸¸ = ¨ t wi = ¨¨ ¸= © ri × ni ¹ ¨ i ¸ ¨ cij vij × ni ¸ ¹ © j =1
¦ ¦
ti
¦c j =1
§ ni · ¨ ¸ ¸ © vij × ni ¹
ij ¨
(2.28)
Let
§ ni · ¸ bij ∆¨¨ ¸ v × n ij i © ¹
(2.29)
which is called the vertex contact wrench here. Then introduce a matrix B:
(
B = b11 , b12 ,..., b1t1 ,..., bk1 , bk 2 ,..., bkt k
)
(2.30)
where k is the number of convex surfaces of a workpiece. Proposition 3: Assume that k convex surfaces of a workpiece are given. The k surfaces are eligible if and only if the convex hull of the corresponding vertex contact wrenches bij strictly contains the origin of R6. Proof: The existence of form-closure fixturing points on the surfaces implies the existence of strictly positive α i such that
104104Y.-H. Liu, D. Ding, and M.-L. Lam k
¦α w i
i
=0
i =1
(2.31)
k
¦α i = 1
∀α i > 0
i =1
Substituting eq. (2.28) into eq. (2.31) results in k
ti
i =1
j =1
¦αi ¦ cijbij = 0
(2.32)
Let β ij = α i cij , thus k
ti
¦¦ β b
ij ij
=0
(2.33)
i =1 j =1
Note that k
ti
¦¦ β
ij
= 1,
β ij > 0
(2.34)
i =1 j =1
This means the convex hull of the points bij contains strictly the origin of R6. From this proposition, by testing whether the convex hull of points in B contains the origin, we can readily check the eligibility of a set of surfaces. For convenience, in the following part of this section we use bi with a single subscript i, instead of bij, to denote the i-th column vector of the wrench matrix B. To find a set of eligible surfaces, we cam apply the grasp synthesis algorithm presented in the previous section. The problem here is similar to the problem of finding a frictional form-closure grasp. Denote the set of convex surfaces of the object by Ω. First, we randomly select 7 surfaces and check whether the convex hull of their vertex wrenches contains the origin of the wrench space. If the convex hull contains strictly the origin, a set of eligible surfaces has been found; otherwise, apply the local search procedure by reducing the distance between the convex hull and the origin until finding a set of eligible surfaces or encountering a local minimum. In case a local minimum is encountered, divide by the separating hyperplane the surfaces in the set Ω into subsets and represent the sub-problems in the subsets as child nodes of the root of the search tree. Note that the vertex wrenches of a surface may spread on both sides of the separating hyperplane, so it is necessary to divide the surface set Ω into three subsets like the case of frictional grasp. The same procedure is applied to the child nodes recursively until a set of eligible surface is detected or all the nodes of the search tree have been processed. 5.2 Optimal Fixturing Points
The next step is to find fixturing points on the set of eligible surface. Usually the automated fixturing point determination needs to perform an exhaustive search to
105 105
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
reach an optimal design solution. We here present a method to calculate the optimal fixturing points under a performance index. A given system of wrenches achieves form-closure when the following equations can be satisfied: 7
ti
7
¦ w = ¦α ¦ c b i
i
i =1
i =1
ij ij
(2.35)
=0
j =1
using the following variables:
β l = α i cij
i = 1,2,...,7
j = 1,2,..., ti
l = j + (i − 1) ∗ ti
(2.36)
The form-closure constraints can be formulated as: N
¦β b
l l
(2.37)
=0
l =1 N
¦β
=1
l
l =1
β ∈ RN > 0 7
It should be noted that all the constraints are linear and involve N =
¦t
vari-
i
i =1
ables. Besides the stability constraint in determining fixturing points, another important factor to be considered is the positioning accuracy of the workpiece provided by fixturing points [30]. The positioning errors of fixturing points result in the localization error of the workpiece. To ensure a minimum influence of the positioning errors of the fixturing points on the positioning accuracy of the workpiece, the fixturing points should be located as far away as possible from the center of mass of the workpiece. So we consider to maximize the Euclidean distance from each fixturing point to the center of mass of the workpiece, i.e., 7
¦α
Maximize :
(2.38)
2 T i ri ri .
i =1
The above equation can be rewritten as: 7
B
D
7
B
D
i =1
A
C
i =1
A
C
¦αi ¦¦ c j ck vTj vk = ¦¦¦ β j β k vTj vk
(2.39)
where
A : j = 1+
i −1
¦t , z
z =1
i
B:
¦t , z
z =1
C : k = 1+
i −1
¦t , z
z =1
i
D ::
¦t
z
z =1
Thus we obtain a Quadratic Programming (QP) problem subject to the constraints eq. (2.37). Maximizing the objective function has the effect of maximizing ri and forcing α i to tend to be equal for all i. The former means minimization
106106Y.-H. Liu, D. Ding, and M.-L. Lam
of workpiece locating errors and the latter implies even distribution of the fixturing forces. Finally, the variables of interest, multiplier cij, can be obtained as follows:
cij =
βl
(2.40)
D
¦β
k
C
It should be noted that the objective function employed here sometimes may produce fixtures with bad sensitivities for certain workpiece, but it works in most of the cases to produce good fixtures that reduce the localization errors of workpieces. Fig. 15 shows two views of the fixturing points, found by this algorithm, on a polyhedral part. The part has eight surfaces and two of them are concave. After decomposing the two concave surfaces, there are fourteen convex surfaces for the part. The vertices and normal vector of each surface are the input parameters. Note that those surfaces of other purposes, e.g., machining features, supporting, should be eliminated from the input set. In this case, the large top face shown in Fig. 15(a) is assumed to be the surface to be machined, so we remove it from the input surface set. The algorithm first randomly picks up three surfaces as the initial set, and then finds the final eligible surface set including seven surfaces is obtained in 0.33 seconds of the CPU time of a SPARC Station 4 using Matlab. Next, the quadratic programming is employed to calculate proper positions for fixturing points on the final surface set selected above. It takes 0.44 seconds of the CPU time to calculate the positions for seven fixturing points. This example ascertained the effectiveness and efficiency of this algorithm.
2 1.5 1 4
0.5
2
4
2
0
1.5
3 −0.5
3
1
−1
7
0.5
1 −1.5
6
0
−2 3
3
−0.5 2 2 1
−1
5
5
−1 1
−2 −3
(a)
0
1 0
−1.5
0
−1
−2 −1
−2 0
−1
1
−3
(b)
Fig. 15 Fixture locations found on a polyhedral part
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
107 107
6 Conclusion In this chapter, we presented a ray-shooting based formulation for stability analysis and synthesis of 3-D form-closure grasps. First, it has been shown that the qualitative test of a 3-D form-closure grasp can be transformed to a ray-shooting problem. Then, based on this transformation we proposed an effective heuristics for guiding the local search for form-closure grasps. A complete and efficient algorithm has been developed for 3-D form-closure grasp synthesis in discrete domain by combining the local search with a recursive problem decomposition strategy. Furthermore, we extended the ray-shooting based formulation to automatic fixture layout design on polyhedral workpieces. Examples have been given to demonstrate the effectiveness and efficiency of the algorithms developed on the basis of this new formulation. It is important to note that this formulation provides a good example of linking techniques in Computational Geometry with mechanics problems. The work presented is based on point-point contacts among the object and the robot fingers, so the results may not apply to soft fingers or other kind of finger models. One of important future works is the generalization of the methods to various finger models. Other future works include optimization of form-closure grasps and planning for dexterous manipulation tasks.
References [1] A. Bicchi, “On the closure properties of robotic grasping,” International Journal of Robotics Research, vol. 14, no. 4, pp. 319–334, 1995. [2] M. Buss, H. Hashimoto, and J. Moore, “Dexterous hand grasping force optimization,” IEEE Transactions on Robotics and Automation, vol. 12, no. 3, pp. 406–418, 1996. [3] M. R. Cutkosky, "On grasp choice, grasp model, and the design of hands for manufacturing task," IEEE Trans. on Robotics and Automation, vol. 5, no. 3, pp. 269-279, 1989. [4] D. Ding, Y. H. Liu, Y. T. Shen, and G. L. Xiang, “An efficient algorithm for computing a 3D form-closure grasp,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robotics and Systems, 2000, vol. 2, pp. 1223–1228. [5] C. Ferrari and J. F. Canny, "Planning optimal grasps," in Proc. IEEE Int. Conf. on Robotics and Automation, pp. 2290-2295, 1992. [6] J. R. Kerr and B. Roth "Analysis of multi-fingered hands," Journal of Robotics Research, vol. 4, no. 4, 1986. [7] L. Han, J. C. Trinkle, and Z. Li, “Grasp analysis as linear matrix inequality problems,” IEEE Transactions on Robotics and Automation, vol. 16, pp. 663– 674, Dec. 2000. [8] M. L. Lam, D. Ding, and Y. H. Liu, "Grasp planning with kinematic constraints", in Proc. IEEE/RSJ Int. Conf. On Intelligent Robots and Systems, 2001, vol. 2, pp. 943-948.
108108Y.-H. Liu, D. Ding, and M.-L. Lam
[9] Z. Li, P. Hsu and S. Sastry, "Grasping and coordinated manipulation by a mutlifingered robot hand," Int. Journal of Robotics Research, pp. 33-50, 1989. [10] Y. H. Liu, “Qualitative test and force optimization of 3-D frictional formclosure grasps using linear programming,” IEEE Transactions on Robotics and Automation, vol. 15, pp. 163–173, Feb. 1999. [11] Y. H. Liu, “Computing n-finger form-closure grasps on polygonal objects,” International Journal of Robotics Research, vol. 18, no. 2, pp. 149–158, 2000. [12] H. Maekawa, K. Komoriya, and K. Tanie, "Manipulation of an unknown object by multi-fingered hands with rolling contact using tactile feedback," in Proc. IEEE Int. Conf. on Intelligent Robots and Systems, pp. 1877-1882, 1992. [13] R. Mattikalli, D. Baraff and P. Khosla, Finding all stable orientation of assemblies with friction, IEEE Trans. On Robotics and Automation, vol. 12, no. 2, pp. 290-301, 1996. [14] X. Markenscoff, L. Ni, and C. H. Papadimitriou, “The geometry of grasping,” International Journal of Robotics Research, vol. 9, no. 1, pp. 61–74, 1990. [15] B. Mishra, J. T. Schwartz, and M. Sharir, “On the existence and synthesis of multifinger positive grips,” Algorithmica, Special Issue: Robotics, vol. 2, no. 4, pp. 541–558, 1987. [16] D. J. Montana, “The condition for contact grasp stability,” in Proc. IEEE Int. Conf. on Robotics and Automation, 1991, vol. 1, pp. 412–417. [17] K. Mulmuley, Computational Geometry: an Introduction through Randomized Algorithms. Englewood Cliffs, NJ: Prentice Hall, 1994. [18] R. M. Murray, Z. Li, and S. S. Sastry, A Mathematical Introduction to Robotic Manipulation. Orlando, FL: CRC, 1994. [19] V.-D. Nguyen, “The synthesis of stable force-closure grasps”, A. I. Laboratory, MIT, Cambridge, MA, Tech. Rep. No. 905, 1986. [20] V.-D. Nguyen, “Constructing force-closure grasps,” International Journal of Robotics Research, vol. 7, no. 3, pp. 3–16, 1988. [21] J. Ponce and B. Faverjon, “On computing three-finger force-closure grasps of polygonal objects,” IEEE Transactions on Robotics and Automation, vol. 11, pp. 868–881, Dec. 1995. [22] J. Ponce, S. Sullivan, A. Sudsang, J. -D. Boissonnat and J. -P. Merlet, “On computing four-finger equilibrium and force-closure grasps of polyhedral objects,” International Journal of Robotics Research, vol. 16, no. 1, pp. 11–35, 1997. [23] F. Reulaux, The Kinematics of Machinery. New York: Macmillan, 1976. [24] R.C. Brost and K. Y. Goldberg, A complete algorithm for designing planar fixtures using modular components, IEEE Trans. on Robotics and Automation, vol. 12, no. 1, pp. 31-46, 1996. [25] J. K. Salisbury and B. Roth, “Kinematic and force analysis of articulated hands,” ASME J. Mechanism, Transmissions, and Automation in Design, vol. 105, pp. 33–41, 1982.
3-D Grasp Analysis and Synthesis Using the Ray-Shooting Technique
109 109
[26] J. C. Trinkle, “On the stability and instantaneous velocity of grasped frictionless objects,” IEEE Transactions on Robotics and Automation, vol. 8, pp. 560–572, Oct. 1992. [27] M. Umetsu and K. Oniki, "Compliant motion control of arm-hand system," in Proc. JSME Int. Conf. on Advanced Mechanics, pp. 429-432, 1993. [28] A. S. Wallack and J. F. Canny, Modular fixture design for generalized polyhedra, in Proc. IEEE Int. Conf. on Robotics and Automation, pp. 830-837, 1996. [29] A. S. Wallack and J. F. Canny, Planning for modular hybrid fixtures, in Proc. IEEE Int. Conf. on Robotics and Automation, pp. 520-527, 1994. [30] M. Y. Wang, “An optimum design for 3-D fixtures synthesis in a point set domain,” IEEE Transactions on Robotics and Automation, vol. 16, pp. 839– 846, Dec., 2000. [31] J. D Wolter and J. C. Trinkle, Automatic selection of fixture points for frictionless assemblies, in Proc. IEEE Int. Conf. on Robotics and Automation, pp. 528-534, May, 1994. [32] T. Yoshikawa and K. Nagai, Evaluation and determination of grasping forces for multifingered hands, in Proc. IEEE Int. Conf. on Robotics and Automation, pp. 245-251, 1987. [33] H. Zhang, K. Tanie, and H. Maekawa, Dextrous manipulation planning by graph transformation, in Proc. IEEE Int. Conf. on Robotics and Automation, pp. 3055-3060, 1996.
110
Laser Visual Sensing and Process Control in Robotic Arc Welding of Titanium Alloys H. Luo, and X.Q. Chen Singapore Institute of Manufacturing Technology 71 Nanyang Drive, Singapore 638075 [email protected]
Abstract. Sensor errors arise from detecting the weld seam profile of titanium alloys with laser-based vision sensors because the surface of such a material is highly reflective to a laser light. In this study, the correct weld seam profile is established from the distorted raw data by signal processing. Furthermore, a fully automatic seam tracking and parameter adjusting system is developed. This system features training and saving of the template, auto-finding of the starting welding point, auto-calibration, detection of seam position and groove dimensions, tracking control and on-line welding parameter adjustment. The system is capable of tracking V-groove, fillet, lap and butt joints of titanium alloys with high accuracy of less than 0.4 mm.
1 Introduction Titanium, the fourth most abundant metal in the earth’s crust, features low density, high strength and excellent resistance to corrosion. Titanium alloys have a fascinating range of applications and are becoming increasingly important in a growing spectrum of industrial sectors. These applications play a key part in modern life and there is huge potential for expanded use of them in fields such as aerospace, automotive, construction and medicine. Obviously, high quality welding of titanium alloys is one of the critical processing technologies that determine their applicability. In order to achieve good welding quality, seam tracking is needed to align the torch along the welding seam in despite of distortion or poor jig fixing. Various approaches for seam tracking during arc welding processes such as through-the-arc sensing [5-7,10], ultrasonic [1] and vision-based sensing [2-4,8,9,11] have been extensively explored. Welding parameters also need to be controlled and adjusted according to weld groove profiles. Among various seam-tracking methods, laser-based vision sensor is the most promising and most widely used technology due to its high speed, high accuracy and high robustness to welding environment. However, laser-based visual sensors are very sensitive to material surface and titanium alloys are very high reflective T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 110−122, 2004. Springer-Verlag Berlin Heidelberg 2004
Laser Visual Sensing and Process Control in Robotic Arc Welding
111 111
metals. Problems arise in seam tracking and parameter control during arc welding of titanium alloys. In this study, two laser-based sensors are investigated to obtain weld seam point and weld groove profile. A titanium alloy welding specification database is set up. Weld seam point is used to move the robot to position the torch along the seam, while weld groove profile is used to extract welding parameters from the database to ensure welding quality.
2 System Design and Experimental Set-Up The experimental set-up consisted of a 5-axis AC servo driven robot, a welding torch, a pulsed power source (Fronius TT2000) and a laser-based vision sensor- a Meta 3D camera or a Keyence position-sensitive-detector (PSD) displacement sensor. The PC interface inside the power source enabled the PC to control and adjust the welding parameters. The sensor head was mounted in front of the welding torch, separated by a look-ahead-distance (LAD) between the sensor and the torch. The computer contained a frame grabber board or a data acquisition board, an 8-axis MEI motion controller card. The host computer served as an image or data acquisition and robotic control system. Figure 1 is a block diagram showing the various components of the system and their relationships. Figure 2 shows the set-up of the 5-axis AC servo driven welding robot. It consists of three Cartesian axes (X, Y, Z), and two rotary axes to control the torch orientation. Power supply
Welding process
Torch
Laser sensor/camera Interface
Robot Groove dimensions
Process parameters
Seam point position
PC Database
Seam tracking
Fig. 1. Block diagram of the robotic welding system
Motion control card
112112H. Luo and X.Q. Chen
Fig. 2. Experimental Set-up
The CPU sends signals to the MEI card, instructing it to move the welding torch and the camera/sensor. The camera/sensor attached to the welding torch captures images/profiles of the weld groove, and sends them to the frame grabber/data acquisition card. Subsequently, the images/profiles are processed to detect the seam point and the width and length of the weld groove. The seam point is passed to the seam-tracking controller to align the torch in relation to the weld seam, while the weld groove dimensions are passed to the welding specification database to extract the welding parameters.
3 Sensing Algorithms for Seam Point and Weld Groove Dimensions Since the surface of the titanium alloys is very reflective to laser light, sensing errors will arise when a laser-based vision system is used. 3.1 Laser Scanning Sensing There are two PSD detectors (Y-direction and Z-direction) inside the Keyence PSD sensor. As the red light laser scans across the work-piece by 30~40 mm in 20 ms, the coordinate values (Y in points, Z in voltage) of 1000 points across the work-piece can be acquired. Then the laser returns to its starting point in 10 ms, which ends a measurement cycle. For the Keyence PSD laser displacement sensor, precise weld groove profile was achieved when the titanium plates were sandblasted. The V-groove profile is shown in Figure 3.
Laser Visual Sensing and Process Control in Robotic Arc Welding
113 113
Fig. 3. V-groove profile of sandblasted titanium plates
However, when this sensor was used for the very shining surface of titanium plates, distorted signals were captured in the profile data. Figure 4 and Figure 5 demonstrate the V-groove profile without a gap between the titanium plates and that with a gap respectively.
Fig. 4. V-groove profile without a gap
It was found that even though the sensed profile was distorted by the reflective surface, points A and B are correct. The distance between point A and C or that between point B and D is exactly the height of the weld groove. The position of point E is the center of the groove. Therefore, the weld groove profile can be constructed from the distorted data. Figure 6 showed the results. Seam point and groove dimensions were available form the profile. 3.2 Structured Laser Vision System The Meta 3D camera was used for this development work. It consists of two laser diodes and a CCD camera. The sensor operates on the principle of active camera at a known angle. The image resulting form the intersection of this light sheet with the
114114H. Luo and X.Q. Chen
work-piece comprises two curved or straight-line stripe segments corresponding to the two pieces of metal to be joined. The camera is located immediately in front of the welding torch. Each stripe produces the weld profile image.
A
C
B
E
D
Fig. 5. V-groove profile with a gap
Fig. 6. Weld profile constructed form corrupted data
For the Meta sensor, since the surface of titanium alloys is very reflective to the laser light, two reflection lines were seen in the image shown in Figure 7.
Reflective lines
Fig. 7. 768 by 572 pixel weld groove image with origin (0, 0) at top-left corner
Laser Visual Sensing and Process Control in Robotic Arc Welding
115 115
It was seen that there existed a -type shape inside the image. Because of its difference from the other part of the image, it was trained and saved as a template. Then, a feature-finding tool using a normalized correlation engine was used to locate the given feature within a new image. Once executed, it indicates whether it successfully finds its model by setting the “found” property to true, and the X and Y coordinates of the identical feature can be read in pixel. It should be noted that the X coordinate represents the Z-axis (vertical axis) of the real world. Furthermore, four lines in the image can be extracted through pixel intensity finding and curve fitting techniques. Their intersection points are found by solving the four equations representing the four lines. Therefore, the weld groove dimensions can be obtained. !
4 Seam Tracking and Control through Structured-Laser Vision The implementation of seam tracking and welding parameter control consists of the following procedures: • Training and saving of the feature template; • Auto-finding of the starting welding point; • Auto-calibrations; • Detection of seam position and groove dimensions; • Tracking control and parameter adjusting. The detailed flow chart is shown in Figure 8. Some key steps in seam tracking and parameter control process are explained in detail. 4.1 Region of Interest (ROI) Auto Placement After loading the trained feature, the feature-finding tool uses it to locate a feature within an image. If the feature was found, a result appeared as a point indicating where the feature was located in the inspection image. Consequently, a training and an inspection region of interest (ROI) are placed with their center at the point. After placing the ROIs, the feature-finding tool re-trains the feature specified by the training ROI as the new feature for the following processes such as calibration process and seam tracking process. In these processes, only the inspection ROI will be searched for the feature, hence significantly reducing image-processing time. 4.2 Auto Calibrations and Start Point Finding In the feature-finding algorithm, the software will return the result in pixel coordinates of the trained feature in the acquired image. Hence, it is necessary to find out the actual size with respect to the real world. The purpose of calibration is to synchronize the movement of the 3-axis robot to the work piece, and it is also used to correlate the pixel size of the image to the real world units in millimeters. In
116116H. Luo and X.Q. Chen Load job inform ation and feature
S tart
• M ove the robot along X direction • G rab an im age No Feature Found Y es • • • • Increase m issing point counter & set m issing flag
C am era at the start of the w eld (X ) Auto R O I p lacem ent, re-train the featu re Axis Y & Z calibration S tart process tim er w ith interval of 0.5s G rab an im age No Feature F ound Y es
Y es
C om pute interm ediate m issing point using line fit m ethod
M issing flag set No • R eset m issing point counter & m issing flag • C om pute the deviation betw een the previous seam point and the current one • C om pute the w eld groo ve dim ensions and extract param eters fro m database No
D eviation is w ithin threshold Y es
• Add the point (Y ,Z ) into the list box • Increase seam point counter No
S eam point counter>= Look A head Fram es Y es S tart w elding & m ake an y correction to the seam path & set w elding param eter Y es
N o of m issing point > Look A head Fram es No
S top the w elding process
R obot idle?
No
Y es E nd of w eld reached
• S ave the seam point into the file • M ove the robot to the start position
E nd
Fig. 8. Flow chart of seam tracking and process control
Laser Visual Sensing and Process Control in Robotic Arc Welding
117 117
order to achieve high reliability, it is always necessary to calibrate every job to obtain the calibration factor. The calibration factor (CF) is the numbers of pixels in one-millimeter drift (pixels per mm). The calibration factor of Y-axis and Z-axis are obtained by finding the pixel coordinates of the trained feature in the acquired image as the reference point (ref_pixel). The torch will then be shifted to the left and right by 1 mm from its current position to capture the respective pixel coordinates of the acquired image (left_pixel and right_pixel). The calibration factor for Y-axis is calculated by the following formulae: CF for Y-axis = abs [(left_pixel-ref_pixel) + (ref_Pixel-right_pixel)]/2 The Z-axis calibration is similar except that shifting the torch up and down by 1mm to capture the pixels coordinates (up_pixel and down_pixel). The calibration factor for Z-axis is calculated by the following formulae: CF for Z-axis = abs [(up_pixel-ref_pixel) + (ref_Pixel-down_pixel)]/2 As the torch is moving along X-axis, the first feature-finding point is where the sensor is at the starting point of the weld seam ( -direction), while the torch is still Look Ahead Distance (LAD) behind. If the sensor is at the center of the weld seam, the Y-coordinate returned by the feature find tool should be 384, which is half of the image length in pixels. If the sensor is not at the center of the weld seam, it is able to move the robot Y-axis until the Y-coordinate reaches that value. Since the welding torch is aligned with the sensor, after moving the -axis by LAD, the torch should be in the center of the weld groove. For Z-axis, the robot can be moved up or down until it reached the desired Z-position where the X-coordinate should return the value, which equals to [desired arc length * CF (Z) in pixels]. After these automatic adjustments, this found point should be the start of the weld seam (X-direction), and the center of the weld groove (Y-direction) and high above the work piece of the desired arc length. This point should be the very start point of welding process. During the welding process, the robot will move in the X-direction at the welding speed, arc length will be kept constant, and the torch will be kept at the center of the groove despite any work piece distortion. !
!
4.3 Missing Point Algorithm Using feature-finding tool, there exist undetected points - missing points. In this study, a two-point linear interpolation method was first used to find the undetected seam point between the previous and the following detected seam point. This algorithm is able to take care of any straight and titled path, and also able to compute the coordinates of several consecutive missing points. Figure 9 shows three consecutive missing points between two detected points. The respective missing point’s Y-coordinate can be computed by the following equation irrespective of whether the seam path has a positive or negative gradient. Mn = P1+ n*(P2-P1) / (No. Of missing point+1)] However, if P1 and P2 are in a curvature path, as shown is Figure 10, the computed missing point is M1* based on the linear interpolation while the actual
118118H. Luo and X.Q. Chen
seam point is M1. This tracking error is due to the fact that only two points are used in the linear interpolation algorithm.
P2 M3
M2 M1 P1
D is t
D is t
D is t
D is t
Fig. 9. Three consecutive missing points
M1
P2
M 1* P1 D ist
D ist
Fig. 10. Wrong computation of seam point (M1*)
The occurrence of this error is very rare, as the interval between two points is very small. For instance, given the image acquisition and processing period is 0.5 ms and the welding speed is 4 mm/s, the interval between two points is 2 mm. In such a short interval, two-point linear interpolation works very well. In a high speed welding application, more points are needed for the interpolation, and curve-fitting method can be adopted to reduce this computation error. 4.4 Seam Path Conversion Algorithm One of the essential portions of this study is to acquire the seam path and convert it into the actual welding path. Once the welding process starts, the X-axis will begin to move from the starting point to the ending position as specified by the welding length. There is always a LAD between the laser beam and the welding torch, which causes difficulty in controlling the correction of the Y and Z-axis movement. Hence, a seam welding path conversion algorithm is used to convert the seam path to the actual welding path, where we need to take into account the correct start
Laser Visual Sensing and Process Control in Robotic Arc Welding
119 119
welding position and to synchronize the movement of the welding torch in the Y and Z-axis with the LAD. The main problem in this algorithm is associated with the LAD. As the tip of the torch lags behind the laser beam by the LAD, the immediate seam points captured by the sensor head cannot immediately be used to move the welding torch. Hence, a delay time is required before moving the welding torch to the desired position so as to compute by the immediate seam point. The delay time is known as look-ahead time, which can be calculated by the following equation. Look-ahead time (sec) =Look-ahead-distance (mm)/welding speed (mm/sec) From the look-ahead time, we can determine the number of look-ahead frames required to execute the control. It is much easier to use the look-ahead frame because all the computation is calculated based on the frame rate (number of frames captured per second). Hence, the number of look-ahead frames is computed as follows: Look-ahead frames (frames) =look-ahead time (sec) * frame rate (frames/sec) The system tracks the number of frames captured upon commencing the welding process. Once the number of frames captured is equal to the look-ahead frames, then the welding machine will be switched on. Next, we need to compute the deviation in Y and Z-axis for the seam point found at the instant the image is acquired. Figure 11 shows the diagram of calculating the actual correction distance.
Torch position
sensor position
1
2
Y
3
4
5
6
!
!
Y+
2 !
!
3 Y+2
!
4
Sensor path !
!
4 X Y
Y+3
!
!
Y+4 !
Y+4 !
Actual seam path
Fig. 11. Calculation diagram for actual correcting distance
Assume the number of look-ahead frames is 4. When the torch is at position 1, the sensor is looking at position 5. At this position, we know that the system already
120120H. Luo and X.Q. Chen
found the seam point at position 5, which is shifted 4 to the left of the current torch position. Before the sensor reaches the position 5, it has already captured the seam point at location 2, 3 and 4. Hence, the system will shift the torch to the left by when it is moving forward to position 2. At the same time, the sensor at position 6 is also shifted by to the left. Therefore, the sensor is only able to detect a 4 shift of the seam path and the system will misinterpret that there will not be any deviation at position 6 as the detected seam point is the same as the previous one. However, the torch should be shifted by when moving form position 5 to position 6. This is because when the torch is at location2, the sensor had already shifted by . In order to eliminate this problem, any shift of the torch away from the reference position (horizon line) in Y-axis will be accumulated. This accumulation will be added to the deviation of the seam point found. The actual deviation at the particular seam point can be computed by the following equation: Shift_dev [k] (pixels) = Result returned by feature find [k] (pixels) + Accumulative Error [k-1] (pixels) – Shift_dev [k-1] (pixels) The accumulative error can be computed by the next equation: Accumulative Error [k] (pixels) = Accumulative Error [k-1] (pixels) + Shift_dev [k] (pixels) !
!
!
!
!
!
As the result is in pixels, it is necessary to convert it into millimetre by dividing the calibration factor. The speed of the Y-axis must also be computed before the move command is executed. The speed of the Y-axis has to be calculated using the following equation: Speed of Y (mm/sec) = [distance in X direction (mm) / Distance in Y direction (mm)]* Speed of X (mm/sec) The same method applied to the tracking in Z-axis to maintain the correct arc length.
5 Results Extensive welding trials were carried out to study the overall performance of the seam tracking and parameter controller. The seam-tracking controller produced satisfactory results with good tracking accuracy of less than 0.4 mm. Figure 12 shows a zigzag V-groove path. The laser vision sensor tracked the actual seam accurately without prior knowledge of the weld preparation. A weld sample of straight line welding is shown in Figure 13. The laser vision sensor captured the weld seam, and extracted groove features. Subsequently, correct welding parameters were inferred from the process database accordingly. Desired welds can be formed even in the presence of groove variations.
121 121
Laser Visual Sensing and Process Control in Robotic Arc Welding
C
"
!
G
F
C
!
!
E
>
D
:
B
8
"
!
A
>
@
<
? 8
!
!
:
;
:
-
"
!
-
!
!
>
=
<
;
:
9
7
6
6
5
"
!
4
3
!
H
H
I
H
H
JH
JI
HK
KI
LH
LI
#H
#I
H
I
$H
$I
%H
I%
&H
&I
H
I
'
'
H
H
H
I
JH
I
I
(
)
*
+
I
,
.
/ 0
1
H
H
H
H
H
2
Fig. 12. Zigzag V-groove seam profile
Fig. 13. Titanium weld sample
6 Conclusions The following conclusions are drawn from the study: 1. High reflectiveness of the surface of the titanium alloys causes the distortion of the range data acquired by laser-based sensors. Correct weld seam profile can be successfully recovered from the distorted raw sensor data. 2. A laser-based camera system has been successfully deployed in a fully automated welding seam tracking system, with a feature finding technique to track the weld seam reliably. This system features training and saving of the template, auto-calibration, detection of seam position and groove dimensions, tracking control and on-line welding parameter adjusting. It is able to find the starting welding point automatically, and recovers missing seam point through two-point linear interpolation during the welding process.
122122H. Luo and X.Q. Chen
3. The system is capable of tracking in V-groove, fillet, lap and butt joints of titanium alloys with high accuracy of less than 0.4 mm.
References: 1.
Ajay Mahajan, Fernando Figueroa (1997) Intelligent seam tracking using ultrasonic sensors for robotic welding. Robotica, Vol. 15 (3): 275-281 2. Chen X Q, Huang S S (1993) An investigation into vision system for automatic seam tracking of narrow gap welding process. Chinese Journal of Mechanical Engineering, Vol. 29 (3): 8-12 3. Chen X Q, Lucas J (1989) Sensors for arrow gap welding. Proceedings of Welding Production Procedures Being Technology – Arc Welding 1989, DVS Berichte No. 127, pp 101-105 4. Chen X Q, J Lucas (1989) A fast vision system for control of narrow gap TIG welding. International Conference on Advances in Joining and Cutting Processes, Harrogate, UK, pp 8-1 to 8-9 5. Cook G E (1986) Computer-based analysis of arc welding signals for tracking and process control. IEEE Transaction On Industrial Electronics, Vol. 34: 1512-1518 6. Dilthey U, Stein L (1994) Through-the-arc-sensing: A multipurpose low-cost sensor for arc welding automation. Welding in the World, Vol.34: 165-171 7. Dilthey U, Stein L Stein, M Oster (1996) Through-the-arc-sensing—an universal and multipurpose sensor for arc welding automation. International Journal for the Joining of Materials, Vol.8 (1): 6-12 8. Kakizaki, Takao Arakawa, Ken-ichi, et al. (1998) Development of a high-speed visual sensing system for robotic seam tracking in automobile body sealing. NTT R&D, Vol.47 (7): 813-818 9. Karsten Haug, Guenter Pritschow (1998) Robust laser-stripe sensor for automated weld-seam-tracking in the shipbuilding industry. IECON Proceedings of Industrial Electronic Conference, Vol.2, IEEE Comp Soc, Los Alamitos, CA, USA, pp 1236-1241 10. Wang J Y, Mohanamurthy P H, Fong M K, Devanathan R, Chen X Q, Chan S P (2000) Development of a closed-loop through-the-arc sensing controller for seam tracking in gas metal arc welding. Gintic Technical Report AT/00/013/AMP 11. Yu, Je-Yong, Na, Suck-Joo. (1998) Study on vision sensors for seam tracking of height-varying weldment Part 2: Applications. Mechatronics, Vol.8 (1): 21-36
Intelligent Technologies for Robotic Welding S.B. Chen,1 T. Qiu,1 T. Lin,1 L. Wu,2 J.S. Tian,2 W.X. Lv,2 and Y. Zhang3 1. Institute of Welding Engineering, Shanghai Jiao Tong University, Shanghai 200030, PR China. Email: [email protected] 2. Welding department, Harbin Institute of Technology 3. Dept. of Mechanical Engineering, Qinghua University [email protected]
Abstract. This paper discusses intelligent technologies for the robotic welding, which contains computer vision sensing, automatic programming for weld path and technical parameters, guiding and tracking seam, intelligent control of welding pool dynamics and quality. Such an welding robotic systems with some artificial intelligent functions could realize collision-free path planning for complex curve seam, detecting welding surroundings by laser scanning technology, identifying the start of weld seam and guiding weld torch, tracking the seam, complete real-time control of pulsed GTAW pool dynamical process by computer vision sensing and intelligent technology. The above intelligent separated units or subsystems were integrated into an intelligentized welding flexible manufacturing cell (IWFMC). Based on discrete event control theory and Petri net modeling method, the related domination strategies for the systems was designed to supervise the IWFMC.
1 Introduction Although using robotic welding is propitious to increase productivity, improve welder laboring status, ensure stable welding quality and realize welding automation for a small batch of products [34,36,21,9], effective application of welding robots in practical production is still obstructed due to complexity and uncertainty in welding process [1,4,14]. Recently, developing intelligentized technology for welding robots has become an attractive research field and inevitable uptrend[7,6,35] At present, welding robots serving in practical production still are teaching and playback type, and can’t often meet quality and diversification requirements of welding production because this type of robots don’t have adaptive ability to circumstance changes and uncertain disturbances during welding process. In practical production, welding conditions are often changing, such as the errors of pre-machining and fitting work-piece would result in differences of gap size and T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 123−143, 2004. Springer-Verlag Berlin Heidelberg 2004
124124S.B. Chen et al.
position, the change of workpiece heat dispersion during welding process would bring on weld distortion and penetration odds . In order to overcome or restrain various uncertain influences on welding quality, it will be an effective approach to develop and improve intelligent technologies for welding robots, such as vision sensing, automatic programming, guiding and tracking, and real-time intelligent control of welding process [2,10]. To obtain enough precise sensing information from welding process under a mass of disturbances is one of key technologies for intelligent welding robots. During robotic welding, intelligent sensing mainly focuses in tracking seam and monitoring pool feature, and vision sensing technology has been paid a hot attention in recent years. Generally, the vision sensing technology can be divided into active method and passive method in welding according to imaging light source. The active vision sensing method requires laser or other assistant light source to illuminate welding region and to restrain welding arc interference for imaging weld pool. Ref[22,3,23,8]reported some designed laser sensor systems for tracking seam and detecting weld pool. The main problem of active vision sensing system is too expensive to apply it in practical welding process. Without assistant light source, the passive vision sensing method only utilizes self-arc to illuminate welding region. Using a filter to wipe off disturbance, the charge coupled device (CCD) camera captures weld pool images in selective narrow spectrum range for picking-up pool features , so that the passive vision method has been considered as a practical technique for monitoring weld pool process. Ref.[38,15,25] investigated the passive vision sensing systems for tracking seam and monitoring weld pool. As is well known, arc welding process contains many complex factors, such as material metallurgy, heat conduction, physical chemistry reactions, etc., it is a typical multi-variable coupling, nonlinear, time-varying, random and uncertain properties, so that it is very difficult to model and control welding dynamics by classical linear system theory. In recent years, some intelligent methods were introduced to welding process, e.g., Ref.[ 17,28,11 ] investigated fuzzy reasoning application in welding quality control, and Ref.[12,29,40,24,37] showed research results artificial neural net works for modeling and control of welding process. In former researches, although some significant results were obtained in vision sensing, modeling and intelligent control of weld quality and pool dynamics[27,26], these intelligent technologies are still not realized effectively in welding robot systems due to complexity of welding process, real-time and opening limits of robot systems. Therefore, developing intelligentized technology for improving current teaching and playback welding robot is necessary and exigent to satisfy high quality and flexible requirements for welding products and advanced manufacturing uptrend.
Intelligent Technologies for Robotic Welding
125 125
2 Intelligentized Welding Robot Systems The principle scheme of an intelligentized welding robot systems is shown as Fig.1, it contains a 9-freedom welding flexible mechanical platform with 6-freedom manipulator and 3-freedom positioner; an automatic programming and simulation subsystem for welding task; a welding seam guiding and tracking unit; an intelligent real-time control unit with vision sensing, modeling and control of weld pool dynamical features; and a central control unit for harmonizing each separated unit. In Fig.1, Wp is welding power, Sp is signal processing interface for laser scanning, IP is image processing interface for weld pool; WPPC is control computer for welding power, TPPC is control computer for tracking seam process, VPPC is process control computer for vision sensing and intelligent control of weld pool dynamics, MPPC is control computer for collaborating the welding flexible mechanical platform with 6-freedom manipulator and 3-freedom positioner; CCPC is central control computer for whole intelligentized welding robot systems. Based on structure functions of intelligent robot system [ 16,30], systems in Fig.1 could be divided into three functional levels: executive level, collaborative level and intelligent level. The executive level contains 9-freedom flexible platform, sensors, controllers, welding power. The collaborative level includes each functional control computers. The intelligent level includes the index command input of human-computer interface, welding task programming and simulation, it administrates whole systems by means of CCPC. The system in Fig.1 is named as an intelligentized welding flexible manufacturing cell (IWFMC).
Fig. 1 The system scheme of intelligentized welding flexible manufacturing cell (IWFMC)
126126S.B. Chen et al.
The photo of the flexible platform with 6-freedom manipulator and 3-freedom positioner is shown as Fig.2, which was designed by our-self, due to its opening framework and maintainability, the platform become a usable equipment for realizing various intelligent technique experiments. The basic mechanical experiment on the platform validated that the system could realize satisfactory collaborating movement, repeating orientated precision, continuous locus and velocity precision for robotic welding tasks.
Fig. 2 The photo of 9-frredom flexible platform
3 Automatic Programming and Simulation for Robotic Welding Task Off-line programming and simulation is a key technology for improving flexibility of welding robots Ref.[ 13,5,19]. The automatic programming and simulation subsystem in Fig.1 includes CAD techniques for features of welding robot and work-piece, path and parameter planning for robotic welding process. 3.1 Robotic welding off-line programming and simulation system based on dynamic graphics The module framework of designed off-line planning system is shown as Fig.3, beside basic functional modules, some advanced modules, such as work-piece feature CAD modeling, welding parameter and path program, were integrated into the system for improving program efficiency. Moreover, this system has added a programming and simulation of arc welding robot with vision and tracking sensors for robotic movement reach-ability. In order to obtain features of various welding work-pieces, it is necessary to establish a basic modeler for identifying geometry features of the object, such as joint and intersecting between sides and plane. The AutoCAD was used in the system to indicate welding workpiece features.
Intelligent Technologies for Robotic Welding
127 127
Advanced Model character CAD
Parameter planning
path planning
Autocad 2000
Geometry modeling
Motion model
Path model
Teaching model
Off-line programming
Program transition
Collision Calibration checking model
Basic Model
Fig. 3 Framework of off-line programming and simulation
An identification of welding seam features passed through two pipes is shown as Fig.4.
Fig. 4 An identified results of welding seam features passed through two pipes
3.2 Welding parameter planning for the robot systems In order to improve intelligence of the programming system, our studies has combined rule-based reasoning RBR , case-based reasoning CBR and artificial neural networks ANN with welding parameter planning so that off-line planning of robotic welding techniques can meet some special requirements in practical production. The framework of welding parameter programmer for the robot systems is shown as Fig.5. !
!
"
"
!
"
128128S.B. Chen et al.
3.3 Path planning technology for arc welding robot
[34]
Path planning technology for arc welding robot contains following two aspects:
Fig. 5 The framework of welding parameter programmer
collision-free path planning and redundancy robotic path planning. The collision-free path planning aims at weld workpiece and circumstance, searching an optimal welding path in manipulator space for weld quality. Redundancy robotic path planning focuses on self-movement in arc robot, searching an optimal kinematics and welding poses path in joint space.
The collision-free path planning of arc robot 1. The collision-free path planning was expressed as an optimization of welding torch poses. An optimizing model was presented to escape collision path, and checking collision algorithm was developed in the system; 2. The heuristic searching technique was used to resolve collision-free path planning of arc welding robot by the designed producing system and so-called quality-step searching method. !
"
Based on the AutoCAD software, a simulation system for collision-free path planning was developed.
Intelligent Technologies for Robotic Welding
129 129
Redundancy robotic path planning 1. An index function with multi-performance synthesis was presented for redundancy robotic path planning; !
!
Using inheriting arithmetic, A harmonizing control algorithm was developed for the redundancy manipulator system with six-freedom robot and three-freedom positioner;
3. Based on the AutoCAD and 3DS software, a simulation system for redundancy robotic path planning was developed.
4 Laser Scanning Sensing Tracking and Real Time Control of Welding Robot Moving Locus During robot welding process, due to various factor influences on welding conditions, such as the machining and assembling errors, heated and remained stress distortion, it is difficult for the teaching and playback robot to satisfy welding quality requirement under welding condition anomaly changes. The IWFMC can realized real autonomous control and correcting robotic manipulations by the following laser sensor for improving robotic adaptive ability, even if without off-line programming and teaching and playback process. 4.1 The laser scanning sensor for weld seam tracking The main performance parameters of the designed laser scanning sensor for weld seam tracking used in IWFMC follows as: 1. Volume:
145mm×120mm×60mm
2. Weight:
600g
3. Scanning angle scope:
±7.5°
4. Scanning times/second:
11.6
5. Data capture ratio:
4000 point/s
6. Across measuring precision: 7. Portrait measuring precision
"
"
"
0.15mm 0.1mm #
8. Suitable to various work-piece surfaces
"
"
"
"
"
9. Strong resisting interference ability: no influence aparting from laser scanning plane 30mm under 200 Amp GTAW arc.
130130S.B. Chen et al.
This sensor can capture the relative position information of the torch to cross section of weld seam in real time, and obtain high precision profile of weld seam cross section. Autonomous control of welding robotic moving locus is one of basic indexes of intelligentized robots. Realizing of welding robot moving locus can relax consistent requirements for work-piece and dependence on welding clamps. In the IWFMC, the moving locus of welding robot can be regulated in real time by laser scanning sensing information and independent on off-line path programming or teaching and playback. 4.2 Laser scanning-based identification of some typical joints Using the above three-dimension laser scanning system, the detected results for several typical seams, such as V-type groove, lap joint, butt seam, are shown as Fig.6. The reticles in Fig.6 indicate welding position determined by measured information.
(
!
"
'
(a) V-type groove joint with little gap
+
)
*
*
'
(b) lap joint
,
-
.
#
#
$
(c) butt seam in 1.5mm gap Fig. 6
%
&
&
(d) butt seam in 0.15mm gap Detected results typical seams
Since identification of butt seam is affected by different gaps, usually, the characteristic differences between small gap and no-gap butt are not evident, it is very difficult to detect and identify the position of these joints. In general, the butt
Intelligent Technologies for Robotic Welding
131 131
seam is so various that it is easily confused with fillet and corner joint due to their similarity. Therefore, some expert rules was used to determined the seam features.
5 Welding Robot Systems with Vision Sensing and Intelligent Control of Pulsed GTAW Pool Dynamics Up till now, no welding robot has provided with function of real-time sensing and control of weld pool dynamics. Therefore, it is significant to realize an intelligentized robot systems with vision sensing and real-time control of welding pool process..
CCD camera Composed filter system Reflecter
Motor
Travel direction
Torch
Reflecter
Weldingdirection Workpiece
(a). photograph of the visual sensor
(b). Structure diagram of the visual sensor
(c) Installation of visual sensor and tracking sensor Fig. 7 The systems of weld pool visual sensing and laser tracking sensors
132
S.B. Chen et al.
5.1 The vision sensing of the pool dynamical process The photograph and the structure diagram of the visual sensor are shown in Fig.7 (a), (b). The visual sensor at the right and the laser scanning sensor at the left were fixed with the end-effector of welding robot. In order to decrease the visual sensor influences on pool images during movement of the torch, a two-step reflecting light path system was adopted. Through a composed filter system , which consisted of a neutral density filter and a narrow band filter, weld pool images and arc light entered the CCD camera. The vision sensor, image collecting board, the computer for image processing and control of weld penetration and pool size, welding power and related interfaces composes the intelligent control subsystem for weld pool dynamic process. 5.2 Picking up dynamical features of pulsed GTAW pool images Calibration of the image sensing system is omitted here for short.
Weld pool Character Parameters Definition In order to investigate weld penetration and weld quality, two weld pool parameters, topside maximum width Wfmax and topside half length Lfmax were defined as Fig.8. arc center
Wfmax
Lfmax
Fig. 8. Definition of characteristic parameters of weld pool during pulsed GTAW
Image Processing and Picking-up features of the weld pool images Under the conditions in Table 1, welding experiment on the robotic system was conducted and typical images are shown as Fig.9
Using image processing algorithms desigined in this paper, characteristic parameters of weld pools were picked-up. Usually, robotic controller cannot ensure that the angle between the direction of detecting and welding travel always was accordant to the complicated seam. On the other hand, the angle was varing during robot welding. Therefore, the algorithm for weld pool image processing
Intelligent Technologies for Robotic Welding
133 133
must be applicable for changing of observation direction during robotic arc welding. Table 1. The experiment conditions of pulsed GTAW Pulse frequency f, Hz Pulse duty ratio , % Peak current Ip, A Base current Ib, A Traveling speed Vw, mm/s !
1 45
Arc Length l, mm T-pole dia. , mm
3 2.4
130 60 2.0
T-pole angle , ° Flux of argon L, l/min Workpiece size mm×mm×mm
30 8.0 250×50×2
"
#
Fig.9(a) was an original image of weld pool captured at the base current time, Fig.9(b) was an image processed through smoothing and filtering noise. The point C in Fig.9(c) was determined as the arc center, The point T was determined as the trail weld pool. Using edge picking-up algorithms, boundary points were determined as Fig.9(d), Fig.9(e) was a pool boundary approached by the quartic polynomial. The pool character parameters were determined by Fig.9(f). The detail image processing algorithms are omitted here.
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 9 Image processing and feature picking-cp of robotic pulsed GTAW pool
5.3 Controller for weld pool dynamic features during pulsed GTAW Due to the complexity of weld pool dynamics during pulsed GTAW, we designed a neuron self-learning Proportional-Summational-Differential (PSD) controller for pulsed GTAW process, shown as Fig.10. Main merits of PSD controller are its modeling-free of the complicated process and its adaptability. Using measured errors between the actual outputs and the expected outputs, the PSD can realize self-learning control of the process. The controlled variables of the system are maximum width and half-length of the weld pool, the outputs of PSD controller are pulse duty ratio and peak current respectively. A neuron self-learning PSD controller’s output follows as
∆u (t ) = ω 1 (t ) x1 (t ) + ω 2 (t ) x 2 (t ) + ω 3 (t ) x3 (t )
(1)
134134S.B. Chen et al.
Fig. 10 Diagram of a neuron self-learning PSD control system for pulsed GTAW
where u(t) is control input
x1 (t ) = e(t )
!
x i (t ) (i=1,2,3) are inputs of neuron, i.e.,
x 2 ( t ) = ∆e(t ) and x 3 (t ) = ∆ e 2 (t ) . ω i (t ) (i=1,2,3) are weight!
ing coefficients corresponding to x i (t ) (i=1,2,3). The weighting coefficients ω i (t ) (i=1,2,3) were modified during neuron self-learning process, the learning rules follows as:
ω i (t + 1) = (1 − m)ω i (t ) + dri (t ) Here m, d>0 "
d is learning ratio and
(2)
ri (t ) is defined as
ri (t ) = z (t )u (t )[e(t ) + ∆e(t )]
(3)
where z(t) is a teaching signal. To ensure convergence and robustness of the learning algorithm, following learning algorithms are adopted, and they compose controlling rules of single neuron PSD, that is 3
∆ u ( t ) = K # ω ' ( t ) x i (t ) 3
Where
i=0
ω i' (t ) = # ω i (t ) i =1
$ω 1 (t + 1) = ω 1 (t ) + d I z (t )u (t )[e(t ) + ∆e(t )] ' %ω 2 (t + 1) = ω 2 (t ) + d P z (t )u (t )[e(t ) + ∆e(t )] 'ω (t + 1) = ω (t ) + d z (t )u (t )[e(t ) + ∆e(t )] 3 D & 3 The parameters dP, dI and dD are regulated according to the actual system.
(4)
Intelligent Technologies for Robotic Welding
135 135
5.4 Experiment of PSD control of weld pool dynamics during robotic welding process In order to verify effectiveness of PSD controller in robot systems, butt welding in robotic pulsed GTAW process was conducted. The workpiece was dumbbell-shaped mild steel plate with 2mm thickness. The size and the shape of the work-piece are shown in Fig.11. The varying curves of peak current and maximum width of the weld pool are shown as Fig.12. The peak value and duty ratio of the pulse current were regulated by neuron self-learning PSD controller during welding process. The Fig.13 is photos of controlled work-piece.
Fig. 11 Shape and the size of the work-piece
Fig. 12 Curves of neuron self-learning PSD control of bead width during robotic pulse GTAW
(a) Topside
(b) Backside
Fig. 13 Photographs of the PSD control of weld work-piece
136136S.B. Chen et al.
6 Integration of Intelligentized Welding Robot Systems The intelligentized welding robot systems has been integrated for uniformly administering robotic collaborating movement, path and parameter planning and simulation, guiding and tracking seam, visual sensing and intelligent control of weld pool dynamic features. The integrated systems could be called as a welding flexible manufacturing cell with some primary intelligent functions, or primary intelligent welding flexible manufacturing cell (IWFMC). 6.1 The integrated structure designs of the IWFMC Based on task characters of arc welding robot systems, the IWFMC was designed in star-shape net framework, as Fig.14. It includes a robotic movement trajectory control unit(RCU), a weld seam tracking unit(STU), welding pool penetration control unit(PPU) and robotic welding power unit(WPU). The communication and exchanging information among these units can be completed by the center monitoring computer(CMC).
-
-
#
-
# #
$
#
$ $
$
* *
! !
&
& &
&
'
' '
'
(
( (
(
!
" !
"
"
! "
!
*
+ *
+
+
, +
,
, ,
! !
)
) )
)
)
) )
)
(
( (
(
Fig. 14 The net topology structure of IWFMC
The information flow in IWFMC systems can be described as following: 1. Based on net communication techniques, robotic controller can down load and run welding task program in real time from CMC, during welding process, the robotic controller can regulate driving signals of joints by tracking sensing information at same time, the penetration unit will provide welding pool states %
Intelligent Technologies for Robotic Welding
137 137
and welding current and speed information to weld power and robotic controller through CMC. 2. The net system of IWFMC was designed by classified control mode and combined with sensing system for realizing sharing control, fusing control and centralizing management by sensing information flow. The CMC can indicate, supervise and correct welding process, path and parameter planning, multimedia graph simulation, data from weld expert system and so on. 6.2 Petri net model for optimization of material flow and information flow in IWFMC The Petri net method in the discrete event dynamic system DEDS theory has very powerful token function for modeling, analysis, performance evaluation and control of event elements and its logic sequence[18,31,39,20]. It is a significant research direction to use Petri net method for optimizing and planning material flow and information flow in welding flexible manufacturing cell/system[33,32] . The discrete event and Petri net are very suitable to little batch and excessive various welding products on Welding Flexible Manufacturing Cell(WFMC) and can improve efficiency of WFMC and welding quality. In order to assure welding product quality and satisfy intelligent requirements for welding flexible manufacturing, a petri net model combined with sensing and executing information during robotic welding process has been established by the following rules: Rule1: if input buffer is empty, then put workpiece to input buffer; Rule2: if positioner is vacancy, then put workpiece to positioner from input buffer; Rule3: if tracking sensor sends regulating signal, then regulate robot running locus; Rule4: if pool feature is exceptional, then regulate weld current and speed; Rule5: if welding is finished and output buffer is empty, then put workpiece to output buffer; Based on the above rules, an automatic Petri net (APN) model for IWFMC has been established as Fig.15. There are two types of directed arc to be added in common Petri net for APN model: inhibited arc, noting as , and enabling arc, noting as . The APN model is activated by exterior spring events, e.g., input information from sensors and action information from executors. The APN model contains 18 places ps1, ps2, p1, p2, , p16 and 18 transitions t1, t2, , t18 , and its marks are explained as following: ps1/ps2 input/output sink, p1/p2: workpiece moving permit/prohibit (from sink to buffer);p3/p4 positioner idle/busy; p5/p6 weld pool sensors (regulating signal)/( no regulating signal); p7/p8 tracking sensor (regulating signal)/ (no regulating signal) ; p9/p10 workpiece moving permit/prohibit (from buffer to sink); p11/p16 input/output buffer states; p12: workpiece ready p13/p14 welding process start/end; p15 post- processing; w1/w2 weight value of inhibited arc. !
"
!
!
"
"
!
"
#
!
"
#
$
$
$
$
$
$
$
%
$
$
138138S.B. Chen et al.
A running process of the APN system can be described as following: In welding process starting, if input buffer p11 is empty, then put workpiece from the input sink to input buffer; if the positioner is vacancy, then put workpiece to positioner from input buffer p12 if workpiece is ready then start to weld p13 during welding process, if weld pool sensor outputs regulating signals, then to regulate welding parameters for control of weld penetration t14; if seam tracking sensor outputs regulating signals, then to regulate manipulator to correct moving locus t15 necessary processing after welding end p15 at the same time, if output buffer is empty p16, ; then down-load the welded workpiece to the output buffer for delivering output sink. Thus whole welding process is completed. !
!
!
!
Fig. 15 Automatic Petri net model of IWFMC
6.3 Functional Realization of composite IWFMC system Based on IWFMC systems with exterior sensors, a running flow chart of the IWFMC with model of welding automation Petri Net – WAPN is shown as Fig.16.
Intelligent Technologies for Robotic Welding
139 139
The work-piece photos of visual tracking and coordinating control experiments on IWFMC are shown as Fig.17. System startup Set-up welding state parameters Download off-line program TPPC
Guide to seam start CCPC monitor system with WAPN
Get to the start?
MPPC
N
Y
Run tracking system WPPC
Run process control
Get to the terminal?
N
Y
Turnoff systems Fig. 16 Running flow chart of WFMC systems with WAPN
Fig. 17 (a) Tracking experiment
VPPC
140140S.B. Chen et al.
Fig. 17 (b) Coordinating experiment Fig. 17 Work-piece photos of the experiments on IWFMC
7 Conclusions The current teaching and playback welding robots need to be improved by intelligentized welding techniques for its effective application in practical production. The intelligentized welding techniques for robot systems conclude the vision sensing, guiding and tracking weld seam, programming path, pose and welding parameters, knowledge modeling and intelligent real-time control of welding dynamics, optimizing supervision of welding flexible manufacturing cell/system, and so on. This paper has shown a primary intelligent welding robot systems with separated sensing and control subsystems, so-called intelligentized welding flexible manufacturing cell—IWFMC. The intelligent technologies contained in this systems will be a useful foundation for us to develop new-type intelligent welding robot in the future.
Intelligent Technologies for Robotic Welding
141 141
Acknowledgments This work was supported by National Natural Science foundation of China, No.59635160 and Science and Technology Committee of Shanghai, China, No.021111116.
References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.
B.F. Kuvin. Welding for Space. Welding Journal. 1992, 71(3): 41-44 Bob Irving. Sensor and Control Continue to Close the Loop in Arc Welding. Welding Journal. 1999, 78(5): 31-36 C.G. Morgan, J.S. Bromley and P.G. Davey. Visual Guidance Techniques for Robot Arc Welding. Proceedings of 3rd Conferences of Robotic Vision Sense Control, 1983: pp615-619 E.J. Morgan-Warren. Remote Repair Welding in Nuclear Reactors. Welding & Mental Fabrication. 1989, (8): 111-116 G. Grunwald, Modeling and Planning for Sensor Based Intelligent Robot Systems (Task-Directed Planning of Cooperating Sensors: A Framework). Machine Perception and Artificial Intelligence. 1995, 21: 143-159 Gerhard Teubel. Experience + Application Up-Date: Automation of A.W.-Operations Using Robot-Technology. Welding in the World. 1994, 34: 75-84 J.D. Lane. Robotic Welding State of the Art. Robotic Welding—International Trends in Manufacturing Technology. IFS (Publications) Ltd.UK. 1987: 1-10 J.E. Agapakis and J. Bolstad. Vision Sensing and Processing System for Monitoring and Control of Welding and Other High Luminosity Processes. International Robots & Vision Automation Conference, 1991: pp23-29 J. L. Pan, ” A survey of welding sciences in 21th century.” Proceeding of 9th Chinese Welding Conference, Tianjun, China, Oct. 1999, (1): D-001-D-017 J.W. Barratt W.F. Clocksin P.G. Davey and A.R. Vialer. Visually Guided Robot Arc-Welding of Thin Sheet Pressings. Proc. of 12th Int. Symp. on Industrial Robot.IFS(Publications) Ltd.UK.1987: 225-230 J.W. Kim and S.J. Na. A Self-Organizing Fuzzy Control Approach to Arc Sensor for Weld Joint Tracking in Gas Metal Arc Welding of Butt Joints. Welding Journal. 1993, Vol. 72(1): pp60s-66s K. Andersen and G.E. Cook. Gas Tungsten Arc Welding Process Control Using Artificial Neural Networks. Proceedings of the 3rd International Conference on Trends in Welding Research, Gatlinburg, Tennessee, USA, 1-5, June, 1992: pp135-142 Krister Brink, etc. Increased Autonomy in Industrial Robotic Systems: A Framework. Journal of Robotic Systems. 1997, 19: 357-373 K. Masubuchi, .A.V. Gaudiano and .I.J. Reynolds Technologies and Practices of Underwater Welding. Proceedings of The International Conference on Underwater Welding. Norway.1983, (6): 49-70 K. Oshima and M. Morita. Sensing and Digital Control of Weld Pool in Pulsed MIG Welding. Transactions of the Japan Welding Society. 1992, Vol. 23(4): pp36-42
142142S.B. Chen et al.
16. K. P. Valavanis, G. N., Saridis, Intelligent Robotics System: Theory, Design and Applications. Boston, 1992: 12-18 17. L. Dimiter. Adaptive Robot Under Fuzzy Control. Fuzzy Sets and Systems. 1985, Vol. 17(1): pp23-28 18. L.E., Holloway, etc. Survey of Petri Net Methods for Controlled Discrete Event Systems. Discrete Event Dynamic Systems: Theory and Applications. 1997, 7(2): 151-190 19. L. Wu, K. Cui, S.B. Chen, Redundancy Coordination of Multiple Robotic Devices for Welding through Genetic Algorithm, ROBOTICA, 2000,18(6), 669-676. 20. M.C. Zhou, Petri Net Synthesis and Analysis of A Flexible Manufacturing System Cell. IEEE Transactions on Systems, Man and Cybernetics. 23(2): 523-531 21. Paul Drews., Gunther Starke. Welding in The Century of Information Technology. Welding in the World. 1994, 34: 1-20 22. R.J. Beatlie, S.K. Cheng and P.S. Logue. The Use of Vision Sensors in Multipass Welding Applications. Welding Journal. 1988, Vol. 67(11): pp28-33 23. R. Kovacevic, Y.M. Zhang and L.Li. Monitoring of Weld Joint Penetration Based on Weld Pool Geometrical Appearance. Welding Journal. 1996, Vol. 75(10): pp317s-329s 24. R. Kovacevic and Y.M. Zhang. Neuro-fuzzy Model-Based Weld Fusion State Estimation. IEEE Transactions on Control Systems Technology. 1997, Vol. 5(4): pp30-42 25. R.W. Richardson and D.A. Gutow. Coaxial Arc Weld Pool Viewing for Process Monitoring and Control. Welding Journal. 1984. 63(3): 43-50 26. S.B. Chen, D. B. Zhao, L. Wu and Y. J. Lou, Intelligent Methodology for Sensing , Modeling and Control of Pulsed GTAW :PART2—Butt Joint Welding, Welding Journal, 2000, 79(6):164s-174s 21. S. Murakami. Weld-line Tracking Control of Arc Welding Robot Using Fuzzy Logic Controller. Fuzzy Sets and Systems. 1989, Vol. 32(2): pp31-36 27. S. B. Chen, Y. J. Lou, L. Wu and D.B. Zhao , Intelligent Methodology for Sensing , Modeling and Control of Pulsed GTAW :PART1—Band-on-Plate Welding, Welding Journal, 2000, 79(6):151s-163s 28. S. Murakami. Weld-line Tracking Control of Arc Welding Robot Using Fuzzy Logic Controller. Fuzzy Sets and Systems. 1989, Vol. 32(2): pp31-36 29. T.G. Lim and H.S. Cho. Estimation of Weld Pool Sizes in GMA Welding Process Using Neural Networks. Journal of Systems and Control Engineering. 1993, Vol. 207(1): pp15-26 30. Timothy Hebert, etc. A Real-Time, Hierarchical, Sensor-Based Robotic System Architecture. Journal of Intelligent and Robotic Systems. 1998, 21: 1-27 31. T.O., Boucher, etc. Petri Net Control on Automated Manufacturing System Cell. Advanced Manufacturing Engineering. 1990, 2(3): 323-331 32. T. Qiu, S. B. Chen, Y. T. Wang, L. Wu, Information Flow Analysis and Petri Net-based Modeling for Welding Flexible Manufacturing Cell. SPIE Intl. Symposium on Intelligent Systems and Advanced Manufacturing. Vol.4192:449-456, Nov. 6-8, 2000, Boston. 33. T. QIU, S. B. Chen, Y. T. Wang, L. Wu, Modeling and Control of Welding Flexible Manufacturing Cell Using Petri Net, Journal of Materials Science & Technology, 2001, 17(1): 181-182 34. Trailer, Manufacturer Depends on Robotic Welding to Boast Production. Welding Journal. 1995, 74(7): 49-51 35. U. Dilthey, L. Stein. Robot System for Arc Welding—Current Position and Future Trends. Welding & Cutting. 1992, (8): E150-E152
Intelligent Technologies for Robotic Welding
143 143
36. West Carrillton, Robot Assure Quality for Auto Seat Manufacturer. Welding Journal. 1995, 74(8): 57-59 37. Y. Kaneko. T. Iisaka and K. Oshima. Neuro-Fuzzy Control of the Weld Pool in Pulsed MIG Welding. Quarterly Journal of the Japan Welding Society. 1994, Vol. 12(3): pp374-378 38. Y. Li and J.E. Middle. Machine Vision Analysis of Welding Region and Its Application to Seam Tracking in Arc Welding. Journal of Engineering Manufacture. 1993, No. 5: pp275-283 39. Y. Narahari, etc. A Petri Net Approach to the Modeling and Analysis of Flexible Manufacturing System. Annals of Operation Research. 1985, (3): 449-472 40. Y. Suga and M. Naruse. Application of Neural Network to Visual Sensing of Weld Line and Automatic Tracking in Robot Welding. Welding in the World. 1994, 34: pp275-284
144
The Seam Tracking System for Submerged Arc Welding Hua Zhang Xuelei Ding Maohua Chen Benqi Da and Chunhua Zou !
!
!
!
Mechatronic Institute Nanchang University, Nanchang 330029, P.R. China [email protected] "
Abstract. The structure and the principle of seam tracking for submerged arc welding were introduced in this paper. The method of seam recognition with computer vision was mainly researched when the environmental light was changed. Intelligent control system for seam tracking has been realized by using a selfmodulated fuzzy control set. The system was tested on the producing of spiral submerged arc welded pipe for outside-welding and the primary results were obtained.
1 Introduction Since submerged arc welding has the advantages of high efficiency, well seamdeform and without arc light interference, it’s widely applied in petrochemical industry and pressure vessel manufacture. Submerged arc welding is adopted in the key engineering project “gas transportation from west to east” in China. Because tracking accuracy is influenced by a number of factors such as changing of pipe diameter, variation of image, flux leaking and so on, it’s not easy to control these factors. So far an effective approach of seam tracking for outside welding is not proposed yet and it is still in the stage of manual control. In practice, workers can only judge seam position according the fact whether the light is just in the middle position of seam. Seam tracking accuracy is influenced workers’ eye fatigue if they watch the light point for a long time. How to get over above disadvantages, increase tracking accuracy, reduce work intensity, replace manual control with automatic control is the primary problem for researchers to solve.
2 Structure and Principle of Seam Tracking System 2.1 Framework of system The standard parameters of this project are described as: pipe : 1016×14.6, welding speed: 0.9 1.5m/min, welding voltage: 32 welding current: 390 1100A, pipe thickness: 7 10mm. !
"
"
"
38v
"
T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 144−151, 2004. Springer-Verlag Berlin Heidelberg 2004
The Seam Tracking System for Submerged Arc Welding
145 145
The hardware of the seam tracking system consists of cross slide industrial control computer interface circuit image acquisition card(VIDEO-PCI-XR), CCD(Charge Coupled Devices) sensor, AC4064 I/O board and so on. The framework of the system is illustrated in Fig.1. #
#
$
#
Intelligent Control
I/O Interface Circuit
Cross Slide(Torch)
Image Processing
Image Acquisition
Vision Sensor(CCD)
%
Welding pipe
&
Fig. 1. Framework of seam tracking system of submerged arc welding for outside-welding
2.2 Principle of system The CCD sensor transfers seam images into video signals. And image acquisition card transfers video signals into digital signals in the computer. Then all above images are in the process of filtering, binarization, Hough transform according to a software programmed with VC++6.0 seam error is obtained after these processing in the end. Seam error is put into intelligent controller, digital quantity is reached by a certain of algorithm ways. Cross slide is adjusted by the digital quantity which is transmitted to analogue quantity through I/O interface circuit. The purpose of precise tracking and error correction is obtained finally.
3 Method of Seam Recognition 3.1
Seam analyzing
First determine the processing region before image preprocessing. It’s not needed that all the image region should be processed, because it has an impact on the real
146146H. Zhang et al.
time processing speed for occupying much more space of the memory and wasting much processing time. The purpose of the process is seam recognition. A region of 300×200(pixels) is selected as the object of image processing to save memory space which contains the seam information, enhance anti-interference capability and speed image processing. This region contains all the information, which is needed for seam recognition. The performance of system is enhanced according to this selection. 3.2
Process of filtering
To make it easy to differentiate feature line from background and reinforce feature line, make the process of filtering. A template is used in the process of filtering; this template is introduced as the following table. Table 1. A template to reinforce vertical orientation
3.3
-1
2
-1
-1
2
-1
-1
2
-1
Process of binarization
Make the process of linear transformation similar to binarization, set gray level of background zero, and gray level of feature line 255. f(x,y) represents the gray level of original image. In term of some criterion get an appropriate threshold value expressed with T. g(x,y) representing gray level of processed image will be achieved after segmentation ,it is described as following: '255 g ( x, y ) = ( ) 0
f ( x, y ) ≥ T
(1)
f ( x, y ) < T
The threshold Selecting To differentiate object and extract seam information from complex background, the selection of threshold plays a key role in technique of threshold segmentation. If the value of threshold is selected too big, it leads to that the object is judged as background incorrectly. On the contrary, anti-results will be attained. So far there are many methods of choosing threshold, the universal methods are histogram threshold partitioning approach, two-dimension maximal entropy partitioning approach, fuzzy threshold partitioning approach and so on. Iterative threshold seg-
The Seam Tracking System for Submerged Arc Welding
147 147
mentation approach is adopted in this paper. This approach is introduced as the following: (A) Determine minimal gray level expressed with ZL and maximal gray level expressed with ZK, suppose the original value of gray level is
ZL + ZK 2
TK =
(2)
(B) TK divides image into two parts ,one is object, the other is background. Determine the gray level of two parts ZO and ZB which are depicted respectively as
Z
O
=
# Z (i, j ) × N (i, j ) z i, j < T K $ % % %% &
(3)
* + + ++ ,
# N (i, j ) z i, j < T K $ % % % &
* + + + ,
# Z (i , j ) × N (i , j ) z (i , j ) > T K ZB = # N (i , j ) ( ) z i, j > T K
(4)
Z(i,j) is the gray level of point (i,j), N(i,j) is the weight of the point (i,j), in generally ,N(i,j)=1.0. (C) Determine the new threshold TK+1, it is written as (5)
Z +Z B T K +1 = O 2 (D) If TK+1-TK then TK is the result. In this system consider TK as the gray level of this image . Otherwise ,K ← K+1 out process (B). !
"
3.4
(
&
)
* +
,
-
#
$
% &
'
&
go on carrying
Hough Transform
Seam information of the butt joint is extracted according to Hough transform after binarization. The obvious advantage is high interference immunity. The essence of Hough transform is to transfer many points to one point in two coordinate systems. Multiple points which represent a straight line or a well-regulated line in a coordinate system is switched to a point in another coordinate system. This point corresponds to parameters of original line. Make the process of Hough transform is to classify some points as well as congregate all the points of a line. Then the
148148H. Zhang et al.
check of the feature of two-dimensional line can be converted to search for maximum in an array of one dimension. Perfect results can be achieved by means of this approach even if there are just a little feature points of butt joint. It is proved that this approach has high reliability and stability when the butt joint parameters are changed greatly. 3.5
Result of seam recognition
It is complex in the producing spot of submerged arc welding for spiratron. There are a great number of interferential factors and these factors are not stable extremely. So the primary task is to remove these disturbances.
(a) Original image (b) filtering result (c) binarization result (d) Hough transform result Fig. 2. the process of image processing
Fig.2.(a) is the picture before image processing, which shows that there are a lot of disturbances on the surface of work piece. According to above described 3×3 filtering template, the vertical orientation characteristic is emphasized and the horizontal direction disturbances are removed effectively on the bottom of the image, shown in Fig.2.(b). A method of selecting threshold automatically is adopted in binarization, so seam feature changed constantly in the process of seam tracking can be highlighted. In Fig.2.(c), background and seam object are distinguished obviously after binarization which is the foundation of Hough transform. It is shown in Fig.2.(c) that there is a little disturbance still. Fig.2.(d) shows that disturbance has been removed and seam has been recognized through Hough transform. Adjust torch and CCD before image processing. The center of image is exactly oriented just before the torch. The center point represents the position of torch. At this time the distance from this point to seam line is seam error, which is one of the controller inputs. In this example the result indicates that the error is 12 pixels after image processing, which is equal to 0.6mm as the resolution ratio of CCD is 0.05 mm/pixel. This time the real error is 0.8mm through accurate measurement. So the processing error is 0.2 mm, the precision of recognition meets the requirement of seam tracking.
149 149
The Seam Tracking System for Submerged Arc Welding
4 Fuzzy Controller of the Seam Tracking System The welding process is subject to a great number of influencing factors, these factors are highly nonlinear and non-determinacy. It is so complex that classical control theory applied in automatic tracking system is not satisfactory. During recent years fuzzy control has been improved increasingly in application of regulating speed and tracking accuracy in seam tracking control system. But the application range of simple fuzzy control is limited. Enough high regulating speed can be achieved when error is smaller, however regulating speed can’t be amended according to the changing of error when error is larger. A conclusion can be drawn that simple fuzzy control is difficulty to meet the requirement of welding in reality yet. Fuzzy-p control theory is adopted in this seam tracking control system in this paper. The concept of Fuzzy-p is expressed as following: Proportional control is adopted to improve response speed and reduce regulating time when error is larger, Fuzzy control is adopted to improve the stability and control precision of response and decrease system overshoot when error is smaller. Switch between two control approaches is realized by designating threshold in advance. It is expressed simply as Proportional control if |E| EP is the absolute value of error if 0 |E| EP Fuzzy control A great number of experiments approved that the effect of tracking are satisfying when partition threshold EP is 1mm,then select EP 1mm. It is key to establish rules of fuzzy control in fuzzy control theory. In formula method of fuzzy control, a universal mathematical model is !
/
,
"
-
U 2
1
.
.
0
E+(1- )EC
(6)
1
is the fuzzy quantity of erWhere E or e is the fuzzy quantity of error, EC or ror variance ratio, U is the output fuzzy quantity, is the adjustment factor. reit vary from 0 to 1. The primary contradiction of flects the weight of e and control system is to remove e when e is large relatively, this time should be increased to enlarge the proportion of e weight in the control rules to improve dynamic characteristics of the system. On the contrary, when e is small the primary contradiction is to restrain system overshoot to make system stable as soon as possible. So should be decreased, meanwhile ,1 is increased to enlarge the proportion of weight in the control rules. The rules of fuzzy control can be modified through amending . Once is fixed, the rules of fuzzy control will be defined. The situation of seam error and working state is varied rapidly due to the influence of welding condition. The fixed fuzzy control can’t meet the demand of seam is obtained through table lookup. tracking. In the practice, When is defined, control rules self modulated online can be realized according to EC. Fig.3 describes the principle of self-modulated fuzzy control seam tracking system. 3
#
1
3
#
1
$
1
1
%
#
3
1
&
&
&
/
'
(
&
*
)
-
+
&
0
1
150150H. Zhang et al.
checking comparision parameter self-tune !
"
K1 d / dt
ec
'
#
fuzzifica -tion
Ec $
%
&
inference engine
defuzzifi -cation
(
K3
object
K2
Fig. 3. Structure of self- modulated fuzzy control seam tracking system
5 Conclusions This system responding quickly can recognize seam for almost all kinds of disturbances, its accuracy rate is high. The resolution ratio of CCD is 0.05 mm/pixel. The application of self- modulated fuzzy controller improves the control characteristic. It meets the need of automatic positioning and automatic tracking as well as easy to manipulate. The primary results have been obtained in the producing spot. It is shown in the results that the tracking accuracy of this system can meet the requirements of outside welding for spiratrons.
References 1. 2. 3. 4.
Gao Xiangdong et al. Application of Network Controller in the Network Controller in the Seam Tracking of Arc-welding Robot. Control Theory and Application, 1999, Vol.16, No.3 Kim J W, Na S J. A Self-Organizing Fuzzy Control Approach to Arc Sensor for Weld Joint Tracking in Gas Metal Arc Welding of Butt Joint. Welding Journal. 1993.72 Zhang Hua, Xian An, Peng Shaopin et al. Inner Weld Auto Tracking and Fusion Control System of Spiral Pipe. Welding Pipe.2002.1Vol.25, No.1 Huang Shisheng, Gao Xingdong, Yu Shiwei. Study On An intelligent Weld Seam Tracking System, Chinese Journal of Mechanical Engineering. 1999(12), Vol.35, No.6
The Seam Tracking System for Submerged Arc Welding
5. 6.
151 151
Hu Shengsun, Hou Wenkao, Qin Baozhong. Study on The Fuzzy-P Controller With Self-modulation Level Factor in Seam Tracking System. Journal of Tianjin University. 1999(3), Vol.32, No2 Wang Xiuyuan Hunag Shisheng, Xue Jiaxiang, et al. Application of Image Processing on The Image Segmentation in Weld Detection. Electric Welding Apparatus. 2000(10), Vol.30, 32~34 !
152
Visual Measurement of Path Accuracy for Laser Welding Robots Dong Du, Bo Sui, Y. F. He, Qiang Chen, and Hua Zhang Department of Mechanical Engineering. Tsinghua University. Beijing 100084. P.R. China [email protected]
Abstract. A sensing method based on binocular vision theory is presented to measure the path accuracy of the robots for laser welding. This method uses two cameras fixed on the end-effector of the industrial robot to capture the images of two marked-lines in the workspace. Based on line matching technology and stereo vision theory, a mathematic model has been set up to calculate the space coordinates of the two cameras from the captured images. This method is used to measure the path accuracy of a 6-DOF industrial robot. The results indicated that this method meet the requirement of continuous path measurement of industrial robots and the detection can be accomplished easily with little reference datum needed.
1 Introduction Industrial robot has been widely used in the application of laser welding recently. In these fields, the path accuracy of the robots is the key problem. But few types of equipment can be used to measure the 3-D location of the end-effector when the robot is running, and those available equipments are very expensive. An instrumentation that can test the variation of path accuracy for industrial robots in a brief way must be developed. Two kinds of measuring methods, which are contact and non-contact method, have been used to measure the position accuracy of industrial robots. However, the contact method cannot be performed continuous path measurement of dynamic trajectory accuracy, as the measuring speed and measuring scope of this kind of methods are very small. In non-contact method, there are two kinds of modes, which are initiative mode and passive mode. The initiative measuring system send signal such as infrared, laser or ultrasonic to the object, and then cipher-out the location of the object by detecting the returned signal from the object [1,2]. To work with this system a very complex and expensive device, difficult to operate, must be set up. So the passive method is considered recently [3-5]. In this article, a binocular vision system using two cameras fixed on the endeffector of the industrial robot has been setup to measure the path accuracy of industrial robots. To work on this system, the stereovision theory and the analytic T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 152−160, 2004. Springer-Verlag Berlin Heidelberg 2004
Visual Measurement of Path Accuracy for Laser Welding Robots
153 153
geometry equation in 3-D space have been studied. Then the projective plane equation of a spatial line with a CCD (Charge Coupled Device) camera has been introduced to get the mathematic model of this system, which is based on linematch technology. The path accuracy of an industrial robot has been measured simply and conveniently using this system, and the measuring results match the demands of continuous path measurement for the laser-welding robot.
2 Camera Projection Model The working process of a CCD camera is usually described by pinhole model (Fig. 1). Three coordinate systems have been used in this model. One is the world coordinate system represented by (XW, YW, ZW), one is the camera coordinate system (xc,yc,zc), the other is the image pixel coordinate system (u, v). The relationship between the retinal image coordinate and the world-coordinate can be written as Eq.2.1.
$XW + $u + %Y , W , % , = MX W zc v == [ L ⋅ F 0 ] M1 % % , % ZW , % , &%1 -, & 1 -
$f $ R t+ % Where , M 1 = % T , M = [ L ⋅ F 0] ⋅ M1 , F = 0 , % &0 1%& 0 $1 %d % x % L=% 0 % %0 % &
0 1 dy 0
(2.1)
0
0+
f
0, ,
0
1 ,-
,
+
u0 ,
, , v0 , , 1, , -
dx and dy is the physical distance between two adjacent pixels. u0 and v0 is the lower case of the pixel corresponding with optical axis. R and t is the rotation and translation matrix mapping from the world coordinate to the camera coordinate. M1 is a 4×4 matrix, mapping from the world coordinate to the camera coordinate system. M is the perspective projection matrix that accounts for both the extrinsic parameters of the camera and the intrinsic parameters of the camera.[3]
154154D. Du et al.
YC YW XC XW
ZC
ZW Fig. 1. Pin hole model
3 Visual Measurement Model for Path Accuracy Two cameras, both focused on the lines marked in the world coordinate system, are used in this measuring instrumentation. When the robot is moving, the relative position of the marked-lines and the cameras will be changed, which can affect the images captured by the CCD in a definite sampling rate. Consequently, the estimation of the path accuracy is to calculate the relative position, which changes in one sampling cycle time. 3.1 Projection model of a spatial line
Fig. 2 shows the relationship between a spatial line and its projection in the image plane. YC
O
ZC
XC
image plane l N
W ZW L YW
XW
Fig. 2. A spatial line and its projection
Visual Measurement of Path Accuracy for Laser Welding Robots
155 155
Suppose the projection l of a spatial line L in the image plane is expressed as an equation: KU=0
Where K = ( k1 , k 2 , k3 ) , U = (u , v ,1)T With reference to Eq.2.1, we can get the following equation by premultiplication the vector K.
[k
1
k2
$ XW + $u + % , $ R t + YW k3 ] %v , = [ K ⋅ L ⋅ F 0] % T , % , = 0 % , &0 1- % ZW , % , &%1-, &1 -
(3.1)
S is used to represent the production of K, L and F. thus Eq.3.1 can be written as SRXW + St = 0
(3.2)
Eq.3.2 describes the characteristics of the projection plane W, which is constructed by the optical center of the camera lens O, space line L and its projection line l. The normal vector N of the plane W can also be obtained from this equation. This vector is a 3×1matrix whose directional number is the production of S and R. This equation is the key to calibrate the camera. 3.2 End-effector position estimation
The projection of two space lines in two CCD cameras is shown in Fig. 3.
Fig. 3. Binocular visual measurement model
156156D. Du et al.
Assumption: CCD1 and CCD2 are fixed on the end-effector of the robot by rigid tools. The rotation and translation from the world coordinate to the two camera coordinates are described as (R1, t1) and (R2, t2) respectively; the CCD1 coordinate to CCD2 coordinate is described as (R3, t3). Then, because
' X CCD1 = R1 X W + t1 * ( X CCD2 = R2 X W + t 2 *X ) CCD2 = R3 X CCD1 + t3 The relationship between (R1, t1), (R2, t2) and (R3, t3) can be described as follows. R2=R3R1, t2=R3t1+t3
(3.3)
There are two marked-lines L1 and L2 in the world coordinate, which are linearly independent. The directional number of them are (m1, n1, p1) and (m2, n2, p2), respectively. The parameter of the first sample point (R0, t0) and world coordinates of A1 and A2 have already been calculated out by the calibration procedure. The rotation and translation of the end-effector from sampling point i to sampling point i+1 is described as (∆Ri, ∆ti) in the world coordinate system. Then the parameter of the number i+1 (Ri+1, ti+1) can be calculated from the following relationship. CCD1: Ri+1,1=∆RiRi,1,
ti+1,1= ∆t i+ ti,1
(3.4)
CCD2: Ri+1,2=R3Ri+1,1=R3∆RiRi,1, ti+1,2= R3ti+1,1+ t3= R3∆t i+ R3 ti,1+ t3
(3.5)
The estimation of the path accuracy of the robot is to calculate out the numerical value of rotation and translation matrices (∆Ri, ∆ti). To achieve this estimation, some geometry constraints should be considered. 1) Perpendicular constraint: The inner product of the normal vector N and projec$$$# tion line OA is nil Showed in figure 4. For example, that the vector Ni+1,11 is perpendicular with vector L1 in figure 4, thus the inner product of Ni+1,11L1 is zero.
' N i +1,11 L1 = Si +1,11 Ri +1 ,1 L1 = 0 ' Si +1 ,11 ∆ Ri Ri L1 = 0 *N L = S R L = 0 *S R ∆ R R L = 0 i +1 , 21 i +1 ,2 1 i i ,1 1 * i +1 ,21 1 * i +1,21 3 #( ( * N i +1 ,12 L2 = Si +1,12 Ri +1,1 L2 = 0 * Si +1,12 ∆ Ri Ri ,1 L2 = 0 *N * ) i +1 ,22 L2 = Si +1 ,22 Ri +1,2 L2 = 0 ) Si +1,22 R3 ∆ Ri Ri ,1 L2 = 0
(3.6)
2) In-plane constraint: The extreme points A1 (XWA1) and A2 (XWA2) must match the equation of the projection plane, which is constructed with marked-line L and its projection l.
Visual Measurement of Path Accuracy for Laser Welding Robots
157 157
' Si +1,11 Ri +1,1 X WA1 + Si +1,11 ti +1 ,1 = 0 * * Si +1,12 Ri +1 ,1 X WA2 + Si +1,12 ti +1 ,1 = 0 # ( * Si +1,21 Ri +1 ,2 X WA1 + Si +1,21 ti +1,2 = 0 *S ) i +1,22 Ri +1 ,2 X WA2 + Si +1,22 t i +1,2 = 0 'Si+111 , ∆ Ri Ri ,1 XWA1 + Si+111 , ( ti ,1 +∆ ti ) = 0 * , ∆ Ri Ri ,1 XWA2 + Si +112 , ( ti ,1 +∆ ti ) = 0 *Si+112 ( *Si+1,21 R3∆ Ri Ri ,1 XWA1 + Si+1,21 [R3 ( ti ,1 +∆ ti ) + t3 ] = 0 *S ) i+1,22 R3∆ Ri Ri ,1 XWA2 + Si+1,22 [R3 ( ti ,1+∆ ti ) + t3 ] = 0
(3.7)
From the Eq.3.6 and Eq.3.7 we can get eight linearly independent equations, where only the matrices ∆Ri and ∆ti are unknown and need to be figured out. In these equations, Si+1is made up of matrix K and the intrinsic parameters of CCD; the former can be obtained from the image-reorganization of lines; the latter can be achieved from the calibration of the CCD. If we use γ, β, θ to represent the rotation along X, Y, Z-axis, the ∆Ri, ∆ti can be written as
$ ∆ti , x + $cγ ⋅ cβ cγ ⋅ sβ − sγ ⋅ cθ cγ ⋅ sβ ⋅ cθ + sγ ⋅ sθ + % , % , ∆Ri = %sγ ⋅ cβ sγ ⋅ sβ + cγ ⋅ cθ sγ ⋅ sβ ⋅ cθ − cγ ⋅ sθ , , ∆ti = % ∆ti , y , %& ∆ti , z ,%& −sβ ,cβ ⋅ sθ cβ ⋅ cθ
(3.8)
where s- and c- is the abbreviation of sine and cosine. There are 6 unknown variables that must be calculated in these eight equations. To solve this problem, least square method is used to get an optimized value. Thus, the deviation of the sampling point i+1 to sampling point i is achieved.
4 Experiment and Results An experiment has been carried out to prove the feasibility of this measuring method. The experimental system is made up of a calibration base, two CCD cameras (resolution 800×600), some fixing tools and an industrial robot, just like figure 4. Two lenses, whose focal lengths are about 50mm and object field angles are about 5.5 degree, are used in the cameras. There are two unparallel lines marked in the calibration base. These lines are very brightness in the black background and easy to be imaged by the cameras. The precision of the system is tested to be little than ±0.2mm. In a common laser-welding application, the deviation of the welding path is not allowed to be above ±0.5mm. Thus, this system can be used to
158158D. Du et al.
measure the path accuracy of industrial robot, which is used in laser-welding application. A 6-DOF (Degree of freedom) articulated robot, whose reach radius is about 1300mm and maximum payload is about 6kg, is measured. The measuring results are list in figure 5 and 6 (the figures of rotation deviation along y-axis and z-axis are elided). From the results, the higher the moving speed, the worse the path accuracy of the robot. The path position accuracy of this robot is more than 4.05mm. This measuring result is proved to be credible by a contact measurement method. Industrial Robot
CCD Camera #1 Fixing Tools
CCD Camera #2
Marked-lines
Fig. 4. System setup
Fig. 5. Measuring results of position error of industrial robot in different moving velocities
Visual Measurement of Path Accuracy for Laser Welding Robots 11.22m/min 45.00m/min 90.00m/min
α/°
2.4
Rotation deviation along x-axis
159 159
2.2 2.0 1.8 1.6 1.4 0
5
10
20 15 Number
25
30
Fig. 6. Measuring results of orientation error along x-axis in different moving velocities
5 Conclusion The measurement method of path accuracy for laser welding robot has been presented. Using this method, the path accuracy of industrial robots can be achieved easily and conveniently. The measurement of a robot moving along a path of straight line is performed successfully. The experimental result demonstrates that the measuring method for industrial robots is capable of measuring path accuracy of laser-welding robot.
Acknowledgements This work is Supported by the Teaching and Research Award Program for Outstanding Young Teachers in Higher Education Institutions of MOE, P.R.C. and the National Nature Sciences Foundation of China ( No. 59975050).
References 1. 2. 3.
Shirnzadeh Bijan. (1998) Laser-interferometry-based tracking for dynamic measurement. Industrial Robot, 25(1): 35–41. Yuan Jing, Yu S L. (1999) End-effector position-Orientation Measurement. IEEE transition on Robotics and automation, 15(3): 592–595. Ma Songde. (1996) A self-calibration technique for active vision systems. IEEE Transactions on robotics and automation, 12(1): 114-120
160160D. Du et al.
4. 5.
Motta JMST, Carvalho GC, McMaster RS. (2001) Robot calibration using a 3D visionbased measurement system with a single camera. Robotics and Computer Integrated Manufacturing, 17(6):487-497 Gordon Wells, Carme Torras. (2001) Assessing image features for vision-based robot positioning, Journal of Intelligent and Robotic Systems, 7(1):95-118
Map Building and Localization for Autonomous Mobile Robots Y.L. Ip, A.B Rad, and Y.K. Wong Department of Electrical Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR, China. [email protected]
Abstract. In this chapter, we present our recent works on map building and localization algorithms for autonomous mobile robots. We start with the segment-based map building via enhanced adaptive fuzzy clustering technique. Based on this algorithm, we develop a dynamic segment-based algorithm in Bayesian state framework. A localization algorithm will be developed in the second part of the chapter. We propose a model-based localization algorithm based on fuzzy-tuned extended Kalman filter. These algorithms will be tested in real-time applications and scenarios.
1 Introduction Many industrial applications require autonomous mobile robots to be able to perform their tasks in partially known or unknown environments with little a priori information. This requirement dictates that the mobile robot is “aware” of its surroundings. It should be able to build an initial map, be able to maintain and update the map and finally to be able to use this map in order to determine its position. The purpose of this chapter is to discuss these issues and suggest some algorithms for map building and localization. The problem of simultaneous map building and localization (SLAM) will not be discussed here. Both map building and location algorithms rely on the perception device. This could be in the form of vision, radar, lasers and ultrasonic sensors. We limit our discussion to ultrasonic perception device. The algorithms described here can be expanded to cover these perception devices with not much difficulty. There are several approaches to solve the map building problem. Among those are occupancy grid or geometric maps. The occupancy grid maps are generated from stochastic estimates of the occupancy state of an object in a given cell. It is rather easy to construct and maintain them with reasonable computing power in small workspace. However, in large environments the storage space and the processing of the maps become matters of concern. This study focuses on geometric methodology. Models based on geometric boundaries approximate the location, orientation and size of the boundaries and are applicable to environments T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 161−189, 2004. Springer-Verlag Berlin Heidelberg 2004
162162Y.L. Ip, A.B. Rad, and Y.K. Wong
susceptible to inaccurate and/or noisy data. The object boundaries can be approximated with straight lines or curves from additional processing of the sensor measurements. The rest of this chapter is organized as follows: Fuzzy clustering and its associated algorithms are reviewed in Section 2. A segment-based map building algorithm is reported in Section3. This algorithm is extended to dynamic case in Section 4. In Section 5, we discuss a localization method based on fuzzy-tuned extended Kalman filter. Experimental results are included in Section 6. The chapter is concluded in Section 7.
2
Fuzzy Clustering Algorithms
The fuzzy clustering algorithms have been used in many applications involving data segmentation. The basic fuzzy clustering algorithm is the fuzzy c-means (FCM) method developed by Bezdek (Bezdek 1981). FCM is aimed at minimizing the sum of the squared distances between the data points and the c cluster centers Vi. The objective function to be minimized is cp J m : M × R → R+ fc n c m 2 c J m ({U ik }, {Vi }) = # # (U ik ) (d ik ) | # U ik = 1 , k = 1,2,...,n (1) i =1 k =1 i =1 where, Mfc is the fuzzy partition space; U∈Mfc is a fuzzy c-partition of X; V={V1,V2,…,Vc} are the cluster center vectors, Vi is a cluster center of p features, Vi∈ℜ p ; X is the matrix of feature vectors, where X={ X1,X2,…,Xn }, Xi∈ℜ p; and dik is given by 2 2 (2) ( d ik ) = x k − vi Here, n is the total number of feature vectors (data points) and c is the number of cluster centers. The index i denotes each clusters, index k denotes each of n number of data points and p is the dimension of ℜ space. The exponent m∈[1,∞) is used to adjust the weighting effect of the membership value. The cluster centers vi can be obtained from the following: n m # (u ik ) x k k =1 vi = n m # (u ik ) k =1
, i = 1,2,..., c
(3)
The fuzzy membership matrix U can be obtained from the following: u ik =
1 2 c d ik m # ( ) −1 j =1 d jk
, i = 1,2,..., c ; k = 1,2,..., n
if dik = 0 then uik = 1 and ujk = 0 for j ≠ i.
(4)
Map Building and Localization for Autonomous Mobile Robots
163 163
The cluster center matrix V is randomly initialized and the fuzzy partition matrix U is created by using Eq. 4. The initialization of V is done by randomly selecting a value for each feature of a cluster center vi within the range of values exhibited by the feature in the data set. The stopping condition is set to ustop. The theoretical details and the proofs of the above are well documented and can be found in (Bezdek 1981). This FCM can not be used directly for line segment detection in a data set. It is more suitable for feature detection of linear shapes. For our purpose, it is best to directly detect the line segment within the data set space. There are several variations of FCM that allows detection of line segments, such as: Fuzzy c-elliptotypes (FCE) algorithm (Bezdek 1981; Runkler and Palm 1996), Adaptive fuzzy clustering (AFC) algorithm (Dave 1989; Phansalkar and Dave 1997). 2.1
Noise Clustering
Noise is a common problem in many environments. The data set obtained by the ultrasonic sensors includes arbitrary distribution of noise and creates numerous problems for minimization of squared error type algorithms. Noise Clustering (NC) was proposed in (Dave 1991) and aims to alleviate this problem. In NC algorithm, the noise is considered as a separate cluster which contain data that cannot be classified into other clusters. Noise prototype is defined as a universal entity such that it is always at the same distance from every point in the data set. From (Dave 1991), the definitions of the noise distance δ is defined by the following equation:
where
ζ
n ' c#−1 # 21 ( d ik ) ( 2 2 δ = ζ ( i =1 k =1 2 ( n(c − 1) 2 ) 3 is the value of the multiplier used to obtain
(5)
δ
from the average of
distances. Based on Eq. 5, δ can be calculated at each iteration of the sequence. The details for the EAFC algorithm along with the NC technique are discussed in next section. 2.2 Enhanced Adaptive Fuzzy Clustering algorithm (EAFC) integrated with NC technique
The main feature of our proposed approach is to divide the AFC algorithm into two phases. Phase I is a standard FCM iterative process whereas phase II is an AFC with the cluster centers initialized by the last cluster center values from phase I. The function of phase I is to select the cluster centers within the data set by the standard FCM algorithm. After obtaining the cluster centers in the data set, the phase II (AFC) is used to calculate the line segments. The previously calculated cluster centers are used for initialization of the fuzzy partition matrix of AFC to obtain the
164164Y.L. Ip, A.B. Rad, and Y.K. Wong
cluster with linear prototypes (line segments in our case). In this algorithm, the type of distance norms is Euclidean. Let ustop_I and ustop_II be the stopping condition in phases I and II respectively. The outline of the proposed algorithm shown as following: Phase I 1. Fix c, 2 ≤ c < n; fix 1 ≤ m < ∞; initialize the cluster center matrix V by using a randomly selected entry in the data set; initialize the fuzzy partition matrix U(ρ = 0) with Eq. 4. 2. Increment ρ = ρ +1. Compute the cluster center vi by Eq. 3. 3. Update membership function U(ρ = ρ+1) by using Eq. 4, along with dik calculation by Eq. 2. 4. If |U(ρ) - U(ρ-1)| < ustop_I then stop, else repeat steps 2-4. Phase II 1. Use the same c as select in Phase I, fix 1 ≤ m < ∞, initialize the cluster center matrix V and the fuzzy partition matrix U by using results from Phase I, initialize mixing coefficient αi, 0 < αi <1. 2. Increment ρ=ρ+1. Compute cluster center vi by using modified Eq. 3, defined below: n m # (u ik ) x k k =1 , i = 1,2,..., c - 1 vi = (6) n m # (u ik ) k =1 3. Compute the fuzzy scatter matrix using Eq. 7 and then obtain their eigenvalues to calculate theαi by using Eq. 8, defined below respectively. n m Τ S fi = # (u ik ) ( x k − vi )( x k − vi ) , i = 1,2,..., c - 1 (7) k =1
4.
τ αi = 1 − ( i ) ξi
(8)
*-(1 − α ) d 2 + α D 2 , i = 1,2,..., c - 1 i ik i ik (γ ik ) = + 2 -, ,i = c δ
(10)
where τi = min(λ1i, λ2i) and ξi = max(λ1i, λ2i), i = 1,2,…,c-1. Update membership function U(ρ = ρ+1) by Eq. 9, along with γik calculation by Eq. 10 and with δ calculation by Eq. 5, defined below. 1 u ik = , i = 1,2,..., c - 1 ; k = 1,2,..., n 2 c γ ik m −1 (9) ) # ( j =1 γ jk
2
where Dik is the distance of the point from the line prototype, and is given by
Map Building and Localization for Autonomous Mobile Robots
( Dik ) 2 = (d ik ) 2 − (< xi − vi , d i >) 2 5.
(ρ)
If |U
165 165 (11)
(ρ-1)
-U
| < ustop_II then stop, else repeat steps 2-5.
3 Segment-Based Map Building via EAFC with NC for Static Environment In the above section, we provide a brief outline of the Enhanced Adaptive Fuzzy Clustering Algorithm with noise clustering approach (EAFC with NC). The main contribution of this algorithm is to extract the line segment from the noise sonar measurements. In this section, we will introduce how to obtain the environment model during the robot traveling in real time. This algorithm is based on the recursive updating of the line segments to extract the features in the map and is based on the following sequence of steps, that analyzes the uncertainly in sensor information with different levels: • As the mobile robot moves in the environment, the consecutive observations n
obtained by the sonar sensors S is given by n n n n n n n S = {( x1 , y1 ), ( x 2 , y 2 ), # , ( x k , y k )}, n = 1, 2, # ,16 (12) Here, n is the number of sonar sensors and k is the number of measurements at each sampling interval. • The world environment is modeled into two-dimension and divided into
squared cells C ( X ,Y ) (local models) (see Fig. 1) to reduce the complexity of data structure. During the robot sampling, the sonar measurements from the environment will transform to global coordinate system. At the same time, the sonar data points are grouped in the related cells defined by
C ( X ,Y )
. The cell
C ( X ,Y )
C ( X ,Y ) = {( x1 , y1 ), ( x 2 , y 2 ), # , ( x % , y % ), B1 , B 2 ,..., Bi }
is
(13)
where X and Y are the cell location in the global frame X-axis and Y-axis, respectively. Index % denotes the number of sensor measurements within the cell. The ith extracted line segments from the corresponding sonar data points is given by
Bi = {φ , ( x S , y S ), ( x E , y E ), ( xV , yV )}
(14)
where φ is the unit-eigenvector of the segment and the end-points of the segment, ( x S , y S ) and ( x E , y E ) are obtained by the EAFC with NC approach. ( xV , yV ) is cluster center of this segment. The cell contains a discrete sample of the outline of the feature within the cell.
C ( X ,Y )
166166Y.L. Ip, A.B. Rad, and Y.K. Wong
(a)
(b) Fig. 1. a) Grouping the sensor measurements in cell C ( X ,Y ) . b) Updating the sensor measurements and line segments simultaneously.
• To extract the line segments in each cell C ( X ,Y ) by the EAFC with NC and store in Bi. • Finally, the similar line segments within a cell are combined together by the compatible line segment merging technique (Ip et al. 2002). It is proposed by the authors’ pervious work. The contribution of the “compatible line segment merging technique” is to merge the similar basic segments together to form a single line segment. The basic segments are discontinuous outlines of the object boundaries in the vicinity of the mobile robot trajectory (Fig. 1). In order to improve the quality of the map, the number of line segments representing the same boundary is merged up by evaluates the relation of the eigenvector in both segments within the cell. In the proposed map building algorithm, the compatible line segment merging technique is implemented in two routines: “local grouping” and “global grouping”. When two segments (i.e. 2 clusters in each cell) are extracted, we apply the compatible line segment merging technique to merge those segments. This routine is called “local grouping”. In “global grouping” routine, the cells are combined by the compatible line segment merging technique based on the pervious updated cells as the center and merging the updated cells with the surrounding cells. Two routines are shown in Fig. 2. The merged segments and the location of the cells are stored in the “global buffer” for display and global map information. However, the segments in the cells are still stored in the local cell memory. This information is saved for the next updating since the pervious
Map Building and Localization for Autonomous Mobile Robots
Fig. 2. The operation of “Local Grouping” and “Global Grouping” routines.
167 167
168168Y.L. Ip, A.B. Rad, and Y.K. Wong
cell information is used for initialization in EAFC with NC. Since the EAFC with NC segment extraction scheme is proposed by the authors’ pervious work. The theoretical details and discussion of the above are well documented and can be found in (Ip et al. 2002).
3.1 Outline of the proposed On-line Segment-based Map Building algorithm
Task 1: Initialization. Fix the number of clusters c, the stopping condition ustop_I and ustop_II for each cell. The size of each cell is required to fix. Task 2: To obtain the sonar sensors measurements S n from the environment (global frame) and transform to global coordinate system. Task 3: To group the sensors measurements into the related cells C( X ,Y ) . Task 4: To extract the line segments in each cell by EAFC with NC approach and then store it in form of Bi in Eq. 14. The information in Bi (i.e. cluster centers ( xV , yV ) and mixing coefficient α ) is used for the initialization of EAFC with NC. Task 5: “Local grouping” routine: to combine the similar cluster segments in each cell by compatible line segment merging technique. Store all the update cells in the buffer for “global grouping”. Repeat Task 2 to 5 for next sampling interval. We use Tlocal to denote the sonar sensors sampling interval. Task 6: “Global grouping” routine: We use Tglobal ( Tglobal > Tlocal ) to denote the “global grouping” sampling interval. We use updated cells (the updated cells are stored in the buffer in Task 5) as a center to merge the surrounding cells. The merged segments and cells location are stored in “global buffer”. Once the mobile robot finishes the navigation in the workspace, the final map information is selected from each cell and the “global buffer”. If the cells are overlapped, we select the segments in “global buffer” instead of that in the cell.
4 Dynamic Segment-Based Map Building via Bayesian State Estimation Framework In the pervious section, our proposed segment-based map building via EAFC with NC for static environment is introduced. In this section, we will extend the proposed map building algorithm to handle a dynamic environment. The definition of dynamic case in this paper is stated as: the object in the modeling environment is not fix and it can be relocated by the human.
Map Building and Localization for Autonomous Mobile Robots
4.1
169 169
Estimating the state of segments
We use the segment-based map building with sonar sensors to extract the object in the environment. We propose using Bayesian update rule to estimate and update the state of segments. Each extracted segment is divided into j parts. Each part is assigned a probability function of its state s ( Bij ) which is conditioned over r observations (sonar sensors measurements) within Bayesian framework. The Bij denotes the jth part at the ith extracted segment in the related cell C( X ,Y ) . The state variable s ( Bij ) associated with a part of segment Bij is defined as a discrete random variable with two states, present and absent, denoted PRESENT and ABSENT. Since the states are exclusive and exhaustive, P [ s ( B ij ) = PRESENT ] + P [ s ( B ij ) = ABSENT ] = 1 . To determine how a sensor reading is used to estimate the state of the a part of a segment, we start by applying Bayes’ theorem (Berger 1985) to a part of segment Bij : P[s( Bij ) = PRESENT | r ] =
p[r | s( Bij ) = PRESENT]P[ s( Bij ) = PRESENT] #s( B ) p[r | s( Bij )]P[ s( Bij )] ij
(15)
where P[ s ( Bij ) = PRESENT ] is prior presented segment probability of a part of segment Bij . P[ s ( Bij ) = PRESENT | r ] is new presented segment probability of a part of segment Bij based on r observations. In Eq. 15, the term p[r | s ( Bij ) = PRESENT ] corresponds to sonar sensor pdf that will be discussed in the following section. In order to estimate the state of complete segment, the average value of the j parts of segment states is calculated as shown below: P[ s ( Bi ) = PRESENT | r ] =
# j P[ s ( Bij ) = PRESENT | r ]
(16)
j
The value of j can be determined by length of segment Bi
(17) σ where σ is range variance of the sonar sensors. Here, we use 40mm. The minimum value of j is fixed at 1. j≈
4.2
Fuzzy tuned sonar sensor model
The sonar pdf is further tuned to accommodate a dynamic environment. A similar idea is proposed in the authors’ pervious work (Chow et al. 2002) for grid-based map building. Here we proposed an enhanced and modified version of tunable sonar pdf model. We select a zero-mean Gaussian distribution to represent the sonar
170170Y.L. Ip, A.B. Rad, and Y.K. Wong
sensor range readings. Only radial uncertainty is considered. The pdf model is shown as follows: * $ − (r − z) 2 . /≤k -k ε , if x < r and k o exp % % σ 2 / ε & 0 r $ − (r − z ) 2 . />k if k o exp% % σ 2 / ε $ − (r − z ) 2 . & 0 r % / , p ( r | z ) = +k o exp % σ 2 / $ − (r − z) 2 . & 0 r / > 0.5 , if x > r and k o exp % % σ 2 / & 0 r , otherwise -0.5 -,
(18)
where x = The distance from the sensor k ε = A parameter that corresponds to the empty space that is tuned by a fuzzy model. k o = A parameter that corresponds to the occupied space and is tuned by a fuzzy model. 2 σ r = The variance of the radial probability. r = The sensor range measurement of the sonar sensor. z = The true parameter space range value. To extend the derivation to two spatial dimensions, consider the example of a range sensor characterized by Gaussian uncertainty in both radial and angular directions. This can be represented as: * $ −θ 2 . % / -1 − (1 − kε ) exp% 2 / & σθ 0 $ 2. $ − (r − z) 2 . % /) exp% − θ / -1 − (1 − k o exp% 2 / %σ 2 / σ & 0 r & θ 0 p(r | z, θ ) = + 2. $ 2. $ -k exp% − ( r − z ) / exp% − θ / - o % σ 2 / %σ 2 / r & 0 & θ 0 -0.5 ,-
where
, if x < r and p ( r | z ) = k ε and (1 − kε ) exp
$ −θ 2 . % / > 0.5 %σ 2 / & θ 0
if x < r and kε < p ( r | z ) < 0.5 ,
$ 2. $ − (r − z ) 2 . /) exp% − θ / > 0.5 %σ 2 / % σ 2 / & 0 r & θ 0
and (1 − k o exp%
$ − (r − z ) 2 . $ − θ 2 . / > 0.5 / exp% 2 / % 2/ & σr 0 & σθ 0
(19)
, if p ( r | z ) > 0.5 and k o exp% %
, otherwise
θ = The azimuth angle measured with respect to the beam central axis. 2 σ θ = The variance of the angular probability.
In this algorithm, k ε and k o are tuned by the fuzzy systems to be further described (see Fig. 4 and 5). In both fuzzy systems, the fuzzy model includes two inputs and a single output with singleton fuzzier, product inference engine and center average defuzzifier (Wang 1997). The tuning strategy is based on the following four variables and their geometric representation is shown in Fig. 3:
Map Building and Localization for Autonomous Mobile Robots
171 171
PmaxB : the highest probability of the extracted segment PmaxBi within the sensor cone. P*B : the probability of the nearest segment within the sonar cone . dr : the distance between the range reading and the extracted segment with the highest probability within the sensor cone. (i.e. range of sensor - location of the extracted segment with the highest probability within the sensor cone) ∆dr : the change of dr. (i.e. ∆ dr = drt − drt −1 , where t is sampling time)
(a)
(b)
Fig. 3. a) Basic condition for tuning kε and for tuning k o .
(c)
k o . b) Condition for tuning kε . c) Condition
The special criteria for tuning these two parameters will be discussed below: For the first criteria to tune these two parameters, the acute angle θ oi between the extracted line segment Bi and valid sonar beam should be located within 90° to θ lim it , where θ lim it is the angle limit. Since the half of valid sonar beam angle is 12.5°. The θ lim it is selected as about 77.5°. Fig. 3(a) shows the situation that satisfies the first criteria. If the angle θ oi exit these limit, these sonar sensor data point will not be used to update the probability of these line (or cell). The parameter k ε corresponds to the probability that a cell is empty (or the segment is absent in our case). In the traditional approach, this parameter does not exist and is implicitly set near to zero. Therefore, when the object detected range of sensor is much larger than its actual location, the probability of the corresponding extracted segment may be reduced to close the zero at once during Bayesian update. In order to avoid this problem, it is suggested to increase k ε when the range of the sensor measurement is much larger than the extracted segment location and vice versa. The corresponding membership function and fuzzy rule-base are shown in Fig. 4 and 5, respectively. This fuzzy system is only activated when the detected range of the sensor measurement is larger than the distance between the corresponding extracted segment and position of sensor (i.e. dr = positive value)
172172Y.L. Ip, A.B. Rad, and Y.K. Wong
and the highest probability of the corresponding extracted segment is larger than Pth (i.e. PmaxB > Pth ), otherwise k ε will be a constant. VS
S
M
L
VL
Plim it 0.5 + 3Plim it 0.5 + Plimit 1.5 + Plimit 4
2
VS
Pth
0.5
4
S
0
Z
L
0.4
0
0.8
VS
MS
M
ML VL
0.5
0.65
0.75
0.85 0.9
6
L
1 + 3Pth 4
1 + Pth 2
3 + Pth 4
VL
1
S
M
L
0.125
0.25
0.375
VL
0.5
Input Space of dr
Input space of ∆ dr
Output consequence fuzzy sets for ko
M
Input Space of PmaxB
Input space of P*B Z
S
VS
MS
M
ML
VL
0.05
0.2
0.3
0.4
0.5
Output consequence fuzzy sets for kε
(b)
Fig. 4. The membership functions and the output consequence fuzzy sets for tuning a) k o and b) k ε .
The parameter k o corresponds to the probability that the cell is occupied (or the segment is present in our case). In conventional approaches, this parameter is fixed. Therefore, when the object is moved towards the sensor, the probability can not reach up again. In our approach, k o is tuned by the fuzzy system. It is suggested to increase k o when the object is moved towards the mobile robot (or sonar sensor). The corresponding membership function and fuzzy rule base are shown in Fig. 4 and 5, respectively. This fuzzy system is only activated when the detected range of sensor is within the range between the location of sensor and existing extracted
Map Building and Localization for Autonomous Mobile Robots
173 173
segment (i.e. dr = negative value) and P*B is lower than 0.5, otherwise k o will be fixed.
(a)
(b)
Fig. 5. a) Fuzzy rule table corresponding to k o . b) Fuzzy rule table corresponding to k ε .
4.3
Outline for updating the state of an extracted segment
Note that the input and output memberships in Fig. 4 and 5 are tuned by simulation trial. All the new extracted segments are initialized with 0.5 probability state. A step-by-step procedure for updating the state of an extracted segment is presented as follows: 1. Obtain the sonar sensor measurements and use the kinematics transformation to obtain global location and store it in the cell C( X ,Y ) . 2. Update and extract the segment in the corresponding cell by the EAFC with NC for all new sonar measurements if the total numbers of data point in the related cells more than 5 data points (designed by human). 3. Locate the highest probability of extracted segment within the new sensor measurement cone, and obtain the dr, if b a then tune the k ε . If dr < b and the probability of the corresponding segment less than 0.5 then tune the k o . Go to step 6. 6. Based on the new tuned value k ε or k o , calculate the probability of the corresponding segment be within the sonar cone by sonar sensor pdf (Eq. 19). Go to step 7. 7. After updating the state of segments, if the probability of state is lower than or equal to Plimit, then the segment and the corresponding data points are removed. Go to step 1 when new sonar measurement(s) is/are available.
174174Y.L. Ip, A.B. Rad, and Y.K. Wong
where Plimit is the lower limit of the probability of the all extracted segments or objects. It is determined by the experimental trial. The Plimit can control the sensitivity of the proposed algorithm. If the value of Plimit is increased, the segment Bi will be deleted after few samplings. Otherwise, a longer time is required to update a true situation. 4.4
Assumptions and limitations
For this paper we will consider a mobile robot moving about in the well-constructed indoor environment with dynamic objects, such as box, furniture or other moving object. The proposed algorithm can enable the mobile robot to be aware of the changes around them. The paper focuses on the segment extraction from the sonar measurements. Therefore, the automatic navigation and the localization system are outside the scope of this study. The parameters in the proposed algorithm are restricted to use sonar sensor. If the mobile robot equipped with laser-range finders, the sensors pdf and sensor variance should be changed.
5 Model Based Localization via Fuzzy Tuned Extended Kalman Filter FT-EKF The capability to acquire the position and orientation of an autonomous mobile robot is an important element for achieving specific tasks requiring autonomous exploration of the workplace. Once we obtain a map (a set of line segment) of the robot workplace by a mobile robot or hand-measured, this map can be provided to a mobile robot for further usage. Here, the world environment is assumed modeled into two-dimensions. Just as humans use two-dimensional floor plans, the robot uses a map, which projects all features into two dimensions. The complexity of the map is thereby greatly reduced. The simplification is often reasonable in practice, especially in man-made environments. 5.1 Mobile robot and sensor modeling and background formulation of extended Kalman filter EKF
We have applied our algorithm to Pioneer 2DX (Pioneer 2 Mobile Robot Operation Manual 2001) mobile robot which is a differential drive type robot. According to a two-dimensional world assumption, the robot configuration w.r.t. the world Τ coordinate can be defined as X = [x y θ ] containing its position ( x, y ) and orientation θ . For the robot motion (see Fig. 6), we assume that the incremental motion is in a straight line (small robot motion) (Konolige 2001). A small robot motion is defined as traveling a short distance “s” from the origin at angle θ and a
Map Building and Localization for Autonomous Mobile Robots
175 175
turn angle α . These two can be integrated to give the global position of the robot. According to this assumption, the updated robot position and orientation are given by a simple function as: ' x + s cos θ 1
X ' = ( y + s sin θ 2
(20)
( θ +α 2 ) 3
The system model describes how the robot pose X (k ) changes with time k in response to the control input u (k ) = [ s (k ),α (k )]Τ and a noise source v(k ) . This noise source v(k ) is assumed to be a zero-mean Gaussian with covariance Q (k ) . The system model is expressed as:
X (k + 1) = F ( X (k ), u (k )) + v (k ), v(k ) ~ N (0, Q(k ))
(21)
where F ( X (k ), u (k )) is the non-linear state transition function. This state transition function is equal to the robot pose update function (Eq. 20) which is given by: ' x ( k ) + s ( k ) cosθ ( k ) 1
F ( X ( k ), u ( k )) = ( y ( k ) + s ( k ) sin θ ( k ) 2
( )
θ (k ) + α (k )
2 3
(22)
Fig. 6. Control command and the assumed small robot motion.
Estimate position can generally be obtained by two methods: internal and external (Konolige 2001). The internal method exploits the motion or forces produced by the robot itself with no regards to the external environment. On the other hand, the external method relies on the signal from the environment to register the robot’s position. In our case, we fuse these two methods together to estimate the robot’s pose. Our Pioneer 2DX mobile robot consists two types of sensor, wheel encoder (for internal method), ultrasonic sensors (for external method). In the following paragraphs, a brief overview of each sensor is introduced. The wheel encoders are used to measure wheel rotation and steering orientation. For short distance travel, it is sensible to assume that only a small error occurs. With this assumption, the error can be eliminated at each localization sampling instant. There are three types of movement errors: k R is range error factor (unit:
176176Y.L. Ip, A.B. Rad, and Y.K. Wong
mm 2 / m ), kθ is turn error factor (unit: deg 2 / deg ) and k D is drift error factor
(unit: deg 2 / m ) (Konolige, 2001; Thrun et al., 2001). For convenience, these errors are represented in variance form. For example: If the robot moves a distance s and turns an angle α , then the variance will be: var(s ) = k R s and
var(θ ) = kθ α + k D s . Since the wheel encoders are installed in the motor shaft, we can estimate the short distance s and turn angle α via the left and right wheel encoders (a simple calculation is used at every sampling time (see (Konolige 2001)). Pioneer 2DX supports both front and rear sonar arrays. Each is equipped with eight Polaroid ultrasonic sonar sensors (Pioneer 2 Mobile Robot Operation Manual, 2001). The sonar positions in all arrays are fixed. One sensor on each side and six facing outward at 20-degree spacing provide 360 degrees coverage of the environment. The local position and orientation of ith sonar (i = 1,2, $ , n S ) is denoted by ( xi , yi ) and θ i , respectively and nS denotes the total number of sensors. Therefore, the global position and orientation of the sonar at time k is given by: i xS ( k ) = x ( k ) + xi cos(θ ( k )) − yi sin(θ ( k )) i y S ( k ) = y ( k ) + xi sin(θ ( k )) + yi cos(θ ( k )) (23) i θ S (k ) = θ (k ) + θi The actual reading of the sonar sensor (observation) can be denoted as set Z (k ) where: Z ( k ) = {z i ( k ) | 1 ≤ i ≤ n s )
(24)
In the 2-D map, it is contain a jth ( j = 1,2,..., n p ) line segments only, n p denote the total number of planes. The form of the line segment representing here is shown as following: plj ( x plj [1], y plj [1], x plj [2], y plj [2])
(25)
where ( x [1], y [1]) : line segment starting point global X-Y coordinate. j pl
j pl
( x plj [2], y plj [2]) : line segment end point global X-Y coordinate. Now we define two more symbols related to the line segment (or plane), a plj where is the angle of the normal line of the related segment (count from positive side of x-axis) and it is given by:
$ x j [1] − x j [ 2] . pl pl / j j % y pl [1] − y pl [ 2] // & 0
j −1% a pl = tan %−
and dplj where distance form the origin to the segment and it is given by:
(26)
Map Building and Localization for Autonomous Mobile Robots
j d pl =
j j j j x pl [1] y pl [2] − x pl [ 2] y pl [1] j j j j 2 2 ( y pl [1] − y pl [ 2]) + ( x [1] − x pl [ 2]) pl
177 177
(27)
From the Eq. 23, 26 and 27, the measurement function of the line segment due to the ith sonar sensor at time k can be calculated as follows: hi (k , plj ) = ρ v (d plj − x Si (k ) cos a plj − y Si (k ) sin a plj ) where
(28)
ρ v is 1 or –1 to determine the visible side, δ δ1 ' a plj ∈ (θ Si (k ) − ,θ Si (k ) + 2 , δ is sonar beam angle,
2 23 ) Only one jth plane is matched with the ith sonar sensor.
Fig. 7. Measurement Model in graphical representation and validated sonar condition.
The graphical relationship is shown in Fig. 7. Finally, the observation model describes how the sonar reading z i (k ) at time k is related to the robot’s position and is given by: z i (k ) = hi ( X (k ), plj ) + wi (k ), wi (k ) ~ N (0, Ri (k ))
(29)
The term wi (k ) represents measurement noise (Wang, 1988). It is also assumed to be a zero-mean Gaussian with covariance Ri (k ) . EKF is often used to combine all measurement data (e.g., for fusing data from different sensors) to get an optimal estimate in a statistical sense. The overall goal of
178178Y.L. Ip, A.B. Rad, and Y.K. Wong
the EKF cycle is to find a new (a posteriori) estimated position Xˆ (k + 1 k + 1) and its covariance Pˆ (k + 1 k + 1) , given the old (a priori) estimated position Xˆ (k k ) and its covariance Pˆ (k k ) at time k. The meaning of Xˆ (k + 1 k + 1) stated is the predicted position Xˆ at time k+1 given all observations up to and including time k+1. The formulation of the EKF is as follows: 1. Filter Initalization. Initialize the mean square error covariance matrix P (0 | 0) , predict the position Xˆ (0 | 0) , state noise covariance model Q(0) and measurement noise covariance model R(0) . 2. Filter Prediction. Calculate the predicted state Xˆ (k + 1 | k ) by using Eq. 22 as: Xˆ (k + 1 k ) = F ( Xˆ (k | k ), u (k )) ' f x (k ) 1 ' x(k ) + s (k ) cosθ (k )1 = (( f y (k )22 = (( y (k ) + s (k ) sin θ (k ) 22 () f θ (k )23 () θ (k ) + α (k ) 23
(30)
Partial differentiate Eq. 30 with respect to the pose ( x(k ), y (k ),θ (k )) and it is given the Jacobian matrix are: ' ∂f x ( k ) ∂f x ( k ) ∂f x (k ) 1 ( ∂x ∂y ∂θ 2 ( 2 '1 0 − s( k ) sin θ (k ) 1 ∂f y ( k ) ∂f y ( k ) ∂f y ( k ) ( 2 = (0 1 s ( k ) cos θ (k ) 2 ∇F = (31) ( ∂x 2 ∂y ∂θ 2 ( 0 0 1 ) 3 ( ∂f (k ) ∂f ( k ) ∂f ( k ) 2 θ θ ( θ 2 ∂y ∂θ 23 () ∂x The predicted error covariance matrix: Pˆ (k + 1 | k ) = ∇F Pˆ (k | k ) ∇F Τ + Q(k )
(32)
1 ' k R s (k ) cos 2 θ (k ) k R s( k ) sin θ ( k ) cos θ (k ) 0 2 ( Q (k ) = (k R s (k ) sin θ ( k ) cos θ ( k ) k R s (k ) sin 2 θ ( k ) 0 2 ( kθ α (k ) + k D s( k ) 23 0 0 )
(33)
Z (k + 1) = {z i (k + 1) | 1 ≤ i ≤ nS }
(34)
where
3. Observation. Obtain the next observation as given by:
4. Measurement Prediction. Determine the predicted measurement by Eq. 28 with the predicated state Xˆ (k + 1 | k ) in the form:
Map Building and Localization for Autonomous Mobile Robots
zˆi (k + 1)) = hi ( plj , Xˆ (k + 1 | k ))
179 179 (35)
5. Innovation. Calculate the difference between the observation and predicted measurement. For the ith sensor, the innovation vii (k + 1) is computed as: vii (k + 1) = z i (k + 1) − zˆi (k + 1)
(36)
6. Variance of the Innovation. Determine the variance given by the innovation. It is formed as: Τ S i (k + 1) = ∇hi Pˆ (k + 1 | k )∇hi + Ri (k + 1)
(37)
where Ri (k + 1) is variance of the sensor measurement, Τ
' ∂hi 1 ( ∂x 2 Τ ' 1 − cos acl ( ∂h 2 ( 2 ∇hi = ( i 2 = ρv − sin acl ( 2 ( ∂y 2 ( ∂hi 2 )( xns sin(θ (k + 1 | k ) − acl ) − yns cos(θ (k + 1 | k ) − acl )32 ( 2 ) ∂θ 3
(38)
Jacobian matrix ∇hi is derived from Eq. 28 by partial differentiation with respect to pose ( x(k + 1 | k ), y (k + 1 | k ),θ (k + 1 | k )) . 7. Validation Gate. v 2 (k + 1) vi (k + 1) S i−1 (k + 1)viΤ (k + 1) = i ≤ g2 (39) S i (k + 1) where g is acceptable different between the measurement and prediction. 8. Kalman Gain. After matching (accepted by the validation gate checking process), a number of measurements and observations are used to update the position of the robot. Then, the filter gain Wi (k + 1) is updated for the every accepted ith sensor measurement and observation: W (k + 1) = Pˆ (k + 1 | k )∇h Τ S −1 (k + 1) (40) i
i
i
9. State vector update At time k+1, the estimated robot pose can be obtained as follow:
Xˆ ( k + 1 | k + 1) = Xˆ ( k + 1 | k ) + # Wi ( k + 1)vi ( k + 1) i
Covariance update: Pˆ (k + 1 | k + 1) = Pˆ (k + 1 | k ) − # Wi ( k + 1) S i (k + 1)Wi Τ ( k + 1) i
10.Repeat step 2 with Xˆ (k + 1 | k + 1) .
(41) (42)
180180Y.L. Ip, A.B. Rad, and Y.K. Wong
5.2
Formulation of proposed FT-EKF
In the traditional adaptation schemes of EKF localization, it is suggested to adapt the process noise covariance Q (k ) with scaling the given priori Q (0) (Jetto et al. 1999a, 1999b). That means, the adaptation scheme is based on changing the scaling to get the updated Q(k + 1) , i.e. factor of Q(k ) Q(k + 1) = βQ (k ), where β is the scaling factor. A problem may arise (degrade the EKF performance) if poor estimated elements in matrix Q (0) (at start) are used. This is a distinct disadvantage in cases where it is difficult to determine the elements in process noise covariance matrix Q (k ) . A novel fuzzy rule based tuning scheme allows us to integrate the heuristic knowledge and simplifies the design knowledge of the matrix Q (k ) . In our proposed algorithm, a simple fuzzy model is employed. The strategy for tuning the parameters (or elements) in Q (k ) will be discussed in following section.
5.2.1 Assumption statement In conventional approaches (Jetto et al. 1999a; Meng et al. 2000), a number of the adaptation schemes of Kalman filter are designed according to covariance matching. The main idea is to make the filter residual covariance, E[vii (k + 1)vii (k + 1) Τ ] , consistent with its theoretical value (Mehra 1972). Note that Kalman gain W (k ) is influenced by both of process noise Q (k ) and measurement noise R (k ) . From Eq. 32, 37 and 40, it can be observed that the value of W (k ) can be increased by either increasing Q (k ) or decreasing R (k ) . We have used this fundamental concept in our algorithm. Now let us go back to basic concept of the Kalman filter adaptation scheme. When the innovation vii (k + 1) approaches zero, the value of Q (k ) (or Kalman gain W (k ) ) is appropriately changed. On the other hand, if the innovation vii (k + 1) is increased, it boosts up the value of Q (k ) in order to reduce the innovation. The main objective is to maintain the innovation process as small as possible. In our approach, three noise factors k R (k ) , kθ (k ) , and k D (k ) are tuned by fuzzy rules. The tuning strategy is based on the assumption of availability of a sufficiently accurate robot pose. The mean of innovation process is similar to the mean square root value of the variance of innovation at time k. It is stated as following:
λ (k + 1) ≈ 1
(43)
Map Building and Localization for Autonomous Mobile Robots
181 181
vi (k + 1)
where λ (k + 1) =
, for all matched values between observation and S i (k + 1) prediction measurement. The details of formulation of the fuzzy tuned EKF will be discussed in the next section.
5.2.2 Adaptive estimation of Q (k ) by fuzzy rule based tuning scheme Before we discuss the tuning condition for k R (k ) , kθ (k ) , and k D (k ) , two additional terms are introduced as follows:
δf xy ( k + 1) = (δf x ( k + 1)) 2 + (δf y ( k + 1)) 2
(44)
#W (k + 1)v(k + 1) , δf (k + 1) = #W (k + 1)v(k + 1) , and δ f ( k + 1) = # W ( k + 1) v ( k + 1) (45)
where δf x (k + 1) =
x
θ
y
y
θ
It can be observed from the above that the elements in the Kalman gain
W (k )
'W x ( k ) 1
can be stated as W ( k ) = (W y (k )2 . ( 2 ()Wθ ( k ) 23
The terms δf x (k + 1) , δf y (k + 1) and δf θ (k + 1) represent any significant change on X-axis, Y-axis and heading of robot, respectively. Similarly, the term δf xy (k + 1) denotes any significant change in distance traveled by the robot. The signal flow diagram and overall structure of the FT-EKF adoption scheme is shown in Fig. 8.
5.2.2.1
Fuzzy system for tuning k R (k )
For tuning k R (k ) , a three inputs and single output fuzzy system has been employed. The three inputs are λ (k ) , δf xy (k ) and v(k + 1) . When the λ (k ) at time k is small enough which implies the factor k R (k − 1) is good enough, we suggest either no change or decrease the factor of k R (k ) a little bit. This depends on the innovation process v(k + 1) at time k+1 and gain action on δf xy (k ) . If an appropriate value of k R (k − 1) is used at time k, the value of v(k + 1) and
δf xy (k ) should be small. Therefore, we suggest no change in k R (k ) . On the other hand, the k R (k ) should be reduced a little bit when a large gain action δf xy (k ) is obtained. According to a similar argument, the rule table for tuning k R (k ) can be
182182Y.L. Ip, A.B. Rad, and Y.K. Wong
formed as show in Table 1a. The output of this fuzzy system should be ∆k R (k ) , i.e. k R (k ) = k R (k − 1) + ∆k R (k ) . For connivance, the input and output parameters are normalized into 0 to 1 and –1 to 1, respectively. The corresponding input and output membership functions are shown in Fig. 9a and Fig. 9b, respectively. Control command
u (k ) = ( s, α ) given by
Navigation System or Human Operator Mobile Robot Platform Sonar sensor readings
Odometry
Z ( k + 1)
Prediction
NO
Matchin
Position Prediction
Update with the predicted position
g YES
Xˆ ( k + 1 k ) = F ( Xˆ (k | k ), u ( k )
Measurement Prediction
Update Innovation
zˆ i ( k + 1) = hi ( p , Xˆ ( k + 1 | k )) l
v ii ( k + 1 ) = z i ( k + 1 ) − zˆ i ( k + 1 )
Covariance Prediction
Kalman gain update
Τ Pˆ(k + 1 | k) = ∇F Pˆ(k | k ) ∇F + Q(k )
Variance of innovation Prediction Si ( k + 1) = ∇hi P(k + 1 | k )∇hi
Τ
T −1 W i ( k + 1) = Pˆ ( k + 1 | k ) ∇ h i S i ( k + 1
Position update
Xˆ ( k + 1 | k + 1) = Xˆ ( k + 1 | k ) +
+ Ri (k + 1)
# W ( k + 1)v ( k + 1 i
i
i
Covariance update
Pˆ(k +1 | k +1) = Pˆ(k +1 | k)
T − #Wi (k +1)Si (k +1)Wi (k +1) i
MAP (priori information)
Past information v (k ),
S ( k ) , W ( k ), u ( k − 1)
Fuzzy rule based tuning system State noise model update Q ( k + 1) Noise model estimation scheme
Fig. 8. Overall structure of FT-EKF.
Map Building and Localization for Autonomous Mobile Robots
5.2.2.2
183 183
Fuzzy system for tuning kθ (k ) and k D (k )
Based on a similar reasoning for tuning in k R (k ) , the fuzzy system for tuning
kθ (k ) and k D (k ) is designed. However, the input δf θ (k ) is used instead of δf xy (k ) . Otherwise, the tuning strategy is the same as k R (k ) . Also note that the output fuzzy subsets are more since the control gain related in heading should be more precise. If a wrong kθ (k ) and k D (k ) are used, the EKF diverge. The same rule base is used in both of factors kθ (k ) and k D (k ) tuning but different scaling factors are applied. The rule table is shown in Table 1b. The output of this fuzzy system should be ∆kθ (k ) and ∆k D (k ) , i.e. kθ (k ) = kθ (k − 1) + ∆kθ (k ) and k D (k ) = k D (k − 1) + ∆k D (k ) . Another important factor is the sequence to update kθ (k ) and k D (k ) . We assign a weighting factor according to the control command u (k − 1) = [ s (k − 1),α (k − 1)]Τ . We then design a two inputs and one output fuzzy system. Since the factor kθ (k ) is related to turning α (k − 1) and the factor
k D (k ) is related to traveling s (k − 1) , the output of this fuzzy system is the so-called weighting factor β , where 0 ≤ β ≤ 1 and the corresponding rule table is shown on Table 1c. Now the weighted output is ∆kθ (k )′ and ∆k D (k )′ . The formulation of those factors are shown as follow:
∆kθ (k )′ = (1 − β )∆kθ (k )
(46)
∆k D (k )′ = β∆k D (k )
(47)
The corresponding input and output membership functions are shown in Fig. 9a and Fig. 9c and 9d, respectively. Table 1. Fuzzy rule table corresponding to a) the output ∆k R ( k ) , b) the output ∆kθ ( k ) and ∆k D (k ) , c) the output β .
λ (k ) : Z / M / L v(k + 1)
Z M L
δf xy (k ) Z Z/Z/Z NMS / PS / PS NMS / PS / PS
M Z / Z / PS NS / Z / PMS NMS / Z / PMS
L NS / Z / Z NS / Z / Z NS / NS / Z
(a)
λ (k ) : Z / M / L v(k + 1)
Z M L
δf θ ( k ) Z
M
Z/Z/Z Z / PS1 / PS3 Z / PS2 / PS3
Z / NS1 / PS1 Z / NS1 / PS1 NS2 / NS2 / PS2
(b)
L
NS1 / NS1 / NS2 NS2 / NS2 / NS3 NS3 / NS2 / NS3
184184Y.L. Ip, A.B. Rad, and Y.K. Wong
s (k − 1)
α (k − 1)
Z M S Z
Z M L
M L M S
L VL L M
(c)
(a)
(b)
(c)
(d)
Fig. 9. a) Input membership functions corresponding to λ ( k ) , v( k + 1) , δf xy (k ) , δf θ (k ) , s (k − 1) and α (k − 1) . Output consequences corresponding to b) ∆k R (k ) , c) ∆kθ ( k ) and ∆k D ( k ) and d) β .
Map Building and Localization for Autonomous Mobile Robots
6
185 185
Experimental Study
In order to evaluate the performance of the proposed dynamic segment-based map building algorithm and FT-EKF localization algorithm, we carried out two experiments with a Pioneer 2-DX mobile robot in indoor campus environment. Pioneer 2-DX mobile robot equipped with a ring of 16 Polaroid ultrasonic sonar sensors and an on-board 20 MHz Siemens 88C166 microprocessor with integrated 32K flash-Rom. The driving mechanism of the robot consists of one castor wheel and two DC reversible motors with wheel encoders. 6.1
Experiment 1
For the first experimental study, the corridor outside the Control Research laboratory was modeled. During the experiment in the corridor, there was one moving object and the doors of two of the offices were open. Figure 10 shows the actual image of the corridor. The total trajectory of the robot was round 8m, lasting about 3.5 minutes, with an average speed of 50mm/s. The on-line segment-based map building result is shown in Figure 11. In this experiment, the dynamic object moved in three different positions (position 1 to position 2 to position 3) which is shown in Figure 10b. In Figure 11a and Figure 11b, the object is placed at position 1. After that, we move the object from position 1 to position 2 as shown in Figures 11b and 11c, respectively. Finally, we moved the object from position 2 to position 3. The Pioneer 2DX was made to perform a “U turn” at position 2. In Figure 11e, we found that the object in position 2 was disappearing when we compared the map with Figure 11d. In Figure 11f, the Pioneer 2 returned to the starting point. It could be observed that the object in position 1 was also disappearing. From this experiment, it was concluded that the proposed on-line segment-based map-building algorithm could build a map in a dynamic environment. Note, when the object is moving, the Pioneer 2DX is stopped until the object is reached to its defined position.
(a)
(b)
Fig. 10. a) Photo of experimental site (corridor), b) 2-D floor plan.
186186Y.L. Ip, A.B. Rad, and Y.K. Wong
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 11. Online mapping during exploration with Pioneer 2DX: a) to f)
6.2
Experiment 2
In the second experiment, the proposed localization algorithm was tested in an indoor environment containing a room and a corridor. The size of area was about (11m × 7.3m). The room was a research office that included some partitions, rubbish bins, desks, and chairs, cabinet and access doors. The layout of the navigation area and different images of the environment are shown in Figure 12. In this experiment, only the office partitions, desks, walls and doors were modeled by several line segments (see Figure 12b). In order to compare the proposed FT-EKF localization algorithm, odometric measurement and conventional EKF localization approach are performed simultaneously. The validation gate of both model-based localization algorithm (convetional EKF and FT-EKF) is 5 (i.e. g = 5). The result trajectories are shown in Figure 13. The results indicate that both EKF and FT-EKF localization algorithms estimate the actual trajectory satisfactory in spite of the failure of odometric measurement. Average of innovation of the EKF localization and FT-EKF localization over all time were equal to 55.301mm and 43.83mm, respectively. The final error is shown in Table 2. It can be deduced that FT-EKF algorithm has smaller final error than that of EKF localization algorithm.
Map Building and Localization for Autonomous Mobile Robots
Box Close
Cabinet Close
Open
Rubbish Bin
Research Office Partitions
187 187
Distilled Water
Desk Chair
Corridor
Close
End Point
Close Close 4m
Close
Box
2m
Approximation Route 2m
4m
Close Starting Point
(a)
(b)
(c) Fig. 12. Experimental environment. a) Floor Plan. b) Modeled Segment. c) Actual environment.
Table 2. Final errors of experiment with Pioneer 2DX (real robot)
∆x (m) ∆y (m)
EKF -0.81 -0.0183
FT-EKF -0.285 -0.014
188188Y.L. Ip, A.B. Rad, and Y.K. Wong
Fig. 13. Result trajectories of experiment with real robot.
7
Conclusions
In this chapter, the important problems of map building and localization were discussed and some new algorithms were proposed. These ideas were tested with simulation studies as well as experimental verification. The next phase of this work include the SLAM problem. The algorithms reported here can be extended for other perception devices.
Map Building and Localization for Autonomous Mobile Robots
189 189
References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.
16.
Berger JO (1985) Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, Berlin. Second Edition. Bezdek JC (1981) Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum, New York Chow KM, Rad AB and Ip YL (2002) Enhancement of Probabilistic Grid-based Map for Mobile Robot Applications. Journal of Intelligent and Robotic Systems 34: pp.155–174 Dave RN (1989) Use of adaptive fuzzy clustering algorithm to detect lines in digital images. Intelligent Robots and Computer Vision VIII, Vol. 1192(2), pp 600-611 Dave RN (1991) Characterization and detection of noise in clustering. Pattern Recognition Letters 12, pp 657-664 Ip YL, Chow KM, Rad AB and Wong YK (2002) Segment-based map building using Enhanced Adaptive Fuzzy Clustering Algorithm for mobile robot applications. Journal of Intelligent and Robotic Systems 35, pp.221–245 Jetto L, Longhi S and Venturini G (1999a) Development and experimental validation of an adaptive extended Kalman filter for localization of mobile robots. IEEE Transactions on Robotics and Automation, vol 15, No. 2, pp 219-229 Jetto L, Longhi S. and Venturini G. (1999b) Localization of a wheeled mobile robot by sensor data fusion based on a fuzzy logic adapted Kalman filter. Control Engineering Practice, 7, pp 763-771 Konolige K (2001) Robot Notes (Robot Motion: Probabilistics Model; Sampling and Gaussian Implementations; Markov Localization). SRI International (http://www.ai.sri.com/~konolige) Mehra RK (1972) Approaches to adaptive filtering. IEEE Transaction Automatic Control, vol 17, Issue: 5 pp 693-698 Meng QH, Sun YC and Cao ZL (2000) Adaptive extended Kalman filter (AEKF)-based mobile robot localization using sonar. Robotica, vol 18, pp 450-473 Phansalkar G and Dave RN (1997) On Generating Solid Models of Mechanical Parts through Fuzzy Clustering, Proceedings of the Sixth IEEE International Conference on Fuzzy Systems, Barcelona, Spain, vol 1, pp 225 -230 Pioneer 2 Mobile Robot Operation Manual. May (2001) ActivMedia Robotics, LLC, v7b Runkler TA, Palm RH (1996) Identification of Nonlinear Systems Using regular fuzzy c-Elliptotype Clustering. Proceedings of the Fifth IEEE International Conference on Fuzzy Systems, New Orleans, Louisiana, vol 2, pp 1026 -1030 Thrun S, Fox D, Burgard W and Dellaert F (2001) Robust Monte Carlo localization for mobile robots. Artificial Intelligence, 128, pp 99-141 Wang LX (1997) A course in fuzzy systems and control. Prentice-Hall International, Inc.
190
Control and Stabilization of the Inverted Pendulum via Vertical Forces D. Maravall Department of Artificial Intelligence Faculty of Computer Science, Universidad Politécnica de Madrid Campus de Montegancedo, 28660 Madrid, Spain [email protected] Abstract. In this chapter, we present a detailed analysis of the possibilities of controlling and stabilizing the inverted pendulum (IP) by means of a vertical force. First, we establish the dynamic equations of the IP under the action of a generic vertical force and then we analyze its control and stabilization. The main conclusion is that the vertical force has an excellent stabilization effect, although it requires a permanent fall of the IP support base when it is the only applied force. Therefore, we investigate the combination of the vertical force with the customary horizontal force, arriving at the stabilization conditions for different formal representations of the system: ordinary differential equations, state variable representation and Liapunov´s direct and indirect methods.
1 Introduction The inverted pendulum (IP) is a widely studied dynamic system, which has received considerable attention in many fields, such as physics, mechanics, applied mathematics, control theory, and the emergent computational techniques known as soft computing [1]-[3]. There are several reasons behind such interest, in particular the importance and ubiquity of the IP in many mechanisms, including robots. Furthermore, its intrinsic theoretical interest and the strong challenges posed by its stabilization and control have made the IP a sort of benchmark, in particular for the comparison of soft computing (artificial neural networks, fuzzy logic, genetic algorithms) and hard computing (ordinary differential equations, input-output control, state variable techniques, like optimal control or Liapunov´s stability). Apart from being a benchmark in control engineering, the IP problem has always been a testbed for computational intelligence theories and models, as it embraces the customary sense-reason-action cycle, typical of intelligent systems [4] and [5]. Furthermore, controlling a pendulum in its unstable top position is not only an interesting physico-mathematical problem with a difficult engineering implementation, as it involves high nonlinearities and fast sensory information procT.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 190−211, 2004. Springer-Verlag Berlin Heidelberg 2004
Control and Stabilization of the Inverted Pendulum
191 191
essing, but it also, and very importantly, poses a strong challenge to any intelligent artificial agent, as it demands the coordination of perception and action and, even, some degree of reasoning. Thus, it is no wonder that IP stabilization has been attempted by means of robotic manipulators and specialized sensors, [6]- [9]. The bibliography on IP is literally overwhelming. However, it is rather surprising that virtually all the technical literature refers to the planar pendulum with one degree of freedom. Only very recently have a few references dealing with the spherical pendulum with two degrees of freedom appeared [9]-[11]. Due to the complex control problems involved, [9] addresses the stabilization of the spherical IP by simultaneously controlling two uncoupled planar pendula (the respective projections on the two orthogonal planes of the intertial coordinate system). References [10] and [11] apply the method of controlled Lagrangians to get theoretical stability conditions for the spherical IP. In practice, the only control action used in the technical literature for IP stabilization is a horizontal force, which is almost universally materialized by means of an electrical cart, i.e., the popular cart-pole system shown in Figure 1.
Fig. 1. The customary cart-pole system.
Note, in Figure 1, that the pendulum is constrained to move within the XOY plane. As mentioned above, the control action is based on the horizontal displacements of the electrical cart. Exceptionally, some authors have considered an alternative control action consisting of an oscillatory vertical force applied to the pendulum pivot. The stabilizing effect of a fast vertical oscillation applied to the pendulum base is known from the early work of Stephenson in 1908 [12] . The Russian physicist Kapitsa was the first, in the fifties, to produce a rigorous demonstration of the stability conditions
192192D. Maravall
of the IP when its suspension base oscillates at a high frequency [13] and, therefore, some authors use the expression Kapitsa pendulum to refer to this stabilization technique [14]. Another control alternative is based on the application of a rotational torque to the pendulum base, as proposed by Furuta and co-workers [15]. In fact, this arrangement leads to a different kind of planar IP, known as the rotational IP [16] or, simply, the Furuta pendulum [17]. With the exception of vibrational control –i.e., based on oscillatory control signals- which is a well-known technique for controlling mechanical systems [18][20], including, as mentioned above, the IP [21]-[23], the only previous work, to our knowledge, that considers the application of vertical forces to stabilize the IP was recently performed by Wu et al. [24] and [25], who employ the IP as a basic element to analyze the postural stability and locomotion of multi-link bipeds. In particular, Wu et al. model the IP base point according to cartilage and ligament behavior in natural joints and they apply horizontal and vertical forces and, also, a rotational torque to the base pivot. Using a very simplified linear model, the resulting overactuated control system is designed by means of Liapunov´s direct method to obtain a desired trajectory of the IP´s center of gravity. In this chapter, we conduct a detailed analysis of the possibilities of controlling and stabilizing the IP by means of a vertical force. The chapter is organized as follows. First, we establish the dynamic equations of the IP under the action of a generic vertical force and then we analyze its control and stabilization. The main conclusion is that the vertical force has an excellent stabilization effect, although it requires a permanent fall of the IP support base when it is the only applied force, which is, obviously, an unfeasible control policy. Therefore, we investigate the combination of the vertical force with the customary horizontal force, arriving at the stabilization conditions for different formal representations of the system: ordinary differential equations, state variable representation and Liapunov´s direct and indirect methods. In particular, we obtain the stability conditions of the IP for a PD control algorithm. An appendix discusses some experimental results, including notable improvements in IP stabilization achieved when combining the customary horizontal force with a vertical force.
2 Stabilization of the Inverted Pendulum with a Vertical Force In Figure 2 we have substituted the customary electrical cart by a platform of mass M, on which the pendulum pivot is mounted. The pendulum has a total mass m and length 2l. Apart from the gravitational force, the only existing external force is purely vertical, Fy The dynamic equations of the system can be straightforwardly obtained by applying Lagrange´s equations
Control and Stabilization of the Inverted Pendulum
193 193
Fig. 2. Planar inverted pendulum supported by a platform subject to a pure vertical force Fy
d $ ∂L * − ∂L = F i; dt %& ∂q#i +, ∂qi
i = 1,2
(1)
L=K−P where L is the Lagrangian, K is the kinetic energy and P the potential energy of the system. Fi stands for the generalized applied forces and qi are the generalized coordinates, which in this case are y and , respectively. As mentioned above, the only existing force is F1 = Fy. !
The kinetic energy is
K = 1 m ( x# p 2 + y# p 2 ) + 1 I θ# 2 + 1 M y# 2 2 2 2
(2)
where I is the inertia of the pendulum, which we will assume to be negligible from now on. The coordinates of the pendulum´s center of mass are
x p = l sinθ ; y p = y + l cosθ
(3)
where y is the vertical coordinate of the platform’s center of gravity, which we assume to coincide with the pendulum hinge. As a consequence, we can also assume that the pendulum mass is virtually concentrated at its top. The potential energy is
P = M g y + m g y p = ( M + m) g y + m g l cosθ
(4)
After some operation, the Lagrangian turns out to be
L = 1 m l 2 θ# 2 + 1 ( M + m ) y# 2 − m l sinθ y# θ# − ( M + m) g y − m g l cosθ 2 2
(5)
194194D. Maravall
By substituting into the Lagrange’s equations, we finally get the equations of the system dynamics
( M + m) ##y − m l sinθ θ## − m l cosθ θ# 2 = Fy − ( M + m) g
(6)
−m l sinθ ## y + m l θ## − m g l sinθ = 0 2
which can be expressed in the standard compact form
M (q) q## + C (q,q# ) q# + G (q ) = τ
(7)
where M(q) is the symmetric, definite positive inertia matrix, C(q,q) is the Coriolis/centripetal matrix and G(q) is the gravity vector
' (M + m) − m l sinθ M (q ) = ( m l 2 /. )− m l sinθ '0 − m l cosθ θ#' (M + m) g C (q,q# ) = ( . ; G (q ) = ( . 0 )− m g l sinθ / )0 / 'F τ = ( y. )0/ ' yq=( . )θ /
;
(8)
It is interesting to compare these equations with the equations for the usual case in which a pure horizontal force, Fx , is applied, as illustrated in Figure 1
( M + m) ##x + m l cosθ θ## − m l sinθ θ# 2 = Fx m l cosθ ## x + m l 2 θ## − m g l sinθ = 0
(9)
In spite of the apparent similarity of formulae (6) and (9), the stabilization of the IP in each case is totally different, as shown in the sequel. As a first step in the analysis of this nonlinear system, let us focus on the situation of practical interest, namely, small IP movements around the unstable openloop position θ = 0, where we can introduce the following approximations sin θ ≈ θ
;
cos θ ≈ 1 − θ 2 / 2
which, substituted into the Lagrangian function (4), yields
(10)
Control and Stabilization of the Inverted Pendulum
L = 1 m l 2 θ# 2 + 1 ( M + m ) y# 2 − m l θ θ# y# − ( M + m) g y − m g l (1 − θ 2 /2) 2 2
195 195 (11)
Then, from the Lagrange’s equations, we get the equation of small movements
( M + m) ##y − m l θ θ## − m l cosθ θ# 2 = Fy − ( M + m) g −θ ## y + l θ## − g θ = 0
(12)
Similarly, for the horizontal force, we get
( M + m) ##x + m l θ## = Fx ## x + l θ## − g θ = 0
(13)
Unlike the dynamic equations of the horizontal case, which are totally linear, the equations of the small IP movements are still nonlinear in the vertical case, making the stability analysis and the control comparatively more difficult. By solving the simultaneous differential equations of the horizontal force in θ, we get
F θ## − M + m g θ = − x Ml
Ml
(14)
The respective characteristic equation yields hyperbolic sines and cosines and, therefore, the pendulum dynamics is unstable. However, the horizontal closed-loop dynamics can be straightforwardly stabilized [26] by introducing a feedback control law, such as a conventional PD algorithm
Fx = k pθ + k d θ#
(15)
Unfortunately, this is not the case for the vertical force, as we then have an autonomous, unforced pendulum dynamics
l θ## − ( g + ## y) θ = 0
(16)
unless we consider vertical acceleration as an external control action. Specifically, we observe that this equation is unstable if the following condition holds
196196D. Maravall
(g + #y#) > 0
(17)
as unbounded hyperbolic solutions are obtained in . On the contrary, if (g+ÿ) < 0, then stable oscillating solutions are obtained, as the characteristic equation has the following roots !
r2 +
g + ## y = 0 → r1,2 = ± j l
g + ## y l
(18)
so that
θ (t) = A cos [ω(t) ⋅ t + ϕ ]
(19)
where the angular frequency (t) is time dependent, due to the variable vertical force. Therefore, when the stable oscillatory condition ( g + ÿ ) < 0 holds, the pendulum oscillates with a constant amplitude and a slowly varying frequency given by !
A = θ 2 (0) +
θ# 2 (0) ω 2 (0)
;
f (t ) = 1 2π
' −θ#(0) . ) ω(0) ⋅ θ (0) /
(20)
ϕ = tan −1 ( g + ## y l
The greater the vertical acceleration, the higher the frequency of oscillation. As for the oscillation amplitude, given the usual condition ’(0) / (0) << (0) , it is virtually equal to the initial deviation (0). !
!
!
!
As a first conclusion regarding the pure vertical force, we can guarantee a stable oscillating behavior of the IP, if and only if the applied vertical force, Fy, produces a negative vertical acceleration such that ## y < − g . As the stability of the IP depends on the vertical acceleration, we must proceed with the analysis of the vertical dynamics, which for small deviation angles of the IP turns out to be
( M + m ) ##y − m l θ θ## = Fy − ( M + m ) g
(21)
By substituting the vertical acceleration given by (21) into the pendulum dynamics, one obtains
Control and Stabilization of the Inverted Pendulum
'
θ## − (
Fy
) ( M + m )l
+
m θ θ##- θ = 0 . M +m /
197 197 (22)
Then, the vertical force that guarantees oscillatory stability of the IP is
Fy < − m l θ θ##
(23)
Assuming that the deviation angle has been stabilized by a vertical force satisfying condition (23), then and its second derivative are sinusoids of opposite sign, so that the upper limit of the stabilizing vertical force is always positive !
Fy < m l θ θ##
(24)
Now, let us suppose, conservatively, that Fy = 0, i.e., the platform-pendulum pair is left in free-fall. In this case, the stability condition holds
m l θ θ## < 0 M +m
( g + ## y) =
(25)
and the respective small oscillations of θ will be of very low frequency
f (t) = 1 2π
θ θ## M +m
(26)
with an amplitude given by (20). Summarizing, the IP can be stabilized by means of the free-fall of the platformpendulum pair, which is obviously an impractical control strategy. For such reason, we need to investigate whether it is possible, at some point after the stabilization of the IP, to stop the platform falling and, even, to raise it to its original position. More specifically, let us introduce a positive vertical force, Fy = (M + m) g, just to balance the force of gravity, in which case the platform stops falling. Thus, the dynamics of the vertical force is now
## y=
m l θ θ## M +m
(27)
which, substituted into the pendulum dynamics given by (16), yields the following condition for maintaining the small oscillations of θ around the vertical position
ml ## > g M +m θθ
(28)
198198D. Maravall
Condition (28) is obviously violated for realistic values of the parameters M, m and l and provided that we are, precisely, hypothesizing small values of the pendulum’s deviation angle and its successive derivatives,. Consequently, the platformpendulum pair would have to remain in free-fall in order to maintain the stable initial oscillations. Our preceding discussion of the IP stability conditions has been based on rather qualitative reasoning. So, let us now take a more rigorous approach by analyzing the Mathieu equation that determines the pendulum dynamics –see expression (16)-. Thus, once the inverted pendulum has been stabilized by applying a negative vertical force, the deviation angle of the pendulum and its two first derivatives are
θ (t ) = A cos(ω t + ϕ ) ; θ#(t) = − ω A sin(ω t + ϕ )
(29)
θ##(t) = − ω A cos(ω t + ϕ ) 2
which, substituted into the vertical dynamics, yields the vertical acceleration
## y=
m l ω 2 A2 'sin 2 (ω t + ϕ ) − cos2 (ω t + ϕ ) - + Fy − g ) / M +m M +m
(30)
whence the pendulum dynamics turns out to be
θ## + '() (
− Fy + m ω 2 A2 cos2(ω t + ϕ ) .θ = 0 / M + m) l M + m
(31)
which is, as mentioned above, the well-known Mathieu equation (32)
## x + (δ + ε cos(ω t ) ) x = 0
The Mathieu equation, in particular its stability conditions, has been extensively analyzed [1] and [27]. The analytical stability conditions of the Mathieu equation confirm the conclusions that we drew from our qualitative discussion, namely, that the IP stabilization with a vertical force – i.e., Fy > 0 and < 0 – is impossible, as virtually all the corresponding negative half-plane encompasses instability regions of the Mathieu equation. Another interesting result derived from the analytical study of equation (32), not directly observed in the qualitative discussion, is that, even for negative vertical forces –i.e., Fy < 0 and > 0 -, there are instability regions that depend on the physical parameters Fy, M, m, l, (t) and (0) that must be carefully analyzed. Given the complex interactions of these parameters and their influence on the instability regions, the most advisable design strategy is to choose as high as possible a and, afterwards, check an that does not drive the system to an instability region. !
!
!
!
!
"
Control and Stabilization of the Inverted Pendulum
199 199
As a general conclusion, the application of a single, sustained vertical force to stabilize and control the IP is unfeasible, although its excellent and, in particular, its fast stabilization effect makes the combination of the vertical force with the customary horizontal force looks very attractive. Therefore, our next and central topic is the stabilization of the IP via the combination of the vertical force with the customary horizontal force.
3 Combination of Horizontal and Vertical Forces After having investigated the stabilization of the IP by means of a vertical force, we are now going to explore its combination with the usual horizontal force. The mechanism for implementing the vertical force, Fy, is a platform of mass m’, mounted on the customary electrical cart, which, as usual, produces the horizontal force, Fx. The total kinetic energy of the cart-platform-pendulum ensemble is
K = 1 M x# 2 + 1 m′( x# 2 + y# 2 ) + 1 m ( x# p 2 + y# p 2 ) 2 2 2
(33)
and the potential energy
P = m′ g y + m g y p
(34)
By applying Lagrange´s equations
d $ ∂L * ∂L +− % = Fi dt %& ∂q#i +, ∂qi
(35)
q1 = x, F1 = Fx ; q2 = y , F2 = Fy ; q3 = θ , F3 = 0 we get the global system dynamics
( M + m′ + m ) ##x + m l cosθ θ## − m l sinθ θ# 2 = Fx ( m + m′) ##y − m l sinθ θ## − m l cosθ θ# 2 = Fy − ( m + m′) g
(36)
cosθ ## x − sinθ ## y + l θ## − g sinθ = 0 To analyze this highly nonlinear system, let us first make the following qualitative remarks.
200200D. Maravall
1. From the study of the pure vertical force case, we know that the only exogenous control action is vertical acceleration, which leads to the unfeasible stabilization strategy of maintaining the platform in free-fall. 2. On the contrary, the pure horizontal force can be used as a feedback control action that straightforwardly stabilizes the IP. 3. Due to the equivalence of the joint (x, ) dynamics of the combined case given by (36) and the pure horizontal case, equation (9), we can, in principle, exploit the feedback stabilization capacity of force Fx by focusing on the joint (x, ) dynamics of the horizontal plus the vertical case and considering vertical acceleration as an external control action. !
!
Thus, let us rewrite the joint (x, ) dynamics of the combined case !
( M + m′ + m ) ##x + m l cosθ θ## − m l sinθ θ# 2 = Fx
(37)
cosθ ## x + l θ## − ( g + ## y ) sinθ = 0 which is equivalent to the pure horizontal case –see equations (9)-, except that the gravity dynamics is perturbed by the term ÿ sin . Remember that this perturbation is generated by the vertical dynamics, given by the second equation of (36). Thus, by considering vertical acceleration as an exogenous element in the pendulum dynamics, the combined forces case turns out to have the same formal structure as the horizontal force case. Therefore, we can tackle the combined case as an ordinary differential equation (ODE) problem and stabilize the IP via the horizontal force Fx using any standard control law, as in [26]. Alternatively, we can also approach the control problem with the state variable representation and stabilize the IP with a plethora of available techniques, including Liapunov’s direct and indirect methods. Let us begin our study with the ODE approach, which conveys a very intuitive and direct physical interpretation. !
3.1 Ordinary Differential Equations Analysis
As usual, we are interested in the neighborhood of the IP vertical position, so that we linearize the system dynamics (38) by approximating sin θ ≈ θ , cos θ ≈ 1 − θ 2 / 2 , θ# 2 ≈ 0 . After solving the ODE system in , we get !
θ## − M + m′ + m ( g + ##y ) θ = −
( M + m′ ) l
Fx ( M + m′ ) l
(38)
which is virtually equivalent to the pure horizontal case
F θ## − M + m g θ = − x Ml
Ml
(39)
Control and Stabilization of the Inverted Pendulum
201 201
Thus, let us apply a PD control law of the error variable
Fx = −k p e − kd e# = k p θ + kd θ#
(40)
Which, substituted into (38), yields
θ## +
kd # ' k p M + m′ + m − θ + )( ( g + ##y ) /.- θ = 0 Ml Ml Ml
(41)
Again, it is equivalent to the close-loop horizontal force case, whose ODE [26] is
θ## +
kd # ' k p − M + m g.θ = 0 θ + )( Ml Ml Ml /
(42)
the only remarkable effect of the vertical acceleration being on the root locus of the combined forces case. As is well-known, both coefficients must be positive to guarantee stability of (41)
kd > 0 → kd > 0 Ml
;
y )) k p > ( M + m′ + m ( g + ##
(43)
The first condition has a straightforward interpretation; namely, it implies that the feedback control force, Fx, must have a component directly proportional to the pendulum angular speed . To illustrate this fact, the four possible states of the IP have been represented in Figure 5. Note that in cases (b) and (c) the pendulum is returning to its vertical position, while in cases (a) and (d) it is moving away from it. In all cases, the orientation of the corresponding control force has been indicated. !
(a)
(b)
(c)
(d)
Fig. 3. The four possible IP states. Observe the orientation of the respective feedback force.
The second stabilization condition in (43) is even more intuitive, as the feedback force must always have the same orientation as the IP’s angular displacement.
202202D. Maravall
Furthermore, the gain kp should guarantee the positiveness of the respective coefficient. Note that when the platform is falling, the vertical acceleration ÿ strengthens IP stabilization. Inversely, when the platform is ascending to recover its original position, the respective positive vertical acceleration detracts from IP stabilization. Therefore, the vertical force component –i.e., the generation of vertical acceleration- must be carefully designed to tackle with this double-sided effect. Roughly speaking, when the pendulum is moving away from the vertical position, the vertical force should be immediately activated to produce a strong negative vertical acceleration . Inversely, with the pendulum recovering its vertical position, we can make Fy > 0 to bring the platform towards its original position. In short, the vertical displacement of the platform must be synchronized with the IP movements. This general control strategy can be succintly formalized as follows
If [sgn(θ ) = sgn(θ# ) ] then ') Fy < 0 → ## y < 0 -/ else ') Fy > 0 → ## y > 0 -/
(44)
Apart from controlling the IP’s deviation angle, which is obviously the main goal, it is also of interest to minimize the platform displacement, which must be constrained to some specific range. To this end, let us distinguish the following three states of the cart-platform-pendulum ensemble. 1. The IP is moving away from the vertical position – i.e., sgn (θ ) = sgn (θ# ) . 2. The IP is returning to the vertical position –i.e., sgn (θ ) ≠ sgn (θ# ) - but is still far from it. 3. As in state 2, but near the vertical position. Note the fuzzy linguistic qualifiers introduced to make a further distinction in the basic IP state returning to the vertical position. Accordingly, we introduce the following control action for each IP state. 1. The pendulum is leaving the vertical position
Fy ( t ) = − )' k pθ θ ( t ) + kdθ θ# ( t ) /-
(45)
2. The pendulum is returning to and is far from the vertical position
Fy ( t ) = − p ') k pθ θ ( t ) + kdθ θ#( t ) -/ − (1 − p ) ') k py y( t ) + kdy y# ( t )-/
(46)
where 0 < p < 1 weights the importance of the two control objectives: the IP deviation angle, , and the platform movement, y. Note that the latter action is aimed at minimizing the vertical displacement. !
Control and Stabilization of the Inverted Pendulum
203 203
3. The pendulum is returning to and is near to the vertical position
Fy ( t ) = − ') k py y ( t ) + kdy y# ( t ) -/
(47)
We must remember that in our preceding discussion the vertical acceleration of the platform-pendulum couple is given by
( m′ + m ) ##y − m l sinθ θ## − m l cosθ θ# 2 = Fy − ( m′ + m ) g
(48)
so that the high nonlinearity of the vertical dynamics must be taken into account in the tuning of the control parameters appearing in formulae (45)-(47). Summarizing the basic philosophy of our combined forces control, the dynamics of the IP is directly controlled by the customary horizontal force, plus the indirect action of a vertical acceleration, ÿ, which, in turn, is controlled by the vertical force given by formulae (45)-(47). Continuing with our qualitative and general discussion of the combined forces control, we are now going to proceed with the stabilization of the IP under the state variable representation.
3.2 State Variable Representation
Although basically similar to the ODE analyis, the state variable representation can be used to extend the possibilities of IP stabilization. In particular, as shown in the sequel, we shall refine the stability conditions by means of Liapunov’s direct method. Following our proposed stabilization strategy, which basically involves controlling the joint (x, ) dynamics by means of the customary horizontal force and, simultaneously, by the vertical acceleration of the platform, let us introduce the following variables change in the pendulum dynamics given by (41) !
x1 = θ , x2 = θ#
(49)
to get the state variable representation. Thus, after some operation, we obtain
x#1 = x2
(50)
204204D. Maravall
( M + m′ + m ) ( g + ##y ) sin x1 − m l sin x1 cos x1 x22 − Fx cos x1 x# 2 = θ## = ( M + m′ ) l + m l sin 2 x1 which is of the more compact form
x#1 = f1 ( x1 , x2 )
x#2 = f 2 ( x1 , x2 )
;
(51)
Liapunov’s indirect method, also known as the first method, approximates the nonlinear dynamics by the first, linear terms of the Taylor development around a certain equilibrium point ( x1eq , x2eq )
x#1 ≈ x# 2 ≈
∂f1 (x1 − x1eq ) + ∂f1 ∂x1 eq ∂x2
eq
∂f 2 ∂x 2
eq
∂f 2 ∂x1
(x
1
eq
− x1eq ) +
(x
2
(x
2
− x2 eq )
(52)
− x2 eq )
In vector form
x# ≈ J eq ( x − xeq )
(53)
where Jeq is the system’s Jacobian particularized into the equilibrium point of interest, in our case, ( x1, x2 ) = (θ ,θ# ) = ( 0,0 ) . After some operation, we get for this equilibrium point
1 ' J eq = ( ( M + m′ + m ) − ∂Fx / ∂x1 ( ( M + m′) l )
0
. F ∂ 1 x − . ( M + m′) l ∂x2 /
(54)
The characteristic equation, det ( λ I − J eq ) = 0 , yields
λ2 −
∂F ∂Fx 1 1 '( M + m′ + m ) ( g + ## y) − x - λ + =0 ∂x1 ./ ( M + m′) l () ( M + m′) l ∂x2
(55)
To guarantee stability, both coefficients must be positive
∂Fx ∂Fx ≡ > 0 # Fx = kd θ# ; kd > 0 ∂x2 ∂θ#
(56)
205 205
Control and Stabilization of the Inverted Pendulum
which coincides with the respective ODE stability condition (43). Additionally, ∂Fx ∂Fx y) ≡ > ( M + m′ + m ) ( g + ## ∂x1 ∂θ
(57)
also coinciding with the second ODE’s stability constraint, provided that we apply a PD control law such that F = k p θ + kd θ# . Both the linearized ODE and Liapunov’s indirect method only guarantee the local stability of the system. Liapunov’s direct method, also known as the second method, is stronger as it provides the global stability conditions. Thus, as for any mechanical system, let us try a Liapunov function based on the total energy of the system
1 1 2 V (x1 , x2 ) = θ# 2 + (1 − cos θ ) = x2 + (1 − cos x1 ) 2 2
(58)
which is definite positive for − π < θ < π , that determines a region beyond the practical interest of IP stabilization. Its first derivative is
V# ( x1, x2 ) = x2 x#2 + sin x1 x#1 = x#1 ( x#2 + sin x1 ) = θ# (θ## + sinθ )
(59)
In order to get the global stability of the equilibrium point of interest ( x1, x2 ) = (θ ,θ# ) = ( 0,0 ) , we must guarantee that V# ( x1 , x2 ) is definite negative in a region comprising the equilibrium point, so that
If (θ# < 0 ) then ')θ## + sinθ > 0-/ else ')θ## + sinθ < 0-/ ∀θ / − π < θ < π ; ∀θ ≠ 0
(60)
Thus, solving θ## in the global nonlinear system given by (37) yields
θ## =
1
( M + m′) l + m l sin 2θ y ) sinθ − m l sinθ cosθ θ# 2 − Fx cosθ -/ ')( M + m′ + m ) ( g + ##
which substituted into (59) gives
(61)
206206D. Maravall
− θ# {F cosθ − [( M + m′ + m ) ( g + ##y ) ( M + m′) l + m l sin 2θ x +( M + m′) l + m l sin 2θ − m l cosθ θ# 2 ] sinθ }
V# ( x1, x2 ) =
(62)
Note that in step (61) we have, again, made use of the control strategy based on considering the vertical acceleration as an exogenous variable in the pendulum dynamics, which is the most feasible procedure for controlling the system when simultaneously applying horizontal and vertical forces. From (62), we propose the control law
Fx = [( M + m′ + m ) ( g + ## y ) + ( M + m′ ) l 2 2 + m l sin θ − m l cosθ θ# /- tgθ + kd θ#
(63)
that with k d > 0 completely guarantees the definite negativeness of the first derivative of the Liapunov function for − π < θ < π
V# ( x1, x2 ) =
− kd θ# 2 ( M + m′) l + m l sin 2θ
(64)
Note that the control law (63) can be expressed for the values of interest of the variable as a conventional PD control law !
Fx = k p tgθ + kd θ# ≈ k p θ + kd θ#
(65)
In fact, the control law (63) is a refinement of the stability conditions that we obtained in the ODE analysis and with Liapunov’s indirect method –see expressions (43) and (56)-(57), respectively-. Thus, by applying Liapunov’s direct method, we have arrived at a more precise and refined control law that guarantees the global IP stability. Consequently, the first step in the actual stabilization of the IP is to design a quantitative control law by introducing specific performance indices such as rise time, settling time, percent overshoot, bandwith etc. Afterwards, the global stability of the IP is guaranteed by additionally constraining the designed control law to satisfy the condition given by (63).
Control and Stabilization of the Inverted Pendulum
207 207
4 Concluding Remarks Although the use of high-frequency oscillating vertical forces for IP stabilization is a well-known technique, the application of generic vertical forces to stabilize the IP has not been, theoretically and practically, fully developed to date. In this chapter, the novel idea of controlling and stabilizing the IP via vertical forces has been introduced and thoroughly analyzed. After having established the dynamic equations of the IP with a generic vertical force applied to its base, we studied IP control and stabilization. The final conclusion is that the vertical force has an excellent and fast stabilization effect, although at the cost of maintaining the IP in free-fall. After this preliminary analysis, the chapter approaches its main contribution, namely, the combination of the customary horizontal force with the vertical force. Roughly speaking, the horizontal force permits a direct stabilization of IP by means of a feedback control action, while the vertical force significantly improves IP stabilization, mainly due to its fast response to external perturbations of the IP equilibrium state. The theoretical analysis of the combined forces has been developed for both the ODE and the variable state representations. In particular, the necessary and sufficient conditions of the local stability of the IP controlled by a PD algorithm have been obtained. Furthermore, by applying Lyapunov’s direct method, the control law, which turns out to be a PD-like feedback action that guarantees the global stability of the IP, has also been obtained. As a general concluding remark, the chapter has demonstrated the excellent properties of the vertical force as regards the stabilization of the inverted pendulum.
Acknowledgments The idea of stabilizing the inverted pendulum via a vertical force originated from endless discussions with my father, Prof. Dario Maravall-Casesnoves of the Royal Academy of Sciences, Madrid. The control law defined by expressions (46) and (47) was proposed by Javier Alonso-Ruiz. Special thanks are due to Prof. C. Zhou of the Singapore Polytechnic for very fruitful discussions and insightful comments and for his invitation to write this chapter.
References 1. 2.
Khalil HK (1996) Nonlinear systems. Second edition. Prentice-Hall, Upper Saddle River, NJ Nelson J, Kraft LG (1994) Real-time control of an inverted pendulum system using complementary neural network and optimal techniques. Proc. American Control Conference, 2553-2554
208208D. Maravall
3. 4. 5.
6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
Zhou C, Ruan D (2002) Fuzzy control rules extraction from perception-based information using computing with words. Information Sciences 142: 275-290 Ghosh BK, Xi N, Tarn TJ (1999) Control in robotics and automation: sensorbased integration. Academic Press, San Diego, CA Maravall D, de Lope J (2002) A reinforcement learning method for dynamic obstacle avoidance in robotic mechanisms. In: Ruan D, D´hondt P, Kerre EE (eds) Computational intelligent systems for applied research. World Scientific, Singapore, 485-494 Schaal S (1997) Learning from demonstration. In: Mozer MC, Jordan M, Petsche T (eds) Advances in neural information processing systems 9. MIT Press, Cambridge, MA, 1040-1046 Sprenger B, Kucera L, Mourad S (1998) Balancing of an inverted pendulum with a SCARA robot. IEEE/ASME Trans. Mechatronics 3 (2): 91-97 Schreiber G, Ott C, Hirzinger G (2001) Interactive redundant robotics: control of the inverted pendulum with nullspace motion. Proc. IROS 2001, 158-164 Chung CY (2002) Balancing of an inverted pendulum with a kinematically redundant robot. Int. J General Systems 31 (1): 1-15 Bloch AM, Leonard NE, Marsden JE (2000) Controlled Lagrangians and the stabilization of mechanical systems I: the first matching theorem. IEEE Trans. Automatic Control 45 (12): 2253-2270 Bloch AM, Chang DE, Leonard NE, Marsden JE (2000) Controlled Lagrangians and the stabilization of mechanical systems II: potential shaping. IEEE Trans. Automatic Control 46 (10): 1556-1570 Stephenson A (1908) On induced stability. Philosophical Magazine 15: 233236 Kapitsa PL (1951) Dynamic stability of a pendulum with a vibrating point of suspension. Zh. Ehksp. Teor. Fiz. 21 (5): 588-598 Fliess M, Levine J, Martin P (1995) Flatness and defect of non-linear systems: introductory theory and examples. Int. J Control 61 (6): 1327-1361 Furuta K, Yamamoto M, Kobayashi S (1992) Swing-up control of inverted pendulum using pseudo-state feedback. J Systems Control Eng. 206 (6): 263269 Widjaja M, Yurkovich S (1995) Intelligent control for swing up and balancing of an inverted pendulum system. Proc. IEEE Conf. Control Applications, 534542 Fantoni I, Lozano R (2001) Non-linear control for underactuated mechanical systems. Springer, Berlin Meerkov S (1980) Principle of vibrational control: theory and applications. IEEE Trans. Automatic Control 25: 755-762 Bogaevski V, Povzner A (1991) Algebraic methods in nonlinear perturbation theory. Springer-Verlag, New York Hillsley KL, Yurkovich S (1993) Vibration control of a two link flexible robot arm. Dynamics and Control 3 (3): 261-280 Acheson D (1993) A pendulum theorem. Proc. Royal Society of London, series A 443: 239-245
Control and Stabilization of the Inverted Pendulum
209 209
22. Baillieul J, Lehman B (1996) Open-loop control using oscillatory inputs. In: Levine WS (ed) The control Handbook. CRC Press & IEEE Press, Boca Raton, FL, 967-980 23. Dimeo RM, Thomopoulos SCA (1994) Novel control of an inverted pendulum. Proc. American Control Conference, 2185-2189 24. Wu Q, Thornton-Trump AB, Sepehri N (1998) Lyapunov stability control of constrained inverted pendulums with general base point motion. Int. J Nonlinear Mechanics 33: 801-818 25. Wu Q (1999) Lyapunov´s stability of constrained inverted pendulums. Proc. American Control Conference, 293-297 26. Raya A, Maravall D (2001) Contribution to the control and stabilization of the pole-cart system. In: Moreno-Diaz R, Buchberger B, Freire JL (eds) Computer aided ystems theory. LNCS 2178, Springer, Berlin, 437-449 27. Jose JV, Saletan EJ (1998) Classical dynamics: a contemporary approach. Cambridge University Press, Cambridge
Appendix. Experimental Results J. Alonso-Ruiz and D. Maravall Department of Artificial Intelligence, Faculty of Computer Science Universidad Politécnica de Madrid, Madrid 28660 SPAIN We briefly present some of the experimental results obtained from computer simulations, in which IP stabilization via the combination of horizontal and vertical forces is investigated and compared with a single horizontal force. We have considered the values of the system parameters to be as follows: cart mass, 2 kg; platform mass, 0.2 kg; pendulum mass and length, 0.1 kg and 0.5 m, respectively. Unless otherwise indicated, distances are in meters (m), time in seconds (s), forces in newtons (N) and angular displacements in radians in all figures. In all the reported examples the respective PD algorithm gains have been obtained to optimize the usual performance indices: rise time, overshoot peak and settling time, either for the combined horizontal and vertical forces or for the single horizontal force. Furthermore, for the combined case, the stability conditions obtained in the theoretical analysis –see expressions (43), (56), (57) and (63)- have been applied. Figure 4 shows the IP trajectories for initial deviations from 5º to 30º in 5º steps.
210210D. Maravall
(a)
(b)
Fig. 4. Instances of IP stabilization with (a) a combination of horizontal and vertical forces and with (b) a single horizontal force.
Note the significant improvement achieved by the addition of the vertical force, which makes the IP stabilize much faster. In particular, the greater the IP initial deviation, the stronger the positive effect of the vertical component. It is also interesting to compare the control efforts, so the respective horizontal forces profiles are depicted in Figure 5. Note the + 20 N restriction over the range of the applied forces. The same initial deviations as in Figure 4 have been considered.
(a)
(b)
Fig. 5. Profiles of the horizontal forces of (a) the combined case and (b) the single horizontal case. The greater the initial deviation, the stronger the applied force.
Note that the reductions in the control effort achieved with the combination of horizontal and vertical forces are significant. Of course, in these cases there is an additional control effort produced by the vertical force, although its respective energy cost is comparatively negligible, because the platform mass is small in comparison to the cart mass. Figure 6 shows the vertical forces profiles and the respective vertical displacements of the platform.
Control and Stabilization of the Inverted Pendulum
211 211
Fig. 6. Vertical forces profiles and platform vertical displacements for the same range of initial deviations as above.
212
Vision Based Motion Planning of Humanoid Robots M. Fikret Ercan and Changjiu Zhou School of Electrical and Electronic Engineering, Singapore Polytechnic, 500 Dover Road., Singapore 139651, Republic of Singapore http://www.robo-erectus.org
Abstract. Vision is the most effective sensing form for humans in understanding their environment, recognition of the objects around them and navigating from one point to another. Humanoid robots mimic human behaviour and they are mainly intended for applications where robots need to work in an environment designed for humans. Naturally, computer vision serves as a dominating sensing unit for them. Computer vision is a powerful tool. In addition to steering the robot, it can also play a key role in achieving robot’s intelligence and its interaction with outside world. This paper discusses various aspects of exploiting computer vision in humanoid robots.
1 Introduction In the past, robots were predominately developed and utilized in factories for manufacturing and transportation purpose. However, a new generation of robots has recently begun to emerge. These are service robots that cooperate with people and assist them in their everyday tasks. Many wheeled mobile robots, operating in indoor or outdoor environments, have been studied extensively for this purpose. Lately, researchers are interested in developing human-like robots named humanoids. The main objective of realizing a human like robot is to develop robots that can operate in an environment designed for humans. Ideally, it is expected that such robot would be a replica of humans with motion, sensing, reasoning, and socializing capabilities and even emotional responses. The research in this field promoted by well-respected researchers [3], though, it was initially perceived as an ambitious research challenge. Recent developments, how-ever, demonstrate that emergence of humanoid robots in our daily life is not too far. A significant example to this is P2 and P3 and Asimo robots developed by Honda [12]. With such astounding demonstrations, research in humanoid robots gaining popularity and it is attracting many research institutions [21, 18, 16]. For instance, humanoid SIG is designed to study integration of perceptual information to control robot [18]. The prototype H7 is developed by Kawada industries to provide an experimental research T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 212−227, 2004. Springer-Verlag Berlin Heidelberg 2004
Vision Based Motion Planning of Humanoid Robots
213 213
platform for full-body integrated sensing and control [16]. Recently, Sony announced a new biped robot [27]. Sony’s the SDR-4X is 50cm tall, weighs 6.5kg, and uses a pair of 64-bit processors. It has speech recognition, speech synthesis, and stereo CCD vision sensors to provide face recognition and distance measuring capabilities. Earlier work in humanoid robots mainly concerned with elemental functions such as bipedal walking of robot, control of algorithms for smooth movement, and multi-sensor fusion. A significant amount of study reported in literature was on the study of bipedal locomotion and many methods introduced for controlling robot walking (see for instance [15]). With the improvements made on the stable bipedal walking of humanoids, attempts to make them autonomous become viable. Integrating a vision based perception system can advance robot locomotion and behaviour. Using computer vision in robotics is not new and there are many vision guided mobile robots reported in the literature [20, 6]. These systems generally utilize a dedicated hardware especially to rapidly extract useful information, such as depth information from stereovision [21], obstacles or land marks, to steer the robot reliably. However, in the case of humanoids integrating a sophisticated computer vision unit does not only assist to steer the robot but also allow it to interpret visual data to recognize its environment, plan a proper motion and interact (even socialize [2]) with its environment autonomously. The capability of a robot’s understanding of its environment is a key parameter to achieve these goals. A well-known approach for this purpose is model-based processing of image data [20, 28]. It is a goal oriented approach and requires knowledge about the robot’s environment through partially modelling of unchanging parts in that environment. Basically, the following objectives are intended to achieve: • Localizing the robot by analyzing the scene and finding a match of the current scene with those stored in a database. • Detect obstacles on a mobile robot’s path. The knowledge based vision system is used to detect unexpected object features in a scene. This is achieved by finding mismatches with the world model defined in the system. • Plan robot navigation by referring to correspondence between image features and expected landmark locations. Perception is achieved by a hypothesis and test method where the model-based system produces hypothesis about the objects in the scene and its relationship with other objects and verifies them with sensory input. Recently, there are other innovative approaches used in vision-guided systems that significantly differ from classical sensory input, analysis and action type of control scheme. Instead, a neural network, which acts as a controller, is used between input pixels and motor drivers (see for instance [1, 22, 3]). This method proposes that a robot will develop a proper behaviour by progressively learning its environment and training its neural network controller. Apparently, a major draw back is the time consuming training process of the network. This is an active research area though application to robots operating in an uncontrolled environment is not immediate. Current research demonstrates solution to rather simple problem cases.
214214M.F. Ercan and C. Zhou
This paper is mainly focused on application of vision in humanoid robots in particular perception and motion planning aspects. In the following section, various features of computer vision will be discussed. Integrating a computer vision system to humanoid robot will be elaborated in section 3 and underlying research issues will be presented. Section 4, describes a humanoid robot and its vision system developed in our institution.
2 Challenges of Computer Vision A governing human sensing is vision; therefore, harnessing an artificial one will benefit attaining autonomy for robots. A truly autonomous system must posses the decision-making ability (intelligence) to work in a non-deterministic world, like a human being. A reliable decision-making is highly depended on the information collected from sensors. As images from the surrounding environment provide the most valuable information, many robots employ a vision based sensory unit [3,20]. However, there are two main factors governing a robust vision system and they can be broadly categorized as software and hardware issues. 2.1 Software issues Unlike the human vision system, which is endowed with natural mechanisms to filter out and reduce the burden of irrelevant information, a vision-guided robot must be able to extract useful information from the input images. However, there is no single or standard solution to achieve this goal. Usually a set of imageprocessing algorithms is selected for the application at hand, and they are acquired from a wide selection of fields such as signal processing, statistics, and artificial intelligence. There is no unambiguous definition on how algorithms are selected, though, they are usually organized in a hierarchical order and the type of image processing can be broadly classified into three levels considering the complexity and data type involved in the operations [28]. Algorithms at the lower level of image processing deal with extracting basic object features, such as corners, size, shape, orientation, linear line segments etc. Operations mainly deal with array of numerical information. The objective of the intermediate level algorithms is to group features extracted from the image. The output of intermediate level algorithms is no more pixels but symbols that represent object features found in the image. In the higher-level, hypothesis regarding the elements in the scene are generated and verified with detected features and finally information about the scene is collected for interpretation. Operations here are model directed and usually utilize spatial relations in object features. Planning and initiating an action, based on the scene analysis, are also conducted at the high-level. The controlling of the computer vision operations is in bottom-up, top-down or hybrid manner. Bottom-up approach requires a complete set of correct features to
Vision Based Motion Planning of Humanoid Robots
215 215
carry out interpretation and this is not possible in practice. On the other hand, topdown analysis is model guided. The features being sought trigger the operations. Hybrid approach combines these two approaches where a bottom up feature extracting is performed first and then followed by a top-down verification and matching phase. The selection of algorithms and the control of vision operations are user dependant and they are not perfect. Furthermore, due to occlusions of objects, noise, and shadows low level image processing may not produce a good result. Consequently, more intelligent algorithms needed for the high level processing to be able to reason with incomplete data. An additional pre-processing is needed in humanoid robotics application to compensate the errors occur in camera position at every step of the robot. 2.2 Hardware issues Vision computation in robotics is highly demanding, mainly due to the amount of data and various types of operations involved. For instance, computations in a typical robotic application must be completed within a constant time frame, which ranges from 25-100 msec. A well-known approach to overcome this hurdle is to employ multi-processor systems. However, considering the computation structure discussed earlier, it is expected that such system should be tailored to the requirements of computer vision. For instance, the system must be capable of manipulating various data types such as pixels, symbols and other complex structures. The system must perform low, intermediate and high-level algorithms simultaneously and it must have a fast data I/O mechanism. A good scalability of computational performance, flexibility and modularity are also desired features. There is a vast amount of research presented to meet computational requirements of computer vision. An extensive discussion on high performance computer vision architectures can be found in the following references [7, 9]. It is difficult to address this great variety of requirements by a single architecture and there are various approaches used in research community. For the purpose of compact and feasible hardware realization, application specific systems are developed. These are usually VLSI based compact systems but they are not flexible [8, 23]. A large number of them are proposed to speed-up low-level image processing such as convolution, median filtering etc. The other common approach is to construct a programmable and flexible system to address variety of algorithms used in computer vision. These systems employed either a heterogeneous multi-layer or a partitionable/reconfigurable structure. Heterogeneous multi-layer systems combine multiple layers of computing platforms and provide a suitable one to each abstract computation level, such as SIMD arrays for low level and MIMD platforms for intermediate and high level processing. When processing continuous stream of images, executing three processing levels simultaneously generates a pipelining effect. On the other hand, partitionable/reconfigurable systems provide a pool of processors that can be partitioned into clusters and configured as SIMD, Systolic, or MIMD depending on the application that is being han-
216216M.F. Ercan and C. Zhou
dled by each cluster [7]. A major shortcoming of these systems is their huge hardware complexity that resulted from efforts to achieve flexibility. Recent systems favour dedicated image-processing units based on VLSI large processor arrays [8, 23], or a network of DSPs [9,11]. Recently, FPGA arrays become popular due to their easy hardware configurability [29]. Problem size for a humanoid robot operating at indoors can be regarded as medium-scale considering the data size and the processing requirements. Basic architectural requirements from vision hardware are the same, but the system should be feasible and hold a low hardware complexity so that it can be realized easily. The other concern in humanoids is apparently the physical limitations. The hardware included to the system has to be compact and power consumption should be minimal. This implies that achieving a compact vision hardware using off the shelve components is rather difficult task.
3 Employing Computer Vision in Humanoid Robots We have reasoned that computer vision can provide significant information to a humanoid about its environment. In order a humanoid to have visual sensing various computer vision areas have to be integrated. We can categorize them as navigation, interaction and coordination. In the following, we will elaborate these areas. 3.1 Navigation Computer vision techniques in this category, deal with gathering information about the outside world by recognition and interpretation of the environment, extracting range information and tracking objects or landmarks. This information is then used to guide the robot such a way that it can negotiate outside world autonomously. There are many techniques fall into this category, though, we will point out three well-known methods: model based scene interpretation (or analysis), stereo depth analysis and active vision. Model based scene analysis: The major goal of the vision is to detect and locate landmarks and obstacles to plan navigation. Interpretation system generates a semantic and symbolic description of a scene in terms of detected objects and their spatial relations. Model based scene analysis would be the most suitable tool for humanoids operating indoor environments. The world model predefined to the robot and its navigation is goal directed. Hence, in each step robot is able to localize itself and how far from the goal by comparing the model and real scene. There are many significant work has been done in this field (see for instance Sandakly et al.[26] as an example). Model-based object recognition or, more generally, scene interpretation is a two-part process. The first phase is to generate a sequence of hypotheses on object identities and poses, and the second phase is to evaluate them based on the object models. The overall structure of computations includes
Vision Based Motion Planning of Humanoid Robots
217 217
algorithms from three image processing level as described earlier in Section 2. Major difficulties involved in scene interpretation are in selecting and tuning image-processing algorithms that deliver object features effectively, optimizing the processing time by selecting relevant data from the scene, and uncertainties and impression of models and data gathered from scene. Among them the most difficult issue to deal with is the uncertainties and many studies using Bayesian probability, evidence theory and fuzzy sets have been presented earlier in the literature [4] . For visual analysis of natural scenes, which include incompletely visible objects in an uncontrolled context, it is a highly intricate task to optimize the match of a model to the data obtained from the scene. Local optimization techniques will usually get stuck in local optima while techniques similar to exhaustive search are limited by time and memory constraints. Consequently, in practice a critical aspect of many object recognition problems is to develop a clever search method. Stereo Analysis and 3D modelling: Humanoids, akin to any autonomous robot, have to interpret 3D structure of their environment. There are two main approaches used in obtaining range information of object or constructing a 3D map of a robot’s environment. One method is using active sensors such as sonar, radar, or laser range finders. Sensors generate the signals and the bouncing signals from the objects provide range information. The second method is using stereovision. Cameras are placed with a fixed baseline and depth can be obtained by computing the disparity in each point in image pairs. Apparently a key problem in stereovision is to find corresponding points in stereo images. Corresponding points are the projections of a single point in 3D world in the different image spaces. The difference in the position of corresponding points in their respective images is called disparity. There are many algorithms presented for the solution. They are generally based on window based correlation technique. The performance of the algorithms is depended on selecting a proper window size and it is not trivial. Adaptive techniques for the solution introduced by Kanade et. Al. [17]. Apparently most of the humanoids presented in the literature employ stereovision to identify robots environment. In some of the applications, stereovision was employed to track a target and to detect mobile objects [18, 21]. An interesting application of stereovision in humanoid robots is demonstrated in [21], where stereovision is used to estimate the grade of the ground. The purpose of the study is to calculate the proper posture of the humanoid so that it can walk not only on flat ground but also on the uneven terrains such as slops. Another particular application of stereovision in humanoids would be to perform leaping over an obstacle. This task is particular to humanoid robots and solution requires utterly different approach than the conventional methods used for obstacle avoidance in wheeled mobile robots. Active Vision: The ability of actively controlling camera parameters is vital to achieving robust, real-time perception for a robot interacting with a complex, dynamic world. Active vision systems have mechanisms that can actively control camera parameters such as orientation, focus, zoom and aperture in response to the requirements of the task and external stimuli. More broadly, active vision incorporates attention that is selectively sensing in space, resolution and time by modify-
218218M.F. Ercan and C. Zhou
ing physical camera parameters or the way image data is being processed. An active vision system actively extracts the information it needs to solve a task. For instance, a camera-equipped robot can change the direction of its camera is looking at, or change the zoom. In other words, an active vision system always interacts with the world with a tight coupling between perception and actions. The processing is also tied closely with the activities it supports, such as navigation or manipulation, allowing simplified control algorithms and scene representations. In this sense, the concept of active vision is different from traditional method of finding detailed reconstructions of the external world. The benefits of active vision were primary interest in a variety of humanoids projects. A well known application of active vision is Kismet humanoid robot where the robot socializes with humans by analyzing their behaviours [2]. 3.2 Gesture recognition Many robots developed today do not interact with people. As humanoids start appearing in our daily life, such as health-care, nursing, and entertainment, humanrobot interaction will be vital. Non-expert users will operate these robots therefore a natural interface that makes instructing these robots as simple as possible would be very useful. In this respect, gesture recognition, face recognition, facial gesture recognition come handy to use in humanoid robots. Gesture recognition: Gestures are an important aspect of human interaction, both interpersonally and in the context of man-machine interfaces. There are many aspects to the modelling and recognition of human gesture as they can be expressed through hands, faces, or the entire body. Gesture recognition is valuable in applications requiring human/robot interaction as it can provide a redundant form of communication or it can help to clarify spoken commands.. For example, the user may say “left” simultaneously pointing to left direction with a hand gesture. The robot needs to recognize only one of the two commands. This is beneficial in situations where speech may be distorted or suppressed by environmental noise. Furthermore, iconic information can easily be given to the robot. For instance, instead of giving coordinates of the target point that robot should move, the user can simply point to that spot. Motion gestures provide additional freedom in the design of robot interaction with gestures. In addition, they reduce the chances of accidentally classifying arm poses as gestures hence they appear better suited for human robot interaction than static pose gestures. While the area of gesture recognition is relatively new, there has been a great deal of activity in the last few years. Some of the vision-based interfaces that allow people to instruct mobile robots via arm gestures can be found in the following references [10, 19]. Face Recognition: Over the years there is a vast amount of search on face recognition and facial expressions [30]. Performance of these algorithms, such as eigenfaces, is depended on accurate detection of faces as they assume that face images are normalized in terms of scale and orientation. In numerous studies, face detection techniques demonstrated such as using motion detection or skin colour.
Vision Based Motion Planning of Humanoid Robots
219 219
Colour provides a computationally efficient yet effective method that is robust under rotations in depth and partial occlusions. A neural network based algorithm is also demonstrated by Rowley et. al. [25], where a neural network examines small windows of an image and decides whether each window contains a face. In a recent study, a model-based approach, using Hausdorff distance has been demonstrated where the face recognition is realized without a face detection step [14]. The detection, recognition and model-based coding of faces have a potential application in humanoid robots as they are interacting with humans. Therefore, it is necessary to detect faces and facial gestures and react accordingly. For example the application presented in [8], demonstrates a robots interaction through facial recognition. The robot interprets the head gesture nodding as “Yes” and shaking as “No”. The face recognition also endow robot with the capability of identifying people. It can be practical for security/surveillance applications or developing personalized robots. 3.3 Coordination Human beings easily use eyes to guide their arms and hands to reach objects, and to perform any particular pose. A robot that can imitate such hand-eye coordination needs the sophisticated skills of perception, spatial reasoning, and communication [13]. This task requires a combination of computer vision methods stereo and active vision components in particular. Grasp reflex would be one of the main objectives. The relative positions of hand and target have to be observed and the motion is refined until the object is grasped. A major challenge is the dynamic nature of the robots interaction with its environment and with human. It is impossible to program the robot anticipating every possible situation; hence the common consensus in achieving this goal is by robot learning. (see for instance, Cambron et al[5] ). 3.4 Key research areas The major areas of visual perception and intelligence envisioned for humanoid studies discussed above are still open to further studies. Although, each topic is a major research area by itself, a key research area is the integration of them. In terms of a full-scale computer vision application, a few systems deployed in the literature and so far fairly limited progress has been achieved. This is mainly due to a lack of robustness and methods for integration of various techniques into full systems. The construction of systems gradually provides the insight to formulate basic methods for the design, analysis, implementation, and evaluation of operational and robust systems that have a wide domain of application.
220220M.F. Ercan and C. Zhou
4 Design of Humanoid Robo-Erectus Currently, we are developing humanoid robots in our research lab [24]. The ultimate goal of this project is to build humanoid robots that can play a soccer game akin to human. The idea of soccer playing robots has been introduced in early 90’s. Apparently, the basic idea behind this initiative was to advance artificial intelligence, control, and robotic fields. Soccer playing is used as a test bed to represent dynamic real world. The response from the rest of the scientific world was strong, and the Robot World Cup Initiative, RoboCup, was started. The first Robot World Cup, RoboCup-97, was held in Nagoya, Japan, in August 1997, and included the participation of more than 40 teams. Since then, RoboCup became an annual event. The vision is to develop a humanoid team which will have the potential of defeating a real world cup winning team of humans in fifty years. Apparently, this challenge poses all the possible set of problems in artificial intelligence field to tackle. The first step of the project was to build a robust humanoid robot that can perform bipedal walk, for straight, curved motions and in-place turns. The second stage of the project includes identifying the goal, objects, team mates and performing basic moves such as kicking, passing and shooting the ball. The third stage of the project would be the acquiring of a team strategy. Currently, we have developed a number of robots with different specifications. These humanoids can perform a steady bi-pedal walk, turn, and ball-kicking type of motions. The first version of the robot had 10 DOF while the latest version has 22 DOF. 4.1 Structure of the humanoid The task of soccer playing requires a humanoid to perform various behaviours and motion types and some of them fairly complex. For the successful realization of the first stage of the project, we had to consider all these motions and provide a high DOF to humanoid. The first stage of the project was completed successfully and demonstrated in RoboCup humanoid walk league 2002. Fig 1 shows our humanoid robot at work. Three control systems developed for these robots. They are PC-based, microcontroller based, and PDA based controllers. Main consensus behind the PDA based controller was to reduce to the cost of humanoid robot and make it more portable and compact. The latest version of the humanoid includes all the limbs and neck joints. In Fig 2, we present the organization of the basic joints and the sensor positions on the robot. 4.2 Humanoid vision unit and related tasks The basic structure of humanoid robot is stable and flexible. Currently, we are tackling with challenges at the second stage of the project that requires a vision-
Vision Based Motion Planning of Humanoid Robots
221 221
Fig. 1. Robo Erectus performing a steady walk
based unit to be incorporated to the system. Our immediate goal is to perform the tasks described in RoboCup humanoid league. These tasks are briefly: 1. Performing a steady walk and making a U turn at a designated landmark 2. Detecting a ball and kicking 3. Executing a penalty kick against a goal keeper. These objectives can comfortably be accommodated via a vision system. Fig 3 depicts a block diagram of the vision unit’s place in robot’s control structure. As shown in this figure, vision unit provides feedback both local and global action planning components of the controller. The first task is about navigation and algorithms discussed earlier in Section 3.1 need to be incorporated. Here, vision unit provides information to global motion and path planning units; they are the scene interpretation and stereo depth information. The scene interpretation reports the spatial relationship about the objects, and between the robot and objects. Here, the whereabouts of landmark and robot’s spatial relation to it is vital to make a proper walk and U-turn around the landmark. The stereo depth information is used to provide 3D-proximity data to robot. We employ a knowledge based top-down approach for the algorithm as the robot is operating in a predefined environment. Our method is similar to that discussed in [20] though in recognition phase we are also utilizing color parameter. The second and third tasks are more complex as they require detecting a ball, approaching to it and kicking. For a penalty kick against a goalkeeper, robot needs to develop a strategy and it should have a reasonable control on kicking the ball to a certain direction. The techniques used here fall into coordination topic discussed earlier in section 3.3. Here, challenges are more on the local motion-planning unit and we are basically dealing with two situations static and dynamic. In the static
222222M.F. Ercan and C. Zhou
Accelerometers and gyros are mounted to provide body posture information
z pitch y
yaw
row
x
Vision system
4 force sensing sensors for each foot
Fig. 2. The humanoid structure and sensory structure
case, robot negotiates with static objects and landmarks. For instance robot needs to detect boundaries of the soccer field, position of the ball etc. In order to detect the ball we use color and shape analysis. The goal tracking function should be considered as a higher level of the basic locomotion control system in order to implement a hierarchical autonomous locomotion control system as shown in Fig. 4. For the static case planning has two major steps. The first step is to determine the path to the goal, and the second step is to calculate the width of each step and the turning angle of Θ. In the dynamic case, we deal with moving objects such as the goalkeeper. However, in this problem both robot and the target are in motion. The moving goal tracking function consists of the following tasks: 1. Detection of the 3D position measurement of the goal in camera coordinates. As the humanoid and the goal are both moving, the goal’s color rather than its shape is utilized to detect the goal. 2. Planning of footstep for one step: For the path and footstep planning, we use the world coordinate system which is more convenient for acquiring the knowledge of the goal motion and the map information. 3. Camera posture control with self motion compensation: In order to keep the goal in the field of view, the camera posture is controlled by the pan and tilt joints of the neck. This can be perceived as an application of active vision in humanoids. That is small deviations in robot’s gazing point due to walking motion is compensated by using head and neck joints. Robot has to keep its dynamic stability. However, unlike wheeled robot, humanoid’s body doesn’t move along with the moving direction. Another, practical issue is vertical position of the camera. Currently, cameras are fixed on the robot’s shoulders, which
Vision Based Motion Planning of Humanoid Robots
223 223
result in rotation of input images up to 12 degrees with the robots bipedal walking. To compensate this rotation, a camera self-motion compensation system is needed. For this purpose we are experimenting two approaches, one of them is using a feedback from the gyroscope integrated into humanoid’s body and using the tilt joint to correct the camera position. It gives a measure of rectification needed to correct the error occurred in camera alignment due to the robot body movement. The second approach is by fixing a landmark in the world model in the scene and counter rotating the image pixels by computing the optical flow between two consecutive image frames. This method is purely in software and computationally demanding. 4. In addition to calculations made for path planning, step size and turning angle in the dynamic case we also calculate the speed of the walk in order to cope with the dynamics of the moving goal. Path Planning
Gait synthesis
Global motion planning
Local motion planning
Humanoid robot
Joint controller
Potentiometers/ encoders
Range sensors/gyros/ force sensors,...
Vision
Fig. 3. A block diagram of vision and other sensory input to robot control mechanism
4.3 Architecture of the vision unit The humanoid project is open ended and the future tasks of the humanoid will be more and more complex. Current vision system utilizes a single CPU and it will be limited for them. Hence, we have to take hardware aspect of the vision system into account and provide a dedicated architecture for it. Our goal is to design a system with minimal hardware complexity while satisfying the basic architectural requirements demanded in a computer vision system in general. For this purpose, currently we are building a multi processor unit with latest parallel DSPs from Analog Devices. This class of processors provides computation flexibility sought for the system as they can efficiently perform both low-level number crunching type algorithms as well as high-level algorithms that require manipulation of data structures. The hardware configuration for data communication is also made flexi-
224224M.F. Ercan and C. Zhou
ble by latest FPGA devices. Processors of the system can be partitioned and pipelining of vision operations can be realized easily since FPGAs used in data communication frontline. Each DSP holds its own memory and four of them share a common memory for rapid data transfer. The vision architecture has two separate units; they are a pool of processors and a image capture unit. A block diagram of this system is presented in Fig 5.
θ
Fig. 4. Path and footstep planning for humanoid turning
The structure of the processing unit is modular and it is built by integrating basic building modules. In each module one of the four DSPs acts as a controller or master processor to the others. In addition to parallel computing, this unit also aimed for multiprogramming. That is modules carry out vision tasks independently, this leads to a two-way communication structure between the modules as well as between the processors of each module. Intra-Layer Communication: although vision system is developed with building blocks available in the market, we organize these basic modules and their communication hierarchy. The first level of the communication is the interaction between the processors of a module. Here, a hybrid communication platform is employed for this purpose. That is in each module processors can directly communicate to each other via a communication link or by using a shared memory among them. When processing images in a parallel architecture, collective communication operations, such as broadcast, scatter and gather, are needed very often and they are time consuming operations. Shared memory is suitable for collective communication, whereas direct links for data exchange between sub-programs running at individual processors. Inter-Layer Communication: This mechanism required for transferring results from one module to another. Communication by using shared memory between modules is most likely to be a bottleneck on the performance and will limit the flexibility. Therefore, we directly connected processors at successive modules.
Vision Based Motion Planning of Humanoid Robots
225 225
The inter-layer communication allows pipelining of vision tasks and it enhances computation performance.
Fig. 5. The architecture of the latest vision unit of the humanoid robot
The scene interpretation algorithm utilizes these modules simultaneously. The pipelining effect is achieved when we process multiple frames. The flow of operations is bottom-up at first and then top-down at the second phase. In the first phase, mainly low- level image processing algorithms are performed for feature extraction and features are grouped to form tokens or vectors to represent objects. At the second phase, matching of objects with model base is performed and using knowledge (world model) about the spatial relations of objects, a description of scene is built. At this stage hypothesis about missing object features or scene parameters are also generated and verified with further processing the input image.
5 Conclusions In this paper, we have discussed issues concerning humanoid robots vision. The desired intelligence and human like behavior of humanoids is strongly dependent
226226M.F. Ercan and C. Zhou
on the superiority of vision systems embedded into their design. Whilst it may seem over ambitious to incorporate so many different aspects of vision into a humanoid robot, desired goal can only be achieved when various visual operations are employed simultaneously. In this paper, we have also briefly introduced the humanoid robot developed in our institution. The ultimate goal of this humanoid robot is to play soccer game. This task is used as a test bed as it embodies all the major techniques discussed above to be incorporated.
Acknowledgments This project is supported by the Singapore Polytechnic R&D fund and Singapore Tote Board fund. Authors wish to acknowledge the many members of the ARICC Centre who have contributed to this effort.
References [1] S. Baluja, “Evolution of an Artificial Neural Network Based Autonomous Land Vehicle Controller,” IEEE Transactions on Systems, Man and Cybernetics-Part B, vol. 26, pp.450-463, 1996. [2] C. Breazeal, “Sociable Machines: Expressive Social Exchange Between Humans and Robots,” Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, PhD Thesis, May 2000. [3] R.A. Brooks, C. Breazeal, M. Marjanovic, B. Scassellati and M.M. William-son, “The Cog Project: Building a Humanoid Robot,” IARP First International Workshop on Humanoid and Human Friendly Robotics, 1998. [4] T. Caelli and W. F. Bischof, Machine Learning and Image Interpretation, New York, Plenum Press, 1997. [5] M. E.Cambron, and R.A. Peters II, “Learning Sensory Motor Coordination for Grasping by a Humanoid Robot,” Proceedings of the 2000 IEEE International Conference on Systems, Man and Cybernetics, vol.5, pp. 870-875, 2000. [6] K. H. Chen and W. H. Tsai, “Vision-Based Autonomous Land Vehicle Guidance in Outdoor Road Environments Using Combined Line and Road Following Techniques,” Journal of Robotic Systems, vol. 14, pp.711-728, 1997. [7] A.N. Choudhary, J.H. Patel and N. Ahuja, “NETRA: A Hierarchical and Partitionable Architecture for Computer Vision Systems,” IEEE Trans. on Parallel and Distributed Systems, vol. 4, pp. 1092-1104, 1993. [8] M. Doi, M. Nakakita, Y. Aoki and S. Hashimoto, “Real-time Vision System for Autonomous Mobile Robot,” IEEE Int’l Workshop on Robot and Human Interactive Communication, pp.442-449, 2001. [9] M.F. Ercan and Y.F. Fung, “The Design and Evaluation of a Multiprocessor System for Computer Vision,” Microprocessors and Microsystems, vol. 24, pp. 365-377, 2000. [10] F. J., Roger, E. Kahn, M. J. Swain, and P. N. Prokopowicz , “Real-Time Gesture Recognition Using the Perseus Architecture,” University of Chicago, Computer Science Technical Report, No:96-04, March 1996
Vision Based Motion Planning of Humanoid Robots
227 227
[11] D.M. Harvey, S. P. Kshirsagar, C.A. Hobson, “Low Cost Scaleable Parallel Image Processing System,” Microprocessors and Microsystems, vol. 25, pp. 143-157, 2001. [12] http://www.honda-p3.com/ [13] R. Jarvis, “Interactive Hand/Eye Coordination between a Human and a Humanoid-A proposal,” International Conference on Intelligent Robots and Systems, pp.1741-1747, 2000. [14] O. Jesorsky, K.J. Kirchberg, and R. W. Frischholz, “Robust Face Detection Using Hausdorff Distance,” Lecture Notes in Computer Science, vol. 2091, pp. 90-95, 2001. [15] S. Kajita and K. Tani, “Study of Dynamic Biped Locomotion Rugged Terrain: Theory and Basic Experiment,” Int’l Conference on Advanced Robotics, pp. 741-746, 1991. [16] http://www.dh.aist.go.jp/h6/H6_H7.html [17] T. Kanade and M. Okutomi, “A Stereo Matching Algorithm with an Adaptive Window; Theory and Experiment,” IEEE Transactions in Pattern Analysis and Machine Intelligence, vol. 16, pp. 920-932, 1994. [18] H. Kitano, H. G. Okuno, K. Nakadai, T. Sabisch and T. Matsui, “Design and Architecture of SIG the Humanoid: An Experimental Platform for Integrated Perception in RoboCup Humanoid Challenge,” Proceedings of the International Conference on Intelligent Robots and Systems, pp. 181-190, 2000. [19] D. Kortenkamp, E. Huber and R. P. Bonasso, “Recognizing and Interpreting Gestures on a Mobile Robot,” Proceedings of AAAI-96,1996. [20] A. Kosaka and A.C. Kak, “Fast Vision-Guided Mobile Robot Navigation Using Model based Reasoning and Prediction of Uncertainties,” Computer Vision and Image Processing: Image Understanding, vol.56, pp. 271-329, 1992. [21] M. Kumagai and T. Emura, “Vision based Walking oh Human Type Biped Robot on Undulating Ground,” Int’l Conference on Intelligent Robots and Systems, pp. 13521357, 2000. [22] S. Nolfi and D. Floreano, Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines, MIT press, London, 2000. [23] S. Okazaki, Y. Fujita, and N. Yamashita, “A Compact Real-time Vision System Using Integrated Memory Array Processor Architecture,” IEEE Transaction on Circuits and Systems for Video Technology, vol. 5, pp. 446-452, 1995. [24] http://www.robo-erectus.org [25] H. A. Rowley, S. Baluja and T. Kanade, “Neural Network-Based Face Detection,” IEEE Transactions in Pattern Analysis and Machine Intelligence, vol. 20, pp. 23-38, 1998. [26] F. Sandakly and G. Giraudon, “3D Scene Interpretation for a Mobile Robot,” Robotics and Autonomous Systems, vol.21, pp. 399-414, 1997. [27] http://www.sony.net/SonyInfo/News/Press/200203/02-0319E/ [28] C.C. Weems, “Architectural Requirements of Image Understanding with Respect to Parallel Processing,” Proceedings of the IEEE, vol. 79, pp. 537-547, 1991. [29] J. Woodfill and B. V. Herzen, “Real-Time Stereo Vision on the PARTS Reconfigurable Computer,” IEEE Symposium on FPGAs for Custom Computing Machines, April 1997. [30] W. Zhao, R. Chellappa, A. Rosenfeld, and J. Phillips, “Face Recognition: A Literature Survey,” Technical Report, University of Maryland, No: CS-TR4167R, 2002.
228
Evolution of Locomotion Controllers for Legged Robots A. Boeing and T. Bräunl Centre for Intelligent Information Processing, University of Western Australia, Perth, Australia [email protected]
Abstract. The construction of a locomotion controller for legged robots is a difficult and time consuming process. This task can be automated through the use of evolutionary techniques and an appropriately chosen control system. The presented approach utilizes a genetic algorithm that evolves the parameters for a spline controller. The controller outputs its control signals to a virtual robot in a mechanical simulator, enabling the robot designer to preview the robots locomotion patterns. This approach has shown to produce walking patterns for a number of robots with differing morphology. Extensions to the basic spline controller allow various forms of sensory information to be encoded enabling the robot to maneuver over irregular terrain. The resulting system has shown to be a simple and efficient method for automated controller design.
1 Legged Robots Legged robots exhibit a number of advantages for locomotion[1]. The mobility offered by legged vehicles is far greater than that of wheeled or tracked vehicles, as they are not limited to paved terrain. The increased mobility offered, allows for a far larger range of applications to legged vehicles. Another incentive for exploring the use of legs for locomotion is that it provides an insight to the systems responsible for human and animal locomotion. Humans are capable of complex movements, whilst maintaining orientation, balance and speed. Robots that could mimic human movements could seamlessly integrate with the human world, enlarging the number of possible applications for legged vehicles. This makes the study of bipedal locomotion particularly attractive. Although there are a multitude of existing locomotion control techniques and well described design processes, the automated generation of these controllers for robots provides significant advantages. Often, the design process is quite complex, time consuming to perform, and requires the control system to be completely redesigned for small alterations to the robot [2]. Furthermore, humans often have difficulty in understanding which sensors to incorporate to provide the best possible feedback to the robot. Designers are often biased towards feedback sensors T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 228−240, 2004. Springer-Verlag Berlin Heidelberg 2004
Evolution of Locomotion Controllers for Legged Robots
229 229
that are present in humans, such as vision and touch. These senses are not necessarily the best sensors for the desired application. By evolving the locomotion controller the robot designer is alleviated from the controller design process, and the control system becomes more flexible, as the robot can improve its controller to cope with environmental or structural changes. Allowing the control system to evolve enables the controller to utilize sensory inputs that would normally be disregarded by human designers. The resulting controllers are more adaptive to the robot’s environment. They are also more robust, flexible, and usually provide superior performance to human designed controllers [2].
2 Control Architectures There are a number of control systems that are applicable to the objective of robot locomotion. Possible control systems range from simple oscillators [3] to neural networks [2] to simple assembly programs [4]. The simplest oscillators consist of a set of sinusoidal function generators whose outputs are combined to generate the control signal for an actuator. These systems can represent a range of outputs by altering the phase and amplitude of the sinusoids [3]. However, these systems are generally incapable of expressing the complexity required for sustained locomotion [5]. Thus, more complicated forms of control are desirable. 2.1 Neural Networks Neural Networks are a popular form of controller for robot locomotion [6,7,8,9,10]. These controllers complement the biological systems believed to be responsible for walking movement in humans and other animals. This form of control allows application of the knowledge learned from neuroscientists studying rhythmical controllers in animals [8]. Neural Networks consist of a set of interconnected processing element nodes whose functionality is based on the animal neuron [11]. Neurons process information by summing the signals that appear at the node’s inputs. Each of the input signals is multiplied by a weight to simulate different input strengths. The weighted sum is then passed through an activation function, which will produce an output if the transformed sum passes a calculated threshold [11]. Traditionally, artificial neurons have been idealized for the sake of mathematical tractability [7]. One of the simplest, and most commonly implemented neuron models, is the sigmoidal neuron [6]. This neuron is a simple extension of the binary neuron model to express a continuous softened step-function. The equations governing the output of the sigmoidal neuron are given below.
yi = # j =1 w ji S j n
(2.1)
230230A. Boeing and T. Bräunl
Si =
1 1 + e − yi
(2.2)
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -10
-8
-6
-4
-2
0
2
4
6
8
10
Fig. 1. Sigmoid Function
There are many extensions possible to this model, and many have been applied to the area of robotic locomotion. 2.2 Splines Splines are piecewise polynomial functions expressed by a set of control points [12] . There are many different forms of splines, each with their own attributes. There are two desirable properties for a spline to posses. Continuity, so that the generated control signal translates to smooth velocity and acceleration changes. Locality of the control points, to reduce the influence of alterations of one control point to the overall shape of the spline. The Hermite spline is expressed by the equations given in (2.3). The curve generated from a Hermite spline passes through the control points that define the spline. Thus, a set of predetermined points can be smoothly interpolated by simply setting the predetermined points as the control points for the Hermite spline. Hermite splines feature both the properties of locality and continuity. However, the disadvantage of the Hermite spline is that the control point tangent values must be specified. The function used to interpolate the control points, given starting point p1, ending point p2, tangent values t1 and t2, and interpolation parameter s, is shown below:
Evolution of Locomotion Controllers for Legged Robots
f ( s ) = h1 . p1 + h2 . p 2 + h3 .t1 + h4 .t 2
231 231
(2.3)
where
h1=2s3-3s2+1 h2=-2s3+3s2 h3=s3-2s2+s h4=s3-s2 0 ≤ s ≤1
3 Evolution Evolutionary algorithms are a form of search and optimization algorithms, which make use of some of the principles of organic evolution [13]. These types of algorithms typically specify what is to be done, rather than how the task should be accomplished [13]. Although evolutionary algorithms are only a basic approximation to the biological evolutionary process in reality, they have been proven to provide a powerful means of problem solving [6,7,3,9,14]. 3.1 Evolving Controllers Genetic algorithms were applied to the evolution of neural controllers for robot locomotion by numerous researchers [6,7,3,9,14]. Lewis et al. [9] successfully generated locomotion patterns for a hexapod robot using a simple traditional genetic algorithm with one point crossover and mutate. Isjpreet [14] evolved a controller for a simulated salamander using an enhanced genetic algorithm. The large number of experiments in this area clearly indicates that genetic algorithms are capable of evolving neural controls which can describe legged robot locomotion [6]. The genetic programming approach has been shown to successfully generate locomotion patterns for various control strategies. Banzhaf et al. [15] demonstrated the pure genetic programming approach to develop assembly programs for robot control. The system was then expanded to control a hexapod robot [16] using a BSpline based approach. Lewis also applied genetic programming to his neural controller [10]. This demonstrated that both the genetic programming approach and the genetic algorithm approach should both be capable of evolving adequate control systems for legged locomotion [10,14]. Parker et al. [17] explored the use of cyclic genetic algorithms for locomotion control of a small hexapod robot. The system demonstrated that the cyclic nature
232232A. Boeing and T. Bräunl
needed to generate the oscillatory motions necessary for legged robot locomotion could be abstracted from the control system and transferred and encoded into the genetic algorithms chromosomes. 3.2 Genetic Algorithm A common implementation of an evolutionary algorithm is the Genetic Algorithm (GA) [13]. As with most evolutionary algorithms the Genetic Algorithm is based on Darwin’s theory of natural selection, ensuring the survival of the fittest. The genetic algorithm operates on a set of encoded variables representing the parameters for the potential solution to a problem. The parameters (or genes) are combined together to form a string of values, referred to as a chromosome [18]. Each of these possible solutions are then assigned a fitness value according to how well it solves the problem. The better solutions are then selected to “reproduce” with other solutions, generating a new set of chromosomes which have inherited features from the chromosomes they were created from. The least fit (worst solutions) are less likely to be selected for reproduction, and thus eventually are removed from the set of chromosomes on which the algorithm operates. The basic methodology for the genetic algorithm is presented below: 1. Randomly initialize a population of chromosomes 2. While the terminating criteria has not been satisfied a) Evaluate the fitness of each chromosome b) Remove the lower fitness individuals c) Generate new individuals, determined by a certain selection scheme, utilizing selected operations. Each iteration of these steps creates a new population of chromosomes. The total set of chromosomes at one iteration of the algorithm is known as a generation. As the algorithm progresses it searches through the solution space, refining the solutions to find one which will fulfill (or come as close as possible to fulfilling) the desired criteria, as described by the fitness function. 3.3 Operators The operators determine the method in which one or more chromosomes are combined to produce a new chromosome. Traditional schemes utilize only two operators: Mutate, and Crossover [18]. Crossover takes two individuals and divides the string into two portions at a randomly selected point inside the encoded bit string. This produces two “head” segments and two “tail” segments. The two tail segments for the chromosomes are then interchanged, resulting in two new chromosomes where the bit string preceding the selected bit position belongs to one parent, and the remaining portion belongs to the other parent. The mutate operator randomly selects one bit in the chromosome string, and inverts the value of the bit. Extensions on these operators are possible by interpreting the chromosome as a string of non-binary values (such as 8 bit integer values, or 32 bit floating point
Evolution of Locomotion Controllers for Legged Robots
233 233
values). Two operators commonly used that rely on this interpretation of the chromosome are the Non-Binary Average, and the Non-Binary Creep operators [19] . Non-Binary Average interprets the chromosome as a string of higher cardinality symbols and calculates the arithmetic average of the two chromosomes to produce the new individual. Likewise Non-Binary Creep treats the chromosomes as strings of higher cardinality symbols and increments or decrements the values of these strings by a small randomly generated amount [19]. 3.4 Selection Schemes In the natural world the organisms which reproduce the most before dying will have the greatest influence on the next generation. In order to simulate this effect in the genetic algorithm a selection scheme is used. The selection scheme determines which individuals of a given population will contribute to form the new individuals for the next generation Roulette Wheel selection and Tournament selection are two commonly used selection schemes [18]. In roulette wheel selection the chance for a chromosome to reproduce is proportional to the fitness of the entity. Thus, if the fitness value returned for one chromosome is twice as high as the fitness value for another, it is then twice as likely to reproduce. Although genetic algorithms will converge to a solution if all of the chromosomes reproduce, it has been shown that by duplicating unchanged copies of the chromosomes future generations will generally have a significant increase in the convergence rate towards the solution. The implemented genetic algorithms attributes are given in Table 1. Table 1. Operators attributes
Operator Name Crossover Mutate Random Replacement Inversion Average Creep
Probability of use 35% 10% 10% 5% 30% 10%
3.5 Staged Evolution A number of possibilities exist in enhancing the performance of a genetic algorithm. Staged evolution is based on the concept of behavioral memory, and increases the convergence rate by introducing a staged set of manageable challenges [10] . Initially limiting the search to a subset of the full solution space enables an
234234A. Boeing and T. Bräunl
approximate solution to be determined. Incrementally increasing the complexity of the problem will increase the solution space, providing the possibility of increased performance as further refinements of the solution are possible. Applying this strategy to a particular problem task, requires that the tasks is capable of being split into further smaller sub-tasks which can be solved in order to contribute to the overall solution. 3.6 Fitness Functions Each problem to be solved required a unique fitness function describing that problem. Given a particular chromosome a fitness function must return a numerical value indicating the appropriateness of the solution with respect to the overall goal [20] . For some applications, such as function optimization problems, the fitness function will simply return the value of the function itself. However, for many applications there is no straightforward performance measurement of the goal, and thus it must be expressed as a combination of the desired factors. The fitness function used to evaluate the robot during the initial phases of evolution was evaluated purely from the distance the robot traveled forward, minus the distance that the robot’s center of gravity lowered. During later phases of evolution the average speed at which the robot moved, and the distance that the robot wavered from the desired path were incorporated. Finally, the distance that the robot was at termination from its desired terminating point was taken into consideration.
4 Simulation There are a number of advantages offered when evolving a system under a simulated environment. Firstly, the risk of damaging the robot hardware during evolution is removed. Secondly, simulation can be performed in far less time than that which is required for a robot to physically perform the desired gait. Simulators also provide the experimenter with the ability to easily modify the environment and model the interaction between objects. Dynamechs [21], the simulation library utilized is a free, extensible and portable mechanical environment simulator. Dynamechs calculates all environment variables and all forces generated by the actuators, in order to describe the movement of all bodies in the system. The robots are expressed by multiple chains linked to a mobile base described with modified Denavit-Hartenberg parameters. The simulation library allows configurable integration step sizes and offers a number of integrators from first order Euler to fifth order Runge-Kutta. This allows configuration of the accuracy and speed of the simulation. The fitness evaluation functions needed to operate the genetic algorithm were added to the package, along with the simulated spline control system and the necessary sensors.
Evolution of Locomotion Controllers for Legged Robots
235 235
5 Spline Control Although a neural controller offers a number of advantages for control, a spline controllers provides a simple and fast method for evolving control. The implemented spline controller consists of a set of joined Hermite splines. One set contains robot initialization information, to move the joints into the correct positions and enable a smooth transition from the robot’s standing state to a travelling state. The second set of splines contains the cyclic information for the robot’s gait. Each spline can be defined by a variable number of control points, with variable degrees of freedom. A pair of start and cyclic splines corresponds to the set of control signals required to drive one actuator within the robot. Cubic Hermite splines were implemented in the controller as they offer a number of desirable characteristics over other splines. The major advantage of the Hermite spline is that the curve passes through all the control points. As the position and tangent are explicitly specified at the ends of each segment, it is easy to create a continuous curve. An example of a simple spline controller is illustrated in Figure 2. Each spline indicates the controllers output value for one actuator.
Fig. 2. Evolved spline controller for a three-jointed limb
The number of control points required for this controller is given by equation (5.1).
a.(i + c) where, a i c
(5.1)
is the number of actuators is the number of initialization control points is the number of cyclic control points
In order to incorporate sensor feedback information into the spline controller, another dimension was added. This resulted in a set of control surfaces for each actuators cyclic information. The initialization splines were not extended as it was assumed that any robot would be set-up on flat terrain. However, the initialization
236236A. Boeing and T. Bräunl
splines can be easily extended to contain information dependent on the terrain’s gradient through a similar method to that used to extend the cyclic splines. The number of control points required for the extended controller is given by equation (5.2). Extending the controller in this form, significantly increase the number of control points required.
a.(i + c.s )
(5.2)
Where, s is the number of control points used to describe the feedback
Fig. 3. Spline controller for a single actuator with feedback
The example controller used an inclinometer to provide the sensory feedback. Thus the controller’s output at any stage is expressed in terms of the current cycle time, and the angle read from the inclinometer. This allows the controller to correct for undesired tilting during normal walking, and also allows for movement over rough terrain. Figure 3 illustrates a resulting control surface for one actuator. 5.1 Spline Controller Encoding In order to evolve the spline controller with a genetic algorithm, the controller’s parameters needed to be encoded into chromosome representations. To enable support for staged evolution the controller needs to be specifically designed such that the evolution can proceed in this manner. The encoded spline controller treated each control point value as an 8 bit fixed point number. In the initial phase
Evolution of Locomotion Controllers for Legged Robots
237 237
of evolution the control point locations within the walking cycle time were equally distributed. This provided each control point with only one degree of freedom, and reduced the available solution space but also significantly reduced the complexity of the chromosome required to describe the controller. In the following stage of evolution, the equally distributed time constraint was removed, providing the control points with an additional dimension of freedom. Finally the tangent values for the control points in the spline were added to the chromosome, allowing final refinement of the solution.
6 Results A number of spline controllers have been evolved for the control of various legged robots. An example of a spline controller which generated locomotion without feedback is shown in Figure 4. The walk is maintained for 10 seconds, before the robot collapsed.
Fig. 4. Walking without Feedback
Fitness vs Generation for Extended Spline Controller 120 100
Top Fitness
60
Average Fitness
40 20
657
616
575
534
493
411
452
370
329
288
206
247
165
83
124
1
0 42
Fitness
80
Generation
Fig. 5. Genetic Algorithm Fitness
238238A. Boeing and T. Bräunl
The increasing fitness function for the control parameters evolved for the gait depicted in Figure 6 is illustrated above. The best and average fitness values increase linearly during the initial phases of evolution. As the algorithm begins to converge towards an appropriate solution the fitness values for the population increase rapidly. The algorithm terminates once it has refined the solution after 500 generations. Incorporating sensory information to provide feedback to the control algorithm overcame the difficulties in maintaining a robots walk. The addition of feedback allowed for continues walking to be achieved. However, integrating feedback significantly impacts on the number of parameters required in the controller. The controller utilized for the gait generation in Figure 4 consists of four initialization control points, and four cyclic control points, with six controlled actuators. Thus, the total chromosome size to represent the complete controller is 48 bytes during its first phase of evolution. The expansion of this controller to enable feedback that samples the sensor with four control points results in an encoded controller size of 120 bytes. This represents a significant increase in the solution space that the genetic algorithm needs to search. Seeding the genetic algorithm with the previously evolved splines resulted in uniform spline surfaces that were capable of expressing the initial movements required in bipedal locomotion. Approaching the extended controller through the staged evolutionary method allowed the controller to be developed in under one hundred hours on an 800MHz Pentium 3 system running Windows NT. The use of an inclinometer also allowed the robot to detect alterations in the terrain. Thus the resulting controller was capable of expressing bipedal gaits which allowed the robot to maneuver over non-uniform terrain.
Fig. 6. Walking over terrain Gaits for non-bipedal legged robots were also successfully evolved utilizing the spline control system. The most successful gait evolved for the tripod robot is illustrated in Figure 6. The robot achieves forwards motion by thrusting its rear leg towards the ground, and lifting its forelimbs. The robot then gallops with is fore limbs to produce a dynamic gait.
Fig. 7. Tripod Gait
Evolution of Locomotion Controllers for Legged Robots
239 239
7 Conclusion and Outlook The proposed evolutionary control system was shown to generate locomotion patterns for a bipedal robot. A system for integrating sensory information into a spline controller was presented. Also, the possible improvements to a robot’s control system when provided with sensory feedback was demonstrated. Future extensions of the system could provide a method for directly downloading the control system to a physical biped robot. Further evolution of the control system on the physical robot could then be used to overcome discrepancies between the simulated and physical world. Another useful extension could be to evolve the robot’s morphology as well as the controller. This would allow the system to suggest structural alterations to robots, which could assist the robot in achieving its desired purpose. Providing control over the placement of sensors on the robot would allow the evolutionary system to optimize the sensor locations such as to provide the optimal feedback to the controller. This should result in further improvements to the adaptability and robustness of the generated control system.
References [1] M. H. Raibert. Legged Robots That Balance Cambridge, MA: The MIT Press, 1986 [2] G. S. Hornby, S. Takamura, J.Yokono, O. Hanagata, T. Yamamoto, M. Fujita, “Evolving Robust Gaits with AIBO” in IEEE International Conference on Robotics and Automation. pp. 3040-3045, 2000. [3] J.Ventrella,“Explorations in the Emergence of Morphology and Locomotion Behavior in Animated Characters “ in R.A. Brooks and P. Maes (eds.) Artifical List IV, pp. 436441, (MIT Press, Cambridge, MA 1994). [4] J. Busch, J. Ziegler, C. Aue, A. Ross, D. Sawitzki, W. Banzhaf, “Automatic generation of Control Programs for Walking Robots Using Genetic Programming” in European Conference on Genetic Programming (EURO GP), 2002. [5] D. Arnold “Evolution of Legged Locomotion”, MSc. thesis, Simon Fraser University, School of Computing Science, 1997. [6] M. Krueger “Genetic Algorithms can Evolve Neural Network Controllers for Robot Locomotion”, BE thesis, University of Western Australia, 2001. [7] R. Reeve “Generating walking behaviors in legged robots”, Ph.D. thesis, University of Edinburgh, 1999. [8] P. Wallen, O Ekeberg, A Lansner, L Brodin, H Traven, and S Grillner. “A computer based model for realistic simulations of neural networks. II. The segmental network generating locomotor rhymicity in the lamprey.” Journal of Neurophysiology, pp. 1939-1950, 1992. [9] M. Lewis, A. Fagg, G. Bekey. “Genetic Algorithms for Gait Synthesis in a Hexapod Robot.” in Recent Trends in Mobile Robots (World Scientific, New Jersey), pp. 317331. 1994.
240240A. Boeing and T. Bräunl
[10] M. Lewis, A. Fagg, A. Solidum, “Genetic Programming approach to the Construction of a Neural Network for Control of a Walking Robot” in IEEE International Conference on Robotics and Automation (Nice, France) pp 2618-23, 1992. [11] K. Gurney, “Neural Nets”, UCL Press Ltd, 2002. [12] Bartels, R. H., Beatty, J. C. and Barsky, B. A. “An Introduction to Splines for use in Computer Graphics and Geometric Models” Morgan Kaufmann, 1987. [13] J.H. Holland, "Adaptation in Natural and Artificial Systems", MIT Press, 1975. [14] A. J. Ijspeert, “Evolution of neural controllers for salamander-like locomotion” in Proceedings of Sensor Fusion and Decentralised Control in Robotics Systems II pp. 168179, 1999. [15] P Nordin, W Banzhaf, “An On-Line Method to Evolve Behaviour and to Control a Miniature Robot in Real Time with Genetic Programming” in Adaptive Behaviour, vol 5 no 2 pp 107-140, 1997. [16] J. Ziegler, W. Banzhaf, “Evolution of Robot Leg Movements in a Physical Simulation” in K. Berns and R. Dillmann, (Eds.), Proceedings of the Fourth International Conference on Climbing and Walking Robots, CLAWAR , (Professional Engineering Publishing 2001). [17] G.B. Parker, D.W. Braun, I. Cyliax, “Learning Gaits for the Stiquito” Proceedings of the 8th International Conference on Advanced Robotics (ICAR'97). (pp. 285-290). 1997. [18] D. Beasley, D. R. Bull, and R. R. Martin, "An Overview of Genetic Algorithms: Part I, Fundamentals" University Computing, vol. 15, no. 2, pp. 58-69, 1993. [19] D. Beasley, D. R. Bull, and R. R. Martin, "An Overview of Genetic Algorithms:Part 2, Research Topics" University Computing, vol. 15, no. 4, pp. 170-181, 1993. [20]F. Busetti, ”Genetic algorithms overview”. Available online: http://citeseer.nj.nec.com/464346.html [Accessed: 28th October 2002] [21]DynaMechs (Dynamics of Mechanisms): A Multibody Dynamic Simulation Library [Online] http://dynamechs.sourceforge.net/ [Accessed: 7th March 2003]
241
Gait Planning for Soccer-Playing Humanoid Robots Zhe Tang1,2, Changjiu Zhou2, and Zengqi Sun1 1
Department of Computer Science and Technology, Tsinghua University, Beijing, P.R. China 2 School of Electrical and Electronic Engineering, Singapore Polytechnic, 500 Dover Road, Singapore http://www.robo-erectus.org
Abstract. Gait planning for humanoid walking and penalty kicking, the two fundamental tasks for the current humanoid soccer competition, is addressed in this paper. First, an overview of some basic research issues in the field of humanoid robotics is presented. Humanoid walking gait planning in both joint space and Cartesian space is then discussed in a detailed way so as to make a clear comparison. By introducing some new concepts such as maximum kicking range and effective kicking range, a novel gait planning method for humanoid penalty kicking is also presented. All the proposed gait synthesizing methods have been successfully implemented on a soccer-playing humanoid robot, named RoboErectus, developed in Singapore Polytechnic.
1 Introduction The desired goals of humanoid robotics research are to develop robots that are able to coexist and collaborate with humans, to replace humans in some of their roles in the interaction between humans and robots, and to extend human capabilities for interacting with environments[1]. So the motion planning for humanoid robot is not only limited to walking, but also include many other tasks, such as kicking, running, jumping and so on. Gait synthesis has been a very important part of work in humanoid robot motion planning. Aiming at participating in robot soccer competitions[2, 3], we have developed a series of humanoid robots named Robo-Erectus in the ARICC Centre of Singapore Polytechnic [4]. Humanoid walking and penalty kicking (PK) are two basic tasks for the current humanoid soccer playing games. We will focus on some basic research issues in the field of gait synthesis for soccer-playing humanoids in this paper. Vukobratovic et al.[5,6] have considered locomotion directly. The fundamental aspect of their approach is the adaptation of human walking data to prescribe T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 241−262, 2004. Springer-Verlag Berlin Heidelberg 2004
242
242
Z. Tang, C. Zhou, and Z. Sun
motion of the lower limbs and the application point of the resultant of the reaction forces among lower limbs and the walking surface. Furthermore, repeatability conditions are added to impose further symmetry conditions on the motion. Vukobratovic and his colleagues adopted a two-stage control synthesis. In the first stage, the control synthesis has to ensure the system’s motion in the absence of any disturbance along the exact nominal trajectories calculated in advance. In the second stage external perturbation is involved, the nominal should be realized with assistance from an additional control force. In this paper, we will only focus on the first stage, designing a desired trajectory for humanoid robot without considering external perturbation. Furusho and Sano[7] achieved smooth 3D walking through a sensor based control on a biped with nine links. They considered the motion in the lateral plane as a regulator problem with two equilibrium states. In the sagittal plane, where the walking control was based on the body speed in forward movement, they made the body movement close to the desired smooth speed by controlling the ankle torque of the support leg. The sole and ankle driving actuators underwent a force/torque feedback control based on the sensor information. An inverted pendulum mode was adopted. Mita and et al.[8] used a linear optimal state feedback regulator to control a seven-link biped. Their work proved the usefulness of typical modern control theory for studies on biped locomotion. Miura and et al.[9] approximated the biped motion of the single support phase as an inverted pendulum, and also gave trajectory planning and stability analysis in their work. Furthermore, their methods were implemented on their two biped robots. Zheng and Hemami[10] have addressed the impact effects of a biped contact with the environment. They showed that the impact may cause large internal impulsive forces in the body, and that the biped can control its initial posture in order to reduce the impulsive component of the internal constraint forces. They also proposed a linear velocity and nonlinear position feedback control method to slow down the biped motion so that it finally reaches a static posture after hitting the ground. In addition, Zheng and Shen[11] developed a biped robot, SD-2, which could climb sloping surfaces using static stabilization criteria. Using a force sensor underneath the foot, their robot was able to detect the transition of the supporting terrain from a flat floor to sloping surface of up to 10 deg. They were the first to propose a biped walking algorithm on a slope surface. Shih[12,13] proposed a gait synthesis of ascending and descending stairs for his 7 DOF biped. The efficiency of the gait synthesis was supported by the biped implementation. However, his method is only applicable to his specific biped: variable length legs and translatable balance weight in the body. Furusho and Masubuchi[14] have proposed some novel approaches to synthesize a five-element biped. They adopted a hierarchical control structure for lower level control. A reduced order model of biped locomotion, which is based on the concept of local feedback, was presented. The results were verified experimentally by using a walking machine and periodicity was demonstrated for several walking patterns. Miyazaki and Arimoto[15] divided biped locomotion into two modes, slow mode and fast mode. The joint motion trajectories were obtained in the two modes respectively. They used the inverted pendulum model to approximate the biped global locomotion. Their control algorithm was verified by
Gait Planning for Soccer-Playing Humanoid Robots
243 243
computer simulation. Kajita and Yamaura[16] used the term “potential energy conserving orbit” to describe a particular class of biped motion trajectories, which was based on an ideal biped model. They proposed a potential energy conserving orbit. Huang and et al.[17] presented a practical method for biped trajectory planning using the third-order Spline interpolation. However, they did not consider the energy consumption, and most of the interpolation positions were decided by experience or trial and error. The rest of this paper is organized as follows. In the following section, for the gait synthesis of humanoid walking, we propose a novel approach to generate trajectories in both Cartesian Space and Joint Space from a robot manipulator perspective. We clarify the difference between the traditional robot manipulators and humanoid robot manipulators. Furthermore, we will also compare the gait planning in the two different spaces. In Section 3, we give a detailed gait synthesis for humanoid penalty kick, including kicking cycles and some basic research issues in penalty kicking. We also propose some new concepts such as max kicking range and effective kicking range. Finally, some remarks on gait planning for soccer-playing humanoid robots are given in Section 4.
2 Gait Synthesis for Humanoid Walking 2.1 Classification Different gait planning methods have been developed for humanoid walking pattern generation. 1. Control goal based method. Researchers have addressed a wide variety of control goals for humanoid walking, such as stable walking on uneven terrain[17], energy efficient walking[16], walking on ascending and descending stairs[12,13]. 2. Control model based method. There are model-based and model-free walking planning schemes. The model-based method generates a walking gait from a mathematical model[5-17]. The model-free method usually makes use of intelligent techniques, such as machine learning[18-24], genetic algorithms[25,26], neural networks[22,23,27] and fuzzy control[19-21]. 3. Stability criteria based method. There are two different kinds of walking, namely dynamic walking and static walking for humanoid robots. Most current dynamic walking schemes use zero moment point (ZMP) as stability criterion[1,17,19,20,21,24,27]. Static walking[11,12,13,22,23] is implemented in slow movement, and utilizes center of gravity (CG) as stability criterion. 4. Trajectories generation space based method. Humanoid walking gait can be planned in either joint space or Cartesian space. Both Cartesian space walking gait planning[12,13,17] and joint space walking gait planning[18,28] will be discussed in this paper. We will also give a clear explanation and comparison of the gait planning methods in both joint space and Cartesian space in this section.
244
244
Z. Tang, C. Zhou, and Z. Sun
2.2 Manipulator Although trajectory generation, one of most important issues in humanoid robots, is different from path planning in manipulator with a fixed base [5], they share many vital attributes. Both the manipulator and the humanoid robot use the same dynamic Newton-Euler equation as follows: ..
.
τ = D ( q ) q + C ( q, q ) + g ( q )
(2.1)
where q is the n × 1 vector of generalized joint displacements, τ is the n × 1 vector of applied generalized forces, D(q ) is the n × n mass matrix, .
C (q, q) is an n × 1 vector of centrifugal and Coriolis terms, and g (q ) is an n × 1 vector of gravity terms. (For details of parameters please refer to [14])
A humanoid robot can be considered as one or several independent manipulators under certain conditions. In this way, many theories in robotic manipulator can also be used for humanoid robots. In this paper, we will demonstrate how manipulator motion planning method can be used for humanoid motion planning. There are two spaces in trajectory generation for manipulator motion: Cartesian space and joint space[29,30,31]. We will present the details of trajectory generation for humanoid robot walking in both spaces. One of the two main differences between the two spaces is that the normal robot manipulator assumes a static base frame (i.e. world coordinate system) which will never move. In Cartesian space, however, the humanoid robot motion involves the movement of every joint. We have to find or assume a base frame to apply the manipulator methods. The other difference is that humanoid robots introduce unpowered degree-of-freedom (DOF), which greatly affects the kinematics of the humanoid robot[5]. 2.3 Stability Criterion
There are two kinds of humanoid robot stability: static stability and dynamic stability. Static stability neglects the dynamics of the robots and takes centre of gravity (CG) as stability criterion, so this stability is applicable only if the robot moves very slowly. Most researches have adopted dynamics stability. The most common dynamic criteria are center of pressure (CP), zero moment point (ZMP) and foot rotation indicator (FRI), which are briefly described as follows[32]: 1. The CP is a point on the foot/ground surface where the net ground reaction force actually acts. 2. The ZMP is the point on the floor at which the moment generated by the reaction force and the reaction torque are balanced. 3. The FRI is the point on the floor/ground surface, inside or outside the base of support, where the net ground reaction force would have to act to keep the foot stationary.
Gait Planning for Soccer-Playing Humanoid Robots
245 245
The ZMP is identical to CP in a different definition. The ZMP was originally introduced for humanoid robot stability in 1969[6]. Using this criteria has yielded many impressive results[1,19,20,21,27,28]. We adopted the ZMP control for our humanoid walking gait synthesis. The ZMP can be calculated from the following equations:
xzmp =
$$ #in=1mi ($$zi + g)xi − #in=1mi $$xi zi − #in=1 IiyΩ iy
(2.2)
#in=1mi ($$zi + g)
$$ #in=1mi ($$zi + g) yi − #in=1mi $$yi zi − #in=1IixΩ ix yzmp = n #i=1mi ($$zi + g) where
$$ and mi is the mass of link i, I ix and I iy are the inertial component, Ω ix
$$ are absolute angular velocity components around the x-axis and the y-axis at Ω iy the center of the gravity of link i, g is the gravitational acceleration, ( xzmp , y zmp , 0 ,) is the coordinate of the ZMP, and ( xi , yi , zi ) is the coordinate of the mass center of link i on an absolute Cartesian coordinate system. The angular accelerations
$$ are small and have little effect on the $$ and Ω Ω iy ix
ZMP position compared with the link acceleration, so the two angular acceleration is normally neglected. 2.4 Walking Phases Humanoid robot walking includes two walking phases, namely, double-support phase and single-support phase[5]. But for each leg, the walking process is composed of a stance phase and swing phase. The double-support phase is the overlap period of two legs’ swing phase. During the double-support phase, the CG or ZMP moves from one foot to another foot, this motion is achieved mainly in lateral plane, to simplify our analysis, our motion planning is restricted to the sagittal plane. Fig.1 gives a walking cycle starting from kth step, where Tc is period for one walking step, Td is duration for double-support phase, and kTc+ Tm corresponds to the point when the swing foot reaches its highest point. 2.5 Trajectory Generation in Cartesian Space Humanoid walking is a periodic motion which alternates between the doublesupport phase and the single-support phase. During the double-support phase, both of the humanoid robot feet are in contact with ground. During the single-support phase, only one foot is in contact with ground to support the humanoid, the other leg swings forward. Our previous work[33] shows that the humanoid robot is prone
246
246
Z. Tang, C. Zhou, and Z. Sun
to falling at the switching point between the double-support phase and the singlesupport phase. In order to control a humanoid robot’s movement, we should first generate the trajectories of humanoid joints. The trajectories should have first-order and second-order derivative continuity. First order derivative continuity guarantees smoothness of joint velocity, while second order guarantees smoothness of acceleration or torque of joint. There are two ways to represent the joint position, Cartesian space or joint space. The current joint trajectory generation falls into these two spaces. Furthermore, it can be proved that the first-order and second-order derivative continuity in Cartesian space can guarantee the first-order and second-order derivative continuity and vice visa. Because the two spaces conversion is a one-toone mapping. In fact, any trajectory generation in Cartesian space has to convert to joint space to control each joint.
Fig. 1. Walking phase.
The trajectory generation in Cartesian space for the humanoid robot can be summarized as follows: (the humanoid robot model is shown in fig.2. This model is a simplified one which neglects the foot and hip, and only considers one leg because the other leg is symmetric.) (1) Defining the position of the hip position (xh(t),yh(t)) at some given time points, then generating the trajectory of the hip using interpolation by polynomials. (2) Defining the position of the ankle position (xa(t),ya(t)) at the switching points and some other interpolation points to constrain the walking pattern. From these positions, the trajectory is generated using polynomial interpolation. (3) Based on the above positions of hip and ankle, the knee position(xk(t),yk(t)) is obtained using the following geometric formulas:
Gait Planning for Soccer-Playing Humanoid Robots
{
x = x + l cos tan −1 $% ( y − y ) ( x − x ) +, − k a 2 a h a & h
247 247
(2.3)
cos −1 $(l 2 + ( x − x )2 + ( y − y )2 − l 2 ) / %& 2 h a h a 1 + . 2l ( x − x )2 + ( y − y )2 , ) / 2 h a h a - 0
{
y = y + l sin tan −1 $ ( y − y ) ( x − x ) + − k a 2 a h a -, &% h cos−1 $(l 2 + ( x − x ) 2 + ( y − y )2 − l 2 ) / %& 2 1 h a h a + . 2l ( x − x )2 + ( y − y )2 , ) / 2 h a h a - 0
(4) From the positions of the joints, the joint angles can be obtained from the following inverse kinematics equation.
θ h = − tan −1 [ ( yk − yh ) ( xk − xh )] ± π / 2
(2.4)
θ a = −(π / 2 ± tan −1 [ ( yk − ya ) ( xk − xa )]) θk = θa + θh
Fig. 2. Simplified model of the humanoid robot
For the humanoid robot with a variable-length leg[12,13,34,35], there is no need generate knee joint trajectory. This kind of robot simplifies control architecture at the expense of more complex mechanical structure.
248
248
Z. Tang, C. Zhou, and Z. Sun
2.6 Trajectory Generation in Joint Space
For the trajectory generation in joint space, we take the two legs as two manipulators. Then the trajectory generation for humanoid motion can be considered as manipulator trajectory generation. The first step for manipulator planning is to define the base frame. Based on humanoid walking phases, we define the two manipulations as follows: (1) During the double-support phase and left-leg stance, the left foot is the base frame of the left manipulator, and the hip is the base frame of the right manipulator. (2) During the right-leg stance phase, the right foot is the base frame of the right manipulator and the hip is the left manipulator’s base frame. We name the foot base frame as the first base frame (BF1), and the hip base frame as the second base frame (BF2). The corresponding legs using the frame are named as the first-leg and the second-leg respectively. The joint angles are defined as following:
' qasi * θ ai (t ) = (qami *q ) aei ' qhsi * θ hi (t ) = (qhmi *q ) hei
t = kTc
(2.5)
t = kTc + Tm t = (k + 1)* Tc t = kTc t = kTc + Tm t = (k + 1)* Tc
where i = 1, 2 .Based on the above analysis, θ as1 = θ ae 2 , θ hs1 = θ he 2 ,
θ as1 = θ ae 2 ,
and θ hs 2 = θ he1 . Furthermore, because the motion of the humanoid robot proceeds to double-support phase in the lateral plane after the single-support phase, the angle velocity in sagittal plane is supposed to be zero, at the start and end of the double support-phase:
θ$ai (kTc) = θ$ai ((k + 1)Tc) = 0
(2.6)
θ$hi (kTc) = θ$hi ((k + 1)Tc) = 0 where i = 1, 2 . From the above via points and start and end conditions, smooth trajectories of the hip and two ankles in joint space can be generated using the third order Spline interpolation. These trajectories can achieve first and second derivate continuity in joint space.
Gait Planning for Soccer-Playing Humanoid Robots
249 249
2.7 Experiment Results
To synthesize a gait for controlling a real humanoid robot, we have developed a gait generator platform see Fig.4 using Visual C++ under Windows 2000. This generator decomposes the humanoid motion into three planes: sagittal, frontal and transverse. Most of humanoid research assumes that the motions in the three planes are independent, which greatly simplifies humanoid motion analysis. The humanoid robot parameters and other values are shown in Fig. 5 and Table1. !
"
Fig. 3. Base frame
2.7.1 Planning in Cartesian Space Let the foot landing angle be Qf = 5 deg and leaving angle be Qb=5 deg respectively. We set the walking step length Ds = 5cm, Hao = 2cm, Lao = 4 cm, and walking cycle 2*Tc = 1.8s, a walking cycle consists of two steps, doublesupport phase Td = 0.2s. The ankle position at the following via points are defined, After defining the via points position, the ankle trajectory is generated by the third-order Spline interpolation. The hip trajectory can be obtained in a similar way. Then the knee position can be obtained from Eq.(2.3). Table 1. Robot parameters Length (cm)
Lhip
Lthigh
Lshank
Lan
Laf
Lab
10
15
15
0
1.5
1.5
250
250
Z. Tang, C. Zhou, and Z. Sun
Table 2. Walking parameters Time (s)
Td
Tm
Tc
0.2
0.6
1
Fig. 4. Simulator interface
Fig. 5 Robot parameters and coordinate
Gait Planning for Soccer-Playing Humanoid Robots
251 251
2.7.2 Planning in Joint Space When the hip and ankle positions at the via points are given, then by Eq.(2.4) the joint angles can be obtained as shown in Table 3. A smooth joint angle trajectory in the joint space is generated using the third order Spline interpolation. However, the definition of the via points is not a trivial task, we normally get the positions through trial-error method and experience. Table 3. Via points value (rad) qas1
qam1
qae1
qhs1
qhm1
qhe1
0.559
0.906
-.02
0.038
0.446
0.643
2.7.3 Comparison between Two Spaces The comparison of experiment results are shown in Figs.6 to 17. The motion planning in the joint space is simpler and computationally less expensive than the motion planning in the Cartesian space, because motion planning in the Cartesian space requires solution of inverse kinematics equation Eq.(2.4). Therefore, the motion planning in joint space is more suitable for on-line trajectory generation. However, the joint space motion planning does not provide straight line motion of the end-effector. Therefore, it is not applicable for humanoid kicking, because the kicking trajectory must be a straight line. From Figs.6 and 7, it can be seen that the density variance of the stick for Cartesian walking planning is smaller than that in the joint space. This indicates that the liner accelerations of every link are smaller. As a result, the trajectory of ZMP is smoother. This is also demonstrated in Figs. 16 and 17. Although the trajectory planning scheme in the joint space makes the joint angular velocity smoother, the liner velocities of every robot link contribute much more to the dynamic walking stability. From this point of view, the walking planning in the Cartesian space is desirable. However, one of drawback for motion planning in the Cartesian space is that it is difficult to predict the joint’s angular velocity during on-line trajectory planning in the Cartesian space[30].
Fig. 6 Consecutive walking for planning in the Cartesian space
252
Z. Tang, C. Zhou, and Z. Sun
252
Fig. 7 Consecutive walking for planning in the joint space
1.3
1.1 Ankle Knee Hip
1.1
0.9
1
0.8
0.9
0.7
0.8
0.6
0.7
0.5
0.6
0.4
0.5
0.3
0.4 0.3
0.2
0
0.1
0.2
0.3
0.4
0.5 t(d)
0.6
0.7
0.8
0.9
1
0.1
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
1.8
2
10
10 X-axis Z-axis
9
X-axis Z-axis
9
8
8
7
7
6
6
Ankle Position(cm)
Ankle Position(cm)
0
Fig. 9. Joint angle for planning in the joint space
Fig. 8. Joint angle for planning in the Cartesian space
5 4
5 4
3
3
2
2 1
1 0
Ankle Knee Hip
1
Angle(rad)
Angle(rad)
1.2
0
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
1.8
2
Fig. 10. Ankle position for planning in the Cartesian space
0
0
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
1.8
2
Fig.11. Ankle position for planning in the joint space
253
Gait Planning for Soccer-Playing Humanoid Robots 18
16 X-axis Z-axis
16
14
14
12 Knee Position(cm)
Knee Position(cm)
X-axis Z-axis
12
10
10
8
8
6
6
0
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
1.8
4
2
Fig. 12. Knee position for planning in the Cartesian space
0
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
2
30 X-axis Z-axis
X-axis Z-axis
25
25
20
20 Hip Position(cm)
Hip Position(cm)
1.8
Fig. 13. Knee position for planning in the joint space
30
15
15
10
10
5
5
0
253
0
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
1.8
0
2
Fig. 14. Hip position for planning in the Cartesian space
0
0.2
0.4
0.6
0.8
1 t(s)
1.2
1.4
1.6
1.8
2
Fig. 15. Hip position for planning in the joint space
2
2 ZMP CG
ZMP CG 0
-2
-2
-4
-4
Y(cm)
Y(cm)
0
-6
-6
-8
-8
-10
-10
-12
5
6
7
8
9
10 X(cm)
11
12
13
14
15
Fig. 16. Fig. X ZMP and CG for planning in the Cartesian space
-12
2
4
6
8
10
12
14
X(cm)
Fig. 17. Fig. X ZMP and CG for planning in the joint space
16
254
254
Z. Tang, C. Zhou, and Z. Sun
3 Humanoid Penalty Kicking For kicking planning in this paper, we assume that the supporting foot keeps static while the kicking leg moves towards the ball. Furthermore, the velocity of the kicking leg at the start and end points is zero, so the moments from reciprocating links are negligible. In the following discussion, CG is used as criteria to evaluate humanoid static stability. Furthermore, humanoid penalty kicking requires that the kicking foot trajectory should be straight in the kicking range, which cannot achieve in joint space planning. Therefore, humanoid penalty kicking planning is conducted in the Cartesian space. 3.1 Basic Research Issues of Humanoid Kicking
3.1.1 Kicking a Ball There are many different methods for kicking a ball. As shown in Fig.18, different kicking directions and different kicking points on the ball may result in different kicking effects. Kicking in a horizontal direction is not the optimal way. But accurate kicking requires accurate information on both the ball and robot localization. To simplify the problem, we only consider kicking in horizontal direction in this paper.
Fig. 18. Kicking a ball
3.1.2 Max Kicking Range and Effective Kicking Range To achieve a longer kicking range, the humanoid robot’s hip should be kept as low as possible, because at the max kicking range point, the kicking leg stretches straight (Fig. 19). The lower the Hip Height (HH) is, the wider the max kicking range is. But the HH is subjected to the constraint that the CG should lie in the
Gait Planning for Soccer-Playing Humanoid Robots
255 255
supporting foot, because if HH is lowered, the CG moves forward. So, there exists a threshold value for the Lowest Hip Height (LHH). Based on the above analysis, the kicking leg speed is zero at the maximum kicking range point. If kicking the ball at that point, there is no momentum for the ball. We need to define another range, effective kicking range (see Fig. 20). To kick the ball into the goal, the minimum ball movement distance is required, and hence the minimum momentum for the foot can be defined. Then there exists a minimum velocity (Vmin) for the kicking foot to kick the ball into the goal. As indicated in Fig. 21, where Ts is the time when the kicking foot enters kicking range. The kicking range, in which the kicking foot velocity is greater than Vmin, is defined as effective kicking range. This effective kicking range is decided by many factors, e.g. the minimum ball movement distance, the floor property (friction) and the foot trajectory planning.
Fig. 19. Max kicking range and lowest hip height
3.2 Gait Planning for Penalty Kicking
3.2.1 Kicking Cycles The proposed kicking action can be decomposed into three steps (Fig. 22): (1) Leaning the body and bending the legs, from t = T0 to T1, (2) Lifting kicking leg, t = T1 to T2, (3) Kicking, t = T2 to T3. For the first step, leaning is to move the center of gravity (CG) to the supporting leg, bending for lifting kicking leg. From the previous discussion, to get a wider kicking range the robot needs to bend as low as possible. At the same time, the robot should lean in the frontal plane in order to move the CG to the supporting foot. During the second step, the kicking leg should lift as high as possible at same time the CG must keep within the supporting area. This position
256
256
Z. Tang, C. Zhou, and Z. Sun
can be obtained using the similar method as described above. For the kicking cycle, the ankle trajectory of kicking foot is first to be generated. At the start and end of a cycle, the kicking leg velocity is zero. This condition can be used to balance the robot.
Fig. 20. Kicking range
Fig. 22. Kicking cycle
Fig. 21. Kicking range vs. velocity
3.2.2 Ankle Trajectory for Kicking Spline interpolation is utilized to generate ankle trajectory of the kicking leg. In the kicking range, the kicking foot position in z-axis should be equal to ball radius. And we define the kicking range starts from zero point. And we assume Ts as the time for the kicking leg swing from the highest point to kicking range (xa(t) = 0). (xa(t), ya(t), za(t) ) is the position of ankle. During the kicking phase, the ankle should meet the following constraints:
Gait Planning for Soccer-Playing Humanoid Robots
' Hx * x a ( t ) = ( Lx * Rx )
t = T2
257 257
(3.1)
t = T2 + Ts t = T3
' Hz * za (t ) = (Rball + Lan *R + Lan ) ball
(3.2)
t = T2 t = T2 + Ts T2 + Ts < t ≤ T3
where ( Hx , Hy , Hz ) is the position of the kicking foot at highest point. Lx is the ankle’s x value at the point the foot enters the kicking range, Rball is the radius of the ball. Because we assume the hip keeps static in this paper, once the ankle trajectory is defined, the knee trajectory is also defined. From Eq.(3.2), it can be seen that za(t) is a constant during T2 + Ts < t ≤ T3 , this condition guarantees the kicking foot in the horizental direction. 3.3 Experiment
The kicking cycle time for this experiment is shown in Table 4. Table 4. Kicking cycle time Time (s)
T0
T1
T2
T3
Ts
0
0.3
1.8
3.3
0.9
Let Hz = 15cm, Lx = 0, Rball = 5cm, and LHH=88cm. According to the constraints (Eqs.(3.1) and (3.2)) and the parameters given in Table 1 and Table 4, an ankle trajectory can be generated as shown in Fig. 23. The kicking foot velocity is given in Fig. 24. Based on this velocity, the effective kicking range can be derived. Fig. 25 shows the three joints angular trajectories. The CG position is given in Fig. 26, which always lies in the supporting foot area, i.e. Lab
258
Z. Tang, C. Zhou, and Z. Sun
258
10
1.8
9
1.6
8
1.4
7
1.2
Velocity(m/s)
Z(cm)
6 5 4
0.8
0.6
3
0.4
2
0.2
1 0 -20
1
-15
-10
-5
0 X(cm)
5
10
15
0 0.9
20
Fig. 23. Ankle trajectory of the kicking leg
1
1.1
1.2
1.3
t(s)
1.4
1.5
1.6
1.7
1.8
4
5
Fig. 24. Velocity of the kicking foot
10
1.2 Ankle Hip Knee
1
9 8
0.8
7
0.6
Z(cm)
Angle(deg)
6
0.4
0.2
5 4
0
3
-0.2
2
-0.4
1
-0.6 0.9
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
0 -5
-4
-3
t(s)
Fig. 25. Angular trajectories of the kicking leg
-2
-1
0 X(cm)
1
2
3
Fig. 26. COG Trajectory
Fig. 27. Humanoid kicking
Gait Planning for Soccer-Playing Humanoid Robots
259 259
4 Concluding Remarks Gait planning for humanoid walking and penalty kicking in both joint space and Cartesian space has been discussed in a detailed way in this paper. By introducing some new concepts such as maximum kicking range and effective kicking range, a novel gait planning method for humanoid penalty kicking is also presented. All the proposed gait synthesizing methods have been successfully implemented on a soccer-playing humanoid robot, named Robo-Erectus, developed in Singapore Polytechnic. Research on humanoid gait planning is still in its early stage. There are still a lot of problems remain unsolved, e.g. how to plan humanoid kicking while maintain the robot’s dynamic stability. It is also noticed that many parameters need to be tuned for both walking planning and penalty kicking planning. How to further develop our humanoid robot with an on-line learning capability to perceive new behavior will be our new research line [19-21].
References [1]
[2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
K. Hirai, M. Hirose, Y. Haikawa, and T. Takenaka, “The development of honda humanoid robot,” Proc. of the International Conference on Robotics & Automation, Leuven, Belgium, May 1998, pp.1321-1326. http://www.fira.net/ http://www.robocup.org/ http://www.robo-erectus.org/ M. Vukobratovic, M. Borovac, B. Surla, D. Stokic, Biped Locomotion: Dynamics, Stability, Control and Application, Springer-Verlag, BerlinHeidelberg, 1990. M. Vukobratovic, D. Juricic, “Contributions to the synthesis of biped locomotion,” IEEE Trans. on Bio&Eng, BME-16, 1969. J. Furusho, A. Sano, “Sensor-based control of a nine-link biped,” Int. Journal of Robotics Research, Vol.9, No.2, April 1990, pp. 83-98. T. Mita, T. Yamaguchi, T. Kashiwase, and T. Kawase, “Realization of a high speed biped using modern control theory,” Int. Journal of Control, Vol. 40, 1984, pp. 107-119. H. Miura, I. Shimoyama, “Dynamic walk of a biped,” Int. Journal of Robotics Research, Vol. 3, No 2, Summer 1984, pp.60-74. Y. F. Zheng, H. Hemami, “Impact effects of biped contact with the environment,” IEEE Tans. Systems. Man. and Cybernetics, Vol. SMC-14, No. 3, May/June 1984, pp.437-443. Y. Zheng, J. Shen, “Gait synthesis for the SD-2 biped robot to climb sloping surface,” IEEE, Trans. Robotics and Automation, Vol. 6, No. 1, Feb 1990, pp. 86-96.
260
260
[12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28]
Z. Tang, C. Zhou, and Z. Sun
C.-L Shih, “Gait synthesis for a biped robot,” Robotica, Vol. 15, 1997, pp.599-607. C.-L Shih, “Ascending and Descending Stairs for a Biped Robot,” IEEE. Trans. On Systems, Man, and Cybernetics- Part A: Systems and Humans, Vol. 29, May 1999, pp. 255-268. J. Furuso, M. Masubuchi, “Control of a dynamic biped locomotion system for steady walking”, Journal of Dynamic System, Measurement, and Control., Vol. 108, June 1986, pp.111-118. F. Miyazaki, S. Arimoto, “A control theoretic study on dynamical biped locomotion,” Journal of Dynamic Systems, Measurement, and Control, Vol. 102, Dec 1980, pp. 233-239. S. Kajita, “Dynamic walking control of a biped robot along a potential energy conserving orbit,” IEEE Transactions on Robotics and Automation, Vol. 8, No. 4, Aug 1992, pp.431-438. Q. Huang, K. Yokoi, S. Kajita, K. Kaneko, H. Arai, N. Koyachi, and K. Tanie, “Planning walking patterns for a biped robot,” IEEE Trans. Robot. Automat, vol. 17, June 2001, pp.280-289. H. Benbrahim, and J. A. Framklin, “Biped dynamic walking using reinforcement learning,” Robotics and Autonomous Systems, Vol. 22, 1997, pp.283-302. C. Zhou, “Nero-fuzzy gait synthesis with reinforcement learning for a biped walking robot,” Soft Computing, Vol. 4, 2000, pp.238-250. C. Zhou, “Robot learning with GA-based fuzzy reinforcement learning agents,” Information Sciences, vol.145, 2002, pp.45-68. C. Zhou, Q, Meng, “Dynamic balance of a biped robot using fuzzy reinforcement learning agents,” Fuzzy Sets and Systems, Vol. 134, No. 1, 2003, pp.169-187. W. Salatian, Y. F. Zheng, “Gait synthesis for a biped robot climbing sloping surface using neural networks, Part 1: static learning,” Proc. IEEE Int. Conf. on R&A, Nice France, May 1992, pp.2601-2606. W. Salatian, Y. F. Zheng, “Gait synthesis for a biped robot climbing sloping surface using neural networks, Part 2: dynamic learning,” Proc. IEEE Int. Conf. on R&A, Nice France, May 1992, pp.2607-2608. C.-M. Chew, Dynamic Biped Walking Assisted by Learning, Ph.D Thesis, Dept. of Mechanical Engineering, MIT, Sep 2000. G. Capi, S. Kaneko, K. Mitobe, L. Barolli, and Y. Nasu, “Optimal trajectory generation for a prismatic join biped robot using genetic algorithm,” Robotics and Autonomous Systems, Vol. 38, 2002, pp.119-128. M.-Y. Cheng, and C.-S. Lin, “Genetic algorithm for control design of biped locomotion,” Journal of Robotic Systems, Vol. 14, 1997, pp. 365-373. A. L. Kun, and W. T. Miller, “Control of variable-speed gaits for a biped robot,” IEEE Robotics and Automation Magazine, Sep 1999, pp. 19-29. T. Furuta, T. Tawara, Y. Okumura, M. Shimizu, K. Tomiyama, “Design and construction of a series of compact humanoid robots and development of biped walk control strategies,” Robotics and Autonomous Systems, Vol. 37, 2001, pp.81-100.
Gait Planning for Soccer-Playing Humanoid Robots
[29] [30] [31] [32] [33] [34] [35]
261 261
J. Koivo, Fundamentals for Control of Robotics Manipulators, John Wiley & Sons Inc, 1989. J.J. Craig, Introduction to Robotics: Mechanics and Control, 2nd Edition, Addison-Wesley, 1989. M. Vukobratovic, M. Kircanski, Kinematics and Trajectory Synthesis of Manipulation Robots, Springer-Verlag, Berlin-Heidelberg, 1986. A. Goswami, “Postural stability of biped robots and the foot-rotation indicator point,” Int. Journal of Robotics Research, vol.18, No.6, 1999, pp.523-533. Z. Tang, C. Zhou and Z. Sun , “Trajectory planning for smooth transition of biped robots,” Proceedings of the IEEE Int. Conf. Robotics and Automation (ICRA’2003), 2003, pp.2455-2460. R. Dunn, R. D. Howe, “Towards smooth bipedal walking,” Proceedings of the 1994 IEEE International Conference on Robotics and Automation, San Diego, CA, May 1994, pp. 2489-2494. R. Dunn, R. D. Howe, “Foot placement and velocity control in smooth bipedal walking,” Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, April 22-28, 1996, pp.578-583.
Appendix A. Cubic Spline Interpolation For n breakpoints x1 < x2 < # < xn , the cubic spline interpolation S ( xi ) = yi , i=1,2,…n, the third-order spline function S(t) is a cubic polynomial for each '
(ti,ti+1), and the first derivative S (t ) and second derivative S ''(t ) are continuous on (t1,tn), S ( x j ) on the subinterval [xj,xj+1] is denoted by the following equation:
Si ( x) = yi + yi ,i +1 ( x − xi ) +
( x − xi )( x − xi +1 ) [2 K i + K i +1 + K i ,i +1 ( x − xi +1 )] 6
for each i=1,2,…n :
yi +1 − yi xi +1 − xi K − Ki K i ,i +1 = i +1 xi +1 − xi yi ,i +1 =
Let
hi = xi +1 − xi , Ki is the solution of the following equations:
262
262
Z. Tang, C. Zhou, and Z. Sun
'2h1 K1 + h1 K 2 = 6( y1,2 − m1 ) * (hi −1 K i −1 + 2(hi −1 + hi ) K i + hi K i +1 = 6( yi ,i +1 − yi −1,i ) i = 2, 3,...n − 1 *h K + 2h K = 6(m − y ) n −1 n n n −1, n ) n −1 n −1
If the initial constraint S(x1)=m1, S(x2)=m2, the following equations are obtained:
'a1 = 1/ 2 * hi +1 ( *ai +1 = 2(h + h ) − h a i = 1, 2,...n − 2 i i +1 i i ) 3 ' *d1 = h ( y1,2 − m1 ) 1 * * 6( yi +1,i + 2 − yi ,i +1 ) − hi di i = 1, 2,...n − 2 (di +1 = 2(hi + hi +1 ) − hi ai * * 3 (mn − yn−1,n ) *d n = hn −1 *) h If an −1 ≠ 2 , ai ≠ 2(1 + i +1 ) (i = 1, 2,...n − 2) then hi 6(mn − yn −1,n ) − hn −1d n −1 ' *Kn = hn −1 (2 − an−1 ) ( *K = d − a K i = n, n − 1,...3, 2 i −1 i −1 i ) i −1
Fuzzy-Neural Impedance Control for Robots Zhong-Li Xu, and Gu Fang School of Engineering and Industrial Design, University of Western Sydney, NSW 1797, Australia. [email protected]
Abstract. In conventional impedance control, the difficulties encountered in obtaining an exact system dynamic model and selecting its impedance parameters have prevented it from being applied to many real world applications. The integration of Fuzzy Logic Control (FLC) and Neural Networks (NNs) into impedance control can not only simplify the design procedure but also improve the controller’s performance. In this paper a fuzzy-neural impedance controller is introduced to control a robot to follow complex spatial edges. To design such a fuzzy-neural controller, firstly, an FLC is designed by trial and error based on the human knowledge of the impedance control. This FLC is then used to train an NN. After the training, the NN, called the fuzzy-neural controller in this paper, can be used to control the robot. By using this method the fuzzy-neural impedance controller can handle the system inexactness and uncertainties effectively. Furthermore, the designing process of this controller is simple. The performance of this fuzzy-neural impedance controller is compared with that of other types of impedance control methods. Simulation results are used to show the effectiveness of such a fuzzy-neural based impedance control.
1 Introduction Robot force control strategies should be employed in robotic contact applications, such as peg-in-hole type of assembly and complex spatial edge following tasks. In general, robot force control can be classified into hybrid position/force control and impedance control [2] [9] [16]. In hybrid position/force control [10] since the force and position are controlled separately and their directions are orthogonal, this control method may cause complexity in task planning due to the separation of the two control subspaces and instability due to its need of switching between the two subspaces [12]. In contrast, because impedance control [5] takes general dynamic relationship that includes both position/force and velocity/force relationship between position error and interacting force into consideration, the control task specification and programming are simplified. Many research works have been reported on implementing impedance control. A six degree-of-freedom (DOF) impedance controlled robot [6] is used to perform contact tasks in assembly, such as in screwing a threaded rod into a threaded hole. An adaptive T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 263−275, 2004. Springer-Verlag Berlin Heidelberg 2004
264
264
Z.-L. Xu and G. Fang
force tracking impedance control [7] is proposed for a robot to perform deburring/milling tasks. In [3] a robot impedance control with non-diagonal stiffness matrix is discussed. However, conventional impedance control assumes that accurate system parameters and the dynamic model are available. Its performance could be severely degraded if these assumptions are not satisfied. Furthermore, impedance parameters are fixed in conventional impedance control method. However, these parameters in reality may vary according to the change of operation conditions. In contrast, fuzzy logic is capable of dealing with inexactness and neural networks (NNs) have the ability of nonlinear mapping and learning. Due to these reasons fuzzy logic and neural network based impedance control strategies are introduced. Fuzzy logic is used in [1] and [14] to derive the robot impedance adaptation for assembly tasks and touching and pushing type of operations. An NN impedance controller is introduced in [15] to regulate the desired impedance of the robot endeffector by learning. In [8] two NN compensation schemes, one position based and the other force based, were proposed to counteract the uncertainties in the robot dynamics and disturbances during applications. In this paper, a fuzzy-neural impedance control strategy, which takes advantage of both the FLC and the NN, is introduced to perform the complex spatial edge following tasks. To generate the training data for this fuzzy-neural controller, an FLC is firstly designed by trial and error based on the knowledge of the conventional impedance control. This FLC is then used to train the NN. The main contribution of the paper is the application of a fuzzy-neural impedance control, which utilizes the advantages of both the FLC and the NN, for robot in following complex spatial contours. The effectiveness of this fuzzy-neural impedance controller is demonstrated by computer simulations. Furthermore, the performance comparison of the proposed fuzzy-neural controller against the conventional impedance controller, an FLC designed by trial and error and an NN based impedance controller has also shown that the fuzzy-neural impedance controller is more robust. This paper is organized as follows. In section 2, the design method of a fuzzyneural impedance controller is introduced. The simulation results of the proposed controller are presented in section 3. The discussions and performance comparison of the proposed controller against other types of impedance controllers are also presented in section 3. Conclusions are then given in section 4.
Fuzzy-Neural Impedance Control for Robots
265 265
2 Fuzzy-Neural Impedance Control
2.1 Impedance control When a robot manipulator is in contact with a work-piece, the desired torque between the robot end-effector and the environment can be controlled using the following equations [13]:
τ = J T [ K∆X + L∆X# ] + h(q, q# ) + c (q ) + f (q) Where τ is an n×1 vector of control torque/forces necessary to drive the robot joint actuators, JT is the transpose of an n×n vector of the robot Jacobian matrix , ∆X and ∆X# are n×1 task space position and velocity error vectors, respectively, K and L are the n×n diagonal stiffness coefficient matrix and damping coefficient matrix in task space, respectively, q and q# are joint position and velocity vectors of the robot manipulator, h(q, q# ) is an n×1 vector of Coriolis and centrifugal forces of the robot, c(q) and f(q) are n×1 vectors of the gravity forces and the frictional forces of the robot, respectively. To use the impedance controller for spatial edge following applications, such as applying sealant to the surfaces of aircraft wings, the robot velocity is relatively low (60 to100cm/min) and the edge surface is smooth. Hence the Coriolis, centrifugal forces, and the frictional forces can be ignored. Therefore, the impedance can be simplified as:
τ = J T [ K∆X + L∆X# ] + c(q)
(1)
The diagonal stiffness matrix K can be determined based on the particular job to be performed. The stability requirement defines that the diagonal damping matrix L should be greater than 2 K . 2.2 Fuzzy-neural impedance control
From the impedance control Eq. 1, it can be seen that when the number of DOF of the robot is six (6), eighteen (18) input variables are required to calculate the six joint forces/torques τ. The inputs are six Cartesian positions P (PX, PY, PZ) and orientations Ω (Ωx, Ωy, and Ωz), or the six joint variables qi, six Cartesian errors Pe and Ωe; and six Cartesian velocity errors P#e , and # e . Theoretically, a fuzzy logic controller (FLC) should be able to map the nonlinear relationship stated in Eq. 1. However, there are two factors that make this fuzzy mapping difficult, if not impossible. Firstly, in general, the number of rules, Nr, required of a FLC can be calculated as Nr = Pn , where P is the number of partitions or membership functions (MFs) of !
266
266
Z.-L. Xu and G. Fang
each input variables, assume all the variables have the same number of MFs and n is the number of the variables. Therefore, to cover the entire control region for 18 input variables, even if each variable is to have only 3 MFs, almost 390 million rules are needed. Obviously, it is practically impossible to design a FLC manually with such an enormous amount of rules. In addition, even when such an FLC exists, the computational time required for this FLC will make the real-time implementation impossible. Although the hierarchical fuzzy logic control (HFLC) [11] can be used in some cases to alleviate this problem, the HFLC is not applicable in this case due to the difficulty in distinguishing which variables are more important than the others. The second factor that hinders the fuzzy logic mapping of Eq. 1 is that the relationship of τ and the input variables are not intuitively clear. This lack of intuitive understanding makes the fuzzy rule design difficult. To overcome the above two problems, the proposed fuzzy-neural impedance controller only calculates the forces/moments in Cartesian space according to the Cartesian errors Pe and Ωe; and Cartesian velocity errors P#e , and # e . In another words, the FLC is to be used to performed the following calculation: !
F = K∆X + L∆X#
(2)
Since the stiffness matrix K and the damping matrix L are diagonal matrices, the above force equation can be decoupled into six equations as: & Fx = K x X e + Lx X# e ) # ) Fy = K yYe + LxYe ) Fz = K x Z e + Lx Z# e ) (3) 'M = K Ω + L Ω # Ωx xe Ωx xe ) x # )M = K Ω + L Ω Ωy ye Ωy ye ) y # )(M z = K Ωz Ω ze + LΩz Ω ze Therefore six individual fuzzy-neural controllers can be designed to calculate the forces and moments along X, Y and Z directions. For each fuzzy-neural controller, there are only two inputs. Hence, the rule space is very manageable. In addition, because the proposed fuzzy-neural controller only maps the force/moment in one direction, the relationship of the inputs and output is intuitively clear. This makes the fuzzy rule design relatively simple. Since the design procedure of each of the six controllers is the same, in this paper the fuzzy-neural controller is only designed to control the three Cartesian forces. In another words, the robot considered is regarded as having only 3 DOF. This allows the simulation to be simplified without loosing generality of the proposed method.
Fuzzy-Neural Impedance Control for Robots
267 267
The FLC designed by trial and error To obtain a fuzzy-neural impedance controller, first, a fuzzy impedance controller is designed by trial and error based on the knowledge of the conventional impedance control expressed in Eq. 3. To design the fuzzy logic controller for each Cartesian direction by trial and error, 5 trapezoidal MFs are used for each input variable. The MFs are positive big (PB), positive small (PS), zero (ZE), negative small (NS) and negative big (NB). On the other hand, 9 triangle MFs are used for each output force (Fx, Fy, Fz). They are positive very big (PVB), positive big (PB), positive medium (PM), positive small (PS), zero (ZE), negative small (NS), negative medium (NM), negative big (NB) and negative very big (NVB). The FLC rule base is in the form expressed below IF Pei is Am and P#ei is Bm THEN Fi is Cn Where i = 1, 2, 3 represents the variables in X, Y and Z directions respectively; Pei and P#ei are the respective position errors and velocity errors at each direction, Fi is the output Cartesian force, m=1, 2, …, 5 is the number of input variable MFs and n=1, 2, …, 9 is the number of output variable MFs. The fuzzy rules are displayed in Table 1. Table 1. Table of Fuzzy Rules Fi
P#ej
NB NS ZE PS PB
NB NVB NVB NB NS ZE
NS NVB NB NS ZE PM
ZE NB NM ZE PM PB
Pej
PS NM ZE PS PB PVB
PB ZE PS PB PVB PVB
The fuzzy-neural impedance controller The input and output data from the FLC designed in the previous section can then be used to train a neural network that learns the fuzzy mapping of the inputs and output as stated in Table 1. After the neural network is properly trained, this neural network, called the fuzzy-neural controller in this paper, can be used to perform the impedance control. The objective of this fuzzy-neural impedance controller is to utilize the advantages of both the FLC and the NN to achieve better control performance for robots. The fuzzy-neural controller for each axis has two inputs, Pei and P#ei , same as those in the FLC described in the previous section. One hidden layer consists of 20 neurons and one output that represents the control force F.
268
268
Z.-L. Xu and G. Fang
3 Simulation Results To demonstrate the effectiveness of the proposed control method, the fuzzy-neural impedance controller is used to control the first three DOF of a PUMA robot. Furthermore its performances are compared with that of the conventional impedance controller, a neural network based impedance controller and the FLC introduced in the first part of section 2.2. 3.1 Simulation setup
The simulations are carried out using Simulink and relevant Matlab Toolboxes. The robot manipulator is configured as Left and Below Arm [4] and the payload is set to 0.5 kg. For comparison purpose, a conventional impedance controller is designed using Eq. 1. The following stiffness matrix is used in simulations: 0 0 * #40000 K = $$ 0 40000 0 ++ (N/m) $% 0 0 5000+,
This stiffness matrix implies that while the control along X and Y directions are position based, the control along the Z direction is a compliant force controller
[13]. In order to achieve an overdamping, L should be greater than 2 K , therefore, the following damping matrix L is chosen: 0 * #500 0 $ L = $ 0 500 0 ++ $% 0 0 175+,
Also for comparison, the data obtained from this conventional impedance controller is then used to train a neural network. This neural network is structured with six inputs representing Pe and P#e , two hidden layers with 15 and 20 neurons, respectively, and three outputs representing Fx, Fy and Fz. After the training is completed this neural network, called the neural network based impedance controller, can be used to control the robot. It is assumed that the reference signals of X, Y and Z directions are all sine waves. In addition, the robustness of the proposed controller is also examined by introducing random disturbances into the simulation model. Simulation results of the robot using four different control methods are presented in section 3.2. The performance of the impedance controller is denoted as ‘Imp’ in the figures. The performance of the neural network based impedance controller is denoted as ‘ImpNN’. The results obtained using the FLC designed using the trial and error method introduced in 2.2 are denoted as ‘FLC’ in the figures. The simulation results of the proposed fuzzy-neural controller are denoted as ‘FuzNN’.
Fuzzy-Neural Impedance Control for Robots
269 269
3.2 Simulation results
As mentioned in 3.1, the reference signals in X, Y and Z are set as sine waves, shown in Fig. 1. From simulation, it is found that the performances of the four controllers along X, Y directions are similar. The tracking errors of the four controllers at X direction without noise, and at Y direction with noise, are shown in Figs. 2 and 3, respectively. It can be seen from Fig. 2 that the fuzzy-neural controller produced the best results, with smoother steady state error and smaller overshoot.
Fig. 1. Reference sine wave inputs for X, Y, and Z directions.
The statistics of the tracking errors when there is no disturbance is also shown in Table 2. The fuzzy-neural controller has the smallest standard deviation of error. Table 2. Statistical information of the X direction tracking error without disturbances Min
Imp FLC ImpNN FuzNN
(mm)
6E-4 4E-4 -5E-4 -0.9E-4
Max
(mm)
16E-4 9E-4 15E-4 3E-4
Median
(mm)
3.1E-6 -2.4E-6 -2.5E-5 0.24E-5
STD
(mm)
0.0002 0.00027 0.00017 0.00004
270
270
Z.-L. Xu and G. Fang
To further evaluate the performance of the controllers, disturbances were added to position and velocity error feedback. The tracking errors of the four controllers along Y direction, when the system is subjected to disturbances, are shown in Fig. 3. It is clear that the fuzzy-neural controller out performed the other three controllers and has a better robustness when the system is subjected to noises. Overall, it has smaller error range, smoother tracking error and less overshoot. The statistics of the tracking errors when disturbances are considered are shown in Table 3.
Fig. 2. The comparison of the X direction control errors.
Table 3. Statistical information of the Y direction tracking error subject to disturbances Min
Max
Median
STD
(mm)
(mm)
(mm)
(mm)
Imp
-15E-4
5.5E-4
4.4E-6
0.0003
FLC
-5.6E-4
13E-4
-5.2E-6
0.00025
ImpNN
-5.6E-4
16E-4
-1.9E-5
0.00021
FuzNN
-4E-4
0.78E-4
0.9E-5
0.00017
Fuzzy-Neural Impedance Control for Robots
Fig. 3. The comparison of the Y direction control errors (subject to noises)
Fig. 4. The tracking error along the Z direction using the impedance controller
271 271
272
272
Z.-L. Xu and G. Fang
Fig. 5. The tracking error along the Z direction using the FLC.
Fig. 6. The tracking error along the Z direction using NN impedance controller.
Fuzzy-Neural Impedance Control for Robots
273 273
Fig. 7. The tracking error along the Z direction using the fuzzy-neural controller.
At Z direction, the fuzzy-neural controller is again the best performer when compared to the other three controllers. The tracking errors of the four controllers, with and without disturbances, are shown in Figs. 4 to 7. Their statistics are also tabled in Tables 4 and 5. It can be seen from Figs. 4 to 7 and Tables 4 and 5, the fuzzy-neural controller has the best control performance in almost every aspects, such as smallest overshoot and error range, smoothest steady state error. Other than that, all four controllers were able to maintain Ze, which is the distance between virtual position and actual position of the robot end-effector, at about 2 to 2.5mm distance. Table 4. Statistical information of the Z direction tracking error without disturbances
Imp FLC ImpNN FuzNN
Min (mm) 0 0 0 0
Max (mm) 0.0029 0.0036 0.0042 0.0023
Median (mm) 0.0025 0.0024 0.0024 0.0020
STD (mm) 0.00026 0.00029 0.00033 0.00020
274
274
Z.-L. Xu and G. Fang
Table 5. Statistical information of the Z direction tracking error subject to disturbances
Imp FLC ImpNN FuzNN
Min (mm) 0 0 0 0
Max (mm) 0.0031 0.0045 0.0041 0.0022
Median (mm) 0.0025 0.0025 0.0024 0.0020
STD (mm) 0.00036 0.00041 0.00043 0.00018
4 Conclusions In this paper a fuzzy-neural impedance controller is introduced to control a robot in complex spatial edge following. This controller is built using a neural network that is trained based on the fuzzy relationship the input and output data described in section 2.2. Because of the advantages of both the FLC and the NN are utilized, this control method provides the best control performances in comparison with the other types of impedance controller. Simulation results have shown that the proposed fuzzy-neural impedance controller has performed satisfactorily. The next stage of the research is to experimentally implement and verify the proposed method in complex spatial edge following applications.
References 1. 2. 3. 4. 5. 6. 7. 8.
Babaci S, Amirat Y, Pontnau J, (1996) Fuzzy Adaptation Impedance of 6 DOF Parallel Robot Application to Peg In Hole Insertion. In: Proceedings of the 5th IEEE International Conference on Fuzzy Systems, LA, USA, vol 3, pp 1770-1776. Bruyninckx H, Schutter JD (1996) Specification of Force Controlled Actions in the ‘Task Frame Formalism’ - A Synthesis. IEEE Transactions on Robotics and Automation 12(4): 581-589. Caccavale F, Siciliano B, Villani L (1999) Robot Impedance Control with Nondiagonal Stiffness. IEEE Transactions on Automatic Control 44(10): 1943-1946. Fu KSR, Gonzalez C, Lee CSG (1987) Robotics: Control, Sensing, Vision, and Intelligence. McGraw-Hill, Singapore. Hogan N (1985) Impedance Control: An Approach to Manipulation, Parts I – III. ASME Journal of Dynamic Systems, Measurement and Control 107(1): 1-24. Jossi D, Donath M (1995) Using Six Degree of Freedom Impedance Control Robots to Perform Contact Tasks in a Workcell. In: Proceedings of the ASME Systems and Control Division. pp 207-216. Jung S, Hsia TC (1999) Adaptive Force Tracking Impedance Control of Robot for Cutting Nonhomogeneous Workpiece. In: Proceedings of the 1999 IEEE International Conference on Robotics and Automation, Detroit, Michigan. pp 1800-1805. Jung S, Hsia TC (1998) Neural Network Impedance Force Control of Robot Manipulator. IEEE Transactions on Industrial Electronics 45(3): 451-461.
Fuzzy-Neural Impedance Control for Robots
9. 10. 11. 12. 13. 14. 15. 16,
275 275
Patarinski SP, Botev RG (1993) Robot Force Control: A Review. Mechatronics 3(4): 377-398. Raibert MH, Craig JJ (1981) Hybrid Position/Force Control of Manipulators. ASME Journal of Dynamic Systems, Measurment, and Control 102: 407-410. Raju GV, Zhou SJ, Kisner RA (1991) Hierachical fuzzy control. International Journal of Control 54(5): 1201-1216. Seraji H, Colbaugh R (1997) Force tracking in impedance control. International Journal of Robotics Research, 106(1): 97-117. Schilling RJ (1990) Fundamentals of Robotics: Analysis & Control. Prentice-Hall, New Jersey. Shibata M, Murakami T, Ohnishi K (1996) A Unified Approach to Position and Force Control by Fuzzy Logic. IEEE Transactions on Industrial Electronics 43(1): 81-87. Tsuji T, Ito K, Morasso PG (1996) Neural Network Learning of Robot Arm impedance in Operational Space. IEEE Transactions on Systems, Man, and Cybernetics 26(2): 290-298. Zeng G, Hemami A (1997) An overview of robot force control. Robotica 15: 473-482.
276
On Singular Perturbation Based Inverse Dynamics Control for a Two-Link Flexible Manipulator Xuefeng Dai1,2, Lining Sun1,2, and Hegao Cai1,2 1
Robotics Research Institute, Harbin Institute of Technology, Harbin 150001, P.R. China 2
Robotics Laboratory, Shenyang Institute of Automation, Chinese Academy of Sciences, P.R. China [email protected]
Abstract. Based on singular perturbation technology, the flexible-link manipulator system model was divided into two sub-systems, one for slow sub-system describing equivalent rigid motion, and one for fast sub-system describing vibration. A novel inverse dynamics control scheme based on the two sub-system models was proposed. The algorithm has no specific demand on the desired trajectory.
1 Introduction The use of lightweight material in manipulator links can achieve higher motion speed, better energy efficiency and improved mobility. Unfortunately, the flexible links result in vibration during and after motion. The control of flexible link manipulator not only need to deal with macro motion, but also need to deal with vibration. The model of flexible link manipulators is a set nonlinear time-varying differential equations [1]. Its control related to almost each branch of control theory [2]. Among these results, singulator perturbation technology is used for a single link manipulator control by Siciliano and Book[3], two-link flexible manipulator control by Wang and Unbehauen[4] and structure optimization of flexible manipulators [5]. Additionally, inverse dynamics control has deterministic physical meaning. In front of the manipulator's sophisticated model, the scheme is hard to be implemented. A novel methedology of realizing inverse dynamics control based on singular technology is proposed in this paper.
T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 276−280, 2004. Springer-Verlag Berlin Heidelberg 2004
On Singular Perturbation Based Inverse Dynamics Control
277 277
2 Modeling Flexible Manipulator It is assumed that the amplitude of the higher modes of the flexible links is very small compared to that of the first mode. The dynamics of a two-link flexible manipulator could be expressed by the following two nonlinear time varying differential equations[1]
Mθ## + g + Mq## + f1 + f 2 = T Mq## + Cq# + Kq + Mθ## + g + f 2 = 0
where
θ = [θ1 θ 2 ]
T
are the joint angulars,
q = [q1
(2.1) (2.2)
q2 ] the local
coordinates of two links, T the control input, M the positive definite and symmetrical inertia matrix and varying vs joint angular θ 2 , C the damping
f1 and f 2 the vectors containing gravitational, Coriolis, and centrifugal terms. M , C and K have the form #a + a2 + 2a3c2 a2 + a3c2 ' M =$ 1 (2.3) a2 () % a2 + a3c2 #C − 2a3θ#2 s2 − 2a3 (θ#1 + θ#2 ' C=$ 1 (2.4) ( # C2 % 2a3θ1s2 ) # K − a3[(2θ##1 + θ##2 ) s2 + (2θ#1θ#2 + θ#22 )c2 ]' (2.5) K=$ 1 ( K 2 − a3 (θ##1s2 − θ#12 c2 ) %0 ) where c2 = cos θ 2 , s2 = sin θ 2 , other variables refer to paper [1]. Considering kii >> −a3[(2θ##1 + θ##2 )s2 + (2θ#1θ#2 + θ#22 )c2 ](i = 1,2) and K2 >> a3 (θ##1s2 −θ#12c2 ) , for the purpose of simplicity in the following, let $ K = diag ( K1 , K 2 ) instead of K . matrix, K the system stiffness matrix, g ,
The end trajectory of manipulator can be expressed as a set of
>, where
y = Φ 0θ + Φ f q
(2.6)
Φ 0 = [l1 l2 ] l1 and l2 are the length of two links, respectively. in general, for the n-th modal Φ f = [φ11 (l ) φ 21 (l )] , where
"
φin ( x) = l ( F1,n sin d n χ + F2,n sinh d n χ + F3,n cos d n χ + F4,n cosh d n χ ) here
χ = x/l #
1 d n ≈ (n + )π , F j ,n ≈ (−1) j +1 (8M t l 2 + ρl 3 ) −1/ 2 . 4
,
278278X. Dai, L. Sun, and H. Cai
3 Inverse Dynamics Control Based on Singular Perturbation Denoting q with q = µ ξ , here µ = Kˆ . From the principle of singular perturbation, Eqs. 2.1 and 2.2 can be rewritten as 2
2
−1
(3.1) Mθ## + g + Mµξ 2 + f1 ( µξ ) + f 2 (θ , µξ ) = T Mµ 2ξ## + Cµξ# + ξ + Mθ## + g + f 2 (θ , µξ ) = 0 (3.2) Let µ = 0 and take into account both f1 and f 2 visibly involving q , Eqs.
(3.1) and (3.2) are reduced to
## Mθ + g = T ξ + Mθ## + g = 0
ξ
(3.3) (3.4)
q can be obtained. y of y can be gained, too. These are
can be resolved from Eqs. (3.3) and (3.4). Furthermore
From Eq. (2.6), the approximated value given by
$ q ≈ µ 2ξ = − K −1T y = Φ θ − Φ Kˆ −1T 0
(3.5) (3.6)
f
yd is known, substitute y in Eq. (3.6) with yd . Hence, one equation containing T as variable is Φ f Kˆ −1T = Φ 0θ − yd (3.7) Since the desired end trajectory of manipulator
Considering the desired motion direction of the manipulator end effector, we
LT = 0 , where L ∈ R 2×1 . Now, from the above −1 #Φ f Kˆ −1 ' #Φ 0θ − yd ' T =$ (3.8) ( $ ( 0 ) % L ) % Up to now, T is still not suitable for control, because there're a last unknown variable θ in Eq. (3.8). Next, discuss how to replace θ with its approximated value θˆ . have
Substituting Eq. (3.8) in Eq. (3.3), we have −1
−1
#Φ f Kˆ −1 ' # yd ' #Φ f Kˆ −1 ' #Φ 0 ' # # Mθ = $ (3.9) ( $ (−g ( $ (θ − $ % L ) %0) % L ) %0) ˆ . Because of M varying slowly, so replace it with a constant matrix M ˆ || , so ignore it for brevity, Eq. 3.9 becomes Notice that || g ||<<|| M
On Singular Perturbation Based Inverse Dynamics Control
##
θˆ = Γˆ θˆ + βˆyd
279 279 (3.10)
−1
−1
ˆ −1 ˆ −1 ˆ = Mˆ −1 #$Φ f K '( #Φ 0 ' βˆ = − Mˆ −1 #$Φ f K '( . Let η1 = θˆ , where Γ $ ( % L ) % L ) %0) # η = θˆ then Eq. (3.10) is converted into !
2
η# = Aη + Byd
where
(3.11)
#0 I ' #0' , I is an identity matrix, B = $ $ ( . From Eq. (3.11) A = $$ ( %Γ 0 ) %β ) t
substituting
θ
with
θˆ
η (t ) = & e A( t −τ ) Byd (τ )dτ
(3.12)
0
in Eq.(3.8), control law
T is derived.
4 Conclusions The algorithm proposed in this paper could be outlined as following,
yd
L, M ˆ βˆ Calculating Γ Given "
$
#
#
Resolving elements. %
η
#
#
C and K , form Mˆ and Kˆ .
A and B .
from Eq. (3.12), constructing
θˆ
by its first two raws
Replacing θ with θˆ , calculating T from Eq. (3.8). In the realization of the algorithm proposed in this paper, there's no specific demand for continuously differentiability to the desired trajectory. And the algorithm could be rearlized easily. On the other hand, inverse dynamics is a kind of open-loop control. Its shortage is evidance. The testbed of single link flexible manipulator had been set up [6]. Now, the two-link flexible manipulator system is under developing. We are perfecting the control algorithm in theory aspect and conducting empirical research meanwhile. &
References 1.
Liu K and Kujath MR (1996) Trajectory optimization for a two-link flexible manipulator. Int. J. Robotics and Automation, 11(2):56-61
280280X. Dai, L. Sun, and H. Cai
2. 3. 4. 5. 6.
Dai XF, Qu DS, Sun LN et al (2002) State of the art for control of flexible link manipulators. In: Proc. 2002 Int. Conf. Contr. and Automation, Xiamen, China, pp: 113-117 Siciliano B and Book WJ (1988) A singular perturbation approach to control of lightweight flexible manipulators. Int. J. Robotics Research, 7(4): 79-90 Wang GL and Unbehauen H(2002) Note on the relative degree of a flexible manipulator and implications to inverse dynamics for tracking control. Robotica, 20(1): 33-48 Moallem M, Khorasani K and Patel RV (1998) Optimum structure design for improving the dynamics of flexible-link manipulators. Int. J. Robotics and Automation, 13(4): 125-131 Zhao HW (2000) Macro/micro powered flexible manipulator system and control scheme research(in Chinese). Ph. D. thesis, Harbin Institute of Technology
Wavelets and Biorthogonal Wavelets for Image Compression Don Hong, Martin Barrett, and Panrong Xiao Department of Mathematics and Computer Sciences, East Tennessee State University, Johnson City, TN 37614, USA [email protected]
Abstract. Digital images are widely used in computer applications. Uncompressed digital images require considerable storage capacity and transmission bandwidth. Efficient image compression solutions are becoming more critical with the recent growth of data intensive, multimedia-based web applications. This paper studies image compression with wavelet transforms. The mathematical properties of several types of wavelets, including Haar, Daubechies orthogonal, and biorthogonal spline wavelets are covered and the Embedded Zerotree Wavelet (EZW) coding algorithm, which is used to code the transformed wavelet coefficients, is introduced. In order to compare wavelet methods, a software tool called MinImage is used to analyze the results of its performance and output to compare the wavelet types.
1 Introduction Data compression is the process of converting data files into smaller files for efficiency of storage and transmission. It plays a very important role to rapid progress being made in information technology. Without compression, it would not be practical to put images, audio, and video alone on websites. It is known that JPEG (Joint Photographic Experts Group) and MPEG (Moving Pictures Experts Group) are standards for representing images and video. Data compression algorithms are used in those standards to reduce the number of bits required to represent an image or a video sequence. Compression is the process of representing information in a compact form. Data compression treats information in digital form, that is, as binary numbers represented by bytes of data with very large data sets. Fox example, a single small 4″ × 4″ size color picture, scanned at 300 dots per inch (dpi) with 24 bits/pixel of true color, will produce a file containing more than 4 Megabytes of data. At least three floppy disks are required to store such a picture. This picture requires more than one minute for transmission by a typical transmission line (64k bit/second ISDN). That is why large image files remain a major bottleneck in a distributed environment. Although increasing the bandwidth is a possible solution, the relatively high cost makes this less attractive. T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 281−303, 2004. Springer-Verlag Berlin Heidelberg 2004
282282D. Hong, M. Barrett, and P. Xiao
Therefore, compression is a necessary and essential method for creating image files with the manageable and transmittable sizes. As we know, a compression algorithm has a corresponding decompression algorithm that, given the compressed file, reproduces the original file. There have been many types of compression algorithms developed. These algorithms fall into two broad types, lossless algorithms and lossy algorithms. A lossless algorithm reproduces the original exactly. A lossy algorithm, as its name implies, loses some data. Data loss may be unacceptable in many applications. For example, text compression must be lossless because a very small difference can result in statements with totally different meanings. There are also many situations where loss may be either unnoticeable or acceptable. In image compression, for example, the exact reconstructed value of each sample of the image is not necessary. Depending on the quality required of the reconstructed image, varying amounts of loss of information can be accepted. This paper is concerned with a certain type of compression that uses wavelets. Wavelets, introduced by J. Morlet [8], are used to characterize a complex pattern as a series of simple patterns and coefficients that, when multiplied and summed, reproduce the original pattern. There are a variety of wavelets for use in compression. Several methods are compared on their ability to compress standard images and the fidelity of the reproduced image to the original image. In order to compare wavelet methods, a software tool called MinImage is used. MinImage was originally created to test Haar and Daubechies wavelets [5]. Additional functionality is added to MinImage to support other wavelet types, such as biorthogonal spline wavelests. The results of MinImage’s performance and output are then analyzed to compare the wavelet types.
2 Wavelet Transform The fundamental idea behind wavelets is to analyze the signal at different scales or resolutions, which is called multiresolution. Wavelets are a class of functions used to localize a given signal in both space and scaling domains. A family of wavelets can be constructed from a mother wavelet. Compared to Windowed Fourier analysis, a mother wavelet is stretched or compressed to change the size of the window. In this way, big wavelets give an approximate image of the signal, while smaller and smaller wavelets zoom in on details. Therefore, wavelets automatically adapt to both the high-frequency and the low-frequency components of a signal by different sizes of windows. Any small change in the wavelet representation produces a correspondingly small change in the original signal, which means local mistakes will not influence the entire transform. The wavelet transform is suited for non-stationary signals, such as very brief signals and signals with interesting components at different scales. Usually, wavelets are functions generated from one single function ψ , which is called mother wavelet, by dilations and translations
283 283
Wavelets and Biorthogonal Wavelets for Image Compression
ψ a ,b ( x ) =| a | −1 / 2 ψ ( where ψ satisfies
.ψ ( x)dx = 0 .
x−b , ) a
The basic idea of wavelet transform is to represent any arbitrary function f as a decomposition of the wavelet basis or write f as an integral over a and b of ψ a,b . Let
a = a 0m , b = nb0 a 0m , with integers
m, n , and real numbers
a 0 > 1, b0 > 0 fixed. Then the wavelet decomposition is
f = # cm ,n ( f )ψ m,n .
For some very special choices of
ψ
and
a 0 > 1, b0 > 0 , the functions ψ
2
constitute an orthonormal basis for L (R). In particular, if we choose
m, n
a0 = 2 ,
b0 = 1 , then there exists ψ , with good time-frequency localization properties such that ψ
m,n
2
becomes an orthonormal basis for L (R) and thus,
c m ,n ( f ) =< ψ m ,n , f >= .ψ m ,n ( x) f ( x)dx .
In image compression, we are dealing with sampled data that are discrete in time. We would like to have discrete representations of time and frequency, which is called the discrete wavelet transform (DWT). 2.1 Discrete Wavelet Transform For wavelets generated by a scaling function ϕ (x ) using a multiresolution analysis (see [3] for examples), we can write c m , n ( f ) =< ψ m , n , f >= ψ m ,n ( x) f ( x )dx as the following algorithm.
.
c m ,n ( f ) = # g 2 n −k a m −1,k ( f ) k
a m ,n ( f ) = # h2 n− k a m −1,k ( f )
(2.1)
k
where
g l = (−1) l h−l +1 and hn = 21 / 2 . ϕ (2 x − n)ϕ ( x)dx. When f is
given in sampled form, then we can assume those samples as the highest order resolution approximation coefficient a 0 ,n . Equation (2.1) gives the coding algorithm on these sampled values with low-pass filter h and high-pass filter g. For orthonormal wavelet bases, these filters give exact reconstruction by the following equation:
a m −1,n ( f ) = # [h2 n −l a m ,n ( f ) + g 2 n−l c m ,n ( f )]. n
284284D. Hong, M. Barrett, and P. Xiao
Compact Supported Orthogonal Wavelets In this section, we will determine the filter coefficients for compact supported orthogonal wavelets. We begin with the scaling function
φ (t ) = # hk 2φ (2t − k ).
By integrating both sides, we have
.
∞
−∞
φ (t )dt = .
k
∞
−∞
#h k
k
2φ (2t − k )dt .
Substituting x = 2k − t and dx = 2dt on the right hand side, we get
.
∞
−∞
1 2
∞
φ (t )dt = # hk 2 . φ ( x) dx = # hk −∞
k
k
1
2
.
∞
−∞
φ ( x)dx .
Then, divide both sides by the integral and we have
#h k
k
= 2.
(2. 2)
We use the orthogonality condition on the scaling function to get another condition on {hk }:
. | φ (t ) |
2
dt = . # hk 2φ ( 2t − k )# hm 2φ ( 2t − m) dt k
m
= ## hk hm 2 . φ ( 2t − k )φ ( 2t − m) dt k
m
= ## hk hm . φ ( x − k )φ ( x − m)dx , k
m
where in the last equation, we substituted 2t by x . The integral on the right hand side is zero except k = m . When k = m , we obtain
#h k
2 k
= 1.
(2. 3)
By using the orthogonality of the translates of scaling function we have Substituting
ϕ (t ) '
. φ (t )φ (t − m)dt = δ
m
.
by scaling function, we get
2 2' 2φ ( 2t − k )3 (# hk 2φ (2t − 2m − l )3 dt k 4 4) k = ## hk hl 2 . φ ( 2t − k )φ (2t − 2m − l ) dt.
. ()# h
k
k
l
By substituting x = 2t , we get
. φ (t )φ (t − m)dt = ## h h . φ ( x − k )φ ( x − 2m − l )dx k
l
k
l
Wavelets and Biorthogonal Wavelets for Image Compression
285 285
= ## hk hl δ k −( 2 m +l ) = # hk hk − 2 m . k
Therefore, we have equation
#h h k
k
l
k
= δm .
k −2 m
(2. 4)
By using equations (2.2)-(2.4), we can generate filter coefficients for scaling function. For k = 2 , from equations (2.2) and (2.3), we have h0 + h1 = 2
The only solution is For
h0 = h1 =
h02 + h12 = 1. 1
2
, which is the Haar scaling function.
k = 4 , from equations (2.2)-(2.4), we have h0 + h1 + h2 = 2
h02 + h12 + h22 = 1 h0 h2 + h1 h3 = 0 .
The solutions to these equations include the 4-tap (filter length is 4) Daubechies scaling function. 1− 3 1+ 3 , 3+ 3 , 3− 3 , h0 = h1 = . h3 = h2 = 4 2 4 2 4 2 4 2 We know the wavelet function is
ψ (t ) = # wk 2φ (2t − k ). k
If the wavelet is orthogonal to the scaling function at the same scale, we have
. φ (t − k )ψ (t − m)dt = 0.
Then, we can obtain the wavelet filter coefficients from the scaling filter coefficients [1].
wk = ±(−1) k hN −k . Haar Wavelet The Haar scaling function is defined as:
* 1 0 ≤ x <1 ,0 otherwise
φ ( x) = +
The scaling function is plotted in Figure 1.
286286D. Hong, M. Barrett, and P. Xiao
Y
1
1
X
Fig. 1. Haar scaling function N
In the scaling equation
ϕ ( x) = # C k ϕ (2 x − k ) , only c0 = c1 = 1 , all the k =0
other coefficients are zeros. Haar wavelets are defined as:
ψ a ,b ( x ) = 2 a / 2 ψ (2 a x − b) , b = 0,1,...,2 a − 1, * 1 0 ≤ x < 1/ 2
where ψ ( x) = +− 1 1 / 2 ≤ x < 1 is the Haar mother wavelet.
- 0 otherwise ,
The Haar mother wavelet function is plotted in Figure 2.
Y 1 1
X
-1
Fig. 2. Haar mother wavelet
Daubechies Orthogonal Wavelets There are no explicit expressions for Daubechies compact support orthogonal wavelets and corresponding scaling functions. Table 1 presents the wavelet filter
Wavelets and Biorthogonal Wavelets for Image Compression
287 287
coefficients for Daubechies 4 taps wavelet [3]. Figure 3 is the scaling and wavelet functions. Table 1. Coefficients for the 4-tap Daubechies low-pass filter H0 H1 H2 H3
.4829629131445341 .8365163037378077 .2241438680420134 -.1294095225512603
Fig. 3. Daubechies 4-tap scaling and wavelet functions
Biorthogonal and Spline Wavelets As we know, most images are smooth. It is reasonable to use a smooth mother wavelet for image analysis. On the other hand, it is also desirable that the mother wavelet is symmetric so that the corresponding wavelet transform can be implemented using mirror boundary conditions, which reduces boundary artifacts. Unfortunately, except for the Haar wavelet (the trivial example), no wavelets are both orthogonal and symmetric. To achieve the symmetric property, we can relax the orthogonality requirements by using a biorthonogal basis. In this case, we keep the same decomposition as in equation (2.1). The reconstruction becomes
~ a m −1, f ( f ) = # [h2 n −l a m ,n ( f ) + g~2 n −l cm ,n ( f )] n
~ ~ where h and g may be different from h, g to obtain exact reconstruction. The
relations between them are given by the following equations.
~ g~n = (−1) n h− n +1 , g n = (−1) n h− n +1 , and
Define
φ ( x ) = # hn φ ( 2 x − n ) n
and
~
#h h n
n
~~
~
n+2k
= δ k ,0 . (2. 5)
φ ( x ) = # hn φ ( 2 x − n ) . k
288288D. Hong, M. Barrett, and P. Xiao
Also define
ψ ( x ) = # g nφ ( 2 x − n)
and ψ~ ( x )
k
n
Then we can rewrite
~ = # g~nφ (2 x − n) .
a m ,n ( f ) and c m ,n ( f ) as:
a m , n ( f ) =< φ m ,n , f >= 2 − m / 2 . φ m , n ( x) f ( x) dx
c m ,n ( f ) =< ψ m ,n , f >= 2 − m / 2 .ψ m ,n ( x ) f ( x ) dx and reconstruction is
f = # < ψ m ,n , f > ψ~m ,n . m,n
The following figure shows the relation between the filter structure and wavelets functions. X
H (n)
~ H (n)
G (n)
~ G ( n)
X
Fig. 4. Filter structure and the associating wavelets
For symmetric filters, the exact reconstruction condition on equation (2.5) can be represented by ~ ~ H (ξ ) H (ξ ) + H (ξ + π ) H (ξ + π ) = 1 ,
~
where H (ξ ) = 2 −1 / 2
~
#h e n
n
− jn ξ
Together with the divisibility of ~
and H (ξ ) = 2 −1 / 2
#h e n
n
~ h and h given in
− jnξ
.
~ H and H , respectively, by (1 + e − jξ ) k and
(1 + e − jξ ) k
We have [4]
' l −1 $ l − 1 + ~ H (ξ ) H (ξ ) = cos(ξ / 2) 2l (# %% p ) p =0 & where R (ξ ) is an odd polynomial in
2 p/ 00. sin(ξ / 2) 2 p + sin(ξ / 2) 2l R (ξ )3 , 1 4 ~ cos(ξ ) and 2l = k + k .
(2. 6)
Spline Filters
~ ~ R ≡ 0 with H (ξ ) = cos(ξ / 2) k e − jkξ / 2 where k = 0 if k is ~ even, k = 1 if k is odd. We have Let us choose
Wavelets and Biorthogonal Wavelets for Image Compression ~ ' l −1 $ l − 1 + H (ξ ) = cos(ξ / 2) 2l − k e jkξ / 2 (# %% ) p =0 & p
Then, we get the corresponding function
~
φ
~ H n and H n (see [3]).
Table 2. Filter Coefficients with
n Hn ~ Hn
0
±1
±2
±3
45/64
19/64
-1/8
-3/64
1/2
1/4
0
0
2 p/ 00. sin(ξ / 2) 2 p 3 . 1 4
which is a B-spline function.
We show one examples from this family with their coefficients
289 289
~ k = 4, k = 2 . Table 2 list the
~ k = 4, k = 2
±4 3/128 0
Fig. 5. Spline examples with
~ l=k=k =4
A Spline Variant with Less Dissimilar Lengths We choose R ≡ 0 , and factor the right hand side of equation (2.6) to break the polynomial into a product of two polynomials in sin(ξ / 2) . Allocate the
290290D. Hong, M. Barrett, and P. Xiao
~
polynomials to H and H respectively to make the length of h and as possible.
~ h as close
The following example is the shortest one in this family (shortest h and corresponds to l = k = 4 [3]. Table 3. Filter coefficients with
n Hn ~ Hn
~ h ). It
l=k=4
0
±1
±2
±3
±4
0.602949
0.266864
-0.078223
-0.016864
0.026749
0.557543
0.295636
-0.028772
-0.045636
0
2.2 Comparison of Wavelet Properties The following Table 4 shows the property comparison of three kinds of wavelets. Haar and Daubechies wavelets have orthogonality, which has some nice features. 1) The scaling function and wavelet function are the same for both forward and inverse transform. 2) The correlations in the signal between different subspaces are removed. Among the three kinds of wavelets, the Haar wavelet transform is the simplest one to implement, and it is the fastest. The major disadvantage of the Haar wavelet is its discontinuity, which makes it difficult to simulate a continuous signal. Daubechies found the first continuous orthogonal compact support wavelet. Note that this family of wavelets is not symmetric. The advantage of symmetry is that the corresponding wavelet transform can be implemented using mirror boundary Property
Table 4. Property comparison of three kinds of wavelets Haar Dau. Dau. Orthogonal Spline Yes No Yes
Explicit Function Orthogonal Symmetric Continuous Compacted support Maximum regularity for order L Shortest scaling function for order L
Yes Yes No Yes
Yes No Yes Yes
No Yes Yes Yes
No
No
Yes
Yes
No
Yes
Biorthogonal
Wavelets and Biorthogonal Wavelets for Image Compression
291 291
conditions to reduce boundary artifacts. That is why we introduce the biorthogonal spline wavelet. For the biorthogonal spline wavelet, the scaling function is a Bspline. The B-spline of degree N is the shortest possible scaling function of order N-1 and B-splines are the smoothest scaling functions for a filter of a given length [12]. Since splines are piecewise polynomial, they are easy to manipulate. For example, it is simple to get spline derivatives and integrals. Next, we discuss how to apply wavelet transform to image compression.
3 Wavelets Applied in Image Compression In order to compare wavelet methods, a software tool coded in Visual C++ called MinImage is used. MinImage was originally created to test Haar and Daubechies wavelets [5]. We add some new functionality to it in order to support other wavelet types. Also, the EZW coding algorithm is implemented to achieve better compression results. In the following, we discuss some implementation details of MinImage.
3.1 Baseline Schema The wavelet image compressor, MinImage, is designed for compressing either 24bit true color or 8-bit gray scale digital images. It was originally created to test Haar wavelet using subband coding. To compare different wavelet types, other wavelet types including Daubechies orthogonal and birothogonal spline wavelets were implemented. Also, the original subband coding was changed to EZW coding to obtain better compression results. A very useful property of MinImage is that different degrees of compression and quality of the image can be obtained by adjusting the compression parameters through the interface. The user can trade off between the compressed image file size and the image quality. The user can also apply different wavelets to different kind of images to achieve the best compression results. Figure 6 is the baseline structure of MinImage compressing sechema. Sampled
Preprocessor
Decompression
DWT
Compressed Image
Fig. 6. The baseline schema of MinImage
EZW Coding
Entropy Coding
292292D. Hong, M. Barrett, and P. Xiao
For more details on the Preprocessor, see [5]. In the rest of the chapter, we will focus on implementation of discrete wavelet transform (DWT) and EZW coding.
3.2 Discrete Wavelet Transform (DWT) The discrete wavelet transform usually is implemented by using a hierarchical filter structure. It is applied to image blocks generated by the preprocessor. Figure 7 shows the 1-D wavelet decomposition (forward transform) and composition (inverse transform) algorithms. original signal x[0..n-1], n = 2k
normalize x[i ] =
x[ i ]
n
,
i
=
h=n
nDecompositionStep = 0
h>1
n
y nDecompositionStep ++
nDecompositionStep > nMaxDecompostionStep
n DecompositionStep( x[0..h-1] )
h=h/2
wavelet coefficients x[0..n-1] Fig. 7. The 1-D wavelet decomposition algorithm in MinImage
y
Wavelets and Biorthogonal Wavelets for Image Compression
wavelet coefficients x[0..n-1], n=2k nStart = n i=0
n
i
y
nStart = nStart MaxDecompostionStep
nStart >> i = = 0
y nStart = 2 h= nStart
n
h <= n
y CompositionStep( x[0..h-1] ) h ++
denormalize x[i ] = x[i ] × n ,
i
=
the original signal x[0..n1] Fig. 8. The 1-D wavelet composition algorithm in MinImage
>>
293 293
294294D. Hong, M. Barrett, and P. Xiao
As we know, images are 2-D signals. A simple way to perform wavelet decomposition on an image is to alternate between operations on the rows and columns. First, wavelet decomposition is performed on the pixel values in each row of the image. Then, wavelet decomposition is performed to each column of the previous result. The process is repeated to perform the complete wavelet decomposition. The following is the 2-D wavelet decomposition algorithm. PROCUDURE 2D-Decomposition( ImageData[ 0..n-1, 0..n-1 ] ) FOR row = 0 TO n-1 DO FOR col = 0 TO n-1 DO ImageData[ row, col ] / = n h=n WHILE h > 1 DO FOR row = 0 TO h-1 DO DecompositionStep( ImageData[ row, 0..h-1 ] ) END FOR FOR col = 0 TO h-1 DO DecompositionStep( ImageData[ 0..h-1, col ] ) END FOR h=h/2 END WHILE END FOR 0 END FOR 1END PROCEDURE
3.3 Embedded Zerotree Wavelet (EZW) Coding After the 2-D wavelet decomposition, the wavelet transform blocks contain the wavelet coefficients. Then, we use Embedded Zerotree Wavelet coding to code the transformed wavelet coefficients. EZW coding was designed by Shapiro [10] to use with wavelet transforms. In fact, EZW coding is more like a quantization method. This method is based on progressive encoding to compress an image into a bit stream. The coding process is implemented by passes. For each pass, a threshold is chosen against which all the wavelet coefficients are measured. If a wavelet coefficient is larger than the threshold, it is encoded and removed from the image; if it is smaller, it is left for the next pass. When all the wavelet coefficients have been visited, the threshold is lowered, and the image is scanned again to add more detail to the already encoded image. This process is repeated until the stopping threshold has been reached. A zerotree is a quad-tree of which all nodes are equal to or smaller than the root. The tree is coded with a single symbol and reconstructed by the decoder as a quad-tree filled with zeroes. EZW coding is based on the observation that wavelet
Wavelets and Biorthogonal Wavelets for Image Compression
295 295
coefficients decrease with scale. When the coefficients are scanned, if the root is smaller than a threshold, it is highly probable that all the coefficients in the quadtree will be smaller than the threshold. If this is the case, then the whole tree can be coded with a single zerotree symbol.
3.4 Entropy Coding Entropy coding is the last stage of MinImage to compress the EZW coded data to a final wavelet image file by using a lossless method. A wrapper class is written to apply the Zlib compression engine to perform this task. Zlib is a free compression library. It compresses the source data by LZ77 (Lempel-Ziv1977) and/or Huffman entropy encoding. Its deflation algorithms are similar with those used by PKZIP (an MSDOS based software by PKWARE, Inc.).
4 Experiments In this section, we analyze the compression results produced by MinImage. These include the subjective and objective qualities of reconstructed images for different wavelet types, timing of composition and decomposition for different wavelet types, and timing of EZW coding algorithm for different compression ratios. PSNR (peak-signal-to-noise-ratio) is the most commonly used value to evaluate the objective image compression quality. It is calculated by the following equation:
PSNR( dB) = 10 log10 where
δ d2
x 2peak
δ d2
,
is called MSE (mean-squared-error) and is calculated by the
following equation:
δ d2 =
1 N
N
# (x n =1
n
− yn ) 2 .
4.1 Objective Quality To compare the objective quality for different wavelet types, we use the standard testing image (see Figure 9). Table 5 illustrates one set of compression results.
296296D. Hong, M. Barrett, and P. Xiao
Fig. 9. Lenna 256*256 Table 5. Comparison of compression results by using different wavelets Wavelet Haar D4 D6 D8 D 10 SP 2_2 SP 2_4 SP 4_4
Compression Ratio 9.177 % 8.613 % 8.469 % 8.426 % 8.336 % 8.321 % 8.278 % 8.235 %
Compressed File size 18048 bytes 16938 bytes 16709 bytes 16570 bytes 16393 bytes 16365 bytes 16280 bytes 16196 bytes
PSNR 35.230 dB 36.515 dB 35.246 dB 36.946 dB 35.643 dB 36.962 dB 37.152 dB 37.644 dB
From the above table, we observe that biorthogonal spline wavelets achieve better compression results than orthogonal wavelets. Haar wavelet has the worst result.
4.2 Subjective Quality To compare the subjective quality for different wavelet types, we choose five different standard testing images. All compressed images have the same compression ratio (1%). For each image, we present the reconstructed images in random order to some volunteers to evaluate the quality based on the original image (graded as 10). The following table 6 shows the result, which is conformed to the objective result. Haar wavelet obtains the worse subjective quality because of its discontinuity, which results in the block artifact in reconstructed images. It is hard to tell the difference of reconstructed images compressed by different Daubechies orthogonal wavelets. Birorthogonal spline wavelets achieve the best subjective quality because those reconstructed images are smoother than the others.
Wavelets and Biorthogonal Wavelets for Image Compression
297 297
Table 6. Comparison of subjective quality Wavelet Haar D4 D6 D8 D 10 SP 2_2 SP 2_4 SP 4_4
Average grade 3.2 5.5 5.4 5.4 5.6 5.9 6.0 6.2
Table 7. Comparison of compression results by using different quantization methods
UQ
Compression Ratio 9.177 %
Compressed File size 18048 bytes
EZW
8.788 %
17282 bytes
UQ
8.613 %
16938 bytes
EZW
8.443 %
16605 bytes
UQ
8.469 %
16709 bytes
EZW
8.058 %
15874 bytes
UQ
8.426 %
16570 bytes
EZW
8.396 %
16511 bytes
UQ
8.336 %
16393 bytes
EZW
7.942 %
15619 bytes
UQ
14.550 %
28615 bytes
EZW
14.510 %
28536 bytes
UQ
12.917 %
25403 bytes
EZW
12.756 %
25086 bytes
Wavelet
Quantization
Haar
D4
D6
D8
D 10
SP 2_2
SP 4_4
PSNR 35.230 dB 35.710 dB 36.515 dB 36.433 dB 35.246 dB 36.580 dB 36.946 dB 37.140 dB 35.643 dB 36.381 dB 40.650 dB 41.190 dB 40.710 dB 41.271 dB
298298D. Hong, M. Barrett, and P. Xiao
4.3 Quantization Methods Originally MinImage used Uniform Quantization to code the transformed wavelet coefficients [5]. EZW coding algorithm is adopted to achieve better compression result. Table 7 illustrates the compression results for these two different methods. From this table, we can verify that EZW coding obtains better objective compression results than Uniform Quantization method for each of the wavelet types.
4.4 JPEG Vs. MinImage JPEG stands for Joint Photographic Experts Group, the original name of the committee that wrote the standard. JPEG is a standardized image compression standard which uses the DCT (Discrete Cosine Transform). As we discussed above, wavelet based compression method should obtain better compression ratios than the DCT based compression method. The following test confirms this result. The standard 512*512 grayscale Lenna image (see Figure 11) is used for this test. The MinImage files on the right side of Figure 10 are compressed by using the SP4_4 wavelet and the EZW coding. The left side images of Figure 10 are compressed by using the JPEG standard algorithm called CJPEG, version 6 (2Aug-95) by Thomas G. Lane (Copyright (C) 1995) [7]. Table 8 lists results for four different compression ratios. Table 8. Comparison of compression results between JPEG and MinImage Compression Method JPEG
Compression Ratio 1.60 %
Compressed File size 4213 bytes
MinImage
1.19 %
3129 bytes
JPEG
3.07 %
8078 bytes
MinImage
2.97 %
7812 bytes
JPEG
6.36 %
16752 bytes
MinImage
6.06 %
15944 bytes
JPEG
12.40%
32640 bytes
MinImage
12.39 %
32618 bytes
PSNR 21.93 dB 28.20 dB 30.40 dB 32.05 dB 34.74 dB 35.24 dB 37.81 dB 38.47 dB
Wavelets and Biorthogonal Wavelets for Image Compression
CJPEG Compressed File Size:4213 bytes PSNR:21.93dB (a)
299 299
MinImage Compressed File Size:3219 bytes PSNR:28.20dB (b)
CJPEG MinImage Compressed File Size:8078 bytes, PSNR:30.40dB Compressed File Size:7812 bytes, PSNR:32.05dB (c) (d) Fig. 10. Comparison of JPEG and MinImage
From the Table 8, we can see that MinImage has better objective (PSNR) quality for all different compression ratios. Since JPEG applies DCT on 8*8 blocks, the block artifact (see image (a) in Figure 10) can be seen from the reconstructed image, especially for low quality image.
300300D. Hong, M. Barrett, and P. Xiao
Fig. 11. Original 512*512 grayscale Lenna
4.5 Timing Analysis In this section, we show the decomposition and composition time comparison (see Figure 12), EZW coding and decoding time comparison, and EZW coding time with different compression ratio (see Figure 13). All timing data were collected from a Dell Pentium III desktop machine running on Windows NT 4.0. Comparison of Decomposition and Composition Time 1.2 Time (Secs)
1 0.8
Decomposition Composition
0.6 0.4 0.2
H a
6
4 D
D
8 D
D 10
Sp 2_ 2
Sp 2_ 4
Sp 4_ 4
0
Wavelet Type
Fig. 12. Comparison of Decomposition and Composition Time
Wavelets and Biorthogonal Wavelets for Image Compression
301 301
Since the wavelet decomposition and composition times are entirely determined by the filter length, Haar wavelet (2 taps) takes the least time and Daubechies 10tap orthogonal wavelet takes the most time. Comparison of EZW Coding and Decoding Time (SP4_4)
Time (Secs)
100 80 60
EZW Coding Time EZW Decoding Time
40 20 0 0.75
1.5
3
6
12
Compression Ratio (%)
Fig. 13. Comparison of EZW Coding and Decoding Time
Figure 13 shows that the EZW coding and decoding take about the same time for same the compression ratio. However, when the compression ratio increases, the EZW coding and decoding time will increase significantly. To explain this, let us recall the EZW coding algorithm. When the stop threshold is high, for every main loop, the transformed wavelet coefficients maintain a good zerotree structure, which makes EZW coding algorithm efficient. On the other hand, with a lower stop threshold, a lot of time will be wasted to check the potential zerotree structures, which are not most of the time for a small threshold. Since the EZW coding takes so much time, especially for higher compression ratios, the total compression time will be dominated by the compression ratio. From Figure 14 we can see that there is not much difference in the EZW coding time between different wavelet types for a same compression ratio.
5 Conclusion and Final Remarks This paper introduced some orthogonal and biorthogonal wavelet filters and introduced the EZW coding method for image compression. A software tool called MinImage is used to compare different wavelet methods. The Haar wavelet transform is the simplest one to implement, and it is the fastest. However, because of its discontinuity, it is not optimal to simulate a continuous signal. Based on our experiments, Haar wavelet obtained the worst compression result, which proves the above statement.
302302D. Hong, M. Barrett, and P. Xiao
100 90 80 70 60 50 40 30 20 10 0
0.75% 1.50% 3.00% 6%
H a
4 D
6 D
D8
D 10
12%
Sp 2_ 2
Sp 4_ 4 Sp _2 _4
Coding Time (Secs)
Comparison of EZW Coding Time
Wavelet Type
Fig. 14. Comparison of EZW Coding time with different wavelets and different compression ratios Daubechies found the first continuous orthogonal compactly supported wavelet. Note that this family of wavelets is not symmetric. The advantage of symmetry is that the corresponding wavelet transform can be implemented using a mirror boundary condition, which reduces boundary artifacts. It was proved that no wavelet could be compactly supported, symmetric, continuous and orthogonal simultaneously. To get the symmetry property, we introduced biorthogonal spline wavelet by relaxing the orthogonal condition. For the biorthogonal spline wavelet, the scaling function is a B-spline. The B-spline of degree N is the shortest possible scaling function of order N-1 and B-splines are the smoothest scaling functions for a filter of a given length. Our experiment also proved that birothogonal spline wavelet outperformed the Daubechies orthogonal wavelet. Compared with the current JPEG image compression standard, MinImage obtains better compression results. However, the run time of MinImage increases significantly to achieve a high quality compression ratio because of the EZW coding algorithm. As an application, MinImage gives the user an easier way to do a variety of tests about wavelet image compressions. It could be improved in several ways. We end the paper with the following remarks. 1. To get the symmetric wavelets, one solution is to relax the orthogonal condition. We derived the biorthogonal spline wavelet, which gives better compression results than orthogonal wavelets for symmetric images. Another solution is using the called multiwavlets [11]. The basic idea is to use a vector of scaling functions to generate multiwavelets with orthogonality, symmetry, continuity, and compact support properties at the same time. In [6], some properties of multiwavelets with multiplicity
Wavelets and Biorthogonal Wavelets for Image Compression
2.
3. 4.
303 303
four are discussed. MinImage could be more powerful if multiwavelets transform methods are added. The EZW coding method [13] is one of the more efficient quantization and coding methods for wavelet coefficients. However, MinImage could be more powerful if some more efficient quantization methods, such as space-frequency quantization [14] or context-based entropy coding [2], can be implemented. The entropy coding library, Zlib, is imported to MinImage. Since it uses traditional coding methods, it may not be the best coding method for EZW coded wavelet coefficients. A basic introduction on image compression from discrete cosine transform (DCT) to wavelets can be found in [9].
References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.
Burrus CS, Gopinath RA, Guo H (1998) Introduction to Wavelets and Wavelet Transfroms. Prentice Hall, Englewood Cliffs, NJ. Chrysafis C, Ortega A (1997) Efficient context-based entropy coding for lossy wavelet image compression. DCC, Data Compression Conference, Snowbird, UT, March, pp. 25 - 27. Daubechies I (1992) Ten Lectures on Wavelets. SIAM, Philadelphia, Pennsylvania. Dunn S (1999) Digital Color. http://davis.wpi.edu/~matt/courses/color. Gu H, Hong D, Barrett M (2003) Image Compression Using the Haar Wavelet Transform. J Computational Analysis and Applications 5: 45-75. Hong D, Wu A (2001) Orthogonal Multivavelets of Multiplicity Four, Computer and Mathematics with Applications 20:1153-1169. McCrosky C (1999) Demo of wavelet compression and JPEG. http://www.cs.usask.ca/faculty/mccrosky/dwt-demo/index.html. Morlet J (1983) Sampling theory and wave propagation. NATO ASI Series, Vol. 1, Issues in Acoustic signal/Image processing and recognition, Berlin, pp. 233261. Saha S (2000) Image Compression - from DCT to Wavelets. http://www.acm.org/crossroads/xrds6-3/sahaimgcoding.html. Shapiro JM (1993) Embedded Image Coding Using Zerotrees of Wavelet Coefficients. IEEE Transactions on Signal Processing 41:3445-3462. Strela V (1996) Multiwavelets: Theory and Application. Ph.D. thesis, Massatchusets Institute of Technology. Unser M (1999) Splines, A Perfect Fit For Signal/Image Processing. IEEE Signal Processing Magazine 16:22-38. Valens C (1999) EZW Coding. http://perso.wanadoo.fr/polyvalens/clemens/ezw. Xiong Z, Ramchandran K, Orchard MT (1997) Wavelet packet image coding using space-frequency quantization. IEEE Trans. Image Processing 7:892-898.
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay F.H. Ho, A.B Rad, Y.K. Wong, and W.L. Lo Department of Electrical Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR, China. [email protected]
Abstract. We present an adaptive fuzzy logic controller, which learns a lowerorder model of the system via an on-line Neural Network (NN) identification algorithm. The identification is based on the estimation of parameters of a First-OrderPlus-Dead-Time (FOPDT) model. The outputs of the NN are three parameters: gain, apparent time delay and the dominant time constant. By combining this algorithm with a fuzzy logic controller with rotating rule-table, an adaptive controller is obtained which -with very little a priori knowledge- can compensate systems with unknown time delay. The simplicity and feasibility of the scheme for realtime control provides a new approach for a variety of real-time applications. Simulation and experimental results are included to demonstrate the adaptive property of the proposed scheme.
1 Introduction Identification and control of time delay systems have attracted a lot of attention due to their theoretical and practical significance [7]. Many processes exhibit delay characteristics in their mathematical formulation. Also, systems that are described by rational transfer functions are generally represented by lower-order transfer functions for the purpose of controller design [13]. Model-based control literature is abundant of studies regarding their modeling, identification and control [11]. Well-known solutions to time delay compensation are control methodologies such as the Smith predictor [16] and Internal Model Control (IMC) [6]. In contrast, one may notice that despite their importance, Soft-Computing [20] research has not paid much attention to such systems. This may be due to the fact that the soft-computing methodologies are essentially model-free approaches and the knowledge of the time delay is only implicitly required. This argument is only valid if the system under control contains a time delay much less than the dominant time constant of the system. However, for effective control of systems for which the ratio of time delay to the time constant is equal to or greater than unity, and in particular the dominant time delay systems, explicit knowledge of the delay is required. In such instances, the control strategy should be able to predict the T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 304−326, 2004. Springer-Verlag Berlin Heidelberg 2004
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
305 305
time delay and hence adjust the control signal accordingly. Neural networks architecture using the Nonlinear Auto-Regressive with eXogenous (NARX) model has been suggested to identify systems with time delay [23]. Another approach using a PID Neural Network has also been suggested in [18]. The problem is also addressed by fuzzy neural network [3] and fuzzy control structure [10]. However, these methods, except [23], require the actual knowledge of the system time delay. In this paper, we present a novel adaptive controller for systems with unknown time delay. As with standard adaptive controllers, an on-line parameter estimation algorithm is integrated with a control design methodology [1]. In the proposed design, an algorithm via neural networks estimates the parameters of a First Order Plus Dead Time (FOPDT) model of a higher-order system from input/output data. Based on the information obtained from this model, in particular, the estimated time delay- the rule-table of a fuzzy controller will be adjusted on-line. The rest of this paper is organized as follow: Section 2 is devoted to the idea of approximating a high-order system with a FOPDT model using neural networks. In Section 3, two adaptive designs based on the PID and a novel Fuzzy Logic Controller (FLC) are derived. Simulations of time responses for different systems and the two controllers are given in Section 4. Stability analysis is performed in Section 5.Experimental results will be included in Section 6. Finally, the paper is concluded in Section 7.
2 Lower Order Approximation of Systems with Neural Networks The study will be restricted to single-input / single-output (SISO) systems. It is well known that a FOPDT model, with the transfer function given by Eq. 1, can approximate many higher order systems [13]: b s m + b1 s m −1 + ... + b m − Ts K ⋅ e −θ ⋅ s Y (s) ≈ = 0 n e 1 − n τs + 1 U (s) + ... + a n s + a1 s
(1)
Here, K is the system gain, τ is the dominant time constant, θ is the apparent dead-time and m≤ n. Y(s) and U(s) are the Laplace transformed output and input signals respectively. The parameters the parameters K, τ and θ can be determined by various methods [15,17,19]. Smith's method [15] is a popular approach whereby the parameters are determined from the process reaction curve by injecting a step signal to the system. Although, this method is convenient and accurate, however, it should be carried out off-line and it is very sensitive to noise. Other methods also suffer from the same drawback and hence cannot be used in the design of an adaptive controller since their parameters cannot be updated on-line.
306306F.H. Ho et al.
2.1 Online Approximating Approach The proposed approach is conceptually simple and is inspired by the process reaction curve method. The block diagram of the on-line approximating approach is shown in Figure 1. As shown in this figure, the on-line identification consists of a standard neural network and a module called the model generator. The control signal u(t) is applied to the high-order system, the neural network, and the FOPDT model generator at the same time. The outputs of the neural network are the three FOPDT parameters, namely, the gain ( K ), the time constant (τ ) and the time delay (θ ). These three parameters are fed to the first-order plus dead time model generator to get the output of the model. The error between the output of the plant and the output of the model is used to train the weights of the neural networks. The training process tends to force the output of the FOPDT model generator to approximate the output of the system. Thus, the inputs of the FOPDT model generator are the approximating parameters of the first-order model for the high-order system. The output of the FOPDT model is expected to match the output of the high-order system after the neural network converges. The transfer function of the FOPDT model generator is rewritten below: Ym (s) K ⋅ e −θ ⋅ s = U (s) s +1
(2)
τ
According to the convolution theorem, the output of the FOPDT model generator can be obtained in the time domain as: K ⋅ e− θ ⋅ s y (t ) = L- 1[ ⋅ U ( s)] m τs + 1
(3)
where L-1 stands for inverse Laplace transform. Therefore, we get: y
m
(t ) = K ⋅ u (t − θ ) − τ
dy
m dt
(t )
(4)
where, u(x) is the input of the FOPDT model generator. The model output ym(t) depends on the parameters K, τ and θ at each time instant. The neural network architecture used for plant modeling is a three-layer feedforward network. The basic structure, having one hidden layer with sigmoid function, is known to be powerful enough to produce an arbitrary mapping among variables [4]. Thus, a three layer network is usually used for control applications. The activation function used here is the standard sigmoid function with range between 0 and 1. To train the above neural network, a direct learning strategy is employed for on-line training. As the desired output of the neural network is unknown, the FOPDT model generator is considered as an additional but not modifiable layer of the neural network. The Back-Propagation (BP) algorithm [5] is used to update the weights of the neural network. The algorithm consists of two passes, forward pass and backward pass. The calculation of the forward pass and updating the connection weights from the input layer to hidden layer are the same as those in the standard BP algorithm. To up date the connection weights from the hidden layer to the
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
307 307
output layer, the momentum technique [5] is employed. The weight adjustment for each iteration is derived below. The error function E is define by: 1 r (5) E = $ ( y − ym )2 2 t =1
u
y
System error K
'
Num
K
÷ Den
τ θ
Neural Network
τ
⋅
"
#
!
$
delay
θ
&
ym
%
X
+
ym
'
Num
÷Den
FOPDT Model Generator
Fig. 1. The architecture of the on-line lower-order modeling of high-order systems
where r is the number of input/output pairs available for training the network, y and ym are the output of the plant and the output at the FOPDT model at any time instant t. Within each time interval from t to t+1, the BP algorithm is used to update the connection weights according to the following relationship: (6) ∂E W
ij
( t + 1) = W
ij
(t ) − η ⋅
∂W
ij
(t )
+ α ⋅ ∆W
ij
(t )
where η and α are the learning rate and the momentum factor respectively. ∆Wij is the amount of the previous weight change. Using the chain rule, one has ∂E ∂W
ij
(t )
=
∂E ∂y
m
= − ( y (t ) − y m ( t )) ⋅
(t )
⋅
∂y ∂φ
m n
(t )
(t )
⋅
∂φ
∂W
n ij
(t )
(t )
∂ y m (t ) ⋅ X i ( t ) ⋅ (1 − X i (t )) ⋅ X j ( t ) ∂φ n ( t )
(7)
where Xi is the output of the ith node of the output layer; Xj is the input vector of the nodes of the jth output layer; Yn(t) is a 3x1 input vector of the FOPDT model (the output vector of the neural network):
308308F.H. Ho et al. ∂y m ( t ) ( ∂y m ( t ) ∂y m ( t ) ∂y m ( t ) / , , = ∂τ ∂θ 01 ∂φ n (t ) )* ∂K
T
(8)
To find the partial derivatives of the output ym(t) of the model generator (FOPDT) w.r.t. gain (K), dominant time constant (T) and apparent dead-time (τ), respectively.
∂ y m (t ) e −θ ⋅s = L -1[ u ( s )] ∂K τ ⋅ s +1 ∂y m (t ) − sKe −θ ⋅s u ( s )] = L -1[ ∂τ (τs + 1) 2 ∂ y m (t ) − sKe − θ S u ( s )] = L -1 [ ∂θ τs + 1
(9) (10) (11)
3 Design of the Adaptive Controller Three designs are derived and their performances are compared with each other. The first one is based on the PID control algorithm. The second and third are implementations of Fuzzy Logic Controllers (FLC). It should be noted that the identification part is the same in all these algorithms. 3.1 On Line PID Tuning Control Method Using NN The control structure for the on-line PID tuning is shown in Figure 2. There are two parts in the control structure of the on line PID tuning method. The first part, which was described in the previous section, is the approximation of high order systems with FOPDT using neural networks, and the second part is the design of the PID controller. The parameters of the PID controller can be obtained from the corresponding parameters of the estimated FOPDT by neural networks. We have used the Ziegler-Nichols ultimate cycle tuning method [24] to compute the parameters of the PID controller: Kp = 0.6Ku Ti = 0.5Tu Td = 0.125Tu
(12)
Here, Kp, Ti, Td, Ku and Tu are the proportional gain, integral time constant, derivative time constant, the ultimate gain and the ultimate period respectively. The ultimate gain and the ultimate period are calculated from the FOPDT model of the high order plant [22]. The output of the PID controller is in the form of u (t ) = K P ( e (t ) +
dy f 1 ) + e ( t ) dt − T d ⋅ Ti dt
(12)
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
e ( t ) = r ( t ) − y ( t ),
y
f
(s) =
309 309 (13)
1 y(s) T 1+ d s 10
where u(t), y(t), r(t), yf(t) are the controller output, process output, set-point, and filtered derivative respectively.
Intput
r
PID Controller
u
Output
PROCESS
y
y +
adjust Kp,Ti & Td
u
Tuning Algorithm Based on First Order Plus Dead-time Model
Neural Network
K
Τ
τ
e
Model Generator
ym
error for training
Fig. 2. Overall structure of the auto-tuning PID controller
3.2 Adaptive Fuzzy Logic Controller (AFLC) for Time Delay Systems Tanaka and Sano [9] suggested that rotating the rule-base of a fuzzy controller, in effect, reduces the number of fuzzy rules and has the property of time delay compensation. Here, we have extended this concept to design an adaptive fuzzy controller, which has the capability of time delay compensation. The complete algorithm is included in the appendix. The control structure for the adaptive FLC is shown in Figure 3. This structure is the same as the PID controller except that it is replaced by a fuzzy controller. The control structure consists of two parts: the first part is the approximation of the high-order system with FOPDT model using neural networks, and the second part is the design of the FLC. The scaling factors of the FLC are also determined on-line based on gain, phase margin and FOPDT model [21]. The parameters of the controller are in the form of ρ (14) G = e
Gde =
r
1 , % 4ω 2θ & 2ω p − p + 1 -∆t & π τ . '
(15) Ge
310310F.H. Ho et al.
Gu =
ω pτ Am KG de
(16)
( 4 − 2α )
(17)
α ≅ max(Ge | e |, G de | ∆ |)
Intput
r
Fuzzy Controller
u
Output
PROCESS
y
y + u Rotation
of the Rule-base via Dead-time Model
Neural Network
K
Τ
τ
e
Model Generator
ym
error for training
Fig. 3. Overall structure of the on line tuning parameters FLC
where θ is the dead time, τ is the time constant and K is the process gain of the FOPDT model. r is the set-point change range. Ge , Gde and Gu are the scaling factors for error, change of error and the control signal respectively. The parameter ρ is a coefficient for the criterion of the Integral Time Absolute Error (ITAE). In this paper, ρ is assigned to 0.2, which gives the system possible compromise between ITAE index to set-point change and load disturbance. The ωp is the phase crossover frequency and is computed from: π Amφm + Am ( Am − 1) (18) 2 ωp = ( Am2 − 1)τ where Am and φm are gain and phase margin respectively. The gain and phase margin pair used in this paper is (3,450) because this pair of gain and phase margins gives good response for both set-point change and load disturbance [21]. Two adaptive fuzzy controllers are implemented. The difference between the two is how the scaling factors are set. In the first algorithm (AFLC1), the scaling factors are set manually during the initialization. In the second algorithm (AFLC2), the scaling factors are set on-line. The implementation of the two adaptive FLC is as follows: 1. 2.
Estimate the First-Order-Plus-Dead-Time (FOPDT) model by NN. Calculate the error and change of error between set-point and the system output.
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
3. 4. 5. 6. 7. 8. 9. 10. 11.
311 311
Calculate the input and output-scaling factor from equation (14-16) [only for AFLC2*] Calculate the Φ0 from equation (A4). Calculate the time delay compensation parameter ∆Φ from equation (A5). Estimate the final angle of rotation of control rule from equation (A3). Calculate z and dz from the coordinate transformation equation (A2). Convert v and dv to fuzzy from. Evaluate fuzzy control value. Compute the crisp value by defuzzification. Deterimine the process output by the control value.
4 Simulation Results To show the adaptive behavior of the algorithm, let us consider two processes as: Process I
Y ( s ) 1.5 e −2.5s = U (s) (1 + s) 2
Process II
1 − 1 .4 s Y (s) = U (s) (1 + s ) 3
The first process is a time delay system and the second process is a nonminimum phase system. First, adaptive control of Process I was simulated for t=150s after which the system was changed from Process I to Process II. Subsequently, the system was switched back to process I. We simulated three controllers: Adaptive PID (APID), Adaptive Fuzzy Controller with off-line setting of scaling factors (AFLC1) and Adaptive Fuzzy Controller with on-line setting of the scaling factors (AFLC2). In the simulation, the set-point was selected to be a square wave with amplitude 0.5 and a period of 80s. Gaussian noise with mean zero and variance of 0.01 was injected at the output of the system. We employed a fourth order Runge Kutta numerical integration algorithm for all time responses and the integration interval was selected to be 0.1s. The neural network also used the same time interval for updating its parameters. The simulation proceeded as follows: the PID controller was initialized with Kp = 1, Ti =1000, Td = 0.0. The architecture of the NN was (1,8,3) and the fuzzy controller was using fixed scale factor with Ge=0.43, Gde=0.6, Gu=0.6. The adaptive fuzzy controller was initialized with Ge=0.0, Gde=0.0, Gu=0.0. The architecture of the NN was (1,16,3). For all controllers a bias term of 0.5 was added to all hidden nodes, the number of hidden units was determined by a trial and error procedure. The connection weights were randomly initialized. The learning rate and the momentum parameter were set at 0.8 and 0.1 respectively. Figures 4-6 show the overall performances of the three controllers. In these figures, the set point and the output, the controller signal and the estimated parameters of gain, apparent time delay and the dominant time constant are shown in top, middle and bottom curves respectively. It should be
312312F.H. Ho et al.
noted that the gains in Process I and Process II are different (1.5 and 1), since some adaptive controllers cannot cope with change in steady state gain of the controlled system. However, the proposed method can successfully track the dynamic change. These simulation studies demonstrate the adaptive property of the proposed algorithm. In all these system changes, the neural networks converged and the estimated parameters of the FOPDT also converged to their steady state values. All controllers are known to provide stable and robust control under various conditions. Tables 1 and 2 show the parameters of FOPDT model approximated by several other methods such as Smith's [14], minimized error [8] and Tsang and Rad [19]. The corresponding ultimate gain and the ultimate period for processes I and II respectively. It should be noted that the parameters from all other methods except the proposed one were obtained off-line, from open loop excitation with unit step and were noise free. Furthermore, the values quoted for the proposed algorithm are based on the last measurement before each system change and not the average value. Fig. 4: Simulation results of the PID controller Table 1. Parameters obtained from different methods for Process I
Smith Method [18] Minimized-error [8] Tsang-Rad [19]
Proposed method for PID Proposed method for FLC
Process I
K 1.50 1.50 1.50 1.49 1.51 -
T 1.65 1.46 1.50 1.40 1.39 -
Process I τ 3.00 3.11 3.09 3.14 3.09 -
Ku 1.06 0.98 1.00 0.97 0.96 1.04
Tu 8.38 8.44 8.45 8.44 8.32 8.44
Table 2. Parameters obtained from different methods for Process II
Smith Method [15] Minimized-error [8] Tsang-Rad [19] Proposed method for PID Proposed method for FLC Process II
K 1.00 1.00 1.00 0.99 1.05 -
T 1.89 1.67 1.78 1.28 1.25 -
Process II τ 2.43 2.55 2.44 2.72 2.71 -
Ku 1.93 1.74 1.85 1.49 1.40 1.54
Tu 7.22 7.35 8.45 7.38 7.33 6.83
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
Set-Point and Process O utput
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
Set-point
0
100
150
t/s
200
250
300
350
400
1 5 0 t/s 2 0 0
250
300
350
400
250
300
350
400
C o n tro l S ig n a l
1 0 .9 0 .8 0 .7 0 .6 0 .5 0 .4 0 .3 0 .2 0 .1 0 0
4
50
O utput
50
100
First Order Model Parameters
θ
3.5 3 2.5
K
2 1.5 1
τ
0.5 0 0
50
100
150
t/s
200
Fig. 5a. Simulation results of the AFLC1
313 313
314314F.H. Ho et al. Set-Point andProcess Output
1
Set-point
0.8 0.6 0.4 0.2
FOPDT auto-tuning FLC 0
t(sec)
-0.2 0
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
50
100
150
200
250
300
350
400
200
250
300
350
400
Control Signal
0
50
100
150
t/s
First Order Model Parameters
4 3.5
θ
3 2.5
Κ
2 1.5 1
τ
0.5 0 0
50
100
150 t/s 200
250
300
Fig. 5b. Simulation results of the AFLC1
350
400
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
315 315
Set-Point and Process Output
1 0.9 0.8
Output
Set-point
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
50
100
150
t/s
200
250
300
350
400
200
250
300
350
400
Control Signal
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
50
100
150
t/s
F irs t O rd e r M o d e l P a ra m e te rs
4 3 .5
θ
3 2 .5
K
2 1 .5 1
τ
0 .5 0 0
50
100
150
t/s
200
250
300
350
400
Fig.6. Simulation results of the AFLC2
5 Experimental Studies 5.1 Plant Description A laboratory-scale process control unit (PCU) from Bytronic [2] was used in this experiment. The system rig consists of a sump, a pump, manual/computer control
316316F.H. Ho et al.
diverter valve and drain valve. The sump water is pumped to the pipeline and the manual flow control valve to the process tank. The water is fed back to the sump via the drain valve, thus completing the cycle. The rig can be used for level, temperature and flow control. The objective in our study was to control the water flow by manipulating the pump voltage. Figure 7 shows the schematic diagram of the process rig. A first-order with time delay model was identified, off-line as:
G(s) =
y ( s ) 1.5e −0.25 s = 0 .5 s + 1 u ( s)
(16)
where y(s) and u(s) are the Laplace transform of the flow rate and pump drive voltage respectively. SYSTEM RIG
cooler indicator stirrer
tank full indicator
flow temp display
cooler
switch level
prt
temp display
dirverter indicator
heater
heat display manual heat input
flow rate display
manual drain
solenoid drain
drain indicator
manual diverter
solenoid diverter
gate valve priming valve
sump temp display
sump
pump
Fig. 7. Bytronic Process control unit
5.2 Experimental Results The purpose of the experiment was to demonstrate the performance of the algorithm on a real system. The plant (Bytronic process control rig) can be represented by a FOPDT model at nominal operating points. However, the plant is inherently nonlinear due to the dead-zone of the pump and the flow meter. The set point was chosen as a staircase signal with amplitude 0.4 l/min, 0.8 l/min and 1.2 l/min and period of 40 sec. The sample time chosen to be 0.05 sec. The on-line process computer sampled the water flow rate at every sample interval and the system parameters were identified by the NN from the pump voltage and the water flow rate. Due to the dead zone in the motor dynamic, the pump drive voltage minimum and maximum were 0.8V and 5V respectively. Three controllers were tested:
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
317 317
Adaptive PID (APID), Adaptive Fuzzy Controller with off-line setting of scaling factors (AFLC1) and Adaptive Fuzzy Controller with on-line setting of the scaling factors (AFLC2). The experiment proceeded as follows: the PID controller was initialized with Kp = 1, Ti =1000, Td = 0.0 and the fuzzy controller was using fixed scale factor with Ge=0.43, Gde=0.6, Gu=0.6. The adaptive fuzzy controller was initialized with Ge=0.0, Gde=0.0, Gu=0.0. The architecture of the NN was (1,8,3). For all controllers a bias term of 0.5 was added to all hidden nodes. The connection weights were randomly initialized. The learning rate and the momentum parameter were set at 0.8 and 0.1 respectively. Figures 8-10 show the performance of the proposed algorithm for these controllers on the real system. It can been seen that the performance of the ZN-PID control system is not very satisfactory, since the derivative action amplifies the inherently noisy flow control system. Such a system can be controlled better by an adaptive PI controller. It was noticed that both FLC algorithms perform better than conventional controller for set-point tracking as well as noise rejection. Besides, although the model of the plant changed at different set-points, the control system maintained a very acceptable performance.
6 Conclusions In this paper, a new adaptive fuzzy control algorithm is presented. The control algorithm is based on the parameters of a FOPDT model, which are obtained using neural networks. The time delay compensation is achieved by rotating the rulebase based on the estimated parameters in particular, the time delay. The performance of the adaptive FLC is compared with a conventional ZN-PID controller. It is noticed that the both simulation and experimental results show that the proposed adaptive control system has superior performance in terms of set-point tracking and noise rejection. The simplicity of the scheme for real-time control provides a new approach for implementing neuro-fuzzy applications for a variety of on-line industrial control problems. Results presented clearly demonstrate the adaptive property of the proposed method.
Acknowledgements The authors gratefully acknowledge the financial support of the Hong Kong Polytechnic University
318318F.H. Ho et al.
Set-Point and Process Output 1.4 1.2
Set-point
1 0.8 0.6 0.4
Output
0.2 0 0
40
80
t/s
120
160
200
120
160
2 00
120
160
200
C ontroller O u tp ut 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0
40
80
Firs t Order Model Param eters 2 1.8 1.6 K 1.4 1.2 1 τ 0.8 0.6 0.4 0.2 θ 0 0
40
80
t/s
t/s
Fig. 8. Experimental results of the PID controller
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay Set-Point and Process Output 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0
40
80
t/s
120
160
200
120
1 60
200
C ontrol S ign al 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0
2 1.8
40
80
t/s
First Order Model Parameters
K
1.6 1.4 1.2 1 0.8 0.6
θ
τ
0.4 0.2 0 0
40
80
t/s
120
Fig. 9. Experimental results of the AFLC1
160
200
319 319
320320F.H. Ho et al. Set-Point and Process Output 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0
40
80
t/s
120
160
200
120
160
200
C o n tro l S ig n a l 1 .4 1 .2 1 0 .8 0 .6 0 .4 0 .2 0 0
40
80
t/s
F irs t O rd e r M o d e l P a ra m e te rs
2 1 .8
K
1 .6 1 .4 1 .2 1
τ
0 .8 0 .6 0 .4 0 .2
θ
0 0
40
80
t/s
120
160
200
Fig. 10. Experimental results of the AFLC2
References [1] Astrom, K.J., Wittenmark, B., Adaptive Control, Addison-Wesley Publishing Company, Inc., Second Edition, 1995.
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
321 321
[2] Bytronic (United Kingdom), User Manual for the Bytronic process control rig (Bytronic), 1994. [3] Chen, S.B., Wu, L., and Wang, Q.L., “Self-learning fuzzy neural networks for control of uncertain systems with time delays,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 27, no.1, pp. 142-148,1997. [4] Cui, X., Shin, K.G., “Direct control and coordination using neural networks,” IEEE Transactions on Systems, man, Cybernetics, vol. 23, no.3, pp. 686-697,1993. [5] Freeman, J.A., Skapura, D.M., Neural Networks: Algorithms, Applications and Programming Techniques, Addison-Wesley Publishing Company, Inc, 1991. [6] Garcia, C.E., “Internal Model Control – 1.A unifying review and some new results,” Industrial and Engineering Chemistry, Process Design and Development, vol. 21, pp. 308-323, 1982. [7] Gawthrop, P.J., Nihtila, M.T., and Rad, A.B., “Recursive parameter estimation of continuous-time systems with unknown time delay,” Control–Theory and Advanced Technology (C-TAT), vol.5, no.3, pp.227-248,1989. [8] K.R.Sundaresan and P.R. Krishnaswamy, “Estimation of Time Delay Time Constant Parameters in Time, Frequency and Laplace Domains,” The Chemical Journal of Chemical Engineering, April, vol. 56, 1978. [9] K. Tanaka, K., Sano, M., Parameter adjustment laws for fuzzy PI controllers for firstorder lags systems with dead-time. In R. Lowen and M. Roubens.(ED): Fuzzy logic : state of the art, pp. 531-540, Kluwer Academic Press, 1993 [10] Li, H. X., and Tso, S.K., “Higher-order fuzzy control structure for higher-order or time delay systems,” IEEE Transactions on Fuzzy Systems, vol. 7, no.5, pp.540-552, 1999. [11] Marshall, J.E., Time delay systems: Stability and performance criteria with applications, Ellis Horwood, New York, 1992. [12] Rad, A.B., “Self-tuning control of systems with unknown time delay – A continuoustime approach,” Control – Theory and Advanced Technology, Transactions on neural networks, vol. 1, no.1, March, pp. 479-497,1994. [13] Rad, A.B., Lo, W.L., and Tsang, K.M., “Self-tuning PID controller using NewtonRaphson search method,” IEEE Transactions on Industrial Electronics, vol.44, no.5, pp. 717-725, 1997. [14] Smith, C.L., Digital Computer Process Control, United States: International Textbook Company, 1972. [15] Smith, C. L., Corripio, A.B., Principles and Practice of Automatic Process Control, John Wiley and Sons, Inc., Second Edition. 1997. [16] Smith, O.J.M., “Closer control of loops with dead time,” Chemical Engineering, vol.53, no.5, pp. 217-219,1957. [17] Sundaresan, K.R., Krishnaswamy, P.R., “Estimation of Time Delay Time Constant Parameters in Time, Frequency, and Laplace Domains,” The Canadian Journal of Chemical Engineering, vol. 56, April, pp. 257-262, 1978. [18] Shu, H.L., Pi, Y.G., “PID neural networks for time delay systems,” Computers and Chemical Engineering, vol.24, pp.859-862, 2000. [19] Tsang, K. M. and Rad, A. B., “A new approach to auto-tuning of PID controllers,” International Journal of Systems Science, vol. 26, no.3, pp.639-658, 1995. [20] Tzafetas, S.G., Soft-computing in Systems and Control Technology, World Scientific Publishing, Singapore, 1999. [21] Xu, J.X., Liu, C., Hang, C.C., “Tuning of fuzzy PI Controllers Based on gain/phase margin specifications and ITAE index,” ISA Transactions, vol.35, pp.79-91, 1996.
322322F.H. Ho et al.
[22] Yu, C.C., Autotuning of PID Controllers, Springer, London, 1999. [23] Yu, D.L., Gomm, J.B. and Williams, D, “Neural model input selection for a MIMO chemical process,” Engineering Applications of Artificial Intelligence, vol.13, pp.1523, 2000 [24] Ziegler J. G. and Nichols N. B., “Optimum Settings for Automatic controllers” Transactions on ASME, Nov, 759-768, 1942
Appendix A The controller will be restricted to Single-Input / Single-Output (SISO) systems. It is well known that a higher order system can be approximated by a first order plus dead time (FOPDT) model in form of:
K ⋅ e −θ ⋅s Y ( s ) b0 s m + b1 s m −1 + ... + b m − Ts (A1) ≈ = e τs + 1 s n + a1 s n −1 + ... + a n U (s) Here, K, τ, θ are the gain, dominant time constant and the apparent dead-time respectively and m≤n. U(s) and Y(s) are the Laplace transformed input and output signals respectively. T is the system time delay and ai and bj are the coefficients of the system transfer function. In this paper, we designed a time delay compensated controller that is specially suited to FOPDT model. The compensated effect is effectively use linear operator transformation of system error and error derivative. In fact, there are two effect can be achieved, First the number of control rules can be reduced. Second, the fuzzy logic controller can be possible to do the time delay compensation. Consider a standard fuzzy rule table (A1) in a symmetric form, and initial z and dz in both diagonal axes. A linear operator A: e→ z that take a vector e and rotates both the fuzzy rule table and vector z in clockwise direction by an angle Φ. The matrix of A is equal to Eq. A2.
e NL NM NS ZO PS PM PL PL ZO PS
PS PM PM PL
PM NS ZO PS PS
z
PL
PS PM PM PL
NS NS ZO PS
PS PM PM
ZO NM NS NS ZO PS
e
PS PM
NS NM NM NS NS ZO PS
PS
NM NL NM NM NS NS ZO PS
Φ (to=45o)
NL NL NL NM NM NS NS ZO
Fig. A1. Illustration of Rotating Table
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
(− sin Φ cos Φ / A=) 0, * cos Φ sin Φ 1
323 323 (A2)
z = Ae
.
where e = [e, e ] ∈ ℜ 2 and z = [ z, dz] ∈ ℜ 2 After the initialization of the rule-table, one diagonal of the rule-table gives the same control action for any dz. In other words, the output of the fuzzy controller depends only on z, (Table A2) hence the number of rules are reduced. Table A1. New rotated rule table
NL Any z NL dz
NM NM
NS ZO PS PM PL NS ZO PS PM PL
The angle of rotation Φ which is the summation of the two angles Φ0 and ∆Φ defined as (A3) Φ = Φο + ∆Φ where Φ0 and ∆Φ are the angle of rotation base on First Order model and the angle for time delay compensation respectively. If the higher order system is modelled by a first-order with delay, three parameters gan, time constant and time delay are obtained. The angle in equation 4 for angle of rotation base on First Order model Φ 0 is obtained using the time constant parameter from the first order model without delay case. Φο = tan −1 (−
kτ 1
k 2τ - 1 - K ⋅ f (e)
)
(A4)
where k1 and k2 are the positive constant parameter in order to make the closeloop reference response stable. In this paper, k1 and k2 are equal to 2. The function f(e) is the control value and K is the static gain of the process. Proof: Given in appendix B The parameter Φ 0 have effect of rotation without consider the time delay. The other parameter ∆Φ in fact has effect for time delay compensation based on the first order with time delay model. It is confirmed from simulation results, the first order plus time delay model parameter τ and θ have influence ∆Φ. e.g ∆Φ=g(τ,θ). The function g for time delay compensation is approximate by a fuzzy system. Table 2 show the time delay compensation parameter ∆Φ as a consequent part for 32 pairs τ and θ obtain by computer simulation for quickest response with 20 percent overshoot as a performance index. The i th rule of the fuzzy system is of the following form: Rule i: If τ is Ai1 and θ is Ain Then ∆Φ is ci where Aij, i=1,2,……,m, j=1,2,……n are the fuzzy sets.
324324F.H. Ho et al. Table A2. Rule table for time delay parameters ∆Φ
τ
∆Φ 1 3 5 7 9 11 13 15
θ
1 0.00 0.351 0.466 0.505 0.526 0.539 0.549 0.557
3 0.099 0.537 0.589 0.611 0.624 0.634 0.642 0.648
5 0.317 0.597 0.636 0.622 0.667 0.676 0.684 0.690
7 0.42 0.632 0.666 0.683 0.695 0.704 0.711 0.717
Given the input data τ and θ, by using a singleton fuzzifier, product fuzzy inference and weighted average defuzzifier, the output of the fuzzy system is inferred as follow: n
m
∆Φ =
$c ∏u i =1 m
i
j =1 n
$∏ u i =1 j =1
Aij
Aij
(x j )
(A5)
(x j )
The implementation of the FLC is as follows: Initial the two axes z and dz in both diagonals for the rule table. 1. 2. Calculate the error and error derivative between set-point and the system output. 3. Calculate the Φ 0 from equation (A4). 4. Calculate the time delay compensation parameter from equation (A5). 5. Estimate the final angle or rotation of control rule from equation (A3). 6. Calculate z and dz from the coordinate transformation equation (A2). 7. Convert z and dz to fuzzy from. 8. Evaluate fuzzy control value. 9. Compute the crisp value by defuzzification. 10. Determine the process output by the control value.
Appendix B Proof of Lemma1: Consider the First Order model without delay K τS + 1 and the closed-loop system governed by ⋅⋅
⋅
(B1)
(B2) e+ k1 e+ k 2 e = 0 where k1 and k2 are positive constant, we have e(t)→0 as t→∝. The close-loop transfer function
Adaptive Neuro-Fuzzy Control of Systems with Unknown Time Delay
325 325
Y ( s) KF = R( s) τs + 1 + KF
where F=f(e,Φ ) τsY + Y + KFY = KFR, Y = R − e
τs ( R − e) + ( R − e) + KF ( R − e) = KFR ⋅ ⋅⋅ ⋅ . − τ e+ R − e − KFe = 0 # −τ e − e− KF e = 0 ⋅⋅ ⋅ 1⋅ 1⋅ e = − e(1 + KF ) # − e(1 + KF ) = − k1 e − k 2 e
τ
τ
⋅ kτ e 1 tan Φ = = − 0 e k τ − 1 − KF 2
⋅
e ( k 2τ − KF − 1) = − k τe, 1
Appendix C .
X m (t ) = AX m (t ) + Bu (t − T ), X m (t ) =
y m (t ) = CX m (t ) + Du (t − T )
t
(t − t 0 ) X m (t 0 ) + + (t − λ ) Bu (λ − T )dλ !
!
t0
(C1)
t
y m (t ) = C (t − t 0 ) X m (t 0 ) + C + (t − λ ) Bu (λ − T )dλ + Du (t − T ) , !
!
t0
(t ) = e At Assuming that all initial states are zeros and D=0, the output equation becomes: !
t
ym (t ) = + h(t − λ , p)u (λ − T )dλ t0
e
− sT
(C2)
Gm ( s) = e − sT L[h(t,p)]
= e − sT C(sI - A)-1B = e− sT
B (s) b0 s m + b1s m −1 + ... + bm = e− sT m Am ( s) s n + a1s n −1 + ... + 1
where θ is the model time delay and h(t, p) =L-1[Gm(s)] is the impulse function of Gm(s). The vector is defined as p = [a1 a2 …an b0 b1 …bm ]. The partial derivatives of the model output with respect to the time delay and the model parameters are as follows.
326326F.H. Ho et al.
∂ym ∂ = ∂T ∂T =
t
t
+ h(t − λ , p)u (λ − T )dλ
t0
∂
+ ∂T [h(t − λ , p)u (λ − T )]dλ
t0 t
= + h(t − λ , p) t0 t
(C3)
∂u (λ − T ) dλ ∂T .
= + [−hm (t − λ , p) u (λ − T )]dλ t0
= − L−1[ se − sT Gm ( s)U ( s)] t
∂ym ∂ hm (t − λ , p)u (λ − T )dλ = ∂ai ∂ai t+0 =
t
∂
+ ∂a [h
m
(t − λ , p)u (λ − T )]dλ =
i
t0
L[ =
∞
∂h ∂h − st ∂ ]= + e dt = a ai ∂a i ∂ ∂ i 0
(C4)
t
∂h(t − λ , p) u (λ − T )dλ ∂ai t0
+
∞
+ h(t , p)e
− st
dt
0
∂G m ( s ) ∂ L[h(t , p)] = ∂a i ∂a i
#
(C5)
∂G ( s ) ∂h = L−1 [ m ] ∂a i ∂a i
Similarly
∂G ( s ) ∂h = L−1 [ m ] ∂bi ∂bi
∂y m t ∂h(t − λ , p) u (λ − T ) dλ = ∂a i t+0 ∂a i = L−1 [e − sT
(C6) n −i
s Bm (s) ∂G m ( s) U ( s )] = L−1 [ −e − sT U ( s)] ∂a i [ Am ( s)] 2
∂y m t ∂h(t − λ , p) u (λ − T )dλ = ∂bi ∂bi t+0 = L−1 [e − sT
(C7)
∂G m ( s) s U ( s)] U ( s)] = L−1 [e − sT ∂bi Am ( s)
∂y m B ( s) = L−1 [− se − sT m U ( s )] ∂T Am ( s)
m −i
(C8) The control signal u(t) is filtered by the filter function in eq. (C6-C8) to find the partial derivatives of ym(t) with respect to various model parameters.
Adaptive Robust Fuzzy Tracking Control for a Class of Nonlinear Systems Yansheng Yang1 and Changjiu Zhou2 1
Navigation College of Dalian Maritime University, Dalian, 116026, P.R. China School of Electrical and Electronic Engineering, Singapore polytechnic, 500 Dover Road, Singapore 139651, Republic of Singapore [email protected] 2
Abstract. The tracking control problem for a class of nonlinear systems with uncertain system function and uncertain gain function, which are the unstructured state-dependent (or non-repeatable) unknown nonlinear functions, is investigated in this paper. A fuzzy logic system is used to approximate the lumped nonrepeatable state-dependent uncertain function, then the fuzzy system is used as the upper bound of uncertainty, and a novel of adaptive robust fuzzy tracking control (ARFTC), that can evaluate the bound parameter of uncertainty on line, is presented. For the proposed algorithm, there are two advantages: (1) the adaptive mechanism with one learning parameterization can be obtained by use of Lyapunov approach; (2) the possible controller singularity problem in some of the existing adaptive control schemes met with feedback linearization techniques can be removed. Finally, the proposed algorithm is verified through simulation.
1 Introduction In recent years, there have been significant research efforts on the adaptive tracking control of nonlinear systems. The control scheme via feedback linearization for nonlinear systems has been proposed in [1]. The fundamental idea of feedback linearization is to transform a nonlinear system dynamic into a linear one. Therefore, linear control or fuzzy logic control (FLC) techniques can be used to acquire the desired performance [1]-[3]. The FLCs have well proven their broad potential in numerous practical industrial applications and have attracted a great deal of public attention and a lot of work has been published in the field. Although FLC systems have the advantages such as no formal mathematical models are needed and the system uncertainties can be coped with, an analytical understanding of this well demonstrated success is lacking, and when designing controllers for an uncertain system, if the structure of the uncertainties in the system is known and the upper bound of uncertainties can be obtained, so it is understood that there exists a class of continuous-time robust controllers that insure the convergence of the state to an arbitrarily small T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 327−344, 2004. Springer-Verlag Berlin Heidelberg 2004
328328Y. Yang and C. Zhou
neighborhood of the origin in a finite period of time [4]. Unfortunately, in industrial control environment, there were some controlled systems which were not only nonlinear, but also characterized by the unstructured state-dependent (or nonrepeatable) unknown functions. Furthermore, as the design of FLCs involves a great deal of heuristics, many theoretical questions regarding various analytical properties of FLCs (e.g. stability, robustness) remain difficult to solve. A solution to the above-mentioned problems was firstly presented by Wang [5]. The fuzzy systems were proposed to uniformly approximate the uncertain nonlinear functions in the designed system by use of the universal approximation properties of certain classes of fuzzy systems presented in Lee [6] , Ying [7], Wang and Mendel [8]. By using Lyapunov based learning law, an adaptive fuzzy control strategy that ensured the stability of the overall system was developed. However, the control method described in Wang [3] has two substantial drawbacks: (1) the method is restricted to plants where the Lie-derivative L∆g Lnf0−1h( x) is constant,
which excludes many systems of practical importance; (2) many parameters need to be turned in the learning laws when there were many state variables in the designed system and many rule bases were used in the fuzzy system, which was used to approximate the nonlinear and uncertain systems. In this paper, an adaptive robust fuzzy tracking control (ARFTC) scheme, that can evaluate the bound parameters of the lumped non-repeatable state-dependent nonlinear function on line, is presented. The main features of the control law proposed can be viewed from two aspects: (1) the Lie-derivative L∆g Lnf0−1h( x) is not constant which can be considered in this method, so the prerequisite for understanding of the controlled system is relaxed; (2) no matter how many rules in the fuzzy system are used, only one parameter needs to be adapted on-line, so the computation load of the scheme has been reduced. This rest of this paper is organized as follows. In Section 2, we will give the preliminaries for the configuration of MISO fuzzy systems and approximation theorem. Section 3 presents the problem formulation for the description of a class of nonlinear systems and tracking control problem. In Section 4, a systematic procedure for the synthesis of adaptive robust fuzzy tracking control (ARFTC) is developed. In Section 5, a simulation example is given to illustrate the effectiveness of the proposed ARFTC for tracking control of an inverted pendulum system. Concluding remarks are given in Section 6.
2 Preliminaries The basic configuration of fuzzy logic systems consists of some fuzzy IF-THEN rules and a fuzzy inference engine. The fuzzy inference engine uses the fuzzy IFTHEN rules to perform a mapping from an input linguistic vector x to an output linguistic variable y . In this paper, we consider a fuzzy system to uniformly approximate a continuous multi-dimensional function y = f ( x ) , where x is input
Adaptive Robust Fuzzy Tracking Control
329 329
vector with r independent x = ( x1 , x2 ,# , xr ) . The domain of xi is θ i = [ ai , bi ] . It T
follows that the domain of x is Θ = θ1 × θ 2 × # ×θ r = [ a1 , b1 ] × [ a2 , b2 ] × # × [ ar , br ]
In order to construct a fuzzy system, the interval [ ai , bi ] is divided into N subintervals 1≤ i ≤ r ai = C0i < C1i < # < CN −1i < CNi = bi ,
On each interval θ i (1 ≤ i ≤ r ) , N + 1 continuous input fuzzy sets, denoted by
A ( 0 ≤ j ≤ N ) , are defined to fuzzify xi . The membership function of Aij is dei j
noted by µ ij ( xi ) , which can be represented by triangular, trapezoid, generalized bell or Gaussian type and so on. Generally, the fuzzy system can be constructed by the K ( K > 1) fuzzy rules as follows: Rl IF x1 is Ahl1 AND x2 is Ahl2 AND …… AND xr is Ahl r !
THEN y is Bhl1h2 #hr , l = 1, 2,# , K where Bhl1h2 #hr is a singleton output fuzzy set for the output variable y . The membership function of Bhl1h2 #hr is 1 only at y = σ l (an arbitrary constant) and 0 at other position. The product fuzzy inference is employed to evaluate the ANDs in the fuzzy rules. After being defuzzified by a typical center average defuzzifier, the output of the fuzzy system is K
fˆ ( x, σ ) =
# σ ( µ ( x ) ⋅ µ ( x )# µ ( x ) ) l =1 K
l h2
1
l hr
2
l h1
1
l h2
l hr
2
µ hl ( x1 ) ⋅ µ hl ( x2 )# µ hl ( xr ) 1
K
#(µ l =1
r
# ( µ ( x ) ⋅ µ ( x )# µ ( x ) ) l =1
where ξ l ( x ) =
l h1
l
2
l h1
r
( x1 ) ⋅ µ ( x2 )# µhl ( xr ) ) l h2
= σ Tξ ( x)
(1)
r
and ξ l ( x ) is called a fuzzy base
r
function. When the membership function µ ij ( xi ) in the ξ l ( x ) is denoted by some
type of membership function, ξ l ( x ) is a known continuous function. When the fuzzy system is employed to approximate the continuous function, two questions of interest may be asked: (1) whether there exists a fuzzy system to approximate any nonlinear function to an arbitrary accuracy? (2) how to determine the parameters in the fuzzy system if such a fuzzy system does exist. For the first question, the following lemma is proven by Wang [3]. Lemma 2.1 Suppose that the input universal of discourse U is a compact set in R r . Then, for any given real continuous function f ( x ) on U and arbitrary ∀ε > 0 , there exists a fuzzy system in the form of expression (1) such that
330330Y. Yang and C. Zhou
sup fˆ ( x, σ ) − f ( x ) ≤ ε
(2)
x∈U
Remark 2.1 If using fˆ ( x, σ ) to estimate the function f ( x ) , there will be N n
unknown constants to be adjusted.
3 Problem Formulation
3.1 Descriptions of uncertain nonlinear systems
Consider an uncertain nonlinear dynamic system in the following form -*ζ& ( t ) = f 0 (ζ ) + ∆f (ζ , w ) + ') g0 (ζ ) + ∆g (ζ , w ) 24 u ( t ) + -, y (t ) = h(ζ )
(3)
where ζ ∈ R n is the state. u ∈ R is the input control. y ∈ R is the output of the system. w is the model uncertainty belonging to a compact set, which includes the parameter and the disturbance uncertainties of the system. f 0 (ζ ) and g 0 (ζ ) are known functions and belong to smooth vector fields in a neighborhood of the origin ζ = 0 with f 0 ( 0 ) = 0 and g 0 ( 0 ) ≠ 0 . ∆f (ζ , w ) is the uncertain system function and ∆g (ζ , w ) is the uncertain control gain function, both of which are con-
tinuous functions depending on the state ζ . For the purpose of designing the controller, the input-state linearization is needed to be performed for the system first. The following assumptions are introduced. Assumption 3.1 The distributions
{
}
Di = span g0 , ad f0 g 0 ,# , ad if0 g 0 , 0 ≤ i ≤ n − 1
are involutive and of constant rank i + 1 in U1 . Assumption 3.2 Suppose
{
}
Di = span g0 , ad f0 g 0 ,# , ad if0 g 0 , 0 ≤ i ≤ n − 1 uncertain function ∆f (ζ , w ) satisfies strict-triangle condition such that
') ∆f (ζ , w ) , Di 24 ∈ Di
0≤i ≤ n−2 T
Finding a diffeomorphism x = φ (ζ ) = ') h, L f0 h,# , Lnf0−1h 24 according to As-
sumptions 3.1 and 3.2,, then Eq. (3) can be transformed into
Adaptive Robust Fuzzy Tracking Control
1 ≤ i ≤ n −1 * x&i = xi +1 + ∆ i n n −1 −1 −1 + x&n = L f0 h (φ ( x ) ) + Lg0 L f0 h (φ ( x ) ) u + ∆ n , y = x1
331 331
(4)
where ∆ i = L∆f Lif−01h (φ −1 ( x ) ) , ∆ n = L∆f Lnf0−1h (φ −1 ( x ) ) + L∆g Lnf0−1h (φ −1 ( x ) ) . In Eq. (4), ∆ i , i = 1, 2,# , n are the functions relating to the uncertainties of the system. From Eq. (4), if ∆ i , i = 1, 2,# , n − 1 does not act within channels implicit in the control input, i.e. unmatched uncertainty hold, it is difficult to design acontroller for the unmatched uncertain system. Therefore, the following assumption is introduced in this paper. Assumption 3.3 The uncertainties of the system satisfy the matched condition such that ∆ i = 0, i = 1, 2,# , n − 1 . According to Assumption 3.3, the uncertain nonlinear system (4) can be rewritten as follows * x&i = xi +1 1 ≤ i ≤ n −1 (5) + x&n = f 0 ( x ) + ∆f ( x, w ) + ') g 0 ( x ) + ∆g ( x, w ) 24 u , y = x1 where x = [ x1 , x2 ,# , xn ] , f 0 ( x ) = Lnf0 h , ∆f ( x, w ) = L∆f Lnf0−1h , g 0 ( x ) = Lg0 Lnf0−1h , T
∆g ( x, w ) = L∆g Lnf0−1h . Another assumption is introduced as follows. Assumption 3.4 The uncertain control gain function ∆g ( x, w ) is confined within a certain range such that 0 < bmin ≤ 1 + g0−1 ( x)∆g ( x, w ) ≤ bmax
(6)
where bmin and bmax are the low and upper bound parameters respectively. The Assumption 3.4 implies that smooth function g 0 ( x) + ∆g ( x, w) is strictly either positive or negative. In the following, without loss of generality, we shall assume 1 + g 0−1 ( x)∆g ( x, w ) ≥ bmin ∀x ∈ R n . Assumption 3.4 is reasonable because g 0 ( x) + ∆g ( x, w) being away from zero is the controllable conditions of system (5). It should be emphasized that the low bound bmin is only required for analytical purposes, its true value is not necessarily known. 3.2 Adaptive fuzzy tracking control problem
In this paper, the control objective is to force y (t ) to track a given reference signal yd (t ) while keeping the states and control bounded, i.e. the output tracking error e1 (t ) = y (t ) − yd (t ) should be small. The given reference signal yd (t ) is as-
332332Y. Yang and C. Zhou
sumed to be bounded and has bounded derivatives up to the nth order for all t ≥ 0 , and yd( n ) (t ) is piecewise continuous. Let ν = )' yd , yd(1) ,# , yd( n −1) 24 (5) can then be described by
T
such that ν is bounded. Suppose e = x − ν . Eq.
*-e&i = ei +1 , 1 ≤ n ≤ n − 1 +& ( n) -,en = f 0 ( x) + ∆f ( x, w) − yd (t ) + )' g 0 ( x) + ∆g ( x, w ) 42 u (t ) For Eq. (7), we have the following transformation for the control input u (t ) = g 0−1 ( x) [ − f 0 ( x) + v (t )]
(7)
(8)
where v(t ) is a new control input. From observation of Eq. (8), the control scheme proposed here consists of two terms: one is − g 0−1 ( x) f 0 ( x) which is used to cancel the nonlinearity in the system and the other is g 0−1 ( x)v(t ) which is mainly used to compensate the effect of uncertainty in the system. Substituting Eq. (8) into Eq. (7), we have *e&i = ei +1 , 1 ≤ n ≤ n − 1 +& (n) ,en = F ( x, w) − yd (t ) + G ( x, w ) v(t )
(9)
where F ( x, w) = ∆f ( x, w) − (1 + g 0−1 ( x)∆g ( x, w)) f 0 ( x) , G ( x, w) = 1 + E ( x, w) and E ( x, w) = g0−1 ( x)∆g ( x, w) . The problem of adaptive fuzzy tracking control design for v(t ) has recently received attention with renewed interest (see, e.g. Wang [3], Fischle and Schroder [9], Su and Stepanenko [10], Yang, Zhou and Jia [11]). If using fuzzy systems to approximate F ( x, w) and G ( x, w) , then an "integral" control law is necessary, i.e., a dynamic feedback
χ& = ϖ ( χ , ξ ( z ), e), ϖ (0, 0, 0) = 0
(10)
v(t ) = α r ( χ , ξ ( z ), e), α r (0, 0, 0) = 0
(11)
where ξ ( z ) is the known fuzzy base functions. And χ ∈ R χ , χ ( ' ) and α r ( ' ) n
n
are smooth functions on R χ × R K × R n . An important quality of the control law is of course the property that the dimension nχ of χ is "as small as possible", and is in particular not dependent on the dimension of the state. However, all the conventional adaptive fuzzy tracking control laws available in the literature have the property that the dimension nχ of χ is 2 N n when using the fuzzy system (cf. Remark 1) to estimate the function F ( x, w) and G ( x, w) . We can state that the dimension nχ of χ is strong dependent on the dimension n of the state. In such a way, the learning time will tend to become unacceptable long for systems of higher order. For the conventional adaptive fuzzy tracking control laws, there is
333 333
Adaptive Robust Fuzzy Tracking Control
another main difficulty which comes from uncertainty G ( x, w) , which is usually approximated by the fuzzy system Gˆ ( x, σ ) . Consequently, the estimate Gˆ ( x, σ ) must be away from zero for avoiding a possible singularity problem. In this paper, we will develop a novel adaptive robust fuzzy tracking controller which does not require to estimate the unknown function Gˆ ( x, σ ) , and therefore avoids the possible controller singularity problem usually met in the traditional adaptive fuzzy tracking control laws. It will result in all the solutions of the closed-loop system (9), (10) and (11), which are uniformly ultimately bounded. Furthermore, the tracking error lim e ( t ) = 0 can be achieved. The outstanding feat →∞
ture of the algorithm proposed in this paper is that the dimension nχ of χ is only one, no matter how many states in the system are investigated and how many rules in the fuzzy system are used.
4 Design of Adaptive Robust Fuzzy Tracking Controller To simplify the design procedure of the adaptive robust fuzzy tracking controller (ARFTC), we define a variable transformation for Eq. (9) as follows n −1
S (e, t ) = en + k1e1 + # + kn −1en −1 = en + # ki ei i =1
(12)
where ki ’s are chosen in such a way that s n + kn −1 s n −1 + # + k1 = 0 is a stable polynomial, hence, we get n −1
S& ( e, t ) = e&n + # ki ei +1 i =1
(13)
Substituting Eq. (9) into Eq.(13), we have
* -e& = e , 1≤ i ≤ n−2 i +1 -- i +e&n −1 = − k1e1 − # − kn −1en −1 + S n −1 - S& = F ( x, w) + ')1 + E ( x, w ) 24 v(t ) − yd( n ) (t ) + # ki ei +1 -, i =1
(14)
Let ζ = [ e1 , e2 ,# , en −1 ] and we use kn −1 to multiply Eq. (12), and substituting it into Eq. (14), we get T
*-ζ& = Aζ + BS +& ( n) ,- S = F ( x, w) − yd (t ) + ')1 + E ( x, w ) 24 v(t ) + kn −1 S − Cζ
(15)
334334Y. Yang and C. Zhou
1 0 # 0 2 ' 0 '02 ' kn −1k1 2 ( 0 3 (03 # 0 1 0 ( k k −k 3 ( 3 ( 3 ( n −1 2 1 )3 $ $ $ $ 3 , B = ( $ 3 , C T = (( where A = ( $ 3. $ ( 3 ( 3 ( 2 3 0 0 # 1 3 ( 0 (03 ()( kn −1 − kn − 2 ) 34 () −k1 −k2 −k3 # −kn −1 34 ()1 34 Owing to A is stable, the positive definite solution P = PT of the Lyapunov equation
AT P + PA + Q = 0
(16)
exists and Q > 0 is chosen by the designer. Construct an ARFTC as follows
v = vequ + vN
(17)
where vequ denotes a certainty equivalent controller and vN denotes a nonlinear controller. Then we define vequ as follows
vequ = −(kn −1 + k 2) S where k > 0 is chosen by the designer. Substituting Eq. (18) into Eq. (15), we can rewrite Eq. (15) as follows *ζ& = Aζ + BS +& 1 (n) - S = − kS + F ' ( x, ζ , yd , w ) + (1 + E ( x, w) ) vN 2 ,
(18)
(19)
where F ' ( x, ζ , yd( n ) , w ) = F ( x, w ) + BT Pζ − Cζ − yd( n ) (t ) .
When the controller is designed based on the feedback linearization technique, the most commonly used control structure is vN = − F '( x) (1 + E ( x) ) if there are no uncertain parameters in the system. When there are uncertainties in the system,
(
)
the nonlinearities F ' x, ζ , yd( n ) , w and 1 + E ( x, w ) are unknown and many adaptive control schemes have been developed, in which the unknown function 1 + E ( x, w ) is usually approximated by a function approximator 1 + Eˆ ( x, w ) . Consequently, 1 + Eˆ ( x, w ) must be away from zero for avoiding a possible singularity problem. In this paper, we will propose a method to design the adaptive robust controller without estimating 1 + Eˆ ( x, w ) so that we can not only reduce the number of parameters needed to be adapted on-line for 1 + Eˆ ( x, w ) and but also avoid the possible controller singularity problem usually met with feedback linearization design when the adaptive fuzzy control is applied. For Eq. (19), the uncertainties F ' and E are the bound function in the control engineering. Hence there exists an unknown low bound for E . If ∆f and ∆g are
Adaptive Robust Fuzzy Tracking Control
335 335
the non-repeatable functions that can be considered to be continuous, then the uncertain function F ' is also a continuous function. In order to design control law, a fuzzy system is employed to approximate the uncertain function F ' . By using Lemma 3.1, the uncertain function F can be approximated by a fuzzy system. Then we can obtain the bound function as follows F '( z ) ≤ σ T ξ ( z ) + ε + D T
where z = [ x, ζ ] , ξ ( z ) = ')ξ1 ( z ) , ξ 2 ( z ) ,# , ξ K ( z ) 24 is the fuzzy base function T
which is known, σ is the weight parameters of the fuzzy system. ε is a parameter for respecting approximating accuracy. yd( n ) (t ) ≤ D . Suppose σ , D and ε are all unknown, above expression can be modified as follows −1 bmin F ' ( z ) ≤ θψ ( z )
(20)
where ψ ( z ) = 1 + ξ1 ( z ) + ξ 2 ( z ) + # + ξ K ( z )
is a known function and
θ = max [ε + D, σ 1 ,# , σ K ] . And E is a bound function which holds Assumption 2.4. The nonlinear controller vN is expressed as follows ( θψ ( z ) S ˆ vN = −θψ ( z ) ( (21) θψ ( z ) S + ς e − at where ς > 0 and a > 0 will be specified by the designer. For the parameter θ , let θˆ be the estimate of θ . Consider a constraint region Ω for approximation parameter θˆ for all t ≥ 0 to be contained inside given conT
θ
straint region Ωθ , the parameter update laws with projection algorithms are necessary. Here, the smooth projection idea proposed by Khalil [12] is used to tackle this problem. Suppose the region Ωθ is a convex hypercube. That is, consider ( ( Ω 01 = θˆ 0 ≤ θ ≤ c and Ωθ 1 = θˆ 0 ≤ θ ≤ c + δ where all the values c and
{
}
{
}
δ > 0 are chosen by the designer. A smooth projection algorithm with respect to θˆ can be obtained as & -*lφ , if θˆ > c and φ > 0 θˆ = + (22) -,lφ , otherwise $ c − θˆ / where l denotes the adaptive gain, φ = ψ ( z ) S and φ = %1 + 0φ . % δ 01 & Theorem 4.1 For the uncertain nonlinear system (3) subject to Assumptions 3.1 to 3.4, suppose that the uncertain function F ' ( z ) is approximated by the fuzzy system, the bound function in form (20) is available, an adaptive robust fuzzy tracking control scheme u (t ) is selected in Eqs. (8), (18) and (21) and the bound
336336Y. Yang and C. Zhou
parameter θˆ of uncertainty is estimated by Eq. (22), then the system tracking error converges to zero asymptotically, i.e. lim ( y (t ) − yd (t ) ) = 0 . t →∞
Proof: Choosing the following Lyapunov function candidate 1 1 1 V ζ , S ,θ% = ζ T Pζ + S 2 + bmin l −1θ% 2 2 2 2 where θ% = θˆ − θ .
(
)
Differentiating V along the trajectory of Eq. (19) and substituting Eq. (18) and (20), so we get 1 1 % %& V& = − ζ T Qζ − kS 2 + S ( F '+ (1 + E )v ) + bmin l −1θθ 2 2 1 1 % &% ≤ − ζ T Qζ − kS 2 + bmin l −1θθ 2 2 ( θψ (X )S ˆ (X )S ( + S F − (1 + E )θψ θψ ( X ) S + ς e − at Substituting Eq. (20) into above equation, we get 1 1 & V& ≤ − ζ T Qζ − kS 2 + bmin l −1θ% lψ ( x ) S − θˆ + bmin ς e− at (23) 2 2 & Now, from the adaptive law in [12] we get l −1θ% lψ ( x ) S − θˆ ≤ 0 and θˆ ∈ Ωθ 1
(
(
for all t ≥ 0 if θˆ ( 0 ) ∈ Ω01 . Then Eq. (23) is
)
)
1 1 V& ≤ − ζ T Qζ − kS 2 + bminς e − at 2 2 ≤ −Vn + bmin ς e − at 1 1 where Vn = ζ T Qζ + kS 2 . Then we get 2 2 t
V ( t ) − V ( 0 ) ≤ . Vn dt + bmin ς (1 − e − at ) ( 2a ) 0
Therefore,
0 ≤ . Vn dt ≤ Vn (0) + bmin ς (1 − e − at ) (2a ) t
0
and
t
lim . Vn dt ≤ Vn (0) + bmin ς (1 − e− at ) (2a) < ∞ x →∞ 0
Since Vn is a uniformly continuous function, according to Barbalat lemma [4], lim Vn = 0 . This implies that lim ζ = 0 and lim S = 0 based on the definition of t →∞
x →∞
t →∞
Vn . As results we get lim ei (t ) = 0, i = 1, 2,# , n − 1 and lim S = 0 which implies t →∞
lim ( y (t ) − yd (t ) ) = 0 t →∞
So the desired result has been obtained.
t →∞
337 337
Adaptive Robust Fuzzy Tracking Control
5 Design of ARFTC for an Inverted Pendulum System To demonstrate the effectiveness of the proposed algorithm, an inverted pendulum system is used for simulation. The Fig. 1 shows the plant composed of a pole and a cart. The cart moves on the rail tracks in horizontal direction. The control objective is to balance the pole starting from an arbitrary condition by supplying a suitable force to the cart. The same case studied has been given in Hwang [13]. The dynamic equations are described by
θ
θ&
u
Fig. 1. An inverted pendulum system
*- x&1 = x2 + -, x&2 = f ( x, w) + g ( x, w)u (t )
(24)
cos x1 mlx22 cos x1 sin x1 mc + m mc + m . + w(t ) , g ( x, w) = $ 4 m cos 2 x1 / $ 4 m cos 2 x1 / l% − l% − 0 0 & 3 mc + m 1 & 3 mc + m 1
g sin x1 − where f ( x, w) =
mc denotes the mass of the pendulum, m is the mass of the vehicle, 2l is the length of the pendulum and u is the applied force. x1 = θ is the angular position from the equilibrium position and x2 = θ& . The trajectory planning problem for a weight-lifting operation is considered in this paper. The desired angle trajectory is assumed to be yd (t ) = 0.1sin(2t ) . In this paper, we use following parameters for simulation: mc = 1 kg , m = 0.1 kg , l = 0.5 m l=0.5. And w(t ) is assumed to be w(t ) = 0.2sin(2t ) . Define five fuzzy sets for each x1 , x2 , e1 with labels Ahi1 (NL), Ahi2 (NM), Ahi3 (ZE), Ahi4 (PM), Ahi5 (PL) which are characterized by the following membership functions ' $ x + π 6 /2 2 ' $ x + π 12 / 2 2 i i µ A1 = exp ( − % 0 3 , µ Ahi2 = exp ( − % 0 3 hi () & π 24 1 34 () & π 24 1 34 (25) ' $ x /2 2 ' $ x − π 12 / 2 2 i i µ A3 = exp ( − % 0 3 , µ Ahi4 = exp ( − % 0 3 hi () & π 24 1 34 () & π 24 1 34
338338Y. Yang and C. Zhou
' $ x − π 6 /2 2 i 0 3 , i = 1, 2,3 () & π 24 1 34
µ A = exp ( − % 5 hi
Let’s set
* f 0 ( x) = 0 - g ( x) = 1 - 0 (26) + - ∆f ( x, w) = f ( x1 , x2 , w) -, ∆g ( x, w) = g ( x1 , x2 , w) − 1 If we get e1 (t ) = x1 (t ) − yd (t ) , e2 (t ) = x2 (t ) − y& d (t ) and k1 = 5 , Q = −2 , then P = 1 and vequ = − ( 5 + 0.5k ) S
where S = e2 + 5e1 . S& = −0.5kS + F ( x, e1 , w) + (1 + E )uN
where F ( x, e1 , w) = f ( x, w) + 5e1 + g ( x, w) − y& d (t ) , E = g ( x, w) .
If g ( x, w ) has a lower bound and the system function F ( x, e1 , w) is a continu-
ous complicated formulation system function, then F is also a continuous function and a fuzzy system is applied to approximate function F . Assuming that the fuzzy system is constructed from the following 75 fuzzy IFTHEN rules R1 : IF x1 is PL AND x2 is PL AND e1 is PL THEN y is α1 Ri : ....... ........
.........
R75 : IF x1 is NL AND x2 is NL AND e1 is NL THEN y is α 75 The fuzzy system can be obtained as follows 75
F ( x, e1 , w ) = # α iξ i ( x, e1 ) + ε i =1
(27)
So the bound function can be written as F ( x, e1 ) ≤ θψ ( x, e1 ) Accordingly the adaptive robust fuzzy tracking control scheme can be obtained for an inverted pendulum system as follows ˆ ( x, e ) S ( θψ 1 u (t ) = −(5 + 0.5k ) S − θψ ( x, e1 ) (28) ˆ ( x, e ) S + 5e−0.01t θψ 1
where k = 180 and θˆ can be calculated by Eq. (22). And l = 1 . c = 0.2 , 75
δ = 0.01 . φ = lψ ( x, e1 ) S , ψ ( x, e1 ) = 1 + # ξ i ( x, e1 ) . i =1
For the simulation, we choose the initial conditions as x1 (0) = 0.1 , x 2 (0 ) = 0 ,
θˆ(0 ) = 0 . The simulation results are shown in Fig. 2 and Fig. 3.
Adaptive Robust Fuzzy Tracking Control
339 339
Fig. 2. Simulation results for Control algorithm in Eq. (28). (Position of the inverted pendulum system. Solid line: actual position, Dashed line: reference position)
Fig. 3. Control force
Before presenting the outstanding advantages of ARFTC developed in this paper, we will briefly review the control law proposed in Wang [3] as follows 1 ' f ( x,θ1 ) − && u = uc + u s = − yd (t ) + k T e 24 gˆ( x,θ 2 ) )
− I1 sgn(eT PB )
1 ' ˆ f ( x , θ1 + f U ( x ) g L ( x) )
(29)
+ gˆ ( x,θ 2 ) uc + g U ( x)uc 42 where k = [2,1]T ,
f ( x) ≤ f U ( x) , g L ( x) ≤ g ( x) ≤ g U ( x) , B = [0,1]T and
Q = diag[10,10] . Then, we solve Lyapunov equation and obtain
340340Y. Yang and C. Zhou
Fig. 4. Simulation results for the adaptive parameter ˆθ
Fig. 5. Simulation results for tracking error
'15 52 P=( 3 ) 5 54 where I 1 = 1 if Ve > V (which is a constant specified by designer), I 1 = 0 if 1 T e Pe . 2 Defining five fuzzy sets the same as those in Eq. (25) for each x1 , x2 , twentyfive fuzzy rules for the systems fˆ ( x,θ1 ) and gˆ ( x,θ 2 ) respectively, and singleton Ve ≤ V . And Ve =
fuzzy model, the product inference and the center-average defuzzification are used. Hence, the function f ( x) and g ( x) can be approximated by the fuzzy systems fˆ ( x,θ ) = θ T ξ ( x) and gˆ( x,θ ) = θ T ξ ( x) where 1
1
1
2
2
2
Adaptive Robust Fuzzy Tracking Control
341 341
θ1 = ')θ11 ,θ12 ,# ,θ125 24 ∈ R 25 ξ1 ( x) = ')ξ11 ( x), ξ12 ( x),# , ξ125 24 ∈ R 25 θ 2 = ')θ 21 ,θ 22 ,# ,θ 225 24 ∈ R 25 ξ 2 ( x) = ')ξ 21 ( x), ξ 22 ( x),# , ξ 225 24 ∈ R 25
with components
ξ11 ( x) = µ h11 ( x1 ) µ h1 2 ( x2 ) D1 ξ12 ( x) = µ h11 ( x1 ) µ h22 ( x2 ) D1 ξ13 ( x) = µ h11 ( x1 ) µ h32 ( x2 ) D1 ,... ξ125 ( x) = µ h51 ( x1 ) µ h52 ( x2 ) D1
and 5
5
D1 = ## µ hi 1 ( x1 ) µ hj2 ( x2 ) i =1 j =1
and the construction of ξ 2 ( x) is similar to ξ1 ( x) . In Wang [3], the following adaptive law to adjust parameter vector θ1 was proposed: *−Γ1eT PBξ1 ( x), if ( θ1 < M f ) or ( θ1 = M f -and eT PBθ1T ξ1 ( x) ≥ 0) θ&1 = + (30) T T T -, P {−Γ1e PBξ1 ( x)} , if ( θ1 = M f and e PBθ1 ξ1 ( x) < 0) where the projection operation P {'} is defined as
P {−Γ1eT PBξ1 ( x)} = −Γ1eT PBξ1 ( x) + Γ1eT PB
θ1θ1T ξ1 ( x) θ1
2
And in Wang [3], the following adaptive law to adjust parameter vector θ 2 was used: T -*−Γ 2 e PBξ 2 ( x)uc , -,0,
θ&2 = +
if eT PBξ 2 ( x)uc < 0 if eT PBξ 2 ( x)uc ≥ 0
(31)
Here, for the parameters M f , V , Γ1 and Γ 2 , please refer to Wang [3]. The simulation results are shown in Fig. 6 to Fig. 8. Fig. 5 shows the simulation results of tracking errors by use of the proposed ARFTC and Fig. 8 shows the simulation results of tracking errors by use of the controller given in Eq. (29). From the results, we can see that the control performance of ARFTC is better than the controller given in Eq. (29). Hence, we can state that the ARFTC satisfies the following advantages that have been described in Section 4: only one function f ( x) is needed to be approximated by fuzzy systems and no matter how many states in the system are investigated and how many rules in the fuzzy system are used, only two parameters are needed to be adapted
342342Y. Yang and C. Zhou
on-line in ARFTC. However, for the traditional methodology (e.g. the control law proposed in Wang [3], even based on five fuzzy sets for each state variable and singleton fuzzy model aforementioned, there are 50 parameters needed to be adapted on-line for the fuzzy system fˆ ( x,θ1 ) and gˆ ( x,θ 2 ) when the fuzzy logic controller is implemented. And the traditional methodology can cause the increase of the number of parameters needed to be adapted exponentially with that of number of state variables or fuzzy sets. The computational complexity can be lessened dramatically and the learning time can be reduced vastly when using ARFTC developed in this paper. Then ARFTC has the potential to provide uncomplicated, reliable and easy-to-understand solutions for a large variety of nonlinear control tasks even for higher order systems.
Fig. 6. Simulation results for Control algorithm in Eq. (29). (Position of the inverted pendulum system: Solid line: actual position, Dashed line: reference position)
Fig. 7. Control force
Adaptive Robust Fuzzy Tracking Control
343 343
Fig. 8. Simulation results for tracking error
6 Conclusions In this paper, the tracking control problem has been considered for a class of nonlinear uncertain systems with the unknown system function and unknown gain function, and fuzzy logic systems have been used to approximate the lumped nonrepeatable state-dependent nonlinear function and the fuzzy system is used as the upper bound function of the uncertainty, then an adaptive robust fuzzy tracking control (ARFTC) algorithm, that can guarantee that the system tracking error converges to zero asymptotically, is proposed. The outstanding features of the algorithm proposed in this paper are that it can avoid the possible controller singularity problem in some of existing adaptive control schemes with feedback linearization techniques and the adaptive mechanism with minimal learning parameterizations, e.g. no matter how many states in the system are investigated and how many rules in the fuzzy system are used, only one parameter is needed to be adapted on-line, so that the computation load of the algorithm can be reduced and it is convenient to realize this algorithm in engineering. Finally, in order to research the efficiency of the algorithm proposed in this paper, it has been applied to an inverted pendulum system. We compare the simulation result by use of the algorithm proposed in this paper with the one by Wang [3] . Simulation results have shown the effectiveness of the proposed ARFTC control scheme.
Acknowledgments This work was supported in part by the Research Fund for the Doctorate Program of Higher Education under Grant No. 20020151005, the Science Foundation under Grant No. 95-06-02-22 and the Young Investigator Foundation under Grant No. 95-05-05-31 of the Ministry of Communications of P. R. China.
344
Y. Yang and C. Zhou
References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
Sastry, S. and Isidori. A., Adaptive control of linearization systems. IEEE Trans. on Automatic Control. Vol. 34, pp.1123-1131, 1989. Isidori, A., Nonliear Control Systems. 2nd ed. Berlin: Springer-Verlay, 1995. Wang, L. X., A Course in Fuzzy Systems and Control. New York: Prentice-Hall Inc., 1997. Slotine, J. E., and Li, W., Applied Nonlinear Control. New York: Prentice-Hall, Inc., 1990. Wang, L. X., Fuzzy systems are universal approximators. In proc. IEEE. International conf. On Fuzzy systems, San Diego, CA, pp.1163-1170, 1992 Lee, C. C., Fuzzy logic in control systems: fuzzy logic controller—Parts I&II. IEEE. Trans. Syst. Man. Cyber. Vol. 20, pp. 404-435, 1990, Ying, H., Sufficient conditions on general fuzzy systems as function approximators. Automatica. Vol. 30, No. 4, pp. 521-525, 1994. Wang, L. X. and Mendel, J. M., Fuzzy basis functions universal approximation, and orthogonal least-squares learning. IEEE Trans. on Neural Networks. Vol. 3, pp. 807-814, 1992. Fischle K. And Schroder D., An improved stable adaptive fuzzy control method. IEEE Trans. on Fuzzy Systems. Vol. 7, No. 1, pp. 27-40, 1999. Su, C. Y. And Stepanenko, Y., Adaptive control of a class of nonlinear systems with fuzzy logic. IEEE Trans. Fuzzy Systems. Vol. 2, pp.285-294, 1994. Yang, Y. S., Zhou, C. and Jia, X. L., Robust adaptive fuzzy control and its application to ship roll stabilization. Information Sciences. Vol. 142, pp. 177-194, 2002. Khalil, H. K., Adaptive output feedback control of nonlinear systems represented by input-output models. IEEE Trans. on Automatic Control. Vol. 41, pp. 177-188, 1996. Hwang, G. C. and Lin, S. C., A stability approach to fuzzy control design for nonlinear systems. Fuzzy Sets and Systems. Vol. 40, pp. 279-287, 1990.
A Complete Algorithm for Attribute Reduction B. Wang1, and S.B. Chen2 1. School of Materials Science & Eng., Shanghai Jiao Tong Univ., Shanghai 200030, P.R. China 2. School of Materials Science & Eng., Shanghai Jiao Tong Univ., Shanghai 200030, P.R. China [email protected]
Abstract. Rough sets theory is an effective mathematical tool dealing with vagueness and uncertainty. It has been applied in a variety of fields such as data mining, pattern recognition or process control. It is significant to find high performance algorithms for attribute reduction. Most of presented algorithms were incomplete and the work on complete algorithms was comparatively little. This paper presented a complete algorithm based on the principle of discernibility matrix and discussed the time complexity of the algorithm. The proof of the completeness of the algorithm was also given in the paper. Finally, the validity of the algorithm was demonstrated by two classical databases in the UCI repository.
1 Introduction Rough sets theory[3] introduced by Pawlak in 1982 is an effective mathematical tool dealing with vagueness and uncertainty. It has been applied in a variety of fields such as data mining, pattern recognition or process control[4]. Attribute reduction is significant in most applications, so that it has been a major concern in the research on rough set theory. Let S be a decision table and we have S=, where U is the universe, the disjoint attribute sets C and D are referred to as the condition and decision attributes respectively. It is necessary to point out U, C and D should be non-empty and finite sets. According to the definition given by Pawlak[3], if P⊆C is a reduction of C, P satisfies follow two conditions: (1)POSP(D)=POSC(D); (2)POSP−{a}(D)≠POSC(D) for each a∈P. In [3], Pawlak presented the original algorithm for attribute reduction whose time complexity was O(2n), where n=|C|. So the algorithm is just a theoretical algorithm and it is limited very much in actual applications. The output of the algorithm, which is a subset of C, satisfies both two conditions above. Some attribute T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 345−352, 2004. Springer-Verlag Berlin Heidelberg 2004
346346B. Wang and S.B. Chen
reduction algorithms with polynomial-time complexity have been introduced, notably those in [1, 2, 5]. These algorithms are more practical but their outputs only satisfy the first condition above. If the output of an attribute reduction algorithm satisfies both two conditions above, we say the algorithm is complete. If the output of an attribute reduction algorithm only satisfies the first condition above, we say the algorithm is incomplete. What we want are complete algorithms with polynomial-time complexity. In [6], firstly, authors introduced a complete algorithm. It is needed to define a total relation over the set of condition attributes beforehand and the order can be considered as prior knowledge. In this way, one can fully use the experience of experts. At the same time, the output of the algorithm is unique for a given order of attributes. However, the algorithm still has limitation when no prior knowledge is available. Secondly, authors presented a optimal reduction algorithm. It is also complete and its time complexity is not lower than O(m3n2) in the worst case. To the author's knowledge, there is no other published complete algorithm with polynomial-time complexity. In this paper, an algorithm based on the principle of discernibility matrix is introduced. We show that the algorithm is a complete attribute reduction algorithm with polynomial-time complexity and its time complexity is O(m2n2) in the worst case. The validity of the algorithm was demonstrated by two classical databases in the UCI repository. This paper is organized as follows. In Section 2, some concepts and definitions are introduced. The algorithm and the analysis of the time complexity of it are given in Section 3. In Section 4, we show that the algorithm is complete. In Section 5, we give results of experiments and the conclusion is in Section 6.
2 Concepts and Definitions Let S= be a decision table. Suppose |U|=n. The discernibility matrix of S denoted by MS is a n×n matrix defined thus: (cij)={ a∈C f (xi a)≠ f (xj a) and w(xi,xj)=1} for i,j=1, 2, ..., n. where !
"
"
1 xi∈POSC(D) and xj∉POSC(D) w(xi,xj) =
1 xi∉POSC(D) and xj∈POSC(D) 1 xi, xj∈POSC(D) and (xi, xj)∉IND(D) 0 otherwise
Table 1. An example of decision tables U x1 x2 x3 x4 x5 x6
a 2 1 1 0 1 2
b 2 2 2 0 0 0
c 0 0 0 0 1 1
d 1 0 1 0 0 1
347 347
A Complete Algorithm for Attribute Reduction
Table 2. The discernibility matrix 1
1 ∅
2 {a}
3 {a}
2
{a}
∅
∅
3
{a}
∅
∅
4 5
{a b} {a b c} ∅
{a b} {b c} {a b c}
!
!
6
!
!
!
!
!
5 {a b c} {b c}
{a b} {b c}
4 {a b} {a b} {a b} ∅ ∅
∅ ∅
{a b c} {a b c} {a c} {a}
{a b c}
{a c}
{a}
∅
!
!
!
!
!
!
!
!
!
!
{b
6 ∅
!
!
c}
!
!
!
!
!
For example, Table 1 is an example of decision talbes and Table 2 is its discernibility matrix. It is necessary to give follow definitions before discussing our algorithm in detail. Let MS be the discernibility matrix of decision table S. The set of non-empty entries of MS will be called the discernibility set of S, denoted by $S. Obviously, the discernibility set of a decision table is the collection of subsets of condition attributes. We associate a subset: Core($S)={a: {a}∈ $S } called the core of $S. Let B⊆C. A S− (B) be the B—elimination set of $S and
A S− (B)={A' A∈$S A'=A−B and A'≠∅ }. A S+ (B) is the B—selection set of $S !
"
A S+
and (B)={A∈$S A∩B=∅}. It is clear that the B—elimination set and the B—selection set are all subsets of condition attributes. To illustrate these definitions, we take the discernibility matrix in Table 2 for example. $S={{a}, {a, b}, {a, b, c}, {b, c}, {a, c}}. Core($S)={a}. Suppose B={a}. !
A S− (B)={{b}, {b, c}, {c}}. A S+ (B)={{b, c}}.
3 The Complete Algorithm and Its Time Complexity In this section, we will introduce a complete attribute reduction algorithm and analyze its time complexity. The input of the algorithm is the discernibility matrix $S of a given decision table S. Algorithm 3.1 (1)R⇐∅, P⇐∅, T⇐∅; (2)$=$S; (3)If $=∅, stop, otherwise continue; (4)P⇐ Core($);
348
B. Wang and S.B. Chen
(5)If P=∅, go to Step 6, otherwise go to Step 7; (6)Select one attribute a in C–(R $ T) randomly, $⇐ A − ({a}) , T⇐T $ {a}, and go to Step3; (7) R⇐R $ P, $⇐ A + (R) and go to Step 3. When the algorithm stop, R is the output of the algorithm, namely the attribute reduction. Fig. 1 is the flow chart of Algorithm 3.1.
START
R⇐∅ !
P⇐∅
T⇐∅ !
$=∅
!
$
Y !
END
N select a
P⇐Core($)
$⇐ $ !
−
({a})
P=∅
!
Y !
N R⇐R $ P !
$⇐ $
Fig. 1 The flow chart of Algorithm 3.1
Suppose |U|=n and |C|=m, where m and n are positive integer. In the worst case, |$|=
n2 − n = N . The time complexity of computing Core($) is O(mN). Let B⊆C. 2
The time complexity of computing A − (B) and A + (B) are also O(mN). Algorithm 3.1 will repeat m times at most so the time complexity of Algorithm 3.1 is O(m(mN+mN))=O(2m2
n2 − n )=O(m2n2) at the worst case. It is clear Algorithm 3.1 2
is faster than the optimal reduction algorithm[6] in the worst case.
A Complete Algorithm for Attribute Reduction
349 349
4 The Completeness of the Algorithm In this section, we show Algorithm 3.1 is complete, namely, the output of the algorithm satisfies the two conditions presented in Section 1. It has been proved those two conditions are equivalent to follow two conditions[6]: (1)For ∀ A∈$S, A # P≠∅; (2)For ∀ a∈P, ∃ A∈$S, such that A # P−{a}=∅. To prove the completeness of the algorithm, we can prove the output of the algorithm, that is R, satisfies these two conditions. For this end, it is worth giving some properties of the algorithm. Step 3 to Step 7 of the algorithm form a loop. Since U and C are finite sets, it is obvious the algorithm terminates absolutely. So we can safely suppose the algorithm iterate k times, where constant k is a positive integer. During one iteration, if Step 6 is executed in the loop, we say the iteration is an elimination iteration, otherwise Step 7 is executed and we say the iteration is a selection iteration. In an elimination iteration one attribute is added to T(the set of attributes eliminated by the algorithm) and in an selection iteration some attributes are added to R. We use the sequence $0, $1, ..., $i, ..., $k−1, $k to indicate the changing of $ during the algorithm running, where i=0, ..., k. The first iteration changes $ from $0 into $1 and the ith iteration changes $ from $i−1 into $i. Accordingly, the sequence R0, R1, ..., Ri, ..., Rk−1, Rk indicates the changing of R and the sequence T0, T1, ..., Ti, ..., Tk−1, Tk indicates the changing of T. We have $0=$S, $k=∅, R0=∅, Rk=R(the output of the algorithm), T0=∅, Tk=T, and if i≤j, Ri⊆Rj and Ti⊆Tj. It is necessary to point out that C−R=T is not always hold. Let C={a, b, c, d, e, f} and the input $S={{a, b, d}, {a, c, d}, {b, c, d}, {a, b, c}, {e, f}}. The output of the algorithm is R={c, d, f}. Table 3 shows how the algorithm runs. Table 3. Running process of the algorithm i 0 1 2 3 4 5
$i {{a, b, d}, { a, c, d}, {b, c, d}, { a, b, c}, {e, f}} {{b, d}, {c, d}, {b, c, d}, {b, c}, {e, f}} {{d}, {c, d}, {c}, {e, f}} {{e, f}} {{f}} ∅
Ri ∅ ∅ ∅ {c, d} {c, d} {c, d, f}
Ti ∅ {a} {a, b} {a, b} {a, b, e} {a, b, e}
Proposition 1 If the ith iteration is an elimination iteration, ∃ A'∈ $i such that A'⊆A for ∀ A∈$i−1, where i=1, ..., k.
350350B. Wang and S.B. Chen
Proposition 2 If the ith iteration is a selection iteration, A # Ri≠∅ if A∉ $ for ∀ A∈$i−1, where i=1, ..., k. Proposition 3 A # Ri≠∅ if ¬∃ A'∈ $i such that A'⊆A for ∀ A∈$i−1 and i=1, ..., k. Proposition 4 A # Rj≠∅ if ¬∃ A'∈ $j such that A'⊆A for ∀ A∈$i , i < j and i, j=1, ..., k. Proposition 1 and Proposition 2 are easy to be proved. By Proposition 1 and Proposition 2 we can show Proposition 3 is right and then we get Proposition 4. Theorem 1 Let $S and R be the input and the output of Algorithm 3.1 respectively. A # R≠∅ for ∀ A∈$S. Proof Let the algorithm iterate k times, where k∈Z and k≥0. If k=0, the theorem is right obviously. If k>0, we have A0=AS and Ak=∅. For ∀ A∈AS, A∈A0 holds. Note that Ak=∅, that is ¬∃ A'∈Ak such that A'⊆A. By Proposition 4, A # Rk≠∅, namely A # R≠∅. Proposition 5 For ∀ A∈$i, ∃ A'∈ $0 such that A⊆A' and A'−A⊆Ti, where i=0, ..., k. Proposition 6 T # R=∅. Both Proposition 5 and 6 are obvious. Theorem 2 Let $S and R be the input and the output of Algorithm 3.1 respectively. For ∀ a∈R, ∃ A∈$S such that A # R−{a}=∅. Proof Let the algorithm iterate k times, where k∈Z and k≥0. If k=0, the theorem is right obviously. If k>0, we have A0=AS and Ak=∅. For ∀ a∈R, ¬∃ i(0
!
5 Experiments In this section, we will process two decision tables came from the UCI repository using the approximate core algorithm. The first decision table is formed from "1984 United States Congressional Voting Records Database". It has 435 objects, 16 condition attributes and 1 decision attributes. The missing values (the mark ? in the data file) in the original data will be regarded as one new value since we do not concern how to deal with missing values in this paper. The attribute reduction acquired by our algorithm includes the first, second, third, forth, ninth, eleventh, thirteenth, fifteenth and sixteenth condition attribute. The total is 9 and these attributes are listed in Table 4.
A Complete Algorithm for Attribute Reduction
351 351
Table 4. The attribute reduction of the first decision table No. 1 2 3 4 9 11 13 15 16
Attribute information handicapped-infants water-project-cost-sharing adoption-of-the-budget-resolution physician-fee-freeze mx-missile synfuels-corporation-cutback superfund-right-to-sue duty-free-exports export-administration-act-south-africa
The second decision table is formed from "Zoo database". It has 101 objects, 16 conditions attributes and 1decison attributes. The original data has no missing value. The attribute reduction acquired by our algorithm includes the third, sixth, ninth, thirteenth and sixteenth condition attribute. The total is 5 and these attributes are listed in Table 5. Table 5. The attribute reduction of the second decision table No. 3 6 9 13 16
Attribute information eggs aquatic backbone legs catsize
Both two attribute reductions satisfy the complete condition, which validate the completeness of the approximate core algorithm.
6 Conclusion This paper introduced a complete algorithm for attribute reduction. The algorithm is based on the principle of discernibility matrix and its time complexity is O(m2n2) in the worst case. The proof of the completeness of the algorithm is also given in the paper.
References 1. 2.
Hu X H, Cercone N. Learning in relational databases: a rough set approach. Computational Intelligence, 1995, 11(2): 333-338 Miao Duo-Qian, Hu Gui-Rong. A heuristic algorithm for reduction of knowledge. Journal of Computer Research and Development. 1999, 36(6): 681 684 !
352352B. Wang and S.B. Chen
3. 4. 5.
Pawlak Z. Rough sets. International Journal of Computer and Information Science, 1982, 11(5): 341-356 Slowinski R. Intelligent Decision Support—Handbook of Applications and Advances of the Rough Sets Theory, Dordrecht: Kluwer Academic Publishers, 1992 Wang Jue et al. Data enriching based on rough set theory. Chinese Journal of Computers. 1998, 21(5): 393 400 Wang Jue, Wang Ju. Reduction algorithms based on discernibility matrix: the ordered attributes method. Journal of Computer Science and Technology. 2001, 16(6): 489-504 !
6.
Preliminary Probe of Embedded Real-Time Operating System and Its Application in Welding Control Wenjie Chen, Shanben Chen, and Tao Lin Institute of Welding Technology, Shanghai Jiao Tong University, Shanghai, 200030, P.R. China [email protected]
Abstract. This paper firstly introduces the concept of real-time and embedded system, then analyses the character of welding and the advantages of adopting RTOS in welding process control. In the end, simulation software was developed to demonstrate the advantages of applying RTOS in welding process control.
1 Introduction Generally speaking, there are two categories of computer application [1]: the general-purpose computer application, such as PCs or workstations, and embedded systems application, such as mobile phones or washing machines. Embedded system means that the computer is built into a system and is not seen by the user as being a computer. Real time software can run on embedded computers or on computers that are doing control functions but are clearly freestanding and not built into the target system. Real time software is a sub-domain of software generally with distinct requirements and characteristics. Real time software differs from conventional software in that its results must not only be numerically and logically correct, they must also be delivered at the correct time. There are two types of real-time systems [4]: hard and soft. In hard real-time systems, tasks have to be performed not only correctly but also on time and missing a deadline has catastrophic results for the system. In a soft real-time system, tasks are performed by the system as fast as possible, but the tasks don't have to finish by specific times and missing a deadline only causes a quality reduction. Embedded systems have several common characteristics [2]: 1) Embedded system takes computer or microprocessor as its center and cooperates with peripheral mechanical or electronic equipments to accomplish a particular purpose. 2) Reactive and real-time: Many embedded systems must continually react to changes in the system’s environment, and must compute certain results in real time without delay. T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 353−357, 2004. Springer-Verlag Berlin Heidelberg 2004
354354W. Chen, S. Chen, and T. Lin
3)
Tightly constrained: Embedded systems often must be sized to fit a incommodious space, must perform fast enough to process data in realtime, and must consume minimum power to extend battery life or prevent the necessity of a cooling fan. In a word, embedded real-time system is an important computer application system. Relatively speaking, these systems have high even rigorous demand of time character, reliability and security. Real-time operating system (RTOS) is the platform of embedded application software. Nowadays, there are many RTOS, both including commercial and individual developed system. Such as pSOS, Vxworks and µC/OS [3]. In this paper, µC/OS was used as platform for simulation software of welding process control.
2 Advantages of Implementation of RTOS in Welding Control The main advantage of adopting RTOS in embedded application is highly improving system’s reliability. Especially for control system, the non-breakdown is a basic requirement. For this problem, adopting some measures such as improving anti-interference ability of hardware system to satisfy electromagnetic compatible demand can do some better. On the other hand, taking some measures on software can improve the reliability of system strikingly. For a long time, conventional developing style is: aiming at certain application and drawing procedure flow chart, then coding. This type of program is so called structured program. While encountered strong interference, a hanging-up at any place of the program will cause breakdown of system and must rely on watchdog of hardware to reset the system. For RTOS managed system, however, this is not necessary. Because for RTOS managed system, there are many independent tasks or processes, the interference can only cause one task or process blocked, while the others are alive. Moreover, the blocked task or process can be killed or reinstated by other tasks or processes. The second advantage of adopting RTOS in embedded system is increasing development efficiency and shortening development period. A complicated application can divide into several independent tasks or modules, and each module’s developing and debugging does not affects the other modules. RTOS provides a nice multitask debugging environment, and the application seems doing several task at the same time. In a sense, computer that has no operating system is no use. In embedded application, only taking CPU embedded into the system meanwhile embedded in oper. ating system is true embedded application [2]
3 Necessary of Implementation of RTOS in Welding Control As for welding, different welding technique has different requirement of output characteristic of welding power source. For example, argon tungsten-arc welding
Preliminary Probe of Embedded Real-Time Operating System
355 355
need constant current external characteristic, manual arc electrode welding need constant potential external characteristic and carbon arc air gouging and submerged-arc welding often needs constant voltage output characteristic. These welding techniques are widely used in boiler, pressure vessel and heavy-duty machinery manufacturing. Moreover, these welding techniques are often used on a product even one joint at the same time. Developing such a power source that can satisfy the various outputs characteristic can bring convenience for users and help users to decrease investment of equipment. Now, embedded microcomputer controlled power source provides this possibility. The application environment of welding power source is very formidable. Most of them are used in noisy workshop or field. In many cases, the high-frequency inference is heavy, and it is very easy to cause microcomputer controlled power source to breakdown. To avoid occurrence of this case, besides the measures such as masking, optoelectronic isolating and electromagnetic compatibility designing, the RTOS should considered to be used in microcomputer controlled welding power source. The character of welding process control is that the parameters needing adjustment is much, for example, the welding current, the welding voltage, the wire feed rate, the gas-flow rate and so on. If all of the adjustments adopt single process to control, then not only the design is very complicated but also the level of reliability is not high. It is easy to lead whole system breakdown just because a little procedure’s problem. While, the system that adopted RTOS is invulnerable. It can divide these tasks into several independent processes; let each process to in charge of each adjustment task. The scheduling, resource’s distribution and inter-process coordination is up to RTOS. The problem of one process cannot lead to the whole system’s breakdown. Another character of welding process control is that needing rapid response to external change. If adopt RTOS, the different process will be assigned a different priority. The scheduling module of RTOS often designed as preemptive mode. As for feedback information, the each process can reactive precisely and promptly. So keep every adjustment in real-time. The real-time is often the critical requirement of welding process control. If adopt RTOS, the system’s transplant ability increase. Because the isolation of RTOS, i.e. the hardware character has been encapsulated by RTOS, the application can easy transplant from this type of CPU to another type of CPU without modification of application codes. While, if not adopt the RTOS, the transplantation must involves massive modification of application codes even entirely recode.
4 Simulation of Welding Process Control Based on RTOS µC/OS-II is a high performance, deterministic real-time kernel and can be embedded in various products. Thousands of people around the world are using µC/OS in all kinds of applications such as cameras, engine controls, network adapters, ATM machines, industrial robots, and many more [5].
356
W. Chen, S. Chen, and T. Lin
To demonstrate the advantages of RTOS in welding process control, this paper use the µC/OS-II as RTOS kernel and develop a simulation application of welding dynamic process. main
Taskstart
TaskController
TaskSpeed
TaskCurrent
Fig.1 Task graph
In this paper, we developed a welding process control simulation application – weldsimu. It’s consists of five independent tasks, µC/OS-II creates two ‘internal’ tasks: the idle task and a task that determines CPU usage rate. Weldsimu creates 3 other tasks. The TaskStart() task is created by main() and its function is to create the other tasks, display the simulation statistics on the screen and save the simulation data to files. The Taskcontroller() is created by TaskStart(). It is a double input and double output fuzzy controller and its function is to control the dynamic process of welding pool. The double inputs are error and error’s change respectively; the double outputs are welding current and welding speed. As we known, the welding current increase, the weld pool width increase, but there is a relatively long lag. The welding speed increase, the heat input decrease, but the lag is relatively short. Therefore, in some cases, for instance, there is a leap of heat transmission condition of work piece, if only adjust welding current, the welding quality can not be ensured, welding current and welding speed be adjusted simultaneously should be better. In this paper, we allocate for welding current and welding speed adjustment each task. In conventional programming, these functions are fulfilled in one and only one task. If there is some problem lead task to hang up, the whole system is breakdown. But in the real-time multitask operating system, one task out of function, the other is still work. The following is the designed task graph. From this task graph, we can know that the application’s update is easy. Because each task has independent stack space and CPU context status, the modification of one task doesn’t affect the other tasks. Fig.2 is the interface of simulation software. From this display, we can observe the parameters of simulation process. There are 5 tasks running in computer and 10 task switches per second. The CUP usage rate is about 16%.
Preliminary Probe of Embedded Real-Time Operating System
357 357
Fig.2 The interface of simulation application
5 Conclusions 1. RTOS has many advantages; the main advantage of adopting RTOS in embedded application is highly improving system’s reliability. The second advantage of adopting RTOS in embedded system is increasing development efficiency and shortening development period. 2. Different welding technique has different requirement of output characteristic of welding power source. Embedded microcomputer controlled power source can satisfy these requirements. 3. The characters of welding process control need the introduction of embedded real-time operating system into welding process control. 4. The simulation of welding process results indicate RTOS is applicable to welding control, and there are many advantages for adopting RTOS in software programming and maintenance.
References 1. 2. 3. 4. 5.
D.M. Auslander, J.R. Ridgely(2000), Design and Implementation of Real Time Software for Control of Mechanical Systems. Dr. Jürgen Sauermann, Melanie Thelen.(2000), Real-time Operating Systems. Jack G. Ganssle(1999), The art of Designing Embedded systems. Labrosse, Jean J.(2000), µC/OS, The Real-Time Kernel. Ted Van Sickle(), Programming Microcontrollers in C, Second Edition.
358
Estimation of Oxygen Concentration Using Dynamic Recurrent Neural Network Based on PCA Dazhong Sun, Fangjun Ye, and Xingsheng Gu Research Institute of Automation, East China University of Science & Technology, Shanghai 200237, P.R. China [email protected]
Abstract. At present, much more research in the field of soft-sensing modeling is concerned. In the method of building soft-sensing models, artificial neural network is more frequently used. In this paper, a model based on dynamic recurrent neural network is studied and the principal component analysis is also used to pretreat the data set before modeling. The generalization result shows that the model is very accurate.
1 Introduction In the process of continuous catalytic reforming (CCR),oxygen concentration is an important variable. In this article, the objective is to build a soft-sensing model for oxygen concentration so that it can be estimated using other easy-to-measure process variables. High estimation accuracy for this model is demanded for economic profit. Through mechanical analysis, we find that there are many secondary variables relating to oxygen concentration. Some of the variables are correlated. That is to say, there exists some redundant information and some information extraction methods must be used to treat the sample data before they are used to build the soft-sensing model. Principal Component Analysis (PCA)[1] among other related techniques have been demonstrated as useful tools for analysis of data and modeling of the systems.
2 Industrial Background The catalytic reforming system is an important chemical process in oil refineries. A typical catalytic reformer consists of a number of reactors, a regeneration tower and a lot of other equipments as shown in Fig. 1. The continuous regeneration of catalyst is an important symbol of the CCR system. Fig.1. shows the simplified flow diagram of the catalyst. The catalyst to be regenerated firstly goes into the T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 358−364, 2004. Springer-Verlag Berlin Heidelberg 2004
Estimation of Oxygen Concentration Using Dynamic Recurrent Neural Network
359 359
catalyst collector from the reactor. In the lift line, the catalyst is lifted by N2 and enters the regeneration tower. Coking, oxidize/chloridize reaction and dry take place in the regeneration tower. At last, the catalyst reaches the reduction zone lifted by highly pure H2.There, the platinum on the catalyst’s surface changes from oxidation state to reduction state which is an important step for the catalyst to be reactivating. The reactivating catalyst then enters the first reforming reactor for use again. In the regeneration process of the catalyst, the oxygen concentration is a very important variable. If the variable can not be well controlled, it is very harmful to the regeneration tower and the catalyst itself. Because of this, many oil refineries have spent much money to buy on-line analyzers. But on-line analyzers have an analytical time delay which may degrade the control performance. Furthermore, on-line analyzers require frequent and high maintain cost. Owing to all of the above disadvantages, many oil refineries have considered using some efficient methods to solve the problems which contribute to the progress of softsensing techniques.
Fig. 1. Simplified flow diagram of the catalyst
3 Elman Network In dynamic recurrent neural networks (DRNN)[2,3], Elman network[4,5] is common and used often. It is a simple recurrent neural network, as shown in Fig.2. It can be seen that in a Elman network, in addition to the input, hidden and the output units, there are also context units. The feedforward weights such as Wih, Wch and Who are modifiable. The recurrent weights Whc is fixed. The output vector is
360360D. Sun, F. Ye, and X. Gu
u(k-1)
Wih Who
Whc
Y(k) Wch
context
X(k)
Xc(k) Fig. 2. Elman network
Y(k) Rm and the input vector is u(k-1) Rr. X(k) Rn is the hidden layer output vector. Xc(k) is a n×n matrix. The relationship between the input and output can be represented as equation (3.1) to equation (3.3). F and G are activation functions of hidden units and output units respectively and they can be adopted as linear mappings. !
!
!
X (k ) = F (Wch X c (k ) + Wih u (k − 1))
(3.1)
X c (k ) = X (k − 1)
(3.2)
Y (k ) = G (Who X (k ))
(3.3)
The dynamic back-propagation (DBP) learning rule is used to train the Elman network. DBP can be presented as follows: ji
∆who = ηδ io x j (k ) ( j = 1,2, # , n; i = 1,2, # , m) qj
∆wih = ηδ hj u q (k − 1) ( j = 1,2, # , n; q = 1,2, # , r ) m
∂x j (k )
i =1
lj ∂wch
ij ∆w = η # (δ io who ) lj ch
where
∂x j (k )
( j = 1,2, # , n; l = 1,2, # , n)
n ′ $ lj ∂xi ( k − 1) ' ( ) ( 1 ) f x k wch = ⋅ − + # % l ( j lj lj ∂wch ∂wch i =1 ) & δ io = ( y di (k ) − y i (k )) g i ′ (⋅) m ′ δ hj = # (δ io whoji ) f j (⋅) i =1
(3.4) (3.5) (3.6)
(3.7) (3.8) (3.9)
Estimation of Oxygen Concentration Using Dynamic Recurrent Neural Network
361 361
At the beginning of the training process, the activation of the hidden units are unknown. They are usually set to one-half of their maximum range. For a sigmoidal activation function the initial values can be set to 0.5. For a hyperbolic tangent activation function they can equal to 0.
4 Model Using Elman Network Based on PCA In this paper, the application of Elman network combined with PCA is to build a soft sensor for oxygen concentration. High estimation accuracy for this model is demanded for economic profit. The data we used is from one oil refining factory. Something must be done prior to build the sensor. 4.1 Data pretreatment
[6]
Data pretreatment is an important issue for developing soft sensors. Firstly, the detection and deletion of singular samples is needed in order to improve generalization results. The singular samples caused by human error or unusual system disturbances are abnormal or wrong. Secondly, normalizing the input and output data sets so that they will have means of zero and standard deviations of 1 in order to avoid having important variables whose magnitudes are small from being overshadowed by less important but with large magnitude variables. The last is to extract meaningful information from the original noisy and redundant data sets. In real process, the data is sampled two times every minute and in half an hour,60 samples are got. In order to eliminate the noise information from the original data, the data is filtered every half an hour before used to build models. 4.2 Model and results Through mechanism analysis, we select 21 secondary variables including temperature, pressure, flux and coke value which influence the oxygen concentration. They will be used to build a model as input variables and the oxygen concentration is the objective variable. Using the PCA algorithm, the data can be adequately described with far fewer factors than the original variables with no significant loss of information. Table 1 lists the results of the PCA. From Table 1, it can be seen that the first five principal components (PC) include almost all of the variance (above 95%). It means that the first five principal components use the most essential information of the original data and the redundant information is eliminated. The first five principal components are used as the input variables of Elman. The selection of the number of hidden nodes of Elman is important and it is usually done by cross validation[7]. The optimal number of hidden nodes is determined according to the testing data root mean square error (RMSE). Usually, the
362362D. Sun, F. Ye, and X. Gu
number of hidden nodes in which a local minimum in the testing RMSE is reached is selected as being optimal. At last, we select 14 hidden nodes. 200 samples are used to train the network and 100 samples are used to check the generalization result of the model. The training and generalization results for the PCA method are shown in Fig.3. and Fig.4.. Fig.3. shows the results of training for the PCA model and Fig.4. is the generalization result of the model The root mean square errors for training and checking are 0.0160 and 0.0173 respectively which show that the model’s generalization ability is very strong. Table 1. The result of PCA PC No.
Eigenvalue
Variance (%)
Total variance (%)
1
2547.493
56.227
56.227
2
880.820
19.441
75.668
3
420.269
9.276
84.944
4
380.932
8.408
93.352
5
149.907
3.309
96.661
6
51.721
1.141
97.802
7
35.866
0.791
98.593
8
27.770
0.613
99.206
9
17.180
0.379
99.585
10
7.477
0.165
99.750
11
4.769
0.105
99.855
12
2.475
0.055
99.910
13
1.530
0.034
99.944
14
1.200
0.026
99.970
15
0.619
0.014
16 17 18
0.254 0.172 0.122
99.984
5.606×10
-3
99.990
3.796×10
-3
99.994
2.693×10
-3
99.997
-3
99.998 99.999
19
0.075
1.655×10
20
0.041
9.049×10-4
0.031
-4
21
6.842×10
100.000
Estimation of Oxygen Concentration Using Dynamic Recurrent Neural Network
Oxygen Concentration
0.65
363 363
Analysis value Elman output
0.6 0.55 0.5 0.45 0
20
40
60
80
100
120
140
160
180
200
280
290
300
Sample Number
Fig. 3. Training result of the Elman model
Oxygen Concentration
0.65
Analysis value Elman output
0.6 0.55 0.5 0.45 200
210
220
230
240
250
260
270
Sample Number
Fig. 4. Generalization result of Elman model
5 Correction of Soft Sensor In a real process, the operating conditions may often change, so if the process drifts away from the conditions in which the soft sensor is built, the soft sensor will not provide satisfying results. In this case, we should consider correcting the model. There are two ways to correct the Elman model. We can change the weights or to change the structure of the neural networks. We firstly collect many new samples in the new operating condition and then use them to train the model.
364364D. Sun, F. Ye, and X. Gu
The frequency that we correct the model depends on the stability of the process. If the process is stable in general, the frequency is low. Otherwise, the frequency is high.
6 Conclusions To estimate hard-to-measure and important variables from the easy-to-measure variables is a promising topic in the industrial fields. The model using Elman network based on PCA has strong generalization ability. It also avoids the time delay of on-line analyzer. The generalization result shows that the model is highly accurate. This can be used for reference by some other similar problems.
References 1. Kaspar MH, Ray WH (1992) Chemometric methods for process monitoring and highperformance controller design. AIChE Journal 38(10): 1593-1608. 2. Shaw AM, Doyle III FJ (1997) Multivariable nonlinear control applications for a high purity distillation column using a recurrent dynamic neuron model. J.Proc.Cont 7(4): 255-268. 3. Karjala TW, Himmelblau DM (1996) Dynamic rectification of data via recurrent neural nets and the extended Kalman filter. AIChE Journal 42(8): 2225-2239. 4. Wenhua Zeng (2001) On line estimation of aviation kerosene specific gravity based on diagonal recurrent neural network.Computer automated measurement & control. 9(2): 50-51. 5. Pham DT, Liu X (1996) Training of Elman networks and dynamic system modeling.International Journal of Systems Science 27(2): 221-226. 6. Wei Zhong, Jinshou Yu (2000) MIMO soft sensors for estimating product quality with on-line correction.Trans IchemE 78(A): 612-620. 7. Osten DW (1988) Selection of optimal regression models via cross validation. Journal of Chemometrics 2: 39-48.
365
Robust Stability of Interval Lur'e Systems: A Bilinear Matrix Inequality Approach J. T. Sun Dept. of Appl. Math., Tongji University, 200092, P.R.China [email protected]
Abstract. This paper provides a sufficient condition for an interval Lur'e system to be globally exponentially stable with a damping factor. The Lur'e system consists of an interval linear dynamical system and a sector-bounded memoryless time-varying nonlinear term. The sufficient condition is described by a simple bilinear matrix inequality. The relationship between the stability of symmetric interval matrix and globally exponentially stability of interval Lur'e system is established.
1 Introduction Many researchers have investigated the stability of feedback systems whose forward path is a linear time-invariant (LTI) system and whose feedback path is a memoryless (possibly time-varying) nonlinearity (Vidyasagar 1993). These feedback systems are called Lur'e systems. The simplest version of the circle criterion guarantees that the Lur’e systems whose linear part is strictly positive real and whose nonlinearity belongs to the sector [0,+∞) is globally exponentially stable. Grujic and Petkovski (1987) discussed the robustness of the Lur'e system. Tesi and Vicino (1991), Dahieh et al. (1993), Mori et al. (1995), Wada et al. (1996,1998) and Sun et al.(2001) have investigated the parametric absolute stability of Lur'e systems. Since the linear matrix inequality (LMI) can be solved numerically efficiently (Boyd et al. 1994), Konishi et al. (1999) presented some sufficient condition for the Lur'e system with the uncertainties satisfying a matching condition to be globally exponentially stable by linear matrix inequality. In the systems with structured parameter uncertainty, there are mainly two kinds of representation of parameter uncertainty. The first representation is when the uncertainty in the matrix A is given by ∆ =DFE with FFT ≤ I , D and E are known matrix, i.e. the uncertainty satisfying a “matching condition”. The second r e p r e s e n t a t i o n i s o n e i n wh i c h t h e u n c e r t a i n t y ∆ i s g i v e n b y ∆=#γi Ai , where the γi are uncertain parameters satisfying | γi | ≤ ri . Gu et al. (1990) T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 365−371, 2004. Springer-Verlag Berlin Heidelberg 2004
366
366
J.T. Sun
showed by an example that the second representation is more general than the first one. In a practical situation one would have an uncertainty upon the components of the matrix A, represented by an upper bound as |∆|# E, which means that |)ij | $ eij , where )ij and eij denote, respectively, the ij-entries of ∆ and E. The matrix E is assumed to be known with positive entries, i.e. the parameter uncertainty is value bounded. It is worthy while noticing that this representation is similar to the second one. Mehdi et al. (1996) presented a linear quadratic design for uncertain systems in state space representation, the parameter uncertainty was value bounded. But interval matrix N[P, Q] ={ A= (aij) n%m }| pij $aij$qij , i=1, & ,n; j=1, &,m; P=( pij ) n%m, Q=( qij ) n%m } is more general representation than the value bounded representation. In this paper, we discuss the robust stability of the interval Lur'e systems whose forward linear system has structured uncertainties and whose feedback path is a sector-bounded memoryless time-varying nonlinearity. The BLMI approach allows us to derive a simple sufficient condition for the interval Lur'e system to be globally exponentially stable with a damping factor. The relationship between the stability of symmetric interval matrix and globally exponentially stability of interval Lur'e system is established.
2 Main Result We consider the interval Lur'e system whose linear forward path is a system (1) and whose feedback path is a memoryless time-varying nonlinearity ( as shown in figure 1. The linear forward path is a linear interval dynamical system
'* x$ (t ) = N [ A, A]x(t ) + N [ B, B]w(t ) ( *) z (t ) = Cx (t ) + Dw(t )
(2.1)
where x(t)' Rn is the state, ω(t) ∈Rm is the input signal and z(t) ∈Rm is the output signal. N [ A, A] is n×n order interval matrix; N [ B, B] is n×m order interval matrix. +
ω(t)
Linear system (2.1)
Ψ(t, z(t)) Nonlinear feedback
Fig. 1 Lur’e system
z(t)
367
Robust Stability of Interval Lur’e Systems
367
The feedback nonlinearity is described by ω(t) = -Ψ(t, z(t)), Ψ: [0,+∞)×R m → R m
(2.2)
We give definitions of the sector condition for the nonlinearity. Definition 1 A memoryless nonlinearity Ψ: :[0,+∞)×R m → R m is said to belong to a sector [0,+∞) if
ZTΨ(t, z) ≥ 0, ∀ t ≥ 0, ∀ z ∈ R m
(2.3)
In this section we derive exponential stability conditions of the interval Lur'e system (1) for the feedback nonlinearity Ψ belonging to a sector [0,+∞). Let A 0 =
A+ A 2
,H =
A− A 2
= (hij ) n×n , B0 =
B+ B 2
,H =
B−B 2
= (h ij ) n×m , E1 =
[ h11 e1 , # , h 1n e1 , # hn1 e n , # hnn e n ]; E 2 = [ h11 e1 , # , h1m e1 , # , h n1 e n , # , h nm e n ]; F1 = [ h11 e1 , # , h1n e n , # , hn1 e1 , # , hnn e n ]T ; F2 = [ h 11 e1 , # , h1m e m , # , h n1 e1 , # , h nm e m ]T . Where E1 , E 2 , F1 , e1 = (1,0, # ,0) T , # , ei = (0, # ,0,1,0, # ,0) T , # , e n = (0, # ,0,1) T ∈ R n and F2 , ei ∈ R m . Further more, we define
#
# ∈R
*
={
nm×nm
# =diag[ε
11 , # , ε 1m , # , ε n1 ,
# , ε nm ], ε ij ≤ 1, i = 1,2, # , n, j = 1,2, # , m}, then N [ A, A] = A0 + E1
B] = B0 + E 2
#F .
# F , N [ B, 1
2
The proof is similar to those of Wu’s et al. paper(1999). System (1) can be described by
#
'* x$ (t ) = ( A0 + E1 F1 ) x(t ) + ( B0 + E 2 ( *) z (t ) = cx(t ) + Dω (t )
# F )ω (t ) 2
(2.4)
Theorem 1 The interval Lur’e system consisting of (2.1) and the feedback (2.2) belonging to a sector [0, +∞) is globally exponentially stable with a damping factor ε if there exists a symmetric interval matrix ∆ $ T11 T12 , - ≥ 0, where T11 = − PN [ A, A] − N T [ A, A]P − 2εP; T12 = C T − =% T 1 %T & 12 D + D .
#
PN [ B, B ]; T21 = C − N T [ B, B ]P
368
J.T. Sun
368
Define a Lyapunov function V = xTPx ≥ 0, P > 0, then $x , % - + 2ω T z − 2εx T Px V$ ( x) = −[ x T , ω T ] 1 %ω & . $ where V (x) is evaluated along the trajectories of the system (1), if Σ1 > 0, ∀t ≥ 0 and Ψ(t, z) belongs to the sector [0, +∞) (i.e. ωTz ≤ 0), then we have Proof
#
V$ ≤ −2εV
(2.5)
Integrating (2.5)from 0 to t yields t
+
t
V$ ( x(t ))dt ≤ − 2ε V ( x(t ))dt then V ( x(t )) ≤ V ( x(0)) exp(−2εt ). From V ( x) ≥
+
0
0
2
ρ min ( P) x , we obtain x(t ) ≤
(
V ( x ( 0 )) ρ min ( P )
)
1 2
exp(−εt ).
This inequality indicates that the interval Lur’e system (2.1) is globally exponentially stable with a damping factor ε. The proof of this theorem is completed. Remark 1 A relationship between the stability of symmetric interval matrix #1 and the globally exponentially stability of interval Lur'e system is established in Theorem 1. Remark 2 The symmetric interval matrix L[ A, A] = { A = (a ij ) n×n a ij ≤ a ij ≤ a ij ,
a ij = a ji , i, j = 1,2, # , n, A = (a ij ) n×n , A = (a ij ) n×n } is stable if and only if LV [ A, A] = { A = (a ij ) n×n a ij = a ij or a ij , a ij = a ji , i, j = 1, # , n} is stable. The proof is similar to those of Shi's paper (1986). 'a ij i= j * is stable, then Remark 3 If M [ A, A] = (mij ) n×n , mij = ( *)max a ij , a ij , i ≠ j
(
)
symmetric interval matrix L[ A, A] is stable. Theorem 2 The interval Lur'e system consisting of (2.1) and the feedback (2.2) belonging to a sector [0, +∞) is globally exponentially stable with a damping factor ε if there exists a symmetric positive-definite matrix P > 0 such that
$ H 11 % %C − BT P 0 &
C T − PB0 T
D +D+
F2T
, -≥0 F2 -.
where H 11 = − PA0 − A0T P − 2εP + P ( E1 E1T + E 2 E 2T ) P + F1T F1 .
(2.6)
369
Robust Stability of Interval Lur’e Systems
369
Proof Consider the equivalent system (2.4), Note $ − PA0 − A0T P − 2εP Y0 = % % C − B0T & $ξ1 , %% -- = 2(ξ 1T PE1 )( &ξ 2 .
C T − PB0 ,. Since ξ 1T ξ 2T T D +D .
(
# F ξ ) + 2(ξ 1 1
T 1
(
PE 2 )(
#F ξ
2 2)
)$ H
ξ 1T PE 2 E 2T Pξ 1 + ξ 2T F2T F2ξ 2 = ξ 1T ξ 2T %% &0 + F1T
#E
T 1
0
11
F2T
$
)%%% &
L11 F2T
#E
PE 2 T 2
P
#F
2
0
, -.
≤ ξ 1T PE1 E1T Pξ 1 + ξ 1T F1T F1ξ 1
,$ ξ 1 , -% -, whereL11 = PE1 F2 -.%& ξ 2 -.
#F
1
P; H 11 = P( E1 E1T + E 2 E 2T ) P + F1T F1 ,
0 , $ H 11 ->0 Y0 + % % 0 FT F 2 2. & by Lyapunov function V(x)=xTPx and similar procedure of proof of Theorem 1, we can obtain the interval Lur's system (2.1) is globally exponentially stable with a damping factor ε.The proof of this theorem is completed. hence
Lemma 1 If A ∈ N [ A, A], (1 − λ )( A − A) ≤ ∆A = A − [λ A + (1 − λ ) A] ≤ λ A − A) =
λT , λ ∈ [0,1], then for any positive number ε , P∆A + ∆A T P ≤ εP T P + nε −1 [max(λ , 1 − λ1 )] 2 diag[( A − A) T ( A − A)] Proof Since x T {εP T P + nε −1 [max(λ , 1 − λ )] 2 diag (T T T )}x = εx T P T Px + n[max( λ ,1− λ )]2
ε
# x ∆a j
#x t
j jk t jk
x j ≥ εx T P T Px +
j ,k
jk ∆a jk
x j ] = εx T P T Px +
i, j ,k
≥ εx T P T Px + ε1 T
# x ∆a i
i, j ,k
ik ∆a jk
1 2ε
[
[max( λ ,1− λ )]2 2ε
# x ∆a i
ik ∆a ik x i
+
j ,i , k
[
#x t
i ik t ik x i
j ,i , k
+
#x t
j jk t jk
xj]
i , j ,k
x j = εx T P T Px + ε1 x T ∆A T ∆Ax ≥ 2 x T ∆AT Px
T
= x [ P ∆A + ∆A P ] x then, Lemma is holds. Theorem 3 The interval Lur'e system consisting of (2.1) and the feedback (2.2) belonging to a sector [0, +∞) is globally exponentially stable with a damping factor ε if there exists a symmetric positive-definite matrix P > 0 and scalar λi ∈ [0, 1] (i = 1,2), λ > 0, such that
370
370
J.T. Sun
$ T11 % %C − BT 1 &
C T − PB1 ,≥0 T22 -.
(2.7)
where T11 = − PA1 − A1T P − 2εP + (λ + 1) PP + nλ −1 [max(λ1 , 1 − λ1 )] 2 diag[( A − A) T ( A − A)], T22 = D T + D + m[max(λ 2 , 1 − λ 2 )] 2 diag[( B − B)]T ( B − B )]. Proof Note $ − PA1 − A1T P − 2εP C T − PB1 , -, where A = λ A + (1 − λ ) A, B = λ B + Y1 = % 1 1 1 1 2 T % C − B1 P D − D -. &
(1 − λ 2 ) B. $ K 11 Since x T ω T %% T & ∆B P
(
)
P∆B ,$ x , -% - = x T ( P∆A + ∆A T P) x + 2ω T ∆B T Px ≤ % 0 -.& ω .
x T {λP T P + nλ −1 [max(λ1 , 1 − λ1 )] 2 diag[( A − A) T ( A − A)]}x + ω T ∆B T ∆Bω + x T PPx ≤ x T {(λ + 1) P T P + nλ −1 [max(λ1 , 1 − λ1 )] 2 diag[( A − A)]T ( A − A)]}x −
(
m[max(λ 2 , 1 − λ 2 )] 2 diag[( B − B ) T ( B − B)]ω T ω = x T
)$ L0
ω T %% &
11
0 ,$ x , -% -, L 22 -.%& ω -.
where K 11 = P∆A + ∆A P; L11 = (λ + 1) P P + nλ [max(λ1 ,1 − λ1 )] 2 diag[( A − T
T
−1
A) T ( A − A)], L 22 = m[max(λ 2 , 1 − λ 2 )] 2 diag[( B − B) T ( B − B ) hence $ P∆A + ∆AT P P∆B ,≥ 0, by similar procedure of proof of Theorem 1, we Y1 + % % 0 -. ∆B T P & can obtain that the interval Lur'e system (2.1) is globally exponentially stable with a damping factor ε. The proof of the theorem is completed. Remark 4 (2.6) and (2.7) are bilinear matrix inequality (BLMI) of P for BLMI, Goh. et al (1994) presented properties and computational methods.
3 Numerical example We consider an uncertain system (2.1) with $ − 0.06 0.97 , $1.1, $ 0 .9 , $ 0.06 1.03 , --, A = %% --, B = %% --, B = %% --, C = (1 1), D = 0.98 A = %% 0 . 9 − − 1 . 97 2 . 04 − − 2 . 03 3 . 06 & . &1.1. & . & . the nonlinear feedback is set as Ψ(t, z(t)) = z(t){1−sin[z(t)]} this nonlinear function belongs to the sector [0, +∞).
(3.8)
Robust Stability of Interval Lur’e Systems
371 371
If we set ε =0.5 and λ =1, λ1=λ2 =0.5 there exists a positive definite matrix !
$4 1 , -- which satisfies the condition in Theorem 2, hence the interval Lur'e P = %% & 1 0 .5 . system consisting of (2.1) and the feedback (3.8) belonging to the sector [0.+∞) is globally exponentically stable with a damping factor 0.5.
References [1]. Vidyasagar, M., (1993) Nonlinear Systems Analysis, Prentice-Hall, London [2]. Grujic LJT, Petkovski DJ (1987) On robustness of Lur'e systems with multiple nonlinearities. Automatica 23: 327-334 [3]. Tesk A, Vicino A (1991) Robust absolute stability of Lur'e control systems in parameter space. Automatica 27: 147-151 [4]. Dahleh M, Tesk A, Vicino A (1993) On the robust Popov criterion for interval Lur'e system. IEEE Tansactions on Automatic Control 38: 1400-1405 [5]. Mori T, Nishimura T, Kuroe Y, Kokame H (1995) Comments on `On the robust Popov criterion for interval Lur'e system. IEEE Tansactions on Automatic Control 40: 136-137 [6]. Wada T, Ikeda M, Ohta Y, Siljak DD (1996) Parametric absolute stability of multivariable Lur'e systems: an LMI condition and application to polytopic systems, Proceedings of the 13th IFAC World Congress, San Francisco, California, USA, pp19-24 [7]. Wada T, Ikeda M, Ohta Y, Siljak DD (1998) Parametric absolute stability of Lur'e systems. IEEE Tansactions on Automatic Control 43: 1649-1653 [8]. Boyd S, Ghaoui LEL, Feron E, Balakrishnan V (1994) Linear Matrix Inequalities in System and Control Theory, Philadelphia, Pennsylvania SIAM [9]. Konishi K, Kokame H (1996) Robust stability of Lur'e systems with time-varying uncertainties: a linear matrix inequality approach. Int.J. of systems Science 30: 3-9 [10]. Gu, K, Zohdy M.A, Loh NK (1990) Necessary and sufficient condition of quadratic stability of uncertain linear system. IEEE Tansactions on Automatic Control 35: 601-604 [11]. Goh KC, Turan L, Safonov MG (1994) Bilinear matrix inequality properties and computational methods. Proc. of Conf. on ACC, Baltimore, pp. 850-855 [12]. Goh KC, Safonov M G, Papavassilopoulos GP (1994) A global optimization approach for the BMI problem .Proc. of Conf. Decision and Control, Tokoy, pp. 2009- 2014 [13]. Shi Z, Gao W (1986)A necessary and sufficient condition for the positive- definiteness of interval symmetric matrices Int. J. of Control 43:325-328 [14]. Wu FX, Shi ZK, Dai GZ (1999) H∞ robust control for interval systems. Acta Automatica Sinica 25: 705-708 [15]. Sun JT, Deng FQ ,Liu YQ (2001) Robust absolute stability of general interval Lur'e type nonlinear control systems.J. of Syst. Engineering and Electronics 12:46-52
372
Thresholding Segmentation and Classification in Automated Ultrasonic Testing Image of Electrical Contact H. D. Chen, Z. J. Cao, Y. W. Wang, and J. Xue Welding Research Institute, School of Materials Sci. & Eng., Xi’an Jiaotong Univ., Xi’an 710049, P.R. China [email protected]
Abstract. This paper is concerned with the thresholding segmentation of ultrasonic C-scan image for defect with artificial intelligence and support vector machines techniques. Wavelet based method is used to remove the noise of image. Segmentation techniques based on thresholding were performed in order to segment the defect in ultrasonic testing image. Experimental results show that this segmentation technique performs well in finding defects in C-scan image of electrical contact.
1 Introduction The increasing use of electrical contact requires that reliable non-destructive evaluation (NDE) methods be available for the detection of defects and damage, or the assessment of quality of these materials. Ultrasonic C-scan is a method commonly used in NDE, which is based on propagation of ultrasonic waves through the material. The waves penetrate the material, bounce at the bottom (or at the defect if exist) and return to the transducer. The time at which the first echo arrives determines the position of the bottom of joint or of the defect. The signal is then digitized along the plate and is visualized in a manner of a gray level image. The main difficulty is then to extract the defects from the background in C-scan images [1] . The images obtained are far from being ideal. Not only do these images contain sensor noise, which is typically present in any kind of image acquisition, but they are also corrupted by non-uniform illumination. That is due to the image acquisition apparatus. For example, the specimen might not be perfectly horizontal when ultrasonic transducer scans it. Wavelet transform is of the ability called “digital microscope”. When ultrasonic testing images are gotten, the original image attached with noise can be decomposed by wavelet transform. There is a difference between the noise frequency spectrum and the frequency spectrum of defect reflection signal, little T.-J. Tarn et al. (Eds.): Robotic Welding, Intelligence and Automation, LNCIS 299, pp. 372−379, 2004. Springer-Verlag Berlin Heidelberg 2004
Thresholding Segmentation and Classification
373 373
useful information is with the low signal-to-noise ratio’s image, which can be eliminated while image reconstruction. There is a large amount of useful information in the sub-band image with high signal-to-noise ratio; the defect image can be obtained after the appropriate defect inspection algorithm. The purpose of image segmentation is to separate the defect image area from the whole image, which is important for subsequent process. Therefore, research on ultrasonic image segmentation technique is of significance for defect detection of automated ultrasonic imaging system in weld. With the rapid development of electronic technology and computer technology, further requirement of joint’s ultrasonic testing is desired. It need not only get the ratio of braze area (RBA), but also get the defect size defect location and defect type in order to carry out quantitative and qualitative analysis. Thus research in ultrasonic testing imaging segmentation method is essential to defect’s identification in weld [2]. In spite of all modern automated inspection methods, defect classification is still a difficult task. Much work, essentially based on pattern recognition techniques and artificial intelligence methods using neural networks, has been published on this topic. The paper is organized as follows. In section two image techniques are talked about and the methods that have been applied to this type of images, next a system overview of the method that implemented is presented and the way that works is introduced. Some theoretical explanations about the methods and techniques are given. Then, segmentation results produced by the proposed method are shown. In section six we conclude that this segmentation technique performs well in finding defects in C-scan image of electrical contact. !
2 Image Processing Technique The image techniques utilized for C-scan image segmentation could be divided into the following groups: threshold-based segmentation [3,4], statistical methods for image segmentation and region growing methods. In the category of threshold-based segmentation was proposed the use of: iterative thresholding, histogram analysis and morphological operations. Some of these techniques are followed by some refinement of the segmented zones. Statistical methods represent another important category in the segmentation process and most of the approaches proposed in this category are using some statistical classifications combined with different image processing techniques in order to segment the C-scan images. Also, the use of statistical methods in texture analysis has been proposed. The region growing techniques applied to C-scan images represents the final category. All these techniques have its advantages and drawbacks, that’s why even more research is demanded in this domain and maybe some combination of the already existing methods or some new image processing techniques will lead to better results. The segmentation problems that are widely encountered by the researchers are the noise introduced with the acquisition of the image, the overlapping
374374H.D. Chen et al.
intensities (different defect structures have different characteristics which results in various signal intensities and these intensities could overlap), and the partial volume effect.
3 Image Segmentation System Now, the system is introduced and all the theoretical implications with this system will be explained in the next section. Fig. 1. shows the system used in order to segment and detect the defect in C-scan images.
Fig. 1. System diagram
The C-scan image acquisition module is the first step in the image segmentation system. In this step it is very likely that noise will be imported, thus the quality of the image could be decreased. That’s why the next step is dealing with the enhancement of the image in order to improve the quality or to perform a de-noising operation. The de-noising method is a wavelet-based method that will be described in the theoretical section. After the de-noising has been performed the image is entering into the classifier module that will distinguish, according to statistical parameters that describe the image. In this step the classifier, which is a support vector machine should also be able to distinguish between defect and background. The last step is the segmentation module of the image to detect the defect. This step uses some prior knowledge from the classification step in order to use an adaptive histogram threshold for a better defect segmentation. After a C-scan image has passed through the whole system, the result is the segmentation of the existing defect.
4 Theoretical Backgrounds 4.1 Image De-noising
[5]
An important part of any image processing system is the pre-processing phase. This phase could imply contrast enhancement techniques or methods for removing the noise. The method that used to perform this task is a wavelet-based one. Lately, wavelets have become a popular tool in many research domains, image-denoising being only one of them. First, a wavelet transform is applied to the image, and then the wavelet coefficients obtained are manipulated in order that the noisy points to be
Thresholding Segmentation and Classification
375 375
eliminated and then inverse wavelet transform are performed in order to reconstruct the image. The manipulation of the wavelet coefficients in a procedure of the former type is based on a classification. This classification is often binary: the coefficients are divided into two groups. The first group contains important, regular coefficients, while the other group consists of coefficients that are catalogued as “too noisy”. These two groups are then processed in an appropriate way. Zero often replaces noisy coefficients. In order to perform this classification a decision is needed in order to know what’s the threshold that distinguishes between these two classes. 4.2 Classification
4.2.1 Statistical parameters A region having a particular defect has a wide variety in its gray level. The question is how we can capture this defect in order to distinguish between different defects. In this case, the original C-scan image is split in small windows and each one of these is characterized using statistical calculus. This area, also called window, can also be varied in its size to capture a sample of different scales to be found here. Note that the precision with which the boundary between regions is known is still a function of the window used. This is always true, and so there is an advantage in using methods that allow the use of small windows. The mean gray level and the standard deviation are known as statistical moments. The mean is related to the first moment, and the standard deviation depends on the second moment. There are others, and these could also be used to characterize defect areas. In general, we have: (1) where N is the number of data points, and n is the order of the moment. Now the skewness can be defined as:
(2)
and the kurtosis as:
(3)
376376H.D. Chen et al.
4.2.2 Support Vector Machines Support Vector Machines (SVMs) are a new technique that introduced in 1995 by Vapnik [6,7]. In terms of theory the SVM are well founded and proved to be very efficient in classification tasks. The advantages of such classifiers are that they are independent of the dimensionality of the feature space and that the results obtained are very accurate, although the training time is very high. Support Vector Machines are feedforward networks with a single layer of nonlinear units. Their design likes the generalized radial basis function (GRBF) networks with good generalization performance as an objective and follows for that reason the principle of structural risk minimization that is rooted in VC dimension theory. The solution for a typical two-dimensional case to has the form shown in Fig. 2. Those training points for which the equality in of the separating plan is satisfied ( i.e.( yt ( xt ⋅ w + b) − 1 ≥ 0 ∀i ) , those which wind up lying on one of the hyper-planes
H1 , H 2 ), and whose removal would change the solution found, are
called Support Vectors (SVs) and are indicated in Fig. 2 by the extra circles.
Fig. 2. Linear separating hyper-planes for the separable case, the support vectors are circled
4.3 Thresholding and Morphological Operations According to the histogram of the C-scan image, the threshold to be used for segmentation can be determined. For a pixel to remain in the image after thresholding, its intensity value must be greater than the threshold value. Thresholding is the most common segmentation technique and has received great interest in the literature.
Thresholding Segmentation and Classification
377 377
Another image processing technique used is opening which is a morphological operation. The morphological operations are well known techniques for the segmentation and enhancement of images and also for recognition purposes. Opening is used to eliminate small details and smoothes the image boundaries. 4.4 Segmentation The purpose of ultrasonic image segmentation is to separate the defect area from the background. If the amplitude of reflection wave signal is large, then the intensity of gray scale value of pixel in the image is large. Because the gray scale value of defect area is different from that of the well-welded area, image segmentation can be utilized to segment the defect image. The gray scale value range of ultrasonic image is 0 255 suppose the gray scale value of a pixel in original image be T ( x, y ) after thresholding segmentation it becomes Tt ( x, y ) then !
"
#
"
"
"
#1 T ( x, y ) ≥ S Tt ( x, y ) = $ %0 T ( x, y ) < S where S , 0 < S < 255 , stands for thresholding. !
(4)
#
The image after thresholding process is a binary image. If the thresholding selected is higher than the actual one, some defect area will be taken for welded area and the calculated welded area would be large then the actual one, vice versa, it would be less than the actual one. Therefore the key of image segmentation is the thresholding selection. The image segmentation techniques used in image processing can be divided into the following groups: threshold-based segmentation, statistical methods for image segmentation and region growing methods. Threshold-based segmentation is widely used in image processing fields. In ultrasonic imaging testing, threshold-based segmentation method is used to segment the defect area in ultrasonic image. There are two ways to choose the threshold: iterative thresholding that is independent of the image or image histogram-based threshold. In order to extract defect object that interested in from the image we have to choose a threshold in such a way that it separates those objects from the rest. One way to choose the threshold is using the histogram of the image. In this paper we performed experiments with histogram-based thresholds. According to the histogram of the C-scan image, the threshold to be used for segmentation can be determined.
5 Experimental Results The training of the Support Vector Machine network was done with 40 images, 20 being of artificial flaw workpiece and 20 of actual workpiece. The result of the classification is around 70%, but it is very clear that they could be improved by training the neural network with more images. The idea of using a neural network
378378H.D. Chen et al.
was to build it and test it in order to use it in the future for a classification of the images. The first step was represented by the pre-processing phase of the images in order to remove the noise that could be introduced to image acquisition. In the figures below an example of image de-noising is shown. The next step after the classification is represented by the segmentation of the C-scan image. The segmentation is a threshold-based, but the threshold was applied after an opening transform. The opening transform was used to enhance the defect and then the threshold was applied. Fig.4 is the results after the segmentation.
(a) Image before de-noising (b) image after wavelet de-noising Fig. 3. Experimental results of switch contact C-scan image
Fig. 4. Segmentation result of the C-scan image
6 Conclusions An image segmentation approach based on wavelet de-noising and SVMs feature extraction and classification has been developed to detect flaws automatically in NDE image. The system implemented is used for detecting the defect in ultrasonic C-scan images. The system involves two major modules, one that performs classification and the other one that segment the defects from the images. Classification module can be used either before the segmentation process or after, depending on what the system is used for. In the segmentation phase the system performed well and detected the defect in the images. But I had no extra information that could be very useful in detecting some flaw patterns. Overall, the system performed well, but there is still a lot to improve for the completeness of knowledge base.
Thresholding Segmentation and Classification
379 379
7 Future Works The work will be continued about this system, so in the following lines I will present some future research that to do. First, hopefully, I will have more artificial defect images and actual ones. When the acquisition of these images will be done, I want to use feature extracted from images in order to get an accurate classification of the images. In my opinion, the classification would be better performed after the segmentation method and the segmentation results to be used in classification.
References 1. 2. 3. 4.
D. E. Bray and R. K. Stanley, Nondestructive evaluation, a tool for design, manufacturing and service. New York: McGraw-Hill, 1989. D. T. Brown and G. Green, “Development of an ultrasonic system for submarine hull acoustic cladding inspection,” Ndt&E Int., vol. 30, no.3, pp. 123-133, 1997. P. Sahoo S. Soltani and A. Wong, “A survey of thresholding techniques,” Comput. Vision, Graph., vol. 41, no.3, pp. 233-260, 1988. J. Kapur, P. Sahoo and A. Wong, A new method for gray-level picture thresholding using the entropy of the histogram. Comput. Vision Graph. vol. 29, no.2, pp. 273-285, 1985. L. D. Wu, Computer Vision. Shanghai: Fudan University Press, 1993. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer Verlag, 1995. V. N. Vapnik, “An Overview of statistical Learning Theory,” IEEE Trans. Neural Networks, vol. 10,no.5, pp.998-1000, 1999. !
!
"
!
5. 6. 7.
Author Index
M. Barrett, 281 A. Boeing, 228 T. Bräunl, 228
Y. J. Lou, 25 H. Luo, 110 W. X. Lv, 123
H. Cai, 276 Z. J. Cao, 372 H. Q. Chen, 372 M. Chen, 144 Q. Chen, 152 S. B. Chen, 25, 56, 123, 345, 353 W. Chen, 353 X. Chen, 110
D. Maravall, 190
B. Da, 144 X. Dai, 276 D. Ding, 80 X. Ding, 144 D. Du, 152 S. Emil, 1 M. F. Ercan, 212 G. Fang, 263 M. Ge, 63 X. Gu, 358 Y. F. He, 152 F.H. Ho, 304 J. Hoffmann, 14 D. Hong, 281 Y. L. Ip, 161 U. Jasnau, 14 M. L. Lam, 80 T. Lin, 123, 353 Y. H. Liu, 80 W. L. Lo, 304
T. Qiu, 123 A. B. Rad, 161, 304 P. Seyffarth, 14 B. Sui, 152 D. Sun, 358 J. Sun, 365 L. Sun, 276 Z. Sun, 241 Z. Tang, 241 J. S. Tian, 123 B. Wang, 345 Y. W. Wang, 372 Y. K. Wong, 161, 304 L. Wu, 25, 56, 123 P. Xiao, 281 Y. Xu, 63 Z. Xu, 263 J. Xue, 372 Y. Yang, 327 F. Ye, 358 J. Q. Yi, 56 H. Zhang, 144, 152 Y. Zhang, 123 D. B. Zhao, 25, 56 C. Zhou, 212, 241, 327 C. Zou, 144
