1969_half 9/16/05 11:11 AM Page 1
Fusion of Optical and Mechatronic Engineering
Mechanical Engineering Series Frank ...
220 downloads
2649 Views
15MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
1969_half 9/16/05 11:11 AM Page 1
Fusion of Optical and Mechatronic Engineering
Mechanical Engineering Series Frank Kreith & Roop Mahajan - Series Editors Published Titles Distributed Generation: The Power Paradigm for the New Millennium Anne-Marie Borbely & Jan F. Kreider Elastoplasticity Theor y Vlado A. Lubarda Energy Audit of Building Systems: An Engineering Approach Moncef Krarti Engineering Experimentation Euan Somerscales Entropy Generation Minimization Adrian Bejan Finite Element Method Using MATLAB, 2nd Edition Young W. Kwon & Hyochoong Bang Fluid Power Circuits and Controls: Fundamentals and Applications John S. Cundiff Fundamentals of Environmental Discharge Modeling Lorin R. Davis Heat Transfer in Single and Multiphase Systems Greg F. Naterer Introductor y Finite Element Method Chandrakant S. Desai & Tribikram Kundu Intelligent Transportation Systems: New Principles and Architectures Sumit Ghosh & Tony Lee Mathematical & Physical Modeling of Materials Processing Operations Olusegun Johnson Ilegbusi, Manabu Iguchi & Walter E. Wahnsiedler Mechanics of Composite Materials Autar K. Kaw Mechanics of Fatigue Vladimir V. Bolotin Mechanics of Solids and Shells: Theories and Approximation Gerald Wempner & Demosthenes Talaslidis Mechanism Design: Enumeration of Kinematic Structures According to Function Lung-Wen Tsai Multiphase Flow Handbook Clayton T. Crowe Nonlinear Analysis of Structures M. Sathyamoorthy Optomechatronics: Fusion of Optical and Mechatronic Engineering Hyungsuck Cho Practical Inverse Analysis in Engineering David M. Trujillo & Henry R. Busby Pressure Vessels: Design and Practice Somnath Chattopadhyay Principles of Solid Mechanics Rowland Richards, Jr. Thermodynamics for Engineers Kau-Fui Wong Vibration and Shock Handbook Clarence W. de Silva Viscoelastic Solids Roderic S. Lakes
1969_title 9/16/05 11:11 AM Page 1
Fusion of Optical and Mechatronic Engineering
Hyungsuck Cho
Boca Raton London New York
A CRC title, part of the Taylor & Francis imprint, a member of the Taylor & Francis Group, the academic division of T&F Informa plc.
1969_Discl.fm Page 1 Monday, September 26, 2005 11:04 AM
Published in 2006 by CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2006 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-10: 0-8493-1969-2 (Hardcover) International Standard Book Number-13: 978-0-8493-1969-3 (Hardcover) Library of Congress Card Number 2005050570 This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data Cho, Hyungsuck Optomechatronics / by Hyungsuck Cho p. cm. Includes bibliographic references and index. ISBN 0-8493-1969-2 (alk. paper) 1. Mechatronics. 2. Optical detectors. TJ163.12.C44 2005 670.42'7--dc22
2005050570
Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com Taylor & Francis Group is the Academic Division of Informa plc.
and the CRC Press Web site at http://www.crcpress.com
Author
Hyungsuck Cho gained his B.S. degree at Seoul National University, Korea in 1971, an M.S. degree at Northwestern University, Illinois in 1973, and a Ph.D. at the University of California at Berkeley, California in 1977. Following a term as Postdoctoral Fellow in the Department of Mechanical Engineering, University of California, Berkeley, he has joined the Korea Advanced Institute of Science and Technology (KAIST) in 1978. He was made a Humboldt Fellow in 1984-1985, won Best Paper Award at the International Symposium on Robotics and Manufacturing, USA in 1994, and the Thatcher Brothers Awards, Institute of Mechanical Engineers, UK in 1998. Since 1993, he has been an associate editor or served on the editorial boards of several international journals, including IEEE Transactions on Industrial Electronics, and has been guest editor for three issues, including IEEE Transactions IE Optomechatronics in 2005. Dr. Cho wrote the handbook Optomechatronic Systems: Technique and Application, has contributed chapters to 10 other books and has published 435 technical papers, primarily in international journals. He was the founding general chair for four international conferences and the general chair or co-chair for 10 others, including the SPIE Optomechatronic Systems Conference held in Boston in 2000 and 2001. His research interests are focused on optomechatronics, environment perception and recognition for mobile robots, optical vision-based perception, control, and recognition, and application of artificial intelligence/ machine intelligence. He has supervised 136 M.S. theses and 50 Ph.D. theses. For the achievements in his research work, he was made POSCO professor from 1995 to 2002.
Preface In recent years, optical technology has been increasingly incorporated into mechatronic technology, and vice versa. The consequence of the technology marriage has led to the evolution of most engineered products, machines, and systems towards high precision, downsizing, multifunctionalities and multicomponents embedded characteristics. This integrated engineering field is termed optomechatronic technology. The technology is the synergistic combination of optical, mechanical, electronic, and computer engineering, and therefore is multidisciplinary in nature, thus requiring the need to view this from somewhat different aspects and through an integrated approach. However, not much systematic effort for nurturing students and engineers has been made in the past by stressing the importance of the multitechnology integration. The goal of this book is for it to enable the reader to learn how the multiple technologies can be integrated to create new and added value and function for the engineering systems under consideration. To facilitate this objective, the material brings together the fundamentals and underlying concepts of this optomechatronic field into one text. The book therefore presents the basic elements of the engineering fields ingredient to optomechatronics, while putting emphasis on the integrated approach. It has several distinct features as a text which make it differ somewhat from most textbooks or monographs in that it attempts to provide the background, definition, and characteristics of optomechatronics as a newly defined, important field of engineering, an integrated view of various disciplines, view of systemoriented approach, and a combined view of macro– micro worlds, the combination of which links to the creative design and manufacture of a wide range of engineering products and systems. To this end a variety of practical system examples adopting optomechatronic principles are illustrated and analyzed with a view to identifying the nature of optomechatronic technology. The subject matter is therefore wide ranging and includes optics, machine vision, fundamental of mechatronics, feedback control, and some application aspects of micro-opto-electromechanical system (MOEMs). With the review of these fundamentals, the book shows how the elements of optical, mechanical, electronic, and microprocessors can be effectively put together to create the fundamental functionalities essential for the realization of optomechatronic technology. Emphasizing the interface between the relevant disciplines involving the integration, it derives a number of basic optomechatronic units. The book
then goes on in the final part to deal, from the integrated perspectives, with the details of practical optomechatronic systems composed of and operated by such basic components. The introduction presents some of the motivations and history of the optomechatronic technology by reviewing the technological evolution of optoelectronics and mechatronics. It then describes the definition and fundamental concept of the technology that are derivable from the nature of practical optomechatronic systems. Chapter 2 reviews the fundamentals of optics in some detail. It covers geometric optics and wave optics to provide the basis for the fusion of optics and mechatronics. Chapter 3 treats the overview of machine vision covering fundamentals of image acquisition, image processing, edge detection, and camera calibration. This technology domain is instrumental to generation of optomechatronic technology. Chapter 4 presents basic mechatronic elements such as sensor, signal conditioning, actuators and the fundamental concepts of feedback control. This chapter along with Chapter 2 outline the essential parts that make optomechatronics possible. Chapter 5 provides basic considerations for the integration of optical, mechanical, and electrical signals, and the concept of basic functional modules that can create optomechatronic integration and the interface for such integration. In Chapter 6, basic optomechatronic functional units that can be generated by integration are treated in detail. The units are very important to the design of optomechatronic devices and systems, since these produce a variety of functionalities such as actuation, sensing, autofocusing, acousticoptic modulation, scanning and switching visual feedback control. Chapter 7 represents a variety of practical systems of optomechatronic nature that obey the fundamental concept of the optomechatronic integration. Among them are laser printers, atomic force microscopes (AFM), optical storage disks, confocal microscopes, digital micromirror devices (DMD) and visual tracking systems. The main intended audiences of this book are the lower levels of graduate students, academic and industrial researchers. In the case of undergraduate students, it is recommended for the upper level since it covers a variety of disciplines, which, though fundamental, involve various different physical phenomena. On a professional level, this material will be of interest to engineering graduates and research/field engineers who function in interdisciplinary work environments in the fields of design and manufacturing of products, devices, and systems. Hyungsuck Cho
Acknowledgments I wish to express my sincere appreciation to all who have contributed to the development of this book. The assistance and patience of Acquiring Editor Cindy Renee Carelli, have been greatly appreciated during the writing phase. Her enthusiasm and encouragement have provided me with a great stimulus in the course of this book writing. In addition, I would like to thank Jessica Vakili, project coordinator, Fiona Woodman, project manager, and Tao Woolfe, project editor of Taylor and Francis Group, LLC, for ensuring that all manuscripts were ready for production. I am also indebted to my former Ph.D students, Drs. Won Sik Park, Min Young Kim and Young Jun Roh for their helpful discussions. Special thanks go to Hyun Ki Lee and all my laboratory students, Xiaodong Tao, Deok Hwa Hong, Kang Min Park, Dal Jae Lee and Xingyong Song who have provided valuable help in preparation of the relevant materials and proofreading the typed materials. Finally, I am grateful to my wife, Eun Sue Kim, and my children, Janette and Young Je, who have tolerated me with patience and love and helped make this book happen.
Contents 1. Introduction: Understanding of Optomechatronic Technology.............1 2. Fundamentals of Optics................................................................................31 3. Machine Vision: Visual Sensing and Image Processing .....................105 4. Mechatronic Elements for Optomechatronic Interface........................173 5. Optomechatronic Integration ....................................................................255 6. Basic Optomechatronic Functional Units ...............................................299 7. Optomechatronic Systems in Practice .....................................................447 Appendix A1
Some Considerations of Kinematics and Homogeneous Transformation......................................565
Appendix A2
Structural Beam Deflection............................................573
Appendix A3
Routh Stability Criterion ...............................................577
Index .....................................................................................................................581
1 Introduction: Understanding of Optomechatronic Technology CONTENTS Historical Background of Optomechatronic Technology ................................ 4 Optomechatronics: Definition and Fundamental Concept ............................. 8 Practical Optomechatronic Systems............................................................ 9 Basic Roles of Optical and Mechatronic Technologies .......................... 12 Basic Roles of Optical Technology..................................................... 13 Basic Roles of Mechatronic Elements................................................ 15 Characteristics of Optomechatronic Technology .................................... 16 Fundamental Functions of Optomechatronic Systems.................................. 20 Fundamental Functions ...................................................................................... 21 Illumination Control.................................................................................... 21 Sensing........................................................................................................... 24 Actuating ....................................................................................................... 24 Optical Scanning .......................................................................................... 24 Visual/Optical Information Feedback Control ....................................... 24 Data Storage.................................................................................................. 25 Data Transmission/Switching ................................................................... 25 Data Display ................................................................................................. 25 Optical Property Variation.......................................................................... 26 Sensory Feedback-Based Optical System Control .................................. 26 Optical Pattern Recognition ....................................................................... 26 Remote Operation via Optical Data Transmission................................. 27 Material Processing...................................................................................... 27 Summary ............................................................................................................... 27 References ............................................................................................................. 28 Most engineered devices, products, machines, processes, or systems have moving parts and require manipulation and control of their mechanical or dynamic constructions to achieve a desired performance. This involves the use of modern technologies such as mechanism, sensor, actuator, control, microprocessor, optics, software, communication, and so on. In the early 1
Optomechatronics
value (performance)
2
optical element
electrical/ electronics
electrical/ electronics
optical element
software
software
electrical/ electronics
electrical/ electronics
optical element
software
electrical/ electronics
mechanical mechanical mechanical mechanical mechanical mechanical element element element element element element
1800
1970
2000
year FIGURE 1.1 Key component technologies contributing to system evolution (not to scale).
days, these have been operated mostly via mechanical elements or devices which caused inaccuracy and inefficiency, thus resulting in difficulty in achieving a desired performance. Figure 1.1 and Figure 1.2 show how the key technologies such as mechanical, electrical, and optical have contributed to the evolution of machines/systems in terms of “value or performance” as years have passed [6]. As can be seen from the figure, tremendous efforts have been made to enhance system performance by combining electrical and electronic hardware with mechanical systems. A typical example is a gear-trained mechanical system controlled by a hardwired controller. This mechanical and electronic, called mechatronic configuration, consisted of two kinds of components: mechanism, and electronics and electric hardware. Because of 10k 10k
mechanism mechanical automation
− 411 +
analog control
μ-processor embedded M/C
optomechatronically embedded system
internet based (teleoperation)
mechatronic technology optomechatronics technology
FIGURE 1.2 Evolution of machines.
Introduction: Understanding of Optomechatronic Technology
3
the hard-wired structural limitation of this early mechatronic configuration, flexibility was not embedded in most systems in those days. This kind of tendency lasted until the mid 1970s when microprocessors came into use for industrial applications. The development of microprocessors has provided a new stimulant for industrial evolution. This brought about a big change, the replacement of many mechanical functions with electronic ones through the role of microprocessor. This evolutionary change has opened up the era of mechatronics, and has raised the autonomy level of machines and systems, at the same time increasing versatility and flexibility. The autonomy and flexibility achieved thus far, however, have a growth limited to a certain degree, since both the hardware and software of the developed mechatronic systems have not been developed so much as to have the capability of realizing many complicated functions autonomously while adapting to changing environments. In addition, information structure has not been developed to have real-time access to appropriate system data and information. There may be several reasons for this delay. The first one may be that in many cases mechatronic components alone may not achieve desirable function or performance as specified for a system design. The second one is that, although mechatronic components alone can work, the results achieved may not be as good as required because of their low perception and execution capability and also inadequate integration between hardware and software. In fact, in many cases measurements are difficult or not even feasible due to inherent characteristics of the systems. In some other cases, the measurement data obtained by conventional sensors are not accurate or reliable enough to be used for further processing. They can sometimes be noisy, necessitating some means of filtering or signal conditioning. The difficulties listed here may limit the enhancement of the functionality and performance of the mechatronic systems. This necessitates the integration of the mechatronic technology with other systems. In recent years, optical technology has been increasingly incorporated at an accelerated rate into mechatronic systems, and as a result, a great number of mechatronic products, machines and systems with smart optical components have been introduced into the market. There may be some compelling reasons for this changing trend. One reason is that the optical technology possesses unique characteristics such as noncontact/noninvasiveness visual perception, and insensitivity to electrical noise. As shown in Figure 1.1, the contribution of the optical technology is growing and enhances the system value and performance, since optical elements incorporating mechatronic elements embedded in the system provide some solutions to the difficult-to-solve technical problems. This emerging trend demonstrates that the optically integrated technology provides enhanced characteristics in such a way that it creates new functionalities that are not achievable with conventional technology alone, exhibits higher functionalities since it makes products and systems function on an entirely different principle or in a more efficient manner, and produces high precision
4
Optomechatronics
and reliability since it can facilitate or enable in-process monitoring and control of the system state. Besides, the technology makes it feasible to achieve dimensional downsizing and allows system compactness, since it has the capability of integrating sensors, actuators, and processing elements into one tiny unit.
Historical Background of Optomechatronic Technology The root of the development of the optically integrated mechatronic technology may be easily found when we revisit the historical background of the technological developments of mechatronics and optoelectronics. Figure 1.3 shows the development of mechatronic technology in the upper line above the arrow and that of optical engineering in the lower line [8]. The real electronic revolution came in the 1960s with the integration of transistor and other semiconductor devices into monolithic circuits following the invention of the transistor in 1948. Then the microprocessor was invented in 1971 with the aid of semiconductor fabrication technology and made a tremendous impact on a broad spectrum of technological fields. In particular, the development created a synergistic fusion of a variety hardware and software technologies by combining them with computer technology. The fusion made it possible for machines to transform analog signal into digital, to compute necessary calculations, to make decisions based upon the computed results and software algorithms, and then to take appropriate action according to the decision and to accumulate knowledge/data/information within their own memory domain. This new functionality has endowed machines and systems with characteristics such as flexibility and adaptability, and the importance of this concept has been recognized among industrial sectors, which accelerated ever wider applications. In the 1980s, the semiconductor technology also created micro-electro-mechanical systems (MEMSs), and this brought about a new dimension of machines and systems, micro-sizing their dimensions. Another technological revolution, so-called opto-electronic integration, has continued during the last 40-plus years ever since the laser was invented in 1960. This was made possible with the aid of advanced fabrication methods such as chemical vapor deposition, molecular-beam epitaxy, and focused-ion-beam micro-machining. These methods enabled integration of optical, electro-optic, and electronic components in a single compact device. The charge coupled device (CCD) image sensor developed in 1974 not only introduced computer vision technology but also opened up a new era of optical technology along with optical fiber sensors which appeared from 1976. The optical components and devices that were developed possessed a number of favorable characteristics, including: (1) noncontact/noninvasive; (2) easy to transduce; (3) having a wide sensing range; (4) insensitive to
FIGURE 1.3 History of optomechatronics. Source: Cho, H.S. and Kim, M.Y., IEEE Transaction on Industrial Electronics, 52:4, 932–943, 2005. q 2005 IEEE.
Introduction: Understanding of Optomechatronic Technology 5
6
Optomechatronics
electrical noises; (5) distributed sensing and communication; and (6) high bandwidth. Naturally, these favorable optical characteristics began to be integrated with those of the mechatronic elements and this integration helped achieve systems of higher performance. When a system or a machine is integrated in this way, namely optically, mechanically, and electronically, it is called an optomechatronic system. Looking back at the development of this system shown in Figure 1.3, we can bring to mind a number of practical examples. The lithography tool that fabricates ICs and other semiconductor devices belongs to this system category: it functions through a series of elaborate mirrors in addition to a light beam, optical units and a stepper servo mechanism that precisely shifts the wafer from site to site. Another representative system is the optical pick-up device mass-produced from 1982. The pickup system reads information off the spinning disc by controlling both the up-and-down and side-to-side tracking of a read head which carries a low-power diode laser beam focused onto the pits of the disc. Since the early days, a great number of optomechatronic products, machines or systems have come out at an increasingly accelerated rate, for the effects that can be achieved with the properties of optical components are significant. As shown in Figure 1.3, through the advancement in microsystems and the advent of MEMS, optomechatronic technology has brought about a new technology evolution, that is, a marriage of optics with microsystems or MEMS. A variety of the components or systems that belong to this category have been developed, and the atomic force microscope (AFM), optical MEMS, and optical switch are some examples among them. As seen from the historical perspective, the electronic revolution accelerated the integration of mechanical and electronic components and later, the optical revolution created the integration of optical and electronic components. This trend enabled a number of conventional systems having very low level autonomy and very low work performance to evolve into those having improved autonomy and performance. Figure 1.4 illustrates practical systems currently in use that evolved from their original old versions. Printed circuit board (PCB) inspection was carried out by the naked eyes of human workers using a microscope until recently, but is now performed by a visual inspection technique. The chip mounter or surface mounting device (SMD), originally mostly performed in a mechanical way based on CAD drawing, is now being carried out by integrated devices such as a part position estimator, visual sensors, and a servo control unit. The coordinate measuring machine (CMM) appeared as a contact then noncontact device, then became a digital electromagnetic type and then an optical type. In recent years, the CMM is actively being researched to introduce it as an accurate, reliable, versatile product, in which a sensor integration technique is to be adopted, as can be seen from the figure. The washing machine shown in Figure 1.6d also evolved from a mechanically operated machine to one having optical sensory feedback and intelligent control function. Table 1.1 illustrates the evolution of some of
touch probe
FIGURE 1.4 Illustrative evolutions.
(c) coordinate measuring machine
(a) PCB inspection
Optical probe
lamp
(d) projector
fluorescent magnetic yolk panel CRT projector
electron gun
electron beam
(b) chip / SMD mounting
mechanical positioning mounter
camera
illumination
LCD projector
B
G
R lens DLP projector
digital micro mirrors
PCB
mounter nozzle
Visual positioning mounter
Fiducial
Introduction: Understanding of Optomechatronic Technology 7
8
Optomechatronics
TABLE 1.1 Evolution in Various Products Technology/Product Data storage disc Printer Projector IC chip mounter
PCB inspection Camera
Coordinate measuring machine (CMM)
Technological Trend Mechanical recording ! magnetic recording·optical recording Dot printer ! thermal printer/ink jet printer ! laser printer CRT ! LCD·DLP projector Partially manual automated assembly ! mechanically automated assembly ! assembly with visual chip recognition Naked eye inspection ! optical/visual inspection Manual film camera ! motorized zoom, auto exposure, auto focusing ! digital camera (CMOS, CCD device) Touch probe ! optical probe ! touch probe þ visual/optical sensing
Source: Cho, H.S. and Kim, M.Y., IEEE Transactions on Industrial Electronics, 52:4, 932–943, 2005. q 2005 IEEE.
products through the presence of optomechatronic technology. In the sequel, we shall elaborate more on this issue and utilize a number of practical systems to characterize optomechatronic technology.
Optomechatronics: Definition and Fundamental Concept It can be observed from the previous discussions that the technology associated with the developments of machines/processes/systems has continuously evolved to enhance their performance and to create new value and new function. Mechatronic technology integrated by mechanical, electronic/electrical, and computer technologies has been certainly taking an important role for such evolution, as can be seen from the historical time line of technology evolution. To make them evolve further towards systems of precision, reliability, and intelligence, however, optics and optical engineering technology needed to be integrated into mechatronics, thus compensating for some limitations in the existing functionalities and creating new ones. The optomechatronics centered in the middle of Figure 1.5 is, therefore, a technology integrated with the optical, mechanical, electrical, and electronic technologies. The technology fusion in this new paradigm is termed optomechatronics or
Introduction: Understanding of Optomechatronic Technology
9
Optics
Optom ech at
r
Op t
ics tron lec oe
ics on
Optomechatronics
Mechanics
M e c h a tr o nic s
Electronics
FIGURE 1.5 The optomechatronic system.
optomechatronic technology [6, 7]. Figure 1.5 shows the integrated technologies that can be achieved by three major technologies: optical, electrical, and mechanical. We can see that optomechatronics can be achieved with a variety of different integrations. We will see in Chapter 5 that these three important combined technologies, optoelectronics, optomechanics, and mechatronics, will be the basic elements for optomechatronics integration. In this section, to provide a better understanding of, and insight into, the system we will illustrate a variety of optomechatronic systems being used in practice and briefly review the basic roles of optical and mechatronic technologies. Practical Optomechatronic Systems Examples of optomechatronic systems are found in many engineering fields such as control and instrumentation, inspection and test, optical, manufacturing, consumer and industrial electronics, MEMS, automotive, and biomedical applications. Here, we will take only some examples of such fields of application. Cameras and motors are typical products which are operated by optomechatronic components. For example, a smart camera [3] is equipped with an aperture control and a focusing adjustment together with an illuminometer to perform well regardless of the ambient brightness change. With this system configuration, new functionalities are created for the enhancement of the performance modern cameras. As shown in Figure 1.6a, the main components of a camera are several lenses, an aperture, a shutter, and a film or an electrical image cell such as CCD or complementary metal oxide semiconductor (CMOS). Images are focused and exposed on the film or the electrical image cell via a series of lenses which effect zooming and focusing of an object. Moving the lenses with respect to the imaging plane results in changes in magnification and focusing points. The amount of light entering
FIGURE 1.6 Illustrations of optomechatronic systems.
(g) fiber scope device for inspection for microfactory
2 mm
SMA coil spring
bending part SMA actuator
image guide fiber light guide fiber
sweeper
laser half-mirror
Photo-detector lens
=Closed(up)
lens
fibers
Non-interrupted signal
mirror
disk
squeeze roll
(i) pipe welding process
ng ldi n we ctio e r i impeder d
contact tip
laser
(f) n×n optical switching system
=Open(down)
cutoff signal MEMS Mirror
input fiber array
(c) optical storage disk
mirror (X-Y scanner)
(h) rapid prototyping process
liquid photopolymer
focusing lens
platform
elevator motorized actuators
PCB
illuminator and camera
micro-positioner
macro-positioner
(e) vision guided micro positioning system
picker
part rack
laser source
output of wash sensor
phototransistor wash sensor drainpipe mechanical part
(d) modern washing machine with optical sensory feedback
motor
light
3-axis piezo electric stage
AFM Tip
AFM Cantilever
(b) atomic force microscope
infrared LED
water supply valve
washing tank water
upper lid
(a) camera
y
position sensitive detector z y
laser
10 Optomechatronics
Introduction: Understanding of Optomechatronic Technology
11
through the lenses is detected by a photosensor and is controlled by changing either the aperture or shutter speed. Recently, photosensors or even CMOS area sensors are used for autofocusing with a controllable focusing lens. A number of optical fiber sensors employ optomechatronic technology whose sensing principle is based on detection of modulated light in response to changes in the physical variables to be measured. For example, the optical pressure sensor uses the principle of the reflective diaphragm, in which deflection of the diaphragm under the influence of pressure changes is used to couple light from an input fiber to an output fiber. An atomic force microscope (AFM) is composed of several optomechatronic components: a cantilever probe, a laser source, a position-sensitive detector (PSD), a piezo-electric actuator and a servo controller, and an x-y servoing stage, as shown in Figure 1.6b, which employs a constant-force mode. In this case, the deflection of the cantilever is used as input to a feedback controller, which, in turn, moves the piezo-electric element up and down in z, responding to the surface topography by holding the cantilever deflection constant. This motion yields a positional variation of light spot at the PSD, which detects the z-motion of the cantilever. The position-sensitive photodetector provides a feedback signal to the piezo-motion controller. Depending upon the contact state of the cantilever, the microscope is classified into contact AFM, intermittent AFM, or noncontact AFM. The optical disc drive (ODD) or optical storage disc is an optomechatronic system as shown in Figure 1.6c. The ODD is composed of an optical head that carries a laser diode, a beam focus servo that dynamically maintains the laser beam in focus, and a fine track voice coil motor (VCM) servo that accurately positions the head at a desired track. The disc substrate has an optically sensitive medium protected by a dielectric overcoat and rotates under a modulated laser beam focused through the substrate to a diffractionlimited spot on the medium. Nowadays a washing machine effectively utilizes optoelectronic components to improve washing performance. It has the ability to feedback control the water temperature within the washing drum and adjust the washing cycle time, depending upon the dirtiness inside the washing water area. As shown in Figure 1.6d, the machines are equipped with an optomechatronic component to achieve such a function. To detect water contamination a light source and a photo-detector are installed at the drain port of the water flowing out from the washing drum, and this information is fedback to the fuzzy controller to adjust washing time or water temperature [44]. The precision mini-robot equipped with a vision system [4] is carried by an ordinary industrial (coarse) robot as shown in Figure 1.6e. Its main function is fine positioning of the object or part to be placed in a designated location. This vision-guided precision robot is directly controlled by visual information feedback, independently of the coarse robot motion. The robot is flexible and low cost, being easily adaptable to change of batch run size, unlike the expensive, complex-and-mass production oriented equipment. This system can be effectively used to assemble wearable computers which
12
Optomechatronics
require the integration of greater numbers of heterogeneous components in an even more compact and light-weight arrangement. Optical MEM components are miniature mechanical devices capable of moving and directing a light beam as shown in Figure 1.6f. The tiny structures (optical devices such as mirrors) are actuated by means of electrostatics, electromagnetics, and thermal actuating devices. If the structure is an optical mirror, the device can move and manipulate light. In optical networks, optical MEMS can dynamically attenuate a switch, compensate, and combine and separate signals, all in an optical manner. The optical MEMS applications are increasing and classified into five main areas: optical switches, optical attenuators, wavelength tunable devices, dynamic gain equalizers, and optical add/drop multiplexes. Figure 1.6g illustrates a fine image fiberscope device [42] which can perform active curvature operations for inspection of a tiny, confined area such as a micro-factory. A shape memory alloy (SMA) coil actuator enables the fiberscope to move through a tightly curvatured area. The device has a fine image fiberscope of 0.2 mm outer diameter with light guides and 2000 pixels. Laser-based rapid prototyping (RP) is a technology that produces prototype parts in a much shorter time than traditional machining processes. One use of this technology is stereo-lithography apparatus (SLA). Figure 1.6h shows the SLA which utilizes a visible or ultraviolet laser and a position servo mechanism to selectively solidify liquid photo-curable resin. The process machine forms a layer with a cross-sectional shape that has been previously prepared from computer-aided design (CAD) data of the product to be produced. By repeating the forming layers in a specified direction, the desired three-dimensional shape is constructed layer by layer. This process solidifies the resin to 96% of full solidification. After building, in a post-curing process the built part is put into an ultraviolet oven to be cured up to 100%. There are a number of manufacturing processes requiring feedback control of in-process state information that must be detected by optoelectronic measurement systems. One such process is illustrated here to help readers to understand the concept of the optomechatronic systems. Figure 1.6i shows a pipe-welding process that requires stringent weld quality control. A structured laser triangulation system achieves this by detecting the shape of a weld bead in an on-line manner and feeding back this information to a weld controller. The weld controller adjusts the weld current according to the shape of element being made. In this situation, no other effective method of instantaneous weld quality measurement can replace the visual in-process measurement and feedback control described here [21]. Basic Roles of Optical and Mechatronic Technologies Upon examination of the functionalities of a number of optomechatronic systems, we can see that there are a number of functions that can be carried
Introduction: Understanding of Optomechatronic Technology
13
out by optical technology. The major functions and roles of optical technology can be categorized into several functional domains as shown in Figure 1.7 [5]. Basic Roles of Optical Technology (1) Illumination: illumination, which is shown in Figure 1.7a, provides the source of photometric radiant energy incident to object surfaces. In general, it produces a variety of different reflective, absorptive, and transmissive characteristics depending on the material properties and surface characteristics of the objects to be illuminated. The illumination source emits spectral energy from a single wavelength which those produces a large envelope of wavelength. (2) Sensing: optical sensors provide fundamental information on physical quantities such as force, temperature, pressure, and strain as well as on geometric quantities such as angle, velocity, etc. This information is obtained by optical sensors using various optical phenomena such as reflection, scattering, refraction, interference, diffraction, and so on. Conventionally, optical sensing devices are composed of a light source, photonic sensors, and optical components such as lenses, beam splitter, and optical fiber as shown in Figure 1.7 b. Recently, numerous sensors have been developed using optical fiber for its advantages in various applications. Optical technology can also contribute to material science. The composition of chemicals can be analyzed by spectrophotometry, which recognizes the characteristic spectrum of light that could be reflected, transmitted, and radiated from the material of interest. (3) Actuating: light can change physical properties of materials by increasing the temperature of the material or affecting the electrical environment. The materials which can be changed by light are lead zirconate titanate (PZT) and SMA. As shown in Figure 1.7c, the PZT is composed of ferroelectrics material, in which the polar axis of the crystal can be changed by applying an electric field. In optical PZT, an electric field is induced in proportion to the intensity of light. The SMA is also used as an actuator. When SMA is illuminated by light, its shape is changed as a memorized shape due to the increase of temperature. On the other hand, when the temperature of SMA is decreased, its shape is recovered. The SMA is used in a variety of actuator, transducer, and memory applications. (4) Data (signal) storage: digitized data composed of 0 and 1 can be stored in media and read optically as illustrated in Figure 1.7d. The principle of optical recording is using light-induced changes in the reflection properties of a recording medium. That is to say, the data are carved in media by changing the optical properties in
FIGURE 1.7 Basic roles of optical technology.
14 Optomechatronics
Introduction: Understanding of Optomechatronic Technology
(5)
(6)
(7)
(8)
15
the media with laser illumination. Then, data reading is achieved by checking the reflection properties in the media using an optical pickup sensor. Data transmitting: light is a good medium for delivering data for its inherent characteristics such as high bandwidth unaffected by external electromagnetic noise. Laser, a light source used in optical communication, has high bandwidth and can contain a lot of data at a time. In optical communication, the digitized raw data such as text or picture are transformed into light signals and delivered to the other side of the optical fiber and decoded as the raw data. As indicated in Figure 1.7e, the light signal is transferred within the optical fiber without loss by total internal reflection. Data displaying: data are effectively understood by end users by visual information. In order to transfer data to users in the form of an image or graph, various display devices are used such as cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED), plasma display panel (PDP), etc. As illustrated in Figure 1.7f, they are all composed of pixel elements consisting of three basic coloremitting cells that are red, green, and blue light. Arbitrary colors can be made by the combination of these three colors. Computing: optical computing is performed by using switches, gates, and flip-flops in their logic operation just like digital electronic computing. Optical switches can be built from modulators using optomechanical, optoelectronic, acousto-optic, and magneto-optic technologies. Optical devices can switch states in about a picosecond or a thousandth of billionth of a second. An optical logic gate can be constructed from the optical transistor. For an optical computer, a variety of circuit elements besides the optical switch are assembled and interconnected, as shown in Figure 1.7g. Light alignment and waveguide are two big problems in the actual implementation of the optical computer. Material property variation: when a laser is focused on a spot using optical components, the laser power is increased on a small focusing area. This makes the highlighted spot of material change its state as shown in Figure 1.7h. Laser material processing methods utilize a laser beam as the energy input, and can be categorized into two groups: (1) the method for changing the physical shape of the materials, and (2) the method for changing the physical status of the materials.
Basic Roles of Mechatronic Elements The major functions and roles of mechatronic elements in optomechatronic systems can be categorized into the following five functional domains: sensing, actuation, information feedback, motion or state control, and embedded intelligence with microprocessor [6].
16
Optomechatronics
First, transducer technology used for sensing nowadays enjoys the integrated nature of mechatronics. The majorities of sensors belong to this category. Second, the drivers and actuators produce a physical effect such as a mechanical movement or a change of property and condition. Third, one of the unique functions of mechatronics is to feedback information for certain objectives. Fourth, the control of motion or state of systems is a basic functionality that can be provided by mechatronics. Last, a mechatronic system implemented with a microprocessor provides many important functions, for example, the stored or programmed control, the digital signal processing, and the design flexibility for the whole system. In addition, advantages of integration within a small space and the low power consumption are attractive features. Characteristics of Optomechatronic Technology Based upon what we have previously observed from various optomechatronic systems, we can summarize the following characteristic points: (1) they possess one or more functionalities to carry out certain given tasks; (2) to produce such functionalities, several basic functional modules are required to be appropriately combined; (3) to achieve combining functions in a desired manner, a certain law of signal (energy) transformation and manipulation needs to be utilized that converts or manipulates one signal to another in a desired form, using the basic mechanical, optical, or electrical one; (4) optomechatronic systems are hierarchically composed of subsystems, which are then composed of units or components. In other words, elements, components, units, or subsystems are integrated to form an optomechatronic system. As we have seen from various illustrative examples of optomechatronic systems discussed above, optomechatronic integration causes all three fundamental signals to interact with each other as shown in Figure 1.8a. Here, three signals imply three different physical variables originated from optical, mechanical, and electrical disciplines. For instance, an optical signal includes light energy, ray and radiation flux, mechanical signal, energy, stress, strain, motion and heat flux, and electrical signal, current, voltage, and magnetic flux, and so on. Depending on how they interact, the properties of the integrated results may be entirely different. Therefore, it is necessary to consider the interaction phenomenon from the point of view of whether the interaction may be efficiently realizable. Optomechatronic integration can be categorized into three classes, depending on how optical elements and mechatronic components are integrated. As indicated in Figure 1.8b, the classes may be divided into the following. (1) Optomechatronically fused type In this type, optical and mechatronic elements are not separable in the sense that if either are removed from the system that they constitute, the system cannot function properly. This implies that those two separate
Introduction: Understanding of Optomechatronic Technology
17
physical inter action
(a) optomechatronic interaction
(b) integration type
dimensional interaction
(c) interaction between three different dimensional worlds FIGURE 1.8 Characteristic features of optomechatronic technology.
elements are functionally and structurally fused together to achieve a desired system performance. Figure 1.9a shows the systems integrated according to this type and include an optically ignited weapon system (left) and a photostrictive actuator (right). The constituent element of each system is shown in the figure where O, M, and E stands for optical, mechanical, and electrical, respectively. (2) Optically embedded mechatronic type In this type, an optical element is embedded into a mechatronic system. The optical element is separable from the system, but the system can function with a decreased level of performance or can function in an entirely different manner. The majority of engineered optomechatronic systems belong to this category, such as servomotor, washers, vacuum cleaners, monitoring and control systems for machines and manufacturing processes, robots, cars, and so on. The anatomy of the servo motor is shown in Figure 1.9b. (3) Mechatronically embedded optical type This type can be found basically from an optical system whose construction is integrated with mechanical and electrical components. Many optical systems require positioning or servoing optical elements and devices to manipulate and align a beam and to control the polarization of the beam. Typical systems that belong to “positioning or servoing” include cameras, optical projectors, galvanometers, series parallel scanners, line
18
Optomechatronics
FIGURE 1.9 Illustration of types of optomechatronic integration. Source: Cho, H.S. and Kim, M.Y., IEEE Transactions on Industrial Electronics, 52:4, 932–943, 2005. q 2005, IEEE.
scan polygons, optical switches, fiber squeezer polarization controllers, and so on. In some other systems, an acoustic wave generator driven by a piezo-element is used to create a frequency shift of a beam. Another typical system is the passive (off-line) alignment of optical fiber–fiber or fiber– waveguide attachment. In this system, any misalignment is either passively or actively corrected by using a micropositioning device to maximize their coupling between fiber – fiber or fiber wave guide. The anatomy of a zoom lens that belongs to this type is shown in Figure 1.9c and consists of five basic elements. Traditional optomechatronic systems usually operate in macro-worlds, if we term a macro-world as a large system whose size can range from mm to a few meters. As briefly mentioned before, due to the miniaturization trend and availability of MEMS and nano-technologies, optomechatronic integration is becoming popular, and thus accelerating the downsizing of engineering products. Optomechatronic integration is therefore achieved in two different physical scales as shown in Figure 1.8c: macro- and micro-nano scales. This length scale difference produces three different types of integration: integration in macro-scale (left), integration in micro/nanoscale (right) and integration in mixed scales (middle). The middle part of the figure implies an employment of combined macro- and micro/nano-systems for the integration. Integration of this type can be found from a variety
Introduction: Understanding of Optomechatronic Technology
19
* atom nanotube virus & bacteria optical lithography
human hair optical fiber 10
10−5 10
−3
MEMS
10−11
10−10 10−9
STM AFM
x-ray nanomanipulation thin film zone plate ring spacing
10−8 10−7
10−6
biological cell
FET
DNA CNT
NEMS
visible light
−4
infrared
* scale in meters
FIGURE 1.10 Scales and dimensions of some entities that are natural and man-made.
of physical systems such as an optical disc (macro) having a micro fineadjustment mechanism, a laser printer that may be operated by micro-lens slits, an AFM, etc. Figure 1.10 depicts scales of some natural entities and some man-made ones. MEMS parts range from hundreds mm to few mm and NEMS parts from few hundreds to tens nm. Nanomanipulation such as atomic lettering using scanning tunneling microscope (STM) is of the order of approximately nm or less. It is known that most of the microscopes currently available have used extremely high resolution. In particular, the extreme case of the smallest scale can be found from STM and AFM whose resolutions range between , 0.1 nm and , 0.5 nm, respectively. One important factor we need to consider in dealing with micro-systems is the so-called “size or scale effect” due to which the influence of the same physical phenomena becomes variable with the length scale of a system. To take some examples: (1) Inertial (volume) force that influences a macro-scale system becomes less significant in micro-systems. Instead, surface effects (force) become rather dominant. This is because in general the inertial force is proportional to volume (L 3), whereas the surface force is in proportion to surface area (L 2). (2) In optical and mechatronic interaction, deflections on the order of wavelengths of light do matter in a microsystem but not in a macrosystem.
20
Optomechatronics
(3) Atomic attractive and repulsive forces become significant in microsystems. (4) The smaller geometrical size material gets stronger. Example (1) may mislead us into neglecting the less significant terms in analyzing microsystems. However, in the case of dynamic analysis and control of the systems, they may not be negligible when the accuracy of the positioning is in high precision or when the range of dominant force operated on the systems is compatible with that of the less significant terms.
Fundamental Functions of Optomechatronic Systems From the discussions in the previous sections, we now elaborate on what type of fundamental functions the optomechatronic integration can produce.
Information feedback control Mechanism design
Precision actuators Optical / Visual Motion control
Artificial intelligence Micro processor
sensors, actuators data display materials processing information processing recognition transmission / switching micro elements
Signal processing Pattern recognition
MEMs
Sensor fusion
FIGURE 1.11 Enabling technologies for optomechatronics.
Sensors & measurement
Introduction: Understanding of Optomechatronic Technology
21
There are a number of distinct functions which originate from the basic roles of optical elements and mechatronic elements discussed. When these are combined, the functional forms may generate the fundamental functions of the optomechatronic system. Figure 1.11 illustrates the technology element needed to produce optomechatronic technology. In the center of the figure, those elements related to optical elements are shown, while mechatronic elements and artificial intelligence are listed on its periphery. These enabling technologies are integrated to form five engines that drive the technology. These include: (1) actuator module, (2) sensor module, (3) control module, (4) signal/ information processing module, and (5) decision making module. It is noted that the integration of these gives a specified functionality with certain characteristics. A typical example is the sensing of an object surface with an AFM. The AFM requires a complicated interaction between sensing element (laser sensor), actuator (piezo-element), controller, and other relevant software. This kind of interaction between modules is very common in optomechatronic systems, and produces the characteristic property of the systems.
Fundamental Functions A number of fundamental functions can be generated by the fusion of optical mechatronic elements. These include (1) illumination control, (2) sensing, (3) actuating, (4) optical scanning, (5) visual/optical information feedback control, (6) data storage, (7) data transmission/switching, (8) data display, (9) optical property variation, (10) sensory feedback based optical system control, (11) optical pattern recognition, (12) remote monitoring/control, and (13) material processing [5]. We will discuss these fundamental functions below.
Illumination Control Illumination needs to be adjusted depending on the optical surface characteristics and surface geometry of objects in order to obtain a good quality image. The parameters to be adjusted include incident angle, and the distribution and intensity of light sources. Figure 1.12a illustrates a typical configuration of such an illumination control system. The system consists of a guardant ring fiber light source, a paraboloid mirror, a toroid mirror, and a positioning mechanism to adjust the distance between the object surface and the parabola mirror. The incident angle is controlled by adjusting the distance, while the intensity of the light source in each region is controlled independently.
object
3-facet mirror xs
object
PSD B
ys
laser zs beam
glass grid
adhesive film
pressure transducer diode strip
(e) motion control : placing diode strip into a glass
camera
detector
opticsservo
(f) data storage/ retrieval
laser
disk
servo
6-D position & orientation laser beam position
tracking control
(b) measurement of a 6-degrees-of-freedom motion of arbitrary objects
PSD A
vacuum-based end effector fiducial mark
cartesian robot
laser source PSD C
FIGURE 1.12 The functionalities achieved by optomechatronic integration.
(d) mirror based scanning
laser
mirror
parabolic mirror
mirror
mirror
lens
laser
(a) illumination control
object lens
half-mirror
camera
mirror
(g) optical cross connect operation by mirror control
scanning mirror
servoing mirror array
(c) shape memory-based optical actuating
spring
motion
light
22 Optomechatronics
23
FIGURE 1.12 Continued.
Introduction: Understanding of Optomechatronic Technology
24
Optomechatronics
Sensing Various types of optomechatronic systems are used to measure various types of physical quantities such as displacement, geometry, force, pressure, temperature, target motion, etc. The commonly adopted feature of these systems is that they are composed of optical, mechanical moving, servoing, and electronic elements. The optical elements are usually the sensing part divided into optical fiber and nonfiber optical transducers. Recently these elements have been accelerating the sensor fusion in which other types of sensors are fused together to obtain necessary information. Figure 1.12b shows a six-dimensional (6D) sensory system employing the optomechatronic principle. The sensor is composed of a 6D microactuator, a laser source, three photosensitive devices (PSD), and a three-facet tiny crystal (or mirror) that transmits the light beam into three PSD sensors. The sensing principle is that the microactuator is so controlled that the top of the tiny mirror is always positioned within the center of the beam. In other words, the output of each PSD sensor is kept identical regardless of object motion. Another typical problem associated with optical sensing is focusing, which needs to be adjusted depending upon the distance between the optical lens and the surface of the objects to be measured. Actuating A number of actuators belong to this category, both optical-based and mechatronic actuation-based. Optical-based actuation utilizes transformation of optical energy into mechanical energy that can be found from a number of practical examples. Figure 1.12c shows one such actuator in which optical energy is used as a heat generator which drives a temperaturesensitive material (e.g., SMA) to move. This actuator can be used to accurately control the movement of mechanical elements. Optical Scanning Optical scanning is used to divert light direction with time in a prescheduled manner and generates a sequential motion of the optical element such as a light source. For high-speed applications, a polygon mirror or galvanometer is effectively used as shown in Figure 1.12d. To operate a scanning system, the scanning step and the scanning angle have to be considered carefully. The scanning ability is useful in applications ranging from laser printing to materials processing. Visual/Optical Information Feedback Control Visual/optical information is very useful in the control of machines, processes, and systems. In this functionality the information obtained by optical sensors is utilized for changing variables operating the systems. A great number of optomechatronic systems require this type of information
Introduction: Understanding of Optomechatronic Technology
25
feedback control. Positioning controlling the position of an object, moving it to a specified location. When a visual sensor is used for the feedback, the motion control is called “visual servoing.” Accurately placing diode strips into a glass grid by using a robot is shown in Figure 1.12e as an illustrative example. A number of practical examples can be found from motion tracking such as following the motion of a moving object and following a desired path based on the optical-sensor information. Many more typical examples can also be found from mobile robot navigation, e.g., optical-based dead reckoning, map building, and vision-based obstacle avoidances. A pipe-welding process shown in Figure 1.6i necessitates an optical information feedback control. The objective of this process control is to regulate the electric power needed for welding in order to attain a specified weld quality. Therefore, the process state relevant to the quality is detected and its instantaneous information is fedback in real-time manner. This principle can also be found from the washing machine shown in Figure 1.6d. In this system, the optical sensor detects the dirtiness of the water and this information is fedback to a controller to adjust the washing time or to adjust the water temperature inside the drum. Data Storage Data storage retrieval is performed by a spinning optical disc and controlled optical units whose main functions are beam focusing and track following, as illustrated in Figure 1.12f. Conventionally, the recording density is limited by the spot size and wavelength of the laser source. Recently, new approaches to increase the recording density are being researched such as near-field optical memory and holographic three-dimensional storing methods. Data Transmission/Switching Optical data switching is achieved by an “all optical network” to eliminate the multiple optical-to-electrical-to-optical (O-E-O) conversions in conventional optical networks [35]. Figure 1.12g illustrates a multi-mirror servo system in which mirrors are controlled to connect a light path any to any ports by using servo controllers. This kind of system configuration is also applied to switch micro-reflective or -deflective lens actuated in front of optical filters. Data Display Digital micro-mirror devices (DMDs) make projection displays by converting white-light illumination into full-color images via spatial light modulators with independently addressable pixels [30]. As schematically illustrated in Figure 1.12h, the DMD developed by Texas Instruments is a lithographically fabricated MEMS system composed of hundreds of thousands of titling aluminum-alloy mirrors (16 mm £ 16 mm) which
26
Optomechatronics
functions as a pixel in the display. Each mirror is attached to an underlying sub-pixel called the yoke, which, in turn, is attached to a hinge support post by way of an aluminum torsion hinge. This allows the mirror to rotate about the hinge axis until the landing tips touch the landing site. This switching action occurs from þ 108 to 2 108 and takes place in several msec. Optical Property Variation Optical properties such as frequency, wavelength, and so on can be tuned by making light waves interact with mechatronic motion. Figure 1.12i illustrates a frequency shifter which consists of an acoustic wave generator, a piezoelectric element, and a high birefringence fiber. In-line fiber optic frequency shifters utilize a traveling acoustic wave to couple light between the two polarization modes of a high-birefringence fiber, with the coupling accompanied by a shift in the optical frequency. Sensory Feedback-Based Optical System Control In many cases, optical or visual systems are operated based on the information which is provided by external sensory feedback, as shown in Figure 1.12j. The systems that require this configuration include: (1) zoom and focus control system for video-based eye-gaze detection using an ultrasonic distance measurement, (2) laser material processing systems whose laser power or laser focus is real-time controlled depending upon the process monitoring, (3) visually-assisted assembly systems in cooperating with other sensory information such as force, displacement, and tactile, (4) sensor fusion systems in which optical/visual sensors incorporate with other sensory systems, and (5) visual/optical systems that need to react to acoustic sound and other sensory information tactile, force, displacement, velocity, etc. Optical Pattern Recognition Three-dimensional pattern recognition shown in Figure 1.12k, uses a laser and a photorefractive crystal as the recording media to record and read holograms [38]. The photorefractive crystal stores the intensity of the interference fringe constructed by the beam reflected from a 3D object and a plane-wave reference beam. The information on the shape of the object is stored in the photorefractive crystal at this point, and this crystal can be used as a template to be compared with another three-dimensional object. To recognize an arbitrary object, an optical correlation processing technique is employed. When an object to be recognized is placed in the right position from the original object with the same orientation, the recorded hologram diffracts the beam reflected from the object to be compared, and the
Introduction: Understanding of Optomechatronic Technology
27
diffracted wave propagating to the image plane forms an image that represents the correlation between the Fourier transforms of the template object beam and the object to be compared. If a robot or a machine is trying to find targets in unknown environments, this system can be effectively used to recognize and locate them in a real-time manner. In this case a scanned motion of the optical recognition system is necessary over the whole region of interest. Remote Operation via Optical Data Transmission Optical data transmission is widely used when data/signals obtained from sensors are subject to external electrical noise or when the amount of data needed to be sent is vast, or when operation is being done at a remote site. Operation of systems at a remote site is ubiquitous nowadays. In particular, internet-based monitoring, inspection, and control are becoming pervasive in many practical systems. Visual servoing of a robot operated in a remote site is a typical example of such a system, as shown in Figure 1.12‘. In the operation room, the robot controls the position of a vibration sensor to monitor the vibration signal with the aid of vision information and transmits this signal to a transceiver. Another typical example is the operation of the visual servoing of mobile robots over the internet. Material Processing Material processing can be achieved by the integration of a laser optical source and a mechatronic servo mechanism. The system produces a material property change or cut surface and heat-treated surface of work pieces. In addition to conventional laser machining, laser micro-machining is becoming popular due to its reduced cost and accuracy. MEMS fabrication, drilling and slotting in medicine, wafer dry cleaning, and ceramic machining employ such micro-machining technology. Figure 1.12m shows a laser surface hardening process [46], which is a typical example of an optical-based monitoring and control system. The laser of high power (4 kW) focused through a series of optical units hits the surface of a workspace and changes the material state of the workpiece. Maintaining a uniform thickness of the hardened surface (less than 1 mm) is not an easy task since it depends heavily upon several process parameters such as the workpiece travel speed, laser power, and the surface properties of the material. Here, the indirect measurement of the coating thickness is made by an infra-red temperature sensor and feedback to the laser power controller.
Summary In recent years, integration of optical elements into mechatronic systems has increasingly accelerated since it produces a synergistic effect, creating new
28
Optomechatronics
functionalities for the systems or enhancing the performance of the systems. This trend will certainly be the future direction of current mechatronic technology and contribute to the advent of a new technological paradigm. This chapter has focused on helping readers to understand optomechatronic technology and systems in which optical, mechanical, electrical/electronics, and information engineering technologies are integrated. In particular, the definition and fundamental concepts of the technology have been introduced. Based upon these, it has been possible to identify types of optomechatronic systems and the fundamental functions that can be created by the integration. Altogether, thirteen functionalities have been identified. They are expected to lead to future developments of such technologies as mechatronics, instrumentation and control, adaptive optics, MEMS, biomedical technology, information processing storage as well as communication. From this point of view, optomechatronics will be a prime technological element that will lead the future direction of various technologies.
References [1] [2] [3] [4] [5] [6] [7] [8]
[9] [10] [11] [12] [13]
Backmutsky, V. and Vaisman, G. Theoretical and experimental investigation of new mechatronic scanners, Proceeding of the IEEE Nineteenth Convention of Electrical and Electronics Engineers in Israel, pp. 363– 366, 1996. Bishop, R.H. The Mechatronics Handbook. CRC Press, Boca Raton, FL, 2002. Cannon Co. Ltd, http:www.canon.com, 2002. Chen, M. and Hollis, R. Vision-guided Precision Assembly. 1999, http://www-2. cs.cmu.edu/afs/cs/project/msl/www/tia/tia_desc.html. Cho, H.S. and Park, W.S. Determining optimal parameters for stereolithography process via genetic algorithm, Journal of Manufacturing Systems, 19:1, 18 – 27, 2000. Cho, H.S. Characteristics of optomechatronic systems, Opto-mechatronic Systems Handbook, chap. 1, CRC Press, Boca Raton, FL, 2002. Cho, H.S. and Kim, M.Y., Optomechatronic technology: the characteristics and perspectives, IEEE Transaction on Industrial Electronics, 52:4, 732– 743, 2005. Ebisawa, Y., Ohtani, M. and Sugioka, A. Proposal of a zoom and focus control method using an ultrasonic distance-meter for video-based eye-gaze detection under free-head conditions, 18th Annual Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 2, pp. 523– 525, Amsterdam, Netherlands, 1997. Geppert, L. Semiconductor lithography for the next millennium, IEEE Spectrum, April, 33 – 38, 1996. Han, K., Kim, S., Kim, Y. and Kim, J. Internet control architecture for internetbased personal robot, Autonomous Robots, 10, 135– 147, 2001. Haran, F.M., Hand, D.P., Peters, C. and Jones, J.D.C. Real-time focus control in laser welding, Measurement Science and Technology, Vol. 7, 1095– 1098, 1996. Higgins, T.V. Optical storage lights the multimedia future, Laser Focus World, September/October, Vol. 31, 1995. Ishi, T. Future trends in mechatronics, JSME, International Journal, Series, III, 33:1, 1 – 6, 1990.
Introduction: Understanding of Optomechatronic Technology [14]
29
Jeong, H.M., Choi, J.J., Kim, K.Y., Lee, K.B., Jeon, J.V. and Park, Y.E., Milli-scale mirror actuator with bulk micromachined vertical combs, Proceedings of the Transducer, Sendai, Japan, pp. 1006– 1011, 1999. [15] Kamiya, M., Ikeda, H. and Shinohara, S. Data collection and transmission system for vibration test, Proceedings of the Industry Application Conference, 3, 1679–1685, 1998. [16] Kayanak, M.O. The age of mechatronics, IEEE Transactions in Industrial Electronics, 43:1, 2 – 3, 1996. [17] Kim, W.S. and Cho, H.S. A novel sensing device for obtaining an omnidirectional image of three-dimensional objects, Mechatronics, 10, 717– 740, 2000. [18] Kim, J.S. and Cho, H.S. A robust visual seam tracking system for robotic arc welding, Mechatronics, 6:2, 141– 163, 1996. [19] Kim, W.S. and Cho, H.S. A novel omnidirectional image sensing system for assembling parts with arbitrary cross-sectional shapes, IEEE/ASME Transactions in Mechatronics, 3:4, 275– 292, 1998. [20] Knopf, G.K. Short course note, SC 255 Opto-Mechatronic System Design, SPIE’s Photonic East, Boston, USA, 2000. [21] Ko, K.W. Cho, H.S., Kim, J.H and Kong, W.I. A bead shape classification method using neural network in high frequency electric resistance weld, Proceeding of World Automation Congress, Alaska, USA. [22] Krupa, T.J. Optical R&D in the Army Research Laboratory, Optics&Photonics News, June, 16 – 39, 2000. [23] Larson, M.C. and Harris, J.S. Wide and continuous wavelength tuning in a vertical cavity surface emitting laser using a micromachined deformable membrane mirror, Applied Physics Letters, 15, 607– 609, 1996. [24] Larson, M.C. Tunable Optoelectronic Devices. 2000, http://www-snow.stanford. edu/~larson/research.html. [25] Lim, T.G. and Cho, H.S. Estimation of weld pool sizes in GMA welding process using neural networks, Proceedings of Institute of Mechanical Engineers, 207, 15 – 26, 1993. [26] Madou, M. Fundamentals of Microfabrication. CRC Press, Boca Raton, FL, 1997. [27] Mahr Co. Ltd, Multiscope 250/400. 2001, http://www.mahr.com/en/content/ products/mess/mms/ms250.html. [28] McCarthy, D.C. Hands on the wheel, cameras on the road, Photonics Spectra, April, 78 – 85, 2001. [29] Mitutoyo Co. Ltd, Quick Vision. 2001, http://www.mitcat.com/e-02.htm. [30] McDonald, T.G. and Yoder, L.A. Digital micromirror devices make projection displays, Laser Focus World, August, 1997. [31] McKee, G. Robotics and machine perception, SPIE’s International Technical Group Newsletter, 9, 2000. [32] Park, I.O., Cho, H.S. and Gweon, D.G. Development of programmable bowl feeder using a fiber optic sensor, 10th International Conference on Assembly Automation, Tokyo, Japan, 1989. [33] Park, W.S., Cho, H.S., Byun, Y.K., Park, N.Y., and Jung, D.K. Measurement of 3-D position and orientation of rigid bodies using a 3-facet mirror, SPIE International Symposium on Intelligent Systems and Advanced Manufacturing, pp. 2– 13, Boston, USA. [34] Pugh, A. Robot Sensors, Tactile and Non-Vision, Vol. 2. Springer-Verlag, Berlin, 1986.
30
Optomechatronics
[35] Robinson, S.D. MEMS technology — micromachines enabling the all optical network, Electronic Components and Technology Conference, pp. 423 – 428, Orlando, FL, 2001. [36] Rogers, C.A. Intelligent materials, Scientific American, September, 122–127, 1995. [37] Roh, Y.J., Cho, H.S. and Kim, J.H. Three dimensional volume reconstruction of an object from X-ray images, Conference on SPIE Optomechatronic Systems Part of Intelligent Systems and Advanced Manufacturing, Vol. 4190, pp. 181–191, Boston, USA, 2000. [38] Shin, S.H. and Javidi, B. Three-dimensional object recognition by use of a photorefractive volume holographic processor, Optics Letters, 26:15, 1161– 1163, 2001. [39] Shiraishi, M. In-process control of workpiece dimension in turning, Annals of the CIRP, 28:1, 333– 337, 1979. [40] Takamasu, K. Development of a nano-probe system, Quarterly Magazine Micromachine, 35, 2001. [41] Toshiyoshi, H., Su, J.G., LaCosse, J. and Wu, M.C. Micromechanical lens scanners for fiber optic switches, Proceedings of the Third International Conference on Micro Opto Electro Mechanical Systems (MOEMS 99), pp. 165– 170, Mainz, Germany, , 1999. [42] Tsuruta, K., Mikuriya, Y. and Ishikawa, Y. Micro sensor developments in Japan, Sensor Review, 19, 37 – 42, 1999. [43] Veeco co. Ltd, Contact AF. 2000, http://www.tmmicro.com/tech/modes/ contact.htm. [44] Wakami, N., Nomura, H. and Araki, S. Fuzzy logic for home appliances, Fuzzy Logic and Neural Networks, pp. 21.1 – 21.23, McGraw Hill, New York, 1996. [45] Wilson, A. Machine vision speeds robot productivity, Vision Systems Design, October, 2001. [46] Woo, H.G. and Cho, H.S. Estimation of hardened layer dimensions in laser surface hardening processes with variations of coating thickness, Surface and Coatings Technology, 102:3, 205– 217, 1998. [47] Zankowsky, D. Applications dictate choice of scanner, Laser Focus World, December, 1996. [48] Zhang, J.H. and Cai, L. An autofocusing measurement system with a piezoelectric translator, IEEE/ASME Transactions in Mechatronics, 2:3, 213– 216, 1997. [49] Zhou, Y., Nelson, B.J. and Vikramaditya, B. Integrating optical force sensing with visual servoing for microassembly, Journal of Intelligent and Robotic Systems, 28, 259– 276, 2000.
2 Fundamentals of Optics CONTENTS Reflection and Refraction ................................................................................... 33 Lenses .................................................................................................................... 36 Refraction at a Spherical Surface............................................................... 37 Multiple Lenses and System Matrices.............................................................. 46 The System Matrices.................................................................................... 50 Computer Ray Tracing ................................................................................ 52 Aperture Stops and Pupils ................................................................................. 53 Aberration ............................................................................................................. 55 Spherical Aberration.................................................................................... 56 Coma Aberration.......................................................................................... 56 Field Curvature ............................................................................................ 57 Astigmatism .................................................................................................. 57 Distortion....................................................................................................... 59 Polarization ........................................................................................................... 61 Coherence.............................................................................................................. 64 Interference ........................................................................................................... 65 Young’s Experiment .................................................................................... 67 Fabry-Perot Interferometer ......................................................................... 69 Michelson Interferometer ........................................................................... 72 Diffraction.............................................................................................................. 74 Double Slits ................................................................................................... 79 Multiple Slits................................................................................................. 81 Circular Aperture......................................................................................... 84 Diffraction Grating....................................................................................... 89 Optical Fiber Transmission................................................................................. 92 Gaussian Beam Optics ........................................................................................ 95 Problems................................................................................................................ 97 References ........................................................................................................... 103
Physical optics is rooted on the fact that light corresponds to the electromagnetic radiation in the narrow slice of spectrum. Optics concerns
31
32
Optomechatronics
with the radiation within this range visible to the human eye but often extends to the region of spectrum near visible region. Classification due to variation in wave length is summarized in Figure 2.1. Visible radiation ranges from 0.35 to 0.75 mm, depending upon types of color of the light their wave length varies. The ultraviolet region extends from 0.01 to 0.35 mm, while infrared radiation extends from the lower end of the visible spectrum to nearly 300 mm. Being conduced as one of the electromagnetic waves, light radiates from a source, propagating straight through a medium at a velocity, c, in a form of combination of electric and magnetic waves oscillating at a frequency v and with a wave length l: The simplest form of such combination of waves is a monochromatic (single color) light wave that is composed of sinusoidal components of electric and magnetic fields, propagating in free space along the direction z as represented in Figure 2.2. The wave here is a plane wave and the electric field is varying parallel to the propagation direction. As can be seen from the figure, the electric and magnetic fields are vector quantities, and thus their magnitude and direction need to be specified. However, both fields are oriented at right angles to each other in nonabsorbing media. Only the direction and magnitude of the electric field need to be specified. We will return to this subject with a little bit more detail when we discuss polarization of light in this chapter. The speed of propagation of the wave in
wavelength (m) 10−10
10−8
frequency (HZ) X-ray X -ray
UV UVlight light
1×1018
1×1016
visible 1×1014
10−6 infrared
1×1012
10−4
10−2
μ-waves μ -waves
FIGURE 2.1 The electromagnetic spectrum: optical radiation.
1×1010
Fundamentals of Optics
33 y E x B
B
E
E
B
B
E
z
FIGURE 2.2 Propagation of a plane-polarized electromagnetic wave.
free space (vacuum) is approximately given by c ¼ 2:998 £ 108 m/sec. The speed in air is just a little bit smaller than this, while that in glass and water is always much smaller than that in vacuum. Throughout this book we will confine ourselves to optical problems concerning visible and monochromatic light. In the first part of this chapter, we will treat a light wave as if it travels from its source along straight lines without any wave motion. Using this treatment, we therefore cannot explain such phenomena as the interference, polarization, diffraction and so on that occur due to wave motion of the light. The regime of this treatment is called “geometric optics” within which the wavelength or wave motion effect is considered negligible compared to the dimensions of the optical system interacting with light waves. In the later part of this chapter, we will treat wave optics, in which wave motion of light will be included, to describe the above phenomena that cannot be explained by geometric optics.
Reflection and Refraction A light wave transmitting through a lens or a prism is typical of wave refraction. A light wave reflecting from the surface of a mirror or a coated reflective surface is typical of wave reflection. To understand these phenomena we will review the laws of reflection and refraction. When a light wave travels through several homogeneous media in sequence, its optical path constitutes a sequence of discontinuous line segments. A simple case of this situation occurs when a light ray is incident upon a plane surface separating two media. There are several laws governing the direction of light propagation at the interface. Here, we will discuss briefly
34
Optomechatronics
FIGURE 2.3 Reflection ur and refraction ut at the boundary of two different media.
the two most frequently used laws, reflection and refraction. When a light wave is incident on an interface lying in the plane of incidence shown in Figure 2.3, a certain fraction of the wave is absorbed or transmitted and the remainder is reflected. This is illustrated using a plane wave in Figure 2.4a. As can be seen from the figure, three waves lie in the plane of incidence and those waves are conveniently represented by three corresponding rays
incident light
normal
A
normal
reflected light
reflected light
incident light
C
qi medium 1 medium 2
B D
refracted light
(a) a beam of plane waves enters the interface
(b) reflection
normal incident light
qi
ni qt
nt refracted light
(c) refraction FIGURE 2.4 A plane wave front incident on the boundary between two media.
qr
Fundamentals of Optics
35
AB; BC; BD: A ray is a line that indicates the direction of wave propagation and is perpendicular to the wave front. Therefore, we will transform this figure into those represented only by rays as shown in Figure 2.4b. The law of reflection states that the angle of reflection equals the angle of incidence. This means that
ui ¼ ur
ð2:1Þ
This is a special case of light reflection. In general cases, most interfaces contain both specular and diffuse surfaces. Diffuse surfaces cause the reflected light to scatter in all directions radiating an equal amount of power per unit area per unit solid angle. The condition given in Equation 2.1 is valid for the specular reflection. When an incoming plane light passes through the boundary between two homogeneous media, the direction of it will bend because of its speed change. The amount it is bent is dependent upon the absolute refraction index of each media, which is defined by n ¼ c=v, where c is speed of light in a vacuum and v is the speed in the medium. The law of refraction, called Snell’s law, states that the refraction angle ut obeys the following relation, ni sin ui ¼ nt sin ut
ð2:2Þ
where ni is the refraction index of the incident medium, while nt is that of the transmitting medium. When ni , nt , that is, the ray enters a medium with higher index (optically more dense), it bends toward the normal. In contrast, when ni . nt , (from optically more to less dense) it bends away from the normal. There are some special cases where further thoughts on Snell’s law are necessary. To explain this, let us suppose that incident light coming from a point source located below an interface travels across the upper medium of lower relative index as shown in Figure 2.5. When ui ¼ 0, that is, an incident normal to the surface ray is transmitted straight across the boundary without change of direction (case 1). However, as the incident angle ui becomes larger and larger, the angle of refraction becomes large. (case 1)
(case 2)
(case 3)
(case 4)
qt nt
qt
qr = qi = 0
nii
qi
S FIGURE 2.5 Various types of refraction phenomena.
qi
90° qc
qi
qi
36
Optomechatronics
TABLE 2.1 Refractive Index of Various Materials Material Vacuum Air (208C) Water vapor Optical glass Quartz Crystal Plexiglas Lucite Zinc crown glass Light flint glass Heavy flint glass Heaviest flint glass Sugar solution (30%) Polymer optical fiber (polymethyl ethacrylate PMMA) Polymer optical fiber (polystyrene) Polymer optical fiber (polycarbonates)
Refractive Index (n) 1.0000000 1.000292 1.00024 1.5 2 01.85 1.4868 2.0 1.488 1.495 1.517 1.575 1.65 1.89 1.38 1.49 1.59 1.5 2 1.57
Finally, there will be a critical angle of incidence that will make ut ¼ 908 (case 3). This angle is called the “critical angle, uc ” which is given by
uc ¼ sin21
nt ni
ð2:3Þ
When the angle of incidence ui . uc , the incident ray exhibits total internal reflection, as shown in case (4). This type of reflection is effectively used for fiber optic transmission, which will be discussed at the end of this chapter. The refraction index of some materials is listed in Table 2.1.
Lenses Figure 2.6 shows when a point source S produces a spherical wave propagating toward a convex or concave lens. The waves expand and intercept the thin lens. In case of the convex lens, the wave fronts contract and converge to a point I, where real image is formed. For the concave lens, the wave fronts expand further so that a virtual image I is created at the backside of the lens. This point is known as a focus of the bundle of rays. This implies that the change in the curvature of the wave fronts occurs due to the refracting power of the lens. One other thing to note is that, in both lenses,
Fundamentals of Optics
37
S
I image point
object point
S object point I image point
(a) convex lens
(b) concave lens
FIGURE 2.6 Interference of a spherical wave with two typical lenses.
if a point source is placed at the image point I, the corresponding image will be located at S. These two points S and I are called “conjugate points”. Refraction at a Spherical Surface A spherical surface is commonly used as a lens surface and can be either concave or convex. Figure 2.7 shows a concave surface of radius R where two rays from a point source S are emanating as indicated by an arrow. One is an axial ray impinging a spherical interface centered at C, which is normal to the surface at its vertex point V. The source is located at a distance so from V. The other ray incident at a point P with an angle ui is refracted with an angle ut , as indicated by an arrow. According to Snell’s law, this ray refracts with the relation, n1 sin ui ¼ n2 sin ut
ð2:4Þ
qt
P
h
qi j
V
n2
s0
n1
FIGURE 2.7 Refraction at a spherical interface.
si
j¢ S R
α I
C
38
Optomechatronics
These two rays appear to be emanating from a point I which is called the image point, whose distance is si from V. The exact relationship between so and si is rather complicated to derive, but we will simplify by assuming ! ! SP ¼ SV
and
! ! IP ¼ IV
ð2:5Þ
by using small angle approximation i.e., sin u . u; cos u . 1: From this relation, it follows from Equation 2.4 that n1 ðw 2 aÞ ¼ n2 ðw0 2 aÞ
ð2:6Þ
Taking the tangents for the angles in the above equation and utilizing the assumption given in Equation 2.5, Equation 2.6 further leads to n1 n n 2 n2 2 2 ¼ 1 ð2:7Þ so si R It is noted that the object distance, so is positive whereas si , the distance of the image (virtual), is negative and R is negative according to the sign convention for spherical refraction surfaces and thin lenses. This equation then can be generalized to include the case of a convex surface. The general form of Equation 2.7 can be expressed as n1 n n 2 n1 þ 2 ¼ 2 ð2:8Þ so si R The above equation is valid for rays that are incident to the surface near to the optical axis such that w and h are small. These rays are known as paraxial rays. The optics dealing with such rays arriving at the paraxial region is called Gaussian optics, named after K. F. Gauss (1777 –1855), the first to develop this formulation. In the preceding discussion, we have dealt with a single spherical surface. In order to relate the source points and image point P we now apply this method to determine the locations of the conjugate points for a thin lens, which represents the thin-lens equation. In this case, two refractions at two spherical surfaces are involved. Referring to Figure 2.8, the first refractions at a spherical interface of radius R1 yields n1 n n 2 n1 þ 2 ¼ 2 so1 si1 R1
ð2:9Þ
where so1 is the distance of the source S from the vertex point V1 and si1 is the distance of the image from the vertex point V1 : At the second spherical surface of radius R2 n2 n n 2 n2 þ 1 ¼ 1 ð2:10Þ so2 si2 R2 Here, the image point P0 acts as a source object point for the second surface, which is considered the second object. Now since the lens thickness is
Fundamentals of Optics
39
R1
R2 P'
P C2
S
s i1
n1
V1
so1
O n2
V2
n1
C1 si2
d
so2
FIGURE 2.8 A point source passing through a spherical lens having two different spherical interfaces.
negligible for a thin lens, i.e., d ! 0, so2 becomes so2 ¼ 2si1
ð2:11Þ
Combining Equation 2.9, Equation 2.10 and Equation 2.11 leads to 1 1 n 2 n1 þ ¼ 2 so si n1
1 1 2 R1 R2
ð2:12Þ
where so1 ¼ so and si2 ¼ si are substituted. It is noted that so and si is measured from either the vertices V1 and V2 or the lens center, O: When a source is located at infinity, we can see that the image distance becomes the focal length f : The focal length of the thin lens is therefore defined as the image distance for an object at infinity. In this case, Equation 2.12 can be rewritten as 1 ðn 2 n1 Þ 1 1 2 ¼ 2 f n1 R1 R2
ð2:13Þ
This equation is referred to as the lens maker’s formula. When a surrounding medium is air, n1 becomes approximately unity, i.e., n1 . 1: In this case, the lens formula becomes 1 1 1 ¼ ðn2 2 1Þ 2 f R1 R2 By using Equation 2.12 and Equation 2.13, the thin lens equation can be expressed as 1 1 1 þ ¼ so si f
ð2:14Þ
40
Optomechatronics
Equation 2.14 is called the Gaussian lens formula. In this formulation we have assumed that all light rays make small angles with the optical axis, which were termed paraxial rays. To help understand the formation of the focal point F, we use Figure 2.9, which illustrates how wave fronts of plane waves pass through the convex and concave lenses. It can be seen that the thicker portion of the lenses causes light to be delayed, thus resulting in convergence for a convex lens and divergence for a concave lens as shown in Figure 2.9a and Figure 2.9b, respectively. In the case of the convex lens, all rays that pass through the lens meet at the focal point F, whereas in the case of the concave lens all the rays that pass through diverge afterwards, as if all the rays appear to emanate from the focal point F: This focal point is called the second of image-side focus. Figure 2.10 summarizes the effect of positive and negative lenses on incident plane wave. The convex lens causes a divergent ray either to converge or diverge less rapidly. In particular, in a situation when a point source S is located at the focal point, the refracted ray becomes parallel, as shown in Figure 2.10d. A concave lens causes a convergent ray either to converge (not shown here) or to diverge (Figure 2.10b). It causes a divergent beam to become more divergent as shown in Figure 2.10c to e. When a convergent beam incident to the lens is directed toward the focal point F, it becomes parallel to the optical axis, as can be seen in Figure 2.10a. As we can see, a convex lens makes a convergent beam converge more rapidly or makes a divergent beam diverge with less speed or converge. In contrast, a concave lens behaves in exactly the opposite way. The output beam properties shown here are made only by a single lens with various angles of incident beam. The output beam properties may be made different depending on how we make the incoming beam incident to the lens. Also, they may be made variable depending on what types of lens or combination of lenses we use in order to manipulate a given incident beam. All of these concerns are involved with optical system design. The beam expander is a good illustration that we can combine two lenses in a proper way to make a beam expand with a desired magnification, as
plane wave
plane wave
F
(a) convex lens FIGURE 2.9 A plane wave transmitting through a lens.
F
(b) concave lens
Fundamentals of Optics
41 F
F
(a)
F
.
F
s
s
F
F
F
(b)
s’
s’
s
s’ F
s
(e)
F
.
s’
(c) s
(d)
F
F
s
F
F
F
s
s’ F
F
.
s’ F
F
s F
s’
F
concave lens
convex lens FIGURE 2.10 The effect of location of point light source.
shown in Figure 2.11. The figure shows two cases of beam expanding. The first one uses two convex lenses and the other one uses one concave and one convex lens. In either case, the lenses have a common focal point. Depending on the focal length of the lenses, the beam diameter at the output is made different. By similar triangles, it is seen that D2 ¼
f2 D f1 1
ð2:15Þ
This implies that a desired output beam diameter D2 can be easily obtained by properly choosing those three variables.
L2
L1
L1 D1
D2
D1
D2 f1
(a)
f1
f2
L2
(b)
FIGURE 2.11 Transmissive beam expanding by combination of two lenses.
f2
42
Optomechatronics
An object is normally a collection of point sources. To find its image, it is desirable to locate the image point corresponding to each point of the object. However, it is sufficient to use a few chief rays in order to determine the object image. To illustrate this, ray diagrams for convex and concave lens chief rays are shown in Figure 2.12. It can be seen that the use of this diagram can facilitate the determination of the location, size, and orientation of an image produced by a lens. The size and location of an image is of particular interest, since they determine magnification. Following the sign convention, the transverse distance above the optical axis is considered positive, while the distance below the axis negative, as can be seen for the integrated image. It follows from the figure that the transverse magnification is expressed by MT ¼
hi s ¼2 i ho so
ð2:16Þ
where ho and hi are the transverse distance, height of the object, and that of its image, respectively. The minus sign for si accounts for an inverted real image as formed by a single thin lens. From Figure 2.12a we can see that in case of a convex lens, the sign and magnitude of the MT depend upon where
A ho
Fi Fo
B
A′
O
f
hi B′
f si
so (a) real image formed by a convex lens
A B′
ho B
Fo
so
A′
hi
O si
(b) virtual image formed by a concave lens FIGURE 2.12 Location and transverse lateral magnification of object image.
Fi
Fundamentals of Optics
43
an object to be image is located. When it is located within the range 1 . so . 2f , 21 , MT , 0: In case of a diverging lens, shown in Figure 2.12b, we can see MT . 0 for si , 0, since in this case all images are virtual and erect. They are always smaller than the object, and lie closer to the lens than the object. The discussed characteristics of the images of real objects are summarized in Table 2.2. When the object is located in front of a convex lens in the range 1 . so . 2f , MT becomes 21 , MT , 0: When it is located so ¼ 2f , it becomes MT ¼ 21: When it is located in the range f , so , 2f , the image is real and inverted and MT becomes magnified MT , 21: Let us take some examples for thin lenses. When an object is located 6 cm in front of a convex lens having focal length 10 cm, the location and magnitude of the image to be formed can be calculated by using the formulas in Equation 2.14 and Equation 2.16, respectively. The image location is obtained by si ¼
6 £ 10 ¼ 215 6 2 10
which indicates that the image is virtual and lies to the left of the lens. The magnification of the image is obtained by MT ¼ 2
si ð215Þ ¼2 ¼ 2:5 so 6
which indicates that the image is erect. In a similar way, the location and size of the image of an object located 10 cm in front of a concave lens of focal length 6 cm can be obtained as follows: location image size
si ¼
10 £ ð26Þ 260 ¼ 10 2 ð26Þ 16
MT ¼ 2
si ð260=16Þ 3 ¼2 ¼ so 10 8
The above simple calculations show that we can vary magnification of an image formed by a single lens by changing its distance from an object. If the lens moves toward the object, by d lying within a range, the image will become larger, but if shifted away from the object, it will become TABLE 2.2 Image Characteristics of Real Objects Lens Convex
Concave
Object Location 1 . so . 2f so ¼ 2f f , so , 2f so , f Any
Image Characteristics
Remarks
Real inverted 21 , MT , 0 Real inverted MT ¼ 21 Real inverted MT , 21 Virtual erect MT . 1 Virtual erect 0 , MT , 1
si . 0 si . 0 si . 0 si , 0 si , 0
44
Optomechatronics
h f
f
h' f < s o < 2 f so - d
d
si
h h'
f
f so = 2f
d
h' s o = 2f
si
f
h
h'
f so + d
2f < s o <∞ si
FIGURE 2.13 Change of magnification and image plane by using a single lens.
smaller. This relation is illustrated in Figure 2.13. An object whose size is indicated by an arrow is located 2f away from the lens; f is the focal length of the lens. If the lens is shifted toward or away from the object by d, the image location si and magnification, M, can be determined by the lens equation as; shifted toward; si ¼
f ð2f 2 dÞ f ; M¼2 f 2d f 2d
shifted away from; si ¼
f ð2f þ dÞ f ; M¼2 f þd f þd
It is noticed that, as the lens moves closer to or away from the object, the image plane becomes further distant away or closer to from the object. So far we have discussed only a single thin lens. In many cases, a train of lenses is used to obtain a desired image of an object by modifying the ray path, and correcting the distortion or aberration magnitude of the image and
Fundamentals of Optics
45
object A
B F1
•
so1
F2
f1
•
L1
L2
O1
• O 2 F1
f2 d
si2 si1
RI
• F2
B' virtual object for lens 2 A'
so2
FIGURE 2.14 Formation of a real image by two convex lenses.
so on. To analyze the optical system involved with this, it is desirable to have a fundamental understanding of a two-lens combination first. To this end, we will now derive the important parameters of the combination such as an equivalent focal length, the location of the image, and the transverse magnification. Let us consider two thin convex lenses having different focal lengths f1 and f2 , which are separated by distance d, as shown in Figure 2.14. Referring to the figure, we can determine the real image (RI) formed by two convex lenses. If no consideration is given to lens L2 the intermediate image, ! ! which is the real image ðA0 B0 Þ of an object ðAB Þ located far left, is formed by lens L1 and can be obtained by the lens formula given in Equation 2.14. si1 ¼
so1 f1 so1 2 f1
ð2:17Þ
The image is real and magnified, since so1 . f1 and f1 . 0 as summarized in Table 2.2. This real image becomes a virtual object for lens L2, since the object is located to the right of the lens. The virtual object is located at so2 ¼ d 2 si1
ð2:18Þ
Utilizing the lens formula for L2 to obtain the image of the object and combining the resulting equation with Equation 2.17 and Equation 2.18, we have si2 ¼
so2 f2 ðso2 2 f2 Þ
ð2:19Þ
This result indicates that the distance of the image of the compound lens can be determined for a given set of parameters such as focal lengths of two lenses, separation distance of the two lenses and the distance of an object that
46
Optomechatronics
are given a priori. The size and orientation of the image can be obtained by total transverse magnification of the compound lens, which is the product of the magnification made by L1 and the magnification by L2; MT ¼ MT1 ·MT2
ð2:20Þ
Upon the use of Equation 2.16, MT becomes MT ¼
f1 si2 dðso1 2 f1 Þ 2 so1 f1
ð2:21Þ
Consider a two convex lens compound system has f1 ¼ 20 cm, f2 ¼ 20 cm, and their separation d ¼ 40 cm. We wish to compute the image of an object located 30 cm from the first lens L1. Applying the lens formula L1 si1 ¼
30 £ 20 ¼ 60 cm 30 2 20
MT1 ¼ 2
60 ¼ 22 30
The intermediate image is real, 60 cm to the right of the first lens, and inverted. The final image formed by the second lens L2 can be obtained by using Equation 2.19. Since so2 ¼ d 2 si1 ¼ 30 2 60 ¼ 230, we have si2 ¼
so2 f2 ð230Þ £ 20 ¼ ¼ 12 so2 2 f2 ð230Þ 2 20
MT2 ¼ 2
12 ¼ þ0:4 ð230Þ
The size and orientation of this image can be obtained by multiplying MT1 and MT2 :Thus MT ¼ MT1 MT2 ¼ ð22Þ £ 0:4 ¼ 20:8 which indicates that the image is reduced and inverted.
Multiple Lenses and System Matrices In general, lenses are classified into two classes depending on whether they make a parallel bundle of light convergent or divergent, as shown in Figure 2.15. In each class, three types of lenses are illustrated. A converging (also called positive) lens is thicker in the center than the periphery, while a diverging (negative) lens is thinner in the center. A lens has two centers of
Fundamentals of Optics biconvex
plano-convex
(a) convex / positive lenses
47 positive meniscus
biconcave plano-concave negative meniscus
(b) concave / negative lenses
FIGURE 2.15 Various types of lenses.
curvatures, the two having sometimes two different radii. For example, one of the surfaces is plane, and the other has a finite radius, as shown in the case of plano-convex or plano-concave. There is no numerical limit where a thin lens and a thick lens are differentiable. In other words, whether a lens is thin or thick depends on the accuracy required for solving a given problem. For instance, a lens needs to be treated as thick, when its thickness along its optical axis cannot be ignored without leading to reviewed errors in analysis. There are many cases in which several lenses must be put together to constitute a compound lens that shapes the optical ray in a desired fashion. For example, a photographic lens consists of five or six lenses for focusing zooming filters, and so on. In this situation, the analytical matrix method is an efficient and effective approach that facilitates the ray analysis. Ray tracing is undoubtedly an effective tool when a designer wishes to design or modify an optical system according to a given specification. However, when an optical system has multiple lenses composed of various types, it is not at all easy to trace some relevant rays through the entire system to compute their ray paths. Let us consider a system of two convex lenses shown in Figure 2.16a, and derive the equivalent focal length of the system. Assume that a ray parallel to the optical axis is incident on lens 1 at a height h1 : The ray after lens 1 enters lens 2 at height h2 : Finally, the ray is focused at a resultant focal point feq : When a ray of light incident upon lens 1 is paralleled to the optical axis at a height h1 as shown in Figure 2.16a, the angle u1 between the refracted ray and the horizontal line parallel to the optical axis is given by
u1 ¼
h1 ¼ p1 h1 f1
ð2:22Þ
if u1 is assumed to be a small angle. In the above, p1 is called the optical power of the lens L1 and represented by 1=f1 : Likewise, when a ray of light parallel to the optical axis is incident upon lens L2 at height h2 as shown in
48
Optomechatronics lens L2
lens L1
h1
h2
θ1
θ2
f1
(a)
f2
(b)
L1
L2
H2
h1
(c)
θ1 h2
θ0∗
F2 feq
d
FIGURE 2.16 Equivalent focal length of a two-lens system.
Figure 2.16b, the angle u2 is given by
u2 ¼
h2 ¼ p2 h2 f2
ð2:23Þ
where p2 is the power of lens 2 and represented by 1=f2 : Let us return to the system of two lenses. After the ray is refracted in lens L1, it is refracted again in lens L2. Behind the second lens, the slope angle up0 is given by
up0 ¼ u1 þ u2 Therefore, the angle up0 can be obtained as follows:
up0 ¼ p1 h1 þ p2 h2
ð2:24Þ
With these results, we then consider two lenses together as a whole. If the focal length of the combined system is denoted by feq which can be measured from a principle plane H2 shown in the figure, angle up0 is expressed by
up0 ¼
h1 ¼ peq h1 feq
ð2:25Þ
Fundamentals of Optics
49
where peq is the equivalent power of the system. From Figure 2.16c, the angle u1 is obtained by geometry h1 2 h2 ¼ p1 h1 d
u1 ¼
ð2:26Þ
where d is the distance between the two lenses. From Equation 2.24 and Equation 2.26 we have peq ¼ p1 þ p2 2 p1 p2 d
ð2:27Þ
Since power peq is 1=feq , the above equation becomes feq ¼
f1 f2 f1 þ f2 2 d
ð2:28Þ
This feq is called the equivalent focal length of the system. We can see that we will have different equivalent focal lengths depending on the distance d. If two lenses are in contact, that is d ¼ 0, then the above result is given by feq ¼
f1 f2 f1 þ f2
ð2:29Þ
Let us take an example of determining the equivalent focal length of a twolens system. The front convex lens has focal length of 50 mm, and the rear concave lens a focal length of 25 mm. The lenses are separated in distance by 30 mm and then moved to another position, 40 mm. To determine the equivalent focal length, we use Equation 2.28, feq ¼
f1 f2 50 £ ð225Þ ¼ ¼ þ250 mm f1 þ f2 2 d 50 2 25 2 30
We consider another case when two positive lenses, L1 and L2, having an identical focal length, f are separated from each other with distance case (1) 14 f and pulled away from each other with distance case (2) f =2: We compute the feq , or the approximate power of the two cases and determine which one is the wider-angle lens. The use of Equation 2.28 yields: for case (1) d¼
f 4
feq ¼
f1 f2 4 ¼ f f1 þ f2 2 d 7
and for case (2) d¼
1 f 2
feq ¼
f1 f2 2 ¼ f f1 þ f2 2 d 3
50
Optomechatronics
Because case (1) has shorter focal length, the system separated 14 f apart is the wider-angle lens. The focal length in the above is the distance between the second principal plane H2 and the focal point F2 as shown in the figure. It is noted that this length is considerably much longer than the actual distance between the first lens to the second. The System Matrices According to the matrix method, as long as paraxial approximation is made, changes in height and direction of ray can be expressed by linear equations. This makes it easy to use the matrix method to specify the properties of the compound optical system. This indicates that, given the position and size of an object, this approach would produce those of the image with a small computational effort. Let us consider a single ray that starts from P0 through P1 : Three frequently used matrices involved with progress of the ray are the translation, refraction, and reflection matrices. Figure 2.17a shows the ray translating from point P0 to P1 by ‘ from the elevation y1 making slope angle a1 relative to the optical axis. For the paraxial ray condition tan a0 < a0 , we have
P1
a0
P0
a1
y1
y0
(a) translation
a'
P
a'
qi a s
θr qt
y' V
P
R j j
R
C
C
a
y' V
n1 n2
(b) refraction at a spherical surface : convex FIGURE 2.17 Forming a matrix for ray tracing.
(c) reflection at a spherical surface : concave
Fundamentals of Optics
51
the following matrix equation representing translation "
#
y1
" ¼
a1
1
‘
0
1
#"
y0
# ð2:30Þ
a0
This represents the location of the ray when traveling from point ðy0 ; a0 Þ through point P1 in the homogeneous medium ðy1 ; a1 Þ: The refraction matrix can be derived when we consider the refraction of a ray traveling in a medium of refractive index n1 at a spherical surface of refractive index n2 as depicted in Figure 2.17b. Since refraction occurs at a point P, the elevation condition yields y0 ¼ y
ð2:31Þ
From the geometry, a 0 is given by
a0 ¼ u t 2
y R
ð2:32Þ
Snell’s law gives n1 ui ¼ n2 ut
ð2:33Þ
Combining Equation 2.31, Equation 2.32, and Equation 2.33 yields
a0 ¼
1 R
n1 n1 21 yþ a n2 n2
ð2:34Þ
Finally, the matrix relating the ray at ða; yÞ to the same ray at ða0 ; y0 Þ is obtained by "
y0
a
0
#
2
1
6 ¼ 4 1 n1 21 R n2
3 0 "y# 7 n1 5 a n2
ð2:35Þ
It is noted that surface, R is negative and y ¼ y 0 , then for a plane interface, R becomes infinity. The reflection matrix can be derived in the same way as for the case of the refraction. Referring to Figure 2.17c, angle a 0 is given by
a0 ¼ u r þ
y R
52
Optomechatronics
It is noted here that by the sign convention R is minus for this concave lens. This leads to
a0 ¼ a þ
2y R
When the law of reflection at point P is applied, the above equation yields the reflective matrix; "
y0
#
a0
2
1 ¼4 2 R
3 0 "y# 5 1 a
ð2:36Þ
Even when an optical system consists of a series of lenses of several different kinds, these three matrix equations can be effectively used to track some major rays by this method. However, this analysis method is formidable to use if there are a number of rays of interest and a number of lens sets in the system. Computer Ray Tracing Computer ray tracing is nowadays a common practice to handle this kind of formidable job. Here, we will treat a very simple ray tracing problem to illustrate the effectiveness of the method. The system to be dealt with is to investigate the effect of variation of the positions of the zooming lenses on the radius of the output beam for a telephoto system as shown in Figure 2.18a. For numerical ray tracing, a computer software developed by the Optical Research Associates was used [8]. The system is composed of three lenses having initially 5 £ magnification, in which a collimated beam of radius
L1 L2
position 3
position 1
(a)
4
output beam
3 2 1
input beam
0 1
2
3 position
4
5
FIGURE 2.18 Beam expanding via three-lens configuration.
distance(d), mm
beam radius, mm (b)
position 5
60
5
−1
5 mm
3.75 mm
2.5 mm
L3
50 30 20 distance : L2-L1
10 0
(c)
distance : L3-L2
40
1
2
3 4 position
5
Fundamentals of Optics
53
0.5 mm is incident at a concave lens. A variety of beam expansion can be obtained when the location of two lenses (L2, L3) while fixing the position of lens L1 is changed. As can be seen from Figure 2.18b, the location change causes the refracted beam to leave the third lens with a beam of radius 2.5 mm shown in Figure 2.18c. Increasing further the separation between L2 and L3 from this point will yield larger radius of the output beam to 5 mm, that is, 10 £ magnification. The corresponding relative distance between L1 and L2 and that between L2 and L3 are plotted in Figure 2.18c.
Aperture Stops and Pupils Aperture plays an important role in optical systems. The aperture modifies the effects of spherical aberration and distortion, and controls the brightness of image. The aperture stop is defined to be the aperture, which limits the maximum amount of light that can be collected. In other words, it provides a measure of the light collecting ability. If an optical system consists of a single lens, the aperture stop is the lens itself. In a typical camera, the iris diaphragm is the aperture stop. In a telescope, the first lens or objective lens is the aperture stop, since it determines the amount of light needed to form a final image. Another important concept is the pupil that determines whether a given particular ray will pass through the entire optical system. The entrance pupil is the image of the aperture stop as seen from object space. In other words, it is the image of the stop through which we look into the entire optical system from the object. If we look into the optical system from the image space, another image of the aperture stop called the exit pupil can be seen to limit the rays forming the image: The object space is a space that contains all physical objects to the left of the lens, while the image space, the space to the right of the last optical element. The location and size of these three openings, the aperture stop, the entrance, and exit pupils have significant effect on the light collecting and also on certain types of aberrations. Therefore, it is often necessary to shift the stop along the optical axis in order to improve performance of optical systems. Figure 2.19 illustrates the limitation of rays for various arrangements of convex lens and aperture stop. Figure 2.19a shows the case when the aperture stop (AS) is placed in front of the lens. It can be seen that the stop indeed determines the amount of incoming light, the maximum of which is limited by the marginal ray passing through the margin of the stop. In this case, the stop serves as an entrance pupil, and the exit pupil is real and located behind the lens. Another ray depicted in the figure is chief or principal ray, which is defined to be any ray from an off-axis object point that passes through the center of the aperture stop. When the optical system is free from aberrations, the ray will also traverse the center of the entrance pupil and leave the center of the exit pupil. Figure 2.19b illustrates the system where the exit pupil is the image of the aperture stop itself and the entrance pupil is the image of the aperture.
54
Optomechatronics (AS) aperture stop marginal ray
f ray
chie
F
chie
f ray
(a)
exit pupil
entrance pupil
AS
chief ray F
chie
f ray
(b)
exit pupil
entrance pupil
FIGURE 2.19 Effects of aperture stop on formation of entrance and exit pupils.
In visual instruments, the observer’s eye is positioned at the center of the exit pupil. In this particular case, the pupil of the system itself is known to vary roughly within the range 2 – 8 mm, depending on the illumination condition. Therefore, the pupil of the instrument must fit into this eye condition. The amount of light admitted through the aperture is very useful in defining two useful parameters, f -number f =#, and numerical aperture, NA. If aperture diameter is denoted by D and focal length of an imaging lens is denoted by f, then the light power per unit area (irradiance) at the image plane is proportional to !2 D Ir / f since the image area is proportional to f : From this quantity, we can define f =# and NA. The quantity f =D is called focal ratio or relative aperture and defined by f -number and thus f =# ¼
f D
If a lens has a focal length, 4.0 cm, and an aperture diameter 1.0 cm, f -number is 4: If for the same lens only aperture is decreased to 0.5 cm, f -number is 8. This indicates that the amount of light received at the image plane increases with smaller f =#:
Fundamentals of Optics
55
The numerical aperture (NA) is defined by NA ¼ nt sin u where n is the refraction index of the medium in which the lens lies, and u is the half angle of the cone of light leaving towards the focusing area. When the lens is used in the air ðn ¼ 1Þ, we notice that the maximum value is unity. When u is small, NA is approximated by NA . tan u ¼ nt
D=2 f
It can be seen that NA represents the amount of the light power gathered in the image plane by the lens. NA is proportional to aperture diameter and inversely proportional to focal length. If the lens is in the air ðnt ¼ 1Þ and the object is located in the far field, f =# and NA are related by f =# ¼
1 2NA
Numerical aperture and f -number are two different methods of defining the same illumination characteristics of an optical system. The f -number is more conveniently applied to the case particularly when the object is in far field, for instance, camera lenses and telescope objectives. The numerical aperture is useful for systems that work at finite conjugates ðso and si finite) in such a case as microscope objectives. We will return to these parameters in Chapters 6 and 7.
Aberration In the previous section, all rays participating in forming an image are assumed paraxial, that is, rays near to the optical axis or making a small angle with it. This would be a good approximation only for the first-order theory for ray tracing. An exact ray tracing in actual case would cause departures from the predictions with the paraxial description. For example, it will no longer be generally true that all the rays leaving a point object will meet an image, as we shall see later. Such deviations from ideal Gaussian behavior are referred to as aberrations. For monochromatic aberrations, there are spherical aberration, coma, and curvature of field, astigmatism, and distortion that deteriorate the image quality. Spherical aberration, astigmatism, and coma make the image unclear, while field curvature and distortion deform the image. There is another important aberration resulting from
56
Optomechatronics
wavelength dependency of the imaging components of optical systems, but we will not treat this chromatic aberration. Spherical Aberration Spherical aberration is one of the most common aberrations in lenses. If all rays emanating from a point source S, not exclusive of rays reflected by a larger lens aperture are assumed to impinge on a convex lens, the image formed can be represented as the point P due to paraxial rays and the point P0 due to nonparaxial rays. This is shown in Figure 2.20. The axial distance between points P and P0 , PP0 is known as the longitudinal spherical aberration. In this case, P0 is to the left of the paraxial focus P and the aberration is positive. In contrast, when P0 is to the right of P, the aberration is considered to be negative. The height above the axis PT is called the transverse spherical aberration. The light rays incident in the intermediate zone of the lens are imaged between point P and P0 : Therefore, there exists no clear sharp image behind the lens because aberrations produce image degradation. If a screen labeled II is inserted between PP0 , we find that the image is composed of a circular disk image constituting the cones of light rays produced from various overlapped axial image points. At some intermediate point between P and P0 we find the best focus or the circle of least confusion having the smallest circular disk. This is generally the best place to observe the image. The amount of spherical aberration for fixed focal length and aperture is influenced by both the object distance and the lens shape. This aberration of a lens can be reduced by inserting an aperture stop to block the light rays near the rim or by designing the surface curvatures of lenses properly. Coma Aberration Coma refers to the variation of magnification with aperture. When a bundle of rays is incident obliquely on a lens as shown in Figure 2.21, the rays
plane of the circle of least confusion j S
a j P
P′ T
longitudinal aberration FIGURE 2.20 Spherical aberration.
Fundamentals of Optics
57
lens
A
R1
hc
Rc R2
hp
B
image plane
FIGURE 2.21 Coma.
passing through the outer zones of the lens form a larger image than the rays through the center portion. If the rays through the outer zone denoted by R1 and R2 make an image at the intersection point A in the image plane, the coma of the lens, hcom , is defined by hcom ¼ hp 2 hc where hp is the height of the image produced by the intersection of the rays R1 and R2 and hc is the height of the image produced by the ray passing through the center of lens. The image has a form which is something like the shape of a comet with its tail, therefore this aberration is named comet. Field Curvature Curvature of field is the aberration of a lens that cannot transform a plane object into a precisely plane image. In other words, object points lying in a plane are imaged onto a curved surface when field curvature is present. As shown in Figure 2.22, a plane object is imaged into a curved surface IIP, on which the object is imaged approximately as a plane only in the paraxial region. This curvature of field is known as Petzval field curvature. The Petzval surface for a convex lens is directed inward toward the object plane, whereas for a concave lens it is directed outward, away from that plane. This surface depends on the focal length of the individual lens components and on their refractive indices, as we might expect. Astigmatism Astigmatism occurs when an object point lies far distant from the optical axis and, therefore, the incident cone of rays strikes the lens asymmetrically. This aberration occurs mainly because off-axis object rays are in different axial sections of the lens, resulting in two slightly different focuses; tangential (meridional) and sagittal (radial). Let us consider Figure 2.23 to see how this
58
Optomechatronics
B′ .
lens
B
A′
A
i
o
object plane
P
image plane
FIGURE 2.22 Appearance of both the astigmatism and the field curvature.
secondary image circle of least confusion
lens chief ray
A optical axis
C
primary image Ft sagittal plane
Fs meridional plane
D object point B FIGURE 2.23 Meridional and sagittal focuses of an oblique light ray: astigmatism.
arises. In the case of an axial object point, the cone of rays is symmetrical with respect to the spherical surface of a lens, arriving at a single focus in the image focal plane. However, when an incident cone of rays strikes obliquely, there occur two distinct focal lengths: one in the meridional plane and the other in the sagittial plane. These two planes are perpendicular to each other, meridional plane containing both the chief ray and the optical axis, and sagittal plane, containing the chief ray, is determined by the plane perpendicular to the meridional plane. As the beam enters the lens, its cross section is initially circular. However, as the beam proceeds to propagate, it becomes gradually elliptical, whose major axis lies in the sagittal plane. At the meridional focus Ft , known as the “primary image” the ellipse completely deforms into a line. Beyond this point, in a particular position, the image becomes circular with blur known as the circle of least
Fundamentals of Optics
59
confusion, where the major and minor axes of the ellipse are equal. When the beam moves farther from this point, its cross section again becomes gradually elliptical whose major axis this time lies in the meridional plane. At the sagittal focus Fs the ellipse deforms into a line in the meriodional plane called the “secondary image.” Distortion Distortion occurs because focal length and lateral magnification more or less vary with different distance from the optical axis. In this case, the lateral magnification Mt may be a function of the distance from the optical axis. This phenomenon can be explained by using Figure 2.24. Let us suppose that a small object ho lies in the axial region of a lens. Its image hi will then appear behind the lens. If the object is shifted upward farther from the optical axis as indicated in the figure, the image hi will be dependent upon where the object is placed in the vertical direction. The Mt will be changed according to the following formula; DMt ¼
M0t 2 Mt h 0 2 hi ¼ i hi Mt
ð2:37Þ
where DMt is the change in lateral magnification, Mt and M0t represent the lateral magnification of the object at two locations, and hi and h0i are the corresponding images at the same locations. If the lateral modification increases with the distance from the axis, the distortion becomes positive, DMt . 0: By contrast, if it decreases with the off-axis distance, the distortion becomes DMt , 0: This phenomenon is well illustrated with a square grid as indicated in Figure 2.25. Pincushion distortion as shown in Figure 2.25b is the case for DMt . 0, positive. This causes each image point outward from the center. On the other hand, as shown in Figure 2.25c, barrel shape distortion
FIGURE 2.24 Change in magnification with the off-axis image plane.
60
Optomechatronics
(a) undistorted
(b) pincushion distortion
(c) barrel distortion
FIGURE 2.25 Distortion due to monochromatic aberrations.
(a) barrel distortion
(b) pincushion distortion FIGURE 2.26 The effect of aperture location on image formation.
occurs for DMt , 0: This causes each image point to move inward toward the center. When measurement of length, width, and dimension requires precision accuracy, DMt needs to be kept typically less than DMt , ^1%: Figure 2.26 illustrates the effect of the location of the aperture stop in distortion. When the aperture stop is inserted between the object and the lens, as shown in Figure 2.26a, barrel distortion occurs. As the aperture stop gets closer to the lens, the barrel distortion is less pronounced than that of the original distance. If the aperture stop is inserted behind the lens, as shown in Figure 2.26b, pincushion distortion occurs. It can be shown that the distortions
Fundamentals of Optics
61
mentioned in the above can be reduced if we add another identical lens behind the aperture stop (see problem P2.10).
Polarization A light wave is oscillating as a transverse electromagnetic wave. As mentioned in the beginning of this chapter, the directions of the electric and magnetic field are always perpendicular to one another. Only the direction of the electric field will suffice to specify the direction of a light wave. Polarization of the wave is a process of characterizing the orientation of such wave oscillation in both time and space. This concept is widely utilized to the design of optical systems, for example, for the control of the amount and direction of light. Before getting into the subject of polarization in detail, let us consider some examples exhibiting such phenomena. Polarized light can be observed from the reflection process. Let us consider when ordinary unpolarized light is reflected from dielectric media like glass, water, plastic and so on, as shown in Figure 2.27. If the light is incident with up , other than 908 there will always be reflected light and refracted light. The characteristics of the reflected light indicate that the light is partially plane polarized in parallel to the plane of incidence as shown in Figure 2.27a. The degree of polarization depends on the incident angle of the incoming light. According to Brewster (1781 – 1868), only at a certain polarization angle that satisfies
up þ ur ¼ 908 unpolarized incident light
partially polarized reflected light
qp qp
unpolarized incident light
fully polarized reflected light
qp
qp interface
interface
p
p
qr
refracted light
(a) partial polarization FIGURE 2.27 Plane polarization by reflection.
qr
refracted light
(b) maximum polarization
62
Optomechatronics
will the reflected light be plane polarized, exhibiting the maximum amount of polarization at this angle, which is shown in Figure 2.27b. From Snell’s law, we can express this angle up in terms of the index of refraction n ð2:38Þ tan up ¼ t ni where ni is the index of refraction of incident medium and nt is that of the transmitting medium. For example, when the light is incident to the surface of a lake, reflection occurs with polarized light at an angle tan up ¼ 1:33 which gives up ¼ 53:18. When a polaroid sheet is placed in front of a propagation wave, only the component of light waves parallel to the transmission axis will pass through the sheet. This is because the device has the property that it transmits light waves in a specific direction without transmitting waves in other directions. The device exhibiting this behavior is called a polarizer. In order to investigate some details of this polarization phenomenon, let us consider when a light wave is oscillating, while remaining within a plane, which is called a “polarized plane.” Figure 2.28 depicts two cases showing y
Ey
E x
Ex
E
Ey
Ex
(a) linearly polarized
z propagation direction
ϕ =180°, 360°, Ex=Ey=const
y
x E Ey Ex z propagation direction (b) circularly polarized
ϕ =90°, 270°, Ex=Ey=const
FIGURE 2.28 Two perpendicular electromagnetic wave oscillations.
Fundamentals of Optics
63
such behavior: The elliptically polarized pattern, which is the most general case, is not shown here. The first figure shows the case when a wave is linearly polarized. The orientation of the electric field E is referenced to some axis perpendicular to the direction of propagation, and does not change with time as E oscillates from a maximum value in one direction to a maximum in the opposite direction. The electric wave oscillates in the plane. Since the wave is a vector quantity, it has amplitude E and direction at any instant of time which are given by qffiffiffiffiffiffiffiffiffiffi E ¼ E2x þ E2y ð2:39Þ Each polarized vector is indicated in one quadrant plane shown as a dotted line. The phase difference between Ex and Ey is p or 2p. The second figure shows circular polarization whose constituent waves have equal amplitude ðEx ¼ Ey Þ and whose phase difference is p=2 or 3p=2: Figure 2.29 illustrates two different states of polarization of light waves. The first figure denotes the case when unpolarized light encounters a polarizer whose transmission axis is in a vertical direction. Therefore, the output light will be polarized in a vertical direction. The second one shows addition of another polarizer whose transmission axis is horizontal. In this polarizer polarizer
.
.
(a) parallel orientation
.
(b) crossed orientation
incident light y
E θ
polarizer Ey = E cosq
x
z (c) Malus's law FIGURE 2.29 Dichroic crystals in two different positions and Malus’s law.
64
Optomechatronics
case, no light is transmitted through the analyzer. The third figure shows when light is polarized with angle u when the vertical transmission axis passes through a polarizer. The polarizer will transmit only the vertical component of the electric field vector. Since light intensity is proportional to the square of amplitude, the irradiance of the transmitted light is obtained by I ¼ I0 cos2 u
ð2:40Þ
where I0 is the maximum intensity of the transmitted wave. This indicates that, when u ¼ 458, the transmitted irradiance becomes 12 I0 , which is half the irradiance of the initial light. This is known as Malus0 s law.
Coherence As to the phenomena of interference and diffraction to be discussed later, we will be concerned with what happens when two or more light waves superimpose each other in some region of space. The phenomena will occur differently depending upon how strongly they are correlated in both time and space. That is to say, when they are radiating perfectly in phase with each other, they will produce a strong response due to the addition of their individual amplitude. On the contrary, when they are not correlated, randomly in phase, their response will become very weak. In reality, these two extremes are not ideally attainable. This means that there are no such cases having a correlation value (coherence function) of either one or zero. Some currently-used laser beams can be approximated by a monochromatic wave having the value slightly less than 1 only if a certain condition is assumed. Partial correlation occurs when they are radiating with constant phase relationships. The coherence therefore is used to describe the correlation between phases of such radiating light waves. The coherence may be conveniently divided into two classifications; temporal and spatial. Temporal coherence refers to correlation in phase at the same location in space at a different time, while spatial coherence refers to correlation in phase at the same time at some different point in radiation space. Michelson interference is a typical case of exhibiting temporal coherence induced by path difference, while Young’s experiment using double slits is a typical case of exhibiting the spatial coherence caused by separation of two slits. The degree of coherence largely depends upon light path difference and the size of the light source. The temporal coherence is characterized by coherent time Dtc , which is the time interval, that is, time coherence length, during which the wave is coherent. If a light source has a long coherent time, Dtc , it has a high degree of temporal coherence. As mentioned before, the spatial coherence is the correlation in phase between some points in the radiation field. Its effect arises when the light’s sources are extended within a finite spatial domain. It is characterized by spatial coherence length D ‘a over which the waves are coherent. Both coherence values, Dtc and D ‘a can be
Fundamentals of Optics
65
derived mathematically by considering the temporal correlation function and spatial interference between the interacting waves.
Interference Interference can be found from several natural phenomena such as soap bubbles or in oil films on wet pavement and so on. Interference occurs when two or more monochromatic light sources are superimposed under some conditions to be discussed later. In a simple configuration of producing interference, a single light source may be used to make light wave travel two different paths and then recombine. Interference between two such waves can be demonstrated where a pinhole point source is places near a plane mirror. This is called Lloyd’s single mirror arrangement. Under this arrangement, there are two light sources: One source is S and the other source is a virtual source S0 from which the reflected light appears to propagate in space. At a point in the region of a screen, the result of the interference between the wave from S and the wave S0 (see Problem P2.17), with the phase change that occurs upon reflection, will be observed. To consider a more general case of interference, let us consider two point sources S1 and S2 emitting monochromatic waves E1 and E2 in a homogeneous medium, as shown in Figure 2.30. For simplicity, we will assume that two waves have the same frequency, v and wavelength l: Let the two waves be represented by E1 ðr1 ; tÞ ¼ E01 cosðk1 r 1 2 vt þ w1 Þ
ð2:41Þ
E2 ðr2 ; tÞ ¼ E02 cosðk2 r 2 2 vt þ w2 Þ where E01 and E02 are the wave amplitude generated by S1 and that of S2, k1 and k2 are the wave numbers of the two waves, and r 1 and r 2 are spatial coordinates in space, respectively. Let the source separation w be much larger than l: According to the principle of superimposition, at an arbitrary point P in space, the two waves will intersect to produce the electric field intensity EP : EP ¼ E1 þ E2 Then the irradiance, I, can be defined by I ¼ 10 ckE2P l
ð2:42Þ
where 10 is the permittivity of free space (vacuum) and c is the speed of light, and kl implies time average of the quantity within the bracket. The irradiance is expressed by using Equation 2.42 I ¼ 10 ckE21 þ E22 þ 2E1 E2 l
ð2:43Þ
66
Optomechatronics P
E1
r1 r2
S1 S2 E2
(a) interaction of waves from two point light source irradiance (I) Imax
Imin 0
∆ϕ
(b) irradiance of interference fringe v.s. phase difference FIGURE 2.30 Interference of two coherent waves.
Taking the time average of the quantity in the bracket, we have I ¼ I1 þ I2 þ I12
ð2:44Þ
where I1 ¼ kE21 l, I2 ¼ kE22 l, I12 ¼ 2kE1 E2 l: In the above equation, I12 is called the “interference term”. Substituting the two waves E1 and E2 given in Equation 2.41 into the above equations and carrying out their time averages, we have I1 ¼
10 c 2 E ; 2 01
I2 ¼
10 c 2 E ; 2 02
I12 ¼ 10 cE01 E02 cos Dw
where, Dw, is expressed by the phase angle difference Dw ¼ ðr 1 2 r 2 Þk þ w1 2 w2
ð2:45Þ
where k ¼ k1 ¼ k2 : In the above equation, the first term is due to the path difference while the second is due to the initial phase difference. Rewriting the irradiance at point P given in Equation 2.44 yields pffiffiffiffiffi I ¼ I1 þ I2 þ 2 I1 I2 cos Dw ð2:46Þ The above equation implies that the irradiance at an arbitrary point P in space is characterized by the path difference term, and the initial phase
Fundamentals of Optics
67
difference w1 2 w2 : If these values vary randomly, the sources S1 and S2 are mutually independent, i.e., incoherent as discussed previously. In this case, the time average of cos Dw becomes zero, which means that no steady state will be observed over a certain period of time. Therefore, cos Dw should not be zero and w1 2 w2 needs to be constant to make interference occur steadily over long period of time. Under this condition, the strength of interference depends on the path difference. In other words, it varies with the point given by r 1 2 r 2, Maximum irradiance occurs when cos Dw ¼ 1; pffiffiffiffiffi Imax ¼ I1 þ I2 þ 2 I1 I2 for Dw ¼ 2mp On the other hand, minimum irradiance occurs when cos Dw ¼ 21 pffiffiffiffiffi Imin ¼ I1 þ I2 2 2 I1 I2 for Dw ¼ ð2m þ 1Þp This relation is shown in Figure 2.30b. It is noticed that when I1 ¼ I2 ¼ I0 , then, Imax ¼ 4I0 and Imin ¼ 0: Young’s Experiment
S2
r2
q
d δ
z
s
Imin
S1
Imax
P r1
intensity (I)
The arrangement of Young’s experiment is schematically shown in Figure 2.31. Suppose that a plane light wave from a monochromatic source to the left of the figure is caused to pass through a narrow slit. According to Huygens’s principle, the propagation of a wave front can be constructed by considering each point on the wave front as a source of new spherical wavelets. Therefore S is regarded to be a new source of a spherical wave, and the spherical wave travels to two double slits S1 and S2. This generates two coherent secondary sources, which will spread out and, in turn, produce interference of two spherical waves. The figure shows a cross-sectional view of the wave fronts in a plane containing the sources S1 and S2. The solid lines
m = 2p m = 3p 2 m=p m= 1p 2
screen (a) schematic for double slit FIGURE 2.31 Double slit interference.
(b) irradiance v. s. distance from optical axis
68
Optomechatronics
indicate their crest, while the dotted lines indicate their valley. Therefore, the waves in solid lines will, when added together, give an intensity that is twice that of a single source. On the other hand, when one wave in the solid line is combined with the other in the dotted line, that is, two waves 1808out of phase are combined, the waves always cancel and therefore no light can be detected. Consider a point P on the screen and the intensity at that point. The intensity is obviously dependent on the distance S1 P and the distance S2 P: If S1 P ¼ S2 P, or S1 P differs from S2 P by an integral number of wave length, then the wave fronts will arrive at P in phase and will reinforce. From the figure, the path difference is given by
d ¼ r1 2 r2 ¼ d sin u
ð2:47Þ
where d is the distance between the slits. Depending upon the path difference d, the interference will be constructive or destructive: Constructive interference occurs when wave fronts add together to give a larger intensity value, while destructive interference occurs when wave fronts are out of phase. Equation 2.47 can be written as constructive;d ¼ S2 P 2 S1 P ¼ ml
1 l 2 This phenomenon is depicted in Figure 2.31b. Let us compute the position of the maxima in the direction of y-axis on the screen. Because u is small, from the trigonometric relationship, we have destructive; d ¼ S2 P 2 S1 P ¼ m þ
sin u . u .
y s
ð2:48Þ
where s is the distance from the slit to the screen and y is the distance from point P to the optical axis. Combining Equation 2.47 and Equation 2.48 yields y¼
s d d
This can be rewritten as s ym ¼ ml; m ¼ ^1; ^2… d
ð2:49Þ
which indicates that the maximum positions of fringes on the screen are determined by the distance between the slits, s, the spacing between two adjacent slit, d, light wave length l, and fringe order m, the spacing of the maximum intensity fringes can be obtained from Equation 2.49 ymþ1 2 ym ¼ Dy .
s l d
ð2:50Þ
where Dy is the distance between two adjacent fringe maxima. Large wavelength and screen distance causes large patterns while small spacing
Fundamentals of Optics
69
causes large fringe spacing. This equation will yield 2% or better for angles less than 208. Let us take example of calculating the location of the specific fringe order. For the given parameters, d ¼ 0:1 mm, s ¼ 1 m, and l ¼ 640 nm, the location of the first order fringe is calculated to be y^1 ¼
1 £ 103 £ 0:64 £ 1023 ¼ ^6:4 mm 1 £ 1021
Thus, the first peaks appear at 6.4 mm above and below the axis of m ¼ 0: Fabry-Perot Interferometer The Fabry-Perot interferometer, developed in 1899 by Fabry and Perot shortly after the Michelson interferometer, utilizes two parallel highly reflecting mirrors or a transparent plate with two reflecting surfaces within which an interference pattern occurs. This instrument is used for high-resolution spectrum analysis, high precision wave length measurement, and determination of such physical parameters as refractive indices. This interferometer principle is also widely used for optical fiber sensors to measure pressures or transparent films of fluid medium. The interferometer consists of two thick glass plates forming an air cavity between them. The interferometer configuration can be made by a film of transparent material, such as a coated layer on a glass substrate and metal oxide layer. The two intersurfaces are parallel; highly reflective surfaces, and polished flat to within 1/20 to 1/100 of a wavelength. To keep high reflectivity they are coated with either a metallic (silver, gold, aluminum) layer or dielectric material depending upon the wavelength of light used. The plate can be fixed or mechanically movable. When they are fixed, the instrument is called an elatron. Consider the arrangement of the interferometer as shown in Figure 2.32a. Suppose that the interferometer having two parallel plates with distance, d, is illuminated by a narrow, monochromatic beam from a broad source. For simplicity only a single ray emitted from the source is depicted. As shown in the figure, the ray which impinges on the left plate with an incident angle of ui is transmitted through the cavity, and then hits the right plate. It is then multiply reflected within the cavity. All these coherent beams will eventually form a spot at point P: Any other rays parallel to this will be brought together at the same point P, which implies that the single beam generates multiple coherent beams in the interferometer. At the interface, the incident beam refracts with an angle ut : The optical path difference between the beams reflected and refracted then becomes
d ¼ 2nc d cos ut where nc is the refraction index of the medium in the cavity. When the two plates are separated in the air, then the path difference d between two
70
Optomechatronics
glass plate
P
collecting lens
qt
screen
d qi S source (a) multiple wave interference Er4 Er3
Et4
Er2
Et3
Er1
Et2 Et1
Eo (b) reflection and transmission from two parallel mirror surfaces FIGURE 2.32 Fabry-Perot interferometer.
successive parallel beams becomes
d ¼ 2d cos ut
ð2:51Þ
This indicates that there will be a constructive interference yielding maximum in the amplitude of the transmitted beam and a destructive interference yielding minimum amplitude. Fringes arising from refraction will appear satisfying maximum amplitude :
d ¼ ml
maximum amplitude :
d¼ mþ
1 l 2
ð2:52Þ
Combining Equation 2.51 and Equation 2.52, we can write ml ¼ 2d cos ut
ð2:53Þ
for maximum amplitude. Because l is very small as compared to d for a given refraction angle ut , we will see that number of fringe m will be very large, usually on the order of 10,000 to 100,000, depending on the plate separation. This number will increase as the plate distance increases, as can be seen from the above equation. The phase difference Dw due to path difference is
Fundamentals of Optics
71
given by Dw ¼
2p 2d cos ut l
ð2:54Þ
As long as the beams satisfy the above angle condition, the transmitted beams overlap at point P. Intensity at point P transmitted through the interferometer can be calculated by considering the set of parallel-reflected waves E1 ; E2 ; …EN as depicted in Figure 2.32b. Let the amplitude of the initial wave incident on the first surface be E0 : In case of no energy absorption, the wave amplitude is decomposed into the amplitude of the reflected wave rr E0 and the amplitude of the transmitted wave rt E0 : The total amplitude of the transmitted waves arrived at point P, EP , becomes EP ¼ Et1 þ Et2 þ · · · þ EtN
ð2:55Þ
Taking into consideration of the phase difference with reference to the wave E1 , Equation 2.55 can be rewritten as 2iðN21ÞDw } EP ¼ r2t E0 {1 þ r2r e2i Dw þ r4r e2i2Dw þ · · · þ r2N r e
where rr and rt denote the reflection and transmission or refraction coefficients, respectively. Since lr2r e2i Dw l , 1, the above expression becomes EP ¼ E0
r2t 1 2 r2r e2i Dw
ð2:56Þ
The intensity at point P is obtained by multiplying Equation 2.56 by its complex conjugate EpP : The resulting equation becomes 1 2 2 rE 1 p 2 t 0 I P ¼ EP EP ¼ 2 ð1 þ r4r Þ 2 2r2r cos Dw
ð2:57Þ
If the intensity of the incoming wave is denoted by I0 ¼ 12 E20 , and no energy absorption condition is assumed, Equation 2.57 is rewritten in the following from I0
IP ¼ 1þ4
rr 1 2 r2r
2
sin2
Dw 2
ð2:58Þ
where the relation, r2t ¼ 1 2 r2r , is used. The intensity transmitted is therefore a function of the plate separation d, the incident angle ui , wave length l, and the reflection coefficient rr : As can be seen from Equation 2.58, intensity varies sharply with the reflection related factor in parenthesis called the
72
Optomechatronics 1
transmitted intensity (T)
rr = 0.3
rr=0.8 0
rr=0.9 2π phase difference (∆j)
0
4π
FIGURE 2.33 The transmitted intensity distribution.
coefficient of finesse F. In terms of this coefficient, IP is rewritten as T¼
IP ¼ I0
1 1 þ F sin2
Dw 2
ð2:59Þ
where T denotes the transmittance and F is expressed by F¼4
rr 1 2 r2r
2
The transmittance distribution function T is termed Airy function. The relationship between the transmission function and phase difference is plotted for various reflection coefficients in Figure 2.33. A maximum intensity in transmission occurs when Dw ¼ 2mp, while a minimum occurs at Dw ¼ ðm þ 1=2Þð2pÞ: From this figure, we can see that a measure of the sharpness of the fringes becomes large for increasing rr value. In the case of rr ¼ 0:9 the irradiance drops off very rapidly on either side of the maximum value, showing a very sharp fringe. It is noted that the important parameters affecting the Dw are the plate separation, d, and refraction angle ut : The Fabry-Perot interferometer can be used to separate two wavelength components that appear in the incident light. Michelson Interferometer Optical path difference provides the cause of interference between waves. A typical interferometer that operates on this principle is the Michelson
Fundamentals of Optics
73
F
movable M2
B
(2) (1) light source S
A
(3) D
beam splitter (4) E (BS) detector or viewer
mirror (fixed)
C
M1
FIGURE 2.34 Michelson interferometer.
interferometer developed in 1880. This device is the best known and most versatile interferometric device. The basic arrangement is shown in Figure 2.34. It is composed of a coherent light from the source, a film (slightly silver coated) beam splitter, and two retroreflecting mirrors, a fixed mirror M1 and a movable mirror M2. When a monochromatic light source is incident to the beam splitter (BS), it is then split into two beams of equal amplitude. The reflected beam 1 and transmitted beam 2 continue to the retroflector M1 and M2, respectively. After being reflected back, their directions are reversed when those return to point D. Therefore, beam 4 includes rays that travel different optical paths: Beam 1 traveled to the fixed M1 and is reflected back to D, whose path is ADCD, while beam 2 traveled to the movable M2 and is reflected back to D, whose path is ABAD. Now, let us suppose that M2 moves to a new position by ‘ from its initial position. In this case, the full path difference between the waves recombined at beam splitter will exhibit the wave interference phenomenon. The optical path difference d made by the two waves described by
d ¼ AFAD 2 ADCD ¼ 2‘ This indicates that if the mirror M2 is moved by ‘, the path difference d is given by two times of the displacement. In other worlds, if ‘ ¼ l=2 then d becomes l a beam wavelength. Since the amplitudes of the two waves are equal, the irradiance due to two beams creating the interference is given according to Equation 2.46 I ¼ 2I0 1 þ cos
4p‘ l
ð2:60Þ
74
Optomechatronics
where I0 is the source amplitude and l is the wavelength of the source. Bright fringes occur at 2mp ¼ ð4p‘Þ=l; m ¼ 0; ^1; ^2…, while dark fringes occur at ð2m þ 1Þp ¼ ð4p‘Þ=l, m ¼ 0; ^1; ^2…: As M2 (object to be measured) moves, its displacement can be measured simply by counting the number of maxima of light intensity obtained by the detector. Its velocity can also be measured by counting the number of maxima per unit time. Therefore, the output signal of the detector varies periodically with ‘, whose period is equal to l=2: This implies that, when a He –Ne laser is used as source and a displacement of an object is ‘ ¼ 2 mm, the Michelson interferometer would produce the number of fringes of the detector output signal; N¼
‘
l=2
¼
4 £ 1023 ¼ 6:3 £ 105 6:33 £ 1029
Diffraction During traveling, a light wave meets an obstacle such as a rectangular object placed in front and a small aperture on an opaque or a transparent screen, light spreading phenomenon occurs beyond the obstacle and as a result produces bright and dark fringes. This spreading of light in the neighborhood of such discontinuities is known as diffraction. Figure 2.35 illustrates the variation of the spreading phenomenon with increasing distance from a slit. In near field, the image pattern due to diffraction somewhat resembles the geometrical shape of obstacles, which is known as Fresnel diffraction. In contrast, in far field, the image pattern does considerably differ from the shape of obstacles due to its spread with traveling distance. This is called Fraunhoffer diffraction. We will deal with only Fraunhoffer diffraction in this section. Consider the arrangement shown in Figure 2.36 for observing Fraunhoffer diffraction. The slit (or aperture) shown in Figure 2.36a is coherently illuminated by means of a point source and a collimation lens, not shown here. A second lens is placed behind the slit and the plane of observation is the second focal place of it. The incident and diffracted wave fronts are therefore plane. The slit lies in the x 2 y plane and has width w and vertical length ‘ð‘ q lÞ: Due to a long slit little diffraction will occur in the direction of the vertical edge (y direction). Then, the problem to find the wave intensity at point P is treated as a one-dimensional problem due to point sources across the width of the slit w as shown in Figure 2.36b. We consider each segment of slit dx as a source and calculate the contributions of all sources by integrating over the entire slit width. Then total wave amplitude arriving at P on a screen located at distance r from a point source is given by EP ¼ Ew
ðw=2 2w=2
eiðkr2vtÞ dx r
ð2:61Þ
Fundamentals of Optics
75
image plane fringe
slit
S FIGURE 2.35 Light wave spread due to an obstacle.
x
xs
y
lens +
−
2
slit 2
2
x
θ
2
w
O
o −
2
θ
z
R P
(a) slit geometry
-
screen
δ
2 slit
r θ R
z
o xp f
P
(b) a schematic of a single-slit diffraction cross section (top view)
FIGURE 2.36 Fraunhoffer diffraction.
where Ew is the source strength per unit width. Now let us consider the optical path difference d between two waves starting at x ¼ 0 and x ¼ w=2: From the geometry it is given by
d ¼ x sin u
76
Optomechatronics
where u is the angle which the OP makes with the horizontal optical axis. Then, the optical path length from the interval dx to the point P is given by rðxÞ ¼ R þ x sin u
ð2:62Þ
since r almost does not change due to this path difference, i.e., r . R: By substituting Equation 2.62 into Equation 2.61, the resulting equation becomes ðw=2 E E EP ¼ w eiðkR2vtÞ eikx sin u dx ¼ w R R 2w=2
sin
kw sin u 2 kw sin u 2
eiðkR2vtÞ
ð2:63Þ
Letting b ¼ ðkw=2Þ sin u, Equation 2.63 can be rewritten as Ew R
EP ¼
sin b iðkR2vtÞ e b
Since the irradiance I is determined by the time average of E2P i.e., I ¼ 10 ckE2P l, then I can be obtained by the above equation, IðuÞ ¼ or IðuÞ ¼
1 0 c Ew 2 R
2
1 0 c Ew 2 R
2
2
sin b b
ð2:64Þ
sinc2 b
The IðuÞ is the irradiance of the light diffracted in a given direction defined by u. This also gives information on the intensity distribution of all light brought into the focal plane. It is noted that if we let the intensity at u ¼ 0 to be Ið0Þ, then the irradiance is simply expressed by IðuÞ ¼ Ið0Þsinc2 b where Ið0Þ ¼
1 0 c Ew 2 R
ð2:65Þ 2
:
The property of the irradiance pattern due to a single slit can be analyzed by examining sinc2 b function: The sinc2 b function becomes unity as b approaches zero, mathematically lim b!0
sin b !1 b
We can see that at this b value the irradiance becomes maximum as shown in Figure 2.38, that is,
b ¼ ðkw/2Þ sin u ¼ 0
Fundamentals of Optics
77 f(b) x = tan b
−3p
−2p
−p
0
p
2p
3p
x=b
b
FIGURE 2.37 Intersection points of the line x ¼ b and the curves x ¼ tan b:
The amplitude of the I(0) is observed to be proportional to the square of slit width w. The secondary maxima along with the above first maxima can be obtained by letting dI=db ¼ 0, which yields the following transcendental equation; tan b ¼ b The b values satisfying the above equation are indicated in Figure 2.37. It can be seen that the points satisfying this equation lie between the minimum values but are not located at the midpoints between these values. At these points, the secondary maxima have rapidly diminishing intensity as can be seen in Figure 2.38. Table 2.3 lists the relative intensities of the first three secondary maxima. The minimum values yielding IðuÞ ¼ 0 can be found from Equation 2.65. These occur at the points satisfying
b¼
kw sin u ¼ mp; 2
m ¼ ^1; ^2; … ^ n…
In terms of wavelength this can be rewritten as ml ¼ w sin u
ð2:66Þ
which can be clearly seen from the figure. The image drawn below the intensity curve reflects the intensity distribution along the screen axis xs in the focal plane.
78
Optomechatronics
1.0
0.7 0.5 0.3 0.1 b: sin q :
−3p
−2p
−p
0
p
−3l/w
−2l/w
−l/w
0
l/w
2p 2l/w
3p 3l/w
FIGURE 2.38 Irradiation pattern for a single-slit Fraunhoffer diffraction.
The image pattern consists of a central bright band and alternate bright and dark band on either side. The image within each band is very much blurred which is typical of diffraction of this kind. The angular width of the first maximum can be determined from Equation 2.66: 2sin Du . 2Du ¼
2l w
ð2:67Þ
for a given l. This result implies that the central maximum will spread as the slit width is narrowed and the wavelength gets large. The spread of the central maximum in the far field can be easily calculated from Equation 2.67.
TABLE 2.3 Relation Intensities of the Secondary Maxima of Diffraction Pattern of a Slit Maxima
b
Central max First max Second max Third max
0 1.43p 2.46p 3.47p
Relative Intensity 1 0.047 0.017 0.008
Fundamentals of Optics
79
Let us suppose that laser light with l ¼ 632 nm passes through a very narrow slit with w ¼ 0:1 mm. Then 2sin u ¼ 2
l 632 £ 1029 ¼ 1:264 £ 1022 ¼2£ w 1 £ 1024
Thus, angular width becomes 2u ¼ 0:7248: If the distance from the slit to screen is R ¼ 5 m, we can also calculate the width of the central maximum. 2Rl 2 £ 5 £ 632 £ 1029 ¼ 63:2 mm ¼ w 1 £ 1024
2xP ¼
Double Slits The diffraction pattern due to two parallel slits, each of which are separated by a distance apart d can be obtained in the same manner as that obtained for the single slit discussed in the above. Referring to Figure 2.31, the wave amplitude at point P can be obtained by integration of the following equation. For the coordinate system whose origin is located at the center of the first slit, EP becomes E ðw=2 ixk sin u E ðdþðw=2Þ ixk sin u dx þ w dx ð2:68Þ e e EP ¼ w R 2w=2 R d2ðw=2Þ where R is the distance from the center of the slit separation to the screen, equal to s in the figure, w is the slit width, and slit separation, d. Introducing two variables associated with slit width w and slit separation d, b ¼ 12 kw sin u, a ¼ 12 kd sin u, the above diffraction integral can be evaluated as EP ¼
2Ew w sin b cos a R b
The corresponding intensity distribution function is now IðuÞ ¼ 4Ið0Þ
2
sin b b
cos2 a
ð2:69Þ
where Ið0Þ ¼
10 c E w w 2 R
2
Equation 2.69 implies that in the u ¼ 0 direction the double slit provides four times the maximum irradiance as compared with the single slit. The equation also tells us that the irradiance is just a product of the irradiances obtained for double slit interference found in Young’s experiment and
80
Optomechatronics
single slit diffraction. The diffraction related factor ðsin b=bÞ2 here constitutes an envelope for the interference fringes given by the factor cos2 a: To illustrate this, Equation 2.69 is plotted for the case of d ¼ 4w: In the Figure 2.39 ðsin b=bÞ2 and cos2 a are plotted together. As can be seen from Figure 2.39 the product of the sine and cosine factors can be interpreted as a modulation of the interference fringe pattern by a single slit diffraction envelope. As discussed in a single slit case, the envelope has a minimum at b ¼ mp ml ¼ w sin u;
m ¼ ^1; ^2…
ð2:70Þ
On the other hand, the interference maxima occur at
a ¼ np nl ¼ d sin u;
n ¼ 0; ^1; ^2…
ð2:71Þ
When the conditions Equation 2.70 and Equation 2.71 are simultaneously met at the same point in the pattern, the diffraction minima coincide with the interference maxima. In this case, there will be no light available at this point participating in the interference. This is called missing order which is given by the following condition; n w or a ¼ m
d¼
n b m
ð2:72Þ
cos2 a
sin2 b b2
1.0
0 b
:
sin q :
−16p
−12p
−8p
−4p
0
4p
8p
12p
16p
−4p
−3p
−2p
−p
0
p
2p
3p
4p
FIGURE 2.39 Interference and diffraction patterns due to double slit.
Fundamentals of Optics
81
For example, when d ¼ 4w the missing orders of interference are given for n ¼ 4m ¼ ^4; ^8; ^12…, as shown in Figure 2.40. The image shown below the curve reflects the characteristics of the diffraction curve, namely, and many black and white stripes corresponding to the curve pattern. In a limiting case when d is very large, i.e., d q w the first missing order appears far from the center of the pattern. This case produces Young’s interference pattern for two slits. Bright fringes occur for a ¼ 0; ^p; ^2p, and so on. The angular separation between fringes is given by Da ¼ p, or approximately, 2Du .
4p 2l ¼ kd d
It is interesting to note that this is identical to the result of Young’s experiment given in Equation 2.49. Multiple Slits When an aperture consists of parallel multiple slits having a large number N of width w and center-to-center separation d, it is called a grating. In this case, the total wave amplitude arriving at point P in the observation plane can be calculated in a manner similar to that of the double slit. When we set up the origin of the coordinates at the center of the first slit in this case, we have the irradiance at point P as " ðdþðw=2Þ ð2dþðw=2Þ Ew ðw=2 ikx sin u e dx þ eikx sin u dx þ eikx sin u dx EP ¼ R 2w=2 d2ðw=2Þ 2d2ðw=2Þ ð3dþðw=2Þ ððN21Þdþðw=2Þ þ eikx sin u dx· · · þ eikx sin u dx 3d2ðw=2Þ
ðN21Þd2ðw=2Þ
ð2:73Þ
where R is the same as defined in Equation 2.68. When we consider only two terms in Equation 2.73, this reduces to the double slit case. If the contribution of the ‘th slit is denoted by E‘, adding all of these terms EP ¼
N21 X ‘20
E‘
and squaring this resultant amplitude, EP yields the following intensity distribution function; I ¼ Ið0Þ
sin b b
2
sin N a sin a
2
ð2:74Þ
where Ið0Þ is the irradiance in the direction u ¼ 0: It is noted that Ið0Þ ¼ N2 I0 : When N ¼ 1, and N ¼ 2, represent the case of single and double slit diffraction, respectively. Here, again, ðsin b=bÞ2 is the diffraction related term whereas ðsin N d=sin aÞ2 is related to the interference related term.
82
Optomechatronics I(θ) 4 I ( 0)
0
b:
−16p
−12p
−8p
−4p
0
4p
8p
12p
16p
sin q : −4p
−3p
−2p
−p
0
p
2p
3p
4p
FIGURE 2.40 A double slit pattern ða ¼ 4bÞ:
The interference pattern can be analyzed from the latter term as in the case of N ¼ 2: When a ¼ 0; ^p; ^2p; ^3p…, the term, which describes interference between slits, becomes maximum. According to L’Hospital’s rule for lim
a!mp
sin N a ¼ ^N; sin a
m ¼ 0; ^1; ^2; ^3…
If we call these maxima principle maxima to differentiate from others, those have a magnitude N2 : Because a ¼ ðkd=2Þsin u, it follows that for N q 2 the maxima occur at the u locations satisfying d sin u ¼ ml;
m ¼ 0; ^1; ^2; ^3…
ð2:75Þ
The examination of the interference term also indicates that there are minima and other maxima between the consecutive principle maxima. There are N 2 1 minima, which make sin N a ¼ 0; sin a
a¼^
p 2p ðN 2 1Þp ;^ ;… ;… N N N
ð2:76Þ
The secondary maxima occur between consecutive minima at ðN 2 2Þ points. These points can be approximated by noting that the term sin N a
Fundamentals of Optics
83
varies much faster than sin a: Therefore, the secondary maxima are determined at the following points when sin N a term has its largest value;
a¼^
3p 5p 7p ;^ ;^ ; ^· · · 2N 2N 2N
ð2:77Þ
In case of N ¼ 4, the primary and secondary maxima are shown along with the minima in Figure 2.41a. From this pattern, we can see that the principal wave patterns are much sharper than those of the double slit case are. In Figure 2.41b, the resultant intensity curve is plotted using Equation 2.74. It is clearly seen that the intensity ratio continues to decrease as due to wave diffraction. The device that utilizes multiple-slit diffraction is called the diffraction grating. In general, the incident wave fronts of light make an angle with the plane of grating, although the incident plane wave here is normal to the grating surface. N2
2 sin Na sina
irradiance, I
α: β: sin q :
π
2π
3π
λ/d
2λ/d
3λ/d
I0
4π π 4λ/d
2 2 sin b sin Na b sina
sin b b
irradiance, I
2 2 sin b sin Na b sina
α: β: sin q :
2
π
2π
3π
λ/d
2λ/d
3λ/d
4π π 4λ/d
FIGURE 2.41 Interference and diffraction patterns due to multiple slits ðN ¼ 4; a ¼ 4bÞ:
(b)
2
(a)
sin Na sina
84
Optomechatronics
Circular Aperture We have discussed diffraction from a slit having width w much smaller than its length ‘. When both dimension of the slit are comparable, it becomes a rectangular aperture. As we can envision the pattern of its diffraction, it produces appreciable spreading over the plane ðx-y coordinates) of the screen. The spread of the pattern, of course, varies with the aperture dimension. Since the spread pattern can be determined in a manner similar to a single slit, we will not treat the details here. Diffraction from a circular aperture is different in its pattern from that of a rectangular or a square aperture. Consider a plane wave propagating in the z-direction incident on an opaque diffraction screen containing a circular aperture as shown in Figure 2.42. Then, the wave amplitude of the optical field at point P due to source strength per unit area EA is expressed by E ð ð iyk sin u dA ð2:78Þ e EP ¼ A R A where the differential area is taken as dA ¼ s dy: Since s is related to qffiffiffiffiffiffiffiffiffiffi s ¼ 2 a 2 2 y2 where a is the radius of the aperture. The integral in Equation 2.78 becomes a ffi 2EA Z iyk sin u qffiffiffiffiffiffiffiffiffi EP ¼ e a2 2 y2 dy R
ð2:79Þ
a
y
dy
s y 0 a x
y′ q
r
p r′
R
z x′
FIGURE 2.42 Coordinate system for diffraction due to a circular aperture.
Fundamentals of Optics
85
If we introduce the quantities u and r defined by u ¼ y=a and r ¼ ka sin u, respectively then Equation 2.79 can be rewritten as EP ¼
2EA a2 ðþ1 iru pffiffiffiffiffiffiffiffi2ffi e 1 2 u du R 21
ð2:80Þ
The above integral has the value pJ1 ðrÞ=r where J1 ðrÞ is the Bessel function of the first kind. Carrying out the integration and substituting the Bessel function, we have EP ¼
2EA a2 pJ1 ðrÞ R r
This implies that the wave amplitudes at point P and its vicinity decay as the distance departs from P, obeying the law of the first order Bessel function. The irradiance for a circular aperture is obtained by IðuÞ ¼ Ið0Þ or IðuÞ ¼ Ið0Þ
2J1 ðrÞ r
2
2J1 ðka sin uÞ ka sin u
2
ð2:81Þ
where Ið0Þ is the irradiance at the center, i.e., u ¼ 0 and given by Ið0Þ ¼
E2A A2 R2
It is noted that the distance R is assumed constant and R q D where D is the aperture diameter, and A is the area of the aperture. The irradiance pattern due to Fraunhoffer diffraction given in Equation 2.81 is plotted in Figure 2.43a. It has an axial symmetry and a towering central maximum, which corresponds to a high irradiance. The image of the irradiance pattern corresponding to the distribution is plotted in Figure 2.43b and c. The central maximum is surrounded by a dark ring that is created by the first zero of the function J1 ðka sin uÞ: This maximum spot is called Airy disk named after G. B. Airy (1801 – 1892). The central disk is surrounded by a dark ring, which is created by the first zero of J1 ðrÞ: We find that the first zero J1 ðrÞ ¼ 0 gives r1 ¼ 3:83: The radius of the center of this dark ring, r1 that gives J1 ðka sin uÞ ¼ 0 is obtained by r1 ¼ 1:22
Rl 2a
ð2:82Þ
When light waves are transmitted through a lens, the lens acts as an aperture. If a lens of an identical size positioned within the aperture is focused on the screen, the focal length is approximately f . R: Then r1 can be rewritten in terms of aperture diameter as r1 . 1:22
fl D
ð2:83Þ
–10
–5
FIGURE 2.43 Irradiance due to a circular aperture.
(b)
–15 (a) 0
0
1.0
I (q ) I (0)
5
10
0.0175
15
I = I (0)(2J1(r)/r)2
(c)
r = (2π/λ)a sinq
86 Optomechatronics
Fundamentals of Optics
87
This equation indicates that the radius of the Airy disk is determined by the effective aperture of diameter D, focal length f, and wavelength of the incident light l. The radius of the Airy disk becomes small as the effective aperture becomes large but decreases as focal length gets large. The diffraction phenomenon can also easily illustrated by varying incident angle of the incoming beam to a thick lens. To illustrate this the Code V was used [8]. Figure 2.44 shows how the Airy disk changes with variation of incident beam angle when an image plane is held at the local plane. The lens has a focal length 2.6 mm and NA is 0.6. The smallest disk in the middle of the figure is obtained when the beam is perfectly collimated. In this case, the entrance pupil is 5.25 mm. However, as the incident beam is deviated with a slightly divergent angle as shown in the left figures, the Airy disk is seen to get larger in size and blurred in image. As the beam gets converged from the collimated case, the Airy disk again exhibits the same behavior as in the above. These phenomena are well depicted in the bottom figures where the image intensities of all six cases are shown. As we shall discuss in Chapter 7, this result implies that the collimator or other lenses should be accurately positioned in order to have accuracy of optical systems to satisfactory level. The resolution can be interpreted as the smallest resolvable separation between the central maximums (Airy disks) created by two equal-irradiance, incoherent, distant point sources separated apart by an angle u as depicted in Figure 2.45. If we introduce Du as an angular measure of the radius of the Airy disk, the disk at far field, for each point source will be spread out over an angular half-width Du about its own center. The angular distance of the Airy disk at far field is can be expressed by Du ¼
1:22l D
ð2:84Þ
where D is the diameter of the aperture. The equation in the above is of great practical importance since the result can be used to define the limit of resolution obtainable with the instruments containing various optical components. To explain this, let us consider the diffraction due to two point sources S1 and S2, shown in Figure 2.45. Those are formed through a single lens, which in this case is considered an aperture. If the separation angle between the two sources, u, is large enough, the formed images are well separated as seen from the figure. As S1 and S2 are brought together close to each other, the two image patterns gradually begin to overlap and finally get to the point where those cannot be resolvable. The limit condition is just resolvable and known as Rayleigh criterion. The limit of the resolvable angular separation of two point sources can be expressed as ðDuÞmin ¼
1:22l D
ð2:85Þ
Objective Le
1.00
ORA
Objective Lens that LGE Designed(01-03-2 Objective Le
FIGURE 2.44 Effect of the incident beam angle on image formation.
(c) intensity variation
(b) variation of Airy disk
MM
05-Feb-05
(a) beam configuration
Objective Lens that LGE Designed(01-03-2
1.00
ORA
MM
05-Feb-05
Objective Lens that LGE Designed(01-03-2 Objective Le
1.00
ORA
MM
05-Feb-05
Objective Lens that LGE Designed(01-03-2 Objective Le
1.00
ORA
MM
05-Feb-05
Objective Lens that LGE Designed(01-03-2
Objective Le
1.00
ORA
MM
05-Feb-05
88 Optomechatronics
Fundamentals of Optics
89
y′
S1 q
z′ = z
q
S2 lens image plane
(a)
qmin r1
(b) FIGURE 2.45 Diffraction limited image achievable by the minimum resolvable separation.
which is given in Equation 2.84. In terms of the center-to-center separation of the two images, ðDy0 Þmin , at the screen or at the focal plane of the lens, the limit of resolution is given by 1:22l ðDy 0 Þmin ¼ f ð2:86Þ D which is equal to r1 as shown in the figure. The diffraction-limited image is shown in Figure 2.46. In general, the resolving power of the imaging system is defined as by either 1=ðDuÞmin or 1=ðDy 0 Þmin : The equation in the above indicates that the resolution can be increased by increasing the diameter of the lens or decreasing the wavelength of light. Therefore, in the fine quality lens, if aberrations are assumed negligible the sharpness of Airy disk attainable by a point object is limited by diffraction. The spreading out of the image pattern due to the diffraction therefore represents the limit of the sharpness of image. As we shall see later, this criterion will be used throughout the book. Diffraction Grating The discussion made on diffraction previously can now be extended to the case of a large number of diffraction elements such as multiple slits, apertures, or obstacles. A periodic array of such elements is called a diffraction grating.
90
Optomechatronics
I2
I1
I1 + I2
(∆q)min
(a) q >> (∆q)min
(b) q ≈ (∆q)min
FIGURE 2.46 Images formed due to diffraction for two different angular separating of sources.
Previously, we have dealt with the case when the incident wave fronts of light make a right angle with the plane of grating surface but here will deal with an arbitrary incident angle. The common form of diffraction grating is an amplitude grating with a rectangular profile of transmittance as shown in Figure 2.47. At its surfaces, light modulation occurs due to the alternate opaque and transparent regions. This can be obtained by vacuum deposition of many opaque (metallic) strips on a glass plate, which forms a periodic array of narrow slits of equal distance. If parallel light beam strikes this grating with an oblique incident angle, ui , diffracted beams from successive slits interfere constructively and destructively. Referring to the right handside of Figure 2.47b, the net path difference between waves from successive slits is given by
d1 2 d2 ¼ dðsin um 2 sin ui Þ
ð2:87Þ
mth order m= +1 m= +1 m= 0 diffracted light m= 0 m= −1 incident light m= −1
(a) diffraction geometry
(b) diffracted light
FIGURE 2.47 Diffraction of light by a transmission amplitude grating.
δ2
qi
qm
δ1 d
Fundamentals of Optics
91
R B
grating lens
R B
white 1 light
f
m = +2
m = +1
m=0 B R B R
m = −1
m = −2
focal plane FIGURE 2.48 Spectra produced by white light.
from which the grating equation can be expressed as dðsin um 2 sin ui Þ ¼ ml;
m ¼ 0; ^1; ^2…
ð2:88Þ
for all waves in phase. In Figure 2.47b only three diffracted waves ðm ¼ 0; ^1Þ are plotted for simplicity. We can see that the diffraction angle differs from diffraction order. When m ¼ 0, the diffraction angle of the zeroth order of interference becomes identical to that of the incident wave, i.e., um ¼ ui : At this direction, the diffracted wave produces the maximum irradiance for all wavelengths l, as we have seen in the previous sections. Higher orders, however, the direction of um varies with wavelength l. These properties are used to measure the wavelength of light and do spectral analysis. Figure 2.48 illustrates how white light can be decomposed into different colors having different wavelength by a diffraction grating. To achieve this a collecting lens is employed to focus each collimating beam of wavelengths to a point in the focal plane. R and B denote the red (R) and blue (B) ends of the light color spectrum. Figure 2.49 shows a reflection phase grating. The gratings have highly reflecting faces of periodically varying optical thickness. Those modify the phase of incident light while the amplitude remains unchanged. This is in contrast with the amplitude grating. As shown in the figure, optical path difference is expressed as given in Equation 2.87 and therefore, the same grating equation as given in Equation 2.88 can be applied.
92
Optomechatronics incident light
qi
m = −1 m=0
diffracted light m = −1 m = +1 m=0
d1 d d2 qm
mth order
m = +1
(a) diffracted lights
(b) diffraction geometry
FIGURE 2.49 Reflection phase grating.
Optical Fiber Transmission Ever since the concept of optical fiber has been used as a means of transmission of signal or data, they have been widely used as transmission media in optical measurement as well as optical communication. In addition, those are becoming extremely important for laser beam delivery for instrumentation and surgical applications. This is due to low, loss transmission, high capacity of carrying information and immunity to electromagnetic interference. An optical fiber is composed of two concentric dielectric cylinders; the core (inner cylinder) and the cladding (outer cylinder). The core having a refractive index nco is clad with the cladding of lower refractive index ncl i.e., nco . ncl : The optical fiber is made based on the phenomenon of total internal reflection within this fiber structural arrangement. Let us elaborate this principle in more detail. As shown in Figure 2.50, rays strike the interface of the core and cladding. Applying Snell’s law at the interface, we have nco sin uco ¼ ncl sin ucl
ð2:89Þ
From this relation we can easily see that, when ucl ¼ 908, the refracted ray travels along the boundary as shown in the middle of Figure 2.50a. The corresponding ucl that yields this phenomenon is called the critical angle uc as discussed previously. As indicated in the right hand side of Figure 2.50a, when uco . uc , all rays striking the interface will be totally reflected back into the core by means of many internal reflections. Thus, the fiber becomes a light guide, as shown in Figure 2.50b. This critical angle uc can be determined from
Fundamentals of Optics
93
θt
ncl nco
θi
θi θi
θc
when θi = θc critical angle
when θi < θc refraction + reflection
θi
θi
when θi > θc total internal refraction
(a) reflection and refraction at various ray incident angles ncl nco θi
θ > θc refracted ray
ni cladding
(b) total internal reflection of rays within an optical fiber FIGURE 2.50 Light propagation in optical fiber.
Equation 2.89
uc ¼ sin21
ncl nco
To yield this critical angle, there will be a range of an angle of incidence for a ray to enter the core. Let us determine the range of the incident angle. If the fiber is surrounded by a medium having a refractive index ni , Schnell’s law gives for some refractive angle u. ni sin ui ¼ nco sin u
ð2:90Þ
If it is assumed here that for this incident angle ui , the refracted ray hits the interface with angle u, as indicated in Figure 2.50b. The angle u to be greater than uc must satisfy the following condition cos u $
ncl nco
ð2:91Þ
From Equation 2.90 and Equation 2.91 we see that the condition for the occurrence of total internal reflection is limited by sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi nco ncl 2 12 sin ui # ni nco
94
Optomechatronics
Since in air ni < 1:0 the maximum ui condition is given by qffiffiffiffiffiffiffiffiffiffiffiffi ð2:92Þ ½ui max ¼ sin21 n2co 2 n2cl qffiffiffiffiffiffiffiffiffiffiffiffi If the above quantity n2co 2 n2cl is defined as the numerical aperture (NA) of the fiber, the above equation be rewritten as ½ui
max
¼ sin21 NA
For example, if nco and ncl are 1.51 and 1.49, respectively, then ½ui max of a fiber in air becomes approximately 14.188. Therefore, NA of the fiber can be an indication of the maximum angle of incidence. Fiber with a wide variety of numerical apertures ranging from 0.2 up to 1.0 is being used, depending on applications. Figure 2.51 depicts the three major types of fiber configuration being used currently. The single mode index fiber depicted in Figure 2.51a has a very narrow core, typically less than 10 mm in diameter. Therefore, it can have only a single mode in which the rays travel parallel to the center through the fiber. The multimode step index fiber shown Figure 2.51b has a core of relatively large diameter, 50 to 200 mm, making it advantageous to launch light into
(a) single-mode step-index fiber
(b) multi-mode step-index fiber
(c) multi-mode graded-index fiber FIGURE 2.51 Major refraction index configurations of optical fibers.
Fundamentals of Optics
95
fibers using LED sources. In addition, it can be easily terminated and coupled. The multimode graded index fiber as shown in Figure 2.51c has a core diameter of 20 to 90 mm. The core employs a nonuniform refractive index, which has higher value at the center decreasing parabolically toward the interface. Due to this index variation, the rays smoothly spiral around the center axis instead of a zigzag path, as seen for the case of the multi mode index fiber. Power losses occur when light propagates through the fiber. It depends upon fiber material, scattering by impurities and defects for glass fiber and absorption for plastic fiber. The attenuation in optical power is usually defined by Pl ¼ 10 log10
Wo Wi
ð2:93Þ
where Pl is expressed in dB, and Wi and Wo are the input and output power, respectively.
Gaussian Beam Optics In modern optics, lasers are the indispensable light source for measurement, communication, data recording, material processing and so forth. In Table 2.4, various lasers are compared with regard to the type, state, wavelength and power range. In most of their applications, it is necessary to shape the laser beam using optical elements. The laser for most cases operates and emits a beam in the lowest order transverse electromagnetic (TEM) mode. In this mode, the beam is a perfect plane wave and has a Gaussian wave front whose characteristics are radically symmetric and can be described by Gaussian function. Although in actual lasers it is not perfectly Gaussian, in Helium Neon (HeNe) and argon-ion lasers it is very close. Figure 2.52 illustrates
TABLE 2.4 Comparison Between Various Laser Sources Type
State
Wave Length (nm)
He–Ne Argon CO2 Semiconductor Excimer (KrF) Nd–YAG He–Cd Ruby
Gas Gas Gas Solid Gas Solid Gas Solid
632.8 488.0, 514.5 10600 850 –910 248 1047–1064 442, 325 694.3
Power Range 0.5– 50 mW 0.5– 10 W 0.5– 10 kW ,10 kW 1–300 W 1 mW– 10 kW 50–150 mW Pulse (,400 J) (0.2 —5 ms)
96
Optomechatronics
r
1 irradiance contour e2
r(z)
I0 e2
I0
r0
I0 e2
0
q
2r0 zr
z
FIGURE 2.52 Spot size variation with propagation distance: Gaussian beam.
the characteristics of Gaussian laser beam such as beam irradiance and divergence profile. As can be seen from the figure, the beam has maximum beam intensity I0 at its center and decreases with distance from the axis. The beam diameter slightly differs depending upon how it is defined. Three widely used definitions are: (1) full power beam diameter, (2) half power beam diameter, and (3) 1=e2 excluded power beam. The full power, beam diameter is the diameter of the laser beam core that contains the total beam power, except the power portion outside 8s which is eight times the standard deviation of the distribution. This corresponds to 99.97% of the total power. The half power beam diameter contains 50% of the total beam power but excluded the rest of the 50% power lying outside 2:35s: The 1=e2 excluded beam diameter is the diameter that contains the beam power exclusive of 13.5% ð1=e2 Þ of the total power. This power corresponds to 4s: This diameter is commonly adopted when referring to the diameter of a laser beam. The divergence angle u of a Gaussian laser beam as indicated in the figure is the angle by which the beam spreads from the center of the waist. This angle is essentially the angle of the asymptotic cone made by 1=e2 irradiance surface. Let us consider the geometry of this beam spread. If rðzÞ is denoted by the radius of the contour at the distance z from the beam waist, it can be expressed by " rðzÞ ¼ r0 1 þ
lz pr20
!2 # 12
ð2:94Þ
where r0 , called the radius of beam waist, is the radius of the irradiance contour at the location where the wave front is flat. From Equation 2.94 we can see that r0 is the radius of the beam waist at z ¼ 0 and that rðzÞ is symmetrical about z ¼ 0: Also, it can be seen that at rðzÞ ¼ r0 the radius of curvature rðzÞ is infinite. For large value of z, that is, at the far field, the equation is largely determined by the term ðlz=pr0 Þ2 , since !2 lz q1 ð2:95Þ pr20
Fundamentals of Optics
97
Using the asymptotic line approaching z ¼ 0 and small angle assumption, we can obtain asymptotic angle u from Equation 2.95 tan21 u . u .
l pr0
ð2:96Þ
This angle is measured at a distance far away from the laser that is z q zr where zr stands for the Rayleigh range which is equal to pr20 =l: The range is defined by the distance from the center of the waist to the propagated location at which the beam wave front has a minimum radius of curvature rmin ðzÞ on either side of the beam waist. Let us have an example to illustrate how much the laser beam will spread for a given HeNe laser having wavelength 632.8 nm and diameter 1.0 mm. From Equation 2.96, we have
u.
632:8 £ 1026 ¼ 4:03 £ 1024 rad pð0:5Þ
Once this is obtained, beam spread rðzÞ can be easily determined at any location of z. Equation 2.94 can be applied to the case when a Gaussian laser beam enters a lens with small divergence angle and is focused at its focal point. Since the lens is located in the far field relative to the focus point as compared with the size of the small focused beam, the focused beam radius can be approximated by r0 ¼
lf pD
ð2:97Þ
where D is the aperture diameter of the lens.
Problems P2.1. Suppose that light wave is incident on the surface of a glass having refractive index nt ¼ 1:5 with an angle u ¼ 458, as shown in Figure P2.1.
45° qg
qa
FIGURE P2.1 A ray transmitting through a glass.
glass
98
Optomechatronics
(1) Determine the refractive angle ug : (2) What will be the transmittive angle ua ? P2.2. A meniscus convex lens made of a glass ðnt ¼ 1:5Þ has its radius of curvature of 28 cm at one side and 50 cm at the other side as shown in Figure P2.2a.
1.5m
1.5m
(a)
(b)
FIGURE P2.2 Meniscus convex lens.
(1) Determine the focal length of this lens. (2) If the same lens is flipped over in respect to the direction of the incident beam as indicated in Figure P2.2b, determine the focal length. (3) In both cases, (1) and (2), determine the image point on the optical axis when an object is placed 1.5 m in front of the lens. P2.3. A bundle of parallel rays transmits through (a) convex lens (b) two lenses composed of a convex lens and a planar convex as indicated in Figure P2.3. Draw a ray diagram in each case in detail and discuss the results of the ray path. f and f 0 are the focal lengths of the two cases.
L1
L1 L2
f
(a)
(b)
f′
FIGURE P2.3 Ray diagram.
P2.4. Design a beam expander using any combination of two types of lenses (concave and convex). The expander is required to expand a beam of 1.5 cm to 9.0 cm. Assume that the absolute value of the focal length of the lens is less than 9.0 cm.
Fundamentals of Optics
99
P.2.5. There are several ways to measure the effective focal length of a lens. One way is to use a collimator. Draw a configuration of this focal collimator, a focal length measurement system, and explain how it works in principle. P2.6. The beam expander is a frequently used device to expand beam diameter. The nonfocusing reflective beam expander is practical for most applications due to simplicity of its configuration. Draw three such beam expanders and explain how they work. P2.7. The image size of an optical system can be conveniently described by a lateral magnification factor, m ¼ h0 =h, where h and h0 are the object height and image height, respectively. If so and si are the object and image distances from the lens, respectively, prove that so and si can be expressed by so ¼
1 21 f m
si ¼ ð1 2 mÞ f P2.8. Consider the astigmatism discussed in Figure 2.23. Suppose that rays focus as a line at fT and fs as shown in Figure P2.8. If the lens is assumed to have a tangential focus ð fT Þ 17.8 cm and a sagittal focus ð fs Þ, 19.2 cm, what will be the diameter of the circle of least confusion. The circle is assumed to be located at the point having an average of the optical powers of the two focal lengths. The diameter of the lens is D: lens
circle of least confusion
D fT
fs
FIGURE P2.8 Location of circle of least confusion.
P2.9. A telescope is an optical device that enlarges the apparent size of an object located at a distance away. Such a device can be configured by combining two thin lenses, L1 and L2, shown in Figure P2.9. The two lenses are separated by distance d, and f1 and f2 are the focal lengths of lenses L1 and L2, respectively. (1) Locate the focal point of each lens and draw optical paths by which light reaches the eye for the two cases shown in Figure P2.9a and b.
100
Optomechatronics L1
L1
L2
L2
eye
(a)
eye
(b)
d L1
d L3
L2
(c) FIGURE P2.9 Inverted and erected images in a telescope.
(2) In Figure P2.9a, the image obtained is inverted. In order to obtain the erected image, repeat the same problem (1) for an optical system shown in Figure P2.9c. In this case, what will be magnification? P2.10. Figure P2.10 shows an aperture placed in an optical system composed of two identical convex lenses. Discuss the effect of the aperture stop on distortion. lens
aperture stop FIGURE P2.10 The effect of aperture stop.
P2.11. Suppose we wish to grind the rim of the lens so that the line between the centers of curvature of two lens surfaces coincides with the mechanical axis, which is defined by the ground edge of the lens. Given an accurate tubular tool on a rotating spindle where the lens can be fasten with wax or pitch, (1) configure an optical system for this grind system and (2) describe in detail how the system works.
Fundamentals of Optics
101
P2.12. Consider that an unpolarized light wave propagates through a polarizer with vertical transmission axis and then a second polarizer with its transmission axis making an angle of 608 with the vertical axis, as shown in Figure P2.12. What will be the polarization angle and the intensity of the transmitted light?
60°
unpolarized light
polarizer
polarizer
FIGURE P2.12 Polarizing light with a polarizer.
P2.13. Counting the number of fringes observed in a Michelson interferometer is used to measure a displacement of an object. When the object is displaced by 0.81 mm, a shift of 2000 fringes is observed. Compute the wavelength of the light source. P2.14. Figure P2.14 shows an arrangement of Fabry-Perot interference. The arrangement utilizes a collimation lens between the source and the interferometer. Discuss what types of fringes will occur. Give the reasons for this. d
S
source
collimating lens
interferometer focusing lens screen
FIGURE P2.14 Fabry-Perot interferometer.
P2.15. When an incident beam contains two wavelengths which are slightly different from each other, the transmitted irradiance of each fringe will be shown in Figure P2.15. If one wavelength l0 is 500 nm and the other differs only by Dl ¼ 0:0052 nm, what will be the spacing of the Fabry-Perot
102
Optomechatronics T
fringe 1 fringe 2
1.0
0.5
∆js ∆l l0
∆ϕ
FIGURE P2.15 Two overlapping wavelength components.
etalon when the mth order of one component coincides with the ðm þ 1Þth order of the other? P2.16. A Fabry-Perot etalon has a reflection coefficient of rr ¼ 0:92 and a plate separation of 1.5 cm. The incident light has two slightly different wavelengths around 490 nm. Determine (1) coefficient of finesse (2) maximum order of interference (3) minimum resolvable wavelength difference Dl: P2.17. The optical system shown in Figure P2.17 called “Lloyd’s mirror” consists of a point source S, a plane mirror M placed near the source, and image plane Si :
point source S
image plane Si
M mirror FIGURE P2.17 An optical system composed of point source S and a plane mirror M.
(1) Draw the relevant optical paths in order to describe the image in the image plane Si : (2) What type of image will appear in the plane Si : P2.18. Suppose that a parallel beam of 632.8 nm light propagates through a single slit. If the distance between the slit and screen is 1 m, determine the width of the slit in order to have the spread of the central maxima, 30 cm.
Fundamentals of Optics
103
P2.19. Suppose that a coherent light wave having the wavelength l ¼ 640 nm passes through double slits whose separation distance is 0.2 mm. The screen is located 1 m away from the slits. Calculate the width of the central maxima. P2.20. In double slit diffraction, the relationship between separation and width is given by d ¼ 5w: (1) Plot the interference term vs. a (2) Plot the diffraction term vs. b (3) Plot the irradiance vs. spatial angle, u P2.21. From Figure 2.40 and Figure 2.41, we can see that the waves appear much sharper for the case of multislits than that of double slits. Give a physical reason in some detail. pffiffi P2.22. Show pthat ffiffi at Rayleigh range zr the beam spreads by a factor of 2, that is, rðzr Þ ¼ 2r0 : P2.23. Suppose that a laser beam spreads out from the beam waist. Determine the beam radius at a location z ¼ 80 m for a beam of radius 0.3 mm laser diode l ¼ 810 nm. P2.24. Suppose that a HeNe laser of radius 5 mm enters a lens having focal length 50 mm. Compute an approximate focus spot diameter.
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
Andonovic, I. and Uttamchandani, D. Principle of Modern Optical Systems, Vol. 1. Artech House, Boston, MA, 1989. Bernacki, B.E. and Mansuripur, M. Causes offocus-error feedthrough in optical-disk systems: astigmatic and obscuration methods, Applied Optics, 33:5, 735–743, 1994. Fan, K.C., Chu, C.L. and Mou, J.I. Development of a low-cost autofocusing probe for profile measurement, Measurement Science and Technology, 12:12, 2137–2146, 2001. Gasvik, K.J. Optical Metrology, 2nd Ed., Wiley, New York, 1996. Heavens, O.S. and Ditchburn, R.W. Insight into Optics, Wiley, New York, 1991. Hecht, E. Optics, 4th Ed., Addison Wesley, Reading, MA, 2001. Mansuripur, M. Classical Optics and its Applications, pp. 222– 239, Cambridge Press, Cambridge, 2002. Code V, Optical Research Associates, 2001. O’Shea, D.C. Elements of Modern Optical Design, pp. 24 – 75, Wiley, New York, 1985. Pedrotti, F.J. and Pedrotti, L.S. Introduction to Optics, pp. 43 – 348, Prentice Hall, Englewood Cliffs, NJ, 1992.
104 [11]
Optomechatronics
Pluta, M. Advanced light microscopy, Principles and Basic Properties, pp. 110 – 129, Vol. 1. Elsevier, Amsterdam, 1988. [12] Rossi, B. Optics, pp. 122– 149, Addison-Wesley, Reading, MA, 1957. [13] Smith, W.J. Practical Optical System Layout: and Use of Stock Lenses, McGraw-Hill Professional, New York, 1997.
3 Machine Vision: Visual Sensing and Image Processing CONTENTS Image Formation................................................................................................ 109 Imaging Devices................................................................................................. 112 Image Display: Interlacing ....................................................................... 115 Image Processing ............................................................................................... 115 Image Representation........................................................................................ 116 Binary Image............................................................................................... 117 Gray Scale Image ....................................................................................... 117 Histogram.................................................................................................... 118 Histogram Modification............................................................................ 118 Image Filtering ................................................................................................... 121 Mean Filter .................................................................................................. 123 Median Filter .............................................................................................. 124 Image Segmentation .......................................................................................... 127 Thresholding............................................................................................... 127 Iterative Thresholding ............................................................................... 131 Region-Based Segmentation ..................................................................... 133 Edge Detection ................................................................................................... 136 Roberts Operator........................................................................................ 138 Sobel Operator............................................................................................ 140 Laplacian Operator .................................................................................... 142 Hough Transform .............................................................................................. 148 Camera Calibration............................................................................................ 152 Perspective Projection ............................................................................... 155 Problems.............................................................................................................. 165 References ........................................................................................................... 172
Visual sensing is prerequisite for understanding the structure of a 3D environment, which requires assessing the shape and position of objects. The sensing is to acquire the image of the environment through a visual or an 105
106
Optomechatronics
imaging system, usually a camera system. When the system sees the scene containing objects, the amount of light energy it acquires depends upon a number of factors, such as shape, optical properties of objects, background, and illumination conditions. Often, these factors are critical to determine the amount of light impinging on the object’s surface. Depending upon these factors, the camera system takes in a certain amount of light reflected and scattered from the scene through its optical system. The received light energy in general determines the intensity of light sensed by the camera, which is the physical quantity that the vision system handles to interpret the scene. In proportion to the intensity of the received light, the imaging hardware device of the camera transforms the physical quantity into an electrical signal. This electrical video signal is captured, processed, and digitized by the frame grabber in a discrete fashion so that a computer can understand the digitized values. After digitization, this signal is sent to a computer for image processing. Utilizing the acquired image, the image processing algorithms carry out several steps of image modification and transformation with a view to extract the information necessary to understand the environments. This information includes a shape, a surface texture, a size of objects, location, orientation of objects, and distance from the vision sensor to objects. Based on this information, a so-called high level algorithm classifies the objects into several groups of dissimilar objects and finally carries out the pattern recognition tasks, if necessary. Machine vision technique comprises a series of these tasks described above. The complete steps needed for image processing, analyzing, and understanding are depicted in Figure 3.1. Let us reiterate in more detail the image analysis part. The objective of the image analysis is to understand the nature of an image by acquiring and interpreting the information necessary to solve a variety of engineering problems. It is composed of several hierarchical steps: (1) (2) (3) (4)
Preprocessing Intermediate processing Feature extraction Scene understanding.
Preprocessing is a stage of selecting an interested region of the image and enhancing the acquired image to be analyzed by eliminating artifacts from the image, filtering noise in the image, making image quantization such as thresholding, and so on. In many cases, the original image acquired usually contains noise, broken lines, blurs, low contrast, and geometric distortion caused by the lens. These unwanted defects make it difficult to accurately extract high-level information such as points, edges, surfaces, and regions. It is, therefore, necessary to eliminate or reduce noise and blurring for the improvement in contrast and the correction of distortion. The processing of carrying out these tasks is called image preprocessing and should yield an improvement to the original image.
Machine Vision: Visual Sensing and Image Processing
107
FIGURE 3.1 Steps for visual information processing.
Since the image usually contains a great deal of pixel data, it is difficult to directly use them to analyze the nature of the acquired scene. To this end, the task in the next stage concerns the extraction of the higher-level information mentioned above, based upon the results obtained by preprocessing. Utilizing the preprocessed image, the intermediate processing stage achieves edge detection, segmentation, and transformation of the image into another domain. Of course, the quality of preprocessing greatly affects that of this stage. In the step for feature extraction, selection of the features is carefully made by considering which information will be useful to best represent the characteristics of objects in the segmented image. The purpose of using the features of the image, rather than the raw image itself, is to make searching and acquiring the representative information easier for processing, thus reducing the amount of time for image analysis. There are many types of features used to extract information as listed in Table 3.1. The frequently used features are geometrical shape, texture, and color selection as clarified in the table. Feature extraction is, therefore, the process
108
Optomechatronics
TABLE 3.1 Commonly Used Feature Sets for High Level Image Processing Feature Set Geometrical shape
Parameter
Description
Center of mass
Area moment
Perimeter
Width
h w
Vertex
Aspect ratio
b a
Principle axis
Elliptic ratio
b a
Circularity Gradient Symmetry Histogram
Orientation Measure for symmetry
Histogram shape (energy, entropy)
Color
Color difference
Color brightness
Texture
Power (spectrum)
Machine Vision: Visual Sensing and Image Processing
109
of acquiring hyper-level image impunity, which is application dependent. In the final stage, these extracted features are utilized for the interpretation of the scene, which is an indispensable stage for inspection, measurement, and recognition of the objects. There are two types of the vision system: active and passive, which are needed to capture the image of a 3D environment. Active vision acquires a 3D image by projecting the light energy onto a scene. In contrast, the passive method utilizes images acquired in the visible region of light spectrum with no use of any external light projection. Due to this advantage, this method has been applied to various types of environments. In this chapter, we will treat the principle of an image formation, an imaging device or a sensor that acquires images, and some fundamental image processing techniques, which include filtering, thresholding, and edge detection. Camera calibration will also be discussed, whose process enables us to interpret the acquired image output in terms of a real world coordinate system. Throughout this chapter, we will limit ourselves to a passive vision method, a monocular system, and a monochrome image.
Image Formation Light emits energy in the ultraviolet (UV), visible, and infrared (IR) portions of the electromagnetic spectrum. UV light is invisible with a very short wavelength in the range of 10 to 400 nm. Visible light is electromagnetic radiant energy with a short wavelength between 400 and 700 nm. In contrast, infrared light has a rather long wavelength between 0.7 and 100 mm. Brightness is related to the radiant flux of a visible light emitted from a source. Depending on how particular objects in the scene to be imaged are illuminated and how they reflect light, the imaging system presents the image of different brightness. For a given illumination condition and scene, the imaging system will present an image with brightness distributed over the scene in a specified manner. The distribution of brightness contains such important information as shape, texture of surface, and color. In addition to these, it will also contain the information on the position and the orientation of the objects, which can be obtained by determining the geometric correspondence between points in the image and those in the scene. The brightness of the imaged surface depends upon optical reflectance characteristics of imaging surface which are very critical to acquiring images. They are, however, not so simple to analyze since the surfaces reflect light in many different ways according to their texture and wavelength. As illustrated in Figure 3.2, there are three typical types of reflectance pattern. They are the specular surface, which reflects light at an angle equal to the incident angle, the Lambertian surface, which uniformly scatters light in such a way that the light has consistent luminance scattered without
110
Optomechatronics specular diffuse
incident light
specular
diffuse
FIGURE 3.2 Radiance depending on surface condition.
dependency on the view angle, and finally, the specular diffuse surface, which scatters incident light with some directivity. When a reflected light contains specular properties to some extent, it sometimes requires special attention and, therefore, normally needs a special illumination condition and imaging methods. Another important parameter to imaging is the field of view, since it determines the angle within which an imaging system can see. The angle is defined by the angle of the cone of directions encompassed by the scene to be imaged, and depends upon the focal length of the system. Normally, a telephoto lens has a narrow field of view due to its large focal length relative to the size of the object to be imaged, while an ordinary lens has relatively broader range of field of view. Before discussing sensing and image processing, let us consider how the image of an object is formed within the image plane and discuss the geometric correspondence between points in the scene and points in the image. There are two projection models to relate the image to the scene: perspective and orthographic. In the perspective projection shown in Figure 3.3a, each point in the image corresponds to a particular direction from which a ray passes through a pinhole. Let P denote a point in the scene with camera coordinates ðxc ; yc ; zc Þ, and Pi denote its image described in the coordinates ðxi ; yi ; zi Þ: If f is the distance of the image plane from the pinhole, we have zi ¼ f : From the optical geometry, the relation between the two coordinate systems are given by xi y f ¼ i ¼ xc yc zc
ð3:1Þ
and from this we have xi ¼ fxc =zc
yi ¼ f yc =zc
ð3:2Þ
Machine Vision: Visual Sensing and Image Processing
111
image plane yi
xi
zi = f
xc
yc
Oi
object zc
P (xc, yc, zc) zc
Oc Pi(xi, yi)
(a) perspective model image plane yi
xi
Oi
xc
yc
P(xc, yc, zc) Oc
Qi(xi, yi)
object
zc Q(xc, yc, zc)
(b) orthographic model FIGURE 3.3 Image formation models.
This implies that as zc increases, the image point of the object point gets closer to the center of the image plane. Orthographic projection shown in Figure 3.3b involves the projections of two parallel lines to the optical axis on the image plane. Consider again a point Pc ðxc ; yc ; zc Þ in the object located at z ¼ zc : The perspective projection model given in Equation 3.1 can be rewritten as xi ¼ 2mxc
ð3:3Þ f where lateral magnification is defined by m ¼ 2 : zc When the depth of the scene is small relative to the average distance of the surface from the camera lens, the magnification m is almost constant. In this case, we can simplify Equation 3.3 by normalizing the coordinate system with m ¼ 2 1, and thus, the orthographic projection is defined by xi ¼ xc
yi ¼ 2myc
yi ¼ yc
ð3:4Þ
This equation implies that, unlike perspective projection, the orthographic projection is not sensitive to changes in depth and can ignore the depth information. This approximation can often be useful for the optical system whose focal length and camera distance are large to compare to the size of the object, as in the case for a microscope observing microparts.
112
Optomechatronics
Imaging Devices There are primarily three types of imaging sensors presently in use: a charge coupled device, a charge injection device, and a photo diode array. The most commonly used device is the charge coupled device (CCD) and, thus, we will describe it in brief. Since AT&T Bell Laboratories invented the CCDs in the late 1960s, the growth has been exploited in almost all application areas. They are currently being used as indispensable imaging detectors of visible, ultraviolet, and x-ray light, and have now become the central element in an imaging system. The CCD visual sensor is composed of an array of tiny photosensitive semiconductor devices called “pixels”. When light is incident on the devices, a charge is generated by absorption of photons and accumulated during a certain period of time. The amount of the charge accumulation is dependent on the intensity of light. The accumulated charge is stored and converted to an electrical signal to be used for image analysis when it is processed properly. To briefly understand how CCDs work, let us consider a unit of CCD cell called “pixel” as shown in Figure 3.4. A semiconductor (p type) has two different states of electrons: One is where the electrons are free to move around in response to an electric field and the other is where the electrons are prevented from moving around. When light enters the silicon surface and photons are absorbed, this causes electrons constrained in motion to move freely, leaving behind a hole, an occupied site. This generates charge, i.e., electron– hole pairs. The figure shows the state of the accumulated charges in a well by introducing an electric field with a voltage þ V through an optically transparent electrode. This state is a result of charge creation by incident light. The amount of the charge here represents the integration of light. After accumulating the charge during a specified integration time, incident lights
transparent electrical contact
+V
e– – e– e e– – e– e e–
depletion region FIGURE 3.4 Generation of charge packet by photons.
p-type semiconductor
Machine Vision: Visual Sensing and Image Processing
113
the charge collected at each pixel needs to be transferred to the adjacent electrode for a sequential move to a readout port. Figure 3.5 illustrates a charge transfer in three phases, using seven electrodes and cycling three voltages (V ¼ 0, 5, and 10 V) on the rectangular grid of electrodes. In Figure 3.5a, the state of the charge due to CCD exposure to incident light is illustrated. A certain amount of charge is accumulated with the electrode voltage V ¼ 10 in the packets A3 and A6. When the voltages on the adjacent wells are increased from 0 to 10 V, the charges become shared between the electrodes E3 and E4 for the packet A3 and E6, and E7 for the packet A6, as shown in Figure 3.5b. Removing the voltages from E3 and E6, the charges in the packets A3 and A6 are completely removed and transferred to the respective adjacent electrodes as indicated in Figure 3.5c. The principle of the transfer of charge from pixel to pixel presented above can be applied to the case for the CCD array configured in Figure 3.6. The CCD system here contains a large pixel array, multiple vertical registers, and usually one horizontal shift register. The charge transfer flow is sequenced V1 applied voltage V2 V3
0 5 10 E1
pixel electrode
E2
E3
E4
E5
E6
E7
0 A3
10
A6
packet
packet
(a) end of exposure V1 V2 V3
10 0 5
A1
A7
A4
(b) charge transfer V1 V2 V3
5 10 0
A2
A5
(c) end of transfer FIGURE 3.5 Charge transfer in a “three phase” CCD.
114
Optomechatronics CCD array configuration
pixel
vertical shift register
output stage
image data
Horizontal CCD Shift Register FIGURE 3.6 Charge transfer configuration for read-out.
as follows. As indicated by dotted lines and arrows, the accumulated charges are transported down to the horizontal shift register by the vertical shift registers. The horizontal register collects one line at a time and moves each packet of charge in a serial manner far right to the output stage. This is called an interline transfer CCD method. Figure 3.7 shows a typical image signal pushed by shift registers. Image signal level is represented by a solid line ranged from 0 to þ 0.714 V, where þ 0.714 V indicates the white level while 0 V the black level. The horizontal synchronization intervals are denoted by the negative pulses with 10.9 m sec. The dotted lines before and after a pulse indicate the last pixel of a line and the first pixel of the next line, respectively. + 0.714V
last pixel of line
white level
video signal 0 – 0.286V
blanking level
first pixel of line
horizontal sync interval 10.9 ms
active line time 52.95ms 1 video line 63.49ms
horizontal line timing FIGURE 3.7 Horizontal synchronization.
black level
Machine Vision: Visual Sensing and Image Processing
3 5
start of odd field frame 2 4
. . . .
477 479
. . . .
480 Active lines
retrace
in the interlaced frame
1
115
478 480 reset to even field frame
FIGURE 3.8 Interlaced image display.
Image Display: Interlacing Interlacing is a method of displaying an image signal by use of the alternate line field structure required by the RS-170 format standard. According to the format standard which is administered by the Electronics Industries Association (EIA), the image on the monitor has 480 lines divided into two fields of 240 lines. As shown in Figure 3.8, they consist of a field containing the odd numbered line depicted as solid lines and a field containing even number lines depicted as dotted lines. Each field is scanned in 1/60th of a second, so that a complete image can be scanned per 1/30 sec. The frequency of scanning is therefore 525 horizontal cycles per 1/30 sec. The time duration for a complete cycle is 63.5 m sec, the forward scan and return trace requiring 52.1 and 11.4 msec, respectively.
Image Processing The image signal sensed by CCD cells is transferred to the acquisition unit called a frame grabber. This captures video signals, buffers or stores the images, and simultaneously displays the image on an external monitor. Figure 3.9 is a simplified version of the grabber and shows its components. The major components are: a video multiplexer that takes in video signals up to N channels, an analogue to digital (A/D) converter that converts analogue
116
Optomechatronics
connected to PC/104 Plus
image grabber
CCD camera
video 1 video 2 …
. video n
video MUX
A-to-D converter amplifier
composite sync HSync VSync
On-board processing unit buffer memory
sync generator
clock syncronization
external trigger
BUS interface
internal clock PCI, ISA and PC104+ BUS Line
FIGURE 3.9 Architecture of an image grabber. Source: Datatranslation Co. Ltd (www.datx.com) and Meteor Co. Ltd (www.natrox.com).
video signal to digital signal, an onboard processing unit that processes various real-time tasks, a frame buffer that stores the image data and buffers its flow, and the interfacing unit that allows to transfer the processed image data onto a host computer (CPU) through bus lines such as PCI, ISA, PC 104þ, and so on. We will discuss some of the characteristic features of a CPU, a multiplexer, an A/D converter, and the data bus later in Chapter 4.
Image Representation Brightness of the image acquired by the camera is defined on every pixel within the CCD array, which is a function of spatial coordinates. To describe the brightness an image function is usually used, which is a mathematical representation of an image that a camera produces on a CCD cell. The function denoted by f ðx; yÞ defines the brightness of the gray level at the spatial location ðx; yÞ: In other words, the brightness of the image at the pixel point ðx; yÞ is represented by f ðx; yÞ: When digitized by the frame grabber, it can be represented by f ði; jÞ where i and j are the ith row and jth column of
Machine Vision: Visual Sensing and Image Processing
117
FIGURE 3.10 Gray image and binary image.
the 2D CCD array cell, respectively. Two types of image will be considered here: binary and gray scale. Binary Image The binary images have two-valued gray levels and are produced from the gray scale image by using a threshold technique to segment the image into regions. Let us consider a 256 £ 256 image of a single object in the field of view of a camera, with a background having different brightness, as shown in Figure 3.10. The object and the background are seen to have approximately uniform brightness. Since, in this case, the object appears darker than the background does, it is easy to segment these into two regions, 0 for the object and 1 for the background. The image represented by this two-valued function is called binary image. Such an image can be obtained by thresholding the gray level image, which we will discuss later. If the pixel intensity is larger than that of the set threshold, it is regarded to possess “1”. On the other hand, if the intensity is below it, it is regarded to possess “0”. The binary image is easier to digitize and store than a full gray-level image, but some information is lost. Therefore this image is usually used for simple image analysis or analysis of simple image. Gray Scale Image In contrast to the binary information, gray scale images have a number of different brightness levels. The levels can be expressed by using eight bit/pixel which expresses 255 different levels. The levels often become
118
Optomechatronics
FIGURE 3.11 Various shapes of histogram.
a burden of computation. And they also have values in the range 0 to 63, or 0 to 1023, corresponding to six or ten bit digitization, respectively. Histogram Histogram representation of an image is a fundamental tool for image analysis and enhancement. The histogram is a plot of the relationship between gray scale value and the number of pixels at that value and, thus, it is a discrete function. As we shall see later, the shape and the magnitude of the histogram are indicative of the characteristic nature of the image, and they are very useful in determining a threshold. Figure 3.11 shows three different histograms where an image has a variety of different distributions over gray level. Shown in Figure 3.11a is the bimodal distribution in which two groups of gray level distributions are somewhat distinct. In this case, the whole image is usually divided into a binary image, that is, black “0” and white “1”. This is one of the simple forms of histogram, which makes it easy to do image analysis. Figure 3.11b shows a very narrow, clustered histogram, which is bad from the viewpoint of image quality. This is because the image has very low contrast, thus making it difficult to discriminate different objects within the image. On the other hand, the histogram shown in Figure 3.11c indicates a spread of histogram over the wide range of gray values, which is good from the view point of image contrast. In this case, it is said to have a high contrast. Histogram Modification When the contrast of an image is low due to poor lighting, incorrect set-up of the camera, or various other reasons, the image is not easily understandable. This is because the differences in the intensity between pixel values are small, so that objects and the background in the image cannot be discerned. In this case, it is necessary to improve the contrast of a poor contrast image.
max.
Ia
Ib
gray level
No. of pixels
(a)
min.
(b)
intensity after enhancement
No. of pixels
Machine Vision: Visual Sensing and Image Processing
119
Ib′ z′
Ia′ Ia Ia′
intensity
Ib′
gray level
z I b
z
intensity before enhancement
(c) Intensity transformation relationship
FIGURE 3.12 Contrast enhancement.
In general, there are two commonly adopted methods of modifying the contrast as shown in Figure 3.12. Let us assume that the original pixel intensities lie in the range Imin # I # Imax ; as shown in Figure 3.12a. This indicates that the image has most of its pixels in the range between Ia and Ib : The first method shown in Figure 3.12b is histogram stretch which extracts a certain range of the intensity values of the interested pixels and modifies them in such a way that a new intensity distribution can make the original image more visually discernable. Let us assume that we are interested in mapping the intensity level range of the original image shown in Figure 3.12a to a new intensity distribution shown in Figure 3.12b. Suppose that we wish to transform the original intensity distribution into a modified distribution. In order words, if a new intensity value z0 is mapped from the corresponding value z, intensity z Ia # z # Ib
!
intensity z0 I 0a # z0 # I 0b
To achieve this, a transformation formula for a histogram stretch can be derived from Figure 3.12c. z0 ¼
I 0b 2 I 0a ðz 2 Ia Þ þ I 0a Ib 2 Ia
ð3:5Þ
The second method called “histogram equalization” shown in Figure 3.13 transforms the original image such that an equal number of pixels is
Optomechatronics
No. of pixels
120
0
dr
r
gray level
No. of pixels
(a)
(b)
0
intensity
ds
s
gray level
FIGURE 3.13 Histogram equalization.
allocated throughout the intensity range of interest. The effect of the histogram equalization is to improve the contrast of an image. In other words, there will be an increase in brightness differences of pixel values near the intensity values at which there are many pixels. In contrast to this, there will be a decrease in brightness differences near the intensity values with few pixels. To illustrate the concept in brief, let us suppose that gray levels described in the original image shown in Figure 3.13a can be transformed in a new gray level distribution having an equal number of pixels within a certain gray value range as illustrated in Figure 3.13b. We wish to find such transformation s ¼ TðrÞ where r and s are the variables representing gray level in the original image and that in the new image, respectively. If the probability density function Pr ðrÞ is assumed to represent the gray level pattern in the range ½r; r þ dr shown in Figure 3.13a, then the transformation will be made to obtain Pr ðsÞ in the range Pr ½s; s þ ds : Because the total number of pixels in the two images will remain equal, we have Pr ðsÞds ¼ Pr ðrÞdr This equation is a basic equation to the histogram equalization process which is essentially to make all the gray levels in the original image equally
Machine Vision: Visual Sensing and Image Processing
121
probable. By letting Pr ðsÞ be a constant C, we have ðs 0
C ds ¼
ðr 0
Pr ðrÞdr
The above equation leads us to obtain the range of the gray value having an equal probability s¼
1 ðr P ðxÞdx C 0 r
where x is the dummy variable. This is illustrated in Figure 3.13b. Other histogram modifications such as histogram stretching, shrink, and sifting can also be performed in a similar manner. Histogram shrink is the opposite of a histogram stretch and decreases image contrast by compressing the gray level. Histogram slide is to translate the histogram in order to make an image either darker or lighter, while retaining the relationship between gray level values.
Image Filtering As mentioned earlier, noise more or less occurs in the image, corrupting a true intensity variation. Filtering an image is to recover its intrinsic visual information when it contains unwanted noise. This can be done by transforming the image intensities in order to remove noise or by performing image enhancement. There are two types of filtering depending upon whether the transformation is made in spatial or frequency domain: spatial domain filter and frequency domain filter. We will discuss briefly only spatial filters such as mean filter and median filter, which are frequently used to remove noise. Before getting into further details of image analysis, it is very helpful to become familiar with the convolution concept in order to understand filtering and edge-detecting procedures. Figure 3.14 indicates three cases of convolution masks. They show the 1 £ 2, 2 £ 1, 2 £ 2, and 3 £ 3 masks defined in an image frame. In the figure, M are the masks operated on the specified pixels, which are also called detection operators. And mij indicates the weighting values placed on the respective pixels. As indicated for a (3 £ 3) mask in Figure 3.15, the convolution process proceeds as follows; Overlay the mask on the image frame, multiply the corresponding pixel value one by one, sum all these results, and place this result at the center of its mask, as shown in the figure. This process continues until it is completed with the final pixel of the image. In a mathematical form, the output of the summation of the convolution results
122
Optomechatronics i 2x2 convolution mask
i
1x2 convolution mask Mx = m11 m12 j
M =
f(i,j ) f (i+1,j ) f(i,j+1)
j
f(i,j )
m11 m12 m21 m22
f(i+1,j )
f(i,j+1) f(i+1,j+1)
My = m11
2x1 convolution mask m 21 image frame
(a) 1×2 and 2×1 convolution masks
(b) 2×2 convolution mask
i
3x3 convolution mask
M= f(i-1,j-1) f(i,j-1) f(i+1,j-1) j
f(i-1,j)
f(i,j )
image frame
m11 m12 m 13 m21 m22 m 23 m31 m32 m 33
f(i+1,j)
f(i-1,j+1) f(i,j+1) f(i+1,j+1)
(c) 3×3 convolution mask FIGURE 3.14 Convolution masks.
3x3 mask
. mask center
FIGURE 3.15 The procedure of applying convolution mark.
image frame
Machine Vision: Visual Sensing and Image Processing
123
can be written as gði; jÞ ¼ f ði; jÞ p mði; jÞ ¼
p n X X
f ði 2 k; j 2 lÞmðk; lÞ
ð3:6Þ
k¼1 l¼1
where: p denotes the convolution of f ði; jÞ with mði; jÞ; gði; jÞ is the sum of results of the multiplication of coincident values, f ði; jÞ represents the input image, and mði; jÞ is called the convolution mask. For the case of a 3 £ 3 mask, the output gð2; 2Þ in the above is computed by gð2; 2Þ ¼ m11 f11 þ m12 f12 þ m13 f13 þ m21 f21 þ m22 f22 þ m23 f23 þ m31 f31 þ m32 f32 þ m33 f33
ð3:7Þ
This procedure is explicitly indicated in the figure. Referring to Equation 3.7, the convolution values at pixel ði; jÞ can be easily computed by the following manner; For 1 £ 2 and 2 £ 1 masks Mx ¼ m11 f ði; jÞ þ m12 f ði þ 1; jÞ;
My ¼ m11 f ði; jÞ þ m21 f ði; j þ 1Þ
ð3:8Þ
For 2 £ 2 masks M½ f ði; jÞ ¼ m11 f ði; jÞ þ m12 f ði þ 1; jÞ þ m21 f ði; j þ 1Þ þ m22 f ði þ 1; j þ 1Þ
ð3:9Þ
For a 3 £ 3 mask M½ f ði; jÞ ¼ m11 f ði 2 1; j 2 1Þ þ m12 f ði; j 2 1Þ þ m13 f ði þ 1; j 2 1Þ þ m21 f ði 2 1; jÞ þ m22 f ði; jÞ þ m23 f ði þ 1; jÞ þ m31 f ði 2 1; j þ 1Þ þ m32 f ði; j þ 1Þ þ m33 f ði þ 1; j þ 1Þ
ð3:10Þ
Mean Filter A mean filter is one of the simplest linear filters and is essentially an average filter that operates on neighboring pixels within an m £ m window. It replaces the center pixel with an average value of the image intensities of the pixels. A convolution mask travels through the image for computation and replacement of the computed average. This can be mathematically expressed by X ~ jÞ ¼ 1 fði; f ðk; lÞ N ðk;lÞ[w
ð3:11Þ
124
Optomechatronics
where N is the total number of pixel within the window W. For example, if a 3 £ 3 mask is given as 1 9
1 9
1 9
1 9
1 9
1 9
1 9
1 9
1 9
~ jÞ becomes then, fði; jþ1 iþ1 X X ~ jÞ ¼ 1 fði; f ðk; lÞ 9 k¼i21 l¼j21
ð3:12Þ
It is noted that the sum of the coefficients of the mask is 1. Median Filter A median filter is effective in coping with impulse noise, while retaining original image details. This is because it does not consider any values which are significantly different from the typical values in the convolution mask. To see how it works, consider Figure 3.16 where a mask (3 £ 3) of the filter is shown. It looks similar to the convolution mask. We can see that the process is not a weighted sum, but employs a nonlinear operation. In this method, all intensity values are sorted in an ascending order for pixels within the mask. Then the value of the middle pixel is selected as the new value for pixel ði; jÞ:
input image
filter window (3x3 case) 14 8 0 4 9 35 6 5 27
FIGURE 3.16 The concept of median filtering.
output image ordered pixels 0 4 5 6 8 9 14 27 35
Machine Vision: Visual Sensing and Image Processing
125
This process is repeated in a successive fashion until all pixel operations are undertaken. The figure shows a 3 £ 3 filter window arriving at a designated location. Putting all the intensity values in ascending order and taking the middle value among them yields the intensity value, “9”. For a 3 £ 3 mask, filtered output value G½f ði; jÞ can be written by G½f ði; jÞ ¼ Med{ f ði 2 1; j 2 1Þ; f ði; j 2 1Þ; f ði þ 1; j 2 1Þ; f ði 2 1; jÞ; f ði; jÞ; f ði þ 1; jÞ; f ði 2 1; j þ 1Þ; f ði; j þ 1Þ; f ði þ 1; j þ 1Þ}
ð3:13Þ
where Med{L} indicates the operator that takes the value of the middle pixel. Often 5 £ 5 and 9 £ 9 masks are used but their filtering methods are the same as that of the above 3 £ 3 operator. Let us consider the original natural image composed of a human face and a bookshelf as shown in Figure 3.17a in order to see the performance of the median filter. It contains salt and pepper noise of 20% intentionally added to the original. A 3 £ 3 median filter is operated on it. As can be seen from Figure 3.17b, the noise is almost removed. The result indicates that this filter is very effective to impulsive noise. In order to obtain the above filtered image by processing the median filter on this image, Matlab M-files are used. In the M-files we write Matlab-specific statements and execute a series of such written statements. Digital image processing methods using Matlab toolbox are well documented in Ref. [7]. Throughout this chapter we will use M-files to create the Matlab statements. The followings are the Matlab codes for obtaining median filtering (Box 3.1).
FIGURE 3.17 Median filtered image.
126
Optomechatronics
Box 3.1. FILTERING : MEDIAN FILTER Matlab source: Median Filter clear all % reading image File image ¼ imread(‘tao.bmp’); % add salt & pepper noise n Percent ¼ 20; [y, x ] ¼ size(image); nMaxHeight ¼ round(y p n Percent/100.0); nMaxWidth ¼ round(x p n Percent/100.0); for I ¼ 1:nMaxHeight, for J ¼ 1:nMaxWidth, cx ¼ round(rand(1) p (x 2 1)) þ 1; cy ¼ round(rand(1) p (y 2 1)) þ 1; aaa ¼ round(rand(1) p 255); if aaa . 128 image(cy,cx) ¼ 255; else image(cy,cx) ¼ 1; end end end % median filtering for i ¼ 1:x for j ¼ 1:y if(i ¼ ¼ 1 ll j ¼ ¼ 1 ll i ¼ ¼ x ll j ¼ ¼ y) % boundary image_out( j,i) ¼ image( j,i); % not changed else for l ¼ 1:3 for k ¼ 1:3 window(l þ (k 2 1) p 3) ¼ image( j þ l 2 2,i þ k 2 2); end end %sorting for l ¼ 1:8 for k ¼ 2:9 if (window(l) . window(k)) temp ¼ window(k); window(k) ¼ window(l); window(l) ¼ temp; end end
Machine Vision: Visual Sensing and Image Processing
127
end image_out( j,i) ¼ window(5); % medain value end end end % showing image figure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_out);
Image Segmentation Thresholding Thresholding is the simplest and still one of the most effective methods to segment objects from their background. Segmentation is an image processing method that splits the image into a number of regions, each having a high level of uniformity in brightness or texture. It reduces a gray scale image to binary black or white pixels. In the binary image, objects appear as black figures on a white background, or as white figures on a black background. One method to extract the objects from the background is to use a thresholding method in which the object and the background pixels have gray levels grouped into two dominant modes. Any point ðx; yÞ having gray level greater than a threshold value, T; that is, f ðx; yÞ . T; belongs to an object point while the point for f ðx; yÞ , T is called a background point. In this case, we need only one threshold value to classify the two, but in other cases, multiple thresholds may be needed when the image contains brightness information rather complex so that it cannot be ideally made binary. In the case of an image containing only two principal brightness regions, a global thresholding is the simplest technique, which partitions the image histogram by using a single threshold, T. Consider the gray level histogram shown in Figure 3.18a, which corresponds to an image f ðx; yÞ: It is composed of light objects on a dark background, having the object and the background pixels grouped into two dominant modes in terms of gray level. The thresholding problem is to find a threshold Tsuch that the two modes are well separated. If a thresholded image, gðx; yÞ; is defined by ( gðx; yÞ ¼
1
if f ðx; yÞ . T
0
if f ðx; yÞ # T
) ð3:14Þ
then the pixels labeled 1 correspond to the objects composed of the object points, whereas the pixels labeled 0 correspond to the background composed
Optomechatronics
number of pixels (N)
number of pixels (N)
128
(a) single threshold
T
T1
intensity
T2
(b) multiple threshold
intensity
FIGURE 3.18 Idealized gray level histogram.
of the background points. By labeling the image pixel by pixel in this way, segmentation is achieved, depending on whether the gray level of the pixel is greater or less than the T value. To illustrate the threshold technique, two scenes with a 256 £ 256 image will be considered. As shown in Figure 3.19, clearly the binary image of the
histogram
number of pixels
10000 background 5000 object 0
0
50
(a) the image of a tree leaf and its corresponding histogram
T1=180
T2 = 150
(b) the processed images with two different thresholds FIGURE 3.19 Thresholding process using single thresholding value.
100 150 intensity
200
250
Machine Vision: Visual Sensing and Image Processing
129
tree leaf differs due to variation of the threshold value. Consider the image of a tree leaf whose corresponding histogram is shown in Figure 3.19a. From the histogram, the leaf is seen to roughly have the intensity range of 0 to 150, while that of the background is 150 to 255. Based on the thresholding operation given in Equation 3.14, the image is thresholded with two different values of T as shown in Figure 3.19b. It can be seen that when the image is thresholded at T ¼ 180; a clear binary image can be obtained. However, as T moves toward a smaller value, for example, 150, the leaf image starts to be deteriorated, showing a slight difference from that of the original. This is because, in this case, the pixels having intensity larger than 150 are classified to background (white region) rather than to part (black). A more complicated scene is the latter case, of which the original image is shown in Figure 3.20a. Thresholding at T ¼ 128 is used to obtain its binary image. It can be observed that, due to the similar gray level among the objects, the background, and the shadow within the scene, the binary image shown in Figure 3.20b does not exactly represent the original face image, but somehow retains some important features. The following is the Matlab source code for obtaining such image (Box 3.2). Returning to Figure 3.18b it shows two types of objects on a dark background, which is a more general case. In this case, we need to define two thresholds T1 and T2 as shown in the figure. The three modes separated by T1 and T2 are: object 1
if f ðx; yÞ . T2
object 2
if T1 , f ðx; yÞ , T2
background
if f ðx; yÞ , T1
FIGURE 3.20 Gray-to-binary image transformation.
ð3:15Þ
130
Optomechatronics
Box 3.2. (THRESHOLD ) Matlab source: Threshold clear all % reading image File image ¼ imread(‘tao.bmp’); [y,x ] ¼ size(image); threshold ¼ 128; % defined by user % threshold for i ¼ 1:x for j ¼ 1:y if image( j,i) . 128 image_th( j,i) ¼ 255; else image_th( j,i) ¼ 0; end end end % showing image figure subplot(1,2,1), imshow(image) subplot(1,2,2), imshow(image_th)
The images having a multimode histogram of this type can be frequently found in most of the real scenes. For instance, when an IC chip is placed on a table, the gray level of this image may form three different distinct ranges; IC body, lead, and background. In this case, image analysis is more difficult than the case of the distinct histogram distribution. Choice of threshold value in this case needs to be made depending upon the information to be extracted from the image. Figure 3.21 shows the images of various electronic parts and bolts. The objects include three chips with white characters engraved on their top surfaces and the image is observed to contain several shaded areas around their bodies. As can be observed from the figure, the contour and the shape of each object become clearer at T ¼ 50 than those of the other two threshold values ðT ¼ 100; 150Þ; and the shadow effect is small. In the case of the electronic chip having several leads of brighter color, the contours are hardly seen from the image. When T is increased to 100, the white characters become clearly recognized and the shadow effect becomes significant. However, at T ¼ 150; the shadow significantly influences the whole object image and, thus, the object and the shadows can not be distinguished easily. This example clearly shows a choice of threshold values should be appropriately made depending upon the objective of the given task.
Machine Vision: Visual Sensing and Image Processing
131
FIGURE 3.21 Thresholding of a variety of electronic parts having different imaging characteristics.
In general, multilevel thresholding becomes a difficult task when the number of separation of regions of interest is large. An approach to this type of problems is to use a single, adaptive threshold. The difficulty also arises when an image is contaminated by some noise, which is frequently encountered in actual machine vision environments. In this case, optimal thresholding techniques can be used in such a way that the probability of the classification error, that is, erroneously classifying an object point as a background point and vice versa, is minimized with respect to T. Iterative Thresholding The previous discussions reveal that there is a need for an optimal thresholding technique which yields the best binary image quality. Some of the techniques to achieve this objective as well as to work for two principal brightness regions include: (1) Iterative threshold method (2) Optimal thresholding using maximum likelihood. The iterative method does not require an estimated function of a histogram and, therefore, works for more general cases, whereas the second method requires an analytical function of the histogram for optimization. In the following, the algorithms for the method will be introduced, based upon
132
Optomechatronics
heuristic approach. Let us assume that an image to be segmented has two brightness regions. The algorithm begins then: (1) Choose an initially estimated value of the threshold value T. (2) Partition the image into two groups R1 and R2 using the T value. (3) Calculate the mean gray value fR1 ðx; yÞ and fR2 ðx; yÞ of the partitioned group R1 and R2 ; respectively. X f ðx; yÞ fR1 ðx; yÞ ¼
fR2 ðx; yÞ ¼
ðx;yÞ[R1
Pixel number of the R1 region X f ðx; yÞ
ð3:16Þ
ðx;yÞ[R2
Pixel number of the R2 region
(4) Choose a new threshold to have T¼
fR1 ðx; yÞ þ fR2 ðx; yÞ 2
ð3:17Þ
(5) Repeat step (2) through step (4) until the mean gray values fR1 and fR2 do not change as iteration proceeds. 2500 frequency
2000 1500 1000 500 0
(a) original image
T= 100
0
50
100 150 intensity (b) histogram of the image
T= 130.7
(c) binary images obtained from the iterative thresholding process FIGURE 3.22 The iterative thresholding process.
200
T= 147
250
Machine Vision: Visual Sensing and Image Processing
133
This algorithm discussed above is illustrated with the example of the electronic part and electric resistance. To begin with, we choose the initial value of T ¼ 100; which is indicated by a histogram in Figure 3.22a. According to Equation 3.16 and Equation 3.17 we can calculate fR1 ðx; yÞ and fR2 ðx; yÞ They are determined by fR1 ðx; yÞ ¼ 186:1
fR2 ðx; yÞ ¼ 75:2
ð3:18Þ
which enables us to obtain a new threshold value. We then use a new value Tð1Þ ¼ 130:7 and its image is indicated in Figure 3.22b for the next iteration. Repeating in this way, we obtain Tð2Þ ¼ 142; Tð3Þ ¼ 145; and finally a steady state T value, Tð4Þ ¼ 147; as shown in Figure 3.22c. It can be seen that, for this single and simple object, the convergence rate is very fast, reaching its optimal value only after four iteration steps. Region-Based Segmentation In the previous section, thresholding was used to partition an image into the regions based on the distribution of the pixel intensity. Another way of segmenting the image is to find the regions directly. Region splitting and merging, and region growing belong to this segmentation category. Here, we will discuss a basic formulation based on splitting and merging technique. As shown in Figure 3.23, the image window contains seven different regions ðR1 ; R2 ; R3 ; R4 ; R5 ; R6 ; R7 Þ having different intensity color and texture. The purpose of this segmentation method is to divide the region into seven segments having identical properties. To generalize this concept, let us define R to represent the entire image region referring to the figure. Then, a segmentation is regarded as a process partitioning R into n subregions R1 ; R2 ; …; Rn : The conditions that must be satisfied during the process are then; ð1Þ
n
< Ri ¼ R
i¼1
ð2Þ Ri is a connected region; ði ¼ 1; 2; …nÞ: ð3Þ Ri > Rj ¼ f
;i and j; i – j:
ð4Þ PðRi Þ ¼ true;
i ¼ 1; 2; …n:
ð5Þ PðRi < Rj Þ ¼ false;
i–j
ð3:19Þ
In the above, f is the null set, and PðRi Þ is a logical predicate over the points in set Ri : Condition (1) implies that every pixel must be in a region while condition (2) indicates connectivity in that all points in a region must be connected. Condition (3) requires that there should be no jointed regions between regions Ri and Rj within R. In other words, they are disjointed between them. Condition (4) indicates that all points in Ri must have the
134
Optomechatronics
FIGURE 3.23 Seven divided different image regions.
same properties such as the intensity and the color. For instance, PðRi Þ ¼ True; if all pixels in Ri have an identical intensity. Condition (5) implies that the property of Ri must be different from that of Rj : Region splitting and merging is, therefore, to subdivide an image into a set of disjointed regions while satisfying the conditions given in the above. During this process, splitting and merging the regions are iteratively carried out. Consider an image frame R composed of four subregions, R1 ; R2 ; R3 ; R4 ; as shown in Figure 3.24. If R1 and R2 satisfy the conditions in Equation 3.19, where R3 and R4 do not, then R3 and R4 need to be segmented into eight subregions, each into four. Once again, these eight regions are segmented into smaller subregions. This segmentation process continues until all the subregions satisfy the conditions. To illustrate this basic concept, a binary image is depicted in the left hand side of Figure 3.25a, which consists of a single object (black) and the background (white). Within each region, the intensity will be assumed to be identical. Then, in the first step, we divide the image into four quadrants, R1 ; R2 ; R3 ; and R4 : We see that two quadrants R2 and R4 satisfy the predicate while the other two do not; PðR1 Þ ¼ false; PðR2 Þ ¼ true; PðR3 Þ ¼ false; PðR4 Þ ¼ true
ð3:20Þ
Therefore, PðR2 Þ and PðR4 Þ are not changed. In the next step, we further divide two “false” regions into subquadrants as shown in the figure. The followings hold for R1: PðR11 < R13 Þ ¼ true; PðR14 Þ ¼ true
ð3:21Þ
Machine Vision: Visual Sensing and Image Processing
R1
R2
R3
R4
R31
R32
R41
R42
R33
R34
R43
R44
135
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
331 333
332 334
341 341
341 341
431 431
431 431
441 441
441 441
(a) partioning the image region R R
R1
R2
R31
R4
R3
R32
R33
R331 R332 R333 R334
R34
R41
R42
R43
R341 R342 R343 R344 R431 R432 R433 R434
R44
R441 R442 R443 R444
(b) tree representation FIGURE 3.24 Segmentation for the split and merge algorithm.
Combining Equation 3.20 and Equation 3.21, we obtain PðR2 < R4 < R11 < R13 Þ ¼ true for “0” intensity level PðR14 Þ ¼ true for “1” intensity level Following the same procedure shown in the above for the subquadrant R13 in Figure 3.25b, we obtain PðR2 < R4 < R11 < R13 < R31 < R33 < R34 Þ ¼ true for “0” intensity level PðR14 UR32 Þ ¼ true for “1” intensity level
ð3:22Þ
One more segmentation step yields the following final conditions. PðR2 < R4 < R11 < R13 < R31 < R33 < R34 < R121 < R122 < R123 Þ ¼ true PðR14 < R32 < R124 Þ ¼ true
ð3:23Þ
The final result, which combines all of these results, is obtained by merging the regions satisfying the conditions specified in Equation 3.19.
136
Optomechatronics Segmentation R1
R2
split
(a) R11
split
(b)
R4
R3 R12
R13
R14
R31
R32
R33
R34
split
R121
R122
R123
R124
FIGURE 3.25 Illustration of the region based segmentation.
Edge Detection In the previous section, we have discussed various methods of image segmentation, discriminating one region from another in the image of interest. Edge detection is one such method for analyzing and identifying image contents, and presents the most common approach for detecting discontinuities in gray level. In this section, we will consider the detection of edges and discuss its basic concepts, the processing steps associated with edge detection, and the edge detectors. As illustrated in Figure 3.26, an edge in the image is the boundary between two regions with relatively distinct gray level value. In other words, the gray level difference is distinct, exhibiting a significant local change in the image intensity as shown in Figure 3.26a. The variation in the gray level f ðx; yÞ in the x-y image plane can be expressed as Gx and Gy in Figure 3.26b. As we go along from point A to point B through C in the image shown in Figure 3.26c, we can see that f ðx; yÞ slowly decreases to somewhere near point C. The gray level has a drastic change and afterwards it changes very little until it
Machine Vision: Visual Sensing and Image Processing
137
y
y
Gy
B
a Gx
C
C
A
G
x
(a) line A-B f(x,y)
x
(b) image f (x,y) G [ f ( x , y )]
A
C
B
(c) profile of a line A-B
A
C
B
(d) magnitude of first derivative
FIGURE 3.26 Edge characteristics.
hits point B. If f ðx; yÞ is differentiated with respect to spatial points along the line A – C – B, the magnitude of the derivative Gðx; yÞ is expressed as shown in Figure 3.26d. Clearly, the maximum value of Gðx; yÞ occurs at point C. This information provides a useful cue in edge detection, as we shall see later. In general, the edge varies with spatial location in the image. There are several standard edge profiles defined depending on its shape: step, ramp line, and roof. Almost all general edge profiles are composed of these standard profiles. Along with these profiles, there usually occurs discontinuity in the image intensity. To mathematically characterize the edge, let us consider the image frame shown in Figure 3.26a of which intensity distribution is defined by f ðx; yÞ: An edge is then characterized by the gradient of the image intensity having two components: magnitude and direction. The image gradient is defined by " G½ f ðx; yÞ ¼
Gx Gy
#
2 6 6 ¼6 4
›f ›x ›f ›y
3 7 7 7 5
ð3:24Þ
The magnitude of the gradient is expressed by qffiffiffiffiffiffiffiffiffiffiffi G½ f ðx; yÞ ¼ G2x þ G2y
ð3:25Þ
138
Optomechatronics
For simplicity of implementation, the gradient magnitude is approximated by addition of the gradient components as; G½ f ðx; yÞ ¼ lGx l þ lGy l
ð3:26Þ
where the direction of the gradient at location ðx; yÞ denoted by aðx; yÞ is given by
aðx; yÞ ¼ tan21
Gy Gx
ð3:27Þ
which is defined with respect to the x axis. It is noted that the above Equation 3.26 and Equation 3.27 require computation of the partial derivatives ›f =›x; ›f =›y at every pixel location. However, the edge taken by CCD camera is discrete in nature and, thus, needs to be numerically approximated by different equations as follows: Gx . f ði þ 1; jÞ 2 f ði; jÞ
Gy . f ði; j þ 1Þ 2 f ði; jÞ
ð3:28Þ
where i and j refer to the ith pixel in x direction and jth pixel in negative direction, respectively. As already discussed, in order to calculate all gradient magnitude values within the image frame of interest, we locate the operator at the left uppermost corner as an initial point and calculate a gradient magnitude at that location. To obtain the next value, we move the mask to the next pixel location. In other words, one value is obtained at each point. This procedure is repeated until the computation is completed for all pixels located within the frame. Gradient operators are in fact the convolution masks that enable us to compute the gradient vector of a given pixel and thus detecting edges, examining small neighborhoods. Table 3.2 summarizes the operators being used for edge detection. The operator approximating the first derivative (Roberts, Sobel) will be examined first and then the second derivative operator (Laplacian). And then the operator called Laplacian of Gaussian (LoG) which combines a Gaussian filter and Laplacian operator will be treated. The edge operators discussed in the above have different characteristics responding differently to edge and noise. For instance, some operators may be robust to noise but miss some critical edges while others may be sensitive to noise although they detect most edges. This necessitates a performance metric for edge detection operator. One such metric called the Pratl Figure of Merit is frequently used for the comparison purpose, but we will not deal with the further details here. Roberts Operator The Roberts operator is a very simple gradient operator that uses a 2 £ 2 neighborhood of the current pixel. The operator is identical to that shown in
Machine Vision: Visual Sensing and Image Processing
139
TABLE 3.2 Properties of Various Edge Operators Detection
Typical Operator/Mask
Roberts
Gmx = Pwett
1
0
0
−1
Gmy =
−1 −1 −1
Gmx = Sobel
1
0 −1
0
1
0
0
Gmy = −1
0
1
1
1
1
−1
0
1
−1
0
1
Gmy = −2 0
2
Gmx = 0
0
0
1
2
1
−3 −3 5
−1
0 −1
∆ Robinson compass mask
−1
−3 5
0
0 −1
0
−1 0
1
5
or
−1
8
−1
−1 −1 −1 −2 −1 0
… G = −1 0 −1 m7
Gmo = −2 0 2 −1 0 15
0
1
m1 m2
Magnitude: maximum of convolution of eight major compass masks Direction: maximum value of eight compass mask Rotationally symmetric direction: sign of the result from two adjacent locations
−1 −1 −1
4 −1
qffiffiffiffiffiffiffiffiffiffiffi Magnitude: m21 þ m22
1
−3 −3 −3
−3 −3 5
2=
0
qffiffiffiffiffiffiffiffiffiffiffi Magnitude: m21 þ m2 m Direction: tan21 2 m1
Direction: tan21
… G = −3 0 5 m7
Gmo = −3 0 5
Laplacian
Edge point only
0 −1
0
−1 −2 −1
Kirsch compass mask
Edge Magnitude/Direction
2
Magnitude: maximum of convolution of eight major compass masks Direction: maximum value of eight compass mask
Figure 3.14b and is written by
Gmx =
1
0
0
−1
Gmy =
0
−1
1
0
ð3:29Þ
Using this operator, Equation 3.26, and Equation 3.28, the magnitude of the gradient value at pixel ði; jÞ is obtained by G½ f ði; jÞ ¼ lf ði; jÞ 2 f ði þ 1; j þ 1Þl þ lf ði; j þ 1Þ 2 f ði þ 1; jÞl
ð3:30Þ
Note that, due to its geometric configuration, the above approximate difference values are assumed to be computed at the interpolated point
140
Optomechatronics
ði þ 1=2; j þ 1=2Þ: As can be observed, this operator is very sensitive to noise because only 2 £ 2 pixels are used to approximate the gradient. Sobel Operator This operator uses a 3 £ 3 mask that considers the neighborhood pixels for the gradient computation as shown in Figure 3.14c. It is expressed by
Gmx =
−1
0
1
−2
0
2
−1
0
1
Gmy =
1
2
1
0
0
0
−1
−2
−1
ð3:31Þ
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 85
85 255 255 255 255 255 255
255 255 255 255 255
85
41
41
255 255 255 255
85
41
0
0
41
255 255 255 85
85 255 255 255 255 255 85 25 5 255 255 255
41
0
0
0
0
41
255 255
85
41
0
0
0
0
0
0
85 255 255 255 41
85 255 255
255 255
41
0
0
0
0
0
0
0
0
41 255 255
255 255
85
85
85
85
85
85
85
85
85
85 255 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
(a) original mosaics mimicking a real image 0
0
0
0
0
0
0
0
0
0 -170 -170 170 170
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 -170 -510 -510 -170 0
0
0
0
0
0
0
0
0
0 -170 -554-812 812 -554 -170 0
0
0
0
0
0
0
0
0 -170 -554 -683 -469 -469 -683 -554 -170 0
0
0
0
0 -170 -554-683 -425 -208 -208 -425 -683 -554 -170 0
0
0 -170 -554 -384 384 554 170
0
0 -170 -554 -683 -299 299 683 554 170
0
0
0
0
0
0 -170 -554- 683-425 -126 126 425 683 554 170
0
0
0
0
0
0
0
0 -170 -554-6 83-425 -167 -41 41 167 425 683 554 170
0
0 -170 -554-683 -425 -167 -41 -41 -167 -425 -683 -554 -170 0
0 -554 -853 -425 -167 -41
0
0
41 167 425 853 554
0
0 -214 -513 -425 -167 -41
0 -768 -894 -167 -41
0
0
0
0
41 167 85 768
0
0
0 -554 -595 -41
0
0
0
0
0
0
41 595 554
0
0
214 683 979 102010201020102010201020 979 683 214
0
0 -170 -170 0
0
0
0
0
0
0
0
0
0
170 510 680 680 680 680 680 680 680 680 510 170
0
0
0
0
0
0
0
0
0
0
0
0
0
0
170 170 0
0
Gx (b) the gradient values Gx and Gy FIGURE 3.27 Sobel operation of a series of mosaic images.
0
0
0
0
-41 -167 -425 -513 -214 0
44 173 299 340 340 340 340 299 173 44
0
0
0
0
0
Gy
0
0
0
0
0
0
0
0
0
Machine Vision: Visual Sensing and Image Processing
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 0
0
0
0
0
240 537 537 240
0 0
240 783 898 898 783 240
0
0
0
0
0
0
141
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
240 783 965 556 556 965 783 240
240 783 965 601 243 243 601 965 783 240
0 0
0
0
0
0
240 537 537 240
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
240 783 898 898 783 240
240 783 965 556 556 965 783 240
240 783 965 601 243 243 601 965 783 240
0
240 783 965 601 236 57
57 236 601 965 783 240
0
0
240 783 965 601 236 57
57 236 601 965 783 240
0
0
593 995 601 236 57
0
57 236 601 995 593
0
0
593 995 601 236 57
0
57 236 601 995 593
0
0
768 895 240 301 340 340 340 340 301 240 895 768
0
0
768 895 240 301 340 340 340 340 301 240 895 768
0
0
593 905 979 102010201020102010201020 979 905 593
0
0
593 905 979 102010201020102010201020 979 905 593
0
0
240 537 680 680 680 680 680 680 680 680 537 240
0
0
240 537 680 680 680 680 680 680 680 680 537 240
0
0
0
0
0
0
0
0
0
0
0
(c)
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
(d)
|G|
0
0
0
240 537 537 240
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
240 783 965 556 556 965 783 240
240 783 965 601 243 243 601 965 783 240
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
T= 500
0
240 783 898 898 783 240
0
0
0
0
240 537 537 240
240 783 898 898 783 240
240 783 965 556 556 965 783 240
240 783 965 601 243 243 601 965 783 240
0
240 783 965 601 236 57
57 236 601 965 783 240
0
0
240 783 965 601 236 57
57 236 601 965 783 240
0
0
593 995 601 236 57
0
57 236 601 995 593
0
0
593 995 601 236 57
0
57 236 601 995 593
0
0
768 895 240 301 340 340 340 340 301 240 895 768
0
0
768 895 240 301 340 340 340 340 301 240 895 768
0
0
593 905 979 102010201020102010201020 979 905 593
0
0
593 905 979 102010201020102010201020 979 905 593
0
0
0
240 537 680 680 680 680 680 680 680 680 537 240
0
0
0
0 0
(e)
0
240 537 680 680 680 680 680 680 680 680 537 240 0
0
0
0
0
0
0
T= 800
0
0
0
0
0
(f)
0
0
0
0
0
0
0
0
0
0
0
0
0
0
T= 1000
FIGURE 3.27 Continued.
The magnitude of the gradient at the center pixel ði; jÞ is calculated in the following manner; Gx ¼ ð f13 þ 2f23 þ f33 Þ 2 ð f11 þ 2f21 þ f31 Þ Gy ¼ ð f11 þ 2f12 þ f13 Þ 2 ð f31 þ 2f32 þ f33 Þ
ð3:32Þ
where G values are easily computed by substituting Equation 3.32 into Equation 3.26. Some characteristics observed from this operator are as follows; First, the operator places an emphasis on pixels located closer to the center pixel. Second, due to this operator configuration, it provides both differencing and smoothing effects. As a result, this operator is one of the most popularly used edge operators. We will consider an example of obtaining the edges of an input image composed of mosaics by using the Sobel operator for a simple image similar to the triangle in Figure 3.27. In the figure the number inside each
142
Optomechatronics
FIGURE 3.28 Sobel and LoG operated images.
pixel represents the image intensity of which gray level is in the range of 0 to 255. By application of the Sobel operator, we wish to obtain its edges when the threshold value of Gðx; yÞ is kept at three different values, 500, 800, and 1000. Utilizing Equation 3.32, Gx ðx; yÞ and Gy ðx; yÞ can be computed and the results are shown in the figure. Combining these two values yields G½ f ði; jÞ as indicated in the figure. The result shows that when T ¼ 500, at the edge pixels, the gradient values are the largest, as expected. We can proceed to the same computational procedure using Equation 3.32 and arrive at the results shown in the figure. It is noted that the threshold value, T, greatly influences the determination of the edge: At T ¼ 800, the edge shape of the triangle is reserved but, at T ¼ 1000, it is completely lost. The Sobel operation is also carried out on the face image considered previously in Figure 3.20a. Figure 3.28a shows the edgedetected results obtained at T ¼ 60: We can see that the detected edges obtained are composed of thick lines and appear to well represent the original image. The Matlab codes to detect the edges by the Sobel operation are presented in the below (Box 3.3). Laplacian Operator So far, we have discussed the gradient operators that use the first derivative of the image intensity. The underlying principle of this method is that the peak of the first derivative occurs at the edge. The Laplacian operator, however, uses the notion that, at edge points where the first derivative becomes extreme, there will be zero crossing in the second derivative. To illustrate the concept, let us introduce the Laplacian of a 2D function f ðx; yÞ in
Machine Vision: Visual Sensing and Image Processing
143
Box 3.3. EDGE DETECTOR : SOBEL Matlab source: Sobel operation clear all % reading image File image ¼ imread(‘tao.bmp’); threshold ¼ 60; % defined by user [y,x ] ¼ size(image); imgs ¼ double(image);% changing data type % Sobel operation for i ¼ 1:x for j ¼ 1:y if(i ¼ ¼ 1 ll j ¼ ¼ 1 ll i ¼ ¼ x ll j ¼ ¼ y) % boundary image_sobel( j,i) ¼ 255; % white else sx ¼ 2 (imgs( j 2 1,i 2 1) þ 2 p imgs( j,i 2 1) þ imgs( j þ 1,i 2 1)) þ imgs( j 2 1,i þ 1) þ 2 p imgs( j,i þ 1) þ imgs( j þ 1,i þ 1); sy ¼ 2 (imgs( j 2 1,i 2 1) þ 2 p imgs( j 2 1,i) þ imgs( j 2 1,i þ 1)) þ imgs( j þ 1,i 2 1) þ 2 p imgs( j þ 1,i) þ imgs( j þ 1,i þ 1); M ¼ sqrt(sx^2 þ sy^2); if(M . threshold) image_sobel( j,i) ¼ 0; % black else image_sobel( j,i) ¼ 255; % white end end end end % showing image figure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_sobel);
an image frame, which is a second order derivative. It is defined as 72 f ðx; yÞ ¼
›2 f ›2 f þ ›x2 ›y2
ð3:33Þ
Consider a 1D edge profile shown in Figure 3.29a of which intensity varies along x direction. Its first and second derivatives are also shown in the figure. As can be observed in Figure 3.29b, the pixels having their first derivative
144
Optomechatronics
above a threshold shown in a dotted line will be all regarded as the edge. This, however, gives a somewhat rough estimation of the edge. If the second derivative is utilized as shown in Figure 3.29c, only a local maximum of the first derivative will be regarded as the edge because it becomes zero in this case. Therefore, all we need to do in the case of the Laplacian operator is to find a zero crossing of the second derivative. This implies that finding the zero crossing position is much easier and more accurate than finding an extreme value of the first derivative. This operator can also be expressed in a digitized numerical form by using different equations in the x and y directions,
›2 f ›Gx ¼ ¼ f ði þ 1; jÞ 2 2f ði; jÞ þ f ði 2 1; jÞ ›x ›x2
ð3:34Þ
›2 f ›Gy ¼ ¼ f ði; j þ 1Þ 2 2f ði; jÞ þ f ði; j 2 1Þ ›y ›y2
Adding these two equations according to the relation given in Equation 3.33 yields the Laplacian operator given in the following form
2=
0
1
0
1
−4
1
0
1
0
ð3:35Þ
∆
f (x, y)
(a)
x peak
f' (x, y)
(b)
a
b
threshold
x
f'' (x, y) zero crossing
(c) FIGURE 3.29 Laplacian operator finding zero crossing of f 00 (x,y).
x
Machine Vision: Visual Sensing and Image Processing
145
From the Laplacian operated image, we can find the edge by finding the position of zero crossing. It can be described as IF { f ½i; j . 0 AND ðf ½i þ 1; j , 0
OR
f ½i; j þ 1 , 0Þ}
{ f ½i; j , 0 AND ðf ½i þ 1; j . 0
OR
f ½i; j þ 1 . 0Þ}
OR ð3:36Þ
THEN f ½i; j ¼ edge Although the Laplacian operator responds very sharply to variation in the image intensity, it is very sensitive to noise due to its second derivative action. Therefore, the edge detection by finding the zero crossing of the second derivative of the image intensity may yield erroneous results. To avoid this the Laplacian of Gaussian (LoG) filter, in which the Gaussian filtering is combined together with the Laplacian operator, is popularly used to filter out noise before the edge enhancement. If the Gaussian filter is denoted by 2
2
gði; jÞ ¼ exp2ði þj Þ=2s
2
ð3:37Þ
Convoluting this equation with f ði; jÞ; we have ~ jÞ ¼ ½gði; jÞ p f ði; jÞ fði;
ð3:38Þ
The output of the LoG is then ~ jÞ ¼ 72 ½gði; jÞ p f ði; jÞ fði;
ð3:39Þ
The above notation implies the discrete version of the continuous function as shown in Equation 3.28. The derivative of convolution yields ~ jÞ ¼ ½72 gði; jÞ p f ði; jÞ fði; In the above the LoG is denoted by ! 2 2 2 i2 þ j2 2 2s2 2 7 gði; jÞ ¼ exp2ði þj Þ=2s 4 s
ð3:40Þ
ð3:41Þ
The edge detection using the LoG filter is summarized in the following three steps: (1) Smoothing operation by a Gaussian filter (2) Enhancing the edges using Laplacian (3) Detecting zero crossing. The same face image discussed in Figure 3.20a is used here to see the effect of the LoG filtering. Its parameter has a mean value ¼ 0 and s ¼ 2.0.
146
Optomechatronics
The filtered result is shown in Figure 3.28b. The image is slightly different from the image obtained by Sobel in that there are many thin lines appearing in the image. This is because the image produced by the LoG operator is the result of the edge detection together with the thinning operation. The Matlab codes for this filter are listed in the below (Box 3.4). Box 3.4. EDGE DETECTOR : LAPLACIAN
OF
GAUSSIAN
Matlab source: Laplacian of Gaussian Operation clear all % reading image file image ¼ imread(‘tao.bmp’); [y,x ] ¼ size(image); % image size % making a log filter mask N ¼ 13; % filter size sigma ¼ 2.0; % sigma half_filter_size ¼ round(N/2); siz ¼ (N 2 1)/2; std2 ¼ sigma^2; [xx,yy ] ¼ meshgrid(2 siz:siz, 2 siz:siz); arg ¼ 2 (xx. p xx þ yy. p yy)/(2 p std2); h ¼ exp(arg); h(h , eps p max(h(:))) ¼ 0; sumh ¼ sum(h(:)); if sumh , ¼ 0, h ¼ h/sumh; end; % now calculate Laplacian h1 ¼ h. p (x. p x þ y. p y 2 2 p std2)/(std2^2); op ¼ h1 2 sum(h1(:))/prod(N); % make the filter sum to zero op ¼ op 2 sum(op(:))/prod(size(op)); % make the op to % sum to zero imgs ¼ double(image);% changing data type % Laplacian operation for i ¼ 1:x for j ¼ 1:y if(i , half_filter_size ll j , half_filter_size ll i . x-half_filter_size-1 ll j . y-half_filter_size-1) % boundary imgs2( j,i) ¼ 255; % white else M ¼ 0; for k ¼ 1:N
Machine Vision: Visual Sensing and Image Processing
147
for l ¼ 1:N M ¼ M þ op(k,l) p imgs( j þ k-half_filter_size, i þ l-half_filter_size þ 2); end end imgs2( j,i) ¼ M; end end end m ¼ y 2 1; n ¼ x 2 1; % image size rr ¼ 2:m 2 1; cc ¼ 2:n 2 1; % The output edge map: e ¼ repmat(false, m, n); b ¼ repmat(false, m, n); b ¼ imgs2; thresh ¼ 0.15 p mean2(abs(b(rr,cc))); %thresh ¼ 1.0; % Look for the zero crossings: þ 2 , 2 þ and their transposes % We arbitrarily choose the edge to be the negative point [rx,cx ] ¼ find(b(rr,cc) , 0 & b(rr,cc þ 1) . 0… &abs(b(rr,cc) 2 b(rr,cc þ 1)) . thresh); % [2 þ ] e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ find(b(rr,cc 2 1) . 0 & b(rr,cc) , 0… &abs(b(rr,cc 2 1) 2 b(rr,cc)) . thresh); % [þ 2 ] e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ find(b(rr,cc) , 0 & b(rr þ 1,cc) . 0… &abs(b(rr,cc) 2 b(rr þ 1,cc)) . thresh); % [2 þ ]0 e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ find(b(rr 2 1,cc) . 0 & b(rr,cc) , 0… &abs(b(rr 2 1,cc) 2 b(rr,cc)) . thresh); % [þ 2 ]0 e((rx þ 1) þ cx p m) ¼ 1; % Most likely this covers all of the cases. Just check to see if there % are any points where the LoG was precisely zero: [rz,cz ] ¼ find(b(rr,cc) ¼ ¼ 0); if , isempty(rz) % Look for the zero crossings: þ 0 2 , 2 0 þ and their %transposes % The edge lies on the Zero point zero ¼ (rz þ 1) þ cz p m; % Linear index for zero points zz ¼ find(b(zero 2 1) , 0 & b(zero þ 1) . 0… &abs(b(zero 2 1) 2 b(zero þ 1)) . 2 p thresh); % [2 0 þ ]0 e(zero(zz)) ¼ 1; zz ¼ find(b(zero 2 1) . 0 & b(zero þ 1) , 0… &abs(b(zero 2 1) 2 b(zero þ 1)) . 2 p thresh); % [þ 0 2 ]0
148
Optomechatronics
e(zero(zz)) ¼ 1; zz ¼ find(b(zero 2 m) , 0 & b(zero þ m) . 0… &abs(b(zero 2 m) 2 b(zero þ m)) . 2 p thresh); % [2 0 þ ] e(zero(zz)) ¼ 1; zz ¼ find(b(zero 2 m) . 0 & b(zero þ m) , 0… &abs(b(zero 2 m) 2 b(zero þ m)) . 2 p thresh); % [þ 0 2 ] e(zero(zz)) ¼ 1; end % normalization image_laplacian ¼ (1 2 e) p 255; % showing image figure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_laplacian);
Hough Transform The Hough transform, developed by Hough (1962) is an effective way of segmenting objects with known shape and size within an image. It is designed specifically to find lines. An advantage of this approach is the robustness of segmentation and the result of the presence of imperfect data or noise. To introduce the underlying concepts of the transform, let us consider the simple problem of detecting a straight line in an image. Referring to Figure 3.30a, let us suppose that a straight line AB composed of infinitely many points including Pi ¼ ðxi ; yi Þ: Let the general equation of the line passing through the point Pi be yi ¼ mxi þ c On the other hand, infinitely
c
image space Pi+n
(xi+2, yi+2) (xi+1, yi+1) (xi,yi)
Pi
parameter m-c space …
(xi+n, yi+n) y = mx + c c = –mx + y
(a) multiple points on a line FIGURE 3.30 A line defined in two different spaces.
x
c = –Xi+nm + Yi+n
…
y
c'
m'
c = –Xim + Yi c = –Xi+1m + Yi+1 c = –Xi+2m + Yi+2 m
(b) the corresponding points mapped into parameter space
Machine Vision: Visual Sensing and Image Processing
149
many lines will pass through ðxi ; yi Þ as long as it will satisfy the equation yi ¼ mxi þ c
ð3:42Þ
for some values of m and c. This implies that the above equation can also be defined in the parameter space composed of the parameters m and c. For example, all the straight lines going through the point Pi can be represented in the m – c space by c ¼ 2xi m þ yi
ð3:43Þ
Similarly, straight lines going through the point Piþn can be described in the same space by c ¼ 2xiþn m þ yiþn
ð3:44Þ
If those two points were on the same line in the image, the two line equation will be intersected at a point denoted by a dot ðm0 ; c0 Þ as indicated in Figure 3.30b. This indicates that any straight line in the image is mapped to a single point in the m – c parameter space, and any part of this line is mapped into the same point. Lines of any direction in an image may pass through any of edge pixels. In some cases, if we consider the slope and the intercept value of those lines to be bounded within a certain range ðmmax ; mmin Þ and ðcmax ; cmin Þ, the parameter space may be digitized into a subdivision of the space, the so called accumulator array composed of the cells of a rectangular structure as shown in Figure 3.31. These cells clearly depict the relation between the image space (x, y) and the parameter space ðm; cÞ: A(0,L-1)
A(K-2, L-1)
c
A(K-1, L-1) cmax
… A(K-1, L-2)
…
…
… A(0,1) cmin
A(0,0) A(1,0) mmin
FIGURE 3.31 Discretization of parameter space (m, c).
A(K-1,0)
… mmax
m
150
Optomechatronics
A cell Aðm; cÞ has the parameter values m and c in the parameter space. For each image point ðxl ; yl Þ in the image plane, when the lines of the allowed directions pass through this pixel, their corresponding parameters m and c are determined by the equation c ¼ 2xl m þ yl , yielding different Aðm; cÞ values. If a line is present in the image in the form of the equation y ¼ m0 x þ c 0 , the value of the Aðm0 ; c 0 Þ will be increased many times. For instance, if the points composing a line are detected M times, the value in Aðm; cÞ will be M, which indicates that M points in the image plane lie on the line y ¼ m0 x þ c 0 : Therefore, lines existing in the image will produce large values of the appropriate cells. This results in local maxima in the accumulator space. In the following, the Hough algorithm for line detection is summarized: Step 1: Quantize the parameter space (m, c) within the range mmin # m # mmax and cmin # c # cmax : Step 2: Form an accumulator array Aðm; cÞ and initialize it to some value. Step 3: Increment the value of an accumulator array Aðm; cÞ in relation to the points composing lines in the image. It is noted that the accuracy of the colinearity of the points lying on a line is dependent upon the number of discrete cells in the ðm; cÞ plane. If, for every point ðxl ; yl Þ, the m axis is quantized into K number, then the number of the corresponding c values will be K. This requires nK computation for each cell of Aðm; cÞ when n points in the image are involved. It is also noted that the larger the K value, the lower the resolution in the image space. The next task is to select a threshold that serves as a criterion to decide whether the value contained in a specific block can be a line or not. For this, we then examine the quantized blocks that contain more points than the threshold value. The blocks that pass this criterion are marked as a line in the processed image. The Hough transform method suffers when the line approaches the vertical (as m ! 1 and c ! 1). One way to avoid this difficulty is to represent a line as x sin u þ y cos u ¼ r
ð3:45Þ
which is depicted in Figure 3.32a. Construction of the accumulator array, in this case, is identical to the previous method used for slope – intercept representation (m, c). Here, again, a straight line in the image plane is transformed to a single point in the ðr; uÞ plane as shown in Figure 3.32b. Suppose that L collinear points are lying on a line x cos ui þ y sin ui ¼ r . This yields L sinusoidal curves that intersect at ðri ; ui Þ in the parameter space. It is noted here that the range of u lies within þ 908 and 2 908, and ranges from 0 to N where N is the image size, N £ N. When u ¼ 0, a line is horizontal and has a positive r . Similarly, when u ¼ 908, the line is vertical and has a positive r in þ x direction and when u ¼ 2908, r has a negative x-intercept.
Machine Vision: Visual Sensing and Image Processing
ρ
y
151
A(0,L-1)
A(K-2, L-1) A(K-1, L-1) …
rmax
A(K-1, L-2) …
…
θ
…
ρ
x
(a) parameter space
rmin
A(0,1) A(0,0) A(1,0)
…
qmin
qmax
A(K-1,0) θ
(b) accumulation cell
FIGURE 3.32 Hough transform in (r, u) plane.
Hough transform to ðr; uÞ the plane follows the same procedure as discussed in the case of ðm; cÞ plane. The only difference is to use Equation 3.45 in order to determine r and u for each pair of ðx; yÞ in the image. Let us take two examples to understand how the transform actually works on the actual images. One example is made of a synthetic image composed of several polygons and the other is a real image of a camera placed upside down. Figure 3.33 illustrates the first example image in which 11 line edges are obtained by an edge operator. To these line images, the Hough transform algorithm is applied and the transformed result is plotted in r – u space. If those lines were perfectly represented by a line, they should be plotted as points in the space. However, the figure shows several streaks instead of points, since the extracted line edges are not a thin line and contain some noise as a result of edge processing. In the last figure, the original lines are successfully reproduced by checking the cells in the accumulator that has the local maximum value. Figure 3.34 shows an image of a camera and its frame from which we wish to find out some straight lines. It is noted from the extracted edges that the image contains several curved lines as well as straight lines. In addition, there appear several broken lines and noises. These lines and noises are all transformed into Hough r – u space. From this result, the straight lines are reproduced in the last figure. We can see that only a few lines are found from the transform although there appear many lines in the original input image. This is because the extracted edges contain noises and thick lines, in addition to circular arcs. The main advantage of using Hough transform lies in the fact that it is insensitive to missing parts of lines, image noise, and other parts of the image, exhibiting nonline characteristics. For example, a noisy or rough straight line will not yield a point in the parameter space but be transformed into a cluster of points. In this way, these are discriminated from straight lines in the image.
152
Optomechatronics
(b) extracted edges
(a) original image
50
50
100
100
150
150
200
200
250
250
300
300
350
350 50
100
150
200
250
(c) the result of Hough transformation
300
350
400
50 100 150 200 250 300 350 400
(d) detected lines
FIGURE 3.33 A Hough transformation of the image composed of polygon objects.
Camera Calibration In the pinhole camera model, a small hole is punched at the optical lens center in the camera coordinate system, through which some of the rays of light reflected by the object pass to form an inverted image of the object on the image plane. This is a camera perspective model as shown in Figure 3.35a, as discussed at the beginning of this chapter. Here, the camera lens is drawn in a dotted line to neglect its presence. A pinhole does not focus and, in fact, limits the entrance of incoming light, requiring long exposure time. No focusing required means that all objects are in focus, which, in turn, implies that the depth of field of the camera is ideally unlimited. However, the pinhole to film (image plane) distance affects the sharpness of the image and the field of view. In actual cameras, the pinhole is sufficiently opened by using a converging lens to avoid the disadvantage of the pinhole model as shown in Figure 3.35b. However, the actual lens
Machine Vision: Visual Sensing and Image Processing
153
(b) extracted edges
(a) original image
50
50
100
100
150 150
200 250
200
300
250
350
50
100
150
200
250
300
(c) the result of Hough transformation
350
300
50
100
150
(d) detected lines
FIGURE 3.34 A Hough transformation of a real object image.
model causes focusing and aberration problems in reality. A difference from the previous model is that, in this model, the lens focuses and projects light rays from an object into the image plane, which is located in a specified distance from the camera lens. Disregarding aberration problem, let us consider the focus variation of the image plane. If an object is located at z ¼ s from the lens as shown in the figure, the following Gaussian lens formula holds 1 1 1 ¼ 0 þ f f s
ð3:46Þ
where f and f 0 are the focal length of the lens and the distance of the image plane from the center point of the lens, respectively. This f is called the effective focal length of the camera or camera constant. As we can see, when
154
Optomechatronics
virtual image object optical center
image sensor
zc real image real image
(a) pin hole model
optical lens
world point P
image plane zc image point P’
f
s
f’
(b) lens model FIGURE 3.35 The perspective camera model.
the object distance, s, varies from infinity to some distance close to the lens, f 0 slightly deviates from the focal length, f : This deviation may cause an error of projection, which cannot be tolerated in the case of the accurate depth measurement at close range. In this case, this parameter must be accurately determined by a camera calibration technique. To analyze the imaging process with a real image rather than the inverted one, we will deal with a virtual image plane denoted in Figure 3.35a. For simplicity, we call this “image plane” in the sequel. The objective of the image analysis is then how we map the image of an object acquired in the image plane into the world coordinate system. Referring to Appendix A1, the mapping conveniently can be described in the following form. up ¼ ½Mp Xw
ð3:47Þ
where up is the image vector of the object described in the image pixel coordinates and is projected from the 3D world coordinates Xw, and Mp is the resulting projection matrix. This is obtained by a perspective model discussed at the beginning of this chapter.
Machine Vision: Visual Sensing and Image Processing
155
Perspective Projection A perspective transformation called the imaging transformation projects 3D points on to a plane. In other words, the transformation changes a 3D aggregate of the objects into a plane surface. To describe this in detail, we need to define four coordinate systems; With reference to Figure 3.36, the first one is the world coordinate system, {Xw } where an object to be imaged in 3D scene is located, the second one is the camera coordinate system {Xc } of which origin is located at the optical lens center, the third is the pixel coordinate system, {up } denoted by the coordinates of the pixels in the digital image, and finally, the fourth is the image coordinate system {ui }; which describes the same image in a different manner from the one described in the pixel coordinate. As shown in Figure 3.37, the image coordinates ðui ; vi Þ are located at the center of the pixel coordinate system ðu0 ; v0 Þ: As shown in the figure, the relationship between the pixel coordinate and image coordinate systems is, therefore, given by up ¼ u0 þ ku ui
vp ¼ v0 þ k v vi
ð3:48Þ
where ku and kv are the inverse of the horizontal and the vertical effective pixel sizes, su and sv, respectively, ku and kv are expressed in units of pixel £ m 21 while su and sv are interpreted as the size in meters for the horizontal and vertical pixels, respectively. Utilizing these coordinate systems, the whole imaging process can be described in the following sequence: coordinate system : coordinate points :
{Xw } 2 3 Xw 6 7 6 Yw 7 4 5
!
{Xc } 2 3 Xc 6 7 6 Yc 7 4 5
!
Zw
Yc
Yw
Xw
Zw Ow world coordinates
{ui } "
!
Zc
Xc
Oc camera coordinate
!
oi
op
up
FIGURE 3.36 The coordinate system for image analysis.
vi
#
{up } "
!
up
#
ð3:49Þ
vp
object (Xc,Yc, Zc) world point: P
(ui,vi)
vi
vp
ui
!
ui
optical axis
virtual image plane
Zc
156
Optomechatronics vi
(uo,vo) o image coordinates
vp
pixel coordinates
ui
up
ku : 1/ su kv : 1/ sv su : horizontal effective pixel size
pixel
sv
su
sv : vertical effective pixel size FIGURE 3.37 Pixel and image coordinates.
The first step to obtain the image interpretation is to obtain a relationship when a point ðXc ; Yc ; Zc Þ in the camera coordinate systems is projected onto the image coordinates ðui ; vi Þ: As indicated in Figure 3.36, under the assumption of the ideal projection, the transformation can be described by the projection equation ui ¼
f Xc ; Zc
vi ¼
f Yc Zc
ð3:50Þ
where f is the focal length of the camera. Note that these equations are nonlinear, since the depth variable Zc is at the denominator of each equation. These relationships can be utilized directly to avoid any complexity that arises from the nonlinearity. We may use alternatively homogeneous coordinates, which enable us to deal them in linear matrix from. As discussed in Appendix A1, the homogeneous coordinates of a point with respect to camera coordinates ðXc ; Yc ; Zc Þ are defined as ðlXc ; lYc ; lZc ; lÞ, where l is a nonzero constant. Conversion of homogeneous coordinates back to Cartesian coordinates can be made by dividing the first three homogeneous coordinates by l. Similarly, we can define the homogeneous coordinates of a point with the pixel coordinates ðup ; vp Þ by ðsui ; svi ; sÞ, where s is a nonzero constant. Then, the relationship in Equation 3.50 called
Machine Vision: Visual Sensing and Image Processing
157
perspective matrix can be conveniently expressed in linear form, 2 3 3 Xc 3 2 2 7 f 0 0 0 6 sui 6 7 76 Yc 7 7 6 6 6 svi 7 ¼ 6 0 f 0 0 76 7 ~c or u~ i ¼ ½Hc X 56 7 5 4 4 6 Zc 7 0 0 1 0 4 5 s 1
ð3:51Þ
~ c in the where u˜i and X˜c are the augmented coordinate vectors of u~ i and X homogenous coordinate space. Using the rigid body transformation, Xc can be related to the world coordinates by the equation 2
lXc
3
2
r11
r12
r13
r22
r23
TX
32
XW
3
2
r T1
76 7 6 6 7 6 TY 7 76 YW 7 6 r T2 76 7¼6 76 7 6 6 ZW 7 6 r T3 r32 r33 TZ 7 54 5 4 0 0 1 1 0T " # .. ~ c ¼ ½Hw X ~ w ¼ ·R· · · .· · ·T· · X ~w X . 0 .. 1 6 7 6 6 lY 7 6 r 6 c 7 6 21 6 7¼6 6 7 6 6 lZc 7 6 r31 4 5 4 0 l
TX
32
XW
3
76 7 76 7 TY 76 YW 7 76 7 76 7 6 ZW 7 TZ 7 54 5 1 1
or ð3:52Þ
where R denotes the 3D rotation matrix, T denotes the 3D translational vector, and ri (i ¼ 1,2,3) is a row vector. Finally, from the relationship given in Equation 3.52, the relationship between the image and the pixel coordinates in the homogeneous coordinate space, 32 3 3 2 2 ui ku 0 u 0 sup 76 7 7 6 6 6 svp 7 ¼ 6 0 kv v0 76 vi 7 or u~ p ¼ ½Hu u~ i ð3:53Þ 54 5 5 4 4 s
0
0
1
1
Therefore, the overall imaging process is described by the relationship, using Equation 3.50 through Equation 3.53. 32 3 2 T XW 2 32 3 r 1 TX 7 76 ku 0 u0 f 0 0 0 6 76 7 6 6 76 76 r T2 TY 76 YW 7 7 6 7 7 6 7 6 u~ p ¼ 6 0 k v 0 f 0 0 or v 0 54 4 56 T 7 76 ð3:54Þ 6 r 3 TZ 76 ZW 7 54 5 0 0 1 0 0 1 0 4 1 0T 1 ~W u~ p ¼ ½H X where [H] is the 3 £ 4 camera projection matrix, which is denoted by 3 2 au 0 u0 7 6 7 ð3:55Þ H ¼ ½C ½R ½T ¼ 6 4 0 av v0 5½R ½T 0
0
1
158
Optomechatronics
where au and av are the image scaling factors, given by au ¼ ku f and av ¼ kv f ; and aspect ratio is given by au =av ¼ ku =kv : The above relationship indicates that au and av are dependent upon the distance of the focal length and the sizes of the pixels in the horizontal and vertical directions, respectively. It is noted that the parameters au ; av ; u0 ; and v0 do not depend on the position and orientation of the camera, and are, therefore, called the intrinsic parameters. The above equation can be rewritten by 2
H11
H14
3
6 u~ p ¼ 6 4 H21
H12
H13
H22
H23
7 ~ H24 7 5Xw
H31
H32
H33
H34
ð3:56Þ
When the above 12 elements are scaled by the value H34 , we have 11 variables to determine. The resulting equation is obtained by 3
2
2
up 7 6 0 6 7 6 s0 6 4 vp 5 ¼ 4 H21
H012
H013
H022
H023
H031
H032
H033
1
H011
3 2 3 Xw 7 6 6Y 7 7 6 7 w 7 6 H024 7 7 56 6 Zw 7 5 4 1 1
H014
ð3:57Þ
where s0 is given by s=H34 : This yields the camera calibration equation from which the relationship between the pixel coordinates and the world coordinates can be established. To illustrate the relationship in more detail, let us consider a point, say ith point, in the world coordinates {X} and the corresponding point imaged in the pixel coordinates {up }: Use of Equation 3.57 leads to 3 H011 7 6 6 0 7 6 H12 7 7 6 7 6 6 0 7 6 H13 7 7 6 7 6 6 0 7 6 H14 7 7 6 7 6 6 0 36 H21 7 7 2 3 i i 7 6 2up Zw 6 u 7 76 0 7 4 p 5 56 H22 7 ¼ 7 6 vp 7 2vip Ziw 6 6 0 7 6 H23 7 7 6 7 6 6 0 7 6 H24 7 7 6 7 6 6 H0 7 6 31 7 7 6 7 6 6 H0 7 6 32 7 5 4 2
2 6 4
i Xw Yiw Ziw 1
0
0
0
0
0
0
i 0 2uip Xw 2uip Yiw
i i 0 Xw Yiw Ziw 1 2vip Xw 2vip Yiw
H033
ð3:58Þ
Machine Vision: Visual Sensing and Image Processing
159
In deriving the above equation, we have used the following relationship; s0 ¼ H031 Xw þ H032 Yw þ H033 Zw þ 1 which can be obtained from Equation 3.57. It needs to be pointed out that, at least, we need 12 equations like these that can be obtained by six calibration points. The equation can be written in a more compact form, Aq ¼ b
ð3:59Þ
where A represents the left-hand side matrix of the H vector and b represents the right-hand side of the equation. Solving for q yields q ¼ ½AT A
21
AT b ¼ Aþ b
ð3:60Þ
where Aþ is the pseudo inverse of A. Here, we have 11 unknowns for qs to be determined and, therefore, need at least six calibration points. Before we proceed to solve Equation 3.60, let us consider the transformation from a point in the image coordinates Pi to a point in world coordinates without a camera perspective model. Referring to Figure 3.38, this can be obtained by considering the absolute position vector of point Pw ðxw ; yw ; zw Þ with respect to the world coordinates, which is equivalent to point Pi ðui ; vi Þ: The position vector is expressed by rw ¼ rc þ ri þ rp
ð3:61Þ
where rw is the absolute position vector of point Pw expressed with respect to the world coordinates, rc is the position vector of the center of the camera with respect to the center of the world, ri is the position vector of the image coordinates with respect to the camera center Oc, and finally rp is the position vector of Pi with respect to the center of the image coordinates, Oi : The camera shown in the figure has usually two rotational motions denoted by the angles a and b, which are called, respectively, tilt and panning. The sequence of positioning point Pi in the image coordinates to the corresponding point Pw in the world coordinates is to obtain the camera transformation matrix in Equation 3.54. The first translation gives 2
1
6 60 6 H1 ¼ 6 6 60 4 0
0 0 1 0 0 1 0 0
xw
3
7 yw 7 7 7 7 zw 7 5 1
ð3:62Þ
160
Optomechatronics Zc β
image coordinates {I}
pan-tilt device Xc α camera coordinates {C}
rp
ri Oi
Oc
Yc
Pi (ui ,vi ,0)
u
v
z optical axis
rc Zw
Ow Yw
Pw(xw, yw, zw)
Xw
world coordinates {W }
FIGURE 3.38 Geometry for camera coordinate transformation.
where xw, yw and zw are the positioning vectors of the center of the camera coordinates. The two rotations, tilting and panning, yield the following transformation: For panning (b) 2
cos b
6 6 sin b 6 H2 ¼ 6 6 6 0 4 0
2sin b
0
cos b
0
0
1
0
0
0
3
7 07 7 7 7 07 5 1
ð3:63Þ
Machine Vision: Visual Sensing and Image Processing
For tilting (a) 2 1 0 6 6 0 cos a 6 H3 ¼ 6 6 6 0 sin a 4 0
0
0 2sin a cos a 0
0
161
3
7 07 7 7 7 07 5 1
ð3:64Þ
The translation from the origin of the camera coordinates with respect to that of the image coordinate is expressed by 3 2 1 0 0 xi 7 6 60 1 0 y 7 6 i7 7 ð3:65Þ H4 ¼ 6 7 6 6 0 0 1 zi 7 5 4 0 0 0 1 where ðxi ; yi ; zi Þ is the position vector of the image coordinate center. As can be seen from the figure, the orientation of the image coordinates is different from that of the camera coordinates. Therefore, the frame needs to be rotated 908 about Zc axis, which yields 2 3 0 21 0 0 6 7 61 0 0 07 6 7 7 H5 ¼ 6 ð3:66Þ 6 7 60 0 1 07 4 5 0 0 0 1 Then, another rotation about YC yields 2 3 0 0 1 0 6 7 6 0 1 0 07 6 7 7 H6 ¼ 6 6 7 6 21 0 0 0 7 4 5 0 0 0 1
ð3:67Þ
If i rp is the position vector of Pi , the transformation of the Pi into the Pw is obtained by w
rp ¼ ½H1 ½H2 ½H3 ½H4 ½H5 ½H6 i rp
ð3:68Þ
This yields the relationship between Pw ðxw ; yw ; zw Þ in the world coordinates and Pi ðui ; vi ; 0Þ in the image plane. We have seen that a point defined in the world coordinates (known) can be mapped into a point defined in the image coordinates (measured) from Equation 3.68. To obtain the transformation matrix H, we need to solve Equation 3.58 or Hij s. However, H contains the intrinsic and the extrinsic parameters, which are
162
Optomechatronics
difficult to obtain by a direct measurement. Furthermore, at least six points located in the world coordinates and the corresponding points in the image coordinates need to be exactly known. This necessitates a measurement technique called “camera calibration.” The calibration method requires geometrically known points called calibration marks. The choice of the marks depends upon the conditions that the camera calibration needs to be carried out, for example, the light conditions, the coloring of the mark and the accuracy of the calibration. The choice of the marks depends upon the conditions under which the camera calibration condition is carried out. Such conditions include the light conditions, the coloring of the mark, and the accuracy of the calibration. Figure 3.39 shows an actual geometrical configuration set for the camera calibration. The calibration rig is composed of an object to be calibrated, a CCD camera, and an optical rail on which the camera can travel within a certain distance. The object contains an evenly spaced pattern of circles filled in white and is located in a known location along the Z axis from the camera position of which center is at Oc The first step is to obtain the image of the pattern and calculate the locations of the center of the circle given in the pixel coordinates, based on the acquired image. Figure 3.40 shows two acquired images taken by the camera at two different locations at Zw ¼ 0 mm and Zw ¼ 250 mm. In this procedure, any information on the circle such as the features of the circle must be given for the calculation. Here, the centroids of the circles are used. As indicated in Figure 3.41, to determine this, we need to use the entire image processing procedures discussed in the previous section, selection of region of interest (ROI), image segmentation, and enhancement, detection, and determination of the mass center of each circle. Table 3.3 indicates the coordinate values of the centroids of the circles with respect to the world coordinate system.
Yw
Xw
Xc
camera
ical
optical center, Oc
opt Yc
FIGURE 3.39 Camera calibration set-up.
Zw
Zc
rail
Machine Vision: Visual Sensing and Image Processing
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
(a) image 1 ( Zw=0mm, “circle_0.raw ”)
163
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
(b) image 2 ( Zw=250mm, “circle_250.raw”)
FIGURE 3.40 The pattern images acquired at two different locations.
Utilizing the locations of the circles obtained in the pixel coordinate values and the world coordinate values, we wish to compute the intrinsic parameters and the calibration transformation matrix given in Equation 3.54. Plugging these data into Equation 3.54, we obtain the following calibration
input image
selection of ROI (region of interest)
segmentation by labeling
edge detection
mass center using edge of each circle
FIGURE 3.41 The procedure to determine the centers of the circle patterns.
164
Optomechatronics
TABLE 3.3 The Centroid of the Circles with respect to World Coordinate System Feature Point
X (mm)
Y (mm)
1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 1-10 1-11 1-12 1-13 1-14 1-15 1-16 1-17 1-18 1-19 1-20 1-21
2120 280 240 0 40 80 120 2120 280 240 0 40 80 120 2120 280 240 0 40 80 120
200 200 200 200 200 200 200 160 160 160 160 160 160 160 120 120 120 120 120 120 120
2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 2-10
280 240 0 40 80 280 240 0 40 80
160 160 160 160 160 120 120 120 120 120
matrix H. 2 0 H11 6 6 0 H¼6 6 H21 4 H031
H012
H013
H022
H023
H032
H033
H014
Z (mm) Feature Point Image 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Image 2 250 250 250 250 250 250 250 250 250 250
3
2
1:8813
7 6 7 6 0:0021 H024 7 7¼6 5 4 0 1
X (mm)
Y (mm)
1-22 1-23 1-24 1-25 1-26 1-27 1-28 1-29 1-30 1-31 1-32 1-33 1-34 1-35 1-36 1-37 1-38 1-39 1-40 1-41 1-42
2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120
80 80 80 80 80 80 80 40 40 40 40 40 40 40 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2-11 2-12 2-13 2-14 2-15 2-16 2-17 2-18 2-19 2-20
2 80 2 40 0 40 80 2 80 2 40 0 40 80
80 80 80 80 80 40 40 40 40 40
250 250 250 250 250 250 250 250 250 250
0:0054
20:3737
21:8833 20:2707 0
20:0012
Z (mm)
321:0669
3
7 7 427:9406 7 5 1 ð3:69Þ
Machine Vision: Visual Sensing and Image Processing
165
Examination of Equation 3.57 indicates that this result enables us to relate a point Pp ðup ; vp Þ in the pixel coordinates to a point Pw ðxiw ; yiw ; ziw Þ in the world coordinates.
Problems P3.1. Figure P3.1 shows an image having a certain gray-level histogram. If we wish to modify the original histogram such that it is stretched over a certain range and that it slides with some offset from the original histogram, as shown in the figure, explain what kind of changes in the image will occur for both cases.
(a) stretching
(b) sliding FIGURE P3.1 Histogram modification.
P3.2. Finding the connected components in an image is common practice in machine vision. A labeling algorithm finds all connected components in the image and assigns a label to all points in the same component. For an image shown in Figure P3.2, explain how the labeling algorithm can work to segment each individual component. P3.3. Binary image varies greatly depending on which threshold value is used. Figure P3.3 shows an image of a tree leaf. Obtain the binary images at T1 ¼ 150 and T2 ¼ 180 and explain the results by comparing the obtained binary images.
166
Optomechatronics
255 255 255 255 255 255 255 255 255 255 255 255 255 255 25 25 255 255 255 255 255 255 255 25 25 25 25 255 255 255 255 255 25 25 25 25 25 25 255 255 255 255 25 25 25 25 25 25 255 255 255 255 255 25 25 25 25 255 255 255 255 255 255 255 25 25 255 255 255 255 255 255 255 255 255 255 255 255 255 255 FIGURE P3.2 A mosaic image.
FIGURE P3.3 Image of a tree leaf.
P3.4. In Figure P3.4(a) and (b), two simple series of mosaics of 1s and 0s mimicking gray level image are given in an 8 £ 10 window. We wish to determine their edge using Robert and Laplacian operators. (1) Obtain the gradient vector Gx and Gy of the image (a), using a Robert operator and find the edge, using T ¼ 220: (2) Obtain the Laplacian operation of the image (b) and find the zerocrossing line from the result.
Machine Vision: Visual Sensing and Image Processing
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 25 25 25 25 255 255 255
167
20 20 20 20 20 80 80 80 80 80 20 20 20 20 20 80 80 80 80 80
255 255 255 25 10 10 25 255 255 255
20 20 20 20 20 80 80 80 80 80
255 255 255 25 10 10 25 255 255 255
20 20 20 20 20 80 80 80 80 80
255 255 255 25 25 25 25 255 255 255
20 20 20 20 20 20 20 20 20 20
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
(a)
20 20 20 20 20 20 20 20 20 20
(b)
FIGURE P3.4 A series of mosaic images.
P3.5. The image of an array resistance is shown in Figure P3.5. Use the Sobel operator to detect the edge at three different arbitrary values of threshold.
FIGURE P3.5 Image of an array resistance.
P3.6. Figure P3.6 contains two different noise levels. Obtain the filtered images of the images using a median filter with mean ¼ 0.0 and standard deviation ¼ 2.0. P3.7. A line in the x-y coordinate frame is shown in Figure P3.7. Let points P1 and P2 be located on the line. (1) Explain why the polar coordinate representation of lines, r ¼ x cos u þ y sin u, is more suitable for line detection using the Hough transform than the standard line representation, y ¼ mx þ c: (2) Explain how the standard line representation y ¼ mx þ c can be converted to the polar coordinate representation, r ¼ x cos u þ y sin u:
168
Optomechatronics
FIGURE P3.6 Images of a pair of cutters with two different noise levels.
y
P2 P1
(−1,1+ 3) (− 2,1) O
x
FIGURE P3.7 A line in x-y coordinate frame.
(3) For the line shown in the figure, show the standard line representation and the polar coordinate representation of the line. P3.8. Let an image plane be with 300 £ 400 pixel size as shown in Figure P3.8. For extracting the line information from the image acquired in this image plane, a Hough transform is generally used. Here, we would like to get the specified resolution of the extracted line, and Dr ¼ 1 pixel. (1) Design the accumulator array AðuÞ: Assume that the image quantization effect is negligible. Determine m and c values where m and c are the horizontal and vertical sizes of the accumulator, respectively. (2) On this image plane, a line denoted by AB is acquired as shown in Figure P3.8. Which cell of the accumulator is corresponding to the line?
Machine Vision: Visual Sensing and Image Processing
169 q = ( 400, 300 )
vp
lA ( 100, 200 )
( 300, 100 ) B up FIGURE P3.8 A line in image coordinate frame.
(3) When this line is reconstructed from the corresponding accumulator cell, how much is the parametric error of the line equation? P3.9. A camera is located relatively to the world coordinate frame {W}, as shown in Figure P3.9. We wish to map a point in world coordinates on to the corresponding points in the image coordinates frame. (1) The transformation between the world coordinates and the camera coordinates can be expressed by using the euler angle (f,u and c)
Yc Xc Zc {C} Zw T = [a, b, c]
f = 90° q = 90°
Yw Xw
{W}
FIGURE P3.9 A configuration of camera and world coordinate frames.
y = 90° T= [0, –100, –100]
170
Optomechatronics
and translation ða; b; cÞ and is given by 2
Xc
3
6 7 6Y 7 6 c7¼ 6 7 6 7 6 Zc 7 4 5 1
c
2 6 6 6 6 6 6 4
Tw
R … 0
32
Xw
3
76 7 .. Yw 7 . T7 76 6 7 76 7 76 .. 7 6 Z . …7 54 w 7 5 .. . 1 1
where c Tw is given by 2
cos f cos u cos c 2 sin f sin c 2cos f cos u sin c 2 sin f cos c cos f sin u a
3
7 6 6 sin f cos u cos c þ cos f sin c 2sin f cos u sin c þ cos f cos c sin f sin u b 7 5 4 2sin u cos c
sin u sin c
cos u
c
Let the intrinsic parameters of the camera be given by f ¼ 16 mm, ku ¼ kv ¼ 50 mm21, u0 ¼ 320, v0 ¼ 240. (1) Find the transformation matrix H in Equation 3.52. (2) If a point Pw is located at Xw ¼ ½200; 600; 200 in the world coordinates, what is the corresponding up in the pixel coordinates? ~w P3.10. Consider a perspective transformation having u~ p ¼ HX where 2
h11
h14
3
6 H¼6 4 h21
h12
h13
h22
h23
7 h24 7 5
h31
h32
h33
h34
and let q1 ¼ h11 h12 q4 ¼ h14 h24 h34 :
h13
q2 ¼ h21
h22
h23
q3 ¼ h31
h32
h33
(1) Show that u0 and v0 can be expressed as u0 ¼ q1 qT3 ; v0 ¼ q2 qT3 : qffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffi (2) Show that au ¼ q1 qT1 2 u20 av ¼ q2 qT2 2 v20 : P3.11. Consider the camera calibration configuration presented in the section “Camera Calibration.” A pattern image is obtained as shown in Figure P3.11.
Machine Vision: Visual Sensing and Image Processing
171
FIGURE P3.11 A pattern image obtained a Zw ¼ 150 mm.
(1) Determine the coordinate values of the centroids of the circles from this image. (2) Compare this result with the one obtained by the camera calibration matrix. The absolute coordinate values of the circles are shown in Table P3.1. Discuss the error caused by calibration.
TABLE P3.1 World Coordinate Values of the Centers of the Circle Patterns Feature Point
X (mm)
Y (mm)
Z (mm)
Feature Point
X (mm)
Y (mm)
Z (mm)
2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120
80 80 80 80 80 80 80 40 40 40 40 40 40 40
150 150 150 150 150 150 150 150 150 150 150 150 150 150
Image 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 3-13 3-14
2120 280 240 0 40 80 120 2120 280 240 0 40 80 120
160 160 160 160 160 160 160 120 120 120 120 120 120 120
150 150 150 150 150 150 150 150 150 150 150 150 150 150
3-15 3-16 3-17 3-18 3-19 3-20 3-21 3-22 3-23 3-24 3-25 3-26 3-27 3-28
172
Optomechatronics
References [1] Ballard, D.H. and Brown, C.M. Computer Vision, Prentice Hall, Englewood Cliffs, NJ, 1982. [2] Bishop, R.H. The Mechatronics Handbook, CRC Press, Boca Raton, 2002. [3] Corke, P.I. Visual Control of Robots, Research Studies Press Ltd., Taunton, UK, 1996. [4] Datatranslation PCI Frame Grabber Product Catalog, Datatranslation Co. Ltd., http://www.datatranslation.com/top_products_pages/imaging-PCI.htm, 2005. [5] Fu, K.S., Gonzalez, R.C. and Lee, C.S.G. Robotics Control, Sensing, Vision, and Intelligence, McGraw Hill lnc., New York, 1987. [6] Gonzalez, R.C. and Woods, R.E. Digital Image Processing, Addison Wesley Publishing Co., Reading, MA, 1992. [7] Gonzalez, R.C., Woods, R.E. and Eddins, S.L. Digital Image Processing Using MATLAB, Prentice hall, Englewood Cliffs, NJ, 2004. [8] Horn, B.K.P. Robot Vision, The MIT Press, McGraw-Hill Book Co., 1986. [9] Jahne, B., Haupecker, H. and Geipler, P. Handbook of Computer Vision and Applications, System and Applications, Vol. 3. Academic Press, New York, 1999. [10] James, R. and Carstens, P.E. Automatic Control Systems and Components, Prentice Hall, Englewood Cliffs, NJ, 1990. [11] Jain, R., Kasturi, R. and Schunck, B.G. Machine Vision, McGraw-Hill, New York, 1995. [12] Metero Frame Grabber Product Catalog, Matrox Co. Ltd., http://www.matrox. com/imaging/products/frame_grabbers.cfm, 2005. [13] Petrou, M. and Bosdogianni, P. Image Processing: The Fundamentals, Wiley, New York, 1999. [14] Ridler, T.W. and Calvard, S. Picture thresholding using an iterative selection method, IEEE Transactions SMC, 8:8, 630–632, 1978. [15] Stadler, W. Analytical Robotics and Mechatronics, McGraw-Hill, New York, 1995. [16] Umbaugh, S.E. Computer Vision and Image Processing, Prentice Hall PTR, Upper Saddle, NJ, 1998. [17] Watson, D.M. Lab Manual, Astronomy 111, The University of Rochester, Rochester, NY, 2000. [18] Xie, M. Fundamentals of Robotics, World Scientific Publishing Co. Pte. Ltd., Singapore, 2003.
4 Mechatronic Elements for Optomechatronic Interface CONTENTS Sensors ................................................................................................................. 175 Capacitive Sensor....................................................................................... 176 Differential Transformer............................................................................ 177 Piezoelectric Sensors.................................................................................. 179 Pyroelectric Sensor..................................................................................... 185 Semiconductor Sensors; Light Detectors................................................ 186 Photodiode .................................................................................................. 189 Other Photodetectors................................................................................. 191 Photovoltaic Detectors ....................................................................... 192 Avalanche Photodiode....................................................................... 192 Signal Conditioning........................................................................................... 193 Operational Amplifiers ............................................................................. 193 Inverting Amplifier ............................................................................ 193 Noninverting Amplifier .................................................................... 194 Inverting Summing Amplifier.......................................................... 195 Integrating Amplifier......................................................................... 195 Differential Amplifier......................................................................... 196 Comparator ......................................................................................... 196 Signal Processing Elements ...................................................................... 197 Filters .................................................................................................... 197 Digital-to-Analog Conversion .......................................................... 198 Analog-to-Digital Converters ........................................................... 199 Sample and Hold Module................................................................. 200 Multiplexer .......................................................................................... 201 Time Division Multiplexing.............................................................. 201 Wheatstone Bridge ............................................................................. 202 Isolator.................................................................................................. 203 Microcomputer System ............................................................................. 204 Microcomputer ................................................................................... 204 Input/Output Interface ..................................................................... 207
173
174
Optomechatronics
Microcontrollers.................................................................................. 207 Sampling of a Signal.................................................................................. 207 Actuators ............................................................................................................. 210 Electric Motors............................................................................................ 210 Piezoelectric Actuator ............................................................................... 212 Voice Coil Motor (VCM) ........................................................................... 216 Electrostatic Actuator ................................................................................ 218 Microactuators.................................................................................................... 220 Shape Memory Alloy (SMA) Actuator................................................... 221 Magnetostrictive Actuator ........................................................................ 222 Ionic Polymer Metal Composite (IPMC) Actuator............................... 223 Signal Display..................................................................................................... 224 Dynamic Systems and Control ........................................................................ 225 Dynamic Systems Modeling .................................................................... 226 Thermal Systems ................................................................................ 227 Spring-Mass-Damper System ........................................................... 227 Fluid System........................................................................................ 228 Optical Disk......................................................................................... 229 Dynamic Response of Dynamical Systems............................................ 230 System Transfer Function ......................................................................... 233 First Order System ............................................................................. 234 Second Order System......................................................................... 235 Higher Order System......................................................................... 236 Laplace Transforms Theorems ......................................................... 236 Open Loop vs. Feedback Control .................................................... 237 System Performance .................................................................................. 238 Basic Control Actions ................................................................................ 241 System Stability .......................................................................................... 244 Problems.............................................................................................................. 245 References ........................................................................................................... 252
As mentioned in the Introduction, mechatronics is an integrated discipline in which optical, mechanical, electrical, and computer technologies are embedded together. Therefore, this discipline encompasses a variety of technical fields ranging from mechanism design, sensor and measurement, signal conditioning and processing, drive and actuator, system control, microprocessor, and so on, as shown in Figure 1.11. These are considered to be the key areas of mechatronics. In this section, we will deal with some detailed fundamental methodologies involved with the technical fields, but, in particular, those that may be effectively combined with optical elements for optomechatronic integration. In the first part, sensors and actuators are introduced. In recent years, a great deal of new materials and transduction methods have been developed. In addition, an abundance of micro sensors and actuators have
Mechatronic Elements for Optomechatronic Interface
175
appeared in recent years owing to development of microfabrication methods; these sensors and actuators are expected to grow at a very fast rate due to down-sizing of macroparts and wide applications of MEMS parts. We will discuss here some of these micro sensors and actuators in brief; the majority of these micro sensors and actuators here means the sensors and actuators that operate on orders of mm range or less, regardless of their physical size. Most sensors to be dealt with have a small range of measurement but high resolution such as capacitive and piezoelectric sensors. Other sensors based on semiconductor technology such as light detecting, piezoresistive are also treated because of the variety of their use in optomechatronic integration. Actuators to be discussed here cover a variety of actuating principles including piezoelectric, capacitive, electromagnetic, material phase transformation, magnetostrictive, and so on. Most of these actuators have a relatively small range of actuation. We will also discuss DC motor which is a rotary actuator and has been used in a variety of optomechatronic systems. We shall make use of some of these actuators in order to produce optomechatronic integration in the later part of this book. In the second part, we will discuss the components for signal conditioning such as operational amplifiers, filters, comparators, multiplexer, microcomputer, and then signal sampling via analog to digital (A/D) converter, sending-out output signal from microcomputer via digital to analog (D/A) converter. In the last part, we will finally discuss some of elementary concepts of system modeling, transfer function, system response, and basic feedback controllers.
Sensors Sensing is the most fundamental technique that senses the physical variables being measured. The sensor is a physical element that does this, and contains one or more transducers within it. The transducer converts or transforms one form of energy to another form in a detectable signal form: The energy includes various types of form-mechanical, electrical, optical, chemical, and so forth. In view of this, the transducer is an essential element of a sensor which needs to be contained within it. The transformed raw signal usually contains low amplitude, noise contaminated, narrow sensing range, nonlinearity, and so on. To make these raw signal forms conditioned into desirable forms, signal conditioning units are usually necessary. Some sensors contain these conditioning units as a part of their body but some are equipped with these units outside. The recent trend, however, due to downsizing of products, is that most of the necessary elements are put together within a single body. Semiconductor sensors belong to one of the groups that lead this trend. This trend can also be found from intelligent sensors in which dissimilar multiple sensors are
176
Optomechatronics
embedded together to measure a variety of different physical quantities. In this section, focusing piezoelectric sensors and light detecting sensors, we will discuss fundamental concepts of several sensors that are frequently used for optomechatronic systems. Capacitive Sensor When two isolated conductive objects or plates are connected to the positive poles of a battery, the plates will receive equal amounts of opposite charge. If the plates are disconnected from the battery, they will remain charged as long as they are in a vacuum. A pair of these plates called a capacitor has a capability of holding electrical charge whose capacitance depends on the magnitude of charge, q, and the potential difference between the plates, V. The capacitance of the parallel plate capacitor is given by q ð4:1Þ C¼ V In terms of the permittivity constant of free space, and the relative permittivity of dielectric material, we can write the capacitance between the plates as C¼
10 1r A d
ð4:2Þ
where 10 is the permittivity of free space, 1r is the relative permittivity of the dielectric material between two plates, A is the area of the plates, and d is the distance between the plates. The relative permittivity is often called the dielectric constant which is defined by 1=10 ¼ 1r, 1 being defined by the
TABLE 4.1 Dielectric Constants of Some Materials (T 5 258C) Material
1r
Vacuum Air Water Benzene Rubber (silicon) Plexiglas Polyesters Ceramic (alumina) Polyethylene Silicon resins Epoxy resins Nylon Lead nitrate
1 1.00054 78.5 2.28 3.2 3.12 3.22 to 4.3 4.5 to 8.4 2.26 3.85 3.65 3.50 37.7
Mechatronic Elements for Optomechatronic Interface
w
V
177
V
d x
(a) area A overlaps
d
(b) gap changes
V
d
dielectric material
(c) dielectric material moves FIGURE 4.1 Change of capacitance between two plates.
permittivity of the material. The dielectric constants of some materials are given in Table 4.1. As shown in Figure 4.1 there are three ways of achieving capacitance variation. In Figure 4.1a, the displacement of one of the plates changes the area of overlap. Figure 4.1b shows the case of varying the distance d between the plates, whereas in Figure 4.1c the displacement of the dielectric material causes a change in capacitance. As one example of such changes, let us suppose that the plate separation d is changed by the amount Dd, and then the resulting change in C will be given by DC ¼ 2
10 1r A Dd d þ Dd d
which indicates a nonlinear relationship between them. This type of sensor has several advantages: noncontact, high accuracy, and high resolution. In addition, stability is another advantage, because it is not influenced by pressure or temperature of the environment. Differential Transformer The differential transformer employing the principle of induction shown in Figure 4.2 provides an AC voltage output proportional to the displacement of the magnetic core passing through the windings. It is essentially a mutual inductance device which consists of three coils symmetrically spaced along an insulated circular tube. The center coil which is the primary coil is energized from an AC power source and the two identical end coils which are the secondary coils are connected together in series. The connection is made in such a way that its outputs are produced out of phase with each
178
Optomechatronics output voltage Ve
magnetic core secondary coil 2
input displacement
primary coil
secondary coil 1
input voltage i FIGURE 4.2 Schematic view of linear variable differential transformer (LVDT).
output voltage
other. When the magnetic core is positioned at the central location the voltage induced (electromotive force emf) in each of the secondary coils will be of the same magnitude, and thus the resulting net output is zero. When the core moves farther from the central position, there will be an increase in the output within a certain limit on either side of the null position. Figure 4.3 illustrates the characteristic curve of output voltage vs. core displacement. The phase difference between the outputs of the two regions is 1808, i.e., out of phase, as can be seen from the figure. Recalling Faraday’s law
A
O
A O
FIGURE 4.3 Typical output voltage of LVDT.
B
core displacement
B core displacement
Mechatronic Elements for Optomechatronic Interface
179
of induction, the voltage (emf) induced in a secondary coil due to change in current i flowing in the primary coil is equal to Ve ¼ 2M
di dt
ð4:3Þ
where M is the coefficient of mutual inductance between two coils. The minus sign indicates the direction of the induced voltage. Since there are two secondary coils in the vicinity of the primary coil, we have two induced emfs for the coils. Thus, if an input current i ¼ i0 sinvt flows through the primary coil, each emf is given by Ve1 ¼ C1 sinðvt 2 wÞ
Ve2 ¼ C2 sinðvt 2 wÞ
where Ve1 and Ve2 denote the emf of the secondary coil one and two, respectively, the constants, C1 and C2, are determined by the position of the core relative to the secondary coils one and two, and w is the phase difference between the primary and secondary voltages. As can be seen from Figure 4.2, the two secondary emfs are connected in series, and therefore the output voltage becomes Ve1 2 Ve2 ¼ ðC1 2 C2 Þsinðvt 2 wÞ It is noticed that the sign and magnitude of ðC1 2 C2 Þ depend on where the magnetic core position is located at the instant of its motion. If C1 ¼ C2 , then the core is located at the null position. If C1 . C2 , the portion of the core positioned in the secondary coil 1 is greater than that in the coil 2, whereas C2 . C1 is the reverse case. This sensor has several advantages for displacement measurement. One of the advantages is noncontact, which eliminates friction and thus gives high resolution. Piezoelectric Sensors Certain materials become electrically polarized when they are subject to mechanical strain. This effect is known as the piezoelectric effect discovered by the French Curie brothers in 1880. The reverse effect is called the inverse piezo effect which is widely used in piezo actuators. Piezoelectric materials can be categorized into single crystals, ceramics, and polymers. Notable among the crystals materials are quartz and Rochelle salt. Barium titanate, lead zirconate titanate (PZT), and ZnO belong to ceramic materials, and polyvinylidene fluoride (PVDF) is an example of a piezoelectric polymer. The piezoelectric effect stays up to the Curie point, since it is dependent upon the temperature of the material. Above that point, the ceramic material has a symmetric polycrystal structure, whereas below the point it has nonsymmetric crystal structure. When it is under the symmetric state, it cannot produce the piezoelectric effect since the centers of the positive and negative charge sites coincide. When it exhibits nonsymmetric structure due
180
Optomechatronics
V
(a) warmed up state
(b) placed in strong electric field V
FIGURE 4.4 Thermal poling of piezo material.
to a phase transformation of the material, the centers of the positive and negative charge sites no longer coincide. In this case, the material exhibits the piezoelectric effect. The Curie point depends on the material, for example, piezo ceramics (150 – 4008C). There are several different procedures for producing the piezoelectric effect called “polarizing” procedure. In the case of material such as PZT and barium titanate, the procedure starts with heating it to a temperature above the Curie point (1208C). This step makes some electric dipoles within the materials which were initially randomly oriented align in a desirable direction. Then, a high DC voltage is applied across the faces, and the material is cooled down while keeping the same electric field across its thickness. Finally, when the electric field is removed, its polarization stays permanent as long as the polarized material is kept below the Curie point. Figure 4.4 shows two states of the electric field in the PZT developed in the course of poling, as indicated with arrows. In Figure 4.4a, the PZT is electrically neutral when no heat is applied, whereas the state of the PZT in Figure 4.4b is electrically charged in a certain direction shown with arrows due to shift of atoms inside the material, when electric field is applied. This phenomenon leads to the development of electric charge on its surface in response to a mechanical deformation: one face of the material becomes positively charged and the opposite face negatively charged, resulting in an electric potential as shown in Figure 4.5a. In Table 4.2, various properties of piezoelectric materials at T ¼ 208C are listed. The charge developed is proportional to the applied force, F, and is expressed as q ¼ Sq F
ð4:4Þ
where Sq is the piezoelectric constant or charge sensitivity of the piezoelectric material and depends upon the orientation of the crystals material and the way in which the force is applied. The charge sensitivity coefficient is listed in Table 4.3. To pick up an accumulated electric charge, conductive electrodes are applied to the crystal at the opposite sides of the material cut surface. From this point of view, the piezoelectric transducer is considered to be a parallel plate capacitor. The capacitance of the piezoelectric material
Mechatronic Elements for Optomechatronic Interface
181
force cable
A + + + + + + + + + +
piezoelectric transducer
electrodes
h
charge charge amplifier amplifier
_ _ _ _ _ _ _ _ _ _
(a)
(b)
force
Cf
Rf
i
Cp
Rp
−
Cc
i
+
(c) piezoelectric transducer cable
charge amplifier
Cp
Rp
Vo
(d)
FIGURE 4.5 Piezoelectric sensor connected to charge amplifier.
between the plates is given by Cp ¼
10 1r A h
ð4:5Þ
where 1r is the relative permittivity of the piezoelectric material, 10 is the permittivity of the free space, and A and h are the area and thickness of the material, respectively. Since Cp ¼ q=V where V is the potential difference between the electrode plates, combining Equation 4.4 and Equation 4.5
TABLE 4.2 Various Properties of Piezoelectric Materials (T 5 208C)
Material Quartz Barium titanate PZT Polyvinylidene fluoride
Density r, 103(kg/m3)
Young’s Modulus E, (109N/m)
Dielectric Constant 1r
Piezoelectric Constant Cp (pC/N)
1.78 5.7 7.5 1.78
77 110 830 0.3
4.5 1700 1200 12
2.3 78 110 d31 ¼ 20, d32 ¼ 2, d33 ¼ 30
182
Optomechatronics
TABLE 4.3 Typical Charge Sensitivities Sq of Piezoelectric Materials Material Quartz SiO2 Single crystal Barium titanate (BaTiO3) Ceramic, poled polycrystalline
Orientation
Sq (pC/N)
X-cut (length longitudinal) X-cut (thickness longitudinal), Y-cut (thickness shear) Parallel to polarization Perpendicular to polarization
2.2 22.0, 4.4 130 256
Source: Dally, J.W., et al. Instrumentation for engineering measurements, John Wiley and Sons Inc., 1984. Reprinted with permission of John Wiley & Sons, Inc.
leads to V¼
Sq h F 10 1r A
ð4:6Þ
The above equation implies that the developed electric voltage is proportional to the charge sensitivity, thickness, and applied force. The developed potential difference is usually very small, and thus it needs to be amplified by an amplifier called a “charge amplifier” as indicated in Figure 4.5b. When this sensor is connected via a cable to this amplifier, the equivalent electric circuit of the piezoelectric transducer and the amplifier circuit may be as shown in Figure 4.5c. The whole electric circuit can be modeled as a charge generator electrically connected to the capacitance Cp and the leakage resistance Rp , the cable represented as a single capacitance Cc , and the charge amplifier composed of Cf and Rf . If the cable and the amplifier circuits are neglected, the equivalent circuit can be modeled as shown in Figure 4.5d. A polyvinylidene fluoride (PVDF) film sensor shown in Figure 4.6 measures a force acting on the cantilever beam which in this case is a concentrated force. The force sensing principle of the sensor utilizes the same concept as that of the piezoelectric effect, as shown in Figure 4.6a. When the PVDF cantilever structure deforms as a result of the applied force, the deformation will produce an electrical charge due to the piezo effect. In general, the relationship between the behavior of the piezoelectric material and electrical behavior is known to exhibit a very complicated phenomenon. According to the property of the linear piezoelectric materials the following relationship holds Di ðxÞ ¼ dij sj ðxÞ þ 1ik Ek ðtÞ
ð4:7Þ
In the case of the PVDF film sensor, Di is the amount of charge on the PVDF per unit area, Ek is the electric field strength, dij is the piezoelectric coefficient
Mechatronic Elements for Optomechatronic Interface
183
F electrode Vo
PVDF film electrode F
(a) force sensing principle PVDF layer A
F h
3 1
w 2
(b) PVDF sensor configuration FIGURE 4.6 PVDF-based piezoelectric force sensor.
of the material, 1ik is the permittivity of the PVDF, and sj ðxÞ is the applied stress. The double indices in dij indicate the direction of the generated charge (first index i) and that of the applied stress (second index j). The double indices in 1ik indicate the direction of the generated charge (first index i) due to the electric field strength acting in the kth direction. It is noted that in the case of piezoelectric materials, the majority of the dielectric constants are zero except; 111, 122, 133, i.e. 1ik ¼ 0 when k – i. According to these properties, for the configuration shown in Figure 4.6b Equation 4.7 can be expressed as D3 ðxÞ ¼ d31 s1 ðxÞ þ 133 E3 ðtÞ Then the charge generated in the PVDF can be determined by integrating the above equation ð q ¼ D3 ðxÞdA where A is the film area. When the above integration is carried out by using
s1 ðxÞ ¼
hð‘ 2 xÞ FðtÞ 2I
induced due to a concentrated force, F(t), the charge developed for the PVDF configuration is obtained by q¼
d31 Ah‘ FðtÞ þ 133 E3 ðtÞA 4I
ð4:8Þ
It can be observed that the developed charge is composed of the contribution due to the applied force and the induced electric field. It is noted here that h denotes the film thickness only, and I denotes the moment
184
Optomechatronics
of inertia which accounts for total thickness consisting of the electrode, film, and surface coating. This charge generates a current dq=dt through the resistance of the PVDF Rp, so the output voltage Vo ðtÞ of the sensor can be determined by Vo ðtÞ ¼ Rp
dq dt
Because E3 ðtÞ ¼ 2
dVo ðtÞ V ðtÞ ¼2 o dh h
the above equation can be rewritten as Rp Cp
dVo ðtÞ dFðtÞ þ Vo ðtÞ ¼ Kp dt dt
ð4:9Þ
where Cp is the capacitance of the PVDF film given by Cp ¼ the constant given by Kp ¼
133 A and Kp is h
Rp d31 Ah‘ 4I
Referring to Figure 4.5d, we can confirm that the above equation represents the equivalent electrical circuit produced by PVDF film. Piezoresistive sensors utilize materials that exhibit a change in resistance when subjected to an external force. A semiconductor gauge usually exhibits large piezoresistive effect. The common material is silicon doped with p-type or n-type material; the p-type yields a positive gauge factor whereas the n-type gives a negative one. Some details of this type of semiconductor material will be discussed in the next subsection. The strain gauges are attached usually to a beam or a diagram as shown in Figure 4.7a. Another type of piezoresistive force sensor shown in Figure 4.7b uses piezoresistive ceramic composite as a sensing element. As shown, it does not use any flexible member in its configuration but uses a bulk material which has both structural strength and force detection function. In recent years, piezoelectric sensors and actuators in micro scale have been well developed due to advancement of microfabrication methods such as deposition of thin films and patterning techniques. For example, applied force piezoresistor piezoresistor
Si substrate
(a) beam-type FIGURE 4.7 Schematics of piezoresistive sensors.
applied force
electrode
(b) bulk-type
ceramic composite
electrode
Mechatronic Elements for Optomechatronic Interface
185
deposition of thin layers of polymers and ceramics over silicon is a commonly adopted method to obtain silicon based piezoelectric structure. This can be easily carried out by sputtering ion beam or vapor deposition methods. The structural shapes commonly used for deposition are rigid substrate, diaphragm, and two layer bimorph. Discussion of these methods will not be made further, for those involved with complex physical phenomena are beyond the scope of this book. Pyroelectric Sensor
polarization
The pyroelectric materials are crystalline materials capable of generating electric charge in response to heat flow. This effect is very closely related to the piezoelectric effect discussed previously. In fact, many crystalline materials exhibit both pyroelectric and piezoelectric properties. When polarized materials are exposed to a temperature change, its polarization, which is the electric charge developed across the material, varies with temperature of the material. A typical relationship is shown in Figure 4.8a. Polarization decreases with increase in temperature and becomes zero or almost zero near the Curie temperature point. The reduction in polarization results in decrease in charge at the surface of material. As a result, a surplus of charge occurs at the surfaces because there are more charges retained at surface before temperature changes. The pyroelectric materials therefore act
temperature, T
(a) polarization of a pyroelectric material in response to heat flow heat flow out
electrodes
pyroelectric material
i
Cp
Rp
heat absorbing layer heat flow in
(b) sensor configuration FIGURE 4.8 Pyroelectric sensor.
(c) equivalent circuit
186
Optomechatronics
as a capacitor. When electrodes are deposited on opposite sides, as shown in Figure 4.8b, these thermally induced surplus charges can be collected by electrodes. The configuration is the pyroelectric sensor that measures the change in the charge that occurs in response to change in temperature. From the relation shown in Figure 4.8a, the sensor will produce the corresponding charge changes Dq in response to the change in temperature DT in the following manner: Dq ¼ kq DT where kq is called pyroelectric charge coefficient and exhibits a nonlinear function of temperature. From the sensor configuration, we can see that the pyroelectric detector does not need any external excitation signal, but requires only an electric circuit that connects the electrodes. The equivalent electric circuit of a pyroelectric sensor is shown in Figure 4.8c. It consists of a capacitor Cp charged by the heat flow induced excess charge and the resistance Rp of the input circuit. Semiconductor Sensors; Light Detectors Semiconductors are crystalline solid material and have several interesting properties: (1) their electrical properties lie in between those of conductor and insulator; roughly speaking, conductors have very low electrical resistivity (a pure metal), and isolators have very high resistivity. (2) their resistance decreases rapidly with increasing temperature, that is, they have a negative coefficient of resistance. For example, silicon is found to have resistivity that decreases about 8%/8C. (3) In a pure state, they have very high resistance such that few electrons are available to carry electric current. Conductors that contain no impurities are called intrinsic semiconductors. However, when impurities are introduced to the pure semiconductor materials, they are called extrinsic semiconductors. The most interesting property of these impure semiconductors is that their conductivity changes, the amount of the change being dependent on the property of the impurity to be added. The impure materials added into the semiconductor materials are called dopants, and the process of adding impurities is called doping. In extrinsic semiconductors, there is current flow within the materials under the action of an applied field. In this situation, bound electrons arranged in energy bands are separated from particular atoms in the materials, and as a result, positively charged holes are formed and will move in the direction opposite to the flow of electrons; a hole is a missing electron in one of the bands. Therefore, current flow in semiconductor materials is due to the contribution of (1) positively charged holes and (2) negatively charged electrons. These two cases are illustrated in Figure 4.9. The p-type semiconductors are created due to an excess of holes. In this case, the contribution to current flow is due to presence of holes. Boron, aluminum,
Mechatronic Elements for Optomechatronic Interface pure semiconductor
flow of holes flow of current
(a) p-type semiconductor
metal electrode
187 pure semiconductor
metal electrode
flow of electron flow of current
(b) n-type semiconductor
FIGURE 4.9 Two types of semiconductors.
gallium are the impurities that serve as p-dopants. In contrast, the n-type semiconductors are created excess of electrons in which current flow is due to presence of electrons. As shown in the figure, it is noted that the current flow in the same direction as that of whole flow. Most semiconductor devices involve a junction at which p-type and n-type doping meet. In other words, these are placed in contact with one another. The resulting device is called a p –n junction. Through this junction, current flows easily from the p-type to the n-type conductor, since positively charged holes easily enter the n-type conductor, whereas electrons (negatively charged) easily enter the p-type conductor. This process is known as diffusion process. In the case of the current flow from the n-type to p-type, there is much greater resistance. There are two types of connection when a voltage source is applied to the junction; forward biased when the p-type is connected to the positive side of the source, and reverse biased when n-type is connected to the positive side, as shown in Figure 4.10a and Figure 4.10b. Figure 4.10c depicts the characteristics of a voltage –current for the p –n junction. In the forward biased connection, current does not flow till a voltage reaches the threshold value Vth , but above the threshold voltage, the current rapidly increases with the voltage. In contrast to this trend, in reverse biased connection, the current is almost zero, reaching, 2i0 , which is called reverse saturation current. This small current flow occurs due to the fact that there exist a small number of electrons in p-type and small number of holes in n-type. The saturation current exists within a wide range of reverse bias voltage. However, when the voltage increases slightly beyond a certain value called the breakdown voltage, Vbr, the junction of the semiconductor diode breaks down. As a result, the current increases rapidly without any further increase in voltage as shown in the figure. Combining the voltage vs. current relation, for both regions of applied voltage, the resulting curve looks as illustrated in Figure 4.11. It is noted that the curve depends upon the material properties and temperature. For instance, the reverse saturation current i0 is of the order of nanoampares for silicon but milliampares for germanian. The current equation in a diode that
(c)
current (i ), mA
Vth
voltage (V ), volt
metal electrode
FIGURE 4.10 Types of connections and their characteristics of voltage vs. current.
(a)
diode
n-type
(b)
Vbr
p-type
io
diode
current (i ), mA
p-type
metal electrode
voltage (V ), volt
n-type
188 Optomechatronics
189
current (i ), mA
Mechatronic Elements for Optomechatronic Interface
Vbr
−io
voltage (V ), volt
FIGURE 4.11 The characteristic curve of the voltage vs. current (silicon).
can represent the above relationship may be approximately expressed by i ¼ i0 exp
qV kT
21
ð4:10Þ
where V is the applied voltage, i0 is the saturation current, q is the charge of an electron, T is the temperature in degree Kelvin, and k is the Boltzman constant. Photodiode Semiconductor junctions are sensitive to light as well as heat. If a p – n junction is reversely biased, it has high resistance. However, when it is exposed to light, photons impinging the junction can excite bound electrons and create new pairs of electrons and holes on both sides of the junction. Consequently, these separate and flow in opposite directions: electrons flow toward the positive side of the voltage source, whereas holes flow toward the negative side. This results in photocurrent, ip which is directly proportional to the irradiance of the incoming light, Ir ip ¼ CI Ir
ð4:11Þ
where CI is the proportionality constant depending on the area of the diode exposed to incoming light. Combining Equation 4.10 and Equation 4.11 yields i ¼ i0 exp
qV kT
2 1 2 ip
ð4:12Þ
190
Optomechatronics
dark current voltage (v), volt
Ir = 0 Ir = I1 Ir = I2
I1 < I2 < I3
current (i ), mA
Ir = I3
FIGURE 4.12 Voltage vs. current characteristics of a photodiode.
The characteristics of the voltage vs. current relation of the photodiode are shown in Figure 4.12. As can be seen, the photo current ip is much larger than the dark current i0 ðIr ¼ 0Þ and gets larger with increasing light irradiance or intensity. This indicates that if we operate the photodiode within the range of a reversed biased voltage we can make the output current directly proportional to the irradiance or intensity of incident light. Since total current is composed of photo current and dark current (reverse leakage), the dark current needs to be kept as small as possible to have high sensitivity of a photodiode. This can be done by cooling photodiodes to very low temperatures. A variety of photodiode detectors are available for converting a light signal to the corresponding electrical signal for position detection. One form of such detector types is multidiode units which are deposited on a common silicon substrate. There are two types of diode configuration, two or four separate diodes. As shown in Figure 4.13, these are all separate identical diodes insulated from each other and produce a photocurrent in proportion to the area upon which the light beam is incident. A two-separate diode type can be used to determine one dimensional displacement whereas a guardant type can be used for measurement of two dimension displacement of the light beam. When the light beam is illuminated on an equal area of each diode, that is, the beam is centered, the output of each diode will be identical. This case yields i1 ¼ i2 for the two diode type, and ix1 ¼ ix2 ¼ iy1 ¼ iy2 for the quadrant and therefore no output will be produced out of the operational amplifier. However, when the illumination is unequal, this will produce
Mechatronic Elements for Optomechatronic Interface
191
i1 OP amp
OP amp
ix
1
∆x
ix
2
i2
iy
∆y
1
iy
OP amp
2
(a) split cell detector
(b) quadrant detector
FIGURE 4.13 Photodiode detectors.
unequal output from each diode, which in turn will produce the output of the amplifier. Other Photodetectors Photo detectors detect electromagnetic radiation in the spectral range from ultraviolet to infrared. These are categorized into two groups: photon detectors and thermal detectors. The photon detectors operate from the ultraviolet to mid infra spectral ranges, whereas thermal detectors separate in the spectral range of the mid and far infra. Photon detectors utilize the “photoelectric” principle that when light is incident on the surface of semiconductor materials the photonic energy is converted to kinetic energy of the electrons. There are three major mechanisms which produce this type of phenomenon. These include the photoemissive, photoconductive, and photovoltaic; photoemissive detectors produce an electrical signal in proportion to the incident light. As shown in Figure 4.14a, it consists of a cathode (negative), the emissive surface deposited on the inside of a glass tube, and an anode (positive) collecting surface. If a proper circuit is used, light incident on the cathode can cause electrons to be liberated and
light photoconductive layer incident light
e e e
electrode anode h
cathode
(a) photo emissive type FIGURE 4.14 Basic structure of photo detectors.
electrode
(b) photoconductor type
192
Optomechatronics
emitted. These emitted electrons reach the anode and make current flow through the external circuit. When the output current is not sufficient even at high intensity of light, adding successively higher electrodes called “dynodes” will provide substantial amplification. This device is known as a photomultiplier tube. Photoconductive detectors made of some bulk semiconductors without a p – n junction sensitively respond to light. As shown in Figure 4.14b, these consist of a photoconductive thin layer fabricated from semiconductor materials such as cadmium selemium (CdSe), cadmium sulfide (CdS), lead telluride (PbTe), and an electrode set at both ends of the conductor. The electrical resistivity of these materials decreases when they are exposed to light. As discussed earlier, in photodiode due to the decrease in resistivity, the output current of the conductor can be quite large. The change in resistance DR is related to the change in conductance Ds; DR ¼
1 ‘ hDs w
where ‘, w and h are the length, width and thickness of the conductor, respectively. A photoconductor often uses a bridge circuit to detect the intensity of light illuminated on the n-type layer. The change in resistivity due to incident light can be effectively detected by this circuit. Photovoltaic Detectors These detectors supply the voltage or current without any external power source, when they are exposed to light. The detector commonly known as a solar cell consists of a sandwich of dissimilar materials such as an iron base coated with a thin layer of iron selenide. When the cell is exposed to light, a voltage is developed across the sandwitch. Semiconductor junction also belongs to one of such materials. A photodiode discussed in Figure 4.12 exhibits the photovoltaic effect, when it is operated in the region where i is negative and V is positive (fourth quadrant of the figure). In this region, the active area of the p –n type photodiode junction is illuminated without any external bias voltage, and a current is produced in proportion to the intensity of the incident light for essentially zero voltage across the diode. Avalanche Photodiode As already discussed in Figure 4.11, an avalanche photodiode operates in the region of near breakdown voltage Vbr : At this breakdown voltage, known as the Zener voltage, current rapidly increases with a small increase in reverse bias. In other words, at near Vbr, even a small change in the illumination on the photodiode causes a large change in the photocurrent.
Mechatronic Elements for Optomechatronic Interface
193
Signal Conditioning Operational Amplifiers An operational amplifier (op-amp) is a complete amplifier circuit supplied as an integrated circuit on a silicon chip and is a basic signal conditioning element. The amplifier has an extremely high gain with typical value higher than 105, and therefore can be regarded as infinite for the purpose of circuit analysis. Figure 4.15 shows the physical sizes of op-amps along with the schematic diagram of its internal circuit representation. We will limit our discussion here to the ideal op-amp which has the following characteristics; (1) high input impedance mega ohms to giga ohms, (2) low output impedance (of the order of 100 V) considered to be negligible, (3) extremely high gain (G ¼ 105 (100 dB) typical value). Figure 4.16 shows six different types of amplifier circuits frequently used in instrumentation and signal conditioning. Inverting Amplifier Since an operational amplifier has a high intrinsic gain and very high input impedance, it is operated essentially with zero input current and voltage (ia ¼ 0, Va ¼ 0). Figure 4.16a shows the inverting mode of the op-amp which has negative feedback. At the junction of the two resistors we have i1 ¼
Vin 2 Va R1
inverting input noninverting input
output
(a) photograph of OP amplifiers
(b) operational amplifier circuit
FIGURE 4.15 Typical operational amplifiers. Source: National Semiconductor Corporation (www.nsc.com).
194
Optomechatronics
i2
Rf
Rf
R1 i1
Vin
a
a
Vout
Rf
R1
V1
Vout
(b) non-inverting amplifier
(a) inverting amplifier
V2
R1
Vin
C R
R2
Vout
(c) summing amplifier
Vin
Vout
(d) integrating amplifier Rf
Rf C
Vin
(e) differentiator
V1 V2 Vout
R1 R2
Vout
(f) differentiator
FIGURE 4.16 Basic operational amplifier circuits.
i2 ¼
Va 2 Vout Rf
ð4:13Þ
where R1 and Rf are the input and feedback resistors, respectively. By Kirchhoff’s current law, i1 ¼ i2 þ ia Hence Vout R ¼2 f Vi R1
ð4:14Þ
The above relationship implies that the voltage gain of the op-amp is determined simply by the ratio of the input and output resistors.
Mechatronic Elements for Optomechatronic Interface
195
Noninverting Amplifier This mode of op-amp operation causes a voltage of the same polarity to appear at the amplifier output. The general use of this mode is schematically illustrated in Figure 4.16b. It has both inverting and noninverting inputs and amplifies the voltage difference between the two inputs. Let us assume that current flows through the inverting terminal input and feedback registers. The voltage at the junction “a” is then equal to Va ¼
R1 V R1 þ Rf out
Since there is no current flow, i.e., ia . 0, this gives Va . Vin : Therefore, we have Vout R ¼1þ f Vin R1
ð4:15Þ
The closed loop gain is again determined by the ratio of resistors. The characteristic of this noninverting input is that the circuit input impedance is the input impedance of the amplifier itself rather than the impedance denoted by amplifier input resistor. Inverting Summing Amplifier The objective of this mode is to obtain the output voltage in terms of the input voltages V1 and V2 : As in the case of the inverting amplifier, the use of Kirchhoff’s law at the summing junction shown in Figure 4.16c yields i1 þ i2 ¼ ia þ i
ð4:16Þ
In terms of voltage drops, the above equation can be written as Vout V V ¼2 1 þ 2 Rf R1 R2 since ia ¼ 0: It is clear that this relationship can be extended to an arbitrary number of input ports. In the case of fixed feedback resistance Rf , the gains of the individual input can be adjusted by varying the input resistances R1 and R2 : When R1 ¼ R2 ¼ Rs , the output of this summing mode given in the above equation is obtained by Vout ¼ 2
Rf ðV þ V2 Þ Rs 1
This relationship can be effectively used when we want to average the input voltages from multiple sensors. Integrating Amplifier An integrating amplifier is used to integrate the incoming input voltage and utilizes a capacitor as shown in Figure 4.16d. The expression for the output
196
Optomechatronics
voltage Vout can be obtained by considering the current flow through the capacitor. Since the charge q ¼ CVout and i ¼ dq=dt, we have Vin dVout ¼ 2C R dt
ð4:17Þ
Solving for the output voltage Vout we have Vout ¼ 2
1 ðt V dt RC 0 in
The output is proportional to the integral of input voltage, Vin : Differential Amplifier A differential amplifier is used to obtain the difference between two input voltages. Two types of the differentiator are shown in Figure 4.16e and Figure 4.16f. The one shown in Figure 4.16e is similar to the integrating amplifier except that the position of resistor and capacitor are interchanged. An expression for the output voltage Vout of this amplifier mode can be developed by using Equation 4.17. In another type of the differentiator shown in Figure 4.16f, there is no current flowing through the amplifier. The voltage potential Va will then be given by Va R2 ¼ V2 R1 þ R2
ð4:18Þ
As with the previous amplifiers, the current flowing through the feedback resistance Rf must be equal to that through the input resistance R1 : Thus V1 2 Va V 2 Vout ¼ a R1 R2
ð4:19Þ
Combining Equation 4.18 and Equation 4.19, we have Vout ¼
R2 ðV 2 V1 Þ R1 2
Comparator When a comparison between two signals is required, an operational amplifier itself can be used as a comparator. As shown in Figure 4.17, a voltage input V1 is applied to the inverting input while another input V2 is applied to the non inverting input. Because of the high gain of the amplifier, only a small difference signal of the order of 0.1 mV is required for the output to compare which of two voltages is the larger. The right hand side of the figure shows the output voltage vs. the input voltage difference. When non inverting input is greater than inverting input, then the output Vout swings from the negative 2Vs to the positive Vs , and vice versa.
Mechatronic Elements for Optomechatronic Interface
197 output Vs
V1 0
V2
input
−Vs FIGURE 4.17 Comparator.
Signal Processing Elements Filters Filters are used to inhibit the presence of a certain band of undesirable frequencies from a dynamic final signal, permitting others to be transmitted, as illustrated in Figure 4.18a. The range of frequencies passed by a filter is a signal containing all frequency components
particular frequency components of interest
electronic filter
practical
0 fC (b) low pass
practical
fC1
(d) band pass FIGURE 4.18 Various signal filters.
f
ideal
signal magnitude 0
signal magnitude
ideal
fC2
f
ideal practical
fC 0 (c) high pass
signal magnitude
signal magnitude
(a) signal filtering
0
f
ideal practical
fC1
(e) band stop
fC2
f
198
Optomechatronics
called the pass band and the range not passed as the stop band. In actual practice, it is not possible to completely remove the unwanted frequency contents due to the dynamic characteristics of the signal. Figure 4.18b to Figure 4.18e show four different ideal filters classified according to the ranges of frequencies. A low pass filter is the one that transmits all frequencies from zero up to some specified frequency, that is, frequencies below a prescribed cut-off frequency. In contrast, a high pass filter transmits all frequencies from a cut-off frequency up to infinity. A band pass filter transmits all frequencies within a specified band whereas a band stop filter rejects all frequencies within a particular band. As can be observed from the figure, the sharp cut-off of the ideal filter cannot be realized, because all filters exhibit a transition band over which the magnitude ratio decreases with the frequency. The rate of transition is known as the filter roll off. Digital-to-Analog Conversion The purpose for a digital-to-analog (D/A) conversion is to convert the information contained in a binary word to a DC output voltage, which is an analog signal. The signal represents the weighted sum of the nonzero bits in word. This indicates that the leftmost bit is the most significant bit (MSB) having the maximum value which is twice the weight of the next less significant bit, and so on. In general, the weighting scheme of an M bit word is given as below bit
M21
bit 2
bit 1
bit 0
2M21
···
22
21
20
2M21
···
4
2
1
Under this scheme, the leftmost bit, M 2 1 is known as the MSB, since its contribution to the numerical value of the word is the largest relative to the other bits. Bit 0 is known as LBS. The number of the digital word determines the resolution of the output voltage. Suppose that the word has M bits and expresses a full scale of an output voltage Vout : In this case, a change of one bit will cause the corresponding change in the output voltage Vout of Vout =2M : The above conversion relationship can be realized electronically in a number of ways one of which is shown in Figure 4.19. This is a simple DAC known as the weighted register DAC and employs an inverting summing amplifier. In the figure Rf is the feedback resistance, R is the base resistance related to the input resistances, and bi ði ¼ 1; 2· · ·MÞ is 0 or 1. The bi is “off” when connected to the ground and “on” is when connected to VR : When all bis are on, DAC gives analog output voltage which is proportional to an input parallel digital signal; bM21 ; bM22 · · ·b2 ; b1 ; b0 : Summing all currents at the summing junction, we have i0 þ i1 þ · · · þ iM21 ¼ ia þ if
ð4:20Þ
Mechatronic Elements for Optomechatronic Interface VR
199
Rf bM-1
20 R
bM-2
21R
bM-3
22R
b1
2 M-2 R
b0
2 M-1 R
if Vout
Va ia
FIGURE 4.19 Digital-to-analog converter.
Equation 4.20 can be rewritten using Ohm’s law as bM21
VR 2 Va V 2V V 2V V 2 Vout þ bM22 R 1 a þ · · · þ b0 RM21 a ¼ ia þ a 0 Rf 2 R 2 R 2 R
Since Va ¼ 0, the output voltage is obtained by Vout ¼ 2
Rf VR R
bM21 b b1 b0 þ M22 þ · · · þ M22 þ M21 20 21 2 2
¼2
Rf V N R R
ð4:21Þ
where N is denoted by N¼
bM21 b b1 b0 þ M22 þ · · · þ M22 þ M21 20 21 2 2
ð4:22Þ
Equation 4.22 indicates that N is the binary number output to D/A converter and the output of the D/A converter Vout is proportional to N. In the above, bM21 is the MSB, bM22 the next significant bit, and b0 is the least significant bit (LSB). If M ¼ 8, there are 8 bits in the converters, switch number seven corresponds to MSB, while switch number 0 corresponds to LSB. When switch numbers one, six, and seven are closed, the bits in Equation 4.22 become 11000012, which corresponds to 9710. Analog-to-Digital Converters The analog-to-digital (A/D) converter is a device that samples an analog input voltage and encodes the sampled voltage as a binary word. A number of different types of A/D converters are available that offer a range of different performance specifications. The most common are successive approximation, ramp, dual ramp and flash. The simplest form of A/D converter is the successive approximation. Figure 4.20 shows the converter circuit in which a D/A converter is used along with a comparator and a time and control unit. This circuit utilizes the output of a counter as the input to a D/A converter. The output of the
200
Optomechatronics clock n bit register
analog input
counter
DAC
parallel digital output FIGURE 4.20 Configuration for successive approximation ADC.
counter is obtained by counting a sequence of pulses generated in a binary fashion by a clock. The D/A converter then converts this value into an analog value, which is compared with the input analog value by the comparator. When the output of the D/A converter equals or exceeds the input analog voltage, counting of the pulses from the clock is stopped by closing a timing and control unit (gate). The value of the counter is then the desired digital value. For example, initially the counter makes a guess that an input analog voltage is greater than or equal to a certain voltage, say, 5 V. The unit then sets MSB’s flip-flop to a logical 1 state. The binary word 1000 is fed to the D/A converter, which feeds 5.0 V to the comparator. If the comparator senses that the analog voltage is greater than the voltage from the D/A converter, the counter leaves the MSB’s flip-flop set to a logical 1 state. Because the first guess is too small to represent the analog input, the counter then makes a second guess, setting the second bit to a 1. Sample and Hold Module The operation of analog-to-digital conversion takes some time. Therefore, it is necessary to hold an instantaneous value of analog input while conversion takes place. This is done by using a sample and hold device whose circuit is shown in Figure 4.21. The circuit consists of an electronic switch to take the
y(t)
Vin
C
(a) sample and hold circuit FIGURE 4.21 Sample/hold device.
original input signal
Vout
(b) sample and hold signal
t
Mechatronic Elements for Optomechatronic Interface
201
sample, a capacitor for the hold, and an operational voltage follower. The operation principle works, during sample and hold states as follows. In the sample state, if the switch is closed, the capacitor is charged with the applied analog input voltage. The voltage follower makes the output voltage follow the input voltage: in the hold state, if the switch is opened, the capacitor retains its charge. The amplifier maintains the output voltage equal to the input voltage at the instant of time until the switch was opened. Therefore, the output voltage is held constant at the value of the input voltage at the instant of time until the switch is closed again. Multiplexer When sampling several input signals from a number of sources it is sometimes desirable to use a single A/D converter by switching one input signal to another by means of a multiplexer circuit such as that shown in Figure 4.22. The multiplexer is an electronic switching device that enables sampling of the input signal sequentially rather than in parallel. Multiplexing therefore is used to enable a single channel to be shared between a number of signal sources. Time Division Multiplexing Figure 4.23 shows a schematic diagram of a time division multiplexer with four channels, 0 1 2 3. When the four input signals are present, the multiplexer selects a signal as an input to a sample/hold device, and then it needs to know which signal is connected to the output line. Thus, each channel has its own binary address signal. For example, if the address signal is 10, the channel line two is switched on to the output line. In this manner, the multiplexer output has a series of sampled signals taken from different channels at different times, as illustrated in the figure. It is noted that the channels are addressed in the order, 0, 1, 2, and 3, and that DT is the sampling
V8 V7 V6 input signals V5 V4 V3 V2 V1
S/H 8-channel mux devices
FIGURE 4.22 Multiplexer with simultaneous sampling.
A/D
multiplexer switch
202
Optomechatronics
channels
0
multiplexer (MUX)
∆T
1
0 1 2 3 4
2
multiplexed signal
3 signal conditioners
∆T
solid state switch
address decoder
S/H
0 1 2 3 4
sample & hold signal
control signal
FIGURE 4.23 Time division multiplexing.
interval. When the multiplexed signals are fed to the sample hold device, it produces the signal types shown in the right hand side of the figure. Wheatstone Bridge The bridge circuit is the most common unit which can be used for obtaining the output of transducers. Of all possible configurations, the Wheatstone bridge is used to the greatest extent. It converts a resistance change in response to a voltage change. As shown in Figure 4.24, the bridge consists of four resistance arms with a constant electrical source (DC), and a meter M. Due to a constant supply voltage this type is called a constant voltage Wheatstone bridge. When the bridge is balanced, i.e., Vout is zero, no current may flow through the meter, which gives iout ¼ 0: Then, the current i1 must be equal to i2 , whereas the current i3 must be equal to i4 : In addition, the potential difference across R1 must be equal to that across R3 , or i1 R1 ¼ i3 R3 and similarly, for the potential difference across R2 and R4 , i2 R2 ¼ i4 R4 :
B R1
R2 i2
i1
A
i3
R4
R3 D
Vs FIGURE 4.24 Constant-voltage Wheatstone bridge.
C
i4
M
Mechatronic Elements for Optomechatronic Interface
203
We therefore obtain the condition for balance R1 R ¼ 3 R2 R4 When this bridge undergoes change in one of the resistances, the balanced condition cannot be satisfied. This means that the output voltage Vout is no longer zero. The Vout under this condition can be obtained in the following manner: Since Vout ¼ VAB 2 VAD , the use of Ohm’s law yields Vout ¼ Vs
R1 R3 2 R1 þ R2 R3 þ R4
ð4:23Þ
If resistance R1 changes by an amount DR1 , Vout will accordingly change to Vout þ DVout : This relation can be written by Vout þ DVout Vs
8 > > <
DR1 R2 R3 1þ 2 R1 R1 R4 ¼ > DR R R3 1 2 > : 1þ þ 1þ R1 R1 R4
9 > > = > > ;
ð4:24Þ
When all resistances are initially equal and Vout ¼ 0, the relation in Equation 4.24 may be simplified as DVout DR1 =R ¼ Vs 4 þ 2ðDR1 =RÞ
ð4:25Þ
where R ¼ R1 ¼ R2 ¼ R3 ¼ R4 , which is a widely used form. Equation 4.25 shows that this type of resistance bridge exhibits nonlinear behavior, but can be considered to be linear, since usually DR1 is very small for most applications. Isolator When a circuit carrying high current or high voltage is connected to the next circuit, the connection may cause the possibility of damage. This can be protected in various ways. One means of such protection is to use an isolator between two circuits. The isolator is a circuit that completely isolates circuits, while keeping them electrically isolated from each other. Figure 4.25 shows a typical isolator circuit using an operational amplifier, and optical signals. The noninverting amplifier used here is a voltage follower discussed previously whose gain is unity. This implies that circuit 2 follows the output of circuit 1 which is the input to the op-amp. The electrical signal at the output of circuit 1 is converted into an optical signal by a light emitting diode (LED). The converted light signal is in turn converted again into the corresponding electrical signal at the output of circuit 2 by using a photo detector such as a photodiode or phototransistor.
204
Optomechatronics
photodiode LED circuit 2
circuit 1 Vin
Vout
light
optoisolator
(a) an isolator using op−amp
(b) optoisolator
FIGURE 4.25 Isolation via optoisolator.
Microcomputer System The microcomputer or microprocessor has become a vital and indispensable tool to solve almost all engineering problems, in practice, signal conditioning, modern complex control or automation problems. Furthermore, when it is interfaced with sensors, actuators, and artificial intelligent tools, it makes modern machines, processes or products or systems smart so that human intervention can be greatly reduced. This trend will be accelerated in the future, because computers will be equipped with smart software in addition to their capabilities interfacing with intelligent hardware tools and systems. In this section, we will briefly discuss the basics of microcomputers in terms of hardware structure. Microcomputer Basically, most of the computers consist of the following essential units and circuitry: (1) (2) (3) (4)
the central processing unit (CPU) memory bus input and output interfaces and devices.
The CPU is a microprocessor, a semi conductor VLSI (very large scale integrated circuit) chip. The main objective is to recognize and carry out program instructions by fetching and decoding them. As shown in Figure 4.26, it consists of control units, an arithmetic and logic unit (ALU), registers accumulator, program counter, and other auxiliary units such as memory controller, interrupt controller, and so on. The control unit determines the timing and sequence of operations involving fetching a program instruction from memory and executing it. The ALU is a logic circuit which can perform arithmetic and logical operations such as addition,
Mechatronic Elements for Optomechatronic Interface
EPROM EPROM EEPROM EEPROM
address bus
205
control bus
RA M ROM
microprocessor ROM RAM
device select lines
I/O interfore unit
data in
data out
data bus
FIGURE 4.26 Schematic diagram of a micro-computer.
subtraction, and logical AND/OR. The microprocessor has several registers such as the status registers containing information on the latest processed results carried out in the ALU, the address register containing the address of data to be manipulated, the instruction register storing an instruction received from the data bus and some other registers. The accumulator is the data register where data to be sent to the ALU is temporarily stored. The program counter is a register which contains the address of the instruction to be executed next. The memory unit is used for several purposes such as storing the instructions simply in numbers and holding data to be processed, immediate answers and the final results of a calculation. The size of the memory unit is specified by the number of storage locations available, for example, 1 k memory (10 bit address signal); 210 ¼ 1024, 64 k ¼ 216 ¼ 65536 locations. ROM (read-only memory) is a permanent memory where no data can be written, but the data can only be read. In this memory, programs that include computer operating systems and dedicated microprocessor applications are stored. RAM (random access memory) is the memory which information can be written into or read out of. It is used for temporary deposits and withdrawals of data or information required for the establishments of programs. The stored information is lost when the power supply is switched off. EPROM is the erasable and programmable ROM. The program is stored by applying voltages to the specified pins of the integrated circuit and producing a pattern of charged or uncharged cells. Erasing the pattern can be done by optical means using ultraviolet light. A bus is a transmitting media consisting of a group of lines. It ties the CPU memory and input – output (I/O) device together by three sets of wire lines. The first wire set, data bus, is used to carry data between the CPU and the
206
Optomechatronics
memory or I/O channels. This bus also transports the instructions from the memory to the CPU with a view to make them executed. The data bus is bidirectional, allowing signals to flow in and out from the CPU. The second set of wires is an address bus that carries a set of signals known as addresses. This address signal indicates which particular memory location is to be selected so that data may be transferred between the CPU and that location. In this situation, only the location is opened to the communication from the CPU. It is noted that on the address bus data always flows from the CPU to the external devices. Most of the popular CPUs typically employ a 16 bit line address; having 64 K locations; 216 ¼ 65536, but some other types use 20 or 24 address lines. The third set is a control bus which carries a variety of timing and control signals out from the CPU. This bus also carries control signals generated by the control and timing units to the CPU, which synchronize and control each internal operation inside the computer. For example, read and write control signals generated by the CPU are used to indicate which way the data is to flow across the data bus and when the data is to be transferred from memory to the CPU and vice versa. As shown in Table 4.4, the binary states of simple logic signals can be used to tell which mode is enabled. In and out control signals generated by the CPU specify whether an input or an output operation is to be performed. In addition, these specify when the addresses and data are valid for operation, as indicated in the table. The CPU interrupts the program being performed and begins execution of the Int signal when it receives an interrupt signal (Int). The functions of the TABLE 4.4 Control Signals for Some Modes of Operation Operation Mode Read 0 0 1 1
Write
Control Signal Function
0 1 0 1
No memory read/write operation requested Memory address and data are valid for write operation Memory address is valid for read operation Not assigned In/Out Control Signal
0 0 1 1
0 1 0 1
No I/O is to be operated I/O address and data are valid for sending out signals I/O address is valid for taking in external signals. Not assigned INT Control Signal
0 1
Not interrupt requested Interrupt requested
Mechatronic Elements for Optomechatronic Interface
207
interrupt signal are performed according to the operation mode, as shown in the table. It is noted that all of these signals are carried by the control bus. Input/Output Interface Input/output devices are connected to the CPU through three bus lines. This enables the CPU to interact with the devices to perform a basic operation or to send out the executed results. All of these interactions occur through the I/O ports whose pins are used for external connections of inputs and outputs. Ports are places where loading and unloading of data take place. The ports can be classified into input port or output port or programmable port depending on whether the signal is input or output. The peripheral (I/O) devices are keyboard, monitor, line printer, memory, A/D, and D/A. In control and instrumentation applications, the inputs may be signals from sensors. These signals are sampled via A/D for computer. In the case of instrumentation, the computed results may be used for analysis of the process being instrumented. In process control, the results will often be converted to analog signal via a D/A converter in order to meet the specification required by actuators involved with control. Microcontrollers The microprocessors and microcontrollers are similar but as far as applications are concerned, these are slightly different in architecture. Microprocessors are primarily used for high speed computing applications. In contrast, the microcontroller is designed to satisfy the applications of signal processing, engineering measurements, industrial automation, system control and so forth. The microcontroller is a digital integrated circuit in which several functions are all brought together on a single chip which includes the CPU, ROM, RAM, ADC, DAC, and serial and parallel ports. From this point of view, the microcontroller is the integration of a microprocessor with memory I/O interface timer, and the converters (ADC and DAC). Sampling of a Signal Sampling of a continuous time signal is a basic process for computer assisted analysis, instrumentation, and control of processes and systems because of the discrete-time nature of the digital computer. Sampling a signal means to replace the signal by its values in a discrete set of points. If sampling is done at an equal time interval Dt, as indicated in Figure 4.27, then the sampling instants are equally spaced in time, therefore the kth sampling time is expressed by tk ¼ kDt;
k ¼ 0; 1; 2· · ·n
ð4:26Þ
If a time-varying continuous signal is represented by f ðtÞ, the sampled version of the signal may be expressed by f ðtk Þ: This is called a sampled
208
Optomechatronics
signal amplitude
f(t )
O 0 ∆t 2∆t 3∆t
k∆t (tk )
(t1) (t2) (t3)
time (t )
FIGURE 4.27 Sampling a continuous signal f ðtÞ:
sequence often referred to as a time series. In this case, f ðtk Þ is said to be sampled with a frequency of fs which is called sampling frequency or sampling rate; fs ¼
1 Dt
ð4:27Þ
amplitude (V)
amplitude (V)
amplitude (V)
amplitude (V)
To see the effect of the sampling frequency, let us consider a 0.16 Hz sine wave which is plotted versus time over time period 20 sec, as shown in 1 0 –1
0
5
10
15
20
5
10
15
20
5
10
15
20
10
15
20
(a) original signal (frequency: 0.16 HZ)
time (t ), sec
1 0 –1
0
time (t ), sec
(b) fs = 1 HZ
1 0 –1
0
(c) fs = 0.37 HZ
time (t ), sec
1 0 –1
0
5
(d) fs = 0.25 HZ
FIGURE 4.28 Sampling a sinusoidal signal with various frequencies.
time (t ), sec
Mechatronic Elements for Optomechatronic Interface
209
Figure 4.28a. This sine wave is sampled with three different frequencies and the results are shown in Figure 4.28b to Figure 4.28d: (b) fs ¼ 1 Hz (c) fs ¼ 0:37 Hz (d) fs ¼ 0:25 Hz: It is apparent that as the sampling frequency decreases the sampled signal loses the original shape of the continuous sine wave. If sampling frequency is very low, that is, sampling rate is too slow, then the sampled signal appears to be entirely different from the original, having a frequency much lower than that of the original. From this observation we can see that the frequency content of the original signal can be reconstructed accurately only when the sample frequency is higher than the twice the highest frequency contained in the analog signal. That is, fs . 2fmax
ð4:28Þ
where fs is the maximum frequency in the analog signal. This is known as Nyquist criterion. In terms of sampling time interval, Dt ,
1
ð4:29Þ
2fmax
In converting a continuous signal to discrete signal form Equation 4.28 provides a criterion for the minimum sampling frequency. Similarly, Equation 4.29 represents a criterion for the maximum time interval. Whenever a signal at a sample rate is less than 2fs , the resulting discrete signal will appear false differently from the original signal due to misinterpretation of the high frequency content of the original. This phenomenon is referred to as aliasing and the false frequency is called the aliasing frequency. Aliasing occurs when fa ,
1 f 2 s
ð4:30Þ
where fa is the aliasing frequency given by fa ¼ lfmax 2 nfs l
for integer n
ð4:31Þ
For example, suppose that a signal is composed of f ðtÞ ¼ A1 sin 2pð120Þt þ A2 sin 2p ð120Þt If this signal is sampled at a rate of 125 Hz, the frequency content of the resulting discrete signal would be fa ¼ l120 2 125l ¼ 5 Hz
210
Optomechatronics
Therefore, the sampled signal appears to be oscillating at frequency of 5 Hz. The aliasing problem can be prevented when a low-pass filter is placed before the signal enters ADC.
Actuators Electric Motors In view of operation principle, electric motors may be categorized into three basic types of actuators; DC, AC, and stepper motor. DC motors utilize DC current source to produce mechanical rotation, while AC motors consume alternating electrical power to generate such motion. In DC motor, depending upon what type of magnetic fields (permanent magnetic type or electromagnet wound field type) is used, these are divided into two; permanent magnetic DC motor and electromagnetic type or wound field DC motor. Stepper motors steps by a specified number of degrees according to each pulse the motor receives from its controller. We will here discuss DC motors only. Figure 4.29a depicts a schematic of a DC servomotor, in which the basic components are included. The stator has magnets which produce an electrical field across the rotor. The rotor is wound with coils through which current is supplied through the brushes. The brush and commutator work together. They are in contact and rub against each other during rotation, as shown in Figure 4.29b. Due to this arrangement, the rotating commutator sends the current through the armature assembly so that current passes through the rotor. Two principles are involved in the motor motion: the motor law and generator law. In case of the motor law, Fleming’s left hand rule is applied to indicate the directions of the electric current. When all three fingers are kept
rotor
stator magnet stator magnet
stator magnet
motor shaft
magnetic flux
coil
ia F
N
brush commutator
B
FIGURE 4.29 Electric DC motor.
stator magnet
ia
S
brush coil (electric conductor)
commutator ia
ia Va
(a) schematic of DC motor
F
(b) principle of DC motor rotation
Mechatronic Elements for Optomechatronic Interface
211
at right angles to each other, magnetic force (F) is indicated by the thumb, magnetic flux (B) by the index finger, current ðia Þ by the middle finger. Because the current passing through the rotor coils (conductor) is in the magnetic field generated by the stator, the rotor will receive the electromagnetic force called the “Lorentz force” which is given by F ¼ B‘c ia
ð4:32Þ
where B is the magnetic flux density in tesla, ia is the armature current in amperes, and ‘c is the effective length of the coil in meters. The above equation converts electrical energy within a magnetic field into mechanical energy so that a rotor can do work. On the other hand, the law of generator as mentioned describes a relation describing the effect of mechanical motion on electricity. The law states that when the rotor coils (conductor) are moving in the magnetic field in the direction perpendicular to the speed v, it generates an electromotive force (emf) Ve in volts which is expressed by Ve ¼ B‘c v
ð4:33Þ
This Ve can cause the corresponding current to flow in an external electric circuit. Based on the observation of these two laws, we can model the dynamics of the motor rotating in the magnetic field by considering the torque acting on the rotor. T ¼ 2B‘c ia r ¼ kt ia where r is the radius of the rotor, and kt is the torque constant. The voltage generated due to the rotation of the rotor in the magnetic field is expressed by Ve ¼ 2B‘c r
du du ¼ 2kb dt dt
ð4:34Þ
where du=dt is the angular velocity and kb is the back emf constant. Referring to Figure 4.30, we have for the electrical side La
dia du þ Ra ia ¼ Va 2 kb dt dt
ð4:35Þ
where Va is the supply voltage to the armature and for the mechanical side I
d2 u du þb ¼T dt dt2
ð4:36Þ
where I is the moment of inertia of the motor, b is the viscous damping coefficient of the motor, and T is torque induced by a supply voltage. When all parameters are known, the angular speed and position of the motor can be determined from the above Equation 4.35 and Equation 4.36. As we shall see later, the motor parameters I, kb , and b determine the dynamic characteristics of the motor.
212
Optomechatronics Ra
La
Va
I
T
ia Ve
(a) electrical model
b
θ T
(b) mechanical model
FIGURE 4.30 Schematic diagram of an armature controlled DC motor.
Piezoelectric Actuator As mentioned earlier, a piezoelectric material deforms or produces a force when it is subject to an electrical field. This concept is a basis of piezoelectric actuator. The relationship between electrical behavior and mechanical behavior has already been treated in piezoelectric transducer but we will briefly discuss it here again to describe basic deformation modes. The relationship can be approximately described when piezoelectric materials are free of applied loads. It is given by S ¼ dij E
ð4:37Þ
where S is the strain in the piezo material, E is the electric field strength, and dij is the coupling coefficient between S and E, which is the piezoelectric charge constant. To get familiar with the physical meaning of the relation, let us consider two basic deformation modes as shown in Figure 4.31. The unchanged dimension of the piezoelectric material is shown in Figure 4.31a, whereas Figure 4.31b and Figure 4.31c, respectively, show the axially and transversely elongated states due to the electric field E applied as indicated in the specified direction. According to the indicated directions of both electric field and strain, dij in Equation 4.37 can be identified. The axial (thickness) mode has dij ¼ d33 , because in this case both directions are directed along axis 3. The transverse mode (longitudinal) has dij ¼ d31 , because the electric field is in the direction of axis 3, while the strain direction is along axis 1.
electrode E 3
h 2
h+∆h 1
(a) original (unchanged)
(b) axial expansion
FIGURE 4.31 Piezoelectric effect: longitudinal and transverse modes.
electrode E _ +
+ _
+∆
(c) transverse expansion
Mechatronic Elements for Optomechatronic Interface
213 do di
z
+x −y
+y d
−x −x −y
(a) stack
(b) bimorph
(c) tube actuator
FIGURE 4.32 Bimorph configuration.
The other deformation modes such as thickness shear and face shear can also be treated in this way. Generally, a single piezoelectric actuator employing the above configuration produces a very small deformation, and therefore a variety of structural configurations are available, which yield relatively large deformations compared to that of single structure. Figure 4.32 indicates three such actuators which include stack, bimorph, and tube types. The stack type is composed of N axial piezo actuators and thus increases N times the deformation of single actuator. The bimorph actuator consists of two similar transverse actuators. It has a cantilever beam configuration which has a large bending mode when subjected to transverse load. These actuators deform in opposite way and therefore produce a large bending. This bimorph actuator is of great importance in piezoelectric microactuator to be discussed later. The cylindrical hollow tube type has outer diameter do and inner diameter di : It is radially polarized and the outer electrode surface is divided into the quadrant. Each of two surfaces has an electrode; þ x and 2 x or þ y and 2 y. When two opposite electrodes are subject to opposite electric field, it bends. Thus, these give two directional motions in both x and y directions. On the other hand, the z motion is generated when all electrodes are subjected to the same electric field. Based on the linearized portion of the relationship given in Equation 4.37, Table 4.5 describes the deformation of various configurations of piezo actuators discussed in the above. The first two denote the single actuator while the other three represent multiple actuators. In the table Vp is an applied voltage and dij is the coupling piezoelectric coefficient. From the relations the applied force can be computed by considering the fact that the piezo generated static force Fp is proportional to the deformation. From Equation 4.6 the force is related to Fp ¼ CF VP where CF is the force sensitivity constant and given by 1 1A CF ¼ 0 r Sq h
ð4:38Þ
214
Optomechatronics
TABLE 4.5 The Displacement of the Piezoelectric as a Function of the Applied Voltage VP Configuration
Mode
Deformation
Geometry electrode E
Axial
Longitudinal (axial)
_ +
D‘ ¼ d33 VP +∆ ‘
electrode E
Transversal
Transversal
Dz ¼
Stack
Longitudinal
D‘ ¼ nd33 VP
Transversal
L2 d ¼ 3 2 d31 VP h
h
d31 VP
_ +
z+∆z z
Bimorph
`
h
δ
do
di
Tube
Transversal
d¼
2L d V d0 2 di 31 P
+x −y
+y
−x −x −y
The relationship in Equation 4.38 shows that the actuation force is proportional to applied voltage. From the discussions we have made in the above it can be summarized that, when a piezoelectric material undergoes a change in geometry due to an applied voltage, the amount and direction of its deflection depend upon the type of piezo electric material and the geometry of the material. Due to this complexity involved, the piezo material exhibits unfavorable nonlinearity between strain and applied voltage, hysteresis, creep, and aging. To reduce or eliminate this inherent nonlinearity there are several methods developed to date. The most frequently used methods are table-loop-up based on feedback control compensation. In general, feedback compensation results in fairly accurate correction of nonlinearity, leaving only 1% nonlinearity. A general feedback control concept will be discussed shortly. So far, we have considered the static relationship between the deformation and electric field under the assumption that the piezoelectric element will reach equilibrium after undergoing an initial deformation as given in Equation 4.37. However, the piezo element is essentially a moving element, so its motion will be governed by certain dynamic characteristics. The piezoelectric ceramics has inherently a spring element kp and an internal
Mechatronic Elements for Optomechatronic Interface electric load applied
215
electrode z z
Vp
bp
kp
Fp= CF (Vp - Ve )
piezoelectric ceramic
(a) electric load applied to a piezoelectric element
(b) equivalent mechanical model
FIGURE 4.33 Equivalent mechanical model of the piezoelectric actuator.
damping element, bp, and thus these need to be taken into account when its dynamic motion is considered in some particular cases. Figure 4.33a shows its equivalent dynamic model composed of a spring-mass-damper system. When an external load Fl is applied to a piezoelectric element, the dynamics of the element can be written as mp
d2 z dz þ bp þ kp z ¼ Fp þ Fl 2 dt dt
ð4:39Þ
where z is the displacement of the piezo actuator in the vertical direction, mp, bp and kp are the effective mass, damping, and elastic stiffness of the piezo ceramics, and Fp is the effective force developed due to the applied net input voltage to the element that can be obtained from the piezoelasticity theory. The first term in the right-head side of Equation 4.39 results from the inverse piezo-effect Fp : Figure 4.33b indicates that although an input command voltage Vp is applied to the actuator, this does not produce the corresponding strain due to the “piezo effect” by the external load. Upon consideration of this effect, effective Fp can be written as Fp ¼ CF ðVp 2 Ve Þ where CF is a piezoelectric force sensitivity constant converting from electric voltage to force, Vp and Ve are the command input voltage and the voltage induced due to external load, respectively. The above Equation 4.39 can be rewritten as mp
d2 z dz þ kp z ¼ CF ðVp 2 Ve Þ þ Fl þ bp dt dt2
ð4:40Þ
This equation of motion describes the displacement of the actuator for a given input voltage, Vp , and external load, Fl. The natural frequency and damping ratio are determined by the system parameters mp, bp and kp. The displacement of the actuator is usually very small and usually ranges within a few mm.
216
Optomechatronics displacement x current i
coil force F
coil magnet yoke
+
R
L
Ve
V
N S
i
−
magnetic flux
(a) voice coil motor
(b) electrical circuit of the VCM leaf spring
yoke coil
base
moving part magnet
(c) actual system configuration FIGURE 4.34 Configuration of voice coil motor.
Voice Coil Motor (VCM) The name “voice-coil motor” originates from the fact that the inventor got the idea for this motor from a speaker. Figure 4.34a illustrates a simple construction of a typical actuator system driven by a VCM, which can freely move in the direction parallel to a mounting rail. The motor is composed of a moving coil (bobbin), two magnets, and a spring-damper system that nullifies the effect of any external disturbance and stabilizes the bobbin motion as well. For clarity, the spring and damper are not shown. The working principle of VCM is the same as that of the DC motor discussed previously and revisited here for better understanding: When a current flows in the coil, this causes Lorentz force which moves the coil. The direction of motion is determined by the direction of the current in the coil and that of the magnetic flow according to Flemming’s left hand rule. In more detail, the magnet M1 generates a magnetic field in the direction of the arrow (x direction) as indicated in the figure. When the bobbin is placed within a magnetic field with magnetic field density B, force F in the x vertical direction is generated according to Fleming’s left hand rule. The force F is given by F ¼ nBi‘c
ð4:41Þ
where n is the number of coil turns, B is the magnetic flux density in tesla, i is the current in ampere flowing in the direction perpendicular to the paper and ‘c is the coil effective length in meters. Figure 4.34b shows the electrical
Mechatronic Elements for Optomechatronic Interface
217
circuit of the VCM which is written by L
di þ Ri ¼ V þ Ve dt
ð4:42Þ
where L and R are the inductance and resistance of the coil, respectively, V is the electrical voltage applied to the coil and Ve is the back emf of the bobbin, respectively. The inductance L can be obtained by Faraday’s and Ampere’s laws and is given by L¼
10 n2 Ac ‘c
where Ac is the cross-sectional area of the coil, and 10 is the permeability of the air. And the resistance R is defined by R¼
r ‘c Ac
where r is the resistivity of the conductor. The back emf is given by Ve ¼ 2nB‘c
dx dx ¼ 2kb dt dt
ð4:43Þ
where dx=dt is the velocity of the bobbin in the upward direction. Rewriting Equation 4.42 by use of Equation 4.43, we have L
di dx þ Ri þ kb ¼V dt dt
ð4:44Þ
Therefore, when the bobbin is supported by a spring as shown in Figure 4.34c, the dynamic equation governing the motion of the bobbin can be written by m
d2 x dx þb þ kx ¼ F dt dt2
ð4:45Þ
where m is the mass of the bobbin, and b existing between the guide and bobbin and k are the damping coefficient and the stiffness of the bobbin, respectively. The actuating force generated by the VCM can adjust the displacement of the bobbin to a desired accuracy, depending upon the choice of these values. We will return to VCM in more detail in Chapter 6. Typical magnetic configurations of linear VCM are shown in Figure 4.35. These are either cylindrical or rectangular in configuration. They have some differences in such characteristics as gap flux density, demagnetization effect, and leakage flux between the magnets and other elements, shielding the leakage. For instance, the outer magnet long coil shown in Figure 4.35a has high leakage between the center yoke and the magnets, whereas inner magnet long coil shown in Figure 4.35b has low leakage factor due to its configuration. Figure 4.35c has low leakage factor and coil inductance in comparison with that of the configuration shown in Figure 4.35d which is the
218
Optomechatronics
N
side yoke
side yoke
S
coil center yoke
N S permanent magnet (a) outer magnet long coil
coil center yoke
N S
(b) inner magnet long coil
N
S
N
S
N
S
N
S
(c) steel plate pole
(d) enclosed magnet long coil
FIGURE 4.35 Configuration of various voice coil motors.
enclosed magnet configuration. This configuration outperforms the previous ones in leakage and uniformity of magnetic field along the path of the coil. Electrostatic Actuator This actuator utilizes an electrostatic force generated by two electrically charged parallel plates. The electrostatic energy stored in this actuator is expressed by U¼
1 CV 2 2
ð4:46Þ
where V is the potential difference between the electrodes and C is the capacitance between two electrodes given by C ¼ 10 1r
A d
ð4:47Þ
In the above, A is the area of opposing electrodes and d is the separation of the electrodes. Consideration of this relation has been already given in the capacitance sensor. Utilizing the relation in Equation 4.46, we can obtain the force acting in an arbitrary direction as Fr ðrÞ ¼
›UðrÞ ›r
ð4:48Þ
Let us elaborate the concept of the electrostatic actuator in two directions. Figure 4.36 illustrates two directional forces that can be actuated by this drive mechanism: (1) overlapped area driving force, (2) gap closing
Mechatronic Elements for Optomechatronic Interface
V
219
V z
z x
(a) overlapped area driving force
(b) gap closing force
kc
bc
V
zs
z
(c) dynamic model of the electrostatic actuator FIGURE 4.36 Electrostatic force actuator.
force. The area driving force shown in Figure 4.36a can be derived using Equation 4.48 and written as Fx ¼
10 1r wV 2 2z
ð4:49Þ
where V is the input voltage applied to the set of the plates, x is the overlap distance whose coordinate is shown in the figure, and w is the width of actuator. The gap closing force per gap is illustrated in Figure 4.36b. The force can be obtained in a similar way as derived in the case of the comb driving force, which can be derived as Fz ¼ 2
10 1r wV 2 2z2
ð4:50Þ
It is noted that the gap closing force increases drastically for a given input voltage V as z decreases. Equation 4.49 and Equation 4.50 present the relationship between the input voltage and the output for a specified gap z or overlap distance x. When this actuator gives an actuating force to a mechanical element, one of the capacitor upper plates moves in z or x directions [19]. Figure 4.36c illustrates the case when the capacitor generates a z-directional force while constrained to a spring kc and a damping element bc. Upon consideration of this dynamic configuration, the equation governing the dynamics of the capacitor plate may be written as mc
d2 z dz 1o AV 2 þ k ðz 2 z Þ ¼ 2 þ b c 0 c dt dt2 2z2
ð4:51Þ
220
Optomechatronics
where A is the charge area of the capacitor and z is the gap between the two plates, which is measured downwards from the equilibrium position, z0. It is given by z ¼ z0 2 zs where z0 is the equilibrium position at which no spring force and capacitor charge occur, and zs is the displacement caused by the spring. This indicates that when z ¼ 0, the displacement of the plate, z, becomes zero. It is important to observe that the driving force appearing in the right hand side of the above equation is a function of displacement z. Examination of this equation shows that at equilibrium the electrostatic force pulling the plate down and the spring force must be equal. This implies that the net force Fn which is the difference between two forces must be zero, which is given by Fn ¼ 2
1AV 2 2 kc ðz 2 z0 Þ 2z2
In order to see the stability of this point, we now differentiate Fn with respect to z, we obtain dFn 1AV 2 2 kc ¼ dz z3
ð4:52Þ
The stability of this point must be dependent upon the sign of the above equation. In order for the capacitor plate to be at a stable equilibrium, the variation of the net force with respect to displacement z must be dFn =dz , 0, which leads to kc .
1AV 2 z3
ð4:53Þ
Detailed discussion on the behavior of the plate dynamics is left for a problem in Chapter 4. We will discuss the dynamic characteristics of this actuator in detail in Chapter 7.
Microactuators There are two categories of microactuators: one is the actuator that drives mechanical elements within micro range regardless of its physical size. The other is the actuator whose physical scale is limited to micro dimension. A number of these microactuators have been developed in the past. Microactuators to be discussed here will include only three actuators that operate on orders up to a few hundred mm. These actuators are made of shape memory alloy, magnetostrictive material, and ionic polymer metal composite. Table 4.6 compares a group of microactuators in terms of deformation range, frequency ranges, and force and so on.
Mechatronic Elements for Optomechatronic Interface
221
TABLE 4.6 Properties of Various Micro-Displacement Actuators Actuator Technology Piezoelectric (BM500) Shape memory (Nitinol) Magnetostrictive (Terfenol-D) Electrostrictive (PMN)
Typical Displacement
Force
Hysteresis
Frequency Range
100 mm (L ¼ 5 cm)
20 kN
8 , 15%
,30 kHz
500 mm (L ¼ 5 cm)
500 N
10 , 30%
,5 Hz
100 mm
1.1 kN
,10%
,4 kHz
65 mm
9 kN
1 , 4%
,1 kHz
Operating Temperature Range 220 , 2508C (Tc ¼ 3658C) up to 4008C (thermal actuation) up to 3008C (Tc ¼ 3808C) 0 , 308C (Tc)
Source: Prasad, E. Sensor Technology limited, 2002.
Shape Memory Alloy (SMA) Actuator SMAs such as titanium nickel alloy (TiNi) are smart materials, which possess thermoelastic martensitic transformation exhibiting shape recovery phenomenon when heated: The alloy has two main phases associated with the upper memory recovery, austenite (high temperature phase) and martensite (low temperature phase). Physically speaking, this means that, when a SMA is stretched from its undeformed state at some temperature below its transformation temperature (high temperature phase), it has the ability to return to the undeformed original shape upon heating. This characteristic results from a transformation in its atomic crystal structure. As shown in Figure 4.37, the martensite state possesses the twinned structure easily to deform while the austenite state has the very rigid cubic structure. This atomic structural change from the twinned martensite to the austenite causes the SMA to generate a large force which can be a very useful property as an actuator. Figure 4.38a depicts
crystal lattice of austenite
FIGURE 4.37 Phase transformation of shape memory alloys.
twinned crystal lattice of martensite
Optomechatronics
contraction,
∆
martensite (finish) austenite (start) cooling
∆T
heating
austenite (finish)
tensile force, F
222
cooling
∆T heating
martensite (start) Tm
temperature (T )
0
Ta
(a) contraction vs. temperature
temperature (T )
Ta
(b) tensile force vs. temperature
FIGURE 4.38 Typical transform hysteresis of shape memory alloys.
the physics involved with phase transformation and shows the relationship between the deformation and the temperature applied. As temperature rises, the deformed material in martensite phase starts to change its phase to austenite. At the temperature Ta , the phase of material is completely turned into austenite. At this phase, the material recovers its length (shape), whose deformation is reversed. It is noted that during this period, elongation is a slightly nonlinear function of temperature. When the material in austenite phase cools from the high temperature Ta , it does not follow up the relationship curve, but the lower curve, slightly deviated from the upper curve. This indicates it undergoes a typical hysteresis effect. As the temperature keeps decreasing, the phase of the material is completely changed into martensite, at temperature Tm : At this phase, the material reaches the deformed state again. This shape recovery property creates a wide range of application areas as an actuator. The drawbacks of this actuator are: nonlinearity due to the hysteresis and slow response. Some of these adverse effects may be partially compensated by applying some advanced control algorithms. In MEMS applications, a variety of methods depositing this SMA film over silicon are used. In this case, actuation is due to the recovery of residual tensile stress in the film. Figure 4.38b shows the relationship of tensile stress vs. temperature for TiNiCu film material. As the film material deposited at temperature Ta cools down, its phase is changed from austenite to martensite. Below temperature Tm , it can be seen that the thermal stress in the film is almost relaxed. The reverse transformation occurs when it is heated from Tm to Ta and causes the film to recover its original shape. Magnetostrictive Actuator Magnetostrictive materials found in 1970s transduce magnetic energy to mechanical energy when these are subjected to an electromagnetic field
Mechatronic Elements for Optomechatronic Interface
223
as indicated in Figure 4.39a. These materials also generate electromagnetic fields when they are deformed by an external force. Therefore, magnetostrictive materials can be used for both actuation and sensing due to this bidirectional coupling. The advantage of this actuator is that it readily responds to significantly lower voltage as compared with piezoelectric actuators (200 to 300 V). Magnetostrictive materials can be deposited over a silicon micromachined cantilever beam as shown in Figure 4.39b when they are used as a micro actuator. The actuator consists of a thin film and a silicon beam. When it is subjected to a magnetic field, the magnetostrictive film expands and causes the beam to bend in the vertical direction. The deflection d is found to show a dependency of field strength (telsa). Typical beam deflection [22] is found to be 100 mm at 0.01 telsa for a 10 mm thick terfenol-D film over a silicon thickness 50 mm and length 20 mm. The recent applications of these materials can be found from a variety of applications: adaptive optics, high force linear motor, active vibration or noise control, and industrial and medical sonar and pump. Ionic Polymer Metal Composite (IPMC) Actuator The IPMC materials possess the susceptibility to interactions with externally applied electric fields and also to their own internal field structure. Due to this property, when an electrical field is applied, the hydrated cations in the materials move to negatively charged electrode side. As a result, the IPMC strip undergoes internal volume change and thus bends towards the anode, negatively charged side, as shown in Figure 4.40. The advantages of this actuator are light weight, relatively large displacement, low input voltage (4 to 7 V) and fast response (msec to sec). The deformation and actuation force are found to be dependent on the applied voltage and geometry of the material. This material can be used as
displacement
(a)
terfenol-D rod
field coil
terfenol-D layer
d (b)
Si
FIGURE 4.39 Configuration of a magnetostrictive actuator.
displacement
224
Optomechatronics
+ – + – + – + – + –
V
+
+ + + + + + – – – – – + – – –
V
FIGURE 4.40 Behavior of an ionic polymer due to an applied electric field.
a sensing element as well. When it is bent, the resulting internal stresses developed cause shifting of mobile charges, which can be detected by a low power amplifier. This constitutes an IPMC sensor.
Signal Display The display element is usually the final data representation element. There are a number of data presentation devices from simple pointer scale and recorders to very large scale display such as cathode-ray-tube (CRT) and plasma display panel (PDP). Here, a brief discussion will be made, being limited to optical or image display devices. As shown in Figure 4.41, optical displays units can be conveniently categorized into small scale alphanumeric displays and large displays that all work in the digital domain. As far as displaying principle is concerned, it is interesting to see that all of these can be grouped into three categories. The first method is to use a light emitting source such as LED. The image optical intensity is directly controlled by controlling input current. The CRT as shown in Figure 4.41a uses this principle, in which deflected electrons controllable by a deflection system within the device interact with semiconductor material such as phosphors formed in dots. Plasma panels in Figure 4.41b are composed of an array of cells known as pixels. The electrodes between two glass substrates produce a gas in a plasma stage, which is made to react with phosphors in each pixel in discharge region. The resulting reaction causes each pixel to produce a desired image at the screen. The second one does not emit light but uses light incoming from other sources. In the liquid crystal device (LCD) depicted in Figure 4.41c, orientation of molecules remains not random but in certain directions, making the optical effect predominant in those directions. This property makes it feasible to modify the crystal structure and thus optical effects by applying electric fields to the LCD. The last display device is to use a signal manipulation technique which is totally different from the conventional display methods in the above. As shown in Figure 4.41d it is based on MEMS technology, and has two types
Mechatronic Elements for Optomechatronic Interface
screen
anode focusing
225
bus electrode
front panel
dielectric phosphor cathode ray tube
x-defection
back panel
y-defection
address electrode
(a) CRT
(b) PDP 7 segment electrode patterns
.. . . .. .
5 liquid crystal 6 7 material glass plate with electrode pattern
reflected light incident light
4
3
21
(c) LCD panel displaying numerals
mirror
(d) DMD schematic
FIGURE 4.41 Large scale display units.
of display, one utilizing optical switching and the other utilizing grating diffraction. The device using optical switching is called a digital mirror device (DMD). The device contains hundreds of millions of tiny mirrors which are embedded in one chip, and is based on micro-electro mechanical system technique (MEMS). Its main role is to steer a light beam into a screen (display unit) in order to project an image in a desired manner by a simple mirror switching action. The innovative concept lies in the fact that in contrast to (LCD) the color content of the image and intensity of the projected image can be flexibly controlled by their tiny motion and switching time. Another device that belongs to this category utilizes grating diffraction. It is called grating light value (GLV) and is composed of a number of pixels which in turn are composed of six diffraction grating made deformable by an actuator. From the aspect of technology integration, DMD projector and GLV are of optomechatronic nature, while LCD is based on an optical-electrical (OE) combination having no moving or deflecting projection of light. We will discuss DMD and GLV in more detail in Chapter 7.
Dynamic Systems and Control Previously, we have seen a variety of physical systems from simple sensors to quite complicated complex systems. Many such systems involve
226
Optomechatronics
phenomena of more than one of the disciplines such as mechanical, electrical, optical and so on. In this case, we need to consider the physical interaction between the discipline involved, which adds more complexity such as nonlinearity, hysteresis and saturation to physical behaviors of the systems. Looking at the systems from a different view, many systems are considered to be static but some are inherently dynamic in nature. In many cases, static condition assumed for static systems often may not be valid due to some physical conditions involved, but may often need to be treated as dynamic systems, as we have already illustrated for the sensors and actuators treated in the previous sections. When it comes to analysis and control of these dynamical systems, characterization of their behaviors is ultimately necessary. To do this physical modeling we first need to take into consideration the properties of physical variables involved and their interaction. Based on this modeling, a control system is to be designed next. Undoubtedly, the performance of the control largely depends on accuracy of the physical model and therefore, modeling needs to be done as accurately as possible. Since most of physical systems encountered in engineering are not ideally simple, some approximation is usually made to make them a linear system which makes controller design relatively much simpler than those not approximated. In this section, we will briefly deal with system modeling, system transfer function, which represents the input – output relationship, and some elementary control techniques. Dynamic Systems Modeling Modeling refers to description of system dynamics in a mathematical form. It involves understanding of the physics involved and mathematical formulation of the physical behavior. Therefore, a fundamental step in building a dynamic model is writing the equations of motion for the system to be modeled. In doing this, we may often find it difficult to completely and accurately describe the system dynamics in detail due to the uncertainty and complexity associated with physical interactions between variables. In this situation, we resort to an identification method with which the unknown or time varying parameters of the system can be identified either from experiments or simulations. There are a number of types of system that can be identified depending on how they behave. These are summarized in Figure 4.42. The lumped model uses lumped variables to represent the systems dynamics while the distributed system uses the partial differential equation in order to describe the system dynamics in both space and time without lumping them. Since the classification given in the figure is fairly standard, one may easily find the description of each model in any control-related books. The systems we will deal with belong to a lumped, continuous and deterministic model. We have already modeled the dynamics of various actuators and sensors. Therefore, we will not illustrate many actuator systems, but instead consider
Mechatronic Elements for Optomechatronic Interface
227
system model
lumped system
discrete system
linear system
deterministic system
time varying system
distributed system
continuous system
nonlinear system
stochastic system
time invariant system
FIGURE 4.42 Types of dynamical systems.
three distinct systems in nature: thermal, vibration and fluid positioning system. Thermal Systems Consider a thermal system as shown in Figure 4.43a. It is assumed that the system at temperature T is exposed to the surrounding environment at temperature T0 : These temperatures are assumed to be uniform throughout the system. When this system has a heat energy inflow, heat energy flow rate through it will be governed by q¼
1 ðT 2 T0 Þ Rt
where q is the heat energy flow rate, and Rt is the thermal resistance. The net inflow into the substance within system causes variation of the temperature of the system. When T0 is constant, the relationship given above can be written as dT 1 ¼ q dt Ct where Ct is the thermal capacity. The above equation indicates that for a given rate of change of heat flow or rate of energy storage q, if the thermal capacitance of the system is large, the rate of temperature change becomes low. The reciprocal of thermal resistance is called “heat conductance.” Spring-Mass-Damper System Many practical systems can be modeled as a spring-mass-damper system if some assumptions are made properly. Figure 4.43b illustrates one such system composed of a two-mass system. From the free-body diagram shown in the figure, we can model the system as m1
d2 x 1 dx1 dx2 ¼ 2b 2 dt dt dt2
2 k1 ðx1 2 x2 Þ þ F
ð4:54Þ
228
Optomechatronics flow in thermal system
valve (orifice)
heat inflow q1(t ) q2 (t ) heat outflow T(t ) T0(t )
(a) thermal system
piston Q A
x
(c) fluid system F x1
m11 k1
F
m m11
b m21 m
k2
k1 (x1–x2) b dx1– dx2 dt dt
x2
k1 (x1–x2) b dx1– dx2 dt dt
m m211 k2x2
(b) vibration system FIGURE 4.43 Various systems for dynamic modeling.
m2
d2 x 2 dx1 dx2 2 ¼b dt dt dt2
þ k1 ðx1 2 x2 Þ 2 k2 x2
It is important to note that the displacements x1 and x2 are taken from their equilibrium position that arises due to gravitational force. Therefore, this force was not included in the above model. Fluid System The fluid flow shown in Figure 4.43c is designed to move a piston in a desired fashion in the x-direction. The physical laws involved with this flow are continuity, resistance and force equilibrium for the piston. When the fluid flows through the orifice (valve) and enters into the empty area of the piston, due to flow resistance at the valve, the mass flow rate Q becomes Q¼
1 pffiffiffiffiffiffiffiffiffiffiffiffi 1 pffiffiffiffi ðP1 2 P2 Þ ¼ DP R R
ð4:55Þ
where R is the orifice flow resistance, and P1 and P2 are the fluid pressures at both sides of the orifice. The flow should meet the continuity law Ardx ¼ Qdt
Mechatronic Elements for Optomechatronic Interface
229
where A is the piston area and r is the fluid density. Due to this fluid flow, the equation governing the motion of the piston is obtained by ADP ¼ m
d2 x dx þb dt dt2
ð4:56Þ
where m and b are the mass and damping coefficient of the piston, respectively. Examination of Equation 4.55 and Equation 4.56 signifies that the model derived is highly nonlinear due to flow through the orifice. However, mass flow rate can be linearized near the normal operation point, Q0 , DP0 , R0 , which will result in a linearized equation. Optical Disk The optical disk shown in Figure 4.44 has two main servo systems composed of a tracking system (radial) and an autofocusing system (vertical). The track following servo is divided into a fine track servo and a coarse motion track servo which are schematically illustrated in the square box. The fine servo system is mounted on the optical pickup and actuated by VCM. As can be seen from the figure, track-following accuracy is largely dependent on this fine servo system. The operation principle is as follows: the laser beam coming out of the laser diode passes through the oblique glass plate (mirror) which is collected by the objective lens. Depending on the location of the lens in the vertical direction, the collected beam may or may not be focused onto the disk surface. The beam reflected from the disk pits travels back through the glass plate and finally is detected by the photodiode. The objective of the track following systems here is to position the laser spot in radial direction with sufficient precision. We will discuss this in detail in Chapter 6.
x
Ω
optical disk
k
objective lens
fine motion unit
b
photo diode
laser diode mirror coarse motion
FIGURE 4.44 Track following system of an optical disk.
230
Optomechatronics
Let us consider the motion of the fine servo system. It is composed of a mass, a damper, and a spring attached to the VCM. The equation of the motion of the fine servo system is described by m
d2 x dx þb þ kx ¼ F dt dt2
ð4:57Þ
Here, m is composed of the mass of the lens and VCM, b is the damping coefficient, and F is the force induced by disk run-out and imbalance of the disk. The unbalance force comes in due to the fact that the disk itself is not perfectly balanced. It should be noted that, although F is very small, this influences the overall accuracy of track following, since track following accuracy is required to be highly precise. Dynamic Response of Dynamical Systems Once the system dynamics is modeled in a mathematical form, we now need to investigate how the system will respond to a given input. As we have modeled a variety of dynamical systems in the previous sections, we know that the complexity may differ from system to system. The simplest system we have observed is a first order system which is the case of the thermal system. And the next is a second order system, which the rest of the illustrative systems belong to. In this manner, dynamic systems may be in general represented by an nth order system whose dynamics is given by a0
dn x dn21 x dm u dm21 u þ a1 þ · · · þ a x ¼ b þ b þ · · · þ bm u n 0 1 n m dt dt dt1 dtm21
ð4:58Þ
where ai and bi are the system parameters and input parameters, respectively. Here, it is important to note that ai and bi may have two different types; time-invariant and time-variant. The two forms make a lot of difference in system analysis: The time invariant case is allowed to utilize the Laplace transformation method as we shall discuss later, whereas the latter case is not permitted, thus making system analysis much more difficult. At this stage we now identify the given system dynamic model as an inputoutput model, which forms the basic of control system. By defining u as an input and x as an output, we will start analyzing system response with a first order system. Let us first consider a first order system which is described by a differential equation of the form a0
dx þ a1 x ¼ b1 u dt
where uðtÞ is an input, and a0 , a1 and b1 are constants. If we let a0 =a1 ¼ t and
Mechatronic Elements for Optomechatronic Interface
231
1.2 t=1
step response, x (t )
1.0 0.8
t=4
0.6
t=3 t=2
0.4 0.2 0.0
0
2
4
6 8 time (t), sec
10
12
14
FIGURE 4.45 Step response for a first order system according to change of time constant.
b1 =a1 ¼ 1 for simplicity, then, this equation is modified as
t
dx þx¼u dt
ð4:59Þ
where t is called “time constant.” When a unit step input is applied to the system, i.e., u ¼ 1, ðt . 0Þ the steady state solution of this equation is obtained by xðtÞ ¼ ð1 2 e2t=t Þ
ð4:60Þ
which signifies that for a unit step input and zero initial condition the final value of xðtÞ reaches unity at a steady state, i.e., at t ¼ 1: The characteristics of the first order system differ largely by time constant, t: When t ¼ t the response becomes xðtÞ ¼ ð1 2 e21 Þ: At this time xðtÞ reaches 0.63 of its steady state value as shown in Figure 4.45. In a time of 3t, xðtÞ rises to 95% of the steady state value. Therefore, it is important to note that time constant t can be a measurement indicating speed of the response. When t p 1, the response is very fast, but, when t is relatively large, the response gets sluggish. The second order system which has the most practical importance in real situations takes the form of the following; a0
d2 x dx þ a1 þ a2 x ¼ b1 u dt dt2
where a0 , a1 , a2 and b1 are constants. This can be modified as d2 x dx þ v2n x ¼ b0 v2n uðtÞ þ 2jvn 2 dt dt
ð4:61Þ
232
Optomechatronics
where 2jvn ¼ a1 =a0 , and a2 =a0 ¼ v2n , b1 =a0 ¼ b0 v2n : In the above, vn is the natural frequency of the system, and j is the damping ratio. The roots of the characteristic equation p2 þ 2jvn p þ v2n ¼ 0 are given by p1 ; p2 ¼ 2jvn ^ pffiffiffiffiffiffiffiffi 2 vn j 2 1: Examination of the roots indicates that the equation can have three different roots depending upon j value; for underdamped case, j , 1, for critically damped case j ¼ 1, for overdamped case j . 1. When the roots j , 1 become complex conjugate, the response of the system becomes for a unit step input and zero initial conditions " ( )# j jvn t cos vd t þ pffiffiffiffiffiffiffiffi sin vd t xðtÞ ¼ b0 1 2 e 1 2 j2 pffiffiffiffiffiffiffiffi where the damped natural frequency is given by vd ¼ vn 1 2 j2 : The above equation reduces to " # e2jvn t xðtÞ ¼ b0 1 2 pffiffiffiffiffiffiffiffi cosðvd t 2 wÞ ð4:62Þ 1 2 j2 j w ¼ tan21 pffiffiffiffiffiffiffiffi 1 2 j2 Figure 4.46 illustrates typical responses for several values of j for b0 ¼ 1:0: As j becomes smaller, the response becomes more oscillatory with larger amplitude while speed of response gets faster. It is seen, however, that the maximum value of the response corresponding to the first peak value is increasingly higher with decrease of j value. As we further decrease in j we can anticipate that a sinusoidal response will eventually appear when j ¼ 0: 1.6 x=0.25
1.4 x=0.55
step response, x (t)
1.2
x=0.75
1.0
x=0.10
0.8 0.6
x=0.15
0.4 0.2 0.0
0
2
4
6 8 time (wn t )
10
12
FIGURE 4.46 Step response for a second order system according to changes of j value.
14
Mechatronic Elements for Optomechatronic Interface
233
When j ¼ 1, the roots are equally at 2vn , and the response becomes xðtÞ ¼ b0 ½1 2 e2vn t 2 vn te2vn t
ð4:63Þ
The response shows no oscillation but approaches the steady state value with slower speed than that of the case j , 1: When j . 1, two roots are real and distinct, and the response becomes xðtÞ ¼ b0 ½1 þ Aep1 t þ Bep2 t
ð4:64Þ
where p1 and p2 are the roots of the system. In this case, the response gets sluggish with even slower speed than the case where j ¼ 1: System Transfer Function As we have seen previously, the response of the system dynamics relating an input to an output has been obtained by solving the differential equation analytically. When the system dynamics come to higher order, the solution method must resort to a numerical approach. Since an analytical solution tool is not easily available, this time domain approach makes it difficult to understand the input –output relationship as well as the characteristics of system dynamics until a numerical solution comes out. To avoid this difficulty we usually transform a differential equation describing system dynamics into an algebraic equation in Laplace domain or s-domain. This transformation method is called Laplace transform which is an indispensable tool for control system analysis and design. This is because the s-domain transformation provides us with a transfer function that relates the input and output of a dynamic system. Once this is obtained in s-domain, we carry out the necessary calculation in algebraic form. To elucidate the concept in more detail, let us suppose that the input and output variables are transformed into Laplace domain (s-domain), respectively, as L½xðtÞ ¼ XðsÞ;
L½uðtÞ ¼ UðsÞ
where L denotes the Laplace transform of the variable in ½· : Utilizing the transformation, we may write the nth order dynamic equations in Equation 4.58, in a Laplace transformed form. The transfer function of the system describing the relationship between the input and output, can be written as GðsÞ ¼
XðsÞ UðsÞ
ð4:65Þ
where GðsÞ denotes the transfer function. It is important to notice that in transforming the differential equation all initial conditions are assumed to be zero. A block diagram representation GðsÞ is given in Figure 4.47. From this relationship, the response XðsÞ in s-domain can be easily obtained by XðsÞ ¼ GðsÞUðsÞ
ð4:66Þ
234
Optomechatronics G(s) 1 ts+1
U(s)
X(s)
(b) first order U(s)
G(s)
X(s) G(s)
(a) generic representation U(s)
wn2 s2
+ 2xwns + w n2
X(s)
(c) second order FIGURE 4.47 A block diagram representation of system transfer function.
if UðsÞ is given. Therefore, the important properties of system dynamics such as stability and characteristics of system response can be analyzed by manipulating the above algebraic equation in s. If necessary, we can easily go back to the time domain from the s-domain in order to do some analysis simply by inverse-transforming the above equation in s L21 ½XðsÞ ¼ L21 ½GðsÞUðsÞ ¼ L21 ½GðsÞ p L21 ½UðsÞ where the symbol p indicates the convolution operation. Let us find out how we obtain these transfer functions by revisiting the differential equations we have dealt with previously. First Order System Consider again the dynamic system described by a first order differential equation given in Equation 4.59. The equation is rewritten here for subsequent discussion
t
dx þx¼u dt
Laplace transform of this is given by
t sXðsÞ þ XðsÞ ¼ UðsÞ which for zero initial condition leads to the following transfer function; GðsÞ ¼
XðsÞ 1 ¼ UðsÞ ts þ 1
A block diagram representation is shown in Figure 4.47b. When UðsÞ is a unit step input, then XðsÞ is given by XðsÞ ¼
1 sðts þ 1Þ
Mechatronic Elements for Optomechatronic Interface
235
It can be shown that Laplace inverse transforming of this equation yields the same differential equation given in Equation 4.60. It is worthwhile to remembering that time constant t determines the speed of the system response, as already discussed. Second Order System Consider the second order system described by the equation given in Equation 4.61 which is rewritten as d2 x dx þ 2jvn þ v2n x ¼ b0 v2n uðtÞ 2 dt dt When we assume Laplace transforming of the above equation with all zero initial conditions we obtain s2 XðsÞ þ 2jvn sXðsÞ þ v2n XðsÞ ¼ b0 v2n UðsÞ from which the transfer function is obtained by GðsÞ ¼
XðsÞ b0 v2n ¼ 2 UðsÞ s þ 2jvn s þ v2n
ð4:67Þ
The block diagram of GðsÞ is described in Figure 4.47c for b0 ¼ 1: As we have already discussed, the roots of denominator of GðsÞ ¼ 0, that is, s2 þ 2jvn s þ v2n ¼ 0 yield three different cases, depending upon j value, as before. pffiffiffiffiffiffiffiffi When j , 1; Two roots are at p1 , p2 ¼ 2jvn ^ jvn 1 2 j2 which are complex conjugates. By using these roots, Equation 4.67 can be rewritten as GðsÞ ¼
v2n ðs1 þ p1 Þðs þ p2 Þ
ð4:68Þ
When UðsÞ is a unit step input, XðsÞ is obtained by XðsÞ ¼
v2n pffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi s s þ jvn þ jvn 1 2 j2 s þ jvn 2 jvn 1 2 j2
By Laplace inverse transforming of this, we have exactly the same response equation given in Equation 4.62. When the system is critically damped, i.e., j ¼ 1 two roots are equally at p ¼ p1 ¼ p2 ¼ 2vn , GðsÞ in this case is given by GðsÞ ¼
v2n ðs þ pÞ2
from which XðsÞ is obtained for UðsÞ ¼ 1=s by XðsÞ ¼
b0 v2n sðs þ vn Þ2
The time domain response of this is identical to that given in Equation 4.63. When j . 1 two roots are distinct and real and located at
236
Optomechatronics
pffiffiffiffiffiffiffiffi p1 ,p2 ¼ 2jvn ^ vn j2 2 1, we have GðsÞ ¼
b0 ·v2n sðs þ p1 Þðs þ p2 Þ
XðsÞ is then given by XðsÞ ¼
b0 ·v2n pffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi sðs þ jvn þ vn j2 2 1Þðs þ jvn 2 vn j2 2 1Þ
The Laplace inverse transform of this equation yields the same time domain response as given in Equation 4.64. Higher Order System When a system is described by the nth order differential equation given in Equation 4.58, a general form of its transfer function is represented by m Y
ðs þ z1 Þðs þ z2 Þ· · ·ðs þ zm Þ j¼1 ¼ n GðsÞ ¼ Y ðs þ p1 Þðs þ p2 Þ· · ·ðs þ pn Þ i¼1
ðs þ zj Þ ð4:69Þ ðs þ pi Þ
if the denominator and nominator are factored out by use of n poles, pi ði ¼ 1,2· · ·nÞ and m zeros zj ðj ¼ 1,2· · ·mÞ: The pi and zj may include the second pffiffiffiffiffiffiffiffi order complex conjugate terms s þ jvn ^ vn j2 2 1: It is noticed that a system to be realizable needs to satisfy the condition n q m: The Laplace transform of this general equation can be made in a similar way to the case of the second order system. This is done by use of the partial fraction method which decomposes the transfer function into several simple first or second order terms. Once the transfer function can be decomposed like this, then control system analysis and design will be much easier than handling the unfactored form. Laplace Transforms Theorems There are several theorems on the Laplace transform that are useful in the analysis and design of control systems. Here, we present them without proof. Final value theorem: If f ðtÞ and df ðtÞ=dt are Laplace transformable, and if f ðtÞ approaches a finite value as t ! 1, then lim f ðtÞ ¼ lim sFðsÞ
t!1
s!0
it is noted that when f ðtÞ is a sinusoidal function such as cos vt, and sin vt, this theorem is not valid, since for these functions lim f ðtÞ does not exist. t!1 This theorem is frequently used to determine the steady state error of the response of a control system.
Mechatronic Elements for Optomechatronic Interface
237
Initial value theorem: If f ðtÞ and df ðtÞ=dt are Laplace transformable and if lim sFðsÞ exists, then s!1 f ð0þ Þ ¼ lim sFðsÞ s!1
where t ¼ 0þ implies the positive side of time at t ¼ 0, and this definition together with t ¼ 02 is useful for a function having discontinuity at some instant of time. This theorem is useful to determine the initial value of the response of a control system at t ¼ 0þ , and thus the slope of the response at that time. The initial value theorem and final value theorem enable us to predict the system behavior in the time domain without actually transforming Laplace transformed functions back to time functions. Open Loop vs. Feedback Control A system normally consists of a series of several transfer function elements as shown in Figure 4.48a. In this case, the block diagram is reduced to obtain the overall transfer function GðsÞ XðsÞ ¼ G1 ðsÞG2 ðsÞG3 ðsÞ ¼ GðsÞ UðsÞ
ð4:70Þ
This equation signifies that once the desired output value is specified we can determine one-to-one correspondence between the input and output. For instance, when a desirable xd ðtÞ is given, we can provide the system with an input UðtÞ so as to make the response xðtÞ ! xd ðtÞ as time t goes to infinity. This is true only when the system is not subject to external disturbance, which will be discussed shortly. This implies that this type of control is called open loop control and does not use any measurement information. In contrast to this, the feedback or closed loop control utilizes an instantaneous measurement information obtained by a sensor or sensors for feedback, as shown in Figure 4.48b. The overall system transfer function can
U(s)
G2(s)
G1(s)
G 3(s)
X(s)
Xd(s)
G(s) equivalent
(a) open loop control FIGURE 4.48 Open loop control vs. feedback control.
G(s)
X(s)
original
original
U(s)
+_
X(s)
Xd(s)
G(s) 1 + G(s) equivalent
(b) feedback control
X(s)
238
Optomechatronics
be obtained upon consideration of error variables which is defined by EðsÞ ¼ Xd ðsÞ 2 XðsÞ Since XðsÞ=EðsÞ ¼ GðsÞ, we finally have XðsÞ GðsÞ ¼ Xd ðsÞ 1 þ GðsÞ
ð4:71Þ
Comparison of Equation 4.70 and Equation 4.71 enables us to examine two basic differences between the open loop control and closed loop control. The first one is that the closed loop control is much less sensitive to the variation of system parameters. To illustrate this let GðsÞ be changed by DGðsÞ from its original function. Then open loop control :
XðsÞ ¼ ½GðsÞ þ DGðsÞ UðsÞ
closed loop control :
XðsÞ ¼
GðsÞ þ DGðsÞ X ðsÞ 1 þ GðsÞ þ DGðsÞ d
In the open loop system, the response XðsÞ is varied by the amount DGðsÞUðsÞ, while in closed loop control XðsÞ is varied approximately by DGðsÞ X ðsÞ 1 þ GðsÞ d Therefore, sensitivity to variation of system parameters is greatly reduced by a factor 1=ð1 þ GðsÞÞ: The second one is that the closed loop control is much less sensitive to external disturbance. To illustrate this let DðsÞ be the Laplace transform of a disturbance dðtÞ, as indicated Figure 4.49, XðsÞ given in Equation 4.70 is modified as open loop control :
XðsÞ ¼ GðsÞUðsÞ þ DðsÞ
closed loop control :
XðsÞ
GðsÞ 1 X ðsÞ þ DðsÞ 1 þ GðsÞ d 1 þ GðsÞ
We can see again that the effect of the disturbance can be drastically reduced with closed loop control. However, one thing to take into careful consideration is that the closed loop system often becomes unstable even if the system is open loop stable. Therefore, the closed loop control system design should take into consideration system stability. In conclusion, the effects of system variation and disturbance on system response can be controllable or even eliminated with some types of control action we will discuss later. System Performance As we have seen already, the system response of the second order system is largely dependent on damping ratio, j and natural frequency, vn : In other words, these system variables determine the response characteristics at both
Mechatronic Elements for Optomechatronic Interface
239 D(s)
(s)
+
G(s)
+
X(s)
open loop
D(s) X d(s)
+_
G(s)
+
+
X(s)
closed loop FIGURE 4.49 Inclusion of an external disturbance.
transient and steady states. This indicates that, once the desired characteristics are specified, we need to determine the corresponding system variables, j and vn according to the specification. The problem is how we define such characteristics. In many practical cases, response characteristics of a dynamical or controlled system are specified in terms of time domain parameters that are indicative of system performance. Because the transient response of a practical system often exhibits damped oscillatory behavior before reaching state, we will define time domain parameters by using an oscillatory transient response curve we have observed from the second order system. In defining such parameters, it is common to use a unit step input: in fact, there are several types of test signals such as impulse, unit step, ramp, parabolic, sinusoidal and random signal as shown in Figure 4.50. The choice of the test signal is dependent upon what type of response a system is required to produce. For example, if the system is a positioning system, the step type test signal may be useful to test its response. Referring to Figure 4.51 which shows a typical response curve of dynamic system for a unit step input, we may specify the time domain parameters commonly used for evaluating the response characteristics. For transient response: (1) rise time ðtr Þ : the time required for the response to rise from 10 to 90%, 5 to 95% or 0 to 100% of its final value, (2) peak time ðtp Þ : the peak time is the time required for the response to reach the first peak of the overshoot,
240
Optomechatronics
(1) impulse
(2) step
(4) parabolic
(3) ramp
(5) sinusoidal
(6) tracking
FIGURE 4.50 Various test signals to evaluate system response.
(3) maximum overshoot ðMp Þ : this quantity is defined by Mp ¼
xðtp Þ 2 xð1Þ £ 100% xð1Þ
For steady state response:
1.2
Mp
response, x (t )
1.0
2%~5%
0.8 0.6 0.4 0.2 0.0 tr
tp
ts
FIGURE 4.51 Characterization of a system response for a unit step input.
time (t )
Mechatronic Elements for Optomechatronic Interface
241
(4) settling time ðts Þ : the time required for the response to reach and stay within a range of 2 to 5% of the final value. (5) steady state error: the error which is the deviation from the final value at steady state eð1Þ ¼ 1 2 xð1Þ According to this specification, we may calculate the following time domain parameters for the second order system. tp ¼
pffiffiffiffiffi2 p 4 3 , Mp ¼ e2ðj 12j Þp , ts ¼ ð2% criterionÞ or ð5% criterionÞ ð4:72Þ vd jvn jvn
From this specification, it is noted that all time domain parameters are a function of j and vn : Clearly, these parameters are a family of parameters that can characterize the system performance. As long as we keep all the parameters ðtp ,Mp ,ts Þ as small as possible, we can achieve the best desirable response, if there is no steady state error. The second order system given in Equation 4.67 has no steady state error if b0 ¼ 1: In general, steady state error appears in most control systems, although it is small. In some cases this small error may not be tolerable which requires careful design of a controller that may reduce or eliminate error altogether. One important thing to be noticed is that there is always a trade off between rise time ðtp Þ and maximum overshoot ðMp Þ because shorter rise time implies larger overshoot, and vice versa. A higher order system can also be specified using these parameters only in certain simple situations. Normally, we need to make an appropriate assumption for system analysis and design or to determine the parameters by obtaining the response numerically (Simulink). Basic Control Actions Most dynamical systems do not exhibit desirable response characteristics unless they are controlled properly. For instance, some systems may not satisfy the transient response requirements while some others may not satisfy the steady state response requirements. In severe cases, they may be inherently unstable for some range of operation. The effective way of correcting the unwanted response characteristics is to use a feedback control method. As we have already pointed out before, this method utilizes the information on the instantaneous state of systems, which is acquired by sensors. Based upon the sensor information, system actuators adjust their manipulating signal so as to reduce or eliminate the error that occurs at any instant of time and eventually the desired response. Figure 4.52 illustrates a generic form of a block diagram of a control system. It is composed of a controller, GC ðsÞ which produces a command signal to an actuator depending upon the error variable, an actuator that actuates the system,
242
Optomechatronics
according to the command signal from the controller, a dynamic system GP ðsÞ to be controlled and finally a sensor GS ðsÞ that measures the instantaneous state of the system. All of these elements are of paramount importance to obtain desired system performance, because each affects system response characteristics. In particular, sensors and actuators must be suitably chosen so that their dynamic characteristics meet the overall system specification. In addition, the control signal should be properly generated by the controller, taking into account the dynamic characteristics of the system, actuators, sensors, and disturbances. There are a variety of controller types that may be categorized into several groups but here we will discuss in brief only three classical controllers; proportional (P) controller, proportional plus integral (PI) controller, proportional plus integral plus derivative (PID) controller. Upon examination of the three controllers listed above, it can be observed that combination of three basic actions enables those controllers to generate appropriate command signals. These are proportional (P), derivative (D), and integral (I) actions. Figure 4.53 shows those three basic actions. Proportional control action is based upon the proportional relationship between the controller output uðtÞ and the error signal eðtÞ which is described by uðtÞ ¼ kp eðtÞ
ð4:73Þ
or in Laplace-transformed quantities UðsÞ ¼ kp EðsÞ
ð4:74Þ
where kp is termed the proportional gain. This control action produces a signal that is linearly proportional to the error between the desired set value and the actual output value. Therefore, this type of action may be said to be one of the simplest control actions among the three. In integral action, the value of controller output uðtÞ is changed at a rate proportional to the integral
disturbance D(s) command signal
X d(s) desired value
+_
error controller E(s) Gc(s)
actuating signal actuator
U(s)
Ga(s)
Gs(s)
measured variable
sensor
FIGURE 4.52 A generic representation of a control system.
M(s)
+
+ Gp(s)
X(s) actual output
Mechatronic Elements for Optomechatronic Interface
desired value
Gc(s)
+_
243
Ga(s)
sensor signal
kp
+_
(a) proportional
+_
ki s
(b) integral
+_
kds
(c) derivative
FIGURE 4.53 Basic control actions.
of the error, whose relationship is given by ðt UðsÞ k uðtÞ ¼ ki eðtÞdt or ¼ i EðsÞ s 0
ð4:75Þ
where ki is the integral constant. This action provides a signal that is a function of all past values of error rather than just the current value. The action is effective in the steady state rather than in the transient state, because in the steady state, the error may be accumulated as time increases. The last type is the derivative action which has the form uðtÞ ¼ kd
de dt
or
UðsÞ ¼ kd s EðsÞ
ð4:76Þ
This control action depends upon the rate of change of the error. As a result, a controller with derivative action exhibits an anticipatory error. As can be observed, this action is effective during the transient period but not at steady state where not much variation in response occurs. It has the effect of stabilizing the response. The PI controller is composed of a proportional (P) action and integral action (I). The control signal is given by ðt k uðtÞ ¼ kp eðtÞ þ ki eðtÞdt ð4:77Þ or UðsÞ ¼ kp þ i EðsÞ s 0 As can be seen, the controller gains kp and ki play the role of a weighting factor on each term. If kp . ki , the proportional controller contributes to the command signal more than the integral part does. On the other hand, if ki . kp , the integral action contributes more. This controller may be effective when a system needs faster response and requires no steady state error due to constant disturbance. Electronic implementation of this controller in a single circuit is shown in Figure 4.54a. Refer to the basic circuits illustrated in Figure 4.16.
244
Optomechatronics
R2 R1
C1
R1
R3 R2
– +
C1
(a) PI controller
– +
(b) PD controller R2
C1
C2
– +
R1
(c) PID controller FIGURE 4.54 Electric implementation of three controllers.
The proportional plus derivative (PD) controller may be represented by uðtÞ ¼ kp eðtÞ þ kd
deðtÞ dt
or
UðsÞ ¼ ðkp þ kd sÞEðsÞ
ð4:78Þ
This controller may be effective when a controlled system requires proper amount of damping without too much loss of response speed. Figure 4.54b shows the circuit implementing this controller. The most general type of controller is basically a three mode controller which is the PID controller defined by ðt deðtÞ or uðtÞ ¼ kp eðtÞ þ ki eðtÞdt þ kd dt 0 ð4:79Þ ki þ kd s EðsÞ UðsÞ ¼ kp þ s When a system to be controlled requires a proper amount of damping, a proper speed and no steady state error to constant external disturbances, this control action may be effective. Again, it is noticed that kp , ki and kd can be interpreted as the weighting factor of each action for the controller. For instance ki q kp , kd implies that integral action contributes greatly in comparison to the other two actions. Figure 4.54c shows a simple PID controller combined in a single amplifier circuit. System Stability Stability of a system is a property that needs to be checked first, because if a system is unstable, then the response characteristics we have discussed in the above is not meaningful at all. System stability can be checked by
Mechatronic Elements for Optomechatronic Interface
245
Im [s]
neurally stable Re[s] = 0; s = ± j ω stable region Re[s] < 0
unstable region Re[s] > 0 O
Re[s]
FIGURE 4.55 Stability in complex s-plane.
examining the denominator of GðsÞ given in Equation 4.69. The stability requires that all roots satisfying the characteristic equation (denominator set equal to zero) must lie in the left-half s plane as indicated in Figure 4.55. To be more specific, all roots that satisfy DðsÞ ¼ ðs þ p1 Þðs þ p2 Þ…ðs þ pn Þ ¼ 0
ð4:80Þ
must be strictly in the left hand side of the complex plane excluding jv axis. In other words, all pi s must have negative real part. If any one of pi s does not meet this requirement, the system is said to be neutrally stable or unstable. This stability check can be made by using the Routh Stability Criterion in Appendix A3. For a neutrally stable system, the roots lie in the jv axis and therefore its response exhibits a typical sinusoidal oscillation with constant amplitude but does not increase with time. For an unstable system, its response to any input will increase monotonically with time or oscillate with increasing amplitude. An important remark to be made here is that whether a linear system is stable or unstable is a property of the system itself, but not a property of external input, unless the input is dependent upon the system response.
Problems P4.1. Consider a capacitance sensor composed of two parallel plates separated by a dielectric material. Its original overlapping length is ‘: Suppose that the upper plate is moved by D‘ as shown in Figure P4.1. Using Equation 4.2, determine the sensitivity function defined by S ¼ DC=D‘:
246
Optomechatronics moving w
fixed d
FIGURE P4.1 A schematic of a capacitance sensor with changing area.
P4.2. When a dielectric material between two plates moves by ‘ in the direction of the x axis, as shown in Figure P4.2, obtain the total capacitance of this sensor. 1r1 and 1r2 are the permittivity of the relative dielectric constants of material 1 and material 2, respectively. plate dielectric εr1
dielectric εr2
x
FIGURE P4.2 A capacitive sensor with dielectric area change.
P4.3. A piezoelectric sensor consists of a piezoelectric transducer, a connecting cable and an external circuit, as shown in Figure P4.3. Assume that it is modeled as a spring-mass-damper system having stiffness kp , damping coefficient bp and mass mp :
applied force, F z
RL i
Cp
Cc
VL
piezoelectric transducer piezoelectric transducer
cable
load
FIGURE P4.3 Piezoelectric sensor configuration.
Suppose that the cable is represented by a pure capacitance element Cc and that the external circuit is represented by a pure load RL : (1) Write down the equation of motion of the sensor in the z direction for a given force F.
Mechatronic Elements for Optomechatronic Interface
247
(2) If the transducer is modeled by a current generator with a capacitance Cp , write down the equation for the electric circuit shown in the figure. (3) Obtain the transfer function between the force F and the output voltage VL : P4.4. Suppose that there is a point light source (object) S distant from a detector (D)
object S FIGURE P4.4 A detector receiving a bundle of rays from an object surface.
detector, D, as shown in Figure P4.4. When high efficiency of sensing is required, suggest a method to increase the efficiency. R2
R1
C
Vin
Vout
FIGURE P4.5 Integral operation by op-amp.
P4.5. Shown in Figure P4.5 is an electronic circuit of the integral operation. Obtain the equation describing the input– output relationship. P4.6. In micro electro-mechanical-systems (MEMS) or micro actuator and sensors, clamped beam structure has a variety of interesting deformable beam h fixed beam
d
V
δ fixed
FIGURE P4.6 A capacitive clamped–clamped type actuator.
248
Optomechatronics
applications. One such application can be found from a capacitive type actuator as shown in Figure P4.6. The two beams are separated initially by d: The upper beam is deformable, while the lower beam is fixed. If the electric field V is applied to the deformable beam, determine the force per unit length acting on the beam element. In the figure, ‘, w and h are the length, width, and thickness of the beam, respectively. P4.7. A spring of mass m is made of shape memory alloy (SMA). The objective of the use of the SMA is to give the mass an oscillatory motion in a proper way (See Figure P4.7).
SMA spring
x m FIGURE P4.7 SMA actuated spring motion.
(1) Describe how the vibratory motion can be produced. (2) Based on this principle, express roughly the equation of the motion, taking into consideration the material hysteresis effect. P4.8. Figure P4.8 shows the use of a Wheatstone bridge to measure strain. When the deformation of the strain gauge occurs, it will cause the gauge resistance to change, DR: If the bridge has equal resistance R, show that Vout is determined by Vout ¼ ðVS Rf =RÞðDR=2Þ: In the figure Vs is the supply voltage and Rf is the feedback resistance of the amplifier. P4.9. A galvanometer recorder utilizes a UV light as a light source, a mirror, and a moving strip of photosensitive paper. The mirror is attached to a moving coil assembly which is suspended by a torsional string and a viscous damper. The coil is moving in a magnetic field under the same principle as that of the electric DC motor. While the paper is moving in the x direction, the mirror reflects an incident beam to a point Pðx; yÞ on the paper as depicted in Figure P4.9.
Mechatronic Elements for Optomechatronic Interface
249
VS Rf R1
R2 i1
i2
i3
i4
R+∆R
i5
i6 na Vout
R4
ng Rf
strain gauge
R+∆R
FIGURE P4.8 Bridge arrangement with a differential amplifier for strain measurement.
(1) If the coil assembly has a moment of inertia I, and is suspended by a torsional spring, kt , and a viscous damper, b, write down the equation governing the mirror motion u: Assume that the magnetic flux density is B, n is the number of coil turns, A is the cross sectional area of the magnet, and i is the current flowing through the coil.
rotational damper
b
θ
mirror
photo sensitive paper
magnetic field B
magnet N tortional spring
P (x,y)
moving
S
i kt
x y S UV light source
FIGURE P4.9 Strip chart-galvanometer recorder.
250
Optomechatronics
(2) Suppose that the mirror is rotated by u from an initial angle uð0Þ ¼ 0, with zero angular velocity du=dtð0Þ ¼ 0: Write down the transfer function between the current iðtÞ and uðtÞ: Discuss the mirror motion in terms of natural frequency and damping ratio of the mirror system. (3) If the mirror is rotated by small amount Du from its initial angular position, what will be the relationship between y and Du ? Assume that the initial angle of incident beam is at u ¼ ui with respect to some reference angle. P4.10. A closed loop control system is shown in Figure P4.10. For control purposes, the microprocessor takes in the information obtained by a sensor whose signal is contaminated by noise as shown in the figure. Then the signal is sampled once within a period Dts and based upon this sampled signal, the controller generates a command signal to an actuator which drives the system. (1) Suppose that the signal of the output variable has a frequency range 10 Hz # fs # 50 Hz, and noise frequency ranges fn $ 60 Hz: What type of filter can be used to reduce noise? What kind of considerations should be given to design such a filter? (2) To ensure a good performance of the control system, what frequency of the control action is recommended? Assume that the control action ð fc Þ is carried out at least twice within one sampling frequency. P4.11. Consider a mirror driven by a gap closing actuator supported by a spring whose stiffness is k as depicted in Figure P4.11. When m is total mass of the mirror unit, z is the displacement from the equilibrium state in downward direction, V is the applied voltage to the actuator, and z0 is the original gap, the equation of motion of the mirror is the same as given Equation 4.51 in the text. (1) Discuss the behavior of the mirror motion depending upon the applied voltage. (2) Describe the behavior of the mirror motion when the mirror approaches the lower plate. (3) Show a block diagram of a feedback control system to maintain the mirror position at a desired position, and explain its control concept. P4.12. Consider the piezoelectric sensor treated in Problem P4.3. If the dynamics of the sensor is given such that its transfer function is described by GðsÞ ¼
VðsÞ tv2n s ¼ FðsÞ ðts þ 1Þðs2 þ 2jvn þ v2n Þ
A/D
controller
measured value
FIGURE P4.10 A signal processing and control for a digital control system.
(b) output signal with noise
O
amplitude
(a) feedback system
xd
desired value
sampled signal
sensor
D/A
actuator
system
time (t )
actuation signal
x( t )
actual output variable
Mechatronic Elements for Optomechatronic Interface 251
252
Optomechatronics
mirror
m leaf spring k
v
z actuator
FIGURE P4.11 A mirror driven by an electrostatic actuator.
(1) Discuss the effect of t on the system response as t decreases from a large value to a very small value. In the case where t # 1, what will be the type of response characteristics? (2) Plot the response of the sensor system to a unit step input of force FðsÞ ¼ 1=s, using the Simulink model in Matlab. The parameters are given in the followings: vn ¼ 1:6 £ 105 rad=sec, j ¼ 0:01, t ¼ 2:0 msec: (3) Determine the steady state error for (a) a unit step input force and (b) a unit ramp input force.
References [1]
Ando, T. Laser beam scanner for uniform halftones, Printing Technologies for Images, Gray Scales and Color, SPIE, Vol. 1458. 1991. [2] Auslander, D.M. and Kempf, C.J. Mechatronics — Mechanical System Interfacing. Prentice Hall, Englewood Cliffs, 1996. [3] Benech, P., Chamberod, E. and Monllor, C. Acceleration measurement using PVDF, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 43: 5, 838– 843, 1996. [4] Bentley, J.P. Principles of Measurement Systems, 4th ed. Prentice Hall, Englewood Cliffs, 2004. [5] Bolton, W. Mechatronics Electronic Control Systems and in Mechanical Engineering. Addison Wesley/Longman, Reading MA, London, 1999. [6] Bradley, D.A., Dawson, D., Burd, N.C. and Loader, A.J. Mechatronics. Chapman and Hall, London, 1991. [7] Dally, J.W., Riley, W.F., and McConnell, K.G. Instrumentation for Engineering Measurements. John Wiley and Sons Inc, New York, 1984. [8] Fraden, J. Handbook of Modern Sensor. American Institute of Physics, New York, 1993. [9] Fukuda, T., Hattori, S., Arai, F. and Matsuara, H. Optical servo systems using bimorph PLZT actuators, The American Society of Mechanical Engineering (ASME), 46, 13 – 19, 1993.
Mechatronic Elements for Optomechatronic Interface [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]
[21] [22]
253
Fukuda, T. Performance improvement of optical actuator by double side irradiation, IEEE Transactions on Industrial Electronics, 42, 455– 461, 5 October, 1995. Gweon, D.G., Special Topics in Dynamics and Control, Lecture Note, MAE850, Korean Advanced Institute of Science and Technology, 2004. Kenjo, T. and Sugawara, A. Stepping Motors and Their Microprocessor Controls, 2nd ed. Oxford Science Publications, Oxford, 1995. Lin, R., Shape Memory Alloys and Their Application, http://www.standford. edu/, richlin1/sma/sma.html, 2005. Luxon, J.T., Parker, D.E., 2nd ed., Industrial Lasers and Applications. Prentice Hall, Englewood Cliffs, 1992. Mahalik, N.P. Mechatronics: Principles, Concepts and Applications. McGraw-Hill Inc, New York, 2003. Near, C.D., et al. Sensor Technology Limited (Product Information), SPIE, Vol. 1916. pp.396 – 404, Oxford, 1993. Pluta, M.. Advanced light Microscopy Principles and Properties. Elsevier, Amsterdam, 1988. Sashida, T. and Kenjo, T. An Introduction to Ultrasonic Motors. Oxford Science Publications, 1993. Senturia, S.D. Microsystem Design. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2001. Shen, Y., et al. A high sensitivity force sensor for microassembly: design and experiments advanced intelligent mechatronics, Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Kobe, Japan, Vol. 2. 2003, pp. 703– 708. Stadler, W. Analytical Robotics and Mechatronics. McGraw-Hill Inc, New York, 1995. Tabib-Azar, M. Microactuators. Kluwer Academic Publisher, Dordrecht, 1998.
5 Optomechatronic Integration CONTENTS Basic Considerations for Integration .............................................................. 256 Basic Functional Modules ................................................................................ 263 Signal Transformation ............................................................................... 263 Signal Manipulation .................................................................................. 264 Signal Sensing............................................................................................. 266 Signal Actuation ......................................................................................... 269 Signal Transmission ................................................................................... 270 Signal Display............................................................................................. 272 Optomechatronic Interface or Integration ..................................................... 273 Basic Two-Signal Integration.................................................................... 273 Fundamental Optomechatronic Integration: Illustrations................... 277 Generic Forms for Optomechatronic Interface ..................................... 279 Integrability................................................................................................. 288 Signal Flow.................................................................................................. 289 Integration-Generated Functionalities............................................................ 291 Problems.............................................................................................................. 294 References ........................................................................................................... 297
In the previous chapters we have reviewed some of the fundamentals of the two engineering fields, optical and mechatronic engineering. During the process of the review, we have seen that basic principles and concepts, and their relevant theories and tools, can be combined to interact with each other to create new concepts or technical elements. In fact, we have seen from the illustrations discussed in Chapter 1 that a variety of the integrated components and systems result from those interactions among optical, mechanical, and electrical technologies. Further, we have seen that such integrations can be derivable from certain principles of physical interaction. Judging from the observations, we can summarize the following characteristic points:
255
256
Optomechatronics
Hierarchical structure of integration: Optomechatronic integration is hierarchically composed of several levels of integration, so its architecture is composed of functional coordination level, structural integration level, and organization level. Fundamental functionalities: Functionality is a task-oriented functional entity required for a system to execute a given task specified by the designer. Such a functionality may be produced by combining several fundamental functional modules. Basic functional modules: The functional modules are the functionoriented modules required to produce a functionality in a specified manner. These functional modules are combined together to create the basic functionalities required by optomechatronic integration. Integration interface: There are certain laws for putting the functional modules together. The integration must consider the interfacing mechanism between the optical, mechanical, and electrical components. At the same time, it must take into consideration the signal flow that represents physical phenomena that occur during the interaction involved with the integration. In this chapter, we will have a more detailed look into physical optomechatronic components and systems, in order to investigate their characteristics regarding the composition of their structure and the fundamental functionalities. To this end, we will consider the basic functional modules that create such functionalities, and study the signal flow in order to facilitate the composition of the modules needed for functionality generation. Here, signal is meant in the broad sense by energy or information. Based upon the analysis of the functional modules, we will discuss the nature of integration in order to find common modalities that might exist among various integrations of the three engineering fields. From these findings, we finally derive the optomechatronic integration and analyze the characteristics of the integration, which include integrability, integration structure, and information signal flow. To better understand the underlying concept, we will illustrate a variety of examples of physical systems from the view point of integration.
Basic Considerations for Integration Looking into the details of optomechatronic technology from this viewpoint, we can see that it involves interaction among three engineering fields, optical, mechanical, and electrical. The interaction between the three signals will differ, depending upon the type of the integration and the strength of contribution of each technical element. The dependency will, therefore, determine the nature of optomechatronic integration. If we can formulate the relationship, the type, and strength of the integration vs. the characteristics
Optomechatronic Integration
257
of the integration, developing integration methodology will become easy for a given set of problems. However, as we might anticipate, it is an extremely difficult task to formulate and analyze the relation even with a heuristic manner. Nevertheless, we will make an effort to dig out the underlying natures of the integration by introducing basic functional modules that can be the basis for the integration, and the analysis of the structure of the integration. As listed in Table 5.1, three engineering fields (O, E, M) have distinct physical variables originating from their own physical phenomena. Integration of these fields implies that these unique, distinct variables are interacting with each other to affect some of the interacting variables, or create new physical variables. As indicated in Figure 5.1, the unique optical variables include light energy, ray intensity, and radiation flux, the mechanical variables include mechanical energy, motion, deformation, strain, and fluid/heat flow, and electrical variables include current, voltage, charge, and magnetic flux. We will represent these unique variables in the following manner: mechanical variables as a mechanical signal, optical
TABLE 5.1 Signals and Basic Elements of Mechanical, Electrical, and Optical Engineering Mechanical Force ( f ) Velocity (v) Power ( p) (mechanical) Energy (E)
Electrical
Optical
Signal Current (i) Voltage (v) Power ( p) (electrical) Energy (E)
Ray (r) Intensity (I ) Irradiance (R) (radiant, luminous) Radiant energy (E )
Basic elements Mass (m) m
Capacitor (c)
Prism
Spring (k)
Coil (L)
Beam splitter
Damping (b)
Resistor (R)
Lens
Gear
Transformer
Cam
Alternator
Belt
Inductor
Bellows
Semiconductor p
Mirror N
S
Stop Aperture
n
Gratings
258
Optomechatronics
energy, ray, intensity, radiation flux ti
na
l
op
ca
sig
l
ical chan me
energy, current, voltage, magnetic flux charge
al ctric ele
signal
energy, motion, strain, fluid/heat flow
signal
FIGURE 5.1 Optical, mechanical, and electrical signals.
variables as an optical signal, and electrical variables as an electrical signal. The signals are expressed in the following form: optical signal ¼ O (energy, ray, intensity, radiation flux…) mechanical signal ¼ M (energy, motion, fluid flow deformation, strain…) electrical signal ¼ E (energy, current, voltage, charge, magnetic flux…) The interaction can occur with various modalities, depending largely upon: (1) which variables are interacting, (2) how many variables are interacting, (3) how and when they are interacting, and (4) finally, the sequence and the duration of the interaction. It will be a formidable, or even an impossible, job to formularize the integration modalities. Rather we will try to find a relevant integration concept, taking into consideration the fundamental questions mentioned above. To start with, we will assume that all three signals contain six or fewer fundamental functional modules, as shown in Figure 5.2. These elements comprise transformation, manipulation, sensing, actuation, transmission, storage, and display of a signal. Let us discuss them one by one, in detail, to identify their characteristics. Signal transformation is the conversion of one signal form into another as a result of a change of physical elements. Signal manipulation is the action of diverting or modifying a signal. Sensing is the measurement of the state of a signal. Signal actuation is the control of a signal in order to maintain it at a certain value. Signal transmission is the action of transmitting a signal to a desired location. Finally, signal display is the presentation of a signal in a presentation element. Optomechatronic integration may often involve the combination of multiples of these functional modules, but, in a simple example, may
Optomechatronic Integration
259
FIGURE 5.2 Basic technological elements for optomechatronic technology.
involve only one single module. For instance, the optical disk shown in Figure 1.6c has several physical components that carry out their own tasks: an optically-encoded rotating disk, a tracking unit that searches a certain data location, a track following unit, and auto focusing and reading units. The rotating disk is a signal storage device that has a signal storage module (optically recorded element), and a signal transformation module (motor) that makes the disk rotate by an electric motor at a specified rotation speed. The tracking unit has a signal transforming component (moving optical head) and a sensing module (optical element) that measures the data track. The auto-focusing and reading unit has a signal control module and a sensing module as well. All of these modules involved here represent optomechatronic integration. Taking another example, the optical switching shown in Figure 1.6f has one functional module, “signal modulation,” which is achieved by optomechatronic integration composed of optical elements (light, mirror) and a mechatronic element (actuator) that actuates the mirror. From these two examples, it may be said that the optomechatronic integration, indeed, is necessary, not only for combining functional modules, but also for making each individual technical component work as desired. The principle of the optomechatronic integration concept departs from this concept. Figure 5.3 depicts a standard procedure which indicates the integration processes involved. According to the procedure, once the design specification is given, the required functionalities are to be checked first, in order for the desired tasks to be carried out by the system or device to be designed. The next thing to do is to identify all the functional modules that might be involved in producing the desired functionalities. Since all those
260
Optomechatronics
design specification
create the functionality
identify the functional modules
integration of optomechatronic components
system integration
verify the performance good
no
yes end FIGURE 5.3 The procedure of optomechatronic integration.
modules may not be feasible for the intended integration, optomechatronic integrability must be checked to investigate its feasibility. After the integrability is confirmed, then optical, mechanical, and electrical hardware elements are to be integrated, so as to meet various requirements set by each individual module or the combined ones. The final stage is to verify the performance of the integration. The steps for optomechatronic design are similar to those standard procedures for designing most engineered products or systems. However, there are several differences. In creating the functionalities, the optomechatronic designer can consider adding new functionalities that may not be realized by mechatronic design alone. Taking a simple example, when we wish to machine a precision mechanical part as specified, we may need to accurately measure its dimensions and surface roughness during the machining operation. This, however, may not be an easy task with the current mechatronic sensor technology. By utilizing optomechatronic concepts, this task becomes feasible by building a non-contacting optical sensor that can work with a rotating and translating machining element (mechatronic element). This implies that, in effect, the optical sensing device creates a new functionality, “optical feedback control,” for the machine
Optomechatronic Integration
261
system. When the optical device is combined with a control unit, the combined system constitutes a complete optical-based feedback control system, which drastically improves the quality of the machined part. In the stage of identifying which modules are necessary to achieve the required functionalities, there will be some difference between the two design methods. A simple difference is that the use of an optical signal may not require signal manipulation or processing (e.g., filtering), while an electrical signal may. However, the major difference comes at the stage of the integration. As discussed earlier, the optomechatronic approach strives to find the aggregated elements in order to achieve the required functionalities more effectively than the mechatronic approach alone can do. Let us consider a typical optomechatronic system, which is an opticalbased heat-treatment of the surface of a mechanical part, as shown in Figure 5.4. In the heat treatment, a moving mechanism (translating, or rotating, or a combination of both) for either a material or optical element is needed to cover the whole area to be heat treated — if we assume that the material is not heated all at once over the whole area. Under this configuration, there may be five divided functionalities involved to achieve a desired quality of heat treatment, although they will not be explained in detail here. Those functionalities are light beam generation for heating the material, control of the actuator for precise light intensity, control of the stage motion in a desired manner (moving range, moving sequence), sensing or monitoring the heat-treated state of the material, and feedback control of this information to regulate the heat-treated state in a desired manner. In order to create these functionalities, we may consider the relevant functional modules, as illustrated in Figure 5.5. To produce an appropriately segmented mirror
mirror
4 kW CW CO2 Laser resonator laser beam lens
sensor fixture IR sensor
coated specimen A
c: >exp now controlled...
measurement & control hardened track XY-table
FIGURE 5.4 Laser surface hardening process.
262
Optomechatronics
light source generation
moving the material or optical head
sensing/control of the heat treated state
FIGURE 5.5 Identifying the functional modules necessary for a heat treatment process.
conditioned optical signal with required power, we first need “signal transformation” that produces an optical signal generated by an electrical signal, optical signal modulation that converts the light beam into a desired form, and optical light control that focuses the beam in order to not only obtain high quality of the heat-treated workpiece, but also to prevent the waste of optical power needed for heat treatment. Note that optical beam control needs its own functional modules to carry out processing, manipulation, and control of the beam injected into the material surface. Another important component of this system is the moving mechanism. Because the moving element should be actuated and positioned as accurately as possible in a desirable manner, it needs a precision electricalto-mechanical actuator, a signal transformation module (which needs signal control), and sensing modules to produce accurate motion of either the material or optical head. Signal sensing and control of the treated material state, therefore, are the important functions of the system that affect treatment quality. The signal sensing has its own signal transformation part that can measure state variables, such as the treated quality, and the dimension of the treated zone. The control of the treated material state requires two signals to be regulated; one is the light power, and the other is the velocity of the moving element. In regulating these variables, the signal sensing part needs two individual sensing devices. The first is the power measurement sensor, and the second is the velocity measurement sensor. It is noted that feedback control will not be classified as a basic functional
Optomechatronic Integration
263
module, but will be identified as a functionality having two functional modules; actuation and sensing.
Basic Functional Modules Signal Transformation In Chapter 1, we observed that a number of systems have signal or energy transformation from one form to another among optical, mechanical and electrical signals. For instance, a digital camera has three important components: CCD cell, auto-zooming by an electric servomotor, and autofocusing by an ultrasonic motor. The CCD cell transduces light information into an electrical signal as a sensor, the ultrasonic motor transduces electrical signal into a mechanical signal as an actuator (as the electric motor does). In actual optical, mechanical, and electrical systems, there are many such signal transformers that convert one signal to another, depending upon the causality between input and output. Table 5.2 configures such transforming elements that convert from one signal form to another. In this table, some of the elements transforming the same kind of signals are not included — for example, optical – optical (OO), electrical –electrical (EE), and mechanical– mechanical (MM) — although they are also important for optomechatronic integration. The first transduction type, optical-to-electrical, denoted by the symbol ðTEO Þ, produces electricity from optical input, which can be derived from several phenomena, such as the photovoltaic phenomenon. The second type, TABLE 5.2 Basic Signal Transformations Transformation
Phenomenon
Symbol
Signal flow
Typical Device
Photovoltaic, pyroelectric, photo emissive Emission of photon
TEO
O ! TEO ! E
Photo diode
E TO
E E ! TO !O
Optical-to-mechanical
Photovoltaic phase transformation
O TM
O O ! TM !M
Mechanical-to-optical
Triboluminiscent
M TO
M M ! TO !O
Light emitting diode Optical actuator, shape memory actuator
Electrical-to-mechanical
Electromagnetism, piezo electric Induction, piezo electric
E TM
E E ! TM !M
TEM
E M ! TM !E
Optical-to-electrical
Electrical-to-optical
Mechanical-to-electrical
Electric motor, piezo actuator Electric generator, piezo sensor
264
Optomechatronics
E Þ; is the reverse case of the first. It produces light from electrical-to-optical ðTO electric input, and is popularly used for either light source or signal conversion for transmission. The third category of the transformation, which O Þ; yields mechanical signals out of optical is optical-to-mechanical ðTM signals. This conversion can be found from optical actuators which utilize some physical phenomena, such as inverse piezo-electric and, material M phase transformation. The fourth one, ðTO Þ mechanical-to-optical, can be found from smart luminiscent materials that generate light when they are E Þ is commonly used in most of subject to stress or friction. The fifth one ðTM the mechatronic devices, including motion generating devices such as electric motors and piezo actuators. The last transformation type ðTEM Þ produces electrical signals that can be observed from mechanical signals from phenomena such as electric induction and piezo electricity. At this point, we can now consider a variety of signal transformation methods by combining some of the basic forms presented in the table. We notice that a number of signal transformations can be obtained as the result of such integration. For instance, suppose that we wish to combine the first, E : The integration then yields: TEO and the fifth, TM
optical ! TEO !electrical ! TME !mechanical which indicates that the optical input signal is transformed into the mechanical output signal. Mathematically, this can be expressed by: E O TEO þ TM ! TM
ð5:1Þ
An important observation we can make from this Equation is that, no matter what type of the intermediate transformation is involved (e.g., electrical signal), the final result is written by the relation between input signal (optical) and output signal (mechanical). Avariety of cases that can illustrate this mode can be found from practical systems. One such example is to adjust an optical system based upon the measured information by a photo detector. Signal Manipulation In electrical signal processing, signal manipulation often needs to be converted into another form while retaining its original information, depending upon the application. The manipulation of optical beams is a process which a number of applications must utilize in order to obtain targeted outputs needed for processing transmission, sensing, or actuating communication via an optical signal. Steering, switching, alteration of amplitude and phase, change of wave form, filtering within a certain frequency band, and noise reduction of optical signals are some of the examples that belong to optical beam manipulation. In this subsection, we will discuss the nature of beam manipulation from the viewpoint of the interaction between optical and mechatronic elements, by dividing the manipulation into two major types, beam modulation and scanning.
Optomechatronic Integration
265
In Table 5.3, various manipulation methods are summarized in the views of the basic operation principle. Almost all methods are operated based on optical manipulation. Mme oo , indicated in the table, denotes the manipulation of the optical beam by both mechanical and electrical elements. The “me” in the bottom of the symbol implies that manipulation is conducted by electrically driving the mechatronic elements. A variety of cases that can illustrate this mode can be found from practical systems. One such example is manipulation of an optical system by diverting its optical signal by means of a mechatronic element. Mm oo and Me oo imply that the manipulation is made by purely mechanical and electrical means, respectively. Let us consider two important methods that belong to signal manipulation in the sequel. (1) Optical signal modulation: In the optical engineering field, the modulation method is also common practice in order to convert an optical signal into a form suitable for certain applications. The major need for optical modulation comes when we want to: (1) (2) (3) (4)
control the intensity and phase of a light beam. control the frequency and wave form of a light beam. impress information on to a carrier signal. have noise reduction.
The technical fields that frequently need modulation can be found among sensing, actuation, control, communication, display, and so on. The modulation can be made primarily by choppers, acousto-opto modulators, electrooptical modulators, photo-elastic modulators, and spatial light modulators. The mechanical optical chopper is typical of optomechatronic choppers,
TABLE 5.3 Types of Signal Manipulation Optical !
o
Mme o
! Optical
Optical !
o
Me o
! Optical
Manipulation type
Basic principle
Acousto-optical modulator Mechanical chopper Photo-elastic modulator Electro-optical modulator Mechanical beam scanner Acousto-optic scanner Electro-optical scanner Mechanical signal manipulation
Bragg deflection Beam chopping by blade Changes in refraction index Changes in refraction index Reflection Bragg deflection Reflection Electromechanical motion
Symbol Mme oo Mme oo Mme oo Me oo Mme oo Mme oo Me oo Me m m
Mme oo optical modulation by mechatronic principle, Me oo optical modulation by electrical method, Me m m mechanical signal manipulation by electrical means.
266
Optomechatronics
as illustrated in Figure 5.6a). In the figure, the optical beam (optical element) incorporates with the motion of a mechatronic unit (rotating chopper blade), and thereby provides a modulated beam as desired. (2) Optical scanning: Scanning is necessary when a light beam or field of view needs to be directed to a certain point or area of an optical field. To achieve scanning, there are two methods, reflective and refractive. In Figures 5.6b,c, these are shown schematically to illustrate the interaction of an optical element (beam) with a mechatronic unit (rotating mirror or prism). In the reflective method, mirrors are utilized, while lens and prisms are used in the refractive method. In terms of scan pattern, there are two classes of the method — fixed pattern or random access. All of these methodologies involved with scanning affect the accuracy and time of scanning. There are three optical scanners most popularly used for practical applications: mechanical scanners, acoustooptical scanners, and electro-optical scanners. Upon examination of the signal manipulation methods shown in the figure, we can see that all manipulations are of optomechatronic nature. Signal Sensing The sensing module is the element that measures physical variables of the process or system being measured. In general, sensing modules operate with two different methods when classified, depending upon which modules are involved. One utilizes signal transformation, while the other uses signal modulation. In other words, sensing can be made by using either signal transformation or signal modulation, as listed in Table 5.2 and Table 5.3, respectively. Therefore, there may be a variety of sensing types classified into several different categories, depending upon the input –output signal form, and the physical phenomena used for sensing.
ψ
ω mirror
incident beam
ω
prism ω
incident beam light beam
reflected beam
refracted beam
(c) refractive scanning
2ω
(a) optical chopper
(b) reflective scanning
FIGURE 5.6 Optical scanning by optomechatronic principle.
(d) scan pattern
Optomechatronic Integration
267
Basically, all the signal transformation and manipulation modes can be used for a sensing element. Combinations of each individual module are also feasible for sensing. The problem is how effectively and accurately each form of sensing can be made. The sensing modules employing the transformation modules TEO , TEM and the modulation modules Mm oo , Me oo are popularly used in actual practice. The sensing module ToM can be used to measure a mechanical signal in terms of an optical signal. For example, when some structures are in fracture or mechanical strain, they produce optical signal. This phenomenon is called “triboluminiscience” as shown in the left hand side of Figure 5.7a. Another luminiscience that belongs to this transformation module is photo luminiscience as shown in the right hand side of Figure 5.7a. The figure shows a portion of a structural surface coated with pressure sensitive paint, which is essentially photoluminescence material. When this structure surface is stressed or strained, the intensity of light reflecting from the surface is varied, depending upon the pressure applied to E seems to be rare, but in the past has been the surface. The sensing module TM actively used in the control of mechatronic devices, or systems such as pneumatic or hydraulic systems.
TABLE 5.4 Signal Sensing and the Related Input – Output Variables Sensing Mode Optical !
TEO
Mechanical !
Electrical !
Phenomenon
! electrical
Photovoltaic, photo emissive
Photo diode storage tube
TEM
Deflection thermo effect, piezoelectric
Strain gauge, piezo sensor, pyro sensor
Mechanical motion
Fly ball governor
TME
! electrical
! mechanical
o
Optical ! Mm o ! optical o
Optical ! Me o ! optical e
Electrical ! Mo e ! electrical Optical !
TEO
Mechanical !
Typical Device
!
TEM
TOE !
! optical
TOE
! optical
Phase, intensity, wave, spectrum
—
—
Fabry-Perot sensor
—
—
Photo emissive —
Image intensifier —
Some other transformation and manipulation modules and their combinations are feasible for sensing, which are not shown here due to space limitation.
268
Optomechatronics
Table 5.4 lists various modules of signal sensing whose principles are originated from signal transformation and modulation. Here, a sensing module measuring the same kind of signal is excluded, as was the case of the signal transformation. As far as transformation type sensing modules are concerned, there are six different types of sensing, whose symbols are identical to those of signal transformation. Among these the sensing E O not shown in the table may become an effective means of modules TO , TM sensing as opposed to the others TEO and TEM , when sensing involves special environments that require such conditions as being inflammable, being contamination-free to noise, and needing fast direct feedback. For instance, if a sensing environment does not permit the sensing module TEO to measure O may be a substitute for the measurement. In this the optical signal, then, TM M case, the module TE should also be replaced by the module Mm oo , which can measure a mechanical signal by modulation of the optical signal. A two-mode sensor module is also feasible by combining two transformation modules together, which was the case discussed in Table 5.4. One such combined sensor module can be seen in an image intensifier, which E as shown in Table 5.4. Figure 5.7b has the signal flow denoted by TEO ! TO illustrates the concept of a typical image intensifier. The intensifier takes a scene with dim incident light and produces a visible image. It consists of an emissive photo cathode surface which causes electrons to be emitted when receiving photons on to a surface, and a phosphor screen to receive the amplified image. In more detail, when photons produced by a dimly lit scene are incident on the cathode surface, the cathode causes electrons to be
λi
λe
emitted light
oxygen-sensitive probe molecules
fracture mechanical loading
structure
base coationg
structure
(a) single-mode transformation sensor photo cathode
induced surface corrugation phosphor screen
structure
photorefractive material light beam 1
collecting lens
(b) two-mode sensor
o
(c) optical actuator ; T M
FIGURE 5.7 Sensors and an actuator employing signal transformation.
light beam 2
Optomechatronic Integration
269
emitted. Under a certain voltage application, the electrostatic lens then focuses the generated electrons on to a phosphor screen. These photo electrons can excite a phosphor on the screen and intensify the illumination of the image. Therefore, the signal flow occurring during the transformation process that produces this image amplification can be written by: E O ! TEO ! TO !O
The other form of a two-mode sensor module is the case of signal flow E : This module senses a mechanical motion by electrical means, TEM ! TO but converts the sensed electrical signal to the corresponding optical signal: E M ! TEM ! TO !O
ð5:2Þ
In this manner, some other multimode sensor modules can also be produced, with even more than two transformation modules. A vast number of sensors employ the modulation method. Here, we list only three kinds: mechanical modulation of optical signal, which is denoted by Mm oo , electrical modulation of optical signal denoted by Me oo, and optical modulation of electrical signal, denoted by Mo ee : The module Mm oo uses the modulation of phase, intensity, wave and spectrum of optical signal that may be caused by mechanical motion. Especially, optical fiber sensors which are popularly used today in various engineering fields adopt this concept, as we shall see later. The sensing module Me oo can be found from the sensors that utilize the interaction of the optical signal with the electro magnetic field. The sensing module, Mo ee is the reverse case of the above modulator, and utilizes modulation of electrical signals by means of optical signals. Signal Actuation Actuating a signal to an arbitrary state is the most common form of the control technology, as we have seen already. As it might be confusing, a signal here means the physical variable to be actuated, and comprises three different types (optical, mechanical and electrical), as we may recall from the previous discussion. Signal actuation is defined here by “actuation of mechanical signal,” to discriminate this type from other types of transformation and modulation modules. O A number of signal actuators employing TM have been developed. One such actuator is shown in Figure 5.7c. The profile of a structural surface that may be altered optically is shown. This optically induced surface deformation employs the principle that photosensitive materials induce stresses and strains when they are exposed to light beam. As shown in the figure two light beams interfere and form a periodic change in the refractive index in a film of a photosensitive material. Due to the changes in the index periodic deformation on the surface of the material results in, whose amplitude is within nm range. This periodic surface corrugations give rise to actuation and can be controllable by the optical interferometric light pattern.
270
Optomechatronics
E Þ is the most commonly-used The electrical-to-mechanical module ðTM actuation mode that actuates mechanical signal by electrical signal. On the other hand, the optical-to-mechanical module is the actuation of a mechanical signal by an optical signal. According to this definition, we can consider two important actuation modules of the transformation type; O E optical-to-mechanical TM and electrical-to-mechanical, TM : When it is necessary for signal actuation to become a remote operation, it involves transmission of a signal, either an optical or electrical. But when sending an electrical signal to a remote site may be not desirable due to noise contamination or a safety problem, an optical means may be a better solution for the transmission. Let us consider the case when an actuator to be operated is located at a remote site. The computer-generated electrical signal is converted into an optical signal, which is then transmitted in an optical signal form. The actuation module at the remote site is involved with the integration of the transformation of the second, the first, and the fifth modes listed in Table 5.2. The resulting transformation mode yields the following mathematical form: E O E E TO þ TRO O þ TE þ TM ! TM
ð5:3Þ
where transmission mode TRO O is included to express signal transmission explicitly. This equation implies that the final transformation mode is operated in the form of the electrical-to-mechanical mode, which is the actuation module of transformation type. A typical example system of this case is an optically operated valve control of a pneumatic system located in a remote site. An electrical signal corresponding to a desired value of the pneumatic valve position is converted into the corresponding optical signal by a LED and is sent to a remote site. This transmitted optical signal is then converted into an electrical signal, which in turn, operates positioning of the servo valve. Depending upon the valve position, the pneumatic servo system is operated. We will treat this in more detail at the end of this chapter. Signal Transmission Signal transmission is a basic form of transporting information or data to or from one location or another. The distance between two locations may be very short, like the length within MEMs parts, short like that of small sensors, or very long like that of communication systems. It is mainly transmitted by means of optical or electrical signal modulation. The transmission should be immune from external noise, exhibit low attenuation, and safe from hazardous interference. For this reason, although application dependent, optical transmission has replaced many of the application areas dominated by electrical transmission. Table 5.5 shows three types of signal transmission, electrical through optical to electrical, optical to
Optomechatronic Integration
271
TABLE 5.5 Types of Signal Transmission Source Signal
Signal Transmission
Signal Flow
Electrical Optical Optical
Operation site ! remote site E Electrical ! optical–optical E ! Me ee ! TO ! TRO O !O o Optical–optical O ! Mme o ! O ! TRO O !O O Optical–mechanical O ! Mme oo ! O ! TRO O ! TM ! M
Electrical Optical Mechanical
Remote site ! operation site E Electrical ! optical–optical E ! Me ee ! TO ! TRO O !O E Optical ! optical O ! TO ! TRO ! O O E Mechanical –optical M ! TEM ! TO ! TRO O !O
optical, and optical to mechanical. The first transmission case is useful when remote operation of mechatronic devices or systems is needed. A simplified optical data transmission system by optical signal is depicted in Figure 5.8. It consists of three major parts, transmitter, optical fiber, and receiver. The transmitter contains a laser or an LED, and appropriate control circuitry: The laser and LED here is a signal transformer that converts an electrical to an optical signal. In a case where the data source is optical, transmitting site
receiving site
laser diode
modulator
optical fiber
optical
(a) electrical-to-optical transmission transmitting site modulator
receiving site optical fiber
optical
optical switch
(b) optical-to-optical transmission transmitting site
receiving site optical
modulator optical switch
optical fiber
(c) optical-to-mechanical transmission FIGURE 5.8 Optical transmission configuration.
mechanical elements
mechanical
272
Optomechatronics
the transforming unit is not necessary and the signal can be sent directly to the input optical fiber unit. When a signal from the signal source is fed into the transmitter, the signal is modulated by the transmitter. Then, the modulated light is launched into the fiber and transmitted to the receiving unit that retrieves the transported information or data. A light detector such as a photo diode (PD) takes the role of receiving signals. The received signal usually goes through a signal processing such as amplification and noise filtering. From the discussion above, it can be seen that in the remote operation shown in Figure 5.8a, signal is transformed from electrical to optical, modulated, transmitted, and transformed back to electrical at the receiving site, and then properly processed and conditioned according to applications. In terms of signal mathematics, the transmission procedure can be written: E E ! Me ee ! TO ! TRO O!O
ð5:4Þ
When a control operation at the sender site is needed, the resulting signal at the remote site is transmitted back to the sender side. In this case, the form of signal mathematics is propagated in a reverse direction as: E e O ˆ TRO O ˆ TO ˆ M e e ˆ E
ð5:5Þ
The second type is a direct optical-to-optical transmission as shown in Figure 5.8b. The input to the fiber is an optical signal modulated by an optical switch Mme oo , and the output signal is the optical signal type, which can be used for a variety of applications. It is noted that some means of providing an optical signal to the operation site is necessary. The signal flow mathematics for this case is expressed by: O ! M meoo ! O ! TRO O!O
ð5:6Þ
The third type is optical-to-mechanical, as indicated in Figure 5.8c. In this case, the transmitted optical signal is directly interacted with mechanical elements at the optic – mechanical interface, thus producing a mechanical signal in a desired manner. The signal flow mathematics is described by: O O ! M meee ! O ! TRO O ! TM ! M
ð5:7Þ
This type is a seemingly unachievable transmission type, but can be found from a practical example where the transmitted optical signal operates on a fluid nozzle which controls fluid flow. This alters a deflection of the jet from the center position. Some of the details will be discussed in the last section of this Chapter. Signal Display The display element is usually the final data representation element. A simple one is a bathroom scale that contains a typical signal display element of the LCD type. There are a number of data presentation devices, from simple pointer scales and recorders to very large scale display devices such
Optomechatronic Integration
273
small-scale alphanumeric
light-emitting diodes (LED)
liquid crystal display (LCD)
large-scale displays (electronic display)
liquid crystal cathod-ray display (LCD) Tube (CRT)
image MEMs based gas plasma discharge intensifier projector
plasma display panel
field emission display
electrolume scene
light emitting diode
FIGURE 5.9 Classification of optical data presentation devices.
TABLE 5.6 Various Signal Display Modes Display Mode
Display Principle
Typical Device
E M ! TEM ! TO !O E E ! TEE ! TO !O E O ! TEO ! TO !O o O ! Mme o ! O O ! Me oo ! O
Photoemissive Photosemissive Photosemissive Optical switching Polarization
Weight indicator (scale) Cathode ray tube, LED, PDP Image intensifier, image converter Digital micromirror, grating light valve (GCV) Liquid crystal display (LCD)
as cathode-ray-tubes (CRT) and plasma display panels (PDP), as shown in Figure 5.9. Since brief discussions on the principles of some of these display units have been made in Chapter 4, we will not discuss them here. As shown in the figure, optical display units can be conveniently categorized into small-scale alphanumeric displays and large displays that all work in a digital domain. As far as the displaying principle is concerned, it is interesting to see that all of these can be grouped into three categories as shown in Table 5.6. The first method is to use a light emitting source like a light-emitting diode (LED). The second one is the polarization method which does not emit light, but uses light incoming from other sources. The last display method is to use a signal manipulation technique.
Optomechatronic Interface or Integration Basic Two-Signal Integration In the previous chapters, we have seen a variety of different types of integration. From the observations of such integrations, we realize that
274
Optomechatronics
to become an optomechatronic element requires an optical element (O), a mechanical element (M), and an electrical element (E) all to be properly integrated together while obeying certain laws. As we shall see later, not all types of the integrations made by these three elements will yield an optomechatronic nature. Integration exhibiting such a nature needs to meet certain conditions called optomechatronic integrability. Among all of the integrations, the basic types are the integration of two signal elements, namely, OE, EO, ME, EM, OM, and MO, if homogeneous integration such as EE, OO and MM are excluded. For example, the photo diode having the transformation TEO in Table 5.1 has the capability of transforming an optical to electrical signal, and does not have mechatronic elements in it. This is an optoelectronic integration that can be denoted by OE. The electric motor (ME) has the capability of transforming an electrical signal to a mechanical signal. And the optical actuator employing shape memory alloy (OM) has the capability of converting optical signals to mechanical. Similar arguments can also be applied to the rest of integrations, such as a laser diode (EO), electric generator (EM), and motion induced light generator (MO). The two signal elements can also be observed from some modulators and manipulators upon examination of the modules listed in Table 5.2 and Table 5.3. For instance, optical transmission itself does not interact with mechatronic elements in transmitting optical signals. It represents an optoelectronic element (OE). Because this two-signal element forms the basis of optomechatronic integration, let us take some more practical illustrations. Figure 5.10 depicts various forms of two-signal integrations of OE, EO, EM, ME, OM, and MO. The light emitting diode (LED) in Figure 5.10a emits electromagnetic radiation over a certain band wavelength when forward biased. This integration represents “EO,” the order of which identifies the input– output relation. As discussed in “Signal Conditioning”, in Chapter 4, a light emitting diode (LED), like an electronic diode, directs electricity flow in one direction. When electricity flows from the cathode (positive lead) to the anode (negative lead), wire encapsulated in the LED body emits light, thus releasing photons. This component has a capability of transforming an electric signal to an optical signal. By using causality relation, this phenomenon can be expressed as electric signal as an input and optical signal as an output. The photo diode shown in Figure 5.10b is a photon detector, which is a semiconductor device in which electrons can be excited from the balance band to the conduction band when incident light hits the junction surface. This integration can be represented by OE, which means that an optical signal as an input produces electrical signal as an output. The two conversion elements in the above can be symbolically represented as: laser emitting diode:E þ O ! EO photo diode:O þ E ! OE Unless specified, we will describe this as OE.
Optomechatronic Integration
reflector
275 motion
electrode
illumination
reflector p type
negative positive
LED chip
n type
piezoelectric material
(b) photo diode
(a) light emitting diode
(c) piezoelectric transducer
motion core S
rotation
coil
arm
separation
S
motion
N
stator
(d) variable capacitance
(e) linear motion variable inductor
stator
rotor
(f) electric motor light
displacement piezoelectric material displacement
optical fiber
bevel painted on black
electrode field coil
deformation directio
terfenol-D rod
(g) magnetostrictive actuator
(h) piezoelectric actuator
diaphram
cavity
gas
light
(j) optical actuator
(i) moving fiber deflection
light
electrostatic bonding
optical fiber
shape memory alloy ( Ti Ni )
glass
light
(k) optical-based shape memory alloy
(l) optical sensor
FIGURE 5.10 Illustrations of various forms of two-signal integration.
Figure 5.10c depicts the phenomenon of piezoelectricity. When a piezoelectric material is subject to external force or pressure which is a mechanical signal, it produces electricity. Figure 5.10d shows the capacitance of an electric capacitor subjected to motion. One plate of the capacitor is made to move freely, while the other plate is fixed. Thus, depending upon the motion applied, this device achieves modulation of electric signal due to change in the capacitance. Figure 5.10e illustrates a typical arrangement of a variable inductor that converts mechanical signal to electrical signal when a coil is subject to a mechanical motion of the core.
276
Optomechatronics
The piezoelectric transducer in the above exhibits the transformation of mechanical signal to electrical signal, while the other devices modulate their signal when they are provided with external mechanical motion. From the point of view of energy conversion, they convert one energy to a different energy form, which, in this case, is mechanical-to-electrical signal conversion. However, from the view point of a functional module, we can call this “signal transformation” rather than “signal modulation.” Symbolically, the above three conversion elements are then described by: M þ E ! ME
ð5:8Þ
Again, the first element M denotes mechanical signal, while the second element E is electrical signal. The meaning of the equation implies a combination of mechanical signal generating elements and electrical signal generating elements, denoted by electromechanical or mechatronic elements. Figure 5.10f shows an electromagnetic motion generator when a coil wound element is exposed to an electromagnetic field, and, subject to the change of the field strength, the element rotates according to the applied electrical signal. The magnetostriction actuator shown in Figure 5.10g utilizes the material property that causes a material to change its length when subjected to an eletro-magnetic field. Thus, this transforms electromagnetic signal to mechanical displacement signal. Another actuator similar to this type is the piezo-electric (PZT) actuator shown in Figure 5.10h. This device transforms electrical signal input to mechanical displacement output. The above elements are produced purely by combination of mechanical and electrical elements, and can be expressed by: E þ M ! ME
ð5:9Þ
Figure 5.10i, Figure 5.10j and Figure 5.10k illustrate optical actuators that transform heat generated by lightwaves to mechanical deformation. The deforming optical fiber works on the following principle. One end of an optical fiber is bevel cut and painted black. When the light illuminates at the other end of the fiber, it exhibits a photo-thermal phenomenon. The optical actuator transforms heat to mechanical signal indirectly, because it uses the expansion of gas to generate a mechanical deformation. The cavity is filled with gas that expands when heated with a light beam. This expansion in the deflection of the diaphragm generates vertical direction. Shape memory alloys such as Ti Ni exhibit thermoelastic martensitic transformation, which yields shape-recovery characteristics upon heating. This results in the generation of mechanical motion. The signal transformation for these types of actuation devices is expressed by: O þ M ! OM
ð5:10Þ
The MO integration shown in Figure 5.10l is an optical fiber sensor configuration in which mechanical deformation provides changes in
Optomechatronic Integration
277
crystal mirror
photodetector sound undefracted
beam
light beam
diffracted piezoelectric actuation material
beam
(a) mechanical
(b) acousto-optical modulator
scanner
light
actuator motion
photo detector laser artifact
angular displacement light source
(c) optical chopper
deformable glass mirror
wave
PZT actuator stack electric leads
(e) grating sensor
laser
(d) optical encoder
(f) deformable mirror
atom or cell
(g) optical twizer
FIGURE 5.11 Illustrative examples of optomechatronic integration.
intensity of a light beam. This type of integration is described by: M þ O ! MO
ð5:11Þ
Fundamental Optomechatronic Integration: Illustrations As presented in Equation 5.8 through Equation 5.11, we have discussed three types of integration, including optical plus electrical (OE or EO), optical plus mechanical (OM or MO), and mechanical plus electrical (ME or EM). The OE (EO) is called optoelectronic integration. Similarly, the OM (MO) and ME (EM) are called optomechanical and mechatronic, respectively. When all of three basic elements, optical, mechanical, and electrical (O, M, E), are combined together, the combination may produce optomechatronic integration. The integration can be achieved with a variety of combinatorial types, which we have seen in Chapter 1. Figure 5.11 depicts various different types of optomechatronic integrations. The first three devices are optical modulators. It can be seen that all of these devices do not change the form of the original input signal, but retain its optical signal form. They modulate direction, wave form, and amplitude, as we shall discuss in detail in Chapter 6. Figure 5.11a is a polygon scanner that diverts a light beam in a desired direction. This scanner consists of a light source (OE) and a rotating mirror (ME) which constitutes optomechatronic integration (OPME). An acousto-opto modulator, shown in Figure 5.11b, modulates the optical signal in such a way that it diverts a light beam in a certain direction by combining light waves with sound waves generated by excitation of a piezoelectric material. It is composed of a light source (OE) and an acoustic generator (ME), resulting in optomechatronic integration (OPME). Figure 5.11c shows an optical chopper, discussed already, that modulates a light beam
278
Optomechatronics
having a wave form and frequency different from the original signal. A light source (OE) and a rotating wheel (ME) constitutes an essential element of optomechatronic integration (OPME). Summarizing all of the above integration types, the modulators can be expressed by: OE þ ME ! OPME The sensors shown in Figure 5.11d,e exhibit all optomechatronic integrations. Shown in Figure 5.11d is an optical angle encoder, which is composed of a light source (OE), a rotating coded wheel (ME), and a separate photo sensor (OE) for each track. It has alternating transparent and opaque areas patterned on the rotating disk. A grating sensor shown in Figure 5.11e also runs on the optomechatronic principle [4]. It measures an artifact topology by using controlled motion of an optical grating device (ME), through the grating of which light (OE) is diffracted differently depending on the surface topology. When the configuration in Figure 5.10l is combined with a photo sensor, the combined device becomes an optical fiber sensor based on the modulation principle. It modulates the amplitude of a light beam passing through the optical fiber (OE), which can be deformed by bimetal displacement due to temperature variation (M). This type of integration can be regarded as a signal transformation, since it converts a mechanical signal (bimetal strip displacement) into an optical signal (light beam amplitude). In summary, the integrated forms of the sensors discussed above are expressed in the following: optical encoder: OE þ ME þ OE ! OPME optical grating sensor: OE þ ME þ OE ! OPME optical fiber sensor: OE þ M ! OPME Two devices shown in Figure 5.11f,g are related to manipulation or actuation that can run on optomechatronic principles. The deformable mirror shown in Figure 5.11f is a mirror that can be locally deformable by multiple-stacked piezo actuators (ME). The state of deformation is controlled by translation of the actuators, depending on the distortion characteristics of incoming light waves (OE). The optical twizer shown in Figure 5.11g is a device that can directly manipulate atoms or nanometer objects by optical means. The motion of small nanoobjects (M) is controlled by the standing light wave generated by a laser light (OE). Therefore, this device can trap objects in the minima of the standing wave condition. From the integration viewpoint, two optical devices are expressed by: deformable mirror: OE þ ME ! OPME optical twizer: OE þ M ! OPME Figure 5.12 shows various types of optomechatronic-based manufacturing processes. Figure 5.12a is an inspection process of measuring the parts on a moving conveyor (ME) by using a CCD camera (OE). The laser welding system in Figure 5.12b is also based on the optomechatronic-based processing concept that the workpiece to be welded by the laser head (OE) is
Optomechatronic Integration
279 laser
camera
optical module/ wave guide
illumination
welding head parts conveyor
(a) inspection of moving parts
work piece
LED
lens
6 axis stage
motor
displacement
(b) laser welding
(c) optical packaging
FIGURE 5.12 Various types of optomechatronic integration used for manufacturing processes.
controlled according to the joining path plan (ME). The laser head is equipped with an autofocusing device, which is composed of a set of lenses (O) and an actuation mechanism (ME). This device itself, therefore, exhibits optomechatronic characteristics. An optical packaging process shown in Figure 5.12c uses a laser beam (OE) to align the photo diode by using a multiaxis adjustable stage (ME). Depending upon the degree of the alignment between the laser beam and the photo diode, the amount of light intensity detected by a photo sensor (OE) becomes different. The manufacturing processes considered above have the following integration form: vision-based inspection: ME þ OE ! OPME laser welding: OE þ O þ ME þ O þ ME ! OPME alignment process: OE þ ME þ OE ! OPME Based on the discussions we have made above, it is clear that optomechatronic features can be found from either one single functional module or more than two modules combined together to generate an output signal. In the case of a single module, optical and mechatronic signals within it are properly integrated to generate an output signal. When multiple functional modules are combined to generate an output signal, the condition for optomechatronic integration should be such that at least one module must exhibit the nature of one optical module, and one mechatronic module must be included for the combination. Generic Forms for Optomechatronic Interface The above observation enables us to define the basic forms of configuration for optomechatronic integration. Figure 5.13 describes two such types, classified according to the change of the signal type between the input and output ports. Figure 5.13a indicates the case when the output signal has a different signal type as a result of integration. Most of the functional modules discussed in the previous section, except modulators, belong to this class.
280
Optomechatronics interface
integration module
input signal signal type :
modulating signal
interface
integration module
input signal
output signal
output signal
input signal = output signal
input signal ≠ output signal
(a) transformation type
(b) manipulation type
FIGURE 5.13 Basic types of optomechatronic integration.
This is called “transformation type” optomechatronic integration. In contrast, Figure 5.13b indicates that the type of output signal is not changed as a result of integration, retaining the same signal as that of the input signal, as can be seen from the modulators discussed in the previous section. This type is termed “modulation type” integration. Let us further examine details of each type of optomechatronic integration. As can be seen from Figure 5.14, the transformation type integration has basically three families of configurations. The thick line indicated in the figure denotes the interface between the input and output signals. The first configuration is the combination of a single-signal with one transformation module, which is shown in Figure 5.14a. When there appears no interacting external signal, a single signal transformer alone interface
O
E
EM
M
OM
O
O
O
M
E
M
MO
M
E
OE
E
ME
M
M
E
functional module
EO
E
O
O
(a) a signal with one transformation module O
TE O
OE
E
E
O
OM
EM
TO M
E
M
TM
TE
ME
EO
O
O
E
E
EM
M
TM
OM
TO M
M
M
TE
TO
MO
O
M
(b) combination of two transformation modules O E M
OPME OME
MO
O
O
O or M or E
(c) coupling between three different signals FIGURE 5.14 Interface for an optomechatronic transformation type integration.
ME
TE
OE
E
E
M
M
E
M
O
TM
E
TM
E
TO
EO
O
Optomechatronic Integration
281
cannot be of optomechatronic nature. The reason is that with such one transformer optomechatronic integration cannot be physically feasible, since an input signal cannot be transformed into another at the interface without an additional transforming unit. For instance, when an optical signal is present at the interface of a mechatronic transformer (ME, EM), the coupling result should produce either a mechanical or electrical signal, so that the transformer can accommodate an optical signal. However, an optical signal cannot be coupled with either one of the two signals, since there is no unit transforming the optical signal to either a mechanical or electrical signal. Therefore, an interacting signal to the transformer must be present that has a signal type different from those inputs and outputs involved with the transformation. In other words, it must occur under the presence of a signal type different from those of the transformer. Figure 5.14a shows all six feasible configurations of integration. The first integration implies the case when an electrical-to-mechanical (E – M) transformation occurs under the influence of an optical signal. Due to the presence of an optical signal, the transformation may result in different characteristics. As we shall see later in “Optomechatronic Actuation” in Chapter 6, an interesting example of this case can be found in which the presence of an optical signal influences an electrostatic force between two capacitive plates. The last integration type in the figure describes an E –O transformation under the influence of a mechanical signal. Likewise, all the rest of the integrations shown in the figure can be physically interpreted. The second configuration is the combination of two functional modules shown in Figure 5.14b. This is essentially a multiprocess signal-transforming element. Some examples of such transformations are illustrated. For example, optically driven motion of mechatronic elements belong to the configuration shown in the first figure. The transformation yields mechanical signal motion to a mechanical element by applying an electrical signal transformed from an optical signal. A practical case of this configuration is a photodiode-driven PZT motion. Optical signal generation by an electrical signal, which in turn is produced by a mechanical signal, is another example of the integration shown in the last figure. A PZT driven laser diode well represents this configuration. Typical examples that belong to this type of integration are given in Table 5.7. The mathematical forms of the transformation integrations given in Figure 5.14 are expressed by: E O TEO þ TM ! TM
E O E TO þ TM ! TM
M TO þ TEO ! TEM
O þ TEM ! TEO TM
E M E þ TO ! TO TM
E M TEM þ TO ! TO
The third configuration is the combination of three single signal elements, denoted by OPME, as shown in Figure 5.14c. This configuration produces an output, any one of the three optical, mechanical, or optical signals. The integration can be made in any order, depending upon the physical
282
Optomechatronics
TABLE 5.7 Symbols and Mathematics for Various Types of Integrations Type of Integration
Symbol
Integration Mathematics
Typical Phenomenon, Device
reflector
Electricalto-optical
EO
reflector
E!O
negative positive
E þ O ! OE
LED chip
illumination p type Optical-toelectrical
OE
O!E O þ E ! OE
n type electrode
Mechanical-toelectrical
ME
motion
M!E M þ E ! ME
piezoelectric material motion
Mechanical-toelectrical
ME
M!E
S
separation
M þ E ! ME
core Mechanical-toelectrical
ME
M!E M þ E ! ME
coil
arm motion
continued
Optomechatronic Integration
283
TABLE 5.7 CONTINUED Type of Integration
Symbol
Integration Mathematics
Typical Phenomenon, Device
rotation
Electrical-tomechanical
EM
E!M E þ M ! EM
S
N
stator
rotor
stator
displacement Electrical-tomechanical
EM
E!M E þ M ! EM
field coil terfenol-D rod displacement
Electrical-tomechanical
EM
piezoelectric material
E!M
electrode
E þ M ! EM
light
Optical-tomechanical
OM
O!M
optical fiber
bevel painted on black
O þ M ! OM
deformation direction diaphram
cavity
Optical-tomechanical
OM
O!M
electrostatic bonding
gas
O þ M ! OM
light
glass
continued
284
Optomechatronics
TABLE 5.7 CONTINUED Type of Integration
Symbol
Optical-tomechanical
OM
Integration Mathematics
Typical Phenomenon, Device
M!O M þ O ! MO
optical fiber
TABLE 5.8 Integration Type and Mathematic for Various Optomechatronic Integrations Integration Type
Symbol
Integration Mathematics
Typical Phenomenon Device, Processes
Object manipulation/actuation
mirror EO ! ME ! OH
OPME
OE ! ME ! OPME
beam
crystal
undefracted
EO ! M ! O
OE ! M ! OPME
diffracted beam
sound
actuation
EO ! ME ! O
piezoelectric material
OE þ ME ! OPME
light beam
continued
Optomechatronic Integration
285
TABLE 5.8 CONTINUED Integration Type
Symbol
O ! ME ! O
Integration Mathematics
Typical Phenomenon Device, Processes light
O þ ME þ O ! OPME
deformable glass mirror
PZT actuator stack
electric leads
OE ! ME
laser
OE þ ME ! OPME
wave
Sensing
EO ! ME ! OE
OPME
atomor cell photo detector
OE þ ME þ OE ! OPME
angular displacement
light source actuator motion
EO ! ME ! OE
OE þ ME þ OE ! OPME
det laser
artifact
Manufacturing system or process laser
EO ! ME ! O ! ME
OPME
OE þ O þ ME þ O þ ME ! OPME
welding head work piece
motor
displacement
continued
286
Optomechatronics
TABLE 5.8 CONTINUED Integration Type
Integration Mathematics
Symbol
Typical Phenomenon Device, Processes camera
EO ! ME ! OE
OE þ ME þ OE ! OPME
illumination
parts conveyor
EO ! ME ! OE
OE þ ME þ OE ! OPME
LED
optical module/wave guide 6 axis stage lens
phenomena involved. The mathematics of this integration is expressed by: O þ M þ E ! OPME In Table 5.8, typical devices that satisfy this relationship are summarized together with integration mathematics. The manipulation type shown in Figure 5.15 has basically six configurations. As we have already seen above, this type produces the same type of input signal as output, by utilizing a signal manipulator in the rectangular box. All manipulators are of the transformation type, operated on a twosignal integration element. The first configuration of the manipulation type shown in Figure 5.15a is a mechatronic type, since optical signal manipulation is achieved by mechatronic signal transformation. The second type in Figure 5.15b is an optomechanical type, whereas the last one is O
E O
EM
O
E
ME
E
M
M
M O
OM
E
O
(a) mechatronic type
E
MO
EO
M
O E
(b) optomechanical type
M
OE
(c) optoelectronic type
FIGURE 5.15 Interface for an optomechatronic modulation type integration.
M
Optomechatronic Integration
287
the optoelectronic type in Figure 5.15c. The mathematics involved with the manipulation-type integration may be expressed by: O ! Memoo ! O
E ! Momee ! E
m M ! Meom !M
O ! Mmeoo ! O
E ! Mmoee ! E
m M ! Moem !M
A physical example of the first signal integration type can be observed from the optical modulator and mechanical beam scanner, as discussed in the “Basic Functional Modules” section. The other integration types may be conceivable, and may be found from some practical examples. In actual practice, optomechatronic integration can take place with the combination of more than two transformation modules. For example, the photo diode-actuated electromechanical device at a remote site belongs to this case. The mathematics of this integration can be given by: O E E E þ TRO TO O þ TE þ T M ! TM E denotes the signal transformation from electrical to optical It is noted that TO from a laser diode, TRO O denotes signal transmission to a remote site in the form of an optical signal, and finally, the signal transformation is back to a mechanical signal. When a piezoelectric sensor signal at a remote site is optically transmitted to a central control station, and then is transformed back to an electrical signal, the mathematics involved in this the integration can be expressed by: E O E þ TRO TEM þ TO O þ TE ! T M
The first term indicates the transformation of a signal from mechanical to electrical by the piezoelectric sensor, the next two terms involve signal transmission, and the last term indicates the transmitted optical signal back to an electrical signal at the control station. Another integration type can take place when a signal transformer and manipulator are combined together. A combination of a signal transformation and a signal manipulator occurs when laser light impinges on to a deformable mirror. The resulting integration is a light distributor, whose mathematics is written by E TO þ Mmeoo ! O
where the first term indicates a transformation for laser light and the second indicates the optical signal modulation by a mechatronic actuator. Another example of this category is an electric motor with an optical encoder. In this integration, the rotational motion of the motor is generated by a signal transformer, and then is measured by the encoder for feedback. The interface diagram for this interface cannot be seen in Figure 5.14 or Figure 5.15 because this interface requires two separate types of modules; one is
288
Optomechatronics
transformation and the other is modulation. In this case, the plus sign for mathematical expression can not be physically feasible, since the causality at the interface between two signal functional modules is not met. To this end, we use the multiplication sign to describe the mathematical expression as: E E ! TM £ Mmeoo ! O
Integrability So far, we have seen a number of different combinations conceivable for optomechatronic integration. However, it can be argued that not all integrations listed here can be expected to make the resulting device or system physically work. From this point of view, it appears that there need to be several considerations for devising a plausible interface between the integrated signals, to make such integrations physically realizable. Two important considerations can be made regarding the integrability. These are the interfaciability, and the output power requirement. The interfaciability refers to causality considerations, whereas output power requirement refers to the ability to produce the power that can be usable at the output side. Causality is the relationship between the signal at the input side and the signal at the receiving unit when they are interconnected at the interface. In other words, the input signal to the interface should be transformable, in order that the signal receiving unit can accommodate it. For example, if the integration shown in Figure 5.14a is considered, each input signal should be interfaciable to the port of each transforming unit. If this is not the case, the integration cannot possibly happen. The same arguments can be applied to the other integration cases shown in Figure 5.14b,c. The other consideration of integrability is the magnitude of energy or power attainable at the output port for a given input signal level. If the output magnitude is very small, the output signal may be affected by noise excitation, or require a very large input signal to attain a relatively large output signal. For a signal within some limited range, this kind of integration becomes problematic, although causality is met. Let us take one interface for an integration, for example, the second type O ! M, in Figure 5.14a. From what we have seen from the piezoelectric actuator, if the required power of the optical signal is too large to attain a desired mechanical signal, this integration module may not be appropriate or feasible. The same requirement on output power will be needed for the other two cases in Figure 5.14b,c. Photo diode-actuated mechatronic elements at remote sites are one such example that may need stringent power requirements. The transmitted optical signal is normally attenuated, and its transmitted optical power may not be large to provide sufficient input to the photo diode. In other words, sufficient power to drive the mechatronic element may not be attainable either due to attenuation during transmission, TRO O , or power loss during transformation. Another example can be found from the devices, such as optically driven mechatronic elements, acousto –optical modulators ðMmeoo Þ:,
Optomechatronic Integration
289
and mechanical scanners ðMmeoo Þ: As we will discuss in the next chapter, all of these devices will need to consider efficiency in their output power. Signal Flow An optomechatronic device or system may be full of functional modules or single signal elements. Figure 5.16 indicates the two generic configurations exhibiting an optomechatronic nature that can be made by optic – mechatronic interface. The arrow indicates the flow of information from one module to another. The open loop structures shown in Figure 5.16a represents the case when the interfacing would occur in a series, that is, all modules are combined in series to generate an output signal. It has three basic forms, which are series, parallel, and series-parallel. The series structure has the configuration of functional modules that are coupled in series such that the output signal of each module flows from one to another. In the parallel structure, however, two modules are put together in parallel, such that their output signals are added together. The series-parallel structure is different from the two in that the divided signals from the proceeding module enter into the two different modules next to it. A device of this type is conceivable when an optical signal is split into two by a beam splitter. Then, each split signal enters into its own autofocusing functional module. The configurations shown in Figure 5.16b have a feedback structure in that the final output is fed back to at least one of the modules within the system. Two feedback structures shown in Figure 5.16b differ from their feed forward (open) loop. The basic case has a single path, while the other form has a multiple path (in this instance, two paths). An example can be illustrated with a camera with autofocusing capability, as mentioned before. In this example, one of the relevant modules in the series has a feedback loop within the integration, as module
module
module
signal
module
+
module
signal
− module
+ + module module
signal
+
module +
module module
(a) open-loop configuration
module module module
(b) feedback configuration
FIGURE 5.16 Basic open loop and feedback configurations for signal flow.
signal
290
Optomechatronics
transformer E
O Er
TEO
o me o
M
+_
E
Er +
_
Md
+
TME
M
O
transformer PSD sensor TEO
laser
E
transformer
(b) measurement of machined workpiece surface optical actuator
TOE
E
o meo
TME
TME
(a) camera
O
TOE
TOM
Mme oo
optical power
gripper
TMM
M
optical mirror gripper
UV light optical fiber iris (valve)
O M
T
optical actuator
TEM
mirror
mirror supporter
motor
sensor
(c) force feedback control of a two-fingered gripper FIGURE 5.17 Illustrations of the closed-loop system.
shown in Figure 5.17a. The optical signal (O) is continuously adjusted to make a feedback control of a lens location focused until a certain image can be focused to a satisfactory level. Another example is an optical power control which has been previously discussed in a laser-controlled machining system. In this example, focusing is made by adjusting the position of the optical system in the laser power system. The schematic diagram illustrated in Figure 5.17b shows an optical sensor-based control for mechatronic system, as discussed in “Optomechatronics: Definition and Fundamental Concept” in Chapter 1. Basically, it has three transforming elements and one modulator: The mechatronic E element denoted by TM has a signal transforming element (from electrical E has a transformation to mechanical signal); a laser diode denoted by TO (from electrical to optical signal); a photo sensor TEO has a signal transforming element (from optical to electrical signal). The module in the middle E represents the interaction with the mechatronic and optical elements. TM The modulation module, Mmeoo ; receives a laser beam produced by a laser E , and impinges it upon the surface of the mechatronic element (e.g., source TO rotating workpiece). The incident beam reflects differently and the reflected beam enters to the photo sensor, TEO depending on the machined surface texture of the workspace being machined. This signal is feedback to compare E with the desired electrical signal. Based on the difference signal, TM module generates an electrical command signal to a driving motor, which acts as a modulation module Mme oo .
Optomechatronic Integration
291
The cascade feedback form shown in Figure 5.17c has basically the same structure as that of the basic feedback form, except the cascade element in the open-loop path. One nature of this feedback form can be found in the case when two optical actuators actuate a mechanical element composed of two grippers which grasp an object with a desired force, as depicted in the figure. The gripping force signal generated by a force sensor can be used to detect the state of gripping, and fed back to the controller element of the controlling optical power coming from a light source. The operation principle of this optically-actuated two-figured gripper is as follows: A O E , actuates two optical actuators TM proper amount of optical power TO and then two grippers accordingly produce small mechanical deformations M Þ due to the gripping force. This force (mechanical signal, M) of an object ðTM is detected by a force sensor ðTEM Þ and fed back to an optical power controller force to compare with the desired mechanical signal ðMd Þ:
Integration-Generated Functionalities In the previous sections we have defined basic functional modules that generate functionalities required for optomechatronic systems, discussed the integrability of the modules to create optomechatronic characteristics, and then studied the signal flow graph that represents the flow of information from input port to final output port. All of these considerations are necessary to design a certain functionality specified for optomechatronic devices, processes, or systems. Our interest here is to know how they are generated by a single module, or by combining the functional modules. In this subsection, we will briefly treat these subjects and illustrate how functional modules are used to produce the required functionalities for an optomechatronic device or system. Any engineering system that includes devices, machines, or processes must possess one or more functionalities in order to perform the desired tasks as specified by the designer. In essence, such a functionality is created by an individual module or combination of functional signal modules. Depending upon how many modules are involved, or how they are interacted with each other, the functionality can have different attributes. Various functionalities created in this way can be combined to make systems exhibit a certain functional behavior. In general, a system or device will need a variety of functionalities to perform a given task. The fundamental functionalities are the common functionalities that are frequently utilized everywhere. Let us consider here some of the fundamental functionalities listed in the Introduction. Figure 5.18 shows a variety of configurations to create a functionality. This configuration is basically similar to those shown for basic configuration in Figure 5.16. Figure 5.18a shows the case when a single module is needed to generate a functionality. The series configuration in Figure 5.18b shows the case when signal flow occurs in a forward direction,
292
Optomechatronics
functional module
input signal
functionality
(a) single module input signal
functional module
functional module
functional module
…
functionality
(b) series input signal
input signal
functional module
input signal + +
functionality
+
−
functional module
functional module
functional module
(c) parallel-series
functional module
(d) feedback
FIGURE 5.18 Various configurations to generate a functionality.
FIGURE 5.19 Functional modules for a pneumatic pressure control system.
functionality
Optomechatronic Integration
293
while the parallel series utilizes a scheme of signal processing in parallel, as shown in Figure 5.18c. Lastly, Figure 5.18d shows a feedback scheme to produce a functionality. Figure 5.19 illustrates the remote control of a pneumatic power system located in a remote site. This system is all optically operated, so that it uses all-optical operation modes, e.g., optical actuators and sensors. The pneumatic, flapper-nozzle valve is controlled by an optical actuator, as considered in the previous discussion. The operation starts when the highpower optical signal transmitted through an optical fiber enters the optical actuator. The actuator, when activated, causes the flexible membrane to move closer towards, or farther from, the nozzle, which accordingly yields a change in backpressure; closer means higher back pressure, while farther means lower backpressure. This change is amplified by a pneumatic gain block. The amplified pneumatic pressure operates a spool valve, and actuates a pneumatic cylinder connected to a system to be controlled. This control system, therefore, belongs to the feedback form shown in Figure 5.18d. Figure 5.20 illustrates the diffraction-based sensing concept that can be used for an atomic force microscope (AFM), which will be treated in Chapter 7. The sensing uses the principle that the variation of grating distance results in the variation of diffraction angle of a laser beam (first order), as discussed in “Diffraction” in Chapter 2. The sensing tip is
actuator motion
photo detector
comb drive
laser
spring AFM tip
artifact
sample (a) diffraction grating sensor measurement function
input signal
+_
actuation module (comb drive)
measurement signal
sensor system
sensing module
(b) functional module diagram FIGURE 5.20 Functional modules for a diffraction grating sensor.
output distance
294
Optomechatronics
supported by a diffraction grating whose end is connected to a comb drive actuator. Since the grating element inherently possesses a spring element due to its material and structural composition, the grating element becomes compressed when the whole sensing unit is scanned through and encounters rough artifact. Note that the support element is fixed at a constant height, as shown in the figure. When laser light is interfaced with this variable grating, it results in the angle variation of the diffracted beam, depending upon how the grating element is displaced.
Problems P5.1. The following are some single-stage transformation modules that interact with a single signal considered in Figure 5.14a. Illustrate a physical device or system corresponding to each case (Figure P5.1).
O
E
ME
M
E
O
O
OM
M
EM
E
M
FIGURE P5.1 Single –stage transformation modules.
P5.2. Illustrate a physical element, device, or system that works according to each of the following: two-stage module composed of transformation and/or modulation type as shown in Figure P5.2.
E
E
O
TO
O
TM
M
E
E
M
TE
E
o
Mme o
O
(2)
(1) M
O
TO
E
TO
(3)
O
O
O
TM
M
M
TE
E
(4)
FIGURE P5.2 Two–stage transformation modules.
P5.3. Repeat the same problem for the multi-stage modules composed of signal transformation and signal modulation as depicted in Figure P5.3.
Optomechatronic Integration
295 O
E
E
TO
O
o
Mme o
O
TE
TM
E
M
O
E
(1) O M
M
TE
E
M
E
TM
o
Mm o
O
TE
(2) FIGURE P5.3 Multi–stage modules composed of signal transformation and modulation.
P5.4. The following signal flow graph indicates a feedback control system in which the position of an optical modulation type device ðMme oo Þ: is varied depending on the magnitude of electrical output of a transformation module ðTEO Þ: The transformation module receives the modulator’s optical output as its input as can be seen from Figure P5.4. (1) Describe a detailed operation principle of the feedback system. (2) Devise an optomechatronic system that obeys this signal flow.
E
E
TO
O
o
Mme o
O
O
TE
E
E
TM
FIGURE P5.4 The signal flow graph for an optomechatronic system.
P5.5. Consider a remote operation of the pneumatic pressure control system shown in Figure 5.19 and redrawn here for this problem in Figure P5.5. (1) Write down signal flow mathematics for this system. (2) Draw a complete signal flow graph describing remote operation of this system. Assume that the sender (control) site uses an LED for signal transformation. P5.6. Figure P5.6 shows an optically-ignited mechatronic system discussed in the Introduction. Repeat the same problem as given in P5.5 (1) and (2).
296
Optomechatronics
FIGURE P5.5 Remote operation of a pneumatic pressure control system.
laser path
sapphire window/case/seal projectile
collimator
propellant charge
FIGURE P5.6 Optically-ignited mechatronic weapon system.
P5.7. Figure P5.7 shows a remote operation of an inspection system, operated by a robot, also discussed in the Introduction. The transmission is carried out by an optical signal. (1) What type of transceiver in the control site do you recommend? (2) What type of transceiver in the remote site do you recommend?A transceiver contains a module for transmission and a module for receiving. control site
remote site
monitor
transceiver
transceiver fiber PC
FIGURE P5.7 Data transmission for remote operations.
sensor
Optomechatronic Integration
297
(3) Write down the signal flow mathematics for this system for each. (4) Draw a complete signal flow graph describing the remote operation of this system. Assume that there is no feedback operation at the control site, that is, one-way operation. P5.8. Laser welding process considered for discussion in “IntegrationGenerated Functionalities” in Chapter 5 is shown in Figure P5.8. (1) Describe in detail how it works to join two metal plates. (2) Based on this figure, draw a complete signal flow graph to carry out the welding task. laser
welding head work piece motor displacement FIGURE P5.8 A laser welding system.
References [1] Cho, H.S. Characteristics of Optomechatronic Systems, Chapter 1, Optomechatronic Systems Handbook. CRC Press, 2002. [2] Cho, H.S. and Kim, M.Y., Optomechatronic technology: the characteristics and perspectives, special issue on optomechatronics: fusion of optical and mechatronic engineering, IEEE Transaction on Industrial Electronics, 52:4, 932– 943, 2005. [3] Hockaday, B.D. and Waters, J.P. Direct optical-to-mechanical actuation, Applied Optics, 29:31, 4629– 4632, 1990. [4] Kim, S.G., Optically sensed in-plane AFM tip with on-board actuator, Lecture No. 6.777, Design and Fabrication of Microelectromechanical Device, Final report of Design Project, MIT, 2002. [5] Liu, K. and Jones, B.E. Pressure sensors and actuators incorporating optical fibre links, Sensors and Actuators, 17:314, 501– 507, 1989. [6] Ogata, K. Modern Control Engineering, International Edition. Prentice-Hall, Englewood Cliffs, NJ, 1970.
6 Basic Optomechatronic Functional Units CONTENTS Optomechatronic Actuation............................................................................. 301 Silicon Capacitive Actuator ...................................................................... 302 Optical Piezoelectric Actuator ................................................................. 304 Mathematical Model of the Photon-Induced Currents................ 305 Induced Strains of the PLZT ............................................................ 306 Photo-Thermal Actuator ........................................................................... 311 Optomechatronic Sensing................................................................................. 316 Optical Sensor............................................................................................. 316 Fabry-Perot Etalon.............................................................................. 318 Fiber Optic Sensors .................................................................................... 321 Automatic Optical Focusing ............................................................................ 326 Optical System Configuration.................................................................. 327 Optical Resolution ..................................................................................... 328 Axial Resolution ................................................................................. 329 Feedback Control of the Objective Lens ................................................ 330 Effect of External Disturbance.......................................................... 336 Focus Measurement ........................................................................... 338 Acoustic-Opto Modulator ................................................................................ 339 Deflector ...................................................................................................... 341 Frequency Shifter ....................................................................................... 345 Tunable Wavelength Filtering .................................................................. 347 Efficiency of Modulation and Speed....................................................... 347 Optical Scanning ................................................................................................ 348 Galvanometer ............................................................................................. 349 Feedback Control of Galvanometer ........................................................ 356 Polygonal Scanner...................................................................................... 363 Correcting Scan Errors....................................................................... 367 Optical Switch .................................................................................................... 367 Thermally Actuated Mirror...................................................................... 369 Electrostatically Actuated Mirror Control ............................................. 371 Lens Controlled Switching ....................................................................... 375 Zoom Control ..................................................................................................... 377
299
300
Optomechatronics
Zooming Principle ..................................................................................... 377 Zoom Control Mechanism........................................................................ 383 Visual Autofocusing .......................................................................................... 386 Image Blurring............................................................................................ 386 Focus Measure............................................................................................ 389 Illumination Control.......................................................................................... 399 Illumination Methods................................................................................ 400 Illumination Control.................................................................................. 403 Illumination Quality Measure.................................................................. 408 Autofocusing with Illumination Control ............................................... 410 Visual (Optical) Information Feedback Control............................................ 411 Visual Feedback Control Architectures .................................................. 414 Fixed Camera Configuration ............................................................ 417 Eye-In-Hand Configuration .............................................................. 420 Feedback Controller Design ............................................................. 421 Optical Signal Transmission............................................................................. 428 Signal Transmission ................................................................................... 428 Power Transmission and Detection ........................................................ 431 Problems.............................................................................................................. 432 References ........................................................................................................... 443 In the previous chapter, we have seen that signal elements (mechanical, optical, and electrical) can be combined in a variety of ways to produce basic functional modules which include transformation, manipulation, transduction, actuation, transmission, storage, and display of the signal. These modules were often shown to be optomechatronic in nature, or may be formed by other natures such as optoelectronic, mechatronic, and so on, as we have discussed in Chapter 5. Due to the presence of optomechatronic interaction, the integration was found to create a variety of different types of functional modules, adding more attributes to the existing ones. Any engineering system that includes devices, machines, or processes must possess one or more functionalities in order to perform the desired tasks as specified by the designer. In essence, such a functionality is created by an individual module or combination of functional signal modules. Depending upon how many modules are involved or how they interact with each other, the functionality may have different attributes. The functionalities created in this way can be combined to make a system exhibit a certain functional behavior as specified by the designer. In general, a system or device will need a certain number of functionalities to perform a given task. But, in case the device carries out a number of different tasks, it will need a number of different functionalities. Generalizing this notion to the case of many systems or devices, we see that a number of functionalities may be frequently used for certain tasks. The fundamental functionalities are those functionalities that are common everywhere in optomechatronic-related engineering fields. In this chapter,
Basic Optomechatronic Functional Units
301
we will consider the fundamental functionalities produced by the modules that are the basis of creating optomechatronic properties. As mentioned briefly in Introduction, these properties include: (1) optomechatronic actuation, (2) optomechatronic sensing, (3) optical autofocusing, (4) AO modulator, (5) optical scanning, (6) optical switching, (7) zoom control, (8) visual autofocusing, (9) illumination control, (10) visual servoing, and (11) optical signal transmission. Because these functionalities are frequently used in most optomechatronic systems, we will focus on understanding their basic concept, hardware configuration, and important factors associated with their performance.
Optomechatronic Actuation The actuating function, one of the fundamental functions needed for actuating optomechatronic systems, is the actuation produced by a mechatronic actuator that drives optical systems, or by an optical drive that actuates mechatronic systems. The types of optomechatronic actuation can be grouped into three classes. The first class is the optical actuators which employ the optically-driven actuation principle. In this case, the energy supplied for actuation comes from a light source, which provides displacement or force to moving mechanisms. A variety of this type of actuator has been developed for different applications. The second class is the mechatronic actuators with embedded optical units: Here, mechatronic actuators implies all non-optical actuators. An electrical motor with an optical encoder is a typical example. The third class encompasses the actuators that drive an optical or optomechatronic system. In this chapter, we will discuss some of the optically driven actuators. Optical actuators function when light is either directly or indirectly transformed into mechanical deformation that generates small displacements of micro- or nanometer scale. Depending upon the interaction of light energy with an absorbing surface, a variety of effects can occur. According to their nature, two types of optical actuations are considered here: photoelectric actuation and photo-thermal actuation. Photo-electric actuation utilizes the conversion of the variations in light intensity into the change in the electric power by means of a p –n junction of semiconductor devices, or piezoelectric material resulting from the generation of photo electrons. The other form of conversion uses the change in capacitance in a capacitor type of actuator configuration. In addition to these types, optical actuators that belong to this photo-electric class use a variety of other conversion methods, including photo-conductivity modulation, photodiode, direct optical manipulation (optical twizer), and so on. In contrast to this principle, photo thermal actuation utilizes the conversion of the variation in light energy into the change in thermal energy.
302
Optomechatronics
Silicon Capacitive Actuator Silicon micro actuator is one of the photo-electric actuators that utilize the photon-generated electrons. This actuator uses change in the stored charges in a capacitor, which results in change in the electrostatic pressure acting on a cantilevered beam. This actuator therefore does not use piezoelectric or thermal effects for silicon microstructures. As shown in Figure 6.1a, the actuator is composed of a cantilever beam (Si) and a ground plate (Cu) on an insulating glass, which forms a parallel plate capacitor [33]. This parallelplate capacitor is given by a simple relation according to Equation 4.2 A d
C0 ¼ 10 1r
where A is the area of the cantilever facing the ground plate, 10 is the permittivity of the free space, 1r is the permittivity constant of the medium between the gap, which in this case is air, and d is the gap between the cantilever and the ground plate. If an electric voltage V0 is applied to the capacitor through a resistor R, the stored charge is obtained by q0 ¼ C0 V0
ð6:1Þ
Now, let us consider the optical actuator subject to the electrical potential V, whose gap is not constant due to the cantilevered beam being clamped at one end. The stored charge will then produce an electrostatic force, which in turn causes a deformation of the cantilever. Since the beam has a clamped-free condition at its ends, the deflection d at the tip can be described by
d ¼ kq q2
ð6:2Þ
where kq is a constant related to the geometry and material property of the cantilever. This relation can be derived from a simple beam theory from which the deflection of a cantilever beam is governed by d4 z 4 ¼ pðxÞ dx4
cantilever (Si) d
d
e ee
V
light ground plane(Cu) glass
R (a) actuator configuration
FIGURE 6.1 Semiconductor capacitive optical actuator.
total current (i). pA
ð6:3Þ
O
light intensity
DC bias voltage, Vb
(b) variation of total current with DC bias voltage
Basic Optomechatronic Functional Units
303
where z is the vertical displacement at the location x along the beam, and pðxÞ is the distributed loading due to the applied voltage. In the above equation, the cantilever has small deflection and the thickness h p ‘: Attention should be given in the above equation that, due to deflection, the electrostatic force is no longer uniform or constant, but depends on x. Nevertheless, we will assume for simplicity a constant loading condition. According to the electrostatic force theory, voltage causes a constant electrostatic force between the ground plate and the undeformed cantilever by the relation given by pðxÞ ¼
10 V 2 2d2
ð6:4Þ
as given in Equation 4.49. Substituting Equation 6.4 into Equation 6.3 and evaluating zðxÞ at the end of the cantilever, we have zðxÞlz¼l ¼ d <
1 0 ‘4 2 V 8d2
ð6:5Þ
It is found that this relation is valid for small d p d: This relates the displacement d to the applied voltage. If we neglect the changes in capacitance formed by the cantilever beam and ground plate, as the applied voltage increases, the displacement can be expressed by using the relation q < C0 V: Substituting this relation into Equation 6.5, we have the same expression as given in Equation 6.2, which relates the displacement at the cantilever tip to the stored charge q due to the applied voltage. This implies that the electrostatic field provides the cantilevered optical actuator with a potential energy field. When monochromatic light (photon) strikes the ground metal plate through the air gap, the photoelectrons are generated through the gap to the cantilever plate. These photo electrons reduce the charge on the capacitor from the q0 at the stressed condition by the following equation ðt ð6:6Þ qðtÞ ¼ q0 2 gAFa dt 0
where Fa is the optical power impinging on the area of the ground plate, A, and g is a constant related to photoelectron flux generation. Examination of Equation 6.5 and Equation 6.6 indicates that generation of photoelectron current can control the cantilever beam position denoted by a dotted line shown in the figure. In more detail, this implies that the generated photoelectron, according to Equation 6.6, causes the stored charge to leak at a rate in proportion to its flux. This results in conversion of the stored potential energy to kinetic energy. Therefore, the cantilever resumes its deformed state toward its unstressed position. To make this actuator continuously operate, it is necessary to charge it with a battery or other current source. Depending upon the magnitude of source current, the induced photocurrent will control the leakage of the charge through the gap, whose amount determines the direction
304
Optomechatronics
of the cantilever motion. The photoelectric actuation differs from an electrostatic actuation in that increase in optical flux relaxes the cantilever to be open toward its undeformed state. Figure 6.1b illustrates how white light illumination on the ground plate creates the photo current with an increase of DC bias voltage, V. Total current includes the current induced due to optical power and that of the induced current due to bias voltage. The figure indicates a nonlinear relationship between the current and the voltage. This relationship differs from magnitude of light intensity, and shows that, as light intensity increases, total current increases accordingly. It is reported that for a bias voltage 6 V and optical power less than 0.1 mW/cm2 for 600 £ 50 £ 1 mm3 cantilever beam with a gap of 12 mm. Optical Piezoelectric Actuator When a piezoelectric material is irradiated by light (UV light) as shown in Figure 6.2, it exhibits two interesting phenomena: (1) photostrictive, (2) pyroelectric. Photostrictive effect is essentially photovoltaic effect caused by the electromotive force which comes from the electrons generated due to the irradiation by the light. Pyroelectric effect is due to a thermal electromotive force when the piezoelectric material such as lead zironate titanate (PLZT) experiences temperature change at the time of the light irradiation. The bimorph piezo element shown in the figure is a ceramic polycrystalline material made of a PZT material such as Pb (Zr, Ti)O3 and has a shape like a rectangular parallelogram [15 – 17]. As shown in the figure, two pieces of the PLZT element after polarization are glued so that each polarization bimorph PZLT light electrode light source
light
light
Bimorph PZLT (a) a bimorph PLZT beam element FIGURE 6.2 Optical piezoelectric actuator.
light source
bimorph PZLT
(b) deflection of PLZT when subjected to UV light
Basic Optomechatronic Functional Units
305
piezoelectric element photovoltaic effect
+ pyroelectric effect
+
mechanical deformation
inverse piezo electric effect
+ +
opto-thermic effect
thermal conduction
thermal deformation
FIGURE 6.3 The energy conversion processes to induce mechanical deformation.
direction is opposite to the others. Electrodes are installed at both ends of the PZT element. When this element is tested to investigate the time history of the displacement at the element tip, it is found that the major factor influencing the cause of displacement is the illumination intensity of UV light. The cause is mainly due to two combined effects, photostritive and thermal effects. Both of these effects convert electric energy into mechanical energy, which is in effect the inverse of piezoelectric effect, so that the actuator may deform from its original state. Therefore, the actuator deformation in total is composed of the contributions by those two effects and by pure thermal deformation. The energy conversion process is schematically illustrated in Figure 6.3. This optical actuator can be utilized for various applications such as optical switch, microvalve, and microrobot. Mathematical Model of the Photon-Induced Currents It is very interesting to see how these two photon-related effects are related to the PLZT deformation. In fact, the energy conversion mentioned in the section above is based upon the relationship. Therefore, it is necessary to obtain appropriate analytical models which can describe the two phenomena, rendering the relationship between induced current and optical energy. However, it is difficult to identify the relationship analytically, since it is involved with physical phenomena occurring in various stages of energy conversion that are complex and nonlinear in nature. Nevertheless, we will model the conversion process based on Figure 6.3. For this modeling, we will assume that the conversion processes can be described as linear models, and that we can use some experimental results to obtain such mathematical models. Some experimental results [15 – 17] show that electromotive current ip ðtÞ due to photovoltaic effect may be expressed by the following first order differential equation dip ðtÞ ¼ 2tp ip ðtÞ þ cp up dt
ð6:7Þ
306
Optomechatronics
where tp is the discharge time constant, cp is the conversion coefficient involved with conversion from photonic energy to electric energy, and up is the irradiated light energy. The pyroelectric current it is governed by the temperature difference between both sides of the PLZT and can be expressed by it ðtÞ ¼ Ct
d {u ðtÞ 2 urm ðtÞ} dt ‘m
ð6:8Þ
where Ct is the coefficient involved with conversion from thermal energy to electric energy, u‘m ðtÞ and urm ðtÞ are the average temperature at both sides of the PLZT due to irradiation, respectively. Induced Strains of the PLZT Once the two photon-induced currents are obtained, the induced strains of both sides of the PLZT can be explicitly expressed as a function of the electric charge by 1 a qðtÞ 2 q 1 1rp ðtÞ ¼ 2 aq qðtÞ 2
1‘p ðtÞ ¼
for the irradiated side
ð6:9Þ
for the back side
where aq is the coefficient of strain conversion from the electric current to mechanical strain for the PLZT element, and qðtÞ is the electric charge produced by the light photonic energy. The electric charge is governed by dqðtÞ ¼ 2tq qðtÞ þ ½ip ðtÞ þ it ðtÞ dt
ð6:10Þ
where tq is the time constant of the electric charge process. On the other hand, the strain 1‘t and 1rt, due to thermal energy produced by the incident light can be expressed by 1‘t ðtÞ ¼ at u‘m ðtÞ
1rt ðtÞ ¼ at urm ðtÞ
ð6:11Þ
where at is the coefficient of thermal expansion of the PLZT. Combining Equation 6.9 and Equation 6.11, we have total strains due to photovoltaic effect, pyroelectric effect, and thermal deformations 1‘ ðtÞ ¼
1 a qðtÞ þ at u‘m ðtÞ 2 q
1 1r ðtÞ ¼ 2 aq qðtÞ þ at urm ðtÞ 2
ð6:12Þ
From these strains, the displacement at the tip of a cantilever due to bending denoted by d‘ ðtÞ can be expressed by
d‘ ðtÞ ¼
3 L2 ½1 ðtÞ 2 1r ðtÞ 8 h ‘
ð6:13Þ
where L is the length of the optical piezoelectric element, h is the thickness of the same element, which can be easily obtained using the beam theory given in Appendix A2.
Basic Optomechatronic Functional Units
307
We are now able to consider control of the optical actuator, based upon the above derived model equations. Accurate control of the actuator is essential for microsystem positioning, because any error occurring from the control may be the order of positioning dimension. To determine the response of the system for a given controller, we need to determine all of the transfer functions. Some are neither simple nor linear. In fact, the relationship between the movement of the optical actuator and the intensity of the UV beam is nonlinear. The coefficients related to photo pyroelectric and thermal phenomena, such as pizeoelectric constant, coefficient of peizoelectric current, heat transfer coefficient, heat capacity, and so on, exhibit highly nonlinear behavior. If the actuator system is operated at a certain nominal point, the relationship can be linearized near that point. The amount of UV light is controlled by a servomotor which can adjust the opening area of an optical fiber by rotating a light valve plate. The relationship between the DC motor and optical power of the UV beam is also highly nonlinear. For simplicity, we will assume all elements are linear. Figure 6.4a shows a servo-controlled optical actuator system in which an electric motor controls the position of the valve. The control system is composed of an optical actuator Gp ðsÞ consisting of two bimorph PLZT elements, a UV beam source Gl(s), a servomotor, Gm(s), a controller Gc ðsÞ, and a displacement sensor Gs ðsÞ: The control objective is to keep the position of the optical actuator xðtÞ as closely as possible to a desired location xd : In other words, the controller must be so designed that the error between xd and x(t), e(t), be driven to zero steady state with fast response. The open loop transfer function between the displacement at the tip dl ðsÞ and UV beam energy Up ðsÞ can be obtained by Laplace transforming both sides of Equation 6.7 through Equation 6.13. This can be written by
d ðsÞ 3 L2 ¼ Gp ðsÞ ¼ l 8 w Up ðsÞ
desired position
Xd
"
C p aq sCt aq þC ðs þ tp Þðs þ tq Þ ðs þ tq Þðs þ tt Þ
controller i Gc (s)
e + _
DC motor Gm (s)
d
# ð6:14Þ
UV radiater up optical actuator Gt (s) Gp (s)
X
sensor Gs (s)
(a) a closed loop optical control system Xd
e +_
controller Gc (s)
i
UV source/motor up Kl
optical actuator Gp (s)
(b) a simplified control system FIGURE 6.4 Block diagram of an optical actuator.
X
actual position
308
Optomechatronics
In deriving the above equation, the induced temperature difference given in Equation 6.8 is approximated by a first-order lag-type transfer function, and C is a constant related to thermal phenomena. For simplicity of controller design, let us suppose that the dynamics of the servomotor-UV beam and sensor are fast enough to be neglected, as compared with that of the optical actuator itself. In this situation, the control system has two elements; plant to be controlled Gp ðsÞ and controller Gc ðsÞ: The control system block diagram shown in Figure 6.4a in this case can be reduced to Figure 6.4b to describe the simplified loop transfer function Go ðsÞ which is given by Go ðsÞ ¼
XðsÞ ¼ Kl Gp ðsÞ IðsÞ
ð6:15Þ
where dðsÞ is replaced by X(s), i is the current input to the UV radiator,Kl is the gain associated with the radiator and servomoter. Since Gp ðsÞ is given in Equation 6.14, we have Go ðsÞ ¼
3kl L2 aq ðb0 s2 þ b1 s þ b2 Þ 8w s3 þ a0 s2 þ a1 s þ a2
where the above coefficients are b0 ¼ CCt aq , b1 ¼ Cp aq þ CCt aq tp , b2 ¼ Cp aq tt , a0 ¼ tp þ tq þ tt , a1 ¼ tp tq þ tp tt þ tq tt , a2 ¼ tp tq tt , and C ¼ 2:0 £ 1024 : Let the desired deformation at the tip of the PLZT be 20 mm, i.e., Xd ðsÞ ¼ 20 £ 1026 £ ð1=sÞ: With the open loop transfer function Go ðsÞ of the optical actuator system and a PID controller given in the form Gc ðsÞ ¼ kp þ
ki þ kd s s
ð6:16Þ
we can simulate the closed loop control system in order to see the control performance of the optical actuator, using the parameters listed in Table 6.1. All the transfer functions are put together and given to the Simulink model. Figure 6.5 shows the simulation results obtained for various controller gains. Figure 6.5a shows the effect of the proportional gain for a given set of the integral and derivative gains ki ¼ 50, kd ¼ 5. It shows that as the proportional gain decreases the response gets sluggish but exhibits higher overshoot, the trend of which appears unlike that of a standard second order system. The steady state is reached in approximately 5 sec. The effect of the integral gain can be seen in Figure 6.5b, from which we can see that as ki increases the response gets faster but has overshoot. The other two integral controller gains ki ¼ 100 and ki ¼ 150 show fast response, and, as expected, the responses exhibit larger overshoot as ki increases. Figure 6.6a exhibits the neutrally stable responses for a set of gain parameters, kp ¼ 0:7; ki ¼ 100; kd ¼ 5: However, when kp gain decreases slightly from the above value, the response gets unstable, as can be seen from Figure 6.6b.
Basic Optomechatronic Functional Units
309
TABLE 6.1 The Parameters Used for the PLZT Simulation Parameters
Symbol
Unit
tp tq tt L w Ct aq Cp kl
Natural discharge constants Natural discharge constants Natural discharge constants Length of the optical piezoelectric element Thickness of the same Pyroelectricity conversion constant Strain conversion coefficient Conversion coefficient Constant
Value
21
s s21 s21 m m CK21 C21 Cm2W21
1.5 0.5 0.075 2.5 £ 1022 1.8 £ 1022 7 £ 1029 1.5 £ 1024 4.2 £ 10212 650
Source: From Fukuda, T. et al. IEEE Transactions on Industrial Electronics, 42(5), 455–461, 1995. q 2005 IEEE.
25
displacement (y), μm
displacement (y), μm
This optical actuator has a variety of applications. Figure 6.7 illustrates an application to a gripper-like structure of a robotic manipulator [15]. The gripper has two bimorph actuators subject to incoming UV beam, while UV beam is transmitted through two optical fibers. The beam transmitted irradiates the mirrors. In this system, a feedback controller can be employed in a similar manner to that discussed earlier to adjust the gripper position, and is schematically shown in Figure 6.7b. The UV beam is controlled by adjusting the iris of the beam radiators, whose movement is controlled by a DC servo motor. The displacement of the optical actuator is assumed to be measured by a non-contact displacement sensor. Referring to the figure, Gl ðsÞ is the transfer function between the controller input and the radiated input of the UV beam, Gop ðsÞ is the transfer function between the UV light and
20 kp = 50 kp = 100 kp = 150
15 10 5 0
0
1
2 3 time (t ), ms
4
5
(a) the effect of proportional controller gain; ki = 50, kd = 5
25 20 ki = 50 ki = 100 ki = 150
15 10 5 0
0
1
2 3 time (t ), ms
4
(b) the effect of integral controller gain; kp = 150, kd = 5
FIGURE 6.5 The controlled response with various controller gains.
5
Optomechatronics 40 35 30 25 20 15 10 5 0
displacement (y), mm
displacement (y), mm
310
0
2
4
6 8 10 time (t ), ms
12
100 80 60 40 20 0 –20 –40 –60 –80 0
14
(a) kp = 0.7, ki = 100, kd = 5
2
4 6 time (t ), ms
8
10
(b) kp = 0.01, ki = 100, kd = 5
FIGURE 6.6 Oscillatory and unstable responses of the optical actuator.
actuator movement, Gp ðsÞ is the gripper transfer function, Gc ðsÞ is the controller transfer function, and Gs ðsÞ is the sensor transfer function. The overall transfer function is obtained by Gc ðsÞGm ðsÞGl ðsÞGop ðsÞGp ðsÞ XðsÞ ¼ Xd ðsÞ 1 þ Gc ðsÞGm ðsÞGl ðsÞGop ðsÞGp ðsÞGs ðsÞ mirror
ð6:17Þ
UV light optical fiber
optical gripper
iris (valve)
mirror
mirror supporter
motor
(a) configuration of optical microgripper
Xd
controllerD/A converter e Gc (s) D/A + –
optical DC motor UV source actuator up f i Gm (s) Gop (s) Gl (s)
Gs (s) A/D converter
sensor
(b) control system block diagram FIGURE 6.7 Microprocessor-controlled optical gripper.
gripper Gp (s)
actual position X
Basic Optomechatronic Functional Units
311
If the same simplified system presented in Figure 6.4b is used and the gripper is approximated by a linear system Gp ðsÞ; we can carry out the same procedure as done previously. This gripper system can be controlled by a built-in microprocessor. In the digital control system as shown in Figure 6.7b, the measured displacement signal is taken via A/D to a microprocessor. Based on this system, the controller command signal is computed in the processor according to the control algorithm which, in this case, is a proportional control action. The computed values are sent to D/A, which converts the discrete values into the corresponding analogue. This analog signal is then used to drive a drive circuit which in turn actuates the DC motor that gives the motion of the DC motor. The displacement of the kind of gripper is found to achieve several hundred mm, although the magnitude is dependent upon the intensity of UV light, the geometry of the gripper and the PLZT material. Photo-Thermal Actuator This actuator employs an indirect means of transforming light energy into mechanical actuation. Typical arrangements of the photo-thermal actuator are illustrated in Figure 6.8. The actuator shown in Figure 6.8a is composed of a light power source, a closed miniature cell that contains air or gas, a light energy absorber, and a flexible diaphragm that generates mechanical motion. The light coming out of an optical fiber is introduced through a trapped air
light shape memory alloy
optical fiber
diaphragm support
collimating lens
support spring
a light energy absorber glass cell back
(a) closed miniature cell
(b) shape memory alloy light
glass silicon diaphragm nozzle
(c) gas field cavity FIGURE 6.8 Photo-thermal actuators.
gas flow
electrostatic bonding spacer glass
312
Optomechatronics
hole in the left end of the cell. This light energy causes the absorber to heat, causing the air or gas trapped in the cell to expand, which in turn causes the membrane to move outward. Another optical manipulation shown in the right-hand side of Figure 6.8b utilizes the expansion of solids that can be generated due to phase-transformation, as can be found from shape memory alloys such as NiTi, Cu – Zn– Al and Cu – Al – Ni. As discussed in the section “Microactuators” in Chapter 4, the shape memory effect is a unique property of such alloys which have the ability to transform from their basic shape to a predetermined shape when heated above a critical “transformation temperature.” The material remembers its original shape when strained and deformed. The silicon diaphragm valve is a typical application of the indirect optical actuator. The valve is optically actuated and used to control the flow of a gas as depicted in Figure 6.8c. When light heats up the gas filled in the cavity, it expands and actuates the diaphragm, thus adjusting the amount of gas or liquid flowing through the nozzle. The diaphragm displacement can be modeled as a function of incident light intensity. Once this is modeled, a similar analysis can be performed to those given above. The configuration of a micromachine employing the principle of Figure 6.8b is illustrated in Figure 6.9 [35,36]. The machine is composed of a body consisting of shape memory alloys and elastic spring and feet made of temperature sensitive ferrites, permanent magnets, and spacers which are assembled in a case separately. The body shown in Figure 6.9a has a capability to stretch when light is projected on to the surface of the shape memory alloy. On the other hand, when light is switched off, the body shrinks. In this way, the body stretches and shrinks by switching on and off a light beam. The foot jointed to the body has a capability of locking and moving freely. In interaction shape memory alloy ferrite
ferrite
magnet spacer
magnet spacer spring (a) system configuration temperature sensitive ferrite magnet spacer
light
A
B
gap floor
floor
(b) foot release-locking mechanism FIGURE 6.9 Optically actuated SMA walking machine. Source: Yoshizawa, T., et al., Proceedings of SPIE, 4190, 212–219, 2001.
Basic Optomechatronic Functional Units
313
with light, the foot is in the state of “free-to-move” or “locking,” as in Figure 6.9b. In a normal state without light (A), ferrite and a magnet are stuck together under a certain magnetic force. When light is projected on to the ferrite surface, a locking state is generated: The temperature of the surface rises and brings about a magnetization change at the Curie temperature. Under this condition, the attractive force between the ferrite and the permanent magnet becomes weakened, causing both the magnet and spacer to drop down on to the floor. This creates a locking motion state at the foot (B). When the light beam disappears, the state of the foot comes back to state (A). When the action of switching the projected light beam on and off is repeated, this kind of movement continues, moving the machine in the right direction, as shown in Figure 6.10. This light-activated machine can walk on a floor, and beneath a ceiling, climb a slope, and move on a curved surface. In any case, since the dynamic properties of the machine are important, it is necessary to minimize the size and weight of the machine. Size and weight are critical, especially when the machine climbs up a slope. In this situation, slipping may occur, causing a retarded motion which is certainly not desirable. As shown in Figure 6.11, when mass m1 moves while mass m2 sticks to the floor, the condition that eliminates slipping between the feet and the floor is obtained by considering the following two inequality conditions 2mF1 2 mðm1 þ mb Þg cos u þ F 2 ðm1 þ mb Þg sin u . 0
mF2 þ mm2 g cosu 2 F 2 m2 g sin u . 0
light on locking
1
B
A
2 (1) locking foot A
stretch locking
walking direction B
A
(2) stretching foot B B
A
locking
(3) locking foot B
shrink B
A
locking (4) shrinking foot A
d
displacement
FIGURE 6.10 Schematic of the walking process made within a cycle motion. Source: Yoshizawa, T., et al., Proceedings of SPIE, 4190, 212– 219, 2001.
314
Optomechatronics N2 light
mbg sinq
B A
m2g
q F1
m1g
displacement
mN2
m2g sinq
m1g sinq mN1
F
F
N1
F2 m2g cosq
mbg m g cosq b
m1g cosq
FIGURE 6.11 Force model of a micromachine climbing an indented slope. Source: Yoshizawa, T., et al., Proceedings of SPIE, 4564, 229–237, 2001.
from which, we have DF ¼ F1 2 F2 .
mt g sin u þ mb g cos u m
ð6:18Þ
where DF is the difference of maximum F1 and minimum sticking force F2, generated by the foot locking-release mechanism, mt is the total weight of the machine, mb is the weight of the body, m is the friction coefficient, and u is the slope angle. This result indicates that the machine needs large DF, smaller mt, and large m to climb an inclined slope, the trend of which depends upon the angle of the inclination. Thus, the machine can move in various conditions (such as a horizontal plane, the ceiling, a vertical wall) and show little difference in speed. Typically, with a halogen lamp (100 V/50 W) the machines moves at an average speed of 4 mm/cycle horizontally, and climbs vertically at 2.9 mm per cycle. As observed from the above discussions, the amount of light beam determines the amount of shrink and stretch of the shape memory, which essentially controls the distance per one moving cycle. When this distance needs to change while during motion, a distance-control system can be built in such a way that the moving distance per movement cycle is kept as desired. Such a control system is illustrated in Figure 6.12. The system is composed of a light source whose intensity is controllable by a source,
desired foot step Sd
error + _
controller
u
light source
I
sensor FIGURE 6.12 A block diagram for the foot step control system.
SMA actuator
F
moving machine
actual foot step S
Basic Optomechatronic Functional Units
315
a shape memory actuator, and a moving machine. Suppose we set a desirable foot-moving distance according to a moving-path plan. When it starts to move, a sensor measuring its absolute position yields an output which is fed back to compare with the desired value. Based on the difference in value between the two, the controller generates a control command signal which activates a drive circuit of the light source. This control action adjusts the amount of light and its intensity necessary to actuate the shape memory alloy (SMA) part at a desired state. Accordingly, the phase transformed SMA yields a desired force, which then moves the machine by a desired distance for its next cycle. The third class of actuators driving optical or optomechatronic systems can be found from a variety of practical systems to be discussed in the later part of this chapter. Here, we introduce a deformable mirror concept for controlling the phase of light wave. A deformable mirror to correct the telescope image distortion caused by the air disturbance is schematically illustrated in Figure 6.13. The first one shown in Figure 6.13a is a monolithic deformable mirror actuated by a PZT ceramic bulk with two-dimensionallyarrayed electrodes. Applying suitable electrical voltage on these electrodes forms a locally deformable mirror surface. A glass mirror actuated by needle shaped piezoelectric ceramic bodies arranged and embedded in a polymer matrix is another example of this class. In both cases, a desirable mirror surface contour is generated by applying suitable voltages on the electrodes, which can correct the wave front distortion. These adaptive characteristics of the mirror can control the phase of the light wave caused by the distortion. Deformable mirrors based on MEMS technology shown in Figure 6.13b are to correct the distorted wave front, which utilize a great member of a tiny
light deformable glass mirror
wave front
lenslet PZT actuator stack
electric leads original locations (a) deformable mirror
(b) lenslet array
FIGURE 6.13 Wave front correction by deformable mirror and lenslet array.
316
Optomechatronics
lenslet. This device provides smoother mirror surface as the number of lens in the array increases.
Optomechatronic Sensing Sensing is a basic functionality that optomechatronic systems possess. Depending upon whether optical or mechatronic elements participate in providing sensing function or not, optomechatronic sensors are classified into the following two groups. The first group represents optical sensors whose sensing function is enabled by incorporating with mechatronic units. Most of the optical sensors belong to this class. The second group represents non-optical sensors integrated with optical units or elements belonging to the reverse case of the above example. In this case, sensors have optical units within their body to constitute a sensing element. Optical Sensor As an illustration of an optical sensor, consider an optical encoder which measures angular or linear motion of mechatronic systems in an optical
photo-electronic sensor encoder disc rotor of motor coil
bearing
photo-electronic sensor motor bearing
(a) DC-tacho / encoder assembly details LED mask wheel AB
phototransistors motor shaft
360˚
A B I 90˚
180˚
channel A
channel B channel I
(b) simplified schematic of an optical encoder (c) diagram of the output signals of a digital encoder
FIGURE 6.14 Optical encoder.
Basic Optomechatronic Functional Units
317
way by adopting the optical photoelectric principle. This sensor is based on the optomechatronic principle, which belongs to the first class, because its measurement principle is based on the interaction of optical signal with mechanical signal (motion). As shown in Figure 6.14a, the rotating encoder is composed of a LED, a light source, a disk with mask, and a phototransistor (See Figure 6.14b). The LED emits light through the indexed code disk. Then, photo transistors react to the light input passed through the masks, yielding electrical outputs of a square wave or quasisine shaped signal, which is used for further processing, such as counting and positioning. The comparison between the phase of the outputs of channels A and B determines direction of rotation, as indicated in Figure 6.14c, which in this case shows counter-clockwise rotation of the shaft. The index channel I is used as reference point for determining the traveled angle of rotation. This is a typical configuration of most electric servomotors with an embedded optical encoder that are being used currently. The principle of velocity feedback of a motor control using the optical encoder is as follows: When a microprocessor inputs a desired velocity command to the motor system, the motor starts to rotate with a certain angular speed. This motion is detected by an encoder whose signal, in turn, is utilized to calculate the speed by code converter. The computed speed is then fed back to a controller embedded within the microprocessor. The controller calculates the error between the desired and actual velocity, based on which controller command signal is given to the PWM circuit. The signal thus generated goes to the drive circuit, which finally alters the motor speed so as to eliminate the velocity error. The linear encoder utilizes the same principle as that of the optical encoder, based on translating grating patterns. As a moving member generates a shift of the moving grating pattern relative to a fixed grating, a series of pulses is generated and counted to detect the position of the member. The interferometric dilatometer is a variation of Michelson type that is based upon opto-mechatronic interaction between the optical and mechatronic units. This type of measurement is necessary when an object to be measured is subject to temperature fluctuations. As discussed in Chapter 2, in the section “Interference,” the Michelson interferometer utilizes the principle of the light intensity change with optical path length difference by every half wavelength, when two light beams with the same wavelength interfere. With this conventional interferometer, fluctuations inevitably cause inaccuracy in measurement and undesirable decrease in sensitivity. The modified measurement system shown in Figure 6.15 employs a mechanical servo actuator made of PMN (lead magnesium niobate based ceramic) which has small hysteresis in the induced strain curve. This servo actuator generates appropriate motion so as to cancel the drift caused by the thermal fluctuation or laser noise. The servo system is the key to improved stability and accuracy.
318
Optomechatronics object
mirror moving y'
C
laser source
mirror (s) actuator
O y
lens
d
half mirror
photo detector FIGURE 6.15 The interferometric dilatometer.
Fabry-Perot Etalon The Fabry-Parot etalon considered in Chapter 2 has a variety of different fringe characteristics controlled by varying one of the mirror separations, d, the index of refraction nt, the incident angle u: Due to this changeability of its
piezoelectric actuator
d mirror plate
medium
mirror plate
(a) configuration of Fabry-Perot Etalon desired x position d +_
controller
servo driver
piezoelectric actuator
Fabry Perot etalon
sensor (b) feedback control schematic for positioning of the Fabry-Perot etalon FIGURE 6.16 Precision positioning of mirrors with piezoelectric actuators.
x
actual position
Basic Optomechatronic Functional Units
319
parameters, and its special characteristics, the device has numerous applications in various fields. We will consider some of its characteristics under the situation that the mirror separation can be made changeable by piezoelectric actuator, as shown in Figure 6.16a. The etalon is composed of two piezoelectric actuators, two transparent mirror plates (or coated plates), and two connecting rods that link the plates. The actuators driven by the servo driver control the positions of two mirror plates according to a given desired mirror spacing. Figure 6.16b illustrates a control system that realizes this position control. The control system requires the precision positioning, and therefore, needs an accurate control. According to the control strategy, the mirror position error is fed back to the servo driver which drives the piezoelectric actuators. The actuators, in turn, actuate the mirror system composed of a lumped mass and a damping element that might exist in the actuators. This kind of control action continues until the mirror separation reaches a desired value. Based on the controllable mirror separation, we will discuss three characteristic features. First, it can enable us to calculate the unknown wavelength by moving the plates of the device and counting the number of interference fringes. As the plate moves to change the distance d, the fringe will move across the screen due to changes in constructive and destructive interference length. If the distance moved is accurately measured by precision sensors, and if the number can be counted by using a photodiode placed at the center of the fringes, we can calculate the wavelength of the incident light from Equation 2.53
l¼
2d m
for the incident angle normal to the plate surface. The second characteristic comes from the concept of resolvable power. So far, only a source with a single wavelength has been considered. When two wavelength components l0 and l1 ¼ l0 þ Dl; are contained in the source, superposition of two fringe systems would result. In this situation, if the wavelengths are very close, they may not be clearly distinguishable from each other. Therefore, it is necessary to establish the minimum resolvable difference in wavelength ðDlÞmin ; that is, the minimum discernable wavelength difference. In order to define this first we normally define the closeness between the two peaks of the two circular fringes, as shown in Figure 6.17. Let P be the overlapping point. It is normally adopted to take this point which is not larger than the point yielding the full width at the half the maximum (FWHM) irradiance. This point can be found by considering the corresponding phase increment between two fringe maxima as shown in the figure. From the transmission equation considered in Equation 2.59, the FWHM DwFM satisfies DwFM 1 ¼ 2 sin21 pffiffi 2 F
320
Optomechatronics T 1.0
T 1.0
FWHM
P ∆jFM 2
∆j ∆jmin
(a) resolving power
wavelength lo – Dl order (m +1)
lo m
lo + Dl (m –1)
wavelength
(b) frequency spectrum range
FIGURE 6.17 Characteristics of the Fabry-Parot etalon.
For large F, this may be rewritten by 4 Dwmin . pffiffi F since Dwmin is nearly equal to DwFM. This value represents the smallest phase increment separating two resolvable fringes, and is related to equivalent minimum increments in wavelength ðDlÞmin : It can be shown that, if the resolving power is defined by R¼
l ðDlÞmin
then it is given by p pffiffi R¼m F 2 For small wavelength intervals, ðDlÞmin is given by: ðDlÞmin ¼
l2 pffiffi p Fd cos u
ð6:19Þ
Therefore, to increase resolving power, it is necessary to increase the order of fringe as well as the coefficient of finesse. The third characteristic of the Fabry-Perot etalon is free spectral range. This concept is important when the two wavelength components contained in the source become very different. Under this condition, the overlapped fringes will separate from each other. As the wave length difference increases, that is, when ^Dl gets large the (m)th order fringe for l0 will approach the (m þ 1)th and (m 2 1)th orders of fringe for the other neighboring wavelengths, l 2 Dl and l þ Dl, respectively. This separation range in wavelength is called the “free spectral range.” This can be easily
Basic Optomechatronic Functional Units
321
derived by ðDlÞfsr ¼
l m
and therefore ðDlÞfsr ¼
l2 2d
for normal incidence. We need to have this value as large as possible. One way to increase this value is to decrease the separation d, but this will decrease the resolving power, resulting in an increase in the ðDlÞmin as can be seen from Equation 6.19. Fiber Optic Sensors The characteristic of fiber optic sensors as distinct from other optical sensors is that they can easily interact with the mechanical elements in detecting motion. The interaction includes displacing one fiber relative to the other (shutter, grating), deforming a moving element with reflective surface placed in close proximity to fibers (Y probe), and the small deformation of a fiber due to the small movement of an element in contact with the fiber (micro bending). As summarized in Table 6.2, there are a variety of optical sensors operated based on this optomechatronic interaction. Most of the sensors utilize the signal modulation discussed in Chapter 5, which includes intensity modulation, wavelength modulation, polarization modulation, and so on. The intensity modulation methods have a disadvantage in that they are susceptible to variations in source intensity or other noise. The wave length modulation method, however, is not dependent upon the undesirable variations.
TABLE 6.2 Fiber Optic Sensors Operated via Optomechatronic Principle Basic Measuring Principle and Methods Modulation Type Intensity
Wavelength Polarization Phase
Method
Motion Provider
Cutting, reflective, attenuation
Shutter grating, mechanical motion, mechanical motion Mechanical resonator Gratings Gratings Mechanical motion Mechanical motion Mechanical motion
Resonator Referencing Digital encoding Spectrum change Polarization change Phase delay
Physical Parameter to be Measured Temperature, pressure, displacement, force Force, pressure acceleration Pressure Displacement Displacement pressure Stress, pressure Displacement
322
Optomechatronics
The intensity modulation principles can be found from a number of optical sensors. The shutter modulation is the simplest configuration of this kind – essentially an optical beam interruption device utilizing displacement of the shutter. For greater sensitivity, a pair of gratings can be situated between two fibers, where one of the grating is held fixed, while the other is moving due to an externally generated motion of mechanical elements. Deflection of a mechanical element (bimetallic strip) shown in Figure 6.18a utilizes the reflective property of strip. It consists of two optic fibers, one as an input, the other as an output. In effect, it utilizes change in reflectance due to the deformation of a mirror-like surface. When any movement of the strip occurs, this causes change in light intensity at the sensor output. The concept of microbending utilizes bending-induced attenuation in fiber optic bundles, as can be seen from Figure 6.18b. Corrugations on the element when displaced due to externally provided load generate a periodic distortion on the fiber. When this occurs, the light beam within the fiber moves orthogonally instead of reflecting, and refracts the fiber wall. The amount of light received at the output is found to be dependent on the corrugation spacing, the core radius of the fiber, and the difference in dielectric index between the core and clad of the fiber defined by ðnco 2 ncl Þ=ncl where nco and ncl are the refractive indices of the core and cladding, respectively. The fiber sensor employing the intensity modulation principle (cutting) shown in Figure 6.18c measures acoustic disturbance in the water. When a sound wave hits the diaphragm, the deformed diaphragm displaces the fiber on the left, thereby changing the amount of the light transmitted across the gap.
source mirror-like surface sensor
bimetallic strip
(a) reflection displacement
photodetector
light source gap
light optical fiber (b) microband force sensor
FIGURE 6.18 Intensity modulation optical sensors.
to photo detector
diaphragm sound wave
(c) optical fiber displacement
air water optic fiber
Basic Optomechatronic Functional Units
323 q
collimating lens input fibre
output fibre f
d
grating angular displacement sensor FIGURE 6.19 Wave length modulation-based optical sensor.
A typical sensor employing the wavelength modulation principle is shown in Figure 6.19. The figure shows a Littrow diffraction grating angular displacement sensor. It consists of a collimating lens and a diffraction grating composed of an array of grooves with spacing d when the grating is displaced by u: The sensor converts angular displacement, u into changes in wavelength of the input beam. Consider an input beam, with broad band radiation emerging from an input fiber located at the focal length f of the lens, is collimated into a parallel beam, and then incident on to the reflection grating at an angle u: As discussed in Chapter 2, the resulting diffraction pattern is that only light with wavelength l satisfying the following equation ml ¼ 2d sin u
m ¼ 1; 2…
ð6:20Þ
will be reflected back along its own path as indicated by an arrow. Since only the first-order reflected beam is dominant, the above relationship can be rewritten for m ¼ 1 as
u ¼ sin21
l 2d
ð6:21Þ
This relates reflected wavelength, l; to grating angle, u: This sensor has a measurement range between 208 – 608; which depends upon the geometry of grating and optical arrangement. The principle of the Fabry-Perrot interferometer discussed in Chapter 2, in the section “Interference” can be applied to measure the pressure of liquid or gas mediums. This interferometric sensing concept utilizes the variation of the distance between the two parallel plates, that occurs due to the movement of one plate relative to the other in response to external pressure change [23, 24]. Figure 6.20a shows the configuration of an interferometric Febry-Perrot pressure sensor. It consists of an F-P interferometer in which a cavity consists of an air gap between an optical fiber and a diaphragm, a multimode optical fiber carrying a single laser source light and a reflected light, and a thin silicon diaphragm capable of being responsive to the change
324
Optomechatronics glass tube
applied pressure
optical fiber
diaphragm (silicon)
internal mirror
phase change
(a) configuration of an optical pressure sensor
Pi Pr dielectric mirror
dielectric mirror
(b) Fabry-Perot interferometer
differential pressure (c) typical response of the pressure sensor
FIGURE 6.20 An optical fiber Fabry-Perot interferometer.
of external pressure. The interferometer has two mirrors coated with dielectric film. The separation distance ‘ forms the cavity length of the interferometer. When pressure is applied to the diaphragm, the diaphragm is deflected, resulting in a change in the optical length of the interferometer shown in Figure 6.20b. This change causes light reflection off two surfaces to bounce back and forth between mirrors and thus interfere with each other. The interference can be either destructive or constructive, as discussed earlier. When two mirrors are assumed to be lossless and their reflectance R is much less than ‘, the reflected optical power Pr is given by Pr ¼ R{1 þ cos w} Pi
ð6:22Þ
where w represents the round-trip phase-change in the interferometer, which is given by w ¼ 4pn‘=l: In the above, Pi is the optical power of the incident beam, n is the reflective index of the material between the two mirrors, l is the optical wave length. The relation indicates that the ratio of the reflected to incident optical power is a function of the reflectance of two mirrors and the round-trip phase difference which depends on the optical length difference between the two mirrors. Now let us consider when an external pressure is applied to the sensor, which results in the deflection of the diaphragm. In this case, the optical fiber will experience longitudinal strain along the fiber axis, which in turn shifts
Basic Optomechatronic Functional Units
325
supporting base
light
bimorph cantilever emitting fiber h
receiving fiber light
d
FIGURE 6.21 Piezoelectric bimorph optical fiber sensor.
round-trip phase difference expressed by
dw ¼ 0:78
4pnðd ‘Þ l
ð6:23Þ
where d ‘ is the change of the cavity length of the interferometer sensor. The factor 0.78 is a correction value to account for the strain optic effect. This phase change is directly measured by the interferometer. Figure 6.20c shows a typical output of a sensor having the cavity length 1 mm, fiber diameter 1.25 mm and the reflectance of the mirror 3.5%. In response to an applied external pressure, we can see that the phase shift of the sensor output is proportional to the applied pressure. The relationship shows strong nonlinearity as applied pressure increases. Combining an optical fiber with a bimorph transducer provides a good means of measuring physical quantities [32]. Precision measurement of high voltage of an electric power system is an example, whose basic structure is shown in Figure 6.21. The operating principle is based on an intensity modulation. The sensor is composed of an emitting fiber, a receiving fiber, and a piezoelectric bimorph transducer. The input fiber is fixed to a supporting plate, while the output (receiving) fiber is fixed to one end of the bimorph transducer. The transducer is composed of piezoelectric plates metalized along their thickness. When a high voltage is applied to the transducer, its free end is deflected by d in the z direction, as discussed in Chapter 4, in the section “Actuators,” which is given by
d¼
3d31 ‘2 V 2 h
ð6:24Þ
where d31 is the piezoelectric coefficient, V is the electric field, and ‘ and h are the geometric dimension of the cantilever. As a result of the displacement, the amount of optical power transmitted from the emitting fiber to the receiving fiber is reduced. When the measured electric field is AC, the bimorph beam will vibrate with the applied electric frequency. The natural frequency of the beam should be made much higher than that measured in order to avoid a resonance phenomenon. If the bimorph actuator of the size
326
Optomechatronics photodiode array
electronics
processor
light source
fibre-optic cable
scanner head
object (a) sensor configuration
(b) typical scanned images obtained by the optical scanner
FIGURE 6.22 Fibre-optic vision sensor.
3:5 mm £ 2:5 mm £ 0:6 mm having a free length of 28 mm, the resonance frequency is found to be 245 Hz. A visual sensor shown in Figure 6.22a whose motion is provided by a scanning unit, is a sensor embedded in a mechatronic unit, thus having a capability to measure an area range of interest [30]. It has two key components; coherent fiber-optic bundles for carrying light from an object to be imaged, and photodiodes to convert the light reflected from the measured object surface. The fiber optic cable can deliver light to illuminate the area of interest of the object. The scanner head has various fiber geometries for imaging, as well as illumination that can be used depending upon the geometry of object to be measured. A typical scanned image obtained for various objects under front lighting is shown in Figure 6.22b. The edges appear not to be sharp due to shadow effects and varying angles of reflectance at the edges.
Automatic Optical Focusing Autofocusing that automatically locates the focal position is a desirable attribute and an important concept in a number of optical systems. It produces clear, sharp images of objects by avoiding blurring of local imaging. For instance, in some types of microscopes, autofocusing makes possible the detection and identification of atoms. For example, a confocal microscope employs the confocal principle in which a point of light source is always in focus with a point inside the sample, using a pinhole placed in front of the object. This confocal system can effectively see a clear image inside thick samples due to this arrangement in which its autofocusing adopts a single point illumination. In this way, it produces a clear image
Basic Optomechatronic Functional Units
327
of objects by locally imaging one point at a time, as we shall discuss in Chapter 7, in the section “Atomic Force Microscope.” In optical hard disks, it enables the optical head to accurately read and write the data by optical means. And in a laser printer, it also helps print better quality images. There are several factors that need to be taken into account to achieve auto focusing: The first is to construct an appropriate optical system configuration. The second is to compose a feedback system that controls the lens position or the object position. The third is to measure the image and find a measure of defocus. All of these factors may differ from application to application. Therefore, an appropriate or optimal choice would be necessary when autofocusing is part of a designed system. In this section, we will deal with the optical system configuration, the lens position control, and sensing problem. Optical System Configuration There are two types of optical system arrangements for autofocusing, depending upon whether the optical system has pinholes or not. In confocal microscope systems, a typical arrangement is a pinhole in front of a detector. The role of the pinhole is that it blocks light from all parts of the object outside this focus. In the situation where there is a point source of light, a point focus
FIGURE 6.23 Basic configuration of the autofocusing system.
328
Optomechatronics
of light inside or at the surface of the object, and a pinhole detector, all three are confocal with each other. Some autofocusing systems do not employ this pinhole system. Figure 6.23 illustrates the basic configuration of the autofocusing system, and demonstrates its principle. When the object is located exactly in the focal plane of the objective, the focused light illuminates a single point on the object surface. The reflected beam spot is then exactly imaged on to a detector through a pinhole, which collects a large amount of the incident energy. A pinhole blocks light from all parts of the object outside this focus. In this situation, when the confocal system is located in the out-of-focus position as indicated in dotted lines, the image is focused at a position before or after the point detector. In these cases, the detector gets only a small amount of the incident energy — the larger the defocus distance, the weaker the signal strength. Therefore, the location of the objective lens system relative to an object under inspection is vital in determining the location of the imaged spot having high signal strength. This indicates that an accurate control of the system in the axial direction depending on the object shape is ultimately required, since a point image can be recorded only when the image is in the focal plane. In the case of a microscope, when this system is scanned over an interested region of the object, a map of 3D image information on the object can be accurately obtained. The brightness of the imaged spot partially depends on the amount of light gathered in by the objective lens of the autofocusing system. To describe this, the f-number is a useful parameter, when the object is located some distance away. However, when both the object and detector are located at a finite distance, as in the case of the confocal system, the numerical aperture denoted by NA is more useful. From the configuration shown in Figure 6.23, NA is expressed by NA ¼ nt sin ua
ð6:25Þ
where nt is the refractive index of the immersing medium adjacent to the objective lens of the confocal system, and ua is the half-angle of the maximum light cone picked up by the lens, depending upon the aperture of the lens. A large value of NA implies a large lens aperture. This is true when the object lens pupil is uniformly filled with light. However, when a small beam enters through it, we do not get the benefit. Therefore, in case of using a laser beam, we do not fill the lens pupil with the light, since its profile is generally Gaussian. Thus there is less light at the pupil edge. Optical Resolution Autofocusing directly concerns the axial resolution, but lateral resolution is also important as a performance factor. To this end we will briefly review the lateral resolution, and then the resolution in axial direction. Lateral resolution is defined by that in the plane of focus. As discussed in Chapter 2, in the section “Diffraction,” diffraction of light by an object and by
Basic Optomechatronic Functional Units
329
an objective lens determines image resolution. The lateral resolution is governed by diffraction, rather than limited by chromatic aberration, a spherical aberration. The image resolution for a given numerical aperture of the objective lens of an optical system is given by Dx ¼
0:61l NAobj
ð6:26Þ
where Dx is the minimum resolvable distance that the diffraction images of two adjacent points in the object can approach each other laterally before they merge. The above equation implies that the larger the numerical aperture, the higher the resolution. Axial Resolution Axial resolution is defined by the minimum resolvable distance that the diffraction image of two points in the object can approach each other along the optical axis before they merge. Another way of defining it is the radius of the first minimum of the diffraction image produced by a light source point of the object. As derived for the lateral resolution, the wave optics can be used to obtain the minimum resolvable length Dz in the optical axis (objective lens) direction. This shows that the image of a point source produced by a diffraction-limited optical system is also periodic above and below the focal plane along the axial direction, as in the case of the in-focus plane. The difference between the lateral and axial resolutions is that the diffraction in the case of axial resolution occurs in 3D space. The minimum resolvable length along the axis of the objective lens is approximately twice as large as that in the plane of focus. In more detail, the distance from the center of the diffraction image to the first minimum along the optical axis is approximately twice as far as it is to the first minimum in the focus plane. This implies that the axial resolving power is approximately one half of the lateral one. The axial resolution is given by Dz ¼
2ln ðNAobj Þ2
ð6:27Þ
which shows that the resolution is inversely proportional to the square of the numerical aperture (NAobj). It is noted that the depth of field of the autofocusing system is the depth of image measured along the optical axis, which appears to be in sharp focus by fine focus adjustment. It is approximately equal to the axial resolution, but is found to be affected by several factors, such as the geometric spreading of the light beam near the focal plane that arises from a single point in the object and the magnification of the objective lens. Due to the use of different criterion for “in focus,” a number of expressions for the depth of field are given in literature. One heuristic measure is to use a rule of thumb that the beam spot or image is regarded to be “in focus” if it is located in a region on either side of the focal (image) plane bounded by the depth of focus. The bound is given by ^ 10%
330
Optomechatronics
of the beam spot diameter, Dw at beam waist. This concept will be discussed in more detail later. Feedback Control of the Objective Lens In the previous section, we discussed briefly the resolution in both lateral and axial directions. Now, with such resolution concepts established, we return to the control of an objective lens in the axial direction, so that the maximum intensity of light can be achieved. The performance of this control is critical to the precision of most of the opto-mechatronic systems such as an autofocusing system, a confocal microscope, an optical pickup device, AFM, inspection and measurement devices, and so on. Referring to Figure 6.24, image detection usually requires either high precision motion control of the objective lens or an object, and thus needs a high precision actuator for such control action. In the open-loop configuration without feedback, the resolution can reach the order of nanometers, but the measurement range is limited by the linear range of the error signal. In addition, this configuration is susceptible and sensitive to external disturbances, such as small vibrations and the reflective light from the object surface. Therefore, an autofocusing system with feedback is ultimately necessary for accurate measurements. A typical autofocusing system, as illustrated in the figure, controls the motion of the objective lens in the direction of optical axis so that it always places a focal point at the image plane. It consists of an actuator, a half mirror plate, a laser diode, and a quadrant photo sensor. A laser beam emanating photo sensor
mirror plate
laser source
objective lens actuator
object surface FIGURE 6.24 Schematic view of the optical autofocusing system.
Basic Optomechatronic Functional Units
zf
controller
actuator driver
Gc (s)
Gu (s)
error +_
331 disturbance D(s) focusing system actuator + + Ga (s) Gp (s)
z
Gs (s) sensor FIGURE 6.25 A schematic of the autofocusing system.
the laser source passes through a polarized beam splitter, a 14 l wave plate, and an objective lens, and finally focuses on the object surface. The beam reflected from the surface travels along the original path, passing through the same splitter, and then, finally, projects on the photodiode. The output of the diode is fed back to the controller to compute the error signal, from which a control command signal is produced. This command signal drives the actuator so that the error may be eliminated as fast as possible. The mirror plate plays both the roles of beam splitter and astigmatic lens. The mirror plate as a beam splitter transmits the beam converging from the objective lens to the four quadrants photo detector. In addition, the tilted plate can generate astigmatism along its principle axis. If this feedback control system is constructed, then it can be drawn as shown in Figure 6.25. We will discuss these control elements one by one. There are many actuators to use, but consideration must be given to several factors, such as accuracy, size, and weight. Table 6.3 compares several characteristics of various linear motion actuators which are frequently used to control high precision motion for various applications. The table shows that PZT is desirable in terms of accuracy and response time, but it may not be desirable from the point of view of moving range. This indicates that the actuator needs to be chosen depending on specifications imposed by the application. For autofocusing applications, voice coil motors and piezo-electric transducers are popularly
TABLE 6.3 Characteristics of Linear Motion Actuators Linear Motion Actuator Servo motor þ ball screw VCM PZT Friction drive Electrostatic actuator
Displacement Range Mm–m Mm–cm #100 mm mm–m #500 mm
Accuracy 0.1 mm– 10 nm ,50 nm ˚ ,A ,10 nm ,10 nm
Response Time .100 ms ,1 ms up to , ms .100 ms ,0.1 ms
332
Optomechatronics
used due to relatively simple structure, fast response, light weight and high precision in comparison with other actuators. Here, we will consider a voice coil motor (VCM) for focusing purposes. Figure 6.26a illustrates a typical autofocusing system driven by a VCM; an objective lens is installed on the VCM and can freely move in a direction perpendicular to the focal plane. The motor is composed of an objective lens on a moving coil (bobbin), two magnets, and a spring-damper system that nullifies the effect of any external disturbance and also stabilizes the bobbin motion. In general, the actuator structure should satisfy the following mechanical requirements: (1) should be as small and light as possible. (2) should maintain the direction of laser beam during actuator motion. (3) should have high natural frequency of the moving bobbin. The magnets M1 and M2 generate magnetic field in the direction of the arrow, as indicated in the figure. When the bobbin is placed within the magnetic field with magnetic field density B, force F in upward vertical direction is generated according to Fleming’s left hand rule. The force is given by F ¼ nBi ‘c
ð6:28Þ
coil for focusing
laser
FL
z
B SN
FR
coil
lens
m k
BS
PD
yoke
B NS
magnet
b
LD M1
(a)
M2
an objective lens installed on a voice coil motor
+ V
_ Rc
Lc
+
_
VbL
–
(b)
electrical circuit of the VCM
FIGURE 6.26 A configuration of the confocal system.
+
VbR
i
Basic Optomechatronic Functional Units
333
where n is the number of coil turns, B is the magnetic flux density in tesla, i is the current in amperes, flowing in the direction perpendicular to the paper and ‘c is the coil effective length in meter. Figure 6.26b shows the electrical circuit of the VCM, which is written by L
di þ Ri ¼ V þ VbL þ VbR dt
ð6:29Þ
where L and R are the inductance and resistance of the coil, respectively, V is the electrical voltage applied to the coil and VbL and VbR are the back emf of the left and right hand side of the bobbin, respectively. The inductance L can be obtained by Faraday’s and Ampere’s laws, and is given by L¼
mn2 Ac ‘c
where Ac is the cross-sectional area of the coil, and m is the permeability. The resistance R is defined by R¼
r ‘c Ac
where r is the resistivity of the conductor. The back emf is given by VbL ¼ VbR ¼ 2nB‘c
dz dz ¼ 2kb dt dt
ð6:30Þ
where dz=dt is the velocity of the bobbin in an upward direction. Rewriting Equation 6.29 by use of Equation 6.30, we have L
di dz þ Ri þ 2kb ¼V dt dt
ð6:31Þ
The dynamic equation governing the vertical motion of the bobbin can be written by m
d2 z dz þb þ kz ¼ F dt dt2
ð6:32Þ
where m is the total mass of the bobbin and lens unit, and b and k are the damping coefficient and the stiffness of the bobbin, respectively. Laplace transforming of Equation 6.31 and Equation 6.32 and combining both, we can obtain the transfer function between vertical motion of the bobbin and electric input voltage to the coil. The transfer function is given by Gp ðsÞ ¼
ZðsÞ nB‘c ¼ VðsÞ Lms3 þ ðLb þ RmÞs2 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s þ Rk
ð6:33Þ
where Gp(s) here includes Ga(s). Let us consider when the above system is controlled by a feedback controller, which is shown in Figure 6.25. The objective of this control is to maintain a vertical displacement of the objective lens at the focal length. If the focal length is denoted by zf, then the controller should bring the system response always within some required
334
Optomechatronics
specifications. For simplicity, in Figure 6.25 we will assume D(s) ¼ 0, and neglect the dynamics of the sensor and driver, i.e., Gu ðsÞ ¼ 1 and Gs ðsÞ ¼ 1 or constant. If we use a PI controller, the controller transfer function is given by Gc ðsÞ ¼ kp 1 þ
ki s
From the block diagram, the overall closed loop transfer function is determined by G¼ ¼
Gc Gp ZðsÞ ¼ Zf ðsÞ 1 þ Gc Gp nB‘c ðkp s þ kp ki Þ Lms4 þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s2 þ ðRk þ nB‘c kp Þs þ nB‘c kp ki ð6:34Þ
The parameter used for the simulation is listed in Table 6.4. In order to investigate the effect of the controller parameter on the response, the system is simulated for various controller gain parameters, using the Simulink model. Figure 6.27 shows the system responses to a desired focal length of the objective lens, zf ¼ 1 mm. For a fixed integral gain, ki ¼ 55, we can see from Figure 6.27a that, as kp increases, the speed of the response increases, but the responses have overshoot. From Figure 6.27b, similar trends can be found with the increase of integral gain for a fixed proportional gain. When the two gains are chosen as, kp ¼ 500, ki ¼ 700, we see that the response amplitude increases with time, exhibiting instability of the lens motion. This leads us to analyze the stability of the control system. Let us consider the stability of the system, and determine the range of stability for various gain parameters. The characteristic equation of the
TABLE 6.4 The Parameters of the Confocal System Used for Simulation Parameters
Symbol
Unit
Value
Resistance Inductance Total mass bobbin and lens Damping coefficient Stiffness Magnetic flux density Number of coil turns Coil effective
R L m b k B n
V H kg N sec/m N/m wb/m2 Turn m
7.5 0.144 £ 1023 0.49 £ 1023 0.175 13.2264 0.454 123 0.01
‘c
Basic Optomechatronic Functional Units 1.2 lens displacement (z), mm
lens displacement (z), mm
1.2 1.0 kp=250 kp=500 kp=750
0.8 0.6 0.4 0.2 0.0
335
0
10
20 30 time (t ), ms
40
50
(a) the effect of proportional gain, kp ; ki = 55
lens displacement (z), mm
8
(c)
1.0 ki=35 ki=55 ki=75
0.8 0.6 0.4 0.2 0.0
0
10
20 30 time (t ), ms
40
50
(b) the effect of integral gain, ki ; kp = 500
kp=500, ki=750
6 4 2 0 −2 −4 −6 −8
0
100
200 300 time (t ), ms
400
500
the unstable response; kp = 500, ki = 750
FIGURE 6.27 The controlled responses of the autofocusing system.
transfer function is given by Lms4 þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s2 þ ðRk þ nB‘c kp Þs þ nB‘c kp ki ¼ 0 When we use the parameters in Table 6.4, the above equation can be written as 7:056 £ 1028 s4 þ 0:0037s3 þ 1:9381s2 þ ð99:198 þ 0:5584kp Þs þ 0:5584kp ki ¼ 0 ð6:35Þ According to the Routh stability criterion discussed in Appendix A3, we can determine the stability condition for the gains kp and ki from Equation 6.35 in the following manner: (1) Since all the coefficients of the characteristic equation should be greater than zero, this yields 99:198 þ 0:5584 kp . 0; kp ki . 0: The above leads to 2177:6468 , kp , 0; ki , 0 or kp . 0; ki . 0 From these conditions, we have kp . 0; ki . 0:
336
Optomechatronics
(2) Arranging the coefficients of the polynomial in rows and columns according to the following pattern yields s4
7:056 £ 1028
1:9381
0:5584kp ki
s3
0:0037
99:198 þ 0:5584kp
0
s2
A1
0:5584kp ki
s1
A2
0
s0
A3
0
We have the following parameter condition for stability 7:056 £ 1028
1:9381
0:0037
99:198 þ 0:5584kp
A1 ¼ 2
0:0037
¼ 1:9381 2 1:907 £ 1025 ð99:198 þ 0:5584kp Þ . 0 0:0037 99:198 þ 0:5584kp A2 ¼ 2
A1
A1 A1
A3 ¼ 2
0:5584kp ki
A2
¼ 0:0011kp ki þ 99:198 þ 0:5584kp . 0
0:5584kp ki 0 A2
¼ 0:5584kp ki . 0
To satisfy all of the above conditions, kp and ki should meet the following condition for stability 0 , kp , 1:82 £ 105 ; 0 , ki ,
9:018 £ 104 þ 507:6364 kp
Using this condition, the range of the stability limit (shaded area) is drawn in Figure 6.28. We see that kp is bounded for all values of ki, while ki is not for small kp value. Effect of External Disturbance Thus far, we did not consider the effect of external disturbance caused by thermal dilation or mechanical vibration that may cause unwanted position deviation. When disturbance, D(s), comes into the VCR, the overall output displacement of the VCR becomes (from Figure 6.25) ZðsÞ ¼
Gc ðsÞGp ðsÞ 1 Zf ðsÞ þ DðsÞ 1 þ Gp ðsÞGc ðsÞ 1 þ Gp ðsÞGc ðsÞ
ð6:36Þ
Basic Optomechatronic Functional Units
337
ki
750 507.64
55.0 0 500
1.82×105
kp
FIGURE 6.28 The area formed by kp and ki for system stability.
The displacement deviation DZðsÞ due to the external disturbance is given as DZðsÞ ¼
DðsÞ 1 þ Gp ðsÞGc ðsÞ
ð6:37Þ
It can be seen that the deviation can be remarkably diminished by a factor of 1=1 þ Gp ðsÞGc ðsÞ compared to the case without feedback. This observation implies that the open loop transfer function should be sufficiently larger than unity in order to minimize the displacement deviation. To investigate the effect of the feedback, let us assume that the disturbance DðsÞ is a step input type, i.e., DðsÞ ¼ 0:1=s Plugging this into Equation 6.37, we have 0:1 s DZðsÞ ¼ 1 þ Gp ðsÞGc ðsÞ For proportional controller, Gc ðsÞ ¼ kp ; we have 0:1 {Lms3 þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnBlc Þ2 }s þ Rk} s DZðsÞ ¼ Lms3 þ ðLb þ RmÞs2 þ {Lk þ Rb þ 2ðnBlc Þ2 }s þ ðRk þ kp nBlc Þ From the final value theorem, the steady state error becomes lim sDZðsÞ ¼
Ds!0
0:1Rk Rk þ nB‘c kp
The results show that in order to reduce the steady state error, kp should be increased as large as possible, but the overshoot and stability must also
338
Optomechatronics
be taken into account. The case of using PI control yields DZðsÞ ¼
0:1{Lms3 þ ðLb þ RmÞs2 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s þ Rk} Lms þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s2 þ ðRk þ nB‘c kp Þs þ nB‘c kp ki 4
Appling the final value theorem again to the above equation, we have lim sDZðsÞ ¼ 0 s!0
which indicates that the integral control action eliminates the steady state error due to the step disturbance. Focus Measurement Accurate measurement of an optical head position (optical information detection lens system) is critical to its precision control. In the previous control system, it is assumed that the instantaneous location of the lens can be fed back to the controller for a feedback signal. Since, in most cases, focal position is not exactly known a priori, finding the error signal may often cause some difficulty. An appropriate criterion that can be used is the degree of defocus (or degree of focus) based on visual image measurement. This method, however, may not be fast enough to be used for precise control of the lens position, because most autofocusing applications, including optical disks, require very fast response for real-time control. This means that the calculation of the focus measure or error signal should be made well within the allowable control time step. A variety of sensors and the corresponding methods have been developed to detect focusing error. Here, we will introduce the astigmatism method, which employs a quadrant photo diode sensor as illustrated in Figure 6.29. The principle of a primary aberration known as astigmatism has been already discussed in the section “Aberration“, in Chapter 2. When at the instant an object plane is moved inside or outside the focal plane of the objective lens, light projected on the photo detector will produce an elliptical pattern shown in plane 1 or plane 3. When the object is exactly at the focal plane, it will have a circular shape shown in plane 2. In more detail, when an object is placed at the focal plane of the objective lens, the beam will scatter, symmetrically yielding a dark circle with a radius R as an output of the photo sensor, shown in the upper figure of the sensor output. As opposed to this, when the objective lens is out of focus, this causes the intensity pattern on the detector to become elliptical, as indicated in the figure. When the lens approaches toward the surface, the output will become elliptic with the major axis in the vertical line, as shown in the middle figure. On the other hand, when the objective lens moves farther away from the object surface, the output will become elliptic again, with the major axis in the horizontal line, bottom figure. Referring to Figure 3.29b, the four quadrant sensor shown in the figure exactly captures this astigmatism phenomenon. When ia, ib, ic, and id represent the output of the four quadrants, respectively, as shown in the
Basic Optomechatronic Functional Units
339
FIGURE 6.29 A schematic view of images formed by an astigmatic lens.
figure, the out-of-focus displacement of the lens Dz can be computed by the following equation Dz ¼ ðia þ ic Þ 2 ðib þ id Þ
ð6:38Þ
This Dz represents the error between the desired z(t), and actual values of the focal length, zf, and is measured at every instant of time. The computed error is used to calculate the controller signal, as indicated in the Figure 6.25.
Acoustic-Opto Modulator When an acoustic wave propagates in a transparent solid block of a medium such as fused quartz, it produces a periodic modulation of the index of refraction via the elasto-optical effect. All acousto-optic (AO) devices utilize the interaction of acoustic and optical waves within the medium. The devices are connected to various laser systems for information handing, optical signal processing, display and spatial and temporal modulation of coherent light. The AO interaction provides four basic properties that are frequently used in various devices. They are classified as follows: (1) Deflection: angular deviation of the light beam which is proportional to the acoustic frequency. (2) Frequency shifting: frequency shift of the deflected beam which is equal to plus or minus the acoustic frequency. (3) Tunable wave length filtering: wave length selection from a large spectral
340
Optomechatronics
FIGURE 6.30 A schematic of acousto-optical modulator.
band source of the beam by changing the acoustic wave frequency. (4) Amplitude modulation: modulation of deflected beam intensity by varying acoustic power. In the AO cell, a piezoelectric transducer is vibrated when driven by a radio high-frequency RF signal. The vibration of the transducer generates an acoustic wave which passes acoustic waves through the quartz, as shown in Figure 6.30. The transducer is placed between two electrodes; the top electrode determines the active limits of the transducer, whereas the ground electrode is bonded to the crystal. The transducer thickness is usually selected to match the acoustic frequency to be generated. The shape of the electrode can be varied for impedance matching or to shape the acoustic wave, while the electrode length is chosen to yield the desired bandwidth and efficiency. An impedance matching circuit is necessary to adapt the AO cell to the impedance of the RF source (in general 50 V) in order to avoid power-returned loss. The RF frequency range covers 1 MHz to 1 GHz, and is band pass in nature. Actual products of AOD or AO modulator are shown in Figure 6.31. A variety of different AO materials are used depending on the wavelength, intensity and polarization of the beam, which in turn vary with the types of applications. For instance, tellurium oxide (TeO2) fused quartz or lithium niobate (Li Nb O3) is used for the visible and near-infra regions, while for the infrared region, germanium is frequently used. The material is antireflection-coated to reduce reflection from the optical surfaces. Table 6.5
Basic Optomechatronic Functional Units
341
FIGURE 6.31 Acousto-optical modulator products. Source: Brimrose Corporation of America, (www.brimrose.com).
show various properties of acousto-optic materials used for AOD. They include optimum optical range for AO applications, incident optical polarization refractive index at various light wavelengths, acoustic velocity, and so on. It is noted from the table that most of the materials are crystal or glass. Deflector Acousto-optical deflection utilizes an acousto-optic effect in which acoustooptic interaction provides a means of modifying the spatial position of an optical beam. This effect occurs when a variation in refractive index arises due to the propagation of an acoustic wave in a modulator medium, as shown in Figure 6.32. Here, as mentioned before, an acoustic wave is generated by electrically actuating mediums such as quartz. When a collimated beam with wavelength l is incident to a collective lens whose
TABLE 6.5 Various Properties of Acousto-Optic Materials Used for AOD
Material
Type
Ge Ge33Ass12Se55 TeO2 SiO2 (fused silica)
Crystal Glass Crystal Glass
Optimum Optical Range for AO Applications (mm)
Incident Optical Polarization
Acoustic Velocity (m/sec)
2.5 –11.0 1.1 –1.7 0.35–0.45 0.2 –2.2
Linear Unpolarized Linear– circular Linear
5500 2520 620 5960
342
Optomechatronics fc modulator medium
0-order undiffracted
Dw collimated beam
acoustic wave
λ
D
lens
Λ
piezoelectric transducer
1st order diffracted lens
FIGURE 6.32 Acousto-optical interaction.
focal length is fc, the collected beam encounters an acoustic wave having a wavelength, L. This is called an acousto-optic interaction. In order to make an effective interaction occur, it is necessary to position the modulator medium at the beam waist, Dw as shown in the figure. As a result of the interaction, a harmonic acoustic wave accompanies variation in the refractive index, which causes the input beam to deflect through a medium of alternating refractive index by the Bragg effect. Passing through this alternating medium has the same effect as passing through diffractive grating. Figure 6.33 illustrates in more detail how an optical beam can be deflected by this phenomenon. The figure shows rarefaction and compression of the medium, as a result of the interaction. By altering the amplitude and frequency of the acoustic wave, the spatial position, intensity, or frequency of an optical beam propagating through the acousto-optic medium can be varied. The acousto-optical modulator or deflection directly utilizes this principle. To derive the deflection angle, let us consider that a light beam of a single frequency is incident at an angle u to the plane of grating, which is the acoustic wave front. For acoustic waves of frequency fa, the wavelength L; and traveling at the speed of sound in a medium, va, the spacing between the planes of index of refraction variation is given by the relation va ¼ Lfa
ð6:39Þ
When the light beam interacts with this acoustically driven medium, it will be deflected to angle given by sin u ¼
ml 2L
ð6:40Þ
where m is the diffraction order, and l is the wave length of the light in air. As can be seen from the figure, use of the difference in beam path between the undiffracted and diffracted is made in the derivation of the above relation.
Basic Optomechatronic Functional Units
343 S m=1 first order
incident light beam θ
a = 2θ
Λ
m=0 zero order traveling acoustic waves
piezoelectric transducer FIGURE 6.33 Diffraction of optical beam in acoustic medium.
According to Heuggen’s principle, the first-order diffracted beam ðm ¼ 1Þ shows the strongest among the diffracted beams, having the largest efficiency. In this case, the angle between a first-order diffracted beam and the undiffracted beam is given by sin
a l lf ¼ a ¼ 2va 2 2L
ð6:41Þ
If a is assumed to be very small, the value is expressed by
a¼
lfa va
This indicates that a light beam can be diffracted by simply varying the frequency of acoustic wave. Acoustic-optical scanners use this relationship, in that the angle of deflection is in proportion to the sound frequency fa. For example, we consider the use of lithium niobate (LiNbO3) material as an AO medium. Suppose that we generate 250 MHz acoustic waves on the substrate whose acoustic velocity in the medium is 6.57 km/sec and refractive index is about 2.2. As a modulating light beam, a red-laser beam from He– Ne laser of wave length l ¼ 632:8 nm is used. Calculate the Bragg deflection angle, u, with respect to the plane of grating. The relation
344
Optomechatronics
in Equation 6.39 gives
L¼
va 6:57 £ 103 m=sec ¼ ¼ 3:285 £ 1025 m fa 2:00 £ 106 Hz
The Bragg deflection angle then becomes " # 29 m /2:2 21 ðl=nt Þ 21 632:8 £ 10 ¼ 0:58 a ¼ sin ¼ sin L 3:285 £ 1025 m This gives u ¼ a=2 ¼ 0:258: We can also compute the range of scan, if the maximum and minimum frequencies are known for scanning, respectively, max( fa) and min( fa). The swept angle range is obtained by the relation
a¼
l lDfa {maxð fa Þ 2 minð fa Þ} ¼ va va
ð6:42Þ
where Dfa denotes the acoustic frequency band width. Equation 6.42 provides the range of sound frequencies needed to produce a given scan angle. In acousto-optical deflectors, resolution and speed of the deflection are the most important parameters. The resolution is dependent on the beam divergence, and again, the given diffraction limited optics. It can be expressed by the number of resolvable spots, which is the scan angle range divided by the angular diffraction spread N¼
a D Dfa ¼ da va a g
where D is the beam diameter, and ag , is the factor related to the aperture geometry. As we shall see later, the term D=va is equal to the acoustic transient time across the optical aperture, tr : This indicates that the resolvable spot N is increased by decreasing the transient time tr for a given acoustic bandwidth Dfa : The increasing tr means that the speed of deflection decreases. Therefore, there is a tradeoff between resolution and speed of AO deflectors. When multiples of AO deflectors are combined, several applications can be considered for scanning. Figure 6.34 illustrates a typical application of multiple AO deflectors in a scanner. As mentioned before, the scanning area is limited by the scan angle, which is usually very small. When a helium – neon laser passes through a glass (tellurium oxide) interacting with acoustic wave having its velocity 0.62 km/sec and frequency 70 MHz, the scan range is roughly within 1.8 mm. This is a rather large scan angle that can be achieved by AOD, if Table 6.5 is examined. The AOD has several advantages in that (1) the scanning speed is much faster than that of a mechanical scanner, (2) it has no moving parts to wear out (3) random access is possible and (4) the accuracy and linearity is very high. On the other hand, it also has some disadvantages, such as small
Basic Optomechatronic Functional Units
345
mirror
laser source lens
X-Y deflector
lens
obeject
FIGURE 6.34 An AOD X-Y scanner.
deflection angle, low diffraction efficiency (about 50 , 70%), use of a single wavelength only due to dispersion of the refracting material, and required chromatic correction. The properties of AO, mechanical scanners and electro-optical scanners are compared in Table 6.6.
Frequency Shifter The acousto-optical device can also be used as a frequency shifter by modifying the frequency of the deflected light. The frequency of the diffracted beam is shifted as a result of the Doppler effect. When an observer and a light source have a relative velocity v, according to relativity theory, the
TABLE 6.6 Properties of Three Modulators Characteristics Transmittance Maximum frequency response (MHz) Rise time (nsec) Single wavelength Aperture Power (W)
Mechanical
Acousto-Optical
Electro-Optical
100 0.005 10000 No Large .10
80 50 25 Yes Small 5
70 100 10 Yes Small 20
346
Optomechatronics
observed frequency is given by 1 þ ðv=cÞ f ¼ f0 qffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 2 ðv2 =c2 Þ
ð6:43Þ
where f0 is the frequency if observer and source are at rest relative to each other. When v p c, the above equation may be simplified as Df v ¼ f0 c
ð6:44Þ
In this equation, the relative velocity is positive for relative motion in the same direction as each other, and negative for motion away from each other. In the acousto-optical modulator, we can easily show that the above equation can be rewritten as Df 2va sin u ¼ f0 c
ð6:45Þ
where f0 in this case is the frequency of the incident beam. In a simplified form, this can be written by Df0 ¼ ^fa
ð6:46Þ
As a consequence, the frequency of the diffracted light can be obtained by fd ¼ f0 ^ fa
ð6:47Þ
where fd is the frequency of the deflected beam. Figure 6.35 illustrates the above relations given in Equation 6.47. In the figure, the plus sign is for the case when the acoustic wave moves along the direction of the beam propagation, and the minus sign is taken when acoustic velocity moves against the propagation direction. Figure 6.36a shows the use of a prism to obtain the frequency shift twice that of the acoustic frequency. Figure 6.36b illustrates multiple use of AO cells with two different acoustic frequencies, which allows production of a variety of frequency shifts. The frequency shift is used for application to hetrodyne st
1 order f0 + fa
f0
0 order f0
q f0 fa RF frequencies (a) f = fo + fa FIGURE 6.35 Frequency shifts due to A-O interaction.
0 order
f0
f - fa fa RF frequencies
(b) f = fo - fa
st
1 order
Basic Optomechatronic Functional Units
f0 + 2fa
prism fa
f0
347 f0 + F f0 fo + fa1 + fa2
f0 + fa
f0
f0 + fa
fa1
fa RF frequencies (a) f = fo + 2fa
fa2 RF frequencies
(b) f = fo + fa1 + fa2
FIGURE 6.36 Frequency shifts for two different A-O-D arrangements.
detection, where precise phase information can be measured. A typical example is a laser Doppler velocimeter (or vibrometer). Tunable Wavelength Filtering According to the relation given in Equation 6.41, the angle of deflection of an acousto-optic deflector is proportional to the optical wavelength. Therefore, it is possible to extract a particular wavelength. This can be done by tuning the center wavelength of the filter passband, which in turn is changed by the frequency of the applied radio frequency signal. It can be shown that the increment in the spectral width can be deduced as Dl ¼
l va · D fa
ð6:48Þ
where D is the beam diameter. This indicates that low value of Dl gives high resolution, which is achieved by the increase of D. Efficiency of Modulation and Speed The important factors that affect the efficiency of the modulator are found to be the divergence angle ratio between the optical beam and acoustic beam, and the acoustic power. To preserve the interaction efficiency on all the bandwidths of the optical beam frequency, it is necessary to reach the Bragg conditions for all angles of the light beam. If dua and duo are denoted by the divergence angle of the acoustic beam and optical beam, respectively, it is known that the ratio duo =dua < 1 should be met in order to have the maximum interaction efficiency. The rise time of the AO modulator depends on the transit time of the acoustic wave propagation across the optical beam, and is proportional to the acoustic traveling time through laser beam. This is given by tr ¼ b
D va
ð6:49Þ
348
Optomechatronics
where b is the constant depending on the laser beam profile. We can see that the only parameter to minimize tr is to reduce the beam diameter. Consequently, it is necessary to focus the incident light beam on the acoustic wave in order to reduce the beam diameter, and thus reduce rise time. As we shall see in the case of the scanners to be discussed in the next section, reducing light beam diameter causes degradation of the scanning resolution. A typical rise time for a 1 mm diameter laser is found to be approximately 0.15 msec.
Optical Scanning Optical scanning is a method of changing the direction of a light beam in a scheduled manner. A variety of applications can be found from laser printers to materials processing. In terms of scanning geometry, there are three categories. These include objective scanning, pre-objective scanning, and post-objective scanning, which are shown in Figure 6.37. Objective scanning is to move a lens to generate a desired path of beam focus, as shown in Figure 6.37a. Therefore, this scanning method does not require a separate scanner, but a scanning lens. Suppose that a collimated beam located a distance x from the optical axis of the lens is directed parallel
scanning lens
beam focus path x
x f
(a) objective scanning
(b) pre-objective scanning scanning device (mirror)
ϕ
FIGURE 6.37 Types of scanning geometries.
scanned surface
ϕ
scanning device (mirror)
collimating objective beam lens
(c) post-objective scanning
objective lens
light beam
L
L
f
Basic Optomechatronic Functional Units
349
to the optical axis of the lens. The beam will then be focused at the focal point of the lens having focal length f. If we want to generate a circular beam path, as indicated in the figure, we need to rotate about the beam axis exactly the same as the trajectory of the beam, as long as the beam remains within the lens aperture. The pre-objective scanning shown in Figure 6.37b utilizes an objective lens in the optical path after the beam is deflected by the scanning mirror. As can be seen from the pre-objective configuration shown in the figure, the diameter of the focus lens must be much larger than the beam diameter in order to scan effectively. Also, an appropriate combination of lenses is needed to correct the curved scan field. It is noted that, in this configuration, the scan length is not linear with scan angle w: The postobjective configuration shown in Figure 6.37c is different from the preobjective in that it uses an objective lens in the optical path before the beam reaches the scanner. The beam enters the objective lens and then is scanned by a scanner represented by a rotating mirror. Due to this arrangement, the scan length (L) is linear with the scan angle, w: since L is equal to R where R is the scan radius. The disadvantage of this configuration is that the scan field is curved considerably more than that of the pre-objective when the focal length of the lens is fixed. To correct this curved focal plane, a dynamic focus system is used which employs a lens translator for focus correction in order to move an objective lens along its optical axis (as we shall see later). Basically, scanning in three dimensions is possible when scanners such as galvanometers and AO cells are appropriately arranged. There are several scanning devices that direct light onto objects with a preplanned rotation speed. There are three frequently used methods: (1) galvanometric scanning (2) acoustic-optical deflection method (3) spinning (polygon) mirror method. Galvanometric scanners use an electromagnetically-driven motor to steer a beam in a particular pattern over a certain angular range. AO scanners, as discussed in the previous section, and also known as acousto-optic beam deflectors, change the direction of the light by an acousto-optic effect — which occurs when a small variation in refractive index arises due to the acoustic wave propagation in the medium of collimated beam. Spinning mirrors have a number of plane-mirror facets parallel to and facing away from a rotation axis of a motor. They provide wide angular range, as much as 3608. Important factors affecting scanning performance include resolution, scan rates, range of scanning angle and pixel placement. In this section, we will consider galvanometers and polygon mirrors, which are widely used due to their high speed scanning. Galvanometer Galvanometric scanners use coated-plane mirrors to improve the reflex activity to deflect a laser beam. The scan head is operated via an optomechatronic principle, in that the optical device “mirror” configured with other optics is actuated by an electromagnetically driven motor.
350
Optomechatronics Y Y
X X
imaging objective (f - θ type) Y Y
scanners
X
scanners
imaging objective
X
lens translator
input beam
(a) pre-objective scan
(b) post-objective scan
FIGURE 6.38 Galvanometer scanning methods.
Typically, a pair of scanners is used to deflect the incident beam in two axes to create the x-y vector. Three-axis systems can be used when a z-axis stage for static field focusing, or a z-axis galvanometer translator for dynamic focusing on tilted, stepped, or curved object surfaces are needed. As already discussed in the introductory part of optical scanning, there are two commonly used scanning configurations in galvanometer; pre-objective, and post-objective scanning [10]. The pre-objective configuration, shown in Figure 6.38a, uses a flat-field scan lens called an f-u lens in the optical path after the beam is deflected by the scanning mirror. On the other hand, the post-objective configuration shown in Figure 6.38b uses an objective lens in the optical path before the beam reaches the mirror. As can be seen from
image plane field curvature
cylindrical lens
image plane
toroidal lens
field curvature optical axis
(a) field curvature formed by toroidal FIGURE 6.39 Correction of field curvature by f-u lens.
optical axis
(b) field curvature formed by cylindrical lens
Basic Optomechatronic Functional Units
351 y
X-Y pincushion
a +
x f - θ lenses motor
f-θ barrel
Y =
composite distortion
X motor collimated beam
FIGURE 6.40 Pre-objective scan distortion.
Figure 6.39, this lens consists of a positive toroidal lens and a negative cylindrical lens. The toroidal lens produces negative field curvature, whereas the cylindrical lens produces positive field curvature. Due to this integrated configuration, a flat scan field can be achieved. In the preobjective configuration, a beam expander designed to fill the aperture of the objective lens transmits paralleled beams into the mirror of the scanner. Then, the deflected beam goes to that of the y-axis. As mentioned earlier, both scanning methods have distortion problem. In a pre-objective scan system, there occurs a distortion called “pincushion” produced by the scan, if a single objective lens is used. To compensate for this error, a flat-field objective lens called an f-u lens is predominantly used, as shown in Figure 6.40. The figure indicates that the barrel distortion due to the f-u lens is added into the pincushion distortion, resulting in much smaller composite distortion. If the f-u lens is simplified by a single lens, it produces a simple relationship between scan angle, u, and spot location, y 0 y0 ¼ fu which indicates a linear relation for a given focal length. Proportionality between the scanning angles u and the image height y0 ensures proportionality between the angular velocity of the deflector and the scanning speed in the image plane. The correction discussed in the above example produces exactly this relationship and therefore a spot location in the scanned field proportional to the scan mirror angle. It is noted here that the entrance pupil of the f-u lens has to lie relatively far outside the region of the location of the deflecting unit. This provides sufficient distance needed for the movement of the deflecting unit, as well as for the entry of the beam bundle.
352
Optomechatronics d Dw = 2 r 0
scanner
z
objective lens curved focal plane
2∆ z
FIGURE 6.41 Fixed focus objective lens of a post-objective scanner.
The post-objective scan system has two types of scanning, which includes a fixed-focus objective lens system and a dynamic-focus objective lens system, as illustrated in Figure 6.41. A fixed system is composed of a focused beam expander and a scanning unit. In this case, the focal length is constant, and thus, the deflected beam behind the deflector unit is incident to a curved focal plane. Let us discuss the fixed-focus objective lens system first. In Gaussian beam optics, as discussed in Chapter 2, “Gaussian Beam Optics,” the beam spreading can be described by " rðzÞ ¼ r0 1 þ
lz pr 20
!2 #1=2 ð6:50Þ
where z is the distance propagated from the surface of the rotating mirror surface, l is the beam wave length, r0 is the radius of the 1=e2 irradiance contour at the surface where the wave front is flat, rðzÞ is the radius of the 1=e2 contour after the wave has propagated a distance z toward the image surface. Suppose that we choose the tolerance of the beam spot size in the scan fixed to be within 5% of r0, that is, rðzÞ ¼ 1:05r0 from the above equation. From Figure 6.41, it can be seen that Dz satisfying rðzÞ ¼ 1:05r0 is given by Dz ¼
0:32pr20 l
In order to keep the scan range in focus, it is necessary to keep it within the depth of focus defined by 2Dz: This means that the variation of focal plane due to the mirror rotation must be smaller than the depth of focus, which may be written by
d , 2Dz
ð6:51Þ
Basic Optomechatronic Functional Units
353
scanner input beam
motor
imaging objective lens
objective lens focal plane
lens translator
lens translator
FIGURE 6.42 Dynamic focus objective lens of a post-objective scanner.
where d is the deviation of the focal plane from the flat one that can be obtained without mirror rotation. Like pre-objective systems, there are distortions generated in the scan field: pincushion, and tangent error. To correct the error, electronic correction is normally used. When the scan field exceeds the depth of focus given in Equation 6.51, a dynamic focus system shown in Figure 6.42, called a lens translator, needs to be employed to achieve a flat-field focus [10]. In this dynamic focus objective system, an objective lens is placed on a slider and translated dynamically in order to relay and focus the beam on to the scan field at any instant. There are several configurations for achieving this objective. Figure 6.43 shows a basic, three thin-lens system for post-objective scanning. The small lens (element 1) located far left is a translating lens, which is actuated by a galvanometric actuator. Elements 2 and 3 can be either plano-convex or best-form signets oriented to minimize spherical aberrations. When a collimated beam enters a translating lens, as shown in the figure, the beam is expanded by a beam expander composed of elements 1 and 2 to fill the aperture of the scanning
D1 f1
z1
1
f2
f 3 D3
2
3
f2
f1
z3
f3
zc
z 2
1
f2
3
z3
FIGURE 6.43 Basic three-element system for dynamic focus scanning.
f3
354
Optomechatronics
head. The focal length of element 3 is designed by considering the optical path length from the element to the target object scanned. This element usually is placed near to the scan head entrance pupil. On the other hand, the focal length of element 2 needs to be determined by considering the combined focal length of elements 2 and 3, f23, so that the designed focal length can yield an appropriate optical objective function. It is noted from the figure that the position of the translator z is not linear with the image focus which is denoted by zc : The translator position, focus change, and focal lengths of the three lenses are related by the two lens equations f1 ¼ f2
D1 D3
1 1 1 ¼ þ f23 f2 f3
1 1 1 ¼ þ f23 f2 2 z f3 þ zc
ð6:52Þ
where f23 is the focal length of the combined lens system composed of lens 2 and lens 3, z is the translator position, and zc is the scan position to be corrected. In order to control the position z, zc must be known by some means. However, zc is not known a priori, and thus needs to be measured in an on-line manner by an appropriate sensor. This will be discussed in the autofocusing section (“Visual Auto-Focusing”). When the maximum range of zc and z are known, we can determine the focal length f2 from the above design considerations. According to Equation 6.52, the focal lengths of elements 1 and 2 can be determined by the following equations f1 ¼ f2 D1 =D3 ;
f2 ¼
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 ðz þ z2 þ 4f3 ðf3 þ zc Þz=zc Þ 2
ð6:53Þ
The variation of the focus can also be derived for a two x-y mirror flat field scanning [19]. Let us consider the case when the incident beam enters into the x-mirror then the y-mirror, and finally hits the surface of the image plane, as illustrated in Figure 6.44. A coordinate system ðx; yÞ is attached to the image plane to describe the image point resulting from the scan angles ux and uy of the two mirrors. The x-mirror is located at point Ox while y mirror is located at Oy : The distance between two mirrors is dxy and the image plane Si is dim distant from the y-mirror surface. Referring to the figure, we consider an arbitrary point Pm located at ðxim ; yim Þ and a point A located at ðO,yim Þ in the target image plane. Point Pim was originally located at the center O of the image plane when ux ¼ 0, uy ¼ 0: But, due to rotation ux and uy, this point reaches to a point ðxim ; yim Þ: From the geometry yim is given by yim ¼ dim tan uy We need to know the x-coordinate value, xim to determine the focus at point Pim : To obtain this value, let us project the image plane on to the virtual image plane of the y mirror located as indicated in the lower part of the figure. xim is then expressed by xim ¼ Ox Atan ux
Basic Optomechatronic Functional Units
Ωx
355 x-mirror
y
Ox
A
incident beam
θx
dxy θy
O
y-mirror Ωy
Oy
dim
Pim (xim , yim)
x Σi image plane
O
x
A y Pim (xim , yim)
FIGURE 6.44 Focal length variation due to mirror rotation.
But Ox A is equal to Ox A ¼ dxy þ Oy A where Oy A is given by qffiffiffiffiffiffiffiffiffiffiffiffi Oy A ¼ d2im þ y2im Therefore, xim is rewritten by qffiffiffiffiffiffiffiffiffiffiffiffi xim ¼ d2im þ y2im þ dxy tan ux
ð6:54Þ
Since Ox Pim is the focal length, we finally obtain the focus at the point ðxim ; yim Þ as sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffi 2 d2im þ y2im þ d2xy þ x2im fim ¼ From this equation, we can get the deviation of the focal length at ðxim ; yim Þ due to rotation which is given by Dfim ¼ fim 2 ðdxy þ dim Þ
ð6:55Þ
356
Optomechatronics
The Dfim expressed in Equation 6.55 is the amount of the correction needed to obtain the flat field scanning. It is noted that this focal length variation is a function of coordinate values in the image plane. In actual practice, drive motors and polygons produce combined errors in the directions of scanning, or at right angles to the scan. These errors result in various scanning anomalies, such as misallocation of initial scanning point, errors in scan line tracking, degraded resolution, and so on. The tracking errors are induced by errors in angular location of polygon facets, errors in facet flatness in being parallel with the axis of rotation of the polygon, errors in fastening of the polygon to the motor shaft, and wobbling. In order to eliminate such tracking errors, dynamic motion, as well as fabrication of the polygon, needs to be carefully examined before it is actually used for scanning. Feedback Control of Galvanometer The foregoing study on optical scanning was made based on the assumption that the dynamics of galvanometers do not influence the optical path of light beams. However, their dynamics normally affect the scanning performance when they are not properly designed. In this regard, we need to analyze the dynamic characteristics of galvanometers. Galvanometric scanners utilize an electromagnetically driven motor to steer a beam. Figure 6.45 illustrates schematically a galvanometric optical scanning device (one-axis). The galvanometer is essentially an electromechanical transducer containing an optical unit, including mirrors. It consists of an amateur (a moving coil assembly), a position transducer, a driving electronics composed of amplifier power supply and logics, a permanent magnet, and a rotating mirror. The mirror is coated with a thin film material to enhance its reflectivity, and mounted on the moving assembly of the motor. There are three types of armatures: moving coil, moving iron, and moving magnet. Here, the assembly of a moving coil is mounted in a magnetic field on a pivoted shaft, as shown in the figure, and its rotation is usually constrained. In general, the driver (amateur) needs to have the following ideal properties: (1) (2) (3) (4) (5)
a high torque to inertia ratio a linear relationship between torque, current, and angular position a dynamically balanced amateur immunity to all internal, external dynamic disturbances capability of low power consumption and good heat dissipation
In (1), the torque to inertia ratio affects the dynamic response of the galvanometer, with amateur of inertia driven by a motor capable of maximum torque. In (2), the linear relationship is static, and emphasizes a linearity between the above three parameters. To avoid any undesirable
Basic Optomechatronic Functional Units
Rg
357
Kt 2
ig
θ mirror
Rs
to the objective lens
+ vg
−
w N
S
permanent magnet ied viscous damping
b
laser
Kt 2
FIGURE 6.45 A schematic of galvanometric optical scanning device.
vibration, perfect balancing of the amateur is needed. Also, when the driver is dynamically perturbed, it should be insensitive to its disturbance effect. The moving coil drivers are simple in structure and low cost. Demand for higher performance requires low inertia, a well-balanced mirror and rotor construction, and good heat conduction. This moving, coil-based scanner is well suited to such applications as low-cost instruments, and vibrations and temperature protected environments. The moving iron drivers have a very high torque to inertia ratio, and therefore, are used in most high-speed scanners. They show nonlinearity between torque, current, and position relationships, but can be compensated by flux path compensation. Moving magnet drivers are high performance optical scanners, and meet all the requirements listed in the above examples to acceptable degrees for most applications. These are used for low and medium speed scanners. The galvanometric scanner is a feedback control system that controls its rotation angle in a desired fashion, based on the information of a position sensor. The role of the transducer is to convert the angular position of the rotor into an electrical signal needed for a control electronic part to generate a control command signal. To be ideal, a sensor needs to have the following characteristics; (1) high resolution. (2) a high bandwidth. (3) a high signal to noise ratio. (4) decoupled from the electromagnetic driver. (5) insensitive to radial motions of the rotor. There are several sensors, typical of which are capacitive sensor and optical interferometric encoders. The optical encoders have low inertia and
358
Optomechatronics
high performance, while the capacitance sensors are commonly used due to low cost and simplicity. Galvanometers can position beams over a wide range of angular displacement (0 , 808) at speeds ranging from 1 to 100 msec. The positioning accuracy can be achieved up to micro radians. Because the scanning speed is primarily dependent on the dynamic characteristics of the moving optical mirror, we will analyze its dynamic properties in brief. As can be seen from the figure, when there is an input voltage to the electrical field, a current-carrying conductor, a coil, experiences an electromagnetic force. This, in turn, generates a torque to the mechanical part (mirror and coil), causing it to rotate. Accordingly, the dynamic equations are composed of two governing equations, one for an electrical part and one for a mechanical part. The differential equation for the electrical field due to an input voltage is given by ig ðRs þ Rg Þ þ Lg
dig du þ BN‘b ¼ Vg dt dt
ð6:56Þ
where Vg is the excitation voltage, ig is the current, Rg is the resistance of the coil and Rs is the signal source resistance, Lg is the coil inductance, B is flux density, N is the number of turns on the coil, b is the width of the coil and ‘ is the coil length. The third term of the left-hand side of the equation denotes a back emf of the coil. The mechanical part consists of a mass (mirror and coils) whose motion is constrained by a tensional spring and built-in damping, either due to the reversed electromagnet effect alone, or combined with damping from enclosed viscous fluid. The driving torque is assumed proportional to the input current. Then, the dynamics of the moving optical mirror is governed by I
d2 u du ðB‘wÞ2 du þ b þ k u þ ¼ Tg t dt dt Rf dt2
ð6:57Þ
where u is the angular displacement of the mirror, I is the moment of inertia of the mass, b is the viscous damping coefficient, kt is the torsional spring constant, and Tg is the driving torque. The fourth term in the left hand side of the equation arises due to the fact that the frame on which the coil is wound is itself a conductor and, therefore, an electromagnetic torque 2Blwied is generated, where ied is the induced eddy current on the frame. If we neglect the effect of inductance within a certain operating frequency range of the galvanometer, we can easily have the resulting simplified equation expressed by d2 u du þ 2zvn þ v2n u ¼ Kv2n Vg 2 dt dt
ð6:58Þ
where K is the constant, vn is the natural frequency and z is the damping coefficient. They are given by K¼
BN‘w rad=V kt ðRs þ Rg Þ
Basic Optomechatronic Functional Units
359
sffiffiffiffi kt vn ¼ I
z¼
b þ ðB‘wÞ2 =Rf þ ðBN ‘wÞ2 =ðRs þ Rg Þ pffiffiffiffi 2 Ikt
where Rf is the resistance of the coil frame. It can be seen from the above equations that increase in sensitivity results when we increase B, N, ‘ and w or decrease kt, Rg and Rs : Also, decrease in kt results in reduced natural frequency, thus making speed of response lower. The damping coefficient shown in the above is composed of three factors: mechanical viscous damping b, electromagnetic damping proportional to du=dt; and back emf proportional to du=dt: Therefore, the influential parameters must be carefully designed in order to have a goal dynamic performance of the galvanometer. Inspection of Equation 6.58 shows that the response of the galvanometer has three different solutions, depending on z: As discussed in Chapter 4, “Dynamic Systems and Control,” these are: underdamped ðz , 1Þ; critical damped ðz ¼ 1Þ and overdamped ðz . 1Þ: Let us consider the underdamped case. When ðz , 1Þ, the dynamic response of the galvanometer is characterized by peak overshoot (Mp), rise time ðtp Þ and settling time (ts), as shown in Figure 6.46. The amount of overshoot depends on the damping ratio z of the galvanometer. When the damping ratio gets small, the overshoot gets large, and when large, the overshoot gets small. Certainly, we need to keep the overshoot as small as possible; however, for small damping, response time gets longer to arrive at steady state. This indicates that damping ratio should be designed in order that, within the permitted overshoot, the response be kept to a minimum. For example, if the accuracy required in a transient period is within ^ 5%, a damping ratio is chosen such that the percentages overshoot gives 1.05. Figure 6.47 illustrates this example: The response curve is tangent to the upper accuracy limit at point Pt, which is a peak. In this case, time required to 95% of the desired galvanometer rotation is given by 0.454 ðvn t=2pÞ: The damping ratio that meets this requirement is shown to be 0.690. When a galvanometer scans over a certain region in repeated fashion, a sinusoidal voltage must be given to it. In this case, ideally, the motion of the galvanometer needs to follow the input signal without distorting its amplitude and introducing phase shift. In reality, however, changes in amplitude and phase of the galvanometer motion will occur and will be a function of input frequency. The frequency response of the system is given for the magnitude ratio between the input and output
QðvÞ 1 ¼ GðvÞ ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi KVg ðvÞ 2 2 ½1 2 ðv=vn Þ þ 2½zðv=vn Þ 2
ð6:59aÞ
360
Optomechatronics q mirror
mirror rotation angle (q)
underdamped: z <1
±2, ±5%
Mp
over damped: z >1 critically damped: z = 1
0
tr
ts
time (wnt /2π)
FIGURE 6.46 Response of a galvanometer to a step input.
response amplitude
tangency point Pt
1.05 1.03 1.0 0.97 0.95
upper limit
P
lower limit
0
0.2
0.4
0.6
0.8
1.0
time (wnt /2π) FIGURE 6.47 Accuracy limits superimposed on the response curves of a galvanometer.
1.2
Basic Optomechatronic Functional Units
361
and for the phase of GðvÞ which is the phase difference between the input and output ( ) 2½zðv=vn Þ 21 /fðvÞ ¼ tan ð6:59bÞ 1 2 ðv=vn Þ2 where GðvÞ is the transfer function between the input KVg ðvÞ and the output Q: In Figure 6.48, the relationship is plotted in terms of frequency ratio v=vn : The ratio QðvÞ/KVg ðvÞ is a measure of the optical mirror dynamic response. Examination of the figure indicates that the amplitude ratio is reasonably constant over a frequency range from DC to approximately 40% of the undamped natural frequency, vn of the galvanometer. Also, the figure shows
amplitude ratio |G(w)|
2.5 ζ =0.2
2.0 1.5
accuracy limit ± 5%
0.4
1.0
0.7 1.0 2.0
0.5 4.0 20.0
0.0 0
1
2 3 frequency ratio (w/wn)
4
5
0
phase
G(w)
–45
ζ=0.2
–90
0.4 0.7
–135 20.0 –180
0
1
4.0
2.0
2 3 frequency ratio (w/wn)
FIGURE 6.48 Amplitude ratio and phase vs. frequency; GðvÞ:
1.0
4
5
362
Optomechatronics
that damping ratio in the range of about 50 to 90% should be used to have reasonable bandwidth. Usually, the frequency response characteristics shown here do not yield a satisfactory response if any modification or correction is not made. To obtain satisfactory results, we normally extend the range of acceptable frequency responses by feedback control gain adjustment. The assumptions for obtaining the dynamic model of the optical mirror are ideally made. In actual practice, the driving torque will be constant only when the coil turns are working in a magnetic field and are moving in a fixed direction relative to it. In addition, nonvisous damping is present to some extent in the form of ordinary spring hysteresis. As discussed in the above analysis, the system parameters such as damping and spring constant determine the characteristics of the galvanometer response. The open-loop system usually does not exhibit satisfactory response characteristics. To improve positioning accuracy and speed, a feedback control system can be considered, as shown in Figure 6.49. The control system is composed of a controller, power amp, a sensor and a mirror unit. We will consider time-domain response of the galvanometer response, but will not treat that of frequency domain. The performance of this position control system is investigated by using the dynamic equation presented in Equation 6.58. In order to simulate its controlled responses, we will use a PI controller and the parameters shown in Table 6.7. Figure 6.50a shows typical responses with various proportional gain parameters ðkp Þ. Those with integral gain ðki Þ fixed at zero are shown in Figure 6.50b. As we can see, the galvanometer response gets faster, but becomes oscillatory as kp increases. With inclusion of an integral gain, the response becomes faster, with large overshoot, but has no steady state error, as expected. To see the effect of external disturbance such as noise, a step noise of magnitude 0:01=s was included, as shown in the Simulink model in Figure 6.51a. The results are shown in Figure 6.51b. The responses of the P control alone to this input noise show a large error
desired scanning angle + – (qd)
controller
driver
FIGURE 6.49 Schematic of a galvanometer control system.
mirror galvanometer
actual angle (q)
Basic Optomechatronic Functional Units
363
TABLE 6.7 The Parameters Used for the Galvanometer Simulation Parameters
Symbol
Unit
Flux density Number of turns on coil Length of coil Breadth of coil Resistance of coil Resistance of signal source Moment of inertia of moving parts axis of rotation Torsional spring constant Viscous damping coefficient Resistance of frame (Ohm)
H N
Gauss
‘
m m V V g m2 N/m N m/(s radian) V
w Rg Rs I kt b Rf
Value 700 50 0.014 0.04 20 100 0.0015 100 0 1
Source: Ernest, O.D. Measurement Systems Application and Design, 4th Ed., McGraw-Hill, New York, 1990.
at steady state. However, the PI controller shows such error is eliminated due to the integral action at steady state. Polygonal Scanner This scanner is a simple mirror rotating on a motor shaft. The rotation of the mirror provides deflection of a ray bundle, which is then imaged by a lens and, finally, on to the image plane. Because the rotation can be made highspeed and repetitive, a polygonal scanner is used in a variety of applications in such areas as high-speed laser printers, facsimile, automated inspection displays, and so on. However, high-speed operations often cause wobbling and bearing wear, which affects the accuracy of scanning and the lifetime of the scanner. 1.4
1.2 1.0
kp=5.5
0.8
kp=2.5
0.6
kp=0.5
response (q), degree
response (q), degree
1.4
0.4 0.2 0.0 0.00
0.02
0.04
0.06
time (t ), sec (a) the effect of P gain; ki = 0 FIGURE 6.50 Responses of the galvanometer.
0.08
0.10
1.2 1.0
ki=50 ki=75 ki=100
0.8 0.6 0.4 0.2 0.0 0.00
0.02
0.04
0.06
time (t ), sec (b) the effect of I gain; kp= 0.5
0.08
0.10
364
Optomechatronics
Step Disturbance +−
++ Gain
Step
nu(s) de(s) PI Controller Transfer Fcn PI
Scope
(a) Simulink model 1.4
response (q), degree
1.2 kp=0.5 ki=50
1.0
kp=2.5 ki=0
0.8
kp=0.5 ki=0
0.6 0.4 0.2 0.0 0.00
0.02
0.04 0.06 time (t ), sec
0.08
0.10
(b) the effect of disturbance FIGURE 6.51 Responses of the galvanometer to a step disturbance.
Many of the important considerations for galvanometic scanning have been already treated in the earlier sections. Most of these considerations are also applied to polygonal scanning, for instance, pre-object and postobjective scanning methods, curved focal plane, and reduction of distortion by an f-u lens. Here, we will discuss resolution of scanning in some detail. There are two typical arrangements of collecting optical beams. In an optical arrangement shown in Figure 6.52a, the collimating beam impinges the scanner surface, and then converges at a focal point. The other arrangement, shown in Figure 6.52b, utilizes the collimating beam that illuminates the facets, then is reflected from the mirror and finally, focused on to an object surface. Both have the curved focal image surface. However, the collimated input beam has an advantage in that it illuminates the full area of facet, thus resulting in better resolution and minimization of modulation of the beam scanning angle.
Basic Optomechatronic Functional Units
365
polygon scanner
polygon scanner mirror
object Ω
Ω
lens
lens
object
(a) convergent input beam
(b) collimated input beam
FIGURE 6.52 Two different types of input beam for a polygon scanner.
The scanning resolution can be analyzed by applying Rayleigh criterion theory to derive the number of resolvable elements that can be produced by the scanner shown in Figure 6.53. In the figure, D is the beam width, or the width of the scanner mirror and Dwmin is the minimum resolvable deflection angle. If it is assumed that the aperture is uniformly illuminated, Equation 2.84 can be applied to obtain the minimum diffraction angle in the scan
Ω +
polygon mirror
D lens
scan plane FIGURE 6.53 Number of beam spot for a polygon mirror.
366
Optomechatronics
direction, which varies with the aperture shape. The angle is expressed by Dwmin ¼
1:22l D
ð6:60Þ
From this minimum resolvable deflection angle, we can now obtain the number of resolvable spots per scan which is given by w ð6:61Þ N¼ Dwmin where w is the total angle that can be scanned by polygonal mirror, and N is the number of spots. Let us consider how we can determine total angle w that a polygonal mirror can scan. As shown in the figure, the polygonal scanner has a multifaceted mirror which is driven by a driving servo motor as discussed for the case of galvanometers. The angle scanned by a polygonal mirror of M facets is written as
w¼
2ð360Þ 4p ðdegÞ or w ¼ ðradÞ M M
ð6:62Þ
The above result is due to the fact that M mirrors are on the circumference of a circle and that the scan angle is doubled by reflection. Equation 6.62 also indicates that the angle doubling affects the angular position of the focused spot at the focal plane and the scan length. Now, if this scan angle is related to a minimum resolvable angle for a circular aperture, the number of spot N can be obtained by N¼
w ¼ Dw
4p D 1:22 Ml
ð6:63Þ
This result implies that N is dependent upon the aperture diameter, the width of a facet D, the beam wavelength l and the scan angle w: The larger D, the number of beam spots gets larger, which indicates that the aperture needs to be fully illuminated over the whole facet. However, if M is increased within the diameter limit sufficient to reflect the beam effectively, the resolution would become low, because this reduces the available scan w; as we can see form Equation 6.63. For instance, suppose that we wish to scan a collimated He-Ne laser beam having 8 mm diameter with a polygon mirror. If M ¼ 16; we should have a spot number 5.1093 £ 103. If we increase the face number M ¼ 40 the spot number N will increase 2.0437 £ 103. At this point it is noteworthy to realize that, once the number of facets, M and scan speed, vs are given, the scanner rotational speed, V is expressed by
V¼
60 vs rpm M
where vs is given in terms of line scans/sec. This relation shows that, as M increases, the rotation speed can be made to decrease. However, increasing M, as mentioned in the above, results in degrading the scan resolution.
Basic Optomechatronic Functional Units
objective scan lens
Ω
367
image plane
polygon mirror
beam collimator
position sensor _
AO deflector
+
desired scan position
control gain
laser source FIGURE 6.54 Optical feedback control system utilizing an AO deflector.
Correcting Scan Errors There are two methods for compensating scan errors, active and passive. The active method is used in the case when errors are not consistently repeatable. As we have already discussed in brief previously, one active method utilizes a dynamic focus system for correcting scan field length. Another active method is to correct scan position error. In this case, active correction for beam angle (based upon the information of the position of the scanned beam) is made. On the contrary, the passive method utilizes additional optical components, such as cylindrical lenses in front of or after the polygon in the light path. Figure 6.54 illustrates an active microprocessor-based scheme to correct the scan error. It utilizes an acousto-optic deflector whose deflection angle is governed by acoustic wave frequency. It is placed in the beam path prior to a polygon mirror, and its deflection angle, which is the correction angle, is controlled based upon the measured information of the scanned beam position. This technique is effective to correct the facet errors in the range of arc minute, which is small. The tolerance of polygon fabrication is within this range.
Optical Switch In conventional mechanical fiber switches, ranges of macro-scale actuators have been adopted in order to control the switching action. Due to this actuator constraint, the structure becomes bulky, and the switching speed often becomes limited due to large mass. Recent advances
368
Optomechatronics mirrors
mirror input fiber
mirror 2
output fiber 1
4
3
output fiber input fiber
(a) on-off switch
(1 ×1 )switch
(b) cross-bar switch (2 × 2 switch)
(c) optical cross connector (n × n switch)
FIGURE 6.55 Configurations of micro-optical switches.
in micro-electro-mechanical systems (MEMS), however, have opened up many new possibilities for optomechatronic switches because movable structures, microactuators, and micro-optical elements can be monolithically integrated on the same substrate. The application of the MEMS is expected to improve the performance of switching devices. Figure 6.55 illustrates some of the architectures of micro-optomechatronic switches in terms of I/O port number. They include a simple on-off gate switch, a 2 £ 2 crossbar switch, and a n £ n optical cross connector. The switching system consists of a single mirror or multiple mirrors that steer the light beam, actuators that position the mirrors, collimating lenses, and input/output fibers through which light comes. There are basically two ways of achieving switching as illustrated in Figure 6.56. In the left-hand side of the figure (Figure 6.56a) switching is achieved by controlling the position of the lenses. Contrary to this, the architectures shown in Figure 6.56b use the control of the position of mirrors for switching, as indicated by dotted lines. With this switching configuration, many channels can be addressed, but here only two channels are drawn for simplicity. It is noted from this system that a telescope lens is normally employed to extend the beams on to the optical fibers. In any configuration, the motion of lens or mirror should be fast and smooth to avoid any undesirable phenomenon that might occur during optical signal transmission. A number of silicon-based actuators have been developed and implemented to realize movable microswitching structures. The commonly used actuators include a comb drive that produces electrostatic force between a pair of multi-fingered rods and combs, thermal actuators that utilize thermal expansion, and microvibrators that use impact actuation to obtain large motion from resonant structures having small-displacement stepper motors that achieve large force. We will discuss some of the microoptical switches that employ the above actuators.
Basic Optomechatronic Functional Units
369 input 1
lens
optical fiber
optical fiber
M1
input 2
mirror array M2 output 1
M4 M3
(a) lens-controlled optical switch
output 2
(b) mirror-controlled optical switch
FIGURE 6.56 Typical methods of achieving optical switching.
Thermally Actuated Mirror The actuation method by heating utilizes the expansion phenomenon when a material is heated. The strain due to the thermal expansion of a metal cantilevered structure clamped at one end is given by elementary mechanics 1x ¼
DL ¼ a DT L
ð6:64Þ
where 1x is the axial strain along x-direction, L is the original length of the material, DL is the elongated length, a is the thermal expansion coefficient and DT is the temperature difference in 8C. A variety of the thermal actuations utilizing this concept has been developed, and one of the popularly used configurations called bimorph or bimetal actuator is illustrated in Figure 6.57. It is composed of two base metals, the siliconoxide in the top having the thermal expansion coefficient a ¼ 2:3 £ 1026 =8C and the aluminum in the bottom having the coefficient a ¼ 25 £ 1026 =8C: Due to this significant difference in a, the heated arm of the aluminum is expanded approximately 10 times larger than that expanded by silicon. This length difference causes the arm to move in an upward motion as indicated in the figure, which can be transformed into lateral movement. The thermal actuator is typically composed of thin-film heaters and four counter parts of bimetal layers. It needs to be fabricated on a planar wafer surface before etching the final micromechanical structures. Therefore, an additional process of depositing a thin-film heater material is required in addition to bulk silicon micromachining. The actuators are found to produce large displacement and high force. A typical application of this actuator to an optical fiber switch is a thermally actuated bistable fiber switch, the top view of which is shown in Figure 6.58 [20]. It consists of two main actuators, one actuator for switching fiber and one actuator for optical fiber clamping, which is not shown here. The former is designed to move horizontally to displace an optical fiber, and the latter moves vertically to open and close the latching mechanism. Let us look into the switching mechanism in more detail by using a schematic shown in the figure. The actuator for switching action is a
370
Optomechatronics SiO2 Al (a) before heat applied
SiO2 Al
(b) after heat applied FIGURE 6.57 Thermal expansion of a bimorph.
U-shaped cantilever composed of two arms heated separately with thin-film heaters. The heat is generated in thin-film heaters on top of each cantilever arm. Due to the heated arm expansion, the upper arm causes the whole switching unit in the fiber to be switched from output fiber a to fiber b. The angular displacement of the actuator-causing switching motion can be obtained by a simple geometric relation shown in the figure tan u ¼
DL w
where w is the width of the arm and DL is the elongated length of the arm. If we assume a small angle and combine this equation with Equation 6.64, then, the resulting equation is written by
u¼
aLDT w
The angular displacement that can be achieved by this thermal actuator for a temperature increase can be easily calculated by the above equation. when U-shaped cantilever heated θ thin film heater
w
a
thin film heater
b L
input optical fiber
output optical fibers
FIGURE 6.58 Thermally actuated by optical fiber switch. Source: Hoffmann, M., et al. IEEE Journal on Selected Topics in Quantum Electronics, 5:1, 46–51, 1999 q2005 IEEE.
Basic Optomechatronic Functional Units
371
To illustrate this, let us calculate the angular displacement u for the Al-SiO2 bimorph. If DT is 5008C and the geometry for the actuator is given by L ¼ 200 mm, w ¼ 20 mm, we have
u¼
0:6 £ 1026 £ 200 £ 1026 £ 500 ¼ 0:003 rad ¼ 0:178 20 £ 1026
Depending upon the application, the shape, geometry, and dimension of the actuator can be designed differently. Electrostatically Actuated Mirror Control The comb drive mechanism has wide applications in actuating optical switches, and utilizes the electrostatic actuator. Figure 6.59a illustrates two directional forces that can be actuated by the comb drive mechanism; (1) comb driving horizontal force (2) gap closing force. The comb driving horizontal force per gap shown in Figure 6.59c can be derived by using Equation 4.51 and written as Fx ¼
10 V 2 w 2z
ð6:65Þ
where 10 is the permittivity of the air, V is the input voltage applied to the set of the plates, x is the overlap distance whose coordinate is shown in the figure, and w is the width of the comb structure. The gap closing force per
w V
V
z
z x
(a) comb driving force
(b) gap closing force
mirror
movable
mirror movable
fixed (anchored)
(c) comb driving actuator
FIGURE 6.59 Comb driving vs. gap closing.
(d) gap closing actuator
372
Optomechatronics
gap is illustrated in Figure 6.59b. In a similar way, as derived in the case of comb driving force, the force can be derived as Fz ¼ 2
10 V 2 w 2z2
ð6:66Þ
Note that the gap closing force increases drastically as gap, z, decreases. Equation 6.65 and Equation 6.66 imply that we can compute the input voltage V in order to obtain the desired gap zd for a given comb structure of width w. Ideally, this may be true, as long as the above theoretical relations hold. In case these relationships do not hold, a feedback control system may be necessary to obtain a desired force or a desired displacement of the comb drive. Figure 6.60 illustrates an example to help us understand a problem involved with a control input voltage generation for a horizontal comb drive system. Its structure is firmly anchored to the ground at its one end, and at the other end through four beam springs whose material have Young’s modulus E and Poisson’s ratio m: Each gap is kept constant at z0 , has width w and operates in the air. If the drive moves in the horizontal direction by x0 from an equilibrium state, the control input voltage V can be computed in the following procedure. Using Equation 6.65, we have the driving force for
_
V
+
x
k0
k0
k0
k0
h zo zo
spring comb drive
(a) comb drive structure
(c) comb drive dimension FIGURE 6.60 Horizontal comb drive mechanism.
L
(b) spring dimension
Fx
Basic Optomechatronic Functional Units
373
all gaps Fx ¼
710 wV 2 z0
ð6:67Þ
where Fx is the total force generated in the x direction. Due to the movement of the comb drive, the spring force acting in the x direction is given by Fx ¼ kx0
ð6:68Þ
where k is the equivalent total spring constant of the spring structure shown in the figure. The cantilever spring constant can be replaced by an equivalent spring constant, and can be derived by considering four cantilevered beams whose dimension is given in the figure. k ¼ k0 ¼
E wh3 1 2 m2 4L3
ð6:69Þ
where k0 is the equivalent spring constant of each plate. Note that this spring force must always be equated by the driving force Fx to stay in equilibrium. Equating Equation 6.68 and Equation 6.69, and solving for V, we have sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Eh3 x0 z0 V¼ 2810 ð1 2 m2 ÞL3 The above relation implies that the voltage input is a complex function of the comb structure geometry, involving a number of parameters. Roughly speaking, it is in proportion to the gap z0 and the overlap x0 and inversely in proportion to the beam spring length L3 : A typical application of this comb drive can be seen in Figure 6.55b, where a mirror is actuated by an electrostatic comb drive actuator in order to accurately steer the light to a desired fiber. The configuration describes a 2 £ 2 switch which uses a double side reflecting vertical mirror. The four fibers are placed into grooves at 908, which are not shown here. When the mirror is out of the optical path, the light travels in a straight path. On the other hand, if it is in the cross state (that is, in the optical path) it is reflected by the mirror, creating two light paths, one from the fiber no.1 to no.4, the other from the fiber no.2 to no.3. It is noted that switching occurs due to the linear motion of the mirror as indicated by the arrow. As mentioned earlier, inserting and removing the mirror in the coupling gap needs to be made within the desirable switching time and with minimum switching loss. Also, it should not yield any jittering motion of the optical switch. Another application of the electrostatic drive is scratch drive actuator (SDA), whose schematic view is described in Figure 6.61a [4]. The actuator consists of three parts: plate, bushing, and substrate electrode. In essence, the plate has a bushing along one side, and forms a capacitance-like structure with the substrate. When a voltage is applied across the capacitance, an electrostatic force can be generated on the plate as given by Equation 6.66. Therefore, when a square pulse is applied periodically between the plate and
374
Optomechatronics plate bushing
(a) V
insulator
substrate
(b)
(c)
(d) dx FIGURE 6.61 A sequence of the motion of the scratch drive actuator (SDA). Source: Akiyama, T. et al. Journal of microelectromechanical systems, 6:1, 10–17, 1997. q2005 IEEE.
substrate, a periodic electrostatic force is generated which results in a forward step motion of the plate, in the substrate plane as shown in the figure. The detailed sequence of its operation is given as follows: When a square pulse starts to be applied between the plate and substrate, the plate is pulled down toward the substrate. Due to the bushing at the front, deformation of the plate is built, as shown in Figure 6.61b. It is noted that in this deformation, near the region of the bushing, the plate deforms as a triangular shape due to the pulling-down motion of the rest of the plate. As the pulse falls, the stored energy retaining the deformed plate starts to be released and the plate begins to regain its original shape as indicated in Figure 6.61c. During this restoring stage of the plate, its rear part moves forward as the bushing remains in contact with the insulator. When the pulse is applied again, the rear part of the plate quickly comes into contact with the insulator, since this part has a smaller separation distance from the insulator. Due to this deformation of the rear part, the front bushing is pushed forward and slides on the surface like a worm. When the voltage is removed, the plate and bushing return to their original position, but translate a small distance, denoted by dx in Figure 6.61d. This completes one cycle motion, dx, in the figure. Repeating this motion due to the applied pulse, the plate can move continuously in a substrate plane. The configuration and velocity of the SDA motion is found to be dependent on the frequency of the applied pulses. Also, the force magnitude
0
375
applied voltage V1 > V2
applied voltage V1 > V2 force (F), μN
velocity (v ), μm/s
Basic Optomechatronic Functional Units
V1 V2
V2
0
frequency (fd), kHz
(a) the effect of frequency on velocity
V1
number of plate (N)
(b) the effect of number of plate on force
FIGURE 6.62 Velocity and force characteristics of a SDA actuator.
is found to be dependent on the number of plates. Figure 6.62 shows the characteristics of their relationship. As might be expected, the higher the driving frequency, the greater the velocity. It also shows that the velocity increases with the voltage applied, since a higher electrostatic force yields a larger pulling force, resulting in a larger step size. Force is found to increase with the number of plates, whose trend depends also on the applied voltage. Lens Controlled Switching A nonblocking n £ n micromechanical lens scanner for optical switching which employs control of the lens position shown in Figure 6.56a is schematically illustrated in Figure 6.63 [34]. It is composed of several tens of input and output fiber ports configured in an x-y stage. A refractive or diffractive microlens located in front of each fiber manipulates the light beam, thus sending the collimated and steered beam into a telescope which magnifies the scan angle. The resulting beam is finally transmitted to an
telescope 1
optical fiber matrix (input fiber port)
y
telescope 2
optical fiber matrix (output fiber port)
x z
2D beam switching device FIGURE 6.63 Non-blocking optical fiber matrix switch. Source: Hiroshi T., et al. MOEMS 99, pp. 165 –170, 1999.
2D beam switching device
376
Optomechatronics f
micro lens f
collimated beam scan angle ∆y/ f
collimated beam ∆y
optical fiber optical fiber actuator (y)
micro lens
(a) unchanged lens position
(b) changed lens position
FIGURE 6.64 Directing beam by lens motion control. Source: Hiroshi, T., et al. MOEMS 99, pp. 165– 170, 1999.
output fiber. Each lens is controlled by an actuator, depending on the beam switching requirement. The role of the microlens is to collimate the beam when the input fiber is placed at the focal length of the lens, as indicated in Figure 6.64a, and then to divert the beam to a certain desired angle, depending on the displacement of the lens in a lateral direction, as shown in Figure 6.64b. The figures show how the beam is collimated by the lens having a numerical aperture NA when an input fiber is located at the focal length, f. When this lens is pushed upward by Dy, the lateral lens motion bends the ray with an angle that satisfies the following condition
u ¼ sin21 ðDy=f Þ This small displacement of the lens is produced by a microactuator such as the electrostatic comb drive discussed previously. In order to obtain such necessary lens motion, a feedback control can be employed for the mirror switching. This one-dimensional lens motion control concept can be extended into two dimensions, a two-dimensional switching can also be achieved by actuating the lens in the x and y directions. Because this nonblocking matrix switch involves a n £ n switching, it requires a large scan angle which can be achieved by using a lens with a short focal length and a microactuator’s large travel distance. However, small focal length causes large spherical aberration. In addition, an electrostatic comb drive mechanism is limited to about 10 mm in traveling distance. Due to this reason, telescopes are used to widen the scan angle so that they can cover the region of the output port. We have seen several cases of mirror or lens control for switching action. In many cases of switching mirror or lens, there may be a need to accurately position mirrors or lenses to achieve effectiveness of light beam steering. To make such a precise control of the mirror position, a position control system can be built as shown in Figure 6.65. The control objective is to generate force, which can position it at a desired position xd : In the figure, V, is the control
Basic Optomechatronic Functional Units
377 x mirror
desired position
xd
+ −
controller
V
mirror position k
x
comb drive
sensor
FIGURE 6.65 Control of the position of a comb drive.
input voltage to be applied to the comb drive actuator. The performance of the control may be dependent on how reliably and accurately the displacement can be measured in a real time manner, since the amount of displacement to be measured is extremely small. To fit this requirement, a noncontacting optical measurement method may be conceivable.
Zoom Control Maintaining the quality of the image produced by a lens throughout its zoom range is the purpose of the zooming control. A zoom lens can achieve this objective. It is an optical device which provides variable magnification without moving the image out of focus, by varying its focal length. In other words, the focal length can be varied continuously along the optical axis, the image position being maintained in a fixed plane by either optical or mechanical means. This can be done by adding another lens that increases the size of the image by apparently increasing the focal length of the original lens. If the focal length is varied, but the image is not maintained, the system is called varifocal. The rate and direction of the lens motion are governed by in-out cams or slots cut into a rotary cylinder. Zooming Principle The zooming range of lenses is measured in its own X factor, called “zoom ratio.” It is given by X¼
longest focal length shortest focal length
For example, 35 to 70 mm is denoted as 2X and 80 , 200 mm is 2.5X. This is an indicative index that describes the power of zoom to increase the linear
378
Optomechatronics object
lens image screen
f
(a)
lens
Initial position
1.5 f
(b)
location P
FIGURE 6.66 Zooming concept by using a single lens.
dimensions of the image it produces. With a 2X zoom lens, the magnification of its image will be two times taller and two times wider than at its least magnification. The added lens is called a magnifier, and changes the magnifying effect with its distance from the original lens. To illustrate how the zooming principle discussed in the above example, let us consider a single lens arrangement shown in Figure 6.66a. When this lens with the focal length f moves forward to a location P at which the image distance is 1.5 f, the question is how large the magnification of the image will be. Assume that object distance is much larger than the image distance. To answer this question, we can draw the ray path through the pin-hole model of the thin lens as shown in Figure 6.66b. The important results we have here are: (1) the image obtained becomes enlarged by moving the lens by a factor of 1.5f/f ¼ 1.5, but only a portion of the object are seen and (2) the obtained image becomes somehow blurred due to the displacement of the lens, which changes the distance from the image plane. In order to have the whole object magnified and in focus, we need an optical system exhibiting autofocusing as well as a variable focal length: The longer the focal length, the larger the magnification factor becomes. Therefore, it is necessary to have a compound lens system at least more than two when an appropriate magnification is required. The procedure for zooming and autofocusing is explained in the flow chart shown in Figure 6.67. To explain the optical zooming principle in more detail, let us consider the system composed of two lenses, as shown in Figure 6.68. Suppose that the positions of two lenses L1, L2 in case (a) yield a focus point F at an image plane. If we move L2 (negative lens) backward, i.e., toward the
Basic Optomechatronic Functional Units
379
image acquisition blurred image autofocusing
fine image magnification ratio setting
lens position control
final focus adjustment
enlarged image
FIGURE 6.67 The zooming procedure.
image (focal) plane in case (b), the resulting focal length becomes shorter than the original, which means that it shifts forward, being deviated from the image plane. It is therefore necessary to make the shifted focal point back to the image plane. One way of achieving this is to move lens L1 (positive) backward, while tuning the position of lens L2 at the same position, so that the image of an object is focused at image plane (case (c)). In this system, the lens shifting the focal length is called “zooming lens,” or “variator,” whereas the lens correcting the focal point is called “correcting lens.” In case of analyzing multi-lens systems like the above system, it is effective to use the matrix method discussed in Chapter 2, “Multiple Lenses and System Matrices.” If f1 and f2 denote the focal length of the first and the second lens for the zoom control system, respectively, the paraxial matrix is determined by 2
" M¼
s11
s12
s21
s22
#
d 61 2 f 6 2 ¼6 6 4 d
3 1 1 d 2 2 þ f1 f2 f1 f2 7 7 7 7 5 d 12 f1
ð6:70Þ
where d is the distance between them. From this matrix, we obtain the equivalent focal length, feq and the length, lF from the optical system to the
380
Optomechatronics zooming lens L2 correcting lens L1
image plane
F Σi d
(a)
zooming lens L2
correcting lens L1
image plane
. F Σi
(b) correcting lens
L1
zooming lens L2
F
image plane
Σi
(c) FIGURE 6.68 A schematic illustrating the zooming principle.
focal plane Si : They are determined by feq ¼
f1 f2 ; f1 þ f2 2 d
lF ¼
ð f1 2 dÞf2 f1 þ f2 2 d
ð6:71Þ
This result implies that by varying the distance d, the focal length of the equivalent system and the distance to image plane can be easily varied for the case of a two-lens zooming system. It is noticed that for this combination of convex and concave lenses, the increase in d results in shorter focal length and image plane distance. For example, suppose that focal lengths of the convex and concave lenses of the zooming system are given by 15 mm and 10 mm, respectively, and that initially the two are separated from each other d ¼ 10 mm. In this case, the paraxial matrix becomes from Equation 6.70 3 2 2 3 2 3 1 " # " # 1 1 6 2 2 30 7 s11 s12 7 6 1 10 7 1 0 6 1 2 15 7 6 ¼4 M¼ 7 ð6:72Þ 5 4 5¼6 4 1 5 s21 s22 10 1 0 1 0 1 10 3
Basic Optomechatronic Functional Units
381
From Equation 6.71, we have feq ¼
21 ¼ 30 mm; s12
lF ¼ s22 feq ¼
1 £ 30 ¼ 10 mm 3
When two lenses become closer, that is, they decrease to 8 mm, they are obtained by feq ¼
21 ¼ 50 mm; s12
lF ¼ s22 feq ¼ 23:3 mm
The results imply that, due to this change, both the equivalent focal length and the distance to the image plane increase, as might be expected. From Equation 6.71 we can obtain the sensitivity of a thin zoom lens with respect to separation distance. By differentiating with respect to d, we have dfeg f1 f2 ¼ dðdÞ ð f1 þ f2 2 dÞ2
dlF 2f22 ¼ dðdÞ ð f1 þ f2 2 dÞ2
ð6:73Þ
It is noted that the two sensitivity functions get relatively large when f1 and f2 are small. Zooming-in and zooming-out utilize this multi-lens operating system. In this system, a zoom lens is moved forward and backward in order to achieve a desired variable magnification. In most cases, it is composed of a group of lenses derived from a Taylor triplet, plus-minus-plus, and moves in a complex way. A three-lens case is illustrated in Figure 6.69. The front convex lens near to the object, Lf is not moving to provide a tight seal. When the concave Lm is in mid-position, the system has an image of a certain size, as shown in Figure 6.69b. However, when this concave lens moves forward (shown in Figure 6.69a), the image becomes smaller, as can be obtained by a wide angle lens system. On the other hand, when the lens moves backward as shown in Figure 6.69c, the image becomes larger due to the enlarged equivalent focal length — as can be seen by a telephoto lens system. It is noted from this figure that the equivalent focal length in each case is different. The focal length of case (c) is much longer than that of case (a), indicating that a desired portion of an object can be zoomed with a certain magnification. Let us consider one more zooming problem with a three-lens system. A software package for lens design and analyis has been used to analyze this zooming system. Figure 6.70 shows the computed relationship between lens position and the corresponding focal length when the first convex lens is held fixed far left at a constant location. Those relationships are denoted by a dotted line for the distance between two lenses L1 and L2, and a chain line for the distance between L1 and L3. To facilitate the calculation of the relative positions of the lenses, two curves are approximated using 4th order
382
Optomechatronics
object
Lf
object
Lf
object
Lf
Lm
image plane
Lb
(a) Lm
image plane
Lb
(b) Lm
image plane
Lb
(c) FIGURE 6.69 Zoom lens illustrating wide-angle position (a) and telephoto position (b).
f1 = 15
f2 = –10
f3 = 10
19 18
case 1
17
film plane
equivalent focal length (feq ), mm
20
16 15
case 2
14 13 12
case 3 18
16
14 12 10 8 6 distance from film (d), mm
FIGURE 6.70 Focal length variation due to lens movement.
4
2
0
Basic Optomechatronic Functional Units
383
polynomial as 2 3 dL2 ¼2411:29174þ112:76932feq 210:70851feq þ0:43622feq 26:51 4 £1023 feq
ð6:74Þ
2 3 4 dL3 ¼2468:11113þ121:31288feq 211:1073feq þ0:43888feq 26:39£1023 feq
where dL2 and dL3 are the positions of the lenses L2, and L3 from film, respectively. When we use these curves, the magnification ratio can be specified by using the equivalent focal length, as long as two lenses are positioned at the two curves. Suppose we wish to change the initial lens configuration of case 1 into that of case 3: case 1 has the equivalent focal length of 19 mm, while case 3 has that of 13 mm, as a result of a change of the configuration. Since the equivalent focal lengths of the two cases are known, the positions of the two lenses can be easily determined from Equation 6.74. The positions of two lenses for case 1 are obtained as follows: the concave lens position of 9.19 mm and the convex lens 4.62 mm, while those for case 3, the concave lens, 17.41 mm and the convex lens, 13.53 mm. We therefore obtain the travel distance of each lens as ðconcavelensÞ¼17:4129:19¼8:22mm ðconvexlensÞ¼13:5324:62¼8:91mm We can see that, as a result of these two lens movements, the image ratio changes to M¼
feq1 13mm ¼ ¼0:68 feq3 19mm
where feq1 and feq3 denote the equivalent focal length of case 1 and case 3, respectively. Zoom Control Mechanism As we have already discussed using Figure 6.69 and Figure 6.70, providing zoom to obtain a desired magnitude of an object image requires adjustment of a focusing lens location. This indicates that from wide angle having the shortest focal length of the system to telephoto location having the longest focal length, lens Lb and lens Lf are required to simultaneously move with a certain relationship. Normally, the change in travel stroke of the two lenses is determined by the design of type and configuration of the lenses. There are basically two methods of controlling lens travel length. One popular method is to use a cam driven mechanism for each lens motion, and the other is to drive each lens independently. In the cam method, two cam mechanisms are driven by one stepper or servo motor, as shown in Figure 6.71. The lens L1 is located and fixed at the front (top), facing objects to be imaged. By a pin connection, the lens group, L2 is designed to move inside a set of three slots
384
Optomechatronics
1st lens (L1)
2nd lens (L2) cam barrel 3rd lens (L3)
motor
2nd lens (L2) 3rd lens (L3) FIGURE 6.71 Zoom lens control by a cam driven mechanism.
of the cam mechanism located right behind lens L1; whereas lens group L3 is designed to move inside another three cam slots located next to the previous cam-slot set. These slot-cams are all provided at the outermost lens barrel, which looks like a tube assembly. A slot-cam set (1) guides lens group L3. The barrel has a gear at its bottom periphery, as shown in the figure. When the motor starts to drive the barrel with this gear connection, the lens of each group moves within each cam curvature, rectilinearly moving along the optical axis of the system, as discussed above. The relationship between the curvatures of the two slot-cams is so designed that it produces exactly the same relationship between the relative displacement as the desired relationship between the lenses. With this cam mechanism, the desired variation of the lens position can be easily implemented. If the barrel is assumed to be designed to move in proportion to the increment of the focal length, the incremental rotation angle of the barrel necessary to move the lenses to the specified position can be easily obtained by Rb Du ¼ Dfeq
ð6:75Þ
where Dfeq is the increment in equivalent focal length due to zooming control, Rb and Du are the radius of the barrel, and the rotation angle of the
Basic Optomechatronic Functional Units lens 1 (fixed) lens 2
385
CCD lens 3 DSP
barrel 1
barrel 2
ultrasonic motor 1
ultrasonic motor 2
video signal processing unit
microprocessor (MPU) FIGURE 6.72 A schematic of microprocessor-based zoom control.
barrel provided by a motor, respectively. From this relation, if Rb ¼ 5 mm, then the angle increment that provides the necessary change in the equivalent focal length can be obtained for by Du ¼
13 2 19 ¼ 21:2 rad ¼ 268:758 5
As we may understand from examination of the slot-cam mechanism, this configuration method may have some difficulty achieving the desired accuracy of fabrication and lens alignment. Also, flexibility in zooming is not so high, due to the fixed travel pattern of the lens. The second zoom control method is designed to reduce the above limitations. The structure shown in Figure 6.72 shows schematically the details of the zooming configuration. The slot-cams are all removed, but, instead, the system utilizes two lens transport mechanisms driven by two ultrasonic motor drives. All the motion of the mechanisms is controlled by a microprocessor control unit (MPU). The motion data of the two lens groups are provided and preprogrammed in the MPU. From the signal generated by MPU based on these, each motor drives its own lens group accordingly: motor 1 drives lens barrel 1 containing the lens group L2, whereas motor 2 containing the lens group L3 drives lens barrel 2. At the other end of the system next to lens group 3, there is an image sensor (CCD) that feeds the information of focusing sharpness through a data signal processing (DSP) unit to the microprocessor in order to achieve autofocusing. The operation principle is as follows: When the system is set to a desired zoom position (i.e., either a telephoto or wide angle), two motors start to simultaneously drive each lens group according to the command signal sent by the MPU. Then, the image sensor located at the other end of the system next to lens
386
Optomechatronics
group L3 acquires the image of a scene, and feeds the information of focusing sharpness to the microprocessor in order to achieve autofocusing. Based upon this feedback information, each ultrasonic motor adjusts the position of its own lens, so that two lens groups move along the optical axis to desired locations. During the motion, the MPU registers the location of each lens group. Near the end of the motion, one lens group (say group 2) can move in fine mode independently from the other group in order to achieve the desired focusing. The information on the focus sharpness at a certain location of the lens group is provided by the CCD sensor. As will be discussed in “Visual Autofocusing,” the focus sharpness can be represented by the focus measure, which is subject to image condition. This focus information is sent to the MPU, which, in turn, utilizes the value to calculate the instantaneous position of the lens groups.
Visual Autofocusing Localization of the focal position of an optical system is a task critical to imaging for many optical systems, including microscopes, optical disks, and so on. The localization is normally carried out by a technique called autofocusing. This is an important procedure to obtain a sharp image, which is ultimately required for dimensional measurements, inspection, object tracking, and visual servoing. For instance, object distance from a focused lens position based on depth-from-focus is useful for measuring industrial parts or products, which usually have varying depth within their own geometry. Electronic cameras can be also automatically adjusted, such that they give a best-focused image by searching for the lens position. This is also important for lighting a microscope, when sharpness, robustness to noise, and speed are ultimately required. Besides these applications, another important application is the well-known approach to find the distance of an object from a certain reference position. All applications involve accurate positioning of either optical lens or image detector or both. Therefore, accurate control of their position is critical. In most cases, servo actuators are used, such as servo motor, ultrasonic motor, and piezoactuator. Image Blurring Let us consider the underlying concept of the visual autofocusing problem. Image information in a simple camera is shown in Figure 6.73a. As shown, suppose that an object (tree) is placed in front at a distance so from a lens having a focal length f. If the image detector is located at distance si to the right of the lens, a point source p on the object will arrive at p0 on the image plane. At this plane, we can see a clear, inverted image of the tree. As discussed in Chapter 2, “Lenses,” the focused image plane depends upon the
Basic Optomechatronic Functional Units
p object
387 image detector
lens
z f s0
p′
si
∑i
(a) image formation in a convex lens (in-focus) image detector
lens
p
z
f so
si ′
(b) image formation in a convex lens (out-focus)
si
p′ p ′′
Db ∑i
FIGURE 6.73 Defocused image formation of an object.
distance of the object, and the focal length of the lens, f. They are related by the formula given in Equation 2.14 1 1 1 þ ¼ so si f
ð6:76Þ
The equation indicates that, as the image plane moves away from the lens, the object point should move closer to the lens in order to get a clear image. When the image plane moves further, i.e., displaced from Si to S 0 i, as shown in Figure 6.73b, the image will be obtained at p00 but will become defocused at that position of the image plane. In this case, the above relationship (Equation 6.76) does not hold, because the image plane is not in focus, giving rise to a blurred image. Another case of causing blur occurs when the lens moves closer toward the image plane or the object moves towards the lens. In any of two cases, Equation 6.76 does not hold, thereby causing a blur of a finite size. We can see from the figure that the point p on the object becomes blurred on the image detector, displaced with a blur circle of diameter Db : According to optics, the blurred image of p00 has the same shape as that of the aperture of the lens, but is scaled by a certain scale factor. In the other method, we can determine blur diameter Db by using geometric and wave optics. Let us consider a simplified version of Figure 6.73b, in order to discuss the size of
388
Optomechatronics object plane Σ'o Σo δ1 δ2
image plane Σi lens Dbg p'
D ′bg
s
so
FIGURE 6.74 Depth of field.
blur. As shown in Figure 6.74, the blur is composed of two contributions, blur due to geometry of the optics and that caused by diffraction limit. When an object is located at the in-focus plane So, its conjugate image point is at a focus point p in the image plane. When it is moved out of So to So0 , the image will be formed in front of the focused image plane Si : Therefore, in the image plane, a blurred image will be formed with a size denoted by mD 0bg where m is the magnification. In this case, a point on S0o will be projected on a point on So, thus making a projected disc of finite size. By letting the displaced distance of the object be denoted by d ¼ s 2 so, we can determine the corresponding the blur size mDbg in the image plane. It can be shown that the defocused blur Dbg can be obtained from the geometric relation by Dbg ¼
2mdðNAÞ n
where NA and n are the numerical aperture and refraction of the optics involved, respectively. In addition, the blur due to diffraction at the lens aperture will occur. The diameter of the blur which is equal to Airy disc is then Dbd ¼
1:22lm ðNAÞ
where l is the wavelength of light. Combing the two contributions, the diameter of the resultant blur can be written as Db ¼ Dbg þ Dbd
ð6:77Þ
This is a theoretically determined amount of blur, but in reality, the blur will not be exactly given like this. Therefore, if we wish to determine its exact value, we must resort to an appropriate measurement method. Normally, this is done by evaluating a measure indicative of whether the measure image is in state of focus or not.
Basic Optomechatronic Functional Units
389
Focus Measure The blurred image can be corrected by searching for the lens position that yields the best focused image. There are two methodologies: One is to control either the position of the lens or the position of the detector relative to each other. The other is controlling the camera position while keeping the relative position at a constant distance. Then, the problem is how to control either one of them so that autofocusing can be achieved with high speed and accuracy, so as to yield the best image of an unknown object in the environment with noise. There are two processes needed to achieve autofocusing. The first one is to determine a focus measure which reflects the degree of focus correctly, as shown in Figure 6.75. The focus value must be simple enough to compute in a fast real-time manner for on-line applications. However, the problem is not so simple, because, in general, image signals contain high frequency components which make the focus value noise-sensitive and scene-dependent. Here, noise includes gray-level noise, excluding the noise coming from the non-front paralleled surfaces. The figure shows how focusing space is related to the focus measure, and indicates a focus measure as function of lens position. The problem is to find the position zf, where the focus measure is maximum within some adjustable range zmim # z # zmax : As discussed before, this range is called “depth of focus.” We will develop an algorithm which finds the best focusing position lens
search space
zf z
zmin
zmax without noise
focus measure (F)
with noise
0 FIGURE 6.75 Search space and its corresponding focus measure.
lens position (z)
390
Optomechatronics
within the focus range, by instantaneously controlling the distance of the focus lens. To do this, we need to establish a reliable focus measure. There are several measures that can be used for autofocusing. Some of these include; (1) gray level (2) gradient magnitude (3) square of gradient magnitude (Laplacian operator) (4) median filter. These focus measures are expressed in the form of their total summation or variance total summation : FT ¼
XX m
Fðm; nÞ
n
1 XX ½Fðm; nÞ 2 Fðm; nÞ variance : FV ¼ mn m n
2
ð6:78Þ
where FT denotes the sum of the measure values of Fðm; nÞ defined within the window of interest having of m £ n pixels, FV is the sum of the variance values defined within the window, and Fðm; nÞ is the statistical mean of Fðm; nÞ: In the above, FT and FV values are usually sensitive to noise, and depend upon edge directivity and type of scene. Therefore, the choice of edge operators introduced in Chapter 3, “Image Processing” needs to be carefully made in consideration of image contents. We will deal with four different Fðm; nÞ values to determine FV in the calculations below. (1) FV based on intensity level variance: Let f ðm; nÞ be the pixel intensity value within a window of the image obtained at a certain focusing time. The image is then normalized by subtracting the mean gray value from the gray level of each pixel Fðm; nÞ ¼ f ðm; nÞ 2 fðm; nÞ where fðm; nÞ is a mean gray level value defined in the same window. (2) FV based on the variance of the Sobel operator: If Gmx and Gmy are the Sobel operator with the convolution kernels as previously defined in Chapter 3, “Image Processing,” Fðx; yÞ can be obtained by qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð6:79Þ Fðm; nÞ ¼ {G x p f ðm; nÞ}2 þ {G y p f ðm; nÞ}2 where p denotes the convolution integral of the quantities in parentheses. With this value obtained, the focus measure based upon the Sobel operator is defined as FV, the variance of the whole Fðm; nÞ within m £ n image window. (3) FV based on Laplacian operator: The Laplacian utilizes two second partial derivatives with respect to horizontal and vertical directions. The Fðx; yÞ value is expressed as Fðx; yÞ ¼ 72 f ðx; yÞ ¼
›2 f ðx; yÞ ›2 f ðx; yÞ þ ›x2 ›y2
ð6:80Þ
The operator 72 was given by Equation 3.34 in Chapter 3, “Image Processing,” and its variance measure FV can be computed by Equation 6.78. (4) Variance FV based on weighted median filter:
Basic Optomechatronic Functional Units
391
It has been pointed out in Chapter 3, in the section “Image Processing” that, in a noisy image, a weighted median filter can not only extract high frequency components from the edges image but also eliminate impulsive noise. As we might recall, the weighted median filter is given by qffiffiffiffiffiffiffiffiffiffiffiffi Wðm; nÞ ¼ w2n þ w2m 1 1 Med{Im23;n ; Im22;n ; Im21;n } þ Med{Im21;n ; Im;n ; Imþ1;n } 4 2 1 2 Med{Imþ1;n ; Imþ2;n ; Imþ3;n } 4 1 1 wm ¼ 2 Med{Im;n23 ; Im;n22 ; Im;n21 } þ Med{Im;n21 ; Im;n ; Im;nþ1 } 4 2 1 2 Med{Im;nþ1 ; Im;nþ2 ; Im;nþ3 } 4 wn ¼ 2
where Med{·} is a median operator. The focus measure in this case is defined by the variance term Fðm; nÞ ¼ Wðm; nÞ
ð6:81Þ
Now, let us consider some practical examples to understand the underlying concept on how the focus measure can be used for autofocusing by evaluating the methods presented above. To investigate this, we need to set up a zooming unit and an autofocusing unit as indicated in Figure 6.76a. It consists of a camera with zoom lenses mounted on a movable stage, an illuminating system, a light source and pattern for a camera calibration, and a beam splitter. By use of this hardware system, a focus measuring system is constructed for autofocusing, as shown in Figure 6.76b. The system is aimed to compute a focus measure, (F) of an image at various locations of the stage. light source object / pattern camera lens CCD
pattern
beam splitter
zoom lens
motor
camera
stage
illuminator
z
frame grabber
microprocessor F value focusing
object
(a) a schematic diagram
of a visual autofocusing system
(b) block diagram of representing the flow of autofocusing
FIGURE 6.76 Finding the relationship between the focus measure and camera position.
392
Optomechatronics
address bus data bus control bus
3 x 8 bit I/O bus
CPU
S/W (manual/auto) manual adjuster
chip set
PCI bus
EPROM
RAM
AGP bus
ROM
I/O Board
power supply motor
motor driver
motion controller
camera motor
frame grabber graphic device
PC
object monitor
stage
linear motion stage
FIGURE 6.77 PC-based controller architecture: autofocusing system & zoom control system.
Figure 6.77 shows the interface between the personal computer and the hardware executing illumination control, zoom control and camera position control. If we look into the interfacing part between the personal computer and the relevant hardware involved with this illumination control system, we can see that there are three major parts. The first part is a motion controller that provides command signals to 2-axis servo drivers through a DA converter, and controls the position of the zoom lens and that of the focus lens. The second part is an illumination controller that provides control current to the 3-layered ring illumination system through the DA converter. In digital I/O, 8 bits per ring is allocated to represent the intensity of each ring, in total 24 bits. The third part is frame grabbers, which capture the image of a scene. The rest of the parts include ADC, DAC flag control, and multiplexer.
FIGURE 6.78 Three image patterns used for autofocusing.
Basic Optomechatronic Functional Units
Wx
393
Wx = 150
Wy = 150
Wy
focus measure window
filtered image
Wx
Wy
focus measure calculation
filtered image FIGURE 6.79 The size of window within image frame.
Three different image patterns are considered here, as shown in Figure 6.78a: a stripe pattern, a rectangular block array, and a human face. At each instant of time, the image is acquired by a frame grabber. This acquired image is processed by the edge processing methods. The image is captured by a CCD area camera (640 £ 480). A window size 150 £ 150 within this image is used as illustrated in Figure 6.79, in order to calculate the variance value of F value. This value is used to represent the focus measure at that position. The set of the focus measures is computed utilizing the equations given in Equation 6.78 through Equation 6.81. The focus value is repeatedly determined by varying the camera position incrementally, Dz ¼ 1 mm: Figure 6.80 shows variation of the computed measure with camera position. For the stripe pattern, the weighted median filter and Sobel operator show much better performance than the Laplacian operator. A similar trend can be seen for the square matrix array. For a more general image like a human face, the Sobel operator has a distinct peak value at a certain lens location, which is proven to be a reliable measure for focusing. The median filter shows a peak which is not so distinct as compared to that of the Sobel operator. For the Laplacian operator, the value almost does not appreciably change with lens location for all three patterns. This may be attributable to some noise included in the patterns. In summary, for the stripe and matrix patterns, the Sobel operator shows better performance
394
Optomechatronics
20
Sobel
15 10 5 0 –20
WMedian
Laplacian
–10 0 10 distance (z), mm
20
4.5 4 3.5 3 2.5 2 1.5 1 0.5 0
(a) stripe pattern
4
16 focus measure
focus measure
25
12
3
Sobel
8
2
WMedian
1
4 Laplacian
0 –20
0 –10 0 10 distance (z), mm
20
(b) matrix pattern 0.8
focus measure
8 6 4 2 0 –20
0.6
WMedian
0.4
Sobel Laplacian
–10 0 10 distance (z), mm
0.2 20
0
(c) human face FIGURE 6.80 Focus measure variation with camera position.
outside the near-focus region, while the weighted median filter yields better within the in-focus near region. The next phase for executing autofocusing is to find the best focused position of the camera in an on-line fashion. In other words, we are interested in finding the position automatically. To solve this problem, we will employ the hill-climbing algorithm which is frequently used in the optimization method. The algorithm is essentially a search method, which determines the direction of the lens movement in order to search for the maximum value of the focus measure. Figure 6.81 illustrates schematically the concept of the algorithm in detail. The algorithm starts with search mode to determine the direction of the initial search. Let us suppose we can move the camera (lens) incrementally with distance Dz within the range zmin # z # zmax and calculate the corresponding focus value at each location. If the focus lens is assumed to be initially located at z ¼ zi ; where zi denotes the ith location of the focus position, we can compute Fðzi Þ and Fðzi þ Dzi Þ; the focus measure at the initial position and z ¼ zi þ Dz position, respectively. The incremental distance, Dz is called a hopping distance. If Fðzi þ DzÞ . Fðzi Þ; the search direction is the same as increasing Dz: Thus, we need to move the lens to a new position, z ¼ zi þ Dzi : If not, we move it backward, and the new position
Basic Optomechatronic Functional Units
395 z*
focus measure ( F(z) )
zfocus df db
F( zi + Dz ) F( zi )
… zi
zi + Dz
zp-1
zp
zp+1
camera position (z)
FIGURE 6.81 Finding the best focus measure by the hill climbing method.
will be at z ¼ zi 2 Dz: This can be written mathematically as ziþ1 ¼ zi þ Dz sgn{Fðzi þ 1Þ 2 Fðzi Þ}
for
Dz . 0
ð6:82Þ
In this manner, the search operation continues until it reaches around the peak zp < zfocus at a time step tp : At this instant, the F value meets the following condition Fðzp21 Þ , Fðzp Þ . Fðzpþ1 Þ
ð6:83Þ
Because this zp is not an exact peak point, it is necessary to refine the focus position. The refined peak can be determined by " # db 2 df Dz z ¼ zp þ £ 2 db þ df where db ¼ Fðzp Þ 2 Fðzp21 Þ; and df ¼ Fðzp Þ 2 Fðzpþ1 Þ: When the peak estimation is roughly completed at z ¼ zp in the above, hill climbing algorithm is performed again, with a small hopping distance dz for further fine tuning. The search starts with zpþj and, j ¼ 1 and continues until it hits the same condition as expressed in Equation 6.83, with p replaced by j. At this point, the equation is evaluated with dz replacing Dz and zp by zp þ N: N is here the integer value that gives z as an estimated peak. The flow chart of describing this procedure is depicted in Figure 6.82. To practice the hill climbing method, let us assume that a focus measure for a human face reveals a Gaussian distribution with respect to camera position z as follows FðzÞ ¼
1 1 exp 2 ðz 2 zÞ2 ; z ¼ 3:5; s ¼ 1:0 2ps 2s 2
396
Optomechatronics
start i = 0, dz = ∆z
focus measure Fi
zi+1 = zi + dz focus measure F i+1 dz = dz
Fi+1 > Fi no
dz = -1 × dz
no
yes
Fi-1 < Fi > Fi+1 ( i > 1)
yes peak estimation z *focus yes
dz = ∆z no end
FIGURE 6.82 The flow chart of the hill climbing algorithm flow.
which is shown in Figure 6.83. When the camera is initially located at z(0) ¼ z0 ¼ 2 2.6, the problem is to find the peak measure, and the best focused position zfocus, using the hill climbing method with the searching parameters Dz ¼ 3:0; dz ¼ 1:0: Let us begin with the initial position in order to find the initial value of F value at this location z0 ¼ 22:6 ! Fðz0 Þ ¼
1 1 exp 2 6:12 ¼ 1:32 £ 1029 2p 2
This gives the initial value. We then proceed to obtain the F value at z1 z1 ¼ z0 þ Dz ¼ 0:4 ! Fðz1 Þ ¼
1 1 exp 2 3:12 ¼ 1:30 £ 1023 2p 2
397
F
Basic Optomechatronic Functional Units
0
∆ FIGURE 6.83 Focus measure of human face image vs. camera position.
Since Fðz1 Þ . Fðz0 Þ; update z as z þ Dz z2 ¼ z1 þ Dz ¼ 3:4 ! Fðz2 Þ ¼
1 1 exp 2 0:12 ¼ 1:58 £ 1021 2p 2
Also, Fðz2 Þ . Fðz1 Þ; let us update z as z þ Dz n o 1 z3 ¼ z2 þ Dz ¼ 6:4 ! Fðz3 Þ ¼ exp 2 12 2:92 ¼ 2:37 £ 1023 2p At this time, we meet the peak condition, Fðz1 Þ , Fðz2 Þ . Fðz3 Þ; and make the first peak estimation " p
z ¼ z2 þ
db 2 df Dz £ 2 db þ df
#
" ¼ 3:4 þ
db 2 df 3 £ 2 db þ df
# ¼ 3:405135
since db ¼ Fðz2 Þ 2 Fðz1 Þ ¼ 0:157058 and df ¼ Fðz2 Þ 2 Fðz3 Þ ¼ 0:155986, we start again hill climbing method with dz ¼ 1:0 z4 ¼ zp ¼ 3:405135 ! Fðz4 Þ ¼
1 1 exp 2 ð20:095Þ2 ¼ 1:58 £ 1021 2p 2
z5 ¼ z4 þ dz ¼ 4:405135 ! Fðz5 Þ ¼
1 1 exp 2 0:9052 ¼ 1:06 £ 1021 2p 2
Since Fðz5 Þ , Fðz4 Þ; update z as z 2 dz z6 ¼ z5 2 dz ¼ 3:405135 ! Fðz6 Þ ¼ Fðz4 Þ ¼ 1:584 £ 1021
398
Optomechatronics
Since Fðz6 Þ . Fðz5 Þ; update again z as z 2 dz z7 ¼ z6 2 dz ¼ 2:405135 ! Fðz7 Þ ¼
1 1 exp 2 ð21:095Þ2 ¼ 8:74 £ 1022 2p 2
We meet the peak condition again, Fðz5 Þ , Fðz6 Þ . Fðz7 Þ; and make the peak estimation for fine motion: " # " # db 2 df dz db 2 df 21 p ¼ 3:405135 þ ¼ 3:478876 z ¼ z6 þ £ £ 2 db þ df 2 db þ df since db ¼ Fðz6 Þ 2 Fðz5 Þ ¼ 0:052778 and df ¼ Fðz6 Þ 2 Fðz7 Þ ¼ 0:071039: Now, the searching procedure is completed, and the final focused position is estimated to be z8 ¼ zfocus ¼ 3:478876: Let us consider an autofocusing problem for a stripe pattern in order to implement the hill climbing method. To carry out autofocusing a system configuration shown in Figure 6.76a is used in which a camera is moving instead of an optical lens. The image is taken by a 640 £ 480 area camera, and the window size is chosen to be 150 £ 150 pixels. Camera motion for searching is given in two different modes: The incremental movement for coarse searching, Dz is 3 mm, and for fine search operation dz is selected to be 1 mm. The variance of gradient magnitude (Sobel operator) value given in
FIGURE 6.84 Finding the focused position using the hill climbing method.
Basic Optomechatronic Functional Units
399
Equation 6.79 is used as the focus measure (F). The procedure to compute FV value at each search position is the same as described previously. In Figure 6.84a, the instantaneous camera position against search time is plotted, while the corresponding FV value is plotted in Figure 6.84b. In Figure 6.84c, the image variation is illustrated for various camera locations z ¼ 19.5, 13.5, 5.4, and 5.9 mm. The effort of focusing starts from z(0) ¼ 19.5 mm. As we can see, when the camera position is moved to the next stop z ¼ 22.5 mm, where FV value decreases slightly to 7. This indicates that searching is made in the opposite direction away from the object. Now, if the searching is made in sequences toward the other direction closer to the object, FV value obtained shows a monotonic increase from the previous value. In this way, searching continues until it reaches a near-maximum value FV ¼ 35 at z ¼ 7 mm. Further increase from this movement toward the object results in a slight decrease in the FV value at z(7). From this moment, the fine tuning method is applied by changing the increment in z from Dz to dz: Finally, the search ends with Fv ¼ 37:5 at z ¼ 5.9 mm. Figure 6.84c clearly depicts the change of the pattern image with a different camera position. As can be seen, the best focused image is obtained at x ¼ 5.9 mm.
Illumination Control A number of visual and optical methodologies, including Moire´, structured light projection, and stereo vision, has been developed to obtain such information as the shape dimension, and the pose of objects, which are needed for various applications. It has been discovered, however, that the optical visual information obtained is vulnerable to external illumination, and the surface reflection properties and shape of objects. The image varies according to different reflective, absorptive, and transmissive characteristics, depending on material properties and surface characteristics of the objects. Also, they are affected by the directivity of illumination angle and the types of illumination, such as backlight, direct or indirect lighting, which are problem-dependent. Therefore, it is most important to design an illumination system that is appropriate to the object to be inspected. In actual cases, objects have a variety of different characteristics in terms of geometrical shape, surface roughness, and reflection properties. This implies that an illumination system must be controllable and adaptable to changes in measuring conditions and environments, and thus needs to be restructured based on the optomechatronic principle. In designing an optical or visual measurement, this principle becomes an important consideration. This is because illumination control has an enhancing effect on accuracy, reliability, and processing time. In this section, we will first treat several considerations
400
Optomechatronics
needed for illumination control and various illumination methods. Then, actual methodologies for control of illumination will be treated in brief by illustrating the systems that are capable of adjusting a directivity and intensity of illumination.
Illumination Methods There are several factors that need to be considered for illumination. First, lighting directivity is important to obtain the maximum contrast between features, e.g., between an object and its background. Optical properties of an object surface produce different patterns of radiance, which are diffuse, specular diffuse, and specular. When specularity is unwanted for a detector, then polarization filers, which employ the principle of polarizing light discussed in Chapter 2, “Aberration,” may be used to reduce the effects of specular reflection. Secondly, the spectral distribution of the power from the light source must be taken into consideration for overall sensitivity and discrimination of the system. Figure 6.85 shows the spectral emission of some common illuminators in a vision system. As can be seen from the figure, an LED light source (for example) has a narrow spectral band. Therefore, it can be effectively used as a light source for illuminating a scene with background light whose spectral content is outside this band. In case of the broad band background light, an optical band pass filter may be used to detect the scene. Longer wavelengths of infrared light are not much affected by the presence of small particles. Therefore, this light source is particularly effective when objects to be detected are surrounded by small particles such as dust or chemical ingredients. Ultraviolet light can be used to identify the presence of fluorescent materials; for instance, when cracks in objects to be detected are sprayed with fluorescent dye.
relative intensity (I)
1 0.8
fluorescent LED
0.6
halogen light He-Ne laser(632.8nm, ∆l = 10–6nm) laser diode tungsten lamp Si blue sensitive photodiode
0.4 0.2 400
600
800
1000
wave length (λ), nm
FIGURE 6.85 Spectrum of various light illuminators.
1200
1400
Basic Optomechatronic Functional Units
401
Thirdly, the diffusive property and size of the source affect the illumination conditions. Diffuse lighting produces rather uniform intensity over the wide illumination area of interest, while small lighting sources produce sharp shadows. Diffusion of light sources can be easily achieved by placing diffusing glasses such as mylar or opalescent glass in front of the sources. The diffusion material serves to enlarge the effective area of the source. Small sources can be produced by optical components such as mirrors, Fresnel lenses, and fiber optics, whose types are dependent upon how they are combined. For instance, to produce a parallel beam, a point light source can be placed at the focal point of a parabolic mirror. To make a focused beam, the source can be put at one focal point of an ellipsoidal mirror. Figure 6.86 depicts various commonly used forms of illumination methods. Figure 6.86a illustrates two types of backlighting. The one on the left is diffuse backlighting, which provides a silhouette image for the target object to be measured and a white image for the background. This can be done by illuminating with respect to the sensing lens. The directional back lighting uses a collimator, which produces all the beams traveling parallel to detector or camera
detector or camera
object
object
lens light source
light source
(a) black lighting detector or camera
incident light
detector or camera diffusing glass
diffusing glass
light source object
(b) directional lighting FIGURE 6.86 Various types of illumination.
object
(c) diffuse front lighting
402
Optomechatronics detector or camera
detector or camera
light source ring
polarizing filter
object
object
(d) fiber optic ring lighting
(e) polarized lighting detector or camera
transparent object
ring
(f) dark field illumination FIGURE 6.86 Continued.
one another. This is effectively used to produce sharply defined shadows on objects, which can be created by collimated light. Figure 6.86b shows a directional lighting whose angle of incidence of the light is arbitrary placed to the object surface. We can see that, as the angle of incidence increases, a larger portion of the incident light is reflected from the flat surface of the test plane. In this case, the visible contrast between the test object and the flat test plane gets reduced. Diffuse front light is illustrated in Figure 6.86c. As mentioned before, diffusion of a light source can be provided by either passive or active diffusing glass in front of a light source. Due to the arrangement, this lighting method eliminates shadows and provides uniform illumination conditions. Fiber optic ring-lighting is illustrated in Figure 6.86d. A ring-lighting device consists of several concentric rings having an array of optical fibers within each ring. Due to its configuration, the ring lights reduce specular reflections from shiny surfaces and provide shadow-free lighting. This method is very useful for highly reflective objects. Polarized lighting is shown in Figure 6.86e. This method reduces specular reflections from object surfaces from any angle by using a polarizing filter placed in front of the camera: Specularly reflected light is usually highly polarized, and the polarized components of the light can be greatly attenuated when the polarizer is at right angles to the other. Figure 6.86f shows a dark field illumination configuration that can be effectively utilized for many objects not visible by diffuse backlighting. In this method, only the
Basic Optomechatronic Functional Units
403
light scattered or reflected by the object will be directed into the optical axis of the camera or detector. Therefore, we can improve image contrast, since, by dark field illumination, the edges of a transparent object appear bright against a dark background. Illumination Control Figure 6.87 shows a simple ring-type illumination that can adjust the directivity and diffusivity of light. The directivity control is achieved by switching on the LEDs in an appropriate layer. Clearly, depending upon the LED location, the incident angle of light varies as illustrated for three cases of incident angles in the figure. Diffusivity of light is controlled by placing a special glass in front of each LED, which changes the degree of its transparency when it gets a certain electric current. Now, suppose that a rectangular object of white color is mounted on a table of black color that effectively absorbs light. Both surfaces are assumed to be lambertian such that they scatter light in a random direction. Under this configuration, there will be a certain incident angle with which the camera can better see an object. From the discussions in Chapter 3, we can see that the incident angle of illumination and the angle of detection are crucial factors to acquisition of an image of good quality. A detailed consideration of this problem is left for exercise at the end of this chapter. With this preliminary knowledge in mind, we will now discuss some of the illumination control systems. Figure 6.88 illustrates an illumination control system, whose configuration is changeable, depending upon the measurement condition. The system can vary the illumination angle to a desired degree by producing a special oblique lighting, and can control intensity as well. The system consists of a lighting unit, an optical mirror system and a unit movable in a vertical direction, carrying the lighting and optical units. The lighting unit has a concentric ring light structure either with LED lights or fiber guided lights. The whole ring is essentially a quadrant ring system, which can arbitrarily combine illuminations from four camera ring illuminator
LED object (block) FIGURE 6.87 Three different illumination angles produced by the ring-type illuminator.
404
Optomechatronics CCD camera
motor for rotating turret pattern projector lenses on turret
front
mirror
mirror
quadrant RGB LED ring lights toroidal mirror
parabolic mirror object
(a) optical system configuration
left
lens
quadrant ring l ights
section 1
elevation stage
section 3
section 4
right
toroidal mirror
section 2
4 parabolic mirrors back
(b) bottom view of quadrant RGB ring lights
FIGURE 6.88 A schematic of a variable illumination system.
different directions. By the optical-path guiding system composed of curved mirrors, illumination incident angle is allowed to be selectable as the different rings are illuminated. It is also noted that segmenting a certain part of the rings allows the light to be directional, which is especially suitable for some object features that lie in a particular direction. More segmentation implies more detailed control of the angle of illumination. The optical mirror system composed of four mirrors shown in Figure 6.88a has the role of providing the beam steering that produces a desired angle of incidence. They are drawn here as a separate unit for laser scanning profiling, but, for visual measurement, the mirror system has a continuous, smooth surface (such as a parabolic mirror and a toroidal mirror), depending on the illumination applications depicted in Figure 6.88b. The illumination incident angle can also be controlled by lowering or lifting the illumination unit, including lighting together with optical units, as shown in Figure 6.89. Incident angles 308, 608, and 808 are illustrated for better understanding. To generate each condition, it can be seen that the distance between the quadrant ring light with the fixed toroidal mirror and the parabolic mirror is varied from d1, through d2, and d3. It makes the optical rays incident from toroidal mirror to parabolic mirror experience a different incident angle in each case. In this arrangement, if it satisfies d1 . d2 . d3, then u1 , u2 , u3, which are the corresponding incident angles to the object. In other words, relatively larger distance makes a larger incident angle to the parabolic mirror, which, in turn, results in a smaller incident angle to the workpiece to be measured.
Basic Optomechatronic Functional Units
quadrant RGB LED ring lights
405
toroidal mirror parabola mirror
d1 d2 d3 30°
60°
(a) using high slope of mirror
80°
(b) using middle slope of mirror
(c) using low slope of mirror
FIGURE 6.89 Illumination angle variation by vertical motion of the optical unit.
The principle of the illumination control discussed in the above example can be applied to the case when higher resolution of an image is required. Consider an optical system that focuses incident light to an object plane, as shown in Figure 6.90. The system is composed of a lighting system (not shown here), a diffraction grating pattern, and a collecting lens. When the pattern is illuminated by a light source as indicated by an arrow, according to diffraction theory, the diffraction angle u is given by sin u ¼ l=w light
light a
grating m = –1
m = +1 m = –1
m=0
q
grating m = +1
q m=0
lens
object plane
(a) a = 0 FIGURE 6.90 Enhancing the image by illumination control.
lens
object plane
(b) a ≠ 0
406
Optomechatronics
light source pattern lens
beam spliter
camera
zoom Zoom
LED illuminations layer 1
motor
stage
layer 2
z
layer 3 object
backlight
variable diffuser
x-y positioning stage
FIGURE 6.91 A flexible vision system with variable illuminators and zooming and autofocusing capabilities.
where l is the wave length of the illuminating light and w is the period of the slit of the pattern. When w is small, the m ¼ ^1 order beam cannot reach the aperture of the lens and only m ¼ 0 order beam will be transmitted, as indicated in Figure 6.90a. This condition deteriorates the image focusing, because the m ¼ þ1 or 2 1 order light beams do not contribute to the image formation. As indicated by the arrow in Figure 6.90b, if illuminating angle a is made oblique, e.g., a – 0, either the m ¼ þ1 or 2 1 order beam will be transmitted through the aperture in addition to m ¼ 0 order beam. In this case, m ¼ þ1 will not be transmitted through the lens. This obviously helps obtain brighter, sharper image quality. This concept is being used for the photomasking process in photolithography. In evaluating illumination quality, it is necessary to take images with a desired magnification and sharpness. For this purpose, referring to Figure 6.91, a flexible system employs a zoom control system and an autofocusing system, to maximize detector coverage for objects of a wide range of sizes, and to maximize the sharpness of the image as well. The system consists of LED light sources, controllable diffusers, a pattern projector, a zoom control unit, an autofocusing camera, and an x-y stage. The main light source is a three-layered ring illumination made of a number of LEDs. A backlight panel mounted on the x-y stage is employed here as a sub-lighting unit to further control the illumination. A controllable diffuser glass is mounted in front of each LED layer. It is an active glass device whose transparency is electrically adjustable, so that it plays a role of controlling the diffusing level of the light emitted from the source, and of changing the spatial distribution of the light. Zoom control is provided by the control of
Basic Optomechatronic Functional Units
407
40
–40
–60 –60
0
–20
–60
–40
0 –4
0
–2
0
20
–40 –20 0 20 40 Y(mm)
–40
20
X(mm)
20
X( 0 mm –20 )
m Y(
–20
40
0
–2
40
) mm Y(
(b) case 2 : all diffusers off FIGURE 6.92 The distribution of the incident light according to diffuser control.
0
0.25 0.2 0.15 0.1 0.05 0 60
0
–4
40
m)
20
(a) case 1 : all diffusers on
–60
20
40
–40
0
40
20
0 X( mm –20 )
60
60
z (mA)
–40 –20 0 20 40 Y(mm)
0.25 0.2 0.15 0.1 0.05
60
z (mA)
the position of the lens whose unit is included in the camera; the auto focusing is achieved by adjusting the location of the camera with respect to the object. To this end, a motion control is provided for the motor in order to move the camera assembly along the optical axis in such a way that the object of interest can be located within the depth of the focus of the lens optics used at the particular magnification. Once this is done with the electrical motor system with linear slides and scales to keep track of zooming position, autofocusing is performed to tune the sharpness of images. The autofocusing utilizes a pattern projection method in which a pattern is generated and projected on to an object surface by a LCD projector. The pattern can be differently designed, depending on the type of applications. The role of the x-y table keeps the object of interest within the field of view of the camera. Using this configuration, illumination characteristics are varied according to a number of combinations of adjusting light intensity and diffuse levels of intensity of each LED layer. In order to investigate quality of the illumination with this system configuration, the performance of the autofocusing unit is evaluated for various conditions in the LED lighting system. The effect of the diffuser on uniformity of illumination is investigated for a simple object geometry, hemisphere. To do this a hemisphere is placed in the center of the x-y table, and the illumination intensity is measured on a visual inspection plane using a photo detector. Figure 6.92 shows the actual intensity distribution of an incident illumination on a surface according to the
X(mm)
408
Optomechatronics
diffuser
J-lead
J-lead
J-lead
J-lead
back bac k llight i g ht
(a) backlight
(b) direct light (layer 1)
(c) direct + indirect (d) diffused indirect light (layer 1,2,3)
(layer 1,2,3 + diffuser)
FIGURE 6.93 Images obtained by variable lighting conditions.
controllable diffuser. The intensity for case 1 is lower, but its distribution spreads wider than that of case 2. This indicates that when diffusers are on, the light intensity is more uniformly distributed over the space of interest. The effect of LED layers is shown in Figure 6.93. The figure depicts several images of an electronic chip under different lighting conditions, which are typical illuminations widely used in visual inspection. To detect the boundary of an object, a silhouette image by backlight, as shown in Figure 6.93a, is found to be useful. To detect leads on the chip, on the other hand, LED lights are used. Figures 6.93b to d show the images changed according to the differently applied illumination conditions. Three different conditions for the angles of incidence are applied. When we use only the top layer, it works as coaxial light. Thus, the flat and specular parts of the leads are highlighted in the image, as shown in Figure 6.93b. By using all LED layers together, we obtain a brighter image, shown in Figure 6.93c. However, there are high lighted spots, specular reflected, on leads. Finally, the use of the diffused LED lights make LED light uniformly illuminated in all directions, although overall intensities of the resulting image are somewhat reduced, as shown in Figure 6.93d. Illumination Quality Measure As seen from the above illustrations, the adjustment of illumination is problem dependent, for the criterion of the image quality should be changed according to which information is important in the image for a particular purpose. For instance, the body or the leads of a chip could often be utilized for chip recognition. To detect the leads, illumination needs to be adjusted so that the reflected light on the body is not imaged, and vice versa. In this
Basic Optomechatronic Functional Units
409
application, adjustment of illumination can be made based on the criterion of how well the obtained image represents the local information of interest. Although there may be several criteria to determine illumination quality, we will consider here one heuristic method that can be used for image segmentation. Let the region R be a region of interest, with m £ n pixels within the image. Using a threshold value, the gray-level image of the region R can be divided into two sets of pixels, which are a set of pixels, A, with higher intensity values than the threshold value, T and the other set of pixels, B, with lower intensity values. This condition is represented mathematically as follows A ¼ {Ii lIi . T for i [ R};
B ¼ {Ii lIi # T for i [ R}
ð6:84Þ
where Ii denotes the intensity value of the ith pixel in the region R, and T is the specified threshold value. One way to represent the quality of the binarized image obtained in Equation 6.84 is to use the average intensity value and the standard deviation of the pixel intensities in the two regions. The quality index can be expressed by combining the two parameters as r ð6:85Þ uimg ¼ ðIA 2 IB Þ þ sA sB where IA and sA are the average and the standard deviation of the pixel intensities in the set A, respectively, IB and sB are the same quantities related to the set B, and r denotes a scale factor. The first term indicates the difference in intensity between the two areas within the image, while the second denotes how uniformly distributed the intensity value of each region is. The threshold value here is determined by an iterative threshold selection algorithm introduced in Chapter 3, “Image Processing,” and the scale factor is set to a desired value depending on which term is of larger importance. It is noted that this quality criterion rates high value when the areas segmented by binarization are far from each other in terms of intensity, and also when the intensity distribution of each region is uniform, having a low standard deviation. In order to see the effectiveness of this evaluation method, illumination quality is investigated for three kinds of chips, capacitor, small ontline integrated circuit (SOIC), and rectangular chip, as shown in Figure 6.94. The purpose of the investigation is to find the best illumination condition that can discriminate one feature region from the other within an image. Finding the illumination condition is changed by controlling the configuration of the illumination system, shown in Figure 6.91. For instance, LED-on in layer 1 implies all LEDS in the layer turned on, while diffuser-off of layer 2 implies all diffusers in the layer to be turned off. In this way, various illumination conditions are produced to investigate the quality. Under each condition, the index value given in Equation 6.85 is computed. Figure 6.94 represents the best results obtained from Equation 6.85 when illumination and diffusing units are properly adjusted. The imaging results reveal that in this
410
Optomechatronics
SOIC
capacitor
RC-chip
FIGURE 6.94 Illumination adjustment results.
illumination system each chip has its own configuration of the illumination rendering a high quality image. The condition yielding the best result is listed in Table 6.8. Autofocusing with Illumination Control In the electronics industry, locating, inspecting, and recognizing surfacemount device packages of various sizes and types are ultimately important to obtain good quality PCB. For instance, in particular applications such as microchip assembly, the illumination condition is required to be varied for better image acquisition according to the types of chips under inspection. In such situations, if the autofocusing system is used with the illumination control system, the combined system can significantly enhance image quality. To illustrate this example, the integrated system used above is implemented to inspect electronic parts. We will consider a rectangular chip, a plastic-leaded chip carrier (PLCC), and SOIC. Figure 6.95(a) TABLE 6.8 Optimal Illumination Configuration for Chip Inspection Layer 1
Capacitor SOIC Rectangular chip
Layer 2
Layer 3
LED
Diffuser
LED
Diffuser
LED
Diffuser
ON ON ON
OFF OFF ON
ON ON ON
ON ON ON
ON OFF OFF
ON OFF OFF
Basic Optomechatronic Functional Units
411
loading chips
Backlight LED lights LED lights Diffuser
object segmentiaton (number of object = N) k =1 Object 1
autofocusing Autofocusi ng Object 2
illumination adjustment chip Chipinspection inspection if k=N yes
no
k=k+1
Object 3
end
(a) the overall procedure of chip inspection
(b) segmentation , autofocusing, and inspection
FIGURE 6.95 Illumination control system with autofocusing capability.
illustrates the overall procedure of chip recognition, which is composed of four steps as follows: The first step is the segmentation of the objects in the camera view, in which only the backlight is utilized. The backlight image is then binarized, and the objects are segmented by a blob analysis. Once the positions and sizes of the objects are identified through the blob analysis, each region is windowed and processed independently, as shown in the figure. The second step is adjusting the focus of each chip in each window, under the situation that all of the LED layers are lighted. In this procedure, each chip is independently focused sequentially. The third step is adjusting the LEDs and diffuser units to achieve a high quality image, which varies with the type of chip. In this application, inspection of chips is conducted to obtain the leads information, such as the number, the size, and the pitch of the leads. Since the leads are all made of specular reflected materials, illumination is adjusted here so that the leads are dominantly imaged. After the procedure of autofocusing and illumination adjustment is completed for the image, chip inspection is performed as the final step. This procedure is sequentially conducted for all objects appearing in the image. The picture shown in Figure 6.95b shows the segmentation and autofocusing results for an image of the three chips. The focused results indicate an image quality satisfactory to find the required information on the leads of each chip.
Visual (Optical) Information Feedback Control Visual or optical information is being widely used in most of mechatronic processes or systems. Among its various applications, feedback control of
412
Optomechatronics
visual (optical) information is becoming essential for precision positioning, monitoring and control, inspection, and pattern recognition, as illustrated in Table 6.9. For instance, current robotic systems needed for a variety of placing, manipulating, and tracking of objects or workpieces are also in need of such a technique, but they have inherent limitations in that they operate on an open loop kinematic chain for executing such tasks with respect to the work environment. That is, they cannot utilize accurate information for execution, mainly due to a lack of capability of sensing the position and orientation of the end point of the robotic system with respect to the object. To overcome this problem, visual sensing is being widely used, since it is known to be the most efficient and useful sensing method among all other available sensors developed to date. It mimics the human sense of vision, and provides such characteristics as wide sensing range, fast measurement, and noncontact sensing. Accordingly, for the task execution utilizing robotic systems, visual servoing or feedback has received increasing attention ever since its inception in the early 1980s. The concept of the camera motion-based visual feedback control is illustrated by using a schematic drawing in Figure 6.96. Suppose that we want to grasp an object by using a hand-held camera system. It is assumed that a target (object) image viewed from the camera is given as shown in the figure. The objective of the visual feedback is to control the motion of the camera system such that the camera image obtained at the current position eventually becomes the target image at the final stage. The problem is how to control the camera motion based on the visual information obtained from the camera. In this sense, visual feedback is the fusing technology which combines the results from high speed image processing and mechanical dynamic motion control. A visual feedback system can be categorized into two groups depending on camera configuration, and also two groups depending on control architecture. In camera configuration, these are two modes; the one fixed in the work space and the other mounted on the end effector, as shown in
TABLE 6.9 Practical Applications of Visual Servoing Application area
Role of Visual Servoing
Assembly of macro parts Assembly of MEMs parts Packaging of MEMs parts Object tracking Object manipulation Optical packaging Parts inspection
Parts guide and positioning Parts guide and positioning Part guide and positioning Object recognition and motion tracking Part recognition and grasping/handling Part recognition/alignment Part positioning, recognition
Characteristics Accuracy Accuracy Speed Accuracy Accuracy, speed Accuracy, speed
Basic Optomechatronic Functional Units
413 camera desired camera frame XC
moving
u
YC camera
ZC
current camera frame
ZC
YC
u
v
desired camera image
XC
v
object
current camera image FIGURE 6.96 The camera motion-based visual feedback control.
Figure 6.97a,b, respectively. When a camera is fixed in the workspace as shown in Figure 6.97a, the camera is related to the coordinates system of the robot by R TC and to the object (work piece) C TO : In this case, the relation between the pose of the camera and that of the robot’s end effector is image
image
camera {C}
OC
{E} end-effector
OC
{C} camera
robot{R}
robot {R} {W}
(a) fixed camera system
object
{W}
(b) eye-in hand system
FIGURE 6.97 Two typical system configurations for visual feedback.
{E} end-effector
object
414
Optomechatronics
unknown and varying with time. Here, E T represents a coordinate’s frame with respect to the frame E, which is shown in the figure. In the eye-in hand configuration (which is the case of the camera mounted on the robot’s end effector), shown in Figure 6.97b, the relationship between the pose of the camera and that of the end effector is known, and often constant. If this relationship is represented by the pose expressed in the coordinates of the camera relative to the end effector, it can be expressed by E TC ; which indicates the location of the origin of camera frame C with respect to the end effector’s frame E. It is noted here that the target image is independent of the motion of the robotic system. Visual Feedback Control Architectures As schematically shown in Figure 6.98, there are two classes of viewing the control architecture: position-based control and image-based control. This is grouped by the view point at which visual information is utilized for the control. The position-based control uses the information of the object pose with respect to the camera. The pose information is obtained from the image acquired at every instant from the camera. We then relate those values to the pose of the object in the world coordinates by using the coordinate transformation discussed in Chapter 3 in the section of “Camera camera
wX
d
+ _
cartesian controller joint controllers amplifiers
wXˆ
robot
joint angle sensors pose estimation
image feature extraction
video
(a) position-based control camera
+
_
feature space controller joint controllers amplifiers
robot
joint angle sensors image feature extraction
(b) image-based control FIGURE 6.98 Two architectures for visual feedback control method.
video
Basic Optomechatronic Functional Units
start
start
input desired image & extract desired features
input desired pose capture current image from camera extract features of current image estimate current pose
415
capture current image from camera command calculated velocity by control law
extract features of current image
command calculated velocity by control law calculate image Jacobian
calcuate feature errors calculate pose error pose error e < δ (user defined)
no
yes end
(a) position-based visual servoing
feature errors, e < δ (user defined)
no
yes end
(b) image-based visual servoing
FIGURE 6.99 Flow charts for two visual servoing methods.
Calibration”. This transformation is carried out at every instant of control. To do this, we extract object features associated with the geometry of the object from the image. Once the visual pose information denoted by w X^ as shown in the figure is obtained, this is compared with a desired pose value w Xd to determine an error defined in the pose space (World coordinates). The error thus obtained is fed back to the World controller, which, in turn, actuates the motion of a robotic system. This control action is repeated until the error is eliminated, or reaches the desired value. In contrast to this, the image-based control method uses the prespecified image features of the object for computing control input without computing the position of the object. This reduces a great deal of computational time and simplifies control methodology. The flow charts shown in the figure 6.99 clearly compare the two control schemes. Feature parameters of an image in the image plane include point, lines, corners, the area of its projected surface, the periphery of its edge, and so on. The choice of these parameters influences the accuracy and speed of visual feedback. To illustrate how the visual feedback control can be accomplished in actual practice, we will consider the image-based visual feedback control. In this control, it is necessary to relate the variation of the feature values obtained in the image to the motion of a robotic system (manipulator), in order to generate the control signal at any instant of time. Therefore, determining the velocity of a robotic system is required to move the robotic system in order to obtain the desired value of the feature values f defined in either pixel or image coordinate discussed in Chapter 3, “Camera Calibration.” To this end, let us define image Jacobian, J defined by ›f/›r,
416
Optomechatronics
which is interpreted as a sensitivity matrix of the feature values, f with respect to position of the end-effector defined in task (world) space, r. This Jacobian can be used to relate the motion of the robotic systems to the variation of feature in camera image with respect to time. The relation can be expressed as df dr ¼ JðrÞ dt dt where: J is a k £ l dimension and given by 3 2 ›f 1 ›f 1 ··· 6 ›r ›r l 7 7 6 1 7 6 6 . ›f ðrÞ .. .. 7 7 JðrÞ ¼ ¼6 . 6 . . . 7 ›r 7 6 7 6 4 ›f k ›f k 5 ··· ›r 1 ›r l
ð6:86Þ
ð6:87Þ
and l and k are the dimension of the task space and that of the feature space, respectively. In this relationship, the rate of feature variation df =dt and the velocity of the robot motion dr=dt are respectively defined by 3 2 du1 6 dt 7 7 6 7 6 6 dv1 7 7 6 6 dt 7 7 6 2 3 7 6 6 . 7 VX 6 .. 7 7 6 7 6 7 6 6 VY 7 6 du 7 7 6 j 7 6 6 7 7 6 6V 7 6 dt 7 dr 6 Z7 df 7; 7 ¼6 ¼6 7 6 6 7 dt 6 dvj 7 dt 6 vX 7 7 6 6 7 6 dt 7 7 6 7 6 6 vY 7 7 6 5 4 6 . 7 6 .. 7 vZ 7 6 7 6 6 du 7 k 7 6 7 6 6 dt 7 7 6 4 dv 5 k
dt
It is noted here that the feature velocity vector is specified for the number of the feature points, k/2, and dr=dt denotes the translational and rotational velocities of the end effectors with respect to camera coordinates. The relationship expressed in Equation 6.87 indicates that, if the change in the pose of the robotic system is given, the change in the image feature can be obtained accordingly, and vice versa. For the visual feedback control, therefore, we need to determine the rate of the motion dr=dt at every instant,
Basic Optomechatronic Functional Units
417
camera OC C C
WE
end-effector {E} O E robot {R}
W C
Pj
point P C
E
VE
camera {C}
fixed frame
PE
{C} W
j
WC
C
Pj
PC
robot {R}
Pj
OC
W
VC point P j
{W}
{W}
(a) coordinates systems for fixed camera
object
(b) coordinates systems for the eye-in hand system
FIGURE 6.100 Coordinate system for the visual feedback control.
so as to make the current feature values rapidly approach the desired value of the given feature. To illustrate the implementation of this underlying concept, we will consider two camera configurations: (1) fixed camera system. (2) eye-inhand system. We will deal with the fixed camera system first. Fixed Camera Configuration Let us suppose that the end-effector is moving with transnational velocity V E and angular velocity C VE which are defined with respect to the camera frame. Referring to Figure 6.100a, they are described by C
C
V E ¼ ½VX ; VY ; VZ
and
C
VE ¼ ½vX ; vY ; vZ
Let Pj be a point attached to the end-effector frame expressed by C
Pj ¼ ½C Xj ; C Yj ; C Zj
T
¼ ½XC ; YC ; ZC
T
From the figure, it can be seen that the displacement vector C Pj with respect to camera frame is given by C
Pj ¼ C PE þ C TE E Pj
ð6:88Þ
where E Pj denotes the displacement vector of point Pj with respect to endeffector frame {E}, C TE is the translation matrix of the end-effector frame {E} with respect to camera frame {C}, and C PE is the displacement vector of the origin OE of end-effector frame {E} with respect to camera frame {C}. Differentiating both sides of Equation 6.88, we have dC P j ¼ C V E þ C VE £ C Pj dt where C VE is the rotation of the end effector with respect to the camera coordinate.
418
Optomechatronics
This relationship enables us to obtain the translational velocity of a point at the end-effector with respect to the camera coordinate system which is given by 3 2 dXC 3 6 dt 7 2 7 6 vY ZC 2 vZ YC þ VX 7 6 6 dYC 7 6 7 7 ¼ 6 vZ XC 2 vX ZC þ VY 7 6 ð6:89Þ 7 4 6 5 6 dt 7 7 6 vX YC 2 vY XC þ VZ 4 dZ 5 C dt These values will now need to be related to those measured in the image coordinate systems. To derive the relation, we will consider the coordinate systems shown in Figure 6.101a. Suppose that a point M is positioned in the camera coordinates, as shown in the figure. And let a point m positioned at the coordinates ðui ; vi Þ be the corresponding projected point of M in the image coordinates. In Chapter 3, “Camera Calibration” we have assumed the image coordinates to camera coordinates YC
XC
(CR E,CPE)
vi
f
XE OC
ZE
ui
Oi
ZC
m
OE
M(Xc,Yc,Zc)
end-effector coordinates YE
(a) end-effector coordinates viewed from camera coordinates vp
vi m
v0
oi
θ op
u0
image coordinates ui pixel coordinates up
(b) image vs. normal coordinates FIGURE 6.101 Relating the image motion to the end-effector motion.
Basic Optomechatronic Functional Units
419
be orthogonal in deriving the relationship between the pixel coordinate and image coordinates are frames. But, in more general cases of CCD pixels, since the pixel coordinates are skewed due to fabrication limitation, they are often not orthogonal each other. The pixel coordinates ðup ; vp Þ are skewed with angle u; and are related to the normalized image coordinates system expressed by ðui ; vi Þ at a point Oi, as shown in Figure 6.101b. From the geometric relationship between the two, the transformation equation was determined by u~ p ¼ ½Hu~ u~ i
ð6:90Þ
where u~ p is the augmented pixel coordinate frame, u~ i is the augmented image coordinate frame, and ½Hu~ is the transformation matrix, which is slightly different from the ½Hu~ defined in Equation 3.53. If ½Hu~ is substituted, Equation 6.90 becomes 32 3 2 3 2 ku 2ku cot u u0 ui up 76 7 6 7 6 6 7 6 vp 7 ¼ 6 0 ð6:91Þ kv =sin u v0 7 5 4 vi 5 4 5 4 1
0
0
1
1
where ku and kv are the scaling factors given in Chapter 3, “Perspective Projection.” To express the relationship in the camera frame in term of the normalized image coordinates, we need to use the perspective projection equations as previously defined in the same section mentioned above, " # " # ui XC f ¼ ð6:92Þ ZC YC vi where f is the focal length of the objective lens of the camera, as shown in the figure. Substituting Equation 6.92 into Equation 6.89, we obtain 3 2 3 2 vi ZC dXC v v Z 2 þ V Y C Z X 7 f 6 dt 7 6 7 6 7 6 7 6 7 6 7 6 dYC 7 6 ui ZC 7 6 7¼6 v 2 v Z þ V ð6:93Þ 6 Z X C Y 7 6 7 6 f 7 6 dt 7 6 7 6 7 7 4 dZ 5 6 4 ZC 5 C ðvi vX 2 ui vY Þ þ VZ dt f This means that once an image is defined and acquired in the image plane, it can be easily expressed in the normalized coordinates. If we use the following relationships ZC
dXC dZC 2 XC dt dt Z2C
dui d X ¼ f C dt dt ZC
¼f
dvi d Y ¼ f C dx dt ZC
dYC dZC ZC 2 YC dt dt ¼f Z2C
and ð6:94Þ
420
Optomechatronics
the motion of the point m in image coordinates is expressed in terms of the motion variable, C V E and C VE of point M defined in camera coordinates system. Substituting Equation 6.94 into Equation 6.93 we obtain the following equation 3 2 VX 7 36 6 VY 7 3 2 f 2 2 2 7 6 u u v f þ u i dui 7 0 2 i 2 i i 2vi 76 7 6 6 dt 7 6 V f ZC ZC f 76 Z 7 7 6 6 7 ð6:95Þ 7 6 6 7¼ 6 7 76 4 dvi 5 6 56 vX 7 4 f vi f 2 þ v2i ui vi 7 6 0 2 2 ui 6 7 dt ZC ZC f f 6 vY 7 5 4
vZ This relation implies that if the rate of the feature vector motion is known, the motion of the end effector can be specified with respect to camera coordinates, and vice versa. Eye-In-Hand Configuration Suppose that a camera mounted on the end-effector moves with the velocities W
VC ¼ ½vX ; vY ; vZ
W
V C ¼ ½VX ; VY ; VZ
where W VC and W V C are defined, respectively, by the angular velocity and transnational velocity of the camera/end-effector with respect to the world coordinate in task space. Let Pj be a point attached to the end-effector. The velocity of the point Pj with respect to the camera frame is expressed by dC Pj ¼ 2W V C 2 W VC £ C Pj dt
ð6:96Þ
In a way similar to that used for the case of the fixed camera system, we obtain the following relationship between the velocity of the camera motion and the feature change obtained in image coordinates 3 2 VX 7 36 6 VY 7 3 2 2 2 2 7 6 f u u v 2f 2 u i i i i dui 7 2 0 vi 7 6 7 6 6 dt 7 6 V ZC ZC f f 76 Z 7 7 6 6 7 7 ð6:97Þ 6 6 7¼ 6 76 7 4 dvi 5 6 56 vX 7 4 f vi f 2 þ v2i ui vi 7 6 2 2ui 6 0 2 7 dt ZC ZC f f 6 vY 7 5 4
vZ
Basic Optomechatronic Functional Units
421
Up to this point, we considered only a single feature point for visual feedback control. In actual cases, we need to use many different feature points to characterize the object to be handled. The above expression can now be extended to more general cases. If the number of feature points is k/2, the above relationships become: 3 f u1 u1 v1 f 2 þ u21 2 0 2 v 6 du1 1 7 72 3 6 ZC1 ZC1 f f 7 V 6 dt 7 6 X 7 7 6 6 76 7 7 6 6 2 2 76 7 7 6 6 f v f þ v u v 1 1 1 1 6 dv1 7 6 0 2 2 2u1 7 VY 7 76 7 6 6 7 ZC1 ZC1 f f 76 6 dt 7 6 6 76 7 7 6 6 76 VZ 7 7 6 6 7 6 . 7 6 .. .. .. .. .. .. 7 6 7 6 . 7¼6 7 . . . . . . 7 76 6 . 7 6 7 76 7 6 6 6 vX 7 7 7 6 6 6 7 2 2 7 6 duk=2 7 6 f þ u 7 6 uk=2 uk=2 vk=2 f 7 7 6 6 k=2 7 7 62 6 2 vk=2 76 0 v 6 Y 74 7 6 dt 7 6 ZC;k=2 ZC;k=2 f f 5 7 7 6 6 7 7 6 6 7 vZ 4 dvk=2 5 6 7 6 vk=2 f 2 þ v2k=2 uk=2 vk=2 f 5 4 0 2 2 2uk=2 dt f f ZC;k=2 ZC;k=2 2
3
2
ð6:98Þ for the eye-in-hand configuration. This derivation completes relating the camera motion velocity to the feature change in image. Feedback Controller Design Now with the known relationship between the motion velocity and the image feature change, we can determine the end-effector velocity at any control instant that can lead to reaching a desired image feature vector. Let the velocity vector in the right-hand side of Equation 6.98 be denoted by u, which is regarded to be the control command inputs to the motion controller of joint servo motors. Assuming that there exists a pseudo inverse J þ, u can be written from Equation 6.98 by u ¼ Jþ ðrÞ
df dt
ð6:99Þ
where Jþ is given by Jþ ¼ ðJT JÞ21 JT
ð6:100Þ
and f represents the feature points in the image coordinate. If the controller employs a proportional control law, for simplicity here, u in Equation 6.99 can be expressed by u ¼ K p Jþ ðrÞð f d 2 f Þ
ð6:101Þ
422
Optomechatronics
where f d is the desired value of f, f d 2 f denotes the error in features, and K p is the proportional controller gain. Here, we will concentrate on point features for visual servoing. The remaining part of this section will be devoted to confirming the above visual feedback concept by illustrating how to solve actual problems. To this end, let us consider a visual feedback system for servoing a camera towards a square block shown in Figure 6.102a. The right hand side of the figure shows the current image of the four feature points of an object obtained at a certain camera location, while the final image of the corresponding feature obtained at another camera location is shown in the left-hand side of the figure. The feature points here denote the characteristic point that can best represent the pose of the object to be tracked. At the desired location, we will assume that the camera sees the feature points ð f1 ; f2 ; f3 ; f4 Þ defined in the desired camera coordinate systems {D} as depicted in the figure. Notice that the image plane is denoted by the pixel coordinates ðup ; vp Þ; and that ðu0 ; v0 Þ is desired camera frame
u v
Ycr
{D} f3d
f4d
XD YD
f2d
current camera frame {cr} u Xcr f3c
ZD
f1d Zw
desired camera image
v
Zcr
f2c
Yw
f3 f2
f1
Xw {W} feature point
camera camera frame
XD
YD
up
virtual image plane vp
{D}
(u0, v0 )
f3
f2
f4 ZD
f1
(b) feature points defined in the camera coordinate system
FIGURE 6.102 Visual feedback control for a square block.
f1c
current camera image
f4
(a) current image vs. desired image
f4c
{W}
Basic Optomechatronic Functional Units
423
the center point of the image coordinates. The problem is to find a control input signal which drives the joint motors holding the camera module so as to locate the camera at the desired position. The camera intrinsic parameters and the feature points are given, respectively, by intrinsic parameters: f ¼ 0:016; av ¼ 1955; av ¼ 1940; u0 ¼ 320; v0 ¼ 240; u ¼ 908 desired feature points in world coordinate frame: f1 ðX1 ; Y1 ; Z1 Þ ¼ ½ 0:015 20:015 0 T ;
f2 ðX2 ; Y2 ; Z2 Þ ¼ ½ 20:015 20:015 0 T ;
f3 ðX3 ; Y3 ; Z3 Þ ¼ ½ 20:015 0:015 0 T ;
f4 ðX4 ; Y4 ; Z4 Þ ¼ ½ 0:015 0:015 0
T
In addition, transformation of the current camera frame with respect to the desired camera frame, that is, the two coordinate frames will be assumed to be at some control instant related by a rotation Rða; b; gÞ which is given by 32 3 2 32 cosa 2sina 0 1 0 0 cosb 0 sinb 6 76 7 76 6 7 76 Rða; b; gÞ ¼ 6 1 0 7 54 0 cosg 2sing 5 ð6:102Þ 4 sina cosa 0 54 0 0
0
1
2sinb 0 cosb
0 sing
cosg
For roll: g ¼ 28; pitch: b ¼ 38; yaw: a ¼ 158; and translation T ¼ ½0:01; 0:01; 20:03 : The transformation between two coordinate frames is determined by 2 3 0:9646 20:2569 0:0596 0:01 6 7 6 0:2585 0:9658 20:0202 0:01 7 6 7 D 7 Tcr ¼ 6 ð6:103Þ 6 7 6 20:0523 0:0349 7 0:9980 20:03 4 5 0 0 0 1 In actual servoing, this information is determined at every instant of time. According to the coordinates shown in the figure, transformation of the world coordinate frame with respect to the desired camera frame gives 2
1
0
0
0
3
6 7 6 0 21 0 7 0 6 7 D 7 TW ¼ 6 6 7 6 0 0 21 0:3 7 4 5 0 0 0 1 It is noted that the ZD axis (optical axis) of the desired frame coincides with the zw axis of the world frame, facing against each other. Given the condition in the above formula, we need to transform the feature points viewed by the desired frame and by the current frame into those seen in the image coordinate frame in order to determine the Jacobian given in Equation 6.87. To do this, we need to follow the following procedures step by step.
424
Optomechatronics
(1) Determine the camera perspective matrix: In the case of the skewed pixel coordinate system, C matrix in the perspective model given in Equation 3.55 is modified as 2
au
6 6 ½C ¼ 6 0 4
0
au cot u u0 av v0 sin u 0 1
0
3
7 07 7 5
ð6:104Þ
0
where u is the skew angle. Plugging the given parameter values into the above Equation 6.104 yields, we have 2
1955
6 ½C ¼ 6 4 0
0
320
1940
240
0
1
0
0
3
7 07 5 0
(2) Determine the coordinate values of the desired image feature points in the pixel coordinates: Let the values be denoted by ½p1d ; p2d ; p3d ; p4d . Utilizing the above perspective matrix obtained and the homogeneous transform matrix of the world coordinate frame with respect to the desired camera frame, DTW, we can compute the coordinate values of the desired frame in the pixel coordinates in the following manner s1 p1d ¼ CD TW f1 ¼ CD TW ½ 0:015 s1 p1d ¼ s1 ½pT1d l1
T
¼ ½ 125:32
20:015
101:1
0:3
0
1
T
¼ ½ 125:32
101:1
T
0:3
T
s1 ¼ 0:3 where s1 ¼ Zc is the scale factor in homogeneous coordinates. This leads to obtaining p1d ¼ ½ 417:75
337
T
Similarly, s2 p2d ¼ CD TW f2 ¼ CD TW ½ 20:015 s2 p2d ¼ s2 ½pT2d l1
T
¼ ½ 66:675
101:1
20:015 0:3
0
1
T
T
¼ ½ 66:675
¼ ½ 66:675
101:1
0:3
T
s2 ¼ 0:3 s3 p3d ¼ CD TW f3 ¼ CD TW ½ 20:015 0:015
0
1
42:9
0:3
T
T
Basic Optomechatronic Functional Units
s3 p3d ¼ s3 ½pT3d l1
T
¼ ½ 66:675 42:9
425 T
0:3
s3 ¼ 0:3 s4 p4d ¼ CD TW f4 ¼ CD TW ½ 0:015 s4 p4d ¼ s4 ½pT4d l1
T
0:015
¼ ½ 125:32 42:9
0 1
T
¼ ½ 125:32
42:9
0:3
T
T
0:3
s4 ¼ 0:3 Therefore, the feature points in the desired image are expressed in the pixel corrdinates by p1d ¼ ½ 417:75
337 T ; p2d ¼ ½ 222:25
337 T ;
p3d ¼ ½ 222:25
143 T ; p4d ¼ ½ 417:75
143
T
(3) In order to determine the Jacobian matrix, we further need to transform the feature points f defined in the world frame into the pixel points in the current image frame by using the homogeneous transformation between the desired camera frame and the current camera frame given in above Equation 6.103. If the transformation from the desired to the current frame 21 , four corresponding points expressed in the current is denoted by DTcr camera frame can be obtained. Since, 21 D s01 p1c ¼ CD Tcr TW f 1 21 D ¼ CD Tcr TW ½0:015 20:015 0 1
¼ C½0:0112 0:0151 0:330 1 ¼ ½ 83:6 108:3 0:330
T
T
T
Hence, p1c ¼ ½ 253:75 328:65 T : Similarly, 21 D s02 p2c ¼ CD Tcr TW f 2 21 D ¼ CD Tcr TW ½20:015 20:015 0 1
¼ ½ 26:48 122:83 0:328 p2c ¼ ½ 80:78 374:75
T
T
T
¼ C½20:0401 0:0228 0:328 1
426
Optomechatronics
21 D s03 p3c ¼ CD Tcr TW f3 21 D ¼ CD Tcr TW ½20:015 0:015 0 1
¼ ½11:513 66:766 0:328 p3c ¼ ½35:06 203:33
T
¼ C½20:0479 20:00621 0:328 1
T
T
21 D s04 p4c ¼ CD Tcr TW f4 21 D ¼ CD Tcr TW ½0:015 0:015 20:3 1
¼ ½68:657 52:242 0:330 p4c ¼ ½207:95 153:23
T
¼ C½20:0189 20:0139 0:330 1
T
T
In summary, in terms of the pixel coordinates the feature points can be expressed as p1c ¼ ½ 253:75 328:65 T ;
p2c ¼ ½80:78 374:75
T
p3c ¼ ½ 35:06 203:33 T ;
p4c ¼ ½207:95 153:23
T
And the depth of field of each feature point defined in the current frame is Z1 ¼ 0:330;
Z2 ¼ 0:328;
Z3 ¼ 0:328;
Z4 ¼ 0:330
Finally, these feature points need to be expressed in terms of the image coordinate frame, because the image Jacobian is expressed in that frame. They are obtained as ½u1c v1c ¼ ½ 20:542 £ 1023 0:731 £ 1023 ; ½u2c v2c ¼ ½ 22:0 £ 1023 1:1 £ 1023 ; ½u3c v3c ¼ ½ 22:3 £ 1023 0:3 £ 1023 ; ½u4c v4c ¼ ½ 20:917 £ 1023 20:716 £ 1023
Basic Optomechatronic Functional Units
427
Therefore, we finally obtain J in Equation 6.97, which can be calculated by 2
f 0 6 6 Z1 6 6 6 f 6 0 6 6 Z 1 6 6 6 f 6 0 6 6 Z2 6 6 6 f 6 6 0 6 Z 2 6 [J ¼ 6 6 f 6 6 6 Z3 0 6 6 6 f 6 6 0 6 Z 3 6 6 6 6 f 6 6 Z4 0 6 6 6 f 4 0 Z4 2
u 2 1c Z1
u v 2 1c 1c f
2
v1c f 2 þ v21c 2 f Z1
2
u2c Z2
2
v2c f 2 þ v22c 2 Z2 f
2
u3c Z3
2
v3c f 2 þ v23c 2 Z3 f
2
u4c Z4
2
v4c f 2 þ v24c 2 Z4 f
2
2
2
u2c v2c f
u3c v3c f
u4c v4c f
3 f 2 þ u21c 2v1c 7 7 f 7 7 7 u1c v1c u1c 7 7 7 f 7 7 2 2 7 f þ u2c 7 2v2c 7 7 f 7 7 7 u2c v2c 7 u2c 7 7 f 7 7 7 2 2 f þ u3c 7 2v3c 7 7 f 7 7 7 u3c v3c 7 u3c 7 7 f 7 7 7 2 2 f þ u4c 7 2v4c 7 7 f 7 7 7 u4c v4c 5 u4c f
4:85 £ 1022
0
1:64 £ 1023
0
4:85 £ 1022
2:17 £ 1023
6 6 6 0 4:85 £ 1022 22:22 £ 1023 6 6 6 4:89 £ 1022 0 6:10 £ 1023 6 6 6 0 4:89 £ 1022 23:35 £ 1023 6 ¼6 6 6 4:89 £ 1022 0 7:01 £ 1023 6 6 6 0 4:89 £ 1022 9:15 £ 1024 6 6 6 6 4:85 £ 1022 0 2:80 £ 1023 4
2:48 £ 1025
1:60 £ 1022
27:31 £ 1024
3
7 7 21:60 £ 1022 22:48 £ 1025 25:42 £ 1024 7 7 7 1:38 £ 1024 1:63 £ 1022 21:10 £ 1023 7 7 7 22 24 23 7 21:61 £ 10 21:38 £ 10 22:00 £ 10 7 7 7 24:31 £ 1025 1:63 £ 1022 3:00 £ 1024 7 7 7 21:60 £ 1022 4:31 £ 1025 22:3 £ 1023 7 7 7 25 22 24 7 7 24:10 £ 10 1:61 £ 10 7:16 £ 10 5 22 25 24 21:60 £ 10 4:10 £ 10 29:17 £ 10
If we define error by e ¼ f d 2 f in the image coordinates, determine the velocity command inputs to the six axis motion controllers at this particular servoing instant. To compute the error values we need to express the features obtained in the pixel coordinates in terms of the image coordinate values. These are obtained by ½ u1d
v1d ¼ ½ 8:0 £ 1024
½ u2d
v2d ¼ ½ 28:0 £ 1024
8:0 £ 1024 ; 8:0 £ 1024
428
Optomechatronics
½ u3d
v3d ¼ ½ 28:0 £ 1024
½ u4d
v4d ¼ ½ 8:0 £ 1024
28:0 £ 1024 ; 28:0 £ 1024
Once these feature values defined in the image coordinates are obtained, we can determine the errors in the feature values, by 2 3 u1d 2 u1c 6 7 6 7 .. e ¼ fd 2 f ¼ 6 7 . 4 5 v4d 2 v4c Then the velocity command inputs are computed from Equation 6.101 u ¼ K p Jþ e ¼ K p ðJT JÞ21 JT e ¼ ½ 0:267
0:135
0:00309
0:394 20:711
0:312
T
which completes the computation of the control inputs that are required to drive the six-motor joint mechanisms.
Optical Signal Transmission As mentioned previously, the advantage of fiber optic transmission is that it provides rapid transmission of data and immunity to noise . In addition, optical fibers are compact and flexible. For this reason, optical data transmission is widely used when data/signals obtained from sensors area subject to external noise, when the amount of data needed to be sent is vast, or when an operation is operated at remote sites. Operation of systems at a remote site is ubiquitous, nowadays. Remote operation of a machine, visual servoing of a robot operated in a remote site, internet-based monitoring, inspection, and control are some of such developments. Signal Transmission A transmission system utilizes several techniques of modulation in transmitting optical signal through a fiber. Figure 6.103 shows a schematic of the intensity modulation based system. It consists of an LED(s) for emitting a light signal of data, and an LED(r) for emitting another light signal of reference for correcting errors occurring during transmission. In addition, it has a pair of current drives for two LEDs, and an optical multiplexer. A receiver has a photodetector (PD(s)) for detecting a light signal of data, a photodetector (PD(r)) for the light signal of reference, a pair of current amplifiers for two photo detectors, an optical demultiplexer, and an analog multiplexer.
Basic Optomechatronic Functional Units
429
transmitter Vi
current drive
LED(s)
receiver
optical multiplexer Vr
current drive
LED(r)
Pi(s)
Po(s)
Po(r)
optical fiber
PD(s)
optical demultiplexer Pi(r)
PD(r)
current amplifier
Vo(s)
analog multiplexer current amplifier
Vo
Vo(r)
FIGURE 6.103 Configuration of the optical fiber transmission system.
The transceiver system simultaneously transmits the light signal of the data and another signal of the reference from the transmitter to the receiver. In the transmitter, the voltages Vi and Vr the input and reference voltages, respectively, are converted to the forward currents of LED(s) and LED(r) by their respective current drives. As a result, the optical output powers are generated from the LEDs, which, in turn, are multiplexed by the optical multiplexer. At the receiver side, the signals and reference lights whose powers are P(s) and P(r), respectively, are demultiplexed through the optical demultiplexer, then fed to PD(s) and PD(r) respectively. The photocurrents of the diodes are converted to electrical voltages Vo ðsÞ and Vo ðrÞ through the current amplifiers, respectively, which can be made in proportion to the P(s) and P(r) values. An optical transceiver containing the transmitter and receiver that can perform the operation explained in the above signal transmission is shown in Figure 6.104. Its role is to launch an optical signal emitted from an LD into the optical fiber and to receive transmitted signal from the fiber by a PD. It consists of a laser diode(LD), photodiode beam spliter and three lenses. The three lenses are used to collimate or focus the laser beam, while the beam splitter is used to couple lenses and other optical devices from or to optical fiber. It can be seen that the laser beam from the LD to the fiber and the transmitted signal input to the PD from the fiber pass through the beam splitter based on confocal scheme. Laser diode (LD) package has a laser diode, an embedded photodiode, and two lenses. In this package, the optical power is controlled by a feedback loop which utilizes the power measurement out of the photodiode. In the transmission system shown in Figure 6.103, Vo needs to be equal to Vi by adjusting the gains of the current drives and amplifiers when the optical fiber is kept straight. However, when the fiber is bent, their relationship is found to be given by Vo ¼
T1 ðl1 ; r1 ; u1 Þ V T2 ðl2 ; r2 ; u2 Þ i
where T1 and T2 are the transmittances of the optical powers passing through a pair of optical powers at wavelength l1 and l2 ; respectively. The transmittances are found to be also dependent upon the bending radius r and
430
Optomechatronics
beam splitter
LD laser diode package
optical fiber PD
lens
lens lens PD
FIGURE 6.104 Configuration of the optical transmitter.
bending angle, u. At the optical connector, coupling loss should be minimized, and distortion of optical signal should also be minimized during the transmission. Figure 6.105 illustrates a full-duplex fiber optic transmission link for a remote operation of a group of machines (robots) controlled by several servo motors. The system carries feed-forward and feedback digital signals over a distance of many meters between the control site and operation site. The control site executes a control algorithm with controller gain dedicated to each motor. At the remote site, this control command is executed via motor power switching circuits, D/A buffer circuits for feedback and feedforward control, and A/D buffer circuits for feedback signals from the angular displacement sensor. Each motor control circuit in the control site is connected to a control computer which provides scheduling of the machine operation, programs the controller gains, and provides the high-level control commands. The inputs to each control circuit from the computer are the sign of the commanded motor current and pulse-width modulation representing the magnitude of the command motor current. This information is sent out through the transmitters, the optical fibers, and
sign magnitude channel 1
channel n
control site
bidirectional interface circuit
transmitter
receiver
receiver
transmitter
machine (robot) sign magnitude channel 1 …
…
central control computer
digital motor control circuits
bidirectional interface circuit
channel n optical link
FIGURE 6.105 An optical transmission link for remote operation.
operation site
power and buffer circuits … channel 1 channel n
motor
…
Basic Optomechatronic Functional Units
431
the receivers, and then finally input to the power and buffer circuits. The fiber optic link has two optical fibers for each motor control; one for feedforward, one for feedback. At both ends, they are connected to identical bidirectional interface circuit boards, each of which contains a transmitter and a receiver. The transceiver A is connected to transceiver B through an optical fiber (for example, a few hundred meters core diameter 50 mm and cladding diameter 125 mm). Power Transmission and Detection In use of optical fibers, we need to consider the efficiency of the coupling between optical source and fiber, since the transmitted power is affected by the focusing system and the geometry of coupling. Figure 6.106 illustrates two cases of coupling; from source to fiber, and from fiber to detector. Therefore, two coupling efficiencies will be taken into consideration. Here, it will be assumed that an optical system focuses the light beam in such a way that the incident angle is equal to half the cone of acceptance of the fiber, which ensures internal reflection. When the coupling of source to fiber shown in Figure 6.106a is considered, it is possible to compute an approximate value of the coupling coefficient Csf. The coefficient is defined as W ðlÞ Csf ¼ i ð6:105Þ SðlÞ where Wi ðlÞ is the power incident on the entire surface of the fiber per unit wave length, and SðlÞ represents the power emitted from the source per unit wave length at l[7]. The geometry of rays incident to the fiber indicates that rays with the cone of acceptance will be transmitted and internally refracted through the core, if Rco is smaller than the source radius Rs : In other words, the Rs is essentially smaller than the aperture of the focusing lens. The effective source area becomes Aeff ¼ pR2a ¼ pd2i tan2 ½ui
max
collecting lens
from source Rs
[q i]max
coupling unit detector fiber
fiber Rco
RD
Rco
[q i]max di
(a) coupling between source and fiber FIGURE 6.106 Coupling of source-fiber and fiber detector.
[q i]max
(b) coupling between fiber and detector
432
Optomechatronics
where Ra is the effective aperture of the lens, di is the distance between lens and fiber, equal to f , and ½ui max is the maximum angle of acceptance. Then, Wi ðlÞ is expressed by Wi ðlÞ ¼ Aeff DuSðlÞ < p2 R2co tan2 ½ui
max SðlÞ
ð6:106Þ
where Du is the solid angle which the fiber core sustends. From Equation 6.106, the coupling coefficient is obtained approximately by Csf ¼
Wi ðlÞ < p2 R2co ðNAÞ2 SðlÞ
ð6:107Þ
where NA ¼ sin½ui max . The coupling of fiber to detector can be obtained in a similar way; the coefficient CfD is simply CfD ¼
WD ðlÞ ¼ 1 for RD . Rco Wf ðlÞ
ð6:108Þ
where RD is the radius of the detector, and Wf ðlÞ and WD ðlÞ are the power leaving the optical fiber and the power transmitted to the detector. In this situation, all the power leaving the fiber transmits through the optical unit of the detector. In case RD , Rco, the coefficient can also be computed in the same way as obtained in the case of source-to-fiber transmission. Light transmitted by a fiber attenuates optical power, and also loses some part of information due to several distortion mechanisms. They include model distortion, material distortion, and waveguide dispersion. The modal distortion occurs because rays take many different travel paths, thus reaching the output end in different times. When an input of a square wave is sent into a fiber, this distortion mechanism causes broadening of pulse. Material distortion occurs, because the refractive index is a function of wave length. There are some fibers that exhibit this phenomenon. Light normally contains several wavelengths, unless it is perfectly monochromatic, and thus, different wavelengths yield different speeds. Waveguide dispersion is a small effect that can occur, resulting from the variation of the refractive index with wavelength, even in the absence of material dispersions. Clearly, depending on the applications, we need to have a careful choice of fiber, and of light source, in order to reduce some loss of information being sent.
Problems P6.1. Consider a robot gripper driven by a PLZT optical actuator control problem shown in Figure P6.1. Gs ; and Guv ðsÞ and Ga ðsÞ are assumed to be
Basic Optomechatronic Functional Units
433
given by Guv ðsÞ ¼ 1
Gs ðsÞ ¼ 1
and
Ga ðsÞ ¼
2 3kl L2 aq ðb0 s þ b1 s þ b2 Þ 3 8w s þ a0 s2 þ a1 s þ a2
All parameters are given in Table 6.1. When a proportional controller is used, Gc ðsÞ ¼ kp ; and Gp ðsÞ is neglected then (1) What will be the steady state error to a step position, xd ðsÞ ¼1/s? (2) If an error is present for this input, how you can eliminate it? (3) In the above problems (1) and (2), the gripper dynamics Gp ðsÞ is neglected. If this cannot be neglected, what type of Gp ðsÞ will it be? If this Gp ðsÞ is included in the transfer function, what will be the effect of the inclusion?
xd
controller
UV radiator and motor
optical actuator
gripper
Gc(s)
Guv(s)
Ga(s)
Gp(s)
x
Gs(s)
FIGURE P6.1 The control system block diagram of the optically operated gripper.
P6.2. Consider the case when mass m2 moves the slope shown in Figure P6.2, while mass m1 sticks to the floor. Determine the condition for no slip between mass m2 and floor. moving SMA actuator m2 fixed m1 q FIGURE P6.2
P6.3. The configuration of the optical actuator employing photothermal effect, shown in Figure P6.3, uses the optical fibers forming three cantilevers whose legs are joined to an equilateral triangle body, as shown in Figure P6.3. The bottom of each leg is cut in a bevel shape and colored black so that it can absorb light to convert it to heat when light enters. If the light is on, this photothermal effect occurring at the bottom of the leg makes the leg bend
434
Optomechatronics light
body
moving fiber optical fiber with black color coated motion direction FIGURE P6.3 Schematic of the optical actuator moving in a straight line.
(deformed). Then, due to this deformed state, the leg stretches forward. Explain in detail how a straight line movement and two-dimensional movement can be generated by appropriately applying light to the fibers. P6.4. Consider the dynamic model at stage (2) shown in Figure 6.10. At this stage, the moving machine has one real foot in the state of locking, while its body stretches with the front foot free to move due to temperature rise in the shadowed area. (1) Draw a free body diagram in this case, as shown in the case of stage (4) of Figure 6.11. (2) Derive the condition of no-slip condition for the climbing-up motion of the machine. P6.5. If Bragg deflection angle 0:258 is required, what frequency of an acoustic wave needs to be generated? A red laser beam is used for an optical scanning purpose. The acousto-optic material is a doped glass listed in Table 6.4. P6.6. An acousto-optic crystal deflects the beam from the angle ui to ud : Suppose that the width of the aperture is 2 mm, and that the velocity of the acoustic wave in the crystal is 700 m/s. Determine the rise-time required to deflect the beam by acoustic wave. Assume that the beam fully fills the aperture of the crystal.
Basic Optomechatronic Functional Units
435
P6.7. Figure P6.7 shows a two-dimensional scanning method for reading or writing. It is composed of a single post-objective galvanometer scanner, an objective lens (not shown), and a moving medium. (1) Discuss a method that can achieve a flat scan field without using any additional optical systems. (2) Consider the case when we employ an active method which uses a z translator shown in Figure 6.42. We wish to correct the deviation of the scan focus length in an on-line fashion. Draw a control system block diagram using Figure P6.7, and explain the operation principle. In (2), devise a sensor that can measure the scan deviation at each instant of scanning. motor
scanner
beam from objective lens
moving
FIGURE P6.7 A configuration of two-dimensional scanning with one scanner.
P6.8. An x – y galvanometer scans through the whole region of a scan field (80 £ 80 mm area), as shown in Figure P6.8. Suppose that two mirrors are
80 A 80 y-axis scanner
−A −80
(0,0)
−80
FIGURE P6.8 Geometry of a galvanometer scanning.
400
mm
436
Optomechatronics
separated by 20 mm. The distance from y-axis mirror to scan plane is 400 mm. Compute the deviation of the focus along the line of AA0 at an interval of 10 mm. P6.9. Derive the simplified equation given in Equation 6.58, obtained by neglecting the effect of inductance of the electric circuit d2 u du þ 2zvn þ v2n u ¼ Kv2n eg 2 dt dt P6.10. Show that the damping ratio used in a galvanometer is 0.690 when the response curve intersects the magnitude u=us ¼ 0:95 at the time t ¼ 0:454ðvn t=2pÞ and meets the tangency point pt at upper accuracy limit, as shown in Figure 6.47. P6.11. Consider a single lens arrangement shown in Figure P6.11. If this lens with focal length f moves forward to a location P at which the image distance is 2f, what will be the magnification of the image size? Assume that object distance is much larger than the image distance. (1) Explain the results by drawing the ray path through the pin-hole model of the thin lens. (2) Describe the defocusing phenomenon when only a single lens is used for zooming.
object
lens
image screen
f initial position lens
2f location P FIGURE P6.11 Magnification change using a single lens.
Basic Optomechatronic Functional Units
437
P6.12. Suppose that a plus lens with a focal length f1 ¼ 15 is placed in front of the minus lens with a focal length f2 ¼ 210 shown in the Figure P6.12. They are located initially at P1 and M1, as shown in the figure. When the minus lens gradually moves backward to two other locations M2, M3, and the plus lens moves to two other locations P2, P3 from their initial position, the resulting focal length variations are shown in the figure. Note that the focusing points correspond to the film plane for all cases. (1) Explain the magnified image obtained at the images plane in case 2 and case 3 as compared with case 1. (2) In order to make the focal length of the imaging system 40mm, design the positions of the plus and minus lenses based on the figure. Explain the image magnification by comparing with the case of initial settings.
d =7.5
70 60
case1 P1
50
M1 d =10
40 30
case 2 P2
20 10
film plane
equivalent focal length (fequ), mm
80
M2
d =14
case 3 P3 50
40
30 20 distance from film (d), mm
M3 10
0
FIGURE P6.12 Focal length variation due to the lens movement in a two-component zoom.
P6.13. Consider a cam-driven mechanism for zoom control system. The relation between the lens position and the equivalent focal length are found to be as given in Table P6.13. We wish to achieve feq ¼ 15 mm from initial feq ¼ 20 mm: (1) Determine a function form (4th order polynomial in term of feq ) that describes the data given in the table. (2) Compute the distance of lens movement L2 and L3 from their initial positions.
438
Optomechatronics
(3) Express the above relation in terms of the rotation angle u of the cam mechanism. Assume the radius of barrel is R (Figure P6.13). TABLE P6.13 The Parameter Values Used for Autozooming d1,2 (mm)
d1,3 (mm)
fequ (mm)
17.33 16.33 15.33 14.33 13.33 12.33 11.33 10.33 9.33 6.33
13.46 12.70 11.87 10.96 10 6.96 7.85 6.62 5.20 3.33
12.98 13.52 14.02 14.51 15 15.52 16.12 16.87 17.96 20
radius: r
motor
f cam profiledL2
cam profiledL3
CCD
q fixed 1st lens
2nd lens
dL2
3rd lens dL3
radius: R M:1 reduction gear
image plane
FIGURE P6.13 A cam-driven system for auto-zooming.
P6.14. Consider an autofocusing system whose configuration is given in Figure P6.14. Suppose that a camera is autofocusing an image of a stripe pattern shown in Figure P6.14. The camera moves by an incremental distance dz ¼ 1 mm ranging from z ¼ 0 mm to z ¼ 8 mm:
Basic Optomechatronic Functional Units
439 z=0 z=1 z=2 z=3 z=4 z= 5 z=6 z=7 z=8 100 pixels 100 pixels
480 pixels
640 pixels FIGURE P6.14 The image of the stripe pattern obtained at various locations.
(1) The focus measure in each image is defined for the area within the window, as indicated, 100 £ 100 pixels. Determine the focus measure for the image obtained at each location. The measure is computed based on the following three operators. (1) Sobel operator, (2) Laplacian, (3) Weighted median. (2) Compare the results of the three methods by plotting the relationship of the focus value vs. camera location, and discuss the differences between them. P6.15. Suppose that a focus measure vs. camera position for a given human face is plotted in Figure P6.15. The distribution of the measure is assumed to be given by
FðzÞ ¼
1 1 exp 2 2 ðz 2 zÞ2 2ps 2s
The exact peak value of the measure needs to be determined by the hill climbing method, starting from zð0Þ ¼ 24:0: Assume that the parameters are at s ¼ 1:0; z ¼ 3:5; z0 ¼ 24:0; Dz ¼ 3:0; dz ¼ 1:0: (1) Show all the procedures needed to determine the peak measure. (2) Plot the camera position vs. time, and the corresponding measure vs. time.
Optomechatronics
focus measure F(z)
440
z
z0
F(z) =
−4.0
1 exp 2ps
1 (z-z)2 2s 2
3.5 camera position (z)
s = 1.0, z = 3.5, z0 = −4.0, ∆z = 3.0, δz = 1.0
FIGURE P6.15 Focus measure of human face image vs. camera position.
P6.16. Consider the illumination concept illustrated in Figure P6.16. Based on the concept, (1) discuss the effect of surface normal, incident angle, and the optical axis of the detection system – and then the relation between these three factors. (2) when the normal surface of an object is to be measured, how we can design an appropriate illumination system. camera ring illuminator
LED
(a) small incident angle
object (block)
(b) middle incident angle
(c) large incident angle
FIGURE P6.16 Three different illumination angles produced by a ring-type illuminator.
P6.17. Figure P6.17 shows a three-layer illumination system to detect the image of a PCB which is composed of several jointed areas of electronic parts, marks, and a background. The PCB to be imaged is shown in the figure.
Basic Optomechatronic Functional Units
441
camera
Contact part
top illumination middle illumination bottom illumination
Board mark
PCB board
(a) vision system for imaging
(b) PCB board to be inspected
FIGURE P6.17 Illumination system for detecting a PCB.
(1) What color of illumination light will be effective to detect the jointed areas and marks when the color of the joints is approximately similar to that of gold, while marks are in white. The camera used here is monocolor. (2) When the illumination ring in the bottom layer is on, what can we expect the image to look like? (3) When the top layer is on, discuss the characteristics of the image that can be detected. P6.18. Eleven fundamental functionalities have been discussed in Chapter 5. Following the methods illustrated in the chapter, draw each of the signal flow diagrams for the systems shown in Figure 6.16, Figure 6.24, Figure 6.34, Figure 6.42 and Figure 6.65. P6.19. Figure P6.19 shows a scene of visual servoing that drives an electric chip on a 6-axis stage to be located at a desired position with a desired pose. Therefore, the objective of this visual control is to move it from the current image in (a) to the desired image (b), as shown in Figure P6.19c. (1) Determine four feature points(corner points) of the electronic chip shown in Figure P6.19a by using Hough transform presented in Chapter 3, “Perspective Projection.” (2) Determine the image Jacobian at the current location by obtaining the feature points in the current image frame. Assume that the camera intrinsic parameters are given by: au ¼ 1955, av ¼ 1940, u0 ¼ 358, v0 ¼ 292, u ¼ 908, f ¼ 0.016. The chip is located 0.3 m away from the camera, along the optical axis of the camera. For simplicity,
442
Optomechatronics
assume that at this instant all the four corner points are located at the same vertical position z ¼ 0:3m. (3) From the image Jacobian obtained in the above question, determine the velocity command input u ¼ K p Jþ e where Kp is the identity gain matrix and J þ is the pseudo inverse of J and e is the error vector.
camera ring illumination
rectangular chip
y X
wX
wy
(b) desired image
wz Z 6 axis stage
(a) current image
desired
experimental rig
current
(c) current image error FIGURE P6.19 Visual servoing for an electronic chip.
P6.20. An acousto-optic deflector is used to diffract the incident light beam at a specified angle, as will be seen in the next chapter. The diffraction angle determines which photo-diode within the array will receive the diffracted beam as shown in Figure P6.20. The output of the diode will then enter the amplifier. lens
AOD
laser source
lens PD array
signal conditioning
q
to actuator amplifier
FIGURE P6.20 AO diffraction-based signal actuation.
Basic Optomechatronic Functional Units
443
(1) How many signal transforming modules are in the system? Explain them in detail. (2) How many signal modulation modules are in the system? Explain them in detail. (3) Draw a signal flow graph of this system, and give it a physical explanation. P6.21. Figure P6.21 shows an optical disk that works on the optomechatronic principle. (1) Explain its operation principle. (2) Repeat the same problems (1), (2) and (3) given in P6.20. disk photo-detector lens laser half-mirror
mirror
FIGURE P6.21 Optical storage disk.
P6.22. Figure P6.22 shows a digital camera. (1) Explain how it works. (2) Repeat the same problems (1), (2) and (3) given in P6.20. pento-penta prism view finder aperture
imaging plane
shutter zoom lens
focusing sensor
FIGURE P6.22 A camera.
References [1] [2]
Acousto-Optics A.A Sa, www.a-a.fr, 2005. Acousto-Optic Report, Crystal Technology, Inc., http://www.crystaltechnology. com/AOMO_app_notes.pdf, 2005.
444
Optomechatronics
[3] Acousto-Optic X-Y Scanning System, Electro-Optical Products Corp., http:// www.eopc.com/dtsxy100_dtsxy250_dtsxy400.html, 2005. [4] Akiyama, T., Collard, D., and Fujita, H. Scratch drive actuator with mechanical links for self-assembly of three-dimensional MEMS, Journal of Microelectromechanical Systems, 6:1, 10 – 17, 1997. [5] AO Application Notes, AA Opto-electronic, http://www.a-a.fr/doc/ AOApplicationnotes.pdf, 2005. [6] Atkins, R.A., Gardner, J.H., Gibler, W.N., Lee, C.E., Okland, M.D., Spears, M.O., Swenson, V.P., Taylor, H.F., McCoy, J.J. and Beshoun, G. Fiberoptic pressure sensors for internal combustion engines, Applied Optics, 33:7, 1315– 1320, 1994. [7] Bentley, J.P. Principles of Measurement Systems, 3rd Ed. Longmans, New York, 1995. [8] Bernacki, B.E. and Mansuripur, M. Causes of focus-error feedthrough in optical-disk systems: astigmatic and obscuration methods, Applied Optics, 33:5, 735– 743, 1994. [9] Choi, K.S., Lee, J.S., and Ko, S.J. New autofocusing technique using the frequency selective weighted median filter for video cameras, IEEE Transactions on Consumer Electronics, 45:3, 820– 827, 1999. [10] Ehrmann, J. S., Optics for vector scanning, Beam Defection and Scanning Technologies, SPIE Vol. 1454, pp. 245-255, 1991. [11] Ernest, O.D. Measurement Systems Application and Design, 4th ed. McGraw-Hill, New York, 1990. [12] Fan, K.C., Chu, C.L., and Mou, J.I. Development of a low-cost autofocusing probe for profile measurement, Measurement Science and Technology, 12:12, 2137– 2146, 2001. [13] Feddema, J.T. and Simon, R.W. CAD-driven microassembly and visual servoing, Proceedings of IEEE International Conference on Robotics and Automation, 2, 1212– 1219, 1998. [14] Fujita, H. and Toshiyoshi, H. Micro-optical devices, Handbook of Microlithography, Micromachining, and Microfabrication, Vol. 2, P.Rai-Choudhury, ed.,.1997. [15] Fukuda, T., Hattori, S., Arai, F., Matsuura, H., Hiramatsu, T., Ikeda, Y. and Maekawa, A. Characteristics of optical actuator-servomechanisms using bimorph optical piezo-electric actuator, Proceedings of IEEE International Conference on Robotics and Automation, 2, 618– 623, 1993. [16] Fukuda, T., Hattori, S., Arai, F. and Matsuura, H., Optical servo systems using bimorph PLZT actuators, Micromechanical Systems, 46, 13 – 20, 1993. [17] Fukuda, T., Hattori, S., Arai, F. and Nakamura, H. Performance improvement of optical actuator by double side irradiation, IEEE Transactions on Industrial Electronics, 42:5, 455–461, 1995. [18] Haralick, R.M. Computer and Robot Vision Volume I and II. Addison Wesley, Reading, MA, 1993. [19] Harshall, G.H. Optical Scanning. Book News, Inc, Portland, 1991. [20] Hoffmann, M., Kopka, P., and Voges, E. All-silicon bistable micromechanical fiber switch based on advanced bulk micromachining, IEEE Journal on Selected Topics in Quantum Electronics, 5:1, 46 – 51, 1999. [21] Hutchinson, S., Hager, G.D., and Corke, P.I. A tutorial on visual servo control, IEEE Transactions on Robotics and Automation, 12:5, 651– 670, 1996. [22] Introduction to Acousto-Optics, Brimrose Corporation of America, http:==www.brimrose.com, 2005.
Basic Optomechatronic Functional Units [23]
445
Joung, J., Kim, K., Kim, I.S. and Park, J. High-sensitivity fiber optic Fabry-Perot pressure sensors for medical applications, Proceedings of SPIE-Optical Engineering for Sensing and Nanotechnology(ICOSN 2001), 4416, 432– 435, 2001. [24] Kao, T.W. and Taylor, H.F. High-sensitivity intrinsic fiber-optic Fabry – Perot pressure sensor, Optics Letters, 21:8, 615– 617, 1996. [25] Matsuba, Y., Otani, Y., and Yoshizawa, T. Two-dimensional movement of photothermal actuator composed of optical fibers, Proceedings of SPIE, 4902, 78– 82, 2002. [26] Mitutoyo Catalog No. E4181, E4142-359, E4242-361, and E4214-361, Mitutoyo Corp., http://www.mitutoyo.com/, 2005. [27] Nussbaum, A. Optical System Design. Prentice Hall, Englewood Cliffs, NJ, 1997. [28] Otani, Y., Matsuba, Y., and Yoshizawa, T. Photothermal actuator composed of optical fibers, Proceedings of SPIE, 4564, 216– 219, 2001. [29] Pech-Pacheco, J., Cristobal, G., Chamorro-Martinez, J. and FernandezValdivia, J. Diatom autofocusing in brightfield microscopy: a comparative study, Proceedings of International Conference on Pattern Recognition, 15:3, 314– 317, 2000. [30] Pugh, A.. Robot Sensors, Tacktile and Non-Vision, Vol. 2. Springer-Verlag, Berlin, 1986. [31] Subbarao, M. and Tyan, J.K. Selecting the optimal focus measure for autofocusing and depth-from-focus, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:8, 864– 870, 1998. [32] Sun, F., Xiao, G., Zhang, Z. and Grover, C.P. Piezoelectric bimorph optical-fiber sensor, Applied Optics, 43:9, 1922–1925, 2004. [33] Tabib-Azar, M. and Leane, J.S. Direct optical control for a silicon microactuator, Sensors and Actuators, 21:1/3, 229–235, 1990. [34] Toshiyoshi, H., Su, J.G.D., LaCosse, J. and Wu, M.C. Micromechanical lens scanners for fiber optic switches, Proceedings of the Third International Conference on Micro Opto Electro Mechanical Systems, 165– 170, 1999. [35] Yoshizawa, T., Hayashi, D., and Otani, Y. Optical driving of a miniature machine composed of temperature-sensitive ferrite and shape memory alloy, Proceedings of SPIE, 4190, 212–219, 2001. [36] Yoshizawa, T., Vsui, T., Yomamoto, M. and Mayashi, D. Miniaturized machine moving in a pipe using photothermal effect, Proceedings of SPIE, 4902, 110 – 115, 2002. [37] Zhang, J.H. and Cai, L. An autofocusing measurement system with a piezoelectric translator, IEEE/ASME Transactions on Mechatronics, 2:3, 213– 216, 1997.
7 Optomechatronic Systems in Practice CONTENTS Laser Printer ....................................................................................................... 448 System Configuration and Units ............................................................. 451 The Laser Optical Elements.............................................................. 452 Photoconductor................................................................................... 452 Toner Charging Development System ............................................ 452 Hot Roll Fuser ..................................................................................... 454 Black Density Control System .......................................................... 455 Printing Performance Specification......................................................... 455 Laser Source ........................................................................................ 456 Acousto-Optical Deflector................................................................. 457 The Optical System.................................................................................... 458 System Configuration ........................................................................ 458 Resolution of Optical Devices.................................................................. 459 Line Scanner: Polygon Mirror .......................................................... 459 Feedback Control of Beam Focus ............................................................ 463 Aperture Control ................................................................................ 465 Optical Storage Disk.......................................................................................... 469 System Configuration................................................................................ 470 Optical System for Focusing .................................................................... 472 Beam Tracking ............................................................................................ 477 Disk Servo Control System....................................................................... 478 Atomic Force Microscope ................................................................................. 484 Measurement Principle ............................................................................. 484 Cantilever Deflection ......................................................................... 493 Optical Measurement of Cantilever Deflection .................................... 496 Optics for Tip Deflection Measurement ......................................... 497 Control of the Scanning Motion .............................................................. 501 Confocal Scanning Microscope........................................................................ 510 Measurement Principle ............................................................................. 510 Beam Scanning ........................................................................................... 513 Nipkow Disk............................................................................................... 518 System Resolution...................................................................................... 520
447
448
Optomechatronics
Focus Measure and Control ..................................................................... 521 Projection Television.......................................................................................... 524 Digital Micromirror Device ...................................................................... 525 Architecture of a DMD Cell.............................................................. 526 Light Pulse Width Modulation ........................................................ 532 Grating Light Valve Display .................................................................... 535 Light Diffraction by Deformable Grating Modulator................... 536 Deflection Control of Microbeam Structures ................................. 538 Visual Tracking System..................................................................................... 540 Image Processing ....................................................................................... 541 Feature Extraction ...................................................................................... 544 Visual Tracking of a Moving Object ....................................................... 548 Zoom Lens Control ............................................................................ 549 Problems.............................................................................................................. 558 References ........................................................................................................... 563
In the previous chapter, we derived a variety of functional components that are characteristics of optomechatronic nature. These components can be identified as a fundamental common tool that can realize optomechatronic integration and in actual practice are becoming the essential parts of many optomechatronic systems. In fact, this emerging trend can be observed in a number of engineering products and systems. In this chapter, we will illustrate several optomechatronic systems in which the optomechatronic components are playing essential roles in performing key functional tasks. The systems to be dealt with here include laser printers, optical storage disks, atomic force microscopes (AFM), confocal microscopes, digital micromirror devices, and visual tracking systems.
Laser Printer Basic operation principles of a laser printer combine the technology of a photocopier with laser light scanning [22]. The printer has several advantages over an inkjet printer. It can write with much greater speed than an inkjet printer and it can draw more precisely due to adoption of an optical system with an unvarying diameter laser beam and optical units with high level of precision and reliability. Another advantage is that its maintenance cost is much lower than that of the inkjet, because toner powder is cheap and lasts a long time. The laser printer uses an electrophotographic process which is operated with a variety of optical and mechatronic units as shown in Figure 7.1. The process develops a latent image on a photoconductive rotating drum when a laser beam from a spinning polygon mirror scans across it.
Optomechatronic Systems in Practice
449
FIGURE 7.1 A schematic of a laser printer.
This image is then coated with toner particles and transferred on to a sheet of paper as it rolls in contact with the drum surface. Due to the interaction of optical units with mechatronic ones, printing quality is affected by how well each unit is interfaced with the other. For instance, even if a laser beam is shaped with appropriate units, the output beam leaving the spinning mirror will be seriously affected if it were to wobble while rotating. Also, the variation of the drum rotation will create an image artifact, called banding, when the shaped output beam hits the surface of the drum. Therefore, interaction problems including those just mentioned are among those that need to be solved for the purpose of ensuring print quality with higher resolution and speed. The basic principle of the printing is static electricity which is an electrical charge built up on an insulated object, such as a balloon. To use this principle, the printer as shown in the figure consists of a rotating metal drum coated with a highly photoconductive material, a charging corona wire charging the drum, a developer roller that coats the drum with toner particles, a densitometer that measures the developed blackness of a pattern on the area of the drum, and a transfer corona wire keeping the paper from clinging to the drum. Figure 7.2 illustrates some of the details of the printing process. The drum assembly is initially negatively charged and can conduct electricity only when it is exposed to light photons. When the printer
450
Optomechatronics hot roll
roll
stationary magnet drum
incoming paper
. . .. ..............
roller
optical switch
.
shield . wire
to
drum
un
.. ........ .... ar .. ........... .. ge d ar ea
ch
ground
ne
positively charged
drum
scanner
g ch ati ar vel ge y d
paper with image
ne ..... ........... .... r a..... ....... .. tta.. .. ch ed
.
.
.
.
FIGURE 7.2 Laser printing process.
receives the image in digital form from the computer, the laser light hits the desired parts of the drum surface according to the switching action of the optical switch and the scanning action of the polygon mirror shown in the figure. These control actions are directed by the printer’s microelectronics. As soon as the scanned light irradiates the designated surface area of the drum, the illuminated area begins to conduct electricity and is discharged as a result. The drum is then covered with a pattern of negatively charged areas (those not exposed by light) and uncharged areas (those exposed by light). After the pattern is set, the developer roller coats the drum with negatively charged black toner particles. They cling to the uncharged area of the drum but not to the negatively charged background that was not exposed to the light. Once the powder pattern is in place the drum rolls over a blank sheet of paper with a positive charge given by the transfer corona wire. The positively charged paper attracts the negatively charged toner particles and pulls them away from the drum as indicated in the figure. Here, it is necessary to keep the rotation speed of the paper at the drum speed in order to make the paper pick up the pattern exactly. As the paper passes through the fuser rollers composed of a pair of heated rollers, the plastic toner powder melts and fuses permanently with the fibers in the paper. Finally, the fuser rolls the paper
Optomechatronic Systems in Practice
451
to the output tray. It is noted here that, because the fuser also heats up, it is important for the paper to pass the rollers very quickly. The discharge lamp shown in the figure takes the roll of erasing the electrical image of the entire photoreceptor surface by illuminating bright light on to the surface. System Configuration and Units As already explained above, the laser printer is of complex optomechatronic nature, because generating the laser pattern and forming an electrostatic image from the pattern involve optomechatronic processes in which optical and mechatronic units are interacted with each other. These interactions are provided by the acousto-optical deflector (AOD), the polygonal scanner and a rotating drum and its drive unit, as shown in the system control block diagram shown in Figure 7.3. According to the logic circuits, the computer sends the basic information to the printer concerning the printing format such as text, graphics, and margin sizes. The information to be used to set up the pattern for modulating the laser beam is created in the microprocessor. In response to the generated digital signal pattern, the AOD deflects the beam either into the optical path to the drum or away from it. When the laser is “on,” the scanner transmits the beam on to the surface of the photo-sensitive rotating drum resulting in a dark spot. The scanning action is performed along the axis of the drum, i.e., (x direction). When one-line scanning for a line is completed, rotation of the drum (y direction) makes the scanner start another line. This results in a raster scan of the drum surface.
motor servo
fuser roller
microprocessor
paper densitometer
motor servo
photoconductor Phot oconductor drum drum f - q lens developer station laser
A-O beam encoder acousto-optical deflector deflector (AOD) FIGURE 7.3 Control system schematic of a laser printer.
polygon mirror
452
Optomechatronics
The Laser Optical Elements The optical system consists of a laser source, a movable mirror, a scanning mechanism, and a series of lenses including a f-u lens. The system receives the page data which makes up the text and images one horizontal line at a time. They come in digital form (tiny dots). According to the data, the laser source emits a pulse of light for every dot to be printed and no pulse for every dot of empty space. Referring to Figure 7.1 and Figure 7.3, the laser beam enters the AOD modulator and the modulated beam is collimated by a collimating lens, which is again shaped ready to enter the mirror facet of the rotating polygon deflector. Because this causes aberrations such as pincushion, a f-u lens is normally placed in front of the electro-photographic drum. The role of the beam scanner is to distribute the laser beam spatially in the horizontal line on to the drum. As the spinning mirror moves, it illuminates the beam through the f-u lens, making the beam move across the drum. It can be seen that the optical assembly scans only one horizontal line for each move. After each scan, a printer mechanism moves the drum up by a notch for preparation of the next line drawing and then a new scanning command is detected by a scan start detector. A print-engine computer synchronizes all of these. Photoconductor The conductor drum contains a charged layer photoconductive element that bears the latent image generated by the light; charge transforming layer (typically 10 to 30 mm), and charge generation layer (typically 0.5 to 5 mm). As indicated in Figure 7.4a, initially the photo drum charges to a high voltage level which is uniform over its surface. When laser light is impinged on the charge generation layer, this generates the photoconduction process in the layers by which the electron-hole pairs are produced. In the process shown in Figure 7.4b, electrons exit at the electrode, while holes are injected into the charge transporting layer. These holes are transported across the layer to the free surface of the photoconductor and neutralize the negative charge to create the latent image. Therefore, the surface of the drum enters the development section with a light exposed area and nonexposed area having approximately 0 and 500 V, respectively. Toner Charging Development System The development section contains a toner hopper and a developer roller, as shown in Figure 7.5. The toner is essentially an electrically charged powder, which is composed of two main ingredients, pigment and plastic. The use of it makes the laser printer distinct from an ink-jet printer. The advantage of the toner over the liquid ink is that it firmly sticks to the fibers of almost any type of paper. The toner powder is supplied by a developer roller from the toner hopper as shown in the figure. The developer is covered with small positively charged magnetic beads, which are attached to a rotating metal
substrate
FIGURE 7.4 Cross sectional view of a photoconductive drum.
(a) corona charge electron hole pair
electrode
charge generation layer
charge transporting layer
light
(b) hole transport neutralization
substrate
latent image
electrode
charged photoconductive surface
Optomechatronic Systems in Practice 453
454
Optomechatronics
f -q lens photoconductive drum
carrier particles with toner W photo conductor (drum)
stationary magnets
developer roller magnetic brush
paper developing region FIGURE 7.5 A schematic of the development system for toner charging.
roller. The developer carries the beads with toner particles into the developing zone. Depending upon which zone light is absorbed, the transported toner particles are transferred. In this way the toner powder is provided to the drum while the roller rotates through the hopper. Hot Roll Fuser In the transfer process, during which the toner attached to the drum is transferred to the paper, the toner particles are weakly attached to the paper. It is therefore necessary to fix them in place to the paper surface permanently. This process is called the fusing process and has two types of fusing, contact and noncontact. The contact type requires fusing by touching the toner slider of the paper, while the noncontact one does not touch. Figure 7.6 illustrates a typical contact type hot roller fuser which is composed of a hot roller heated by a lamp, a backup roller (pressure roller) applying pressure to the hot roller, and a thin layer coated with elastomer used for high thermal stability and low surface energies. The low energies employed here yield minimum toner sticking. It can be seen that fusing quality mainly depends upon the temperature (T) of the interface between the paper and toner, the pressure ( p) applied to the toner parties, and time (t) in the nip zone.
Optomechatronic Systems in Practice
455 elastomer core coating
hot roller
nip zone pressure roller backup roller W
FIGURE 7.6 Contact-type hot roll fuser system.
Black Density Control System An important unit at this stage of printing process is the one which controls the blackness of the pattern printed on an area of the drum. A reflective densitometer measures the blackness of the developed area and this information is sent to the microprocessor. The processor then compares with the desired blackness and, if any deviation occurs, it adjusts appropriate development parameters to regulate the proper developed density. Printing Performance Specification There are several factors that affect the printing quality of the laser printer. Three primary factors that represent the print quality are resolution of spot size and waiting speed of the printer and laser power consumption. We will examine some of these factors that determine quality. Let us begin our discussion with a series of characters that can be formed on the drum surface. Suppose that the scanning system scans the drum in such way that a character is formed within n £ m dot matrix as illustrated in Figure 7.7. If the printer has a printing rate of Nc characters per second, it must produce Ps dots per second Ps ¼ n £ m £ N c
ð7:1Þ
To ensure printing image quality proper overlap between adjacent dots is necessary, as can be seen from the H character in the figure. To this end we need to specify the horizontal separation and the vertical separation between dots. The horizontal separation ‘h is determined by the modulation speed
456
Optomechatronics
FIGURE 7.7 A laser scanned H character.
of the AOD, but the vertical separation ‘v is determined by the positioning accuracy of the drum controlled by a rotation motor. The writing resolution can be also easily determined in terms of number of dots. For example, if the printer is assumed to print the entire width of an A4 page (210 £ 297 mm), with ‘h ¼ 30 mm, ‘v ¼ 40 mm, the required number of dots per scan in horizontal direction is given by N¼
210 ¼ 7000 dots 0:03
As we shall see later, the number of dots per scan is related to the geometry of the polygon scanner and the diameter of the beam spot entering into the mirror facet of the scanner. The above discussion makes it necessary to analyze the optical system of the printer in order to determine the design parameters that meet the specifications required for the resolution, speed, and exposure laser level of the printer. We will deal with this topic in the subsequent subsections. Laser Source The laser needs to have long working life span and low power consumption while its wavelength should be short to make the drum photosensitive. Nowadays, due to advancement of diode laser technology, laser diodes are
Optomechatronic Systems in Practice
457
more conveniently chosen over the He– Ne laser as a printer source in view of cost and size, but we will work with the He– Ne laser to become familiar with the operation of the AOD. Provided the specifications are of the required resolution, and the printing speed is that given in Equation 7.1, we can calculate the laser power consumption level. If the printer has a resolution of 25 dots per mm, that is, dot size is given by Dspot ¼ 2rspot ¼ 40 mm where Dspot and rspot are the diameter and radius of the dot, respectively. The dot area denoted by Ad approximately becomes Ad ¼ 40 mm £ 40 mm ¼ 1600 mm2 Now, consider the case when the printer prints 36,000 characters per second each of which is composed of 18 £ 25 dot matrices. If the energy level of a laser is given by 1.6 mJ/cm2, laser exposures per dot is given by 2.56 £ 1025 mJ. Therefore, the laser power level P‘ becomes P‘ ¼ ð2:56 £ 1025 mJÞ £ 1:62 £ 107 =sec ¼ 0:41 mW Acousto-Optical Deflector Equation 7.1 determines the time for one dot to be printed, which is the reciprocal of Ps : Let us suppose that the time for printing one dot would be 40 nsec. This indicates that if an AOD cell is used, the rise time must be shorter than 40 nsec. According to Equation 6.41, the laser beam diameter to be modulated inside the modulator affects the rise time as Tr ¼ b
Ds va
where Tr is the rise time of the AOD, Ds is the beam diameter illuminating a facet of the polygon, and va is the velocity of the sound. If b is unity, and the time for printing one dot is required to be less than 40 nsec, then Ds ¼ 40 £ 1029 sec £ 6 £ 106 mm=sec ¼ 240 mm in the case of using fused quartz as a modulator. In early laser printers adopting a hard-sealed He– Ne laser with a beam diameter 1 mm, the beam must be compressed by a factor of four or five in order to obtain the 240 mm diameter by a beam compressor as depicted in Figure 7.1. We shall see later that when a laser beam compressed for modulation enters the polygonal scanner, its beam diameter needs to be expanded by a beam expander to increase the writing resolution whose expansion ratio depends upon the area of the scanner facet.
458
Optomechatronics
The Optical System System Configuration Figure 7.8 shows a complete arrangement of the optical system redrawn for better understanding of the optical beam flow from the laser source to the photoconductive drum surface. The optical system shown here is so composed that the effects of some undesirable factors affecting print quality are reduced. In general, the optical system without compensation has several factors that critically affect print quality. If the major ones are listed here: aberration (geometrical and wavefront), focusing, and beam spot divergence. Aberration causes field curvature on the scan plane, defocusing causes blurring effects and beam divergence affects resolution of imaging. Unless all of these are solved, good printing quality cannot be guaranteed. We will briefly discuss some of the correction methods that can drastically reduce these unfavorable optical effects. Correcting the field curvature is done with an f-u lens as discussed in Chapter 6, “Optical Scanning.” To reiterate, here we will consider a lens assembly which consists of the combination of several lenses whose configurations depend on performance requirements, some have only two, for instance, a toroidal lens, and a cylindrical lens. Referring to Figure 6.39, the cylindrical lens having a concave surface forms the scan line bent outward from the photodrum surface. In this situation we can use this lens to flatten out the scan line along the surface. Figure 7.9 illustrates the principle of correction by drawing the scan lines with and without correction. As can be seen from the figure, the solid line bent inward towards the lens denotes the focus line of the image obtained by a single convex lens (toroidal surface), while the one bent outward from the drum surface represents the result obtained by a concave cylindrical surface. Combining these two curves together, a solid line can be obtained, which is the result of the correction by the f-u lens. This result shows that when two lenses are combined, the focus shift is greatly reduced, otherwise it remains large with a single toroidal convex lens as indicated in the figure. Further correction needs to be made for higher demands of printing cylindrical toroidal concave surface surface
aperture
photo drum
polygon mirror laser source
collimator
cylindrical lens
f - q lens
image plane
FIGURE 7.8 Optical arrangement for the laser printing system. Source: Donald C. O’Shea, Elements of Modern Optical Design, John Wiley & Sons, 1985. Reprinted with permission of John Wiley & Sons, Inc.
Optomechatronic Systems in Practice
459
corrected lens inward
scan position
image plane
outward
cylindrical concave lens without correction (toroidal convex lens)
lens
−1.0
0 (normalized)
1.0
drum
FIGURE 7.9 Correction of field curvature using a single cylindrical concave lens.
performance. In this case several more lenses may be added in order to eliminate the