Optomechatronics: Fusion of Optical and Mechatronic Engineering

1969_half 9/16/05 11:11 AM Page 1 Fusion of Optical and Mechatronic Engineering Mechanical Engineering Series Frank ...

Author: Hyungsuck Cho

212 downloads 1634 Views 15MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

1969_half 9/16/05 11:11 AM Page 1

Fusion of Optical and Mechatronic Engineering

Mechanical Engineering Series Frank Kreith & Roop Mahajan - Series Editors Published Titles Distributed Generation: The Power Paradigm for the New Millennium Anne-Marie Borbely & Jan F. Kreider Elastoplasticity Theor y Vlado A. Lubarda Energy Audit of Building Systems: An Engineering Approach Moncef Krarti Engineering Experimentation Euan Somerscales Entropy Generation Minimization Adrian Bejan Finite Element Method Using MATLAB, 2nd Edition Young W. Kwon & Hyochoong Bang Fluid Power Circuits and Controls: Fundamentals and Applications John S. Cundiff Fundamentals of Environmental Discharge Modeling Lorin R. Davis Heat Transfer in Single and Multiphase Systems Greg F. Naterer Introductor y Finite Element Method Chandrakant S. Desai & Tribikram Kundu Intelligent Transportation Systems: New Principles and Architectures Sumit Ghosh & Tony Lee Mathematical & Physical Modeling of Materials Processing Operations Olusegun Johnson Ilegbusi, Manabu Iguchi & Walter E. Wahnsiedler Mechanics of Composite Materials Autar K. Kaw Mechanics of Fatigue Vladimir V. Bolotin Mechanics of Solids and Shells: Theories and Approximation Gerald Wempner & Demosthenes Talaslidis Mechanism Design: Enumeration of Kinematic Structures According to Function Lung-Wen Tsai Multiphase Flow Handbook Clayton T. Crowe Nonlinear Analysis of Structures M. Sathyamoorthy Optomechatronics: Fusion of Optical and Mechatronic Engineering Hyungsuck Cho Practical Inverse Analysis in Engineering David M. Trujillo & Henry R. Busby Pressure Vessels: Design and Practice Somnath Chattopadhyay Principles of Solid Mechanics Rowland Richards, Jr. Thermodynamics for Engineers Kau-Fui Wong Vibration and Shock Handbook Clarence W. de Silva Viscoelastic Solids Roderic S. Lakes

1969_title 9/16/05 11:11 AM Page 1

Fusion of Optical and Mechatronic Engineering

Hyungsuck Cho

Boca Raton London New York

A CRC title, part of the Taylor & Francis imprint, a member of the Taylor & Francis Group, the academic division of T&F Informa plc.

1969_Discl.fm Page 1 Monday, September 26, 2005 11:04 AM

Published in 2006 by CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2006 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-10: 0-8493-1969-2 (Hardcover) International Standard Book Number-13: 978-0-8493-1969-3 (Hardcover) Library of Congress Card Number 2005050570 This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data Cho, Hyungsuck Optomechatronics / by Hyungsuck Cho p. cm. Includes bibliographic references and index. ISBN 0-8493-1969-2 (alk. paper) 1. Mechatronics. 2. Optical detectors. TJ163.12.C44 2005 670.42'7--dc22

2005050570

Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com Taylor & Francis Group is the Academic Division of Informa plc.

and the CRC Press Web site at http://www.crcpress.com

Author

Hyungsuck Cho gained his B.S. degree at Seoul National University, Korea in 1971, an M.S. degree at Northwestern University, Illinois in 1973, and a Ph.D. at the University of California at Berkeley, California in 1977. Following a term as Postdoctoral Fellow in the Department of Mechanical Engineering, University of California, Berkeley, he has joined the Korea Advanced Institute of Science and Technology (KAIST) in 1978. He was made a Humboldt Fellow in 1984-1985, won Best Paper Award at the International Symposium on Robotics and Manufacturing, USA in 1994, and the Thatcher Brothers Awards, Institute of Mechanical Engineers, UK in 1998. Since 1993, he has been an associate editor or served on the editorial boards of several international journals, including IEEE Transactions on Industrial Electronics, and has been guest editor for three issues, including IEEE Transactions IE Optomechatronics in 2005. Dr. Cho wrote the handbook Optomechatronic Systems: Technique and Application, has contributed chapters to 10 other books and has published 435 technical papers, primarily in international journals. He was the founding general chair for four international conferences and the general chair or co-chair for 10 others, including the SPIE Optomechatronic Systems Conference held in Boston in 2000 and 2001. His research interests are focused on optomechatronics, environment perception and recognition for mobile robots, optical vision-based perception, control, and recognition, and application of artiﬁcial intelligence/ machine intelligence. He has supervised 136 M.S. theses and 50 Ph.D. theses. For the achievements in his research work, he was made POSCO professor from 1995 to 2002.

Preface In recent years, optical technology has been increasingly incorporated into mechatronic technology, and vice versa. The consequence of the technology marriage has led to the evolution of most engineered products, machines, and systems towards high precision, downsizing, multifunctionalities and multicomponents embedded characteristics. This integrated engineering ﬁeld is termed optomechatronic technology. The technology is the synergistic combination of optical, mechanical, electronic, and computer engineering, and therefore is multidisciplinary in nature, thus requiring the need to view this from somewhat different aspects and through an integrated approach. However, not much systematic effort for nurturing students and engineers has been made in the past by stressing the importance of the multitechnology integration. The goal of this book is for it to enable the reader to learn how the multiple technologies can be integrated to create new and added value and function for the engineering systems under consideration. To facilitate this objective, the material brings together the fundamentals and underlying concepts of this optomechatronic ﬁeld into one text. The book therefore presents the basic elements of the engineering ﬁelds ingredient to optomechatronics, while putting emphasis on the integrated approach. It has several distinct features as a text which make it differ somewhat from most textbooks or monographs in that it attempts to provide the background, deﬁnition, and characteristics of optomechatronics as a newly deﬁned, important ﬁeld of engineering, an integrated view of various disciplines, view of systemoriented approach, and a combined view of macro– micro worlds, the combination of which links to the creative design and manufacture of a wide range of engineering products and systems. To this end a variety of practical system examples adopting optomechatronic principles are illustrated and analyzed with a view to identifying the nature of optomechatronic technology. The subject matter is therefore wide ranging and includes optics, machine vision, fundamental of mechatronics, feedback control, and some application aspects of micro-opto-electromechanical system (MOEMs). With the review of these fundamentals, the book shows how the elements of optical, mechanical, electronic, and microprocessors can be effectively put together to create the fundamental functionalities essential for the realization of optomechatronic technology. Emphasizing the interface between the relevant disciplines involving the integration, it derives a number of basic optomechatronic units. The book

then goes on in the ﬁnal part to deal, from the integrated perspectives, with the details of practical optomechatronic systems composed of and operated by such basic components. The introduction presents some of the motivations and history of the optomechatronic technology by reviewing the technological evolution of optoelectronics and mechatronics. It then describes the deﬁnition and fundamental concept of the technology that are derivable from the nature of practical optomechatronic systems. Chapter 2 reviews the fundamentals of optics in some detail. It covers geometric optics and wave optics to provide the basis for the fusion of optics and mechatronics. Chapter 3 treats the overview of machine vision covering fundamentals of image acquisition, image processing, edge detection, and camera calibration. This technology domain is instrumental to generation of optomechatronic technology. Chapter 4 presents basic mechatronic elements such as sensor, signal conditioning, actuators and the fundamental concepts of feedback control. This chapter along with Chapter 2 outline the essential parts that make optomechatronics possible. Chapter 5 provides basic considerations for the integration of optical, mechanical, and electrical signals, and the concept of basic functional modules that can create optomechatronic integration and the interface for such integration. In Chapter 6, basic optomechatronic functional units that can be generated by integration are treated in detail. The units are very important to the design of optomechatronic devices and systems, since these produce a variety of functionalities such as actuation, sensing, autofocusing, acousticoptic modulation, scanning and switching visual feedback control. Chapter 7 represents a variety of practical systems of optomechatronic nature that obey the fundamental concept of the optomechatronic integration. Among them are laser printers, atomic force microscopes (AFM), optical storage disks, confocal microscopes, digital micromirror devices (DMD) and visual tracking systems. The main intended audiences of this book are the lower levels of graduate students, academic and industrial researchers. In the case of undergraduate students, it is recommended for the upper level since it covers a variety of disciplines, which, though fundamental, involve various different physical phenomena. On a professional level, this material will be of interest to engineering graduates and research/ﬁeld engineers who function in interdisciplinary work environments in the ﬁelds of design and manufacturing of products, devices, and systems. Hyungsuck Cho

Acknowledgments I wish to express my sincere appreciation to all who have contributed to the development of this book. The assistance and patience of Acquiring Editor Cindy Renee Carelli, have been greatly appreciated during the writing phase. Her enthusiasm and encouragement have provided me with a great stimulus in the course of this book writing. In addition, I would like to thank Jessica Vakili, project coordinator, Fiona Woodman, project manager, and Tao Woolfe, project editor of Taylor and Francis Group, LLC, for ensuring that all manuscripts were ready for production. I am also indebted to my former Ph.D students, Drs. Won Sik Park, Min Young Kim and Young Jun Roh for their helpful discussions. Special thanks go to Hyun Ki Lee and all my laboratory students, Xiaodong Tao, Deok Hwa Hong, Kang Min Park, Dal Jae Lee and Xingyong Song who have provided valuable help in preparation of the relevant materials and proofreading the typed materials. Finally, I am grateful to my wife, Eun Sue Kim, and my children, Janette and Young Je, who have tolerated me with patience and love and helped make this book happen.

Contents 1. Introduction: Understanding of Optomechatronic Technology.............1 2. Fundamentals of Optics................................................................................31 3. Machine Vision: Visual Sensing and Image Processing .....................105 4. Mechatronic Elements for Optomechatronic Interface........................173 5. Optomechatronic Integration ....................................................................255 6. Basic Optomechatronic Functional Units ...............................................299 7. Optomechatronic Systems in Practice .....................................................447 Appendix A1

Some Considerations of Kinematics and Homogeneous Transformation......................................565

Appendix A2

Structural Beam Deﬂection............................................573

Appendix A3

Routh Stability Criterion ...............................................577

Index .....................................................................................................................581

1 Introduction: Understanding of Optomechatronic Technology CONTENTS Historical Background of Optomechatronic Technology ................................ 4 Optomechatronics: Deﬁnition and Fundamental Concept ............................. 8 Practical Optomechatronic Systems............................................................ 9 Basic Roles of Optical and Mechatronic Technologies .......................... 12 Basic Roles of Optical Technology..................................................... 13 Basic Roles of Mechatronic Elements................................................ 15 Characteristics of Optomechatronic Technology .................................... 16 Fundamental Functions of Optomechatronic Systems.................................. 20 Fundamental Functions ...................................................................................... 21 Illumination Control.................................................................................... 21 Sensing........................................................................................................... 24 Actuating ....................................................................................................... 24 Optical Scanning .......................................................................................... 24 Visual/Optical Information Feedback Control ....................................... 24 Data Storage.................................................................................................. 25 Data Transmission/Switching ................................................................... 25 Data Display ................................................................................................. 25 Optical Property Variation.......................................................................... 26 Sensory Feedback-Based Optical System Control .................................. 26 Optical Pattern Recognition ....................................................................... 26 Remote Operation via Optical Data Transmission................................. 27 Material Processing...................................................................................... 27 Summary ............................................................................................................... 27 References ............................................................................................................. 28 Most engineered devices, products, machines, processes, or systems have moving parts and require manipulation and control of their mechanical or dynamic constructions to achieve a desired performance. This involves the use of modern technologies such as mechanism, sensor, actuator, control, microprocessor, optics, software, communication, and so on. In the early 1

Optomechatronics

value (performance)

2

optical element

electrical/ electronics

electrical/ electronics

optical element

software

software

electrical/ electronics

electrical/ electronics

optical element

software

electrical/ electronics

mechanical mechanical mechanical mechanical mechanical mechanical element element element element element element

1800

1970

2000

year FIGURE 1.1 Key component technologies contributing to system evolution (not to scale).

days, these have been operated mostly via mechanical elements or devices which caused inaccuracy and inefﬁciency, thus resulting in difﬁculty in achieving a desired performance. Figure 1.1 and Figure 1.2 show how the key technologies such as mechanical, electrical, and optical have contributed to the evolution of machines/systems in terms of “value or performance” as years have passed [6]. As can be seen from the ﬁgure, tremendous efforts have been made to enhance system performance by combining electrical and electronic hardware with mechanical systems. A typical example is a gear-trained mechanical system controlled by a hardwired controller. This mechanical and electronic, called mechatronic conﬁguration, consisted of two kinds of components: mechanism, and electronics and electric hardware. Because of 10k 10k

mechanism mechanical automation

− 411 +

analog control

μ-processor embedded M/C

optomechatronically embedded system

internet based (teleoperation)

mechatronic technology optomechatronics technology

FIGURE 1.2 Evolution of machines.

Introduction: Understanding of Optomechatronic Technology

3

the hard-wired structural limitation of this early mechatronic conﬁguration, ﬂexibility was not embedded in most systems in those days. This kind of tendency lasted until the mid 1970s when microprocessors came into use for industrial applications. The development of microprocessors has provided a new stimulant for industrial evolution. This brought about a big change, the replacement of many mechanical functions with electronic ones through the role of microprocessor. This evolutionary change has opened up the era of mechatronics, and has raised the autonomy level of machines and systems, at the same time increasing versatility and ﬂexibility. The autonomy and ﬂexibility achieved thus far, however, have a growth limited to a certain degree, since both the hardware and software of the developed mechatronic systems have not been developed so much as to have the capability of realizing many complicated functions autonomously while adapting to changing environments. In addition, information structure has not been developed to have real-time access to appropriate system data and information. There may be several reasons for this delay. The ﬁrst one may be that in many cases mechatronic components alone may not achieve desirable function or performance as speciﬁed for a system design. The second one is that, although mechatronic components alone can work, the results achieved may not be as good as required because of their low perception and execution capability and also inadequate integration between hardware and software. In fact, in many cases measurements are difﬁcult or not even feasible due to inherent characteristics of the systems. In some other cases, the measurement data obtained by conventional sensors are not accurate or reliable enough to be used for further processing. They can sometimes be noisy, necessitating some means of ﬁltering or signal conditioning. The difﬁculties listed here may limit the enhancement of the functionality and performance of the mechatronic systems. This necessitates the integration of the mechatronic technology with other systems. In recent years, optical technology has been increasingly incorporated at an accelerated rate into mechatronic systems, and as a result, a great number of mechatronic products, machines and systems with smart optical components have been introduced into the market. There may be some compelling reasons for this changing trend. One reason is that the optical technology possesses unique characteristics such as noncontact/noninvasiveness visual perception, and insensitivity to electrical noise. As shown in Figure 1.1, the contribution of the optical technology is growing and enhances the system value and performance, since optical elements incorporating mechatronic elements embedded in the system provide some solutions to the difﬁcult-to-solve technical problems. This emerging trend demonstrates that the optically integrated technology provides enhanced characteristics in such a way that it creates new functionalities that are not achievable with conventional technology alone, exhibits higher functionalities since it makes products and systems function on an entirely different principle or in a more efﬁcient manner, and produces high precision

4

Optomechatronics

and reliability since it can facilitate or enable in-process monitoring and control of the system state. Besides, the technology makes it feasible to achieve dimensional downsizing and allows system compactness, since it has the capability of integrating sensors, actuators, and processing elements into one tiny unit.

Historical Background of Optomechatronic Technology The root of the development of the optically integrated mechatronic technology may be easily found when we revisit the historical background of the technological developments of mechatronics and optoelectronics. Figure 1.3 shows the development of mechatronic technology in the upper line above the arrow and that of optical engineering in the lower line [8]. The real electronic revolution came in the 1960s with the integration of transistor and other semiconductor devices into monolithic circuits following the invention of the transistor in 1948. Then the microprocessor was invented in 1971 with the aid of semiconductor fabrication technology and made a tremendous impact on a broad spectrum of technological ﬁelds. In particular, the development created a synergistic fusion of a variety hardware and software technologies by combining them with computer technology. The fusion made it possible for machines to transform analog signal into digital, to compute necessary calculations, to make decisions based upon the computed results and software algorithms, and then to take appropriate action according to the decision and to accumulate knowledge/data/information within their own memory domain. This new functionality has endowed machines and systems with characteristics such as ﬂexibility and adaptability, and the importance of this concept has been recognized among industrial sectors, which accelerated ever wider applications. In the 1980s, the semiconductor technology also created micro-electro-mechanical systems (MEMSs), and this brought about a new dimension of machines and systems, micro-sizing their dimensions. Another technological revolution, so-called opto-electronic integration, has continued during the last 40-plus years ever since the laser was invented in 1960. This was made possible with the aid of advanced fabrication methods such as chemical vapor deposition, molecular-beam epitaxy, and focused-ion-beam micro-machining. These methods enabled integration of optical, electro-optic, and electronic components in a single compact device. The charge coupled device (CCD) image sensor developed in 1974 not only introduced computer vision technology but also opened up a new era of optical technology along with optical ﬁber sensors which appeared from 1976. The optical components and devices that were developed possessed a number of favorable characteristics, including: (1) noncontact/noninvasive; (2) easy to transduce; (3) having a wide sensing range; (4) insensitive to

FIGURE 1.3 History of optomechatronics. Source: Cho, H.S. and Kim, M.Y., IEEE Transaction on Industrial Electronics, 52:4, 932–943, 2005. q 2005 IEEE.

Introduction: Understanding of Optomechatronic Technology 5

6

Optomechatronics

electrical noises; (5) distributed sensing and communication; and (6) high bandwidth. Naturally, these favorable optical characteristics began to be integrated with those of the mechatronic elements and this integration helped achieve systems of higher performance. When a system or a machine is integrated in this way, namely optically, mechanically, and electronically, it is called an optomechatronic system. Looking back at the development of this system shown in Figure 1.3, we can bring to mind a number of practical examples. The lithography tool that fabricates ICs and other semiconductor devices belongs to this system category: it functions through a series of elaborate mirrors in addition to a light beam, optical units and a stepper servo mechanism that precisely shifts the wafer from site to site. Another representative system is the optical pick-up device mass-produced from 1982. The pickup system reads information off the spinning disc by controlling both the up-and-down and side-to-side tracking of a read head which carries a low-power diode laser beam focused onto the pits of the disc. Since the early days, a great number of optomechatronic products, machines or systems have come out at an increasingly accelerated rate, for the effects that can be achieved with the properties of optical components are signiﬁcant. As shown in Figure 1.3, through the advancement in microsystems and the advent of MEMS, optomechatronic technology has brought about a new technology evolution, that is, a marriage of optics with microsystems or MEMS. A variety of the components or systems that belong to this category have been developed, and the atomic force microscope (AFM), optical MEMS, and optical switch are some examples among them. As seen from the historical perspective, the electronic revolution accelerated the integration of mechanical and electronic components and later, the optical revolution created the integration of optical and electronic components. This trend enabled a number of conventional systems having very low level autonomy and very low work performance to evolve into those having improved autonomy and performance. Figure 1.4 illustrates practical systems currently in use that evolved from their original old versions. Printed circuit board (PCB) inspection was carried out by the naked eyes of human workers using a microscope until recently, but is now performed by a visual inspection technique. The chip mounter or surface mounting device (SMD), originally mostly performed in a mechanical way based on CAD drawing, is now being carried out by integrated devices such as a part position estimator, visual sensors, and a servo control unit. The coordinate measuring machine (CMM) appeared as a contact then noncontact device, then became a digital electromagnetic type and then an optical type. In recent years, the CMM is actively being researched to introduce it as an accurate, reliable, versatile product, in which a sensor integration technique is to be adopted, as can be seen from the ﬁgure. The washing machine shown in Figure 1.6d also evolved from a mechanically operated machine to one having optical sensory feedback and intelligent control function. Table 1.1 illustrates the evolution of some of

touch probe

FIGURE 1.4 Illustrative evolutions.

(c) coordinate measuring machine

(a) PCB inspection

Optical probe

lamp

(d) projector

fluorescent magnetic yolk panel CRT projector

electron gun

electron beam

(b) chip / SMD mounting

mechanical positioning mounter

camera

illumination

LCD projector

B

G

R lens DLP projector

digital micro mirrors

PCB

mounter nozzle

Visual positioning mounter

Fiducial

Introduction: Understanding of Optomechatronic Technology 7

8

Optomechatronics

TABLE 1.1 Evolution in Various Products Technology/Product Data storage disc Printer Projector IC chip mounter

PCB inspection Camera

Coordinate measuring machine (CMM)

Technological Trend Mechanical recording ! magnetic recording·optical recording Dot printer ! thermal printer/ink jet printer ! laser printer CRT ! LCD·DLP projector Partially manual automated assembly ! mechanically automated assembly ! assembly with visual chip recognition Naked eye inspection ! optical/visual inspection Manual ﬁlm camera ! motorized zoom, auto exposure, auto focusing ! digital camera (CMOS, CCD device) Touch probe ! optical probe ! touch probe þ visual/optical sensing

Source: Cho, H.S. and Kim, M.Y., IEEE Transactions on Industrial Electronics, 52:4, 932–943, 2005. q 2005 IEEE.

products through the presence of optomechatronic technology. In the sequel, we shall elaborate more on this issue and utilize a number of practical systems to characterize optomechatronic technology.

Optomechatronics: Deﬁnition and Fundamental Concept It can be observed from the previous discussions that the technology associated with the developments of machines/processes/systems has continuously evolved to enhance their performance and to create new value and new function. Mechatronic technology integrated by mechanical, electronic/electrical, and computer technologies has been certainly taking an important role for such evolution, as can be seen from the historical time line of technology evolution. To make them evolve further towards systems of precision, reliability, and intelligence, however, optics and optical engineering technology needed to be integrated into mechatronics, thus compensating for some limitations in the existing functionalities and creating new ones. The optomechatronics centered in the middle of Figure 1.5 is, therefore, a technology integrated with the optical, mechanical, electrical, and electronic technologies. The technology fusion in this new paradigm is termed optomechatronics or

Introduction: Understanding of Optomechatronic Technology

9

Optics

Optom ech at

r

Op t

ics tron lec oe

ics on

Optomechatronics

Mechanics

M e c h a tr o nic s

Electronics

FIGURE 1.5 The optomechatronic system.

optomechatronic technology [6, 7]. Figure 1.5 shows the integrated technologies that can be achieved by three major technologies: optical, electrical, and mechanical. We can see that optomechatronics can be achieved with a variety of different integrations. We will see in Chapter 5 that these three important combined technologies, optoelectronics, optomechanics, and mechatronics, will be the basic elements for optomechatronics integration. In this section, to provide a better understanding of, and insight into, the system we will illustrate a variety of optomechatronic systems being used in practice and brieﬂy review the basic roles of optical and mechatronic technologies. Practical Optomechatronic Systems Examples of optomechatronic systems are found in many engineering ﬁelds such as control and instrumentation, inspection and test, optical, manufacturing, consumer and industrial electronics, MEMS, automotive, and biomedical applications. Here, we will take only some examples of such ﬁelds of application. Cameras and motors are typical products which are operated by optomechatronic components. For example, a smart camera [3] is equipped with an aperture control and a focusing adjustment together with an illuminometer to perform well regardless of the ambient brightness change. With this system conﬁguration, new functionalities are created for the enhancement of the performance modern cameras. As shown in Figure 1.6a, the main components of a camera are several lenses, an aperture, a shutter, and a ﬁlm or an electrical image cell such as CCD or complementary metal oxide semiconductor (CMOS). Images are focused and exposed on the ﬁlm or the electrical image cell via a series of lenses which effect zooming and focusing of an object. Moving the lenses with respect to the imaging plane results in changes in magniﬁcation and focusing points. The amount of light entering

FIGURE 1.6 Illustrations of optomechatronic systems.

(g) fiber scope device for inspection for microfactory

2 mm

SMA coil spring

bending part SMA actuator

image guide fiber light guide fiber

sweeper

laser half-mirror

Photo-detector lens

=Closed(up)

lens

fibers

Non-interrupted signal

mirror

disk

squeeze roll

(i) pipe welding process

ng ldi n we ctio e r i impeder d

contact tip

laser

(f) n×n optical switching system

=Open(down)

cutoff signal MEMS Mirror

input fiber array

(c) optical storage disk

mirror (X-Y scanner)

(h) rapid prototyping process

liquid photopolymer

focusing lens

platform

elevator motorized actuators

PCB

illuminator and camera

micro-positioner

macro-positioner

(e) vision guided micro positioning system

picker

part rack

laser source

output of wash sensor

phototransistor wash sensor drainpipe mechanical part

(d) modern washing machine with optical sensory feedback

motor

light

3-axis piezo electric stage

AFM Tip

AFM Cantilever

(b) atomic force microscope

infrared LED

water supply valve

washing tank water

upper lid

(a) camera

y

position sensitive detector z y

laser

10 Optomechatronics

Introduction: Understanding of Optomechatronic Technology

11

through the lenses is detected by a photosensor and is controlled by changing either the aperture or shutter speed. Recently, photosensors or even CMOS area sensors are used for autofocusing with a controllable focusing lens. A number of optical ﬁber sensors employ optomechatronic technology whose sensing principle is based on detection of modulated light in response to changes in the physical variables to be measured. For example, the optical pressure sensor uses the principle of the reﬂective diaphragm, in which deﬂection of the diaphragm under the inﬂuence of pressure changes is used to couple light from an input ﬁber to an output ﬁber. An atomic force microscope (AFM) is composed of several optomechatronic components: a cantilever probe, a laser source, a position-sensitive detector (PSD), a piezo-electric actuator and a servo controller, and an x-y servoing stage, as shown in Figure 1.6b, which employs a constant-force mode. In this case, the deﬂection of the cantilever is used as input to a feedback controller, which, in turn, moves the piezo-electric element up and down in z, responding to the surface topography by holding the cantilever deﬂection constant. This motion yields a positional variation of light spot at the PSD, which detects the z-motion of the cantilever. The position-sensitive photodetector provides a feedback signal to the piezo-motion controller. Depending upon the contact state of the cantilever, the microscope is classiﬁed into contact AFM, intermittent AFM, or noncontact AFM. The optical disc drive (ODD) or optical storage disc is an optomechatronic system as shown in Figure 1.6c. The ODD is composed of an optical head that carries a laser diode, a beam focus servo that dynamically maintains the laser beam in focus, and a ﬁne track voice coil motor (VCM) servo that accurately positions the head at a desired track. The disc substrate has an optically sensitive medium protected by a dielectric overcoat and rotates under a modulated laser beam focused through the substrate to a diffractionlimited spot on the medium. Nowadays a washing machine effectively utilizes optoelectronic components to improve washing performance. It has the ability to feedback control the water temperature within the washing drum and adjust the washing cycle time, depending upon the dirtiness inside the washing water area. As shown in Figure 1.6d, the machines are equipped with an optomechatronic component to achieve such a function. To detect water contamination a light source and a photo-detector are installed at the drain port of the water ﬂowing out from the washing drum, and this information is fedback to the fuzzy controller to adjust washing time or water temperature [44]. The precision mini-robot equipped with a vision system [4] is carried by an ordinary industrial (coarse) robot as shown in Figure 1.6e. Its main function is ﬁne positioning of the object or part to be placed in a designated location. This vision-guided precision robot is directly controlled by visual information feedback, independently of the coarse robot motion. The robot is ﬂexible and low cost, being easily adaptable to change of batch run size, unlike the expensive, complex-and-mass production oriented equipment. This system can be effectively used to assemble wearable computers which

12

Optomechatronics

require the integration of greater numbers of heterogeneous components in an even more compact and light-weight arrangement. Optical MEM components are miniature mechanical devices capable of moving and directing a light beam as shown in Figure 1.6f. The tiny structures (optical devices such as mirrors) are actuated by means of electrostatics, electromagnetics, and thermal actuating devices. If the structure is an optical mirror, the device can move and manipulate light. In optical networks, optical MEMS can dynamically attenuate a switch, compensate, and combine and separate signals, all in an optical manner. The optical MEMS applications are increasing and classiﬁed into ﬁve main areas: optical switches, optical attenuators, wavelength tunable devices, dynamic gain equalizers, and optical add/drop multiplexes. Figure 1.6g illustrates a ﬁne image ﬁberscope device [42] which can perform active curvature operations for inspection of a tiny, conﬁned area such as a micro-factory. A shape memory alloy (SMA) coil actuator enables the ﬁberscope to move through a tightly curvatured area. The device has a ﬁne image ﬁberscope of 0.2 mm outer diameter with light guides and 2000 pixels. Laser-based rapid prototyping (RP) is a technology that produces prototype parts in a much shorter time than traditional machining processes. One use of this technology is stereo-lithography apparatus (SLA). Figure 1.6h shows the SLA which utilizes a visible or ultraviolet laser and a position servo mechanism to selectively solidify liquid photo-curable resin. The process machine forms a layer with a cross-sectional shape that has been previously prepared from computer-aided design (CAD) data of the product to be produced. By repeating the forming layers in a speciﬁed direction, the desired three-dimensional shape is constructed layer by layer. This process solidiﬁes the resin to 96% of full solidiﬁcation. After building, in a post-curing process the built part is put into an ultraviolet oven to be cured up to 100%. There are a number of manufacturing processes requiring feedback control of in-process state information that must be detected by optoelectronic measurement systems. One such process is illustrated here to help readers to understand the concept of the optomechatronic systems. Figure 1.6i shows a pipe-welding process that requires stringent weld quality control. A structured laser triangulation system achieves this by detecting the shape of a weld bead in an on-line manner and feeding back this information to a weld controller. The weld controller adjusts the weld current according to the shape of element being made. In this situation, no other effective method of instantaneous weld quality measurement can replace the visual in-process measurement and feedback control described here [21]. Basic Roles of Optical and Mechatronic Technologies Upon examination of the functionalities of a number of optomechatronic systems, we can see that there are a number of functions that can be carried

Introduction: Understanding of Optomechatronic Technology

13

out by optical technology. The major functions and roles of optical technology can be categorized into several functional domains as shown in Figure 1.7 [5]. Basic Roles of Optical Technology (1) Illumination: illumination, which is shown in Figure 1.7a, provides the source of photometric radiant energy incident to object surfaces. In general, it produces a variety of different reﬂective, absorptive, and transmissive characteristics depending on the material properties and surface characteristics of the objects to be illuminated. The illumination source emits spectral energy from a single wavelength which those produces a large envelope of wavelength. (2) Sensing: optical sensors provide fundamental information on physical quantities such as force, temperature, pressure, and strain as well as on geometric quantities such as angle, velocity, etc. This information is obtained by optical sensors using various optical phenomena such as reﬂection, scattering, refraction, interference, diffraction, and so on. Conventionally, optical sensing devices are composed of a light source, photonic sensors, and optical components such as lenses, beam splitter, and optical ﬁber as shown in Figure 1.7 b. Recently, numerous sensors have been developed using optical ﬁber for its advantages in various applications. Optical technology can also contribute to material science. The composition of chemicals can be analyzed by spectrophotometry, which recognizes the characteristic spectrum of light that could be reﬂected, transmitted, and radiated from the material of interest. (3) Actuating: light can change physical properties of materials by increasing the temperature of the material or affecting the electrical environment. The materials which can be changed by light are lead zirconate titanate (PZT) and SMA. As shown in Figure 1.7c, the PZT is composed of ferroelectrics material, in which the polar axis of the crystal can be changed by applying an electric ﬁeld. In optical PZT, an electric ﬁeld is induced in proportion to the intensity of light. The SMA is also used as an actuator. When SMA is illuminated by light, its shape is changed as a memorized shape due to the increase of temperature. On the other hand, when the temperature of SMA is decreased, its shape is recovered. The SMA is used in a variety of actuator, transducer, and memory applications. (4) Data (signal) storage: digitized data composed of 0 and 1 can be stored in media and read optically as illustrated in Figure 1.7d. The principle of optical recording is using light-induced changes in the reﬂection properties of a recording medium. That is to say, the data are carved in media by changing the optical properties in

FIGURE 1.7 Basic roles of optical technology.

14 Optomechatronics

Introduction: Understanding of Optomechatronic Technology

(5)

(6)

(7)

(8)

15

the media with laser illumination. Then, data reading is achieved by checking the reﬂection properties in the media using an optical pickup sensor. Data transmitting: light is a good medium for delivering data for its inherent characteristics such as high bandwidth unaffected by external electromagnetic noise. Laser, a light source used in optical communication, has high bandwidth and can contain a lot of data at a time. In optical communication, the digitized raw data such as text or picture are transformed into light signals and delivered to the other side of the optical ﬁber and decoded as the raw data. As indicated in Figure 1.7e, the light signal is transferred within the optical ﬁber without loss by total internal reﬂection. Data displaying: data are effectively understood by end users by visual information. In order to transfer data to users in the form of an image or graph, various display devices are used such as cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED), plasma display panel (PDP), etc. As illustrated in Figure 1.7f, they are all composed of pixel elements consisting of three basic coloremitting cells that are red, green, and blue light. Arbitrary colors can be made by the combination of these three colors. Computing: optical computing is performed by using switches, gates, and ﬂip-ﬂops in their logic operation just like digital electronic computing. Optical switches can be built from modulators using optomechanical, optoelectronic, acousto-optic, and magneto-optic technologies. Optical devices can switch states in about a picosecond or a thousandth of billionth of a second. An optical logic gate can be constructed from the optical transistor. For an optical computer, a variety of circuit elements besides the optical switch are assembled and interconnected, as shown in Figure 1.7g. Light alignment and waveguide are two big problems in the actual implementation of the optical computer. Material property variation: when a laser is focused on a spot using optical components, the laser power is increased on a small focusing area. This makes the highlighted spot of material change its state as shown in Figure 1.7h. Laser material processing methods utilize a laser beam as the energy input, and can be categorized into two groups: (1) the method for changing the physical shape of the materials, and (2) the method for changing the physical status of the materials.

Basic Roles of Mechatronic Elements The major functions and roles of mechatronic elements in optomechatronic systems can be categorized into the following ﬁve functional domains: sensing, actuation, information feedback, motion or state control, and embedded intelligence with microprocessor [6].

16

Optomechatronics

First, transducer technology used for sensing nowadays enjoys the integrated nature of mechatronics. The majorities of sensors belong to this category. Second, the drivers and actuators produce a physical effect such as a mechanical movement or a change of property and condition. Third, one of the unique functions of mechatronics is to feedback information for certain objectives. Fourth, the control of motion or state of systems is a basic functionality that can be provided by mechatronics. Last, a mechatronic system implemented with a microprocessor provides many important functions, for example, the stored or programmed control, the digital signal processing, and the design ﬂexibility for the whole system. In addition, advantages of integration within a small space and the low power consumption are attractive features. Characteristics of Optomechatronic Technology Based upon what we have previously observed from various optomechatronic systems, we can summarize the following characteristic points: (1) they possess one or more functionalities to carry out certain given tasks; (2) to produce such functionalities, several basic functional modules are required to be appropriately combined; (3) to achieve combining functions in a desired manner, a certain law of signal (energy) transformation and manipulation needs to be utilized that converts or manipulates one signal to another in a desired form, using the basic mechanical, optical, or electrical one; (4) optomechatronic systems are hierarchically composed of subsystems, which are then composed of units or components. In other words, elements, components, units, or subsystems are integrated to form an optomechatronic system. As we have seen from various illustrative examples of optomechatronic systems discussed above, optomechatronic integration causes all three fundamental signals to interact with each other as shown in Figure 1.8a. Here, three signals imply three different physical variables originated from optical, mechanical, and electrical disciplines. For instance, an optical signal includes light energy, ray and radiation ﬂux, mechanical signal, energy, stress, strain, motion and heat ﬂux, and electrical signal, current, voltage, and magnetic ﬂux, and so on. Depending on how they interact, the properties of the integrated results may be entirely different. Therefore, it is necessary to consider the interaction phenomenon from the point of view of whether the interaction may be efﬁciently realizable. Optomechatronic integration can be categorized into three classes, depending on how optical elements and mechatronic components are integrated. As indicated in Figure 1.8b, the classes may be divided into the following. (1) Optomechatronically fused type In this type, optical and mechatronic elements are not separable in the sense that if either are removed from the system that they constitute, the system cannot function properly. This implies that those two separate

Introduction: Understanding of Optomechatronic Technology

17

physical inter action

(a) optomechatronic interaction

(b) integration type

dimensional interaction

(c) interaction between three different dimensional worlds FIGURE 1.8 Characteristic features of optomechatronic technology.

elements are functionally and structurally fused together to achieve a desired system performance. Figure 1.9a shows the systems integrated according to this type and include an optically ignited weapon system (left) and a photostrictive actuator (right). The constituent element of each system is shown in the ﬁgure where O, M, and E stands for optical, mechanical, and electrical, respectively. (2) Optically embedded mechatronic type In this type, an optical element is embedded into a mechatronic system. The optical element is separable from the system, but the system can function with a decreased level of performance or can function in an entirely different manner. The majority of engineered optomechatronic systems belong to this category, such as servomotor, washers, vacuum cleaners, monitoring and control systems for machines and manufacturing processes, robots, cars, and so on. The anatomy of the servo motor is shown in Figure 1.9b. (3) Mechatronically embedded optical type This type can be found basically from an optical system whose construction is integrated with mechanical and electrical components. Many optical systems require positioning or servoing optical elements and devices to manipulate and align a beam and to control the polarization of the beam. Typical systems that belong to “positioning or servoing” include cameras, optical projectors, galvanometers, series parallel scanners, line

18

Optomechatronics

FIGURE 1.9 Illustration of types of optomechatronic integration. Source: Cho, H.S. and Kim, M.Y., IEEE Transactions on Industrial Electronics, 52:4, 932–943, 2005. q 2005, IEEE.

scan polygons, optical switches, ﬁber squeezer polarization controllers, and so on. In some other systems, an acoustic wave generator driven by a piezo-element is used to create a frequency shift of a beam. Another typical system is the passive (off-line) alignment of optical ﬁber–ﬁber or ﬁber– waveguide attachment. In this system, any misalignment is either passively or actively corrected by using a micropositioning device to maximize their coupling between ﬁber – ﬁber or ﬁber wave guide. The anatomy of a zoom lens that belongs to this type is shown in Figure 1.9c and consists of ﬁve basic elements. Traditional optomechatronic systems usually operate in macro-worlds, if we term a macro-world as a large system whose size can range from mm to a few meters. As brieﬂy mentioned before, due to the miniaturization trend and availability of MEMS and nano-technologies, optomechatronic integration is becoming popular, and thus accelerating the downsizing of engineering products. Optomechatronic integration is therefore achieved in two different physical scales as shown in Figure 1.8c: macro- and micro-nano scales. This length scale difference produces three different types of integration: integration in macro-scale (left), integration in micro/nanoscale (right) and integration in mixed scales (middle). The middle part of the ﬁgure implies an employment of combined macro- and micro/nano-systems for the integration. Integration of this type can be found from a variety

Introduction: Understanding of Optomechatronic Technology

19

* atom nanotube virus & bacteria optical lithography

human hair optical fiber 10

10−5 10

−3

MEMS

10−11

10−10 10−9

STM AFM

x-ray nanomanipulation thin film zone plate ring spacing

10−8 10−7

10−6

biological cell

FET

DNA CNT

NEMS

visible light

−4

infrared

* scale in meters

FIGURE 1.10 Scales and dimensions of some entities that are natural and man-made.

of physical systems such as an optical disc (macro) having a micro ﬁneadjustment mechanism, a laser printer that may be operated by micro-lens slits, an AFM, etc. Figure 1.10 depicts scales of some natural entities and some man-made ones. MEMS parts range from hundreds mm to few mm and NEMS parts from few hundreds to tens nm. Nanomanipulation such as atomic lettering using scanning tunneling microscope (STM) is of the order of approximately nm or less. It is known that most of the microscopes currently available have used extremely high resolution. In particular, the extreme case of the smallest scale can be found from STM and AFM whose resolutions range between , 0.1 nm and , 0.5 nm, respectively. One important factor we need to consider in dealing with micro-systems is the so-called “size or scale effect” due to which the inﬂuence of the same physical phenomena becomes variable with the length scale of a system. To take some examples: (1) Inertial (volume) force that inﬂuences a macro-scale system becomes less signiﬁcant in micro-systems. Instead, surface effects (force) become rather dominant. This is because in general the inertial force is proportional to volume (L 3), whereas the surface force is in proportion to surface area (L 2). (2) In optical and mechatronic interaction, deﬂections on the order of wavelengths of light do matter in a microsystem but not in a macrosystem.

20

Optomechatronics

(3) Atomic attractive and repulsive forces become signiﬁcant in microsystems. (4) The smaller geometrical size material gets stronger. Example (1) may mislead us into neglecting the less signiﬁcant terms in analyzing microsystems. However, in the case of dynamic analysis and control of the systems, they may not be negligible when the accuracy of the positioning is in high precision or when the range of dominant force operated on the systems is compatible with that of the less signiﬁcant terms.

Fundamental Functions of Optomechatronic Systems From the discussions in the previous sections, we now elaborate on what type of fundamental functions the optomechatronic integration can produce.

Information feedback control Mechanism design

Precision actuators Optical / Visual Motion control

Artificial intelligence Micro processor

sensors, actuators data display materials processing information processing recognition transmission / switching micro elements

Signal processing Pattern recognition

MEMs

Sensor fusion

FIGURE 1.11 Enabling technologies for optomechatronics.

Sensors & measurement

Introduction: Understanding of Optomechatronic Technology

21

There are a number of distinct functions which originate from the basic roles of optical elements and mechatronic elements discussed. When these are combined, the functional forms may generate the fundamental functions of the optomechatronic system. Figure 1.11 illustrates the technology element needed to produce optomechatronic technology. In the center of the ﬁgure, those elements related to optical elements are shown, while mechatronic elements and artiﬁcial intelligence are listed on its periphery. These enabling technologies are integrated to form ﬁve engines that drive the technology. These include: (1) actuator module, (2) sensor module, (3) control module, (4) signal/ information processing module, and (5) decision making module. It is noted that the integration of these gives a speciﬁed functionality with certain characteristics. A typical example is the sensing of an object surface with an AFM. The AFM requires a complicated interaction between sensing element (laser sensor), actuator (piezo-element), controller, and other relevant software. This kind of interaction between modules is very common in optomechatronic systems, and produces the characteristic property of the systems.

Fundamental Functions A number of fundamental functions can be generated by the fusion of optical mechatronic elements. These include (1) illumination control, (2) sensing, (3) actuating, (4) optical scanning, (5) visual/optical information feedback control, (6) data storage, (7) data transmission/switching, (8) data display, (9) optical property variation, (10) sensory feedback based optical system control, (11) optical pattern recognition, (12) remote monitoring/control, and (13) material processing [5]. We will discuss these fundamental functions below.

Illumination Control Illumination needs to be adjusted depending on the optical surface characteristics and surface geometry of objects in order to obtain a good quality image. The parameters to be adjusted include incident angle, and the distribution and intensity of light sources. Figure 1.12a illustrates a typical conﬁguration of such an illumination control system. The system consists of a guardant ring ﬁber light source, a paraboloid mirror, a toroid mirror, and a positioning mechanism to adjust the distance between the object surface and the parabola mirror. The incident angle is controlled by adjusting the distance, while the intensity of the light source in each region is controlled independently.

object

3-facet mirror xs

object

PSD B

ys

laser zs beam

glass grid

adhesive film

pressure transducer diode strip

(e) motion control : placing diode strip into a glass

camera

detector

opticsservo

(f) data storage/ retrieval

laser

disk

servo

6-D position & orientation laser beam position

tracking control

(b) measurement of a 6-degrees-of-freedom motion of arbitrary objects

PSD A

vacuum-based end effector fiducial mark

cartesian robot

laser source PSD C

FIGURE 1.12 The functionalities achieved by optomechatronic integration.

(d) mirror based scanning

laser

mirror

parabolic mirror

mirror

mirror

lens

laser

(a) illumination control

object lens

half-mirror

camera

mirror

(g) optical cross connect operation by mirror control

scanning mirror

servoing mirror array

(c) shape memory-based optical actuating

spring

motion

light

22 Optomechatronics

23

FIGURE 1.12 Continued.

Introduction: Understanding of Optomechatronic Technology

24

Optomechatronics

Sensing Various types of optomechatronic systems are used to measure various types of physical quantities such as displacement, geometry, force, pressure, temperature, target motion, etc. The commonly adopted feature of these systems is that they are composed of optical, mechanical moving, servoing, and electronic elements. The optical elements are usually the sensing part divided into optical ﬁber and nonﬁber optical transducers. Recently these elements have been accelerating the sensor fusion in which other types of sensors are fused together to obtain necessary information. Figure 1.12b shows a six-dimensional (6D) sensory system employing the optomechatronic principle. The sensor is composed of a 6D microactuator, a laser source, three photosensitive devices (PSD), and a three-facet tiny crystal (or mirror) that transmits the light beam into three PSD sensors. The sensing principle is that the microactuator is so controlled that the top of the tiny mirror is always positioned within the center of the beam. In other words, the output of each PSD sensor is kept identical regardless of object motion. Another typical problem associated with optical sensing is focusing, which needs to be adjusted depending upon the distance between the optical lens and the surface of the objects to be measured. Actuating A number of actuators belong to this category, both optical-based and mechatronic actuation-based. Optical-based actuation utilizes transformation of optical energy into mechanical energy that can be found from a number of practical examples. Figure 1.12c shows one such actuator in which optical energy is used as a heat generator which drives a temperaturesensitive material (e.g., SMA) to move. This actuator can be used to accurately control the movement of mechanical elements. Optical Scanning Optical scanning is used to divert light direction with time in a prescheduled manner and generates a sequential motion of the optical element such as a light source. For high-speed applications, a polygon mirror or galvanometer is effectively used as shown in Figure 1.12d. To operate a scanning system, the scanning step and the scanning angle have to be considered carefully. The scanning ability is useful in applications ranging from laser printing to materials processing. Visual/Optical Information Feedback Control Visual/optical information is very useful in the control of machines, processes, and systems. In this functionality the information obtained by optical sensors is utilized for changing variables operating the systems. A great number of optomechatronic systems require this type of information

Introduction: Understanding of Optomechatronic Technology

25

feedback control. Positioning controlling the position of an object, moving it to a speciﬁed location. When a visual sensor is used for the feedback, the motion control is called “visual servoing.” Accurately placing diode strips into a glass grid by using a robot is shown in Figure 1.12e as an illustrative example. A number of practical examples can be found from motion tracking such as following the motion of a moving object and following a desired path based on the optical-sensor information. Many more typical examples can also be found from mobile robot navigation, e.g., optical-based dead reckoning, map building, and vision-based obstacle avoidances. A pipe-welding process shown in Figure 1.6i necessitates an optical information feedback control. The objective of this process control is to regulate the electric power needed for welding in order to attain a speciﬁed weld quality. Therefore, the process state relevant to the quality is detected and its instantaneous information is fedback in real-time manner. This principle can also be found from the washing machine shown in Figure 1.6d. In this system, the optical sensor detects the dirtiness of the water and this information is fedback to a controller to adjust the washing time or to adjust the water temperature inside the drum. Data Storage Data storage retrieval is performed by a spinning optical disc and controlled optical units whose main functions are beam focusing and track following, as illustrated in Figure 1.12f. Conventionally, the recording density is limited by the spot size and wavelength of the laser source. Recently, new approaches to increase the recording density are being researched such as near-ﬁeld optical memory and holographic three-dimensional storing methods. Data Transmission/Switching Optical data switching is achieved by an “all optical network” to eliminate the multiple optical-to-electrical-to-optical (O-E-O) conversions in conventional optical networks [35]. Figure 1.12g illustrates a multi-mirror servo system in which mirrors are controlled to connect a light path any to any ports by using servo controllers. This kind of system conﬁguration is also applied to switch micro-reﬂective or -deﬂective lens actuated in front of optical ﬁlters. Data Display Digital micro-mirror devices (DMDs) make projection displays by converting white-light illumination into full-color images via spatial light modulators with independently addressable pixels [30]. As schematically illustrated in Figure 1.12h, the DMD developed by Texas Instruments is a lithographically fabricated MEMS system composed of hundreds of thousands of titling aluminum-alloy mirrors (16 mm £ 16 mm) which

26

Optomechatronics

functions as a pixel in the display. Each mirror is attached to an underlying sub-pixel called the yoke, which, in turn, is attached to a hinge support post by way of an aluminum torsion hinge. This allows the mirror to rotate about the hinge axis until the landing tips touch the landing site. This switching action occurs from þ 108 to 2 108 and takes place in several msec. Optical Property Variation Optical properties such as frequency, wavelength, and so on can be tuned by making light waves interact with mechatronic motion. Figure 1.12i illustrates a frequency shifter which consists of an acoustic wave generator, a piezoelectric element, and a high birefringence ﬁber. In-line ﬁber optic frequency shifters utilize a traveling acoustic wave to couple light between the two polarization modes of a high-birefringence ﬁber, with the coupling accompanied by a shift in the optical frequency. Sensory Feedback-Based Optical System Control In many cases, optical or visual systems are operated based on the information which is provided by external sensory feedback, as shown in Figure 1.12j. The systems that require this conﬁguration include: (1) zoom and focus control system for video-based eye-gaze detection using an ultrasonic distance measurement, (2) laser material processing systems whose laser power or laser focus is real-time controlled depending upon the process monitoring, (3) visually-assisted assembly systems in cooperating with other sensory information such as force, displacement, and tactile, (4) sensor fusion systems in which optical/visual sensors incorporate with other sensory systems, and (5) visual/optical systems that need to react to acoustic sound and other sensory information tactile, force, displacement, velocity, etc. Optical Pattern Recognition Three-dimensional pattern recognition shown in Figure 1.12k, uses a laser and a photorefractive crystal as the recording media to record and read holograms [38]. The photorefractive crystal stores the intensity of the interference fringe constructed by the beam reﬂected from a 3D object and a plane-wave reference beam. The information on the shape of the object is stored in the photorefractive crystal at this point, and this crystal can be used as a template to be compared with another three-dimensional object. To recognize an arbitrary object, an optical correlation processing technique is employed. When an object to be recognized is placed in the right position from the original object with the same orientation, the recorded hologram diffracts the beam reﬂected from the object to be compared, and the

Introduction: Understanding of Optomechatronic Technology

27

diffracted wave propagating to the image plane forms an image that represents the correlation between the Fourier transforms of the template object beam and the object to be compared. If a robot or a machine is trying to ﬁnd targets in unknown environments, this system can be effectively used to recognize and locate them in a real-time manner. In this case a scanned motion of the optical recognition system is necessary over the whole region of interest. Remote Operation via Optical Data Transmission Optical data transmission is widely used when data/signals obtained from sensors are subject to external electrical noise or when the amount of data needed to be sent is vast, or when operation is being done at a remote site. Operation of systems at a remote site is ubiquitous nowadays. In particular, internet-based monitoring, inspection, and control are becoming pervasive in many practical systems. Visual servoing of a robot operated in a remote site is a typical example of such a system, as shown in Figure 1.12‘. In the operation room, the robot controls the position of a vibration sensor to monitor the vibration signal with the aid of vision information and transmits this signal to a transceiver. Another typical example is the operation of the visual servoing of mobile robots over the internet. Material Processing Material processing can be achieved by the integration of a laser optical source and a mechatronic servo mechanism. The system produces a material property change or cut surface and heat-treated surface of work pieces. In addition to conventional laser machining, laser micro-machining is becoming popular due to its reduced cost and accuracy. MEMS fabrication, drilling and slotting in medicine, wafer dry cleaning, and ceramic machining employ such micro-machining technology. Figure 1.12m shows a laser surface hardening process [46], which is a typical example of an optical-based monitoring and control system. The laser of high power (4 kW) focused through a series of optical units hits the surface of a workspace and changes the material state of the workpiece. Maintaining a uniform thickness of the hardened surface (less than 1 mm) is not an easy task since it depends heavily upon several process parameters such as the workpiece travel speed, laser power, and the surface properties of the material. Here, the indirect measurement of the coating thickness is made by an infra-red temperature sensor and feedback to the laser power controller.

Summary In recent years, integration of optical elements into mechatronic systems has increasingly accelerated since it produces a synergistic effect, creating new

28

Optomechatronics

functionalities for the systems or enhancing the performance of the systems. This trend will certainly be the future direction of current mechatronic technology and contribute to the advent of a new technological paradigm. This chapter has focused on helping readers to understand optomechatronic technology and systems in which optical, mechanical, electrical/electronics, and information engineering technologies are integrated. In particular, the deﬁnition and fundamental concepts of the technology have been introduced. Based upon these, it has been possible to identify types of optomechatronic systems and the fundamental functions that can be created by the integration. Altogether, thirteen functionalities have been identiﬁed. They are expected to lead to future developments of such technologies as mechatronics, instrumentation and control, adaptive optics, MEMS, biomedical technology, information processing storage as well as communication. From this point of view, optomechatronics will be a prime technological element that will lead the future direction of various technologies.

References [1] [2] [3] [4] [5] [6] [7] [8]

[9] [10] [11] [12] [13]

Backmutsky, V. and Vaisman, G. Theoretical and experimental investigation of new mechatronic scanners, Proceeding of the IEEE Nineteenth Convention of Electrical and Electronics Engineers in Israel, pp. 363– 366, 1996. Bishop, R.H. The Mechatronics Handbook. CRC Press, Boca Raton, FL, 2002. Cannon Co. Ltd, http:www.canon.com, 2002. Chen, M. and Hollis, R. Vision-guided Precision Assembly. 1999, http://www-2. cs.cmu.edu/afs/cs/project/msl/www/tia/tia_desc.html. Cho, H.S. and Park, W.S. Determining optimal parameters for stereolithography process via genetic algorithm, Journal of Manufacturing Systems, 19:1, 18 – 27, 2000. Cho, H.S. Characteristics of optomechatronic systems, Opto-mechatronic Systems Handbook, chap. 1, CRC Press, Boca Raton, FL, 2002. Cho, H.S. and Kim, M.Y., Optomechatronic technology: the characteristics and perspectives, IEEE Transaction on Industrial Electronics, 52:4, 732– 743, 2005. Ebisawa, Y., Ohtani, M. and Sugioka, A. Proposal of a zoom and focus control method using an ultrasonic distance-meter for video-based eye-gaze detection under free-head conditions, 18th Annual Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 2, pp. 523– 525, Amsterdam, Netherlands, 1997. Geppert, L. Semiconductor lithography for the next millennium, IEEE Spectrum, April, 33 – 38, 1996. Han, K., Kim, S., Kim, Y. and Kim, J. Internet control architecture for internetbased personal robot, Autonomous Robots, 10, 135– 147, 2001. Haran, F.M., Hand, D.P., Peters, C. and Jones, J.D.C. Real-time focus control in laser welding, Measurement Science and Technology, Vol. 7, 1095– 1098, 1996. Higgins, T.V. Optical storage lights the multimedia future, Laser Focus World, September/October, Vol. 31, 1995. Ishi, T. Future trends in mechatronics, JSME, International Journal, Series, III, 33:1, 1 – 6, 1990.

Introduction: Understanding of Optomechatronic Technology [14]

29

Jeong, H.M., Choi, J.J., Kim, K.Y., Lee, K.B., Jeon, J.V. and Park, Y.E., Milli-scale mirror actuator with bulk micromachined vertical combs, Proceedings of the Transducer, Sendai, Japan, pp. 1006– 1011, 1999. [15] Kamiya, M., Ikeda, H. and Shinohara, S. Data collection and transmission system for vibration test, Proceedings of the Industry Application Conference, 3, 1679–1685, 1998. [16] Kayanak, M.O. The age of mechatronics, IEEE Transactions in Industrial Electronics, 43:1, 2 – 3, 1996. [17] Kim, W.S. and Cho, H.S. A novel sensing device for obtaining an omnidirectional image of three-dimensional objects, Mechatronics, 10, 717– 740, 2000. [18] Kim, J.S. and Cho, H.S. A robust visual seam tracking system for robotic arc welding, Mechatronics, 6:2, 141– 163, 1996. [19] Kim, W.S. and Cho, H.S. A novel omnidirectional image sensing system for assembling parts with arbitrary cross-sectional shapes, IEEE/ASME Transactions in Mechatronics, 3:4, 275– 292, 1998. [20] Knopf, G.K. Short course note, SC 255 Opto-Mechatronic System Design, SPIE’s Photonic East, Boston, USA, 2000. [21] Ko, K.W. Cho, H.S., Kim, J.H and Kong, W.I. A bead shape classiﬁcation method using neural network in high frequency electric resistance weld, Proceeding of World Automation Congress, Alaska, USA. [22] Krupa, T.J. Optical R&D in the Army Research Laboratory, Optics&Photonics News, June, 16 – 39, 2000. [23] Larson, M.C. and Harris, J.S. Wide and continuous wavelength tuning in a vertical cavity surface emitting laser using a micromachined deformable membrane mirror, Applied Physics Letters, 15, 607– 609, 1996. [24] Larson, M.C. Tunable Optoelectronic Devices. 2000, http://www-snow.stanford. edu/~larson/research.html. [25] Lim, T.G. and Cho, H.S. Estimation of weld pool sizes in GMA welding process using neural networks, Proceedings of Institute of Mechanical Engineers, 207, 15 – 26, 1993. [26] Madou, M. Fundamentals of Microfabrication. CRC Press, Boca Raton, FL, 1997. [27] Mahr Co. Ltd, Multiscope 250/400. 2001, http://www.mahr.com/en/content/ products/mess/mms/ms250.html. [28] McCarthy, D.C. Hands on the wheel, cameras on the road, Photonics Spectra, April, 78 – 85, 2001. [29] Mitutoyo Co. Ltd, Quick Vision. 2001, http://www.mitcat.com/e-02.htm. [30] McDonald, T.G. and Yoder, L.A. Digital micromirror devices make projection displays, Laser Focus World, August, 1997. [31] McKee, G. Robotics and machine perception, SPIE’s International Technical Group Newsletter, 9, 2000. [32] Park, I.O., Cho, H.S. and Gweon, D.G. Development of programmable bowl feeder using a ﬁber optic sensor, 10th International Conference on Assembly Automation, Tokyo, Japan, 1989. [33] Park, W.S., Cho, H.S., Byun, Y.K., Park, N.Y., and Jung, D.K. Measurement of 3-D position and orientation of rigid bodies using a 3-facet mirror, SPIE International Symposium on Intelligent Systems and Advanced Manufacturing, pp. 2– 13, Boston, USA. [34] Pugh, A. Robot Sensors, Tactile and Non-Vision, Vol. 2. Springer-Verlag, Berlin, 1986.

30

Optomechatronics

[35] Robinson, S.D. MEMS technology — micromachines enabling the all optical network, Electronic Components and Technology Conference, pp. 423 – 428, Orlando, FL, 2001. [36] Rogers, C.A. Intelligent materials, Scientiﬁc American, September, 122–127, 1995. [37] Roh, Y.J., Cho, H.S. and Kim, J.H. Three dimensional volume reconstruction of an object from X-ray images, Conference on SPIE Optomechatronic Systems Part of Intelligent Systems and Advanced Manufacturing, Vol. 4190, pp. 181–191, Boston, USA, 2000. [38] Shin, S.H. and Javidi, B. Three-dimensional object recognition by use of a photorefractive volume holographic processor, Optics Letters, 26:15, 1161– 1163, 2001. [39] Shiraishi, M. In-process control of workpiece dimension in turning, Annals of the CIRP, 28:1, 333– 337, 1979. [40] Takamasu, K. Development of a nano-probe system, Quarterly Magazine Micromachine, 35, 2001. [41] Toshiyoshi, H., Su, J.G., LaCosse, J. and Wu, M.C. Micromechanical lens scanners for ﬁber optic switches, Proceedings of the Third International Conference on Micro Opto Electro Mechanical Systems (MOEMS 99), pp. 165– 170, Mainz, Germany, , 1999. [42] Tsuruta, K., Mikuriya, Y. and Ishikawa, Y. Micro sensor developments in Japan, Sensor Review, 19, 37 – 42, 1999. [43] Veeco co. Ltd, Contact AF. 2000, http://www.tmmicro.com/tech/modes/ contact.htm. [44] Wakami, N., Nomura, H. and Araki, S. Fuzzy logic for home appliances, Fuzzy Logic and Neural Networks, pp. 21.1 – 21.23, McGraw Hill, New York, 1996. [45] Wilson, A. Machine vision speeds robot productivity, Vision Systems Design, October, 2001. [46] Woo, H.G. and Cho, H.S. Estimation of hardened layer dimensions in laser surface hardening processes with variations of coating thickness, Surface and Coatings Technology, 102:3, 205– 217, 1998. [47] Zankowsky, D. Applications dictate choice of scanner, Laser Focus World, December, 1996. [48] Zhang, J.H. and Cai, L. An autofocusing measurement system with a piezoelectric translator, IEEE/ASME Transactions in Mechatronics, 2:3, 213– 216, 1997. [49] Zhou, Y., Nelson, B.J. and Vikramaditya, B. Integrating optical force sensing with visual servoing for microassembly, Journal of Intelligent and Robotic Systems, 28, 259– 276, 2000.

2 Fundamentals of Optics CONTENTS Reﬂection and Refraction ................................................................................... 33 Lenses .................................................................................................................... 36 Refraction at a Spherical Surface............................................................... 37 Multiple Lenses and System Matrices.............................................................. 46 The System Matrices.................................................................................... 50 Computer Ray Tracing ................................................................................ 52 Aperture Stops and Pupils ................................................................................. 53 Aberration ............................................................................................................. 55 Spherical Aberration.................................................................................... 56 Coma Aberration.......................................................................................... 56 Field Curvature ............................................................................................ 57 Astigmatism .................................................................................................. 57 Distortion....................................................................................................... 59 Polarization ........................................................................................................... 61 Coherence.............................................................................................................. 64 Interference ........................................................................................................... 65 Young’s Experiment .................................................................................... 67 Fabry-Perot Interferometer ......................................................................... 69 Michelson Interferometer ........................................................................... 72 Diffraction.............................................................................................................. 74 Double Slits ................................................................................................... 79 Multiple Slits................................................................................................. 81 Circular Aperture......................................................................................... 84 Diffraction Grating....................................................................................... 89 Optical Fiber Transmission................................................................................. 92 Gaussian Beam Optics ........................................................................................ 95 Problems................................................................................................................ 97 References ........................................................................................................... 103

Physical optics is rooted on the fact that light corresponds to the electromagnetic radiation in the narrow slice of spectrum. Optics concerns

31

32

Optomechatronics

with the radiation within this range visible to the human eye but often extends to the region of spectrum near visible region. Classiﬁcation due to variation in wave length is summarized in Figure 2.1. Visible radiation ranges from 0.35 to 0.75 mm, depending upon types of color of the light their wave length varies. The ultraviolet region extends from 0.01 to 0.35 mm, while infrared radiation extends from the lower end of the visible spectrum to nearly 300 mm. Being conduced as one of the electromagnetic waves, light radiates from a source, propagating straight through a medium at a velocity, c, in a form of combination of electric and magnetic waves oscillating at a frequency v and with a wave length l: The simplest form of such combination of waves is a monochromatic (single color) light wave that is composed of sinusoidal components of electric and magnetic ﬁelds, propagating in free space along the direction z as represented in Figure 2.2. The wave here is a plane wave and the electric ﬁeld is varying parallel to the propagation direction. As can be seen from the ﬁgure, the electric and magnetic ﬁelds are vector quantities, and thus their magnitude and direction need to be speciﬁed. However, both ﬁelds are oriented at right angles to each other in nonabsorbing media. Only the direction and magnitude of the electric ﬁeld need to be speciﬁed. We will return to this subject with a little bit more detail when we discuss polarization of light in this chapter. The speed of propagation of the wave in

wavelength (m) 10−10

10−8

frequency (HZ) X-ray X -ray

UV UVlight light

1×1018

1×1016

visible 1×1014

10−6 infrared

1×1012

10−4

10−2

μ-waves μ -waves

FIGURE 2.1 The electromagnetic spectrum: optical radiation.

1×1010

Fundamentals of Optics

33 y E x B

B

E

E

B

B

E

z

FIGURE 2.2 Propagation of a plane-polarized electromagnetic wave.

free space (vacuum) is approximately given by c ¼ 2:998 £ 108 m/sec. The speed in air is just a little bit smaller than this, while that in glass and water is always much smaller than that in vacuum. Throughout this book we will conﬁne ourselves to optical problems concerning visible and monochromatic light. In the ﬁrst part of this chapter, we will treat a light wave as if it travels from its source along straight lines without any wave motion. Using this treatment, we therefore cannot explain such phenomena as the interference, polarization, diffraction and so on that occur due to wave motion of the light. The regime of this treatment is called “geometric optics” within which the wavelength or wave motion effect is considered negligible compared to the dimensions of the optical system interacting with light waves. In the later part of this chapter, we will treat wave optics, in which wave motion of light will be included, to describe the above phenomena that cannot be explained by geometric optics.

Reﬂection and Refraction A light wave transmitting through a lens or a prism is typical of wave refraction. A light wave reﬂecting from the surface of a mirror or a coated reﬂective surface is typical of wave reﬂection. To understand these phenomena we will review the laws of reﬂection and refraction. When a light wave travels through several homogeneous media in sequence, its optical path constitutes a sequence of discontinuous line segments. A simple case of this situation occurs when a light ray is incident upon a plane surface separating two media. There are several laws governing the direction of light propagation at the interface. Here, we will discuss brieﬂy

34

Optomechatronics

FIGURE 2.3 Reﬂection ur and refraction ut at the boundary of two different media.

the two most frequently used laws, reﬂection and refraction. When a light wave is incident on an interface lying in the plane of incidence shown in Figure 2.3, a certain fraction of the wave is absorbed or transmitted and the remainder is reﬂected. This is illustrated using a plane wave in Figure 2.4a. As can be seen from the ﬁgure, three waves lie in the plane of incidence and those waves are conveniently represented by three corresponding rays

incident light

normal

A

normal

reflected light

reflected light

incident light

C

qi medium 1 medium 2

B D

refracted light

(a) a beam of plane waves enters the interface

(b) reflection

normal incident light

qi

ni qt

nt refracted light

(c) refraction FIGURE 2.4 A plane wave front incident on the boundary between two media.

qr

Fundamentals of Optics

35

AB; BC; BD: A ray is a line that indicates the direction of wave propagation and is perpendicular to the wave front. Therefore, we will transform this ﬁgure into those represented only by rays as shown in Figure 2.4b. The law of reﬂection states that the angle of reﬂection equals the angle of incidence. This means that

ui ¼ ur

ð2:1Þ

This is a special case of light reﬂection. In general cases, most interfaces contain both specular and diffuse surfaces. Diffuse surfaces cause the reﬂected light to scatter in all directions radiating an equal amount of power per unit area per unit solid angle. The condition given in Equation 2.1 is valid for the specular reﬂection. When an incoming plane light passes through the boundary between two homogeneous media, the direction of it will bend because of its speed change. The amount it is bent is dependent upon the absolute refraction index of each media, which is deﬁned by n ¼ c=v, where c is speed of light in a vacuum and v is the speed in the medium. The law of refraction, called Snell’s law, states that the refraction angle ut obeys the following relation, ni sin ui ¼ nt sin ut

ð2:2Þ

where ni is the refraction index of the incident medium, while nt is that of the transmitting medium. When ni , nt , that is, the ray enters a medium with higher index (optically more dense), it bends toward the normal. In contrast, when ni . nt , (from optically more to less dense) it bends away from the normal. There are some special cases where further thoughts on Snell’s law are necessary. To explain this, let us suppose that incident light coming from a point source located below an interface travels across the upper medium of lower relative index as shown in Figure 2.5. When ui ¼ 0, that is, an incident normal to the surface ray is transmitted straight across the boundary without change of direction (case 1). However, as the incident angle ui becomes larger and larger, the angle of refraction becomes large. (case 1)

(case 2)

(case 3)

(case 4)

qt nt

qt

qr = qi = 0

nii

qi

S FIGURE 2.5 Various types of refraction phenomena.

qi

90° qc

qi

qi

36

Optomechatronics

TABLE 2.1 Refractive Index of Various Materials Material Vacuum Air (208C) Water vapor Optical glass Quartz Crystal Plexiglas Lucite Zinc crown glass Light ﬂint glass Heavy ﬂint glass Heaviest ﬂint glass Sugar solution (30%) Polymer optical ﬁber (polymethyl ethacrylate PMMA) Polymer optical ﬁber (polystyrene) Polymer optical ﬁber (polycarbonates)

Refractive Index (n) 1.0000000 1.000292 1.00024 1.5 2 01.85 1.4868 2.0 1.488 1.495 1.517 1.575 1.65 1.89 1.38 1.49 1.59 1.5 2 1.57

Finally, there will be a critical angle of incidence that will make ut ¼ 908 (case 3). This angle is called the “critical angle, uc ” which is given by

uc ¼ sin21

nt ni

ð2:3Þ

When the angle of incidence ui . uc , the incident ray exhibits total internal reﬂection, as shown in case (4). This type of reﬂection is effectively used for ﬁber optic transmission, which will be discussed at the end of this chapter. The refraction index of some materials is listed in Table 2.1.

Lenses Figure 2.6 shows when a point source S produces a spherical wave propagating toward a convex or concave lens. The waves expand and intercept the thin lens. In case of the convex lens, the wave fronts contract and converge to a point I, where real image is formed. For the concave lens, the wave fronts expand further so that a virtual image I is created at the backside of the lens. This point is known as a focus of the bundle of rays. This implies that the change in the curvature of the wave fronts occurs due to the refracting power of the lens. One other thing to note is that, in both lenses,

Fundamentals of Optics

37

S

I image point

object point

S object point I image point

(a) convex lens

(b) concave lens

FIGURE 2.6 Interference of a spherical wave with two typical lenses.

if a point source is placed at the image point I, the corresponding image will be located at S. These two points S and I are called “conjugate points”. Refraction at a Spherical Surface A spherical surface is commonly used as a lens surface and can be either concave or convex. Figure 2.7 shows a concave surface of radius R where two rays from a point source S are emanating as indicated by an arrow. One is an axial ray impinging a spherical interface centered at C, which is normal to the surface at its vertex point V. The source is located at a distance so from V. The other ray incident at a point P with an angle ui is refracted with an angle ut , as indicated by an arrow. According to Snell’s law, this ray refracts with the relation, n1 sin ui ¼ n2 sin ut

ð2:4Þ

qt

P

h

qi j

V

n2

s0

n1

FIGURE 2.7 Refraction at a spherical interface.

si

j¢ S R

α I

C

38

Optomechatronics

These two rays appear to be emanating from a point I which is called the image point, whose distance is si from V. The exact relationship between so and si is rather complicated to derive, but we will simplify by assuming ! ! SP ¼ SV

and

! ! IP ¼ IV

ð2:5Þ

by using small angle approximation i.e., sin u . u; cos u . 1: From this relation, it follows from Equation 2.4 that n1 ðw 2 aÞ ¼ n2 ðw0 2 aÞ

ð2:6Þ

Taking the tangents for the angles in the above equation and utilizing the assumption given in Equation 2.5, Equation 2.6 further leads to n1 n n 2 n2 2 2 ¼ 1 ð2:7Þ so si R It is noted that the object distance, so is positive whereas si , the distance of the image (virtual), is negative and R is negative according to the sign convention for spherical refraction surfaces and thin lenses. This equation then can be generalized to include the case of a convex surface. The general form of Equation 2.7 can be expressed as n1 n n 2 n1 þ 2 ¼ 2 ð2:8Þ so si R The above equation is valid for rays that are incident to the surface near to the optical axis such that w and h are small. These rays are known as paraxial rays. The optics dealing with such rays arriving at the paraxial region is called Gaussian optics, named after K. F. Gauss (1777 –1855), the ﬁrst to develop this formulation. In the preceding discussion, we have dealt with a single spherical surface. In order to relate the source points and image point P we now apply this method to determine the locations of the conjugate points for a thin lens, which represents the thin-lens equation. In this case, two refractions at two spherical surfaces are involved. Referring to Figure 2.8, the ﬁrst refractions at a spherical interface of radius R1 yields n1 n n 2 n1 þ 2 ¼ 2 so1 si1 R1

ð2:9Þ

where so1 is the distance of the source S from the vertex point V1 and si1 is the distance of the image from the vertex point V1 : At the second spherical surface of radius R2 n2 n n 2 n2 þ 1 ¼ 1 ð2:10Þ so2 si2 R2 Here, the image point P0 acts as a source object point for the second surface, which is considered the second object. Now since the lens thickness is

Fundamentals of Optics

39

R1

R2 P'

P C2

S

s i1

n1

V1

so1

O n2

V2

n1

C1 si2

d

so2

FIGURE 2.8 A point source passing through a spherical lens having two different spherical interfaces.

negligible for a thin lens, i.e., d ! 0, so2 becomes so2 ¼ 2si1

ð2:11Þ

Combining Equation 2.9, Equation 2.10 and Equation 2.11 leads to 1 1 n 2 n1 þ ¼ 2 so si n1

1 1 2 R1 R2

ð2:12Þ

where so1 ¼ so and si2 ¼ si are substituted. It is noted that so and si is measured from either the vertices V1 and V2 or the lens center, O: When a source is located at inﬁnity, we can see that the image distance becomes the focal length f : The focal length of the thin lens is therefore deﬁned as the image distance for an object at inﬁnity. In this case, Equation 2.12 can be rewritten as 1 ðn 2 n1 Þ 1 1 2 ¼ 2 f n1 R1 R2

ð2:13Þ

This equation is referred to as the lens maker’s formula. When a surrounding medium is air, n1 becomes approximately unity, i.e., n1 . 1: In this case, the lens formula becomes 1 1 1 ¼ ðn2 2 1Þ 2 f R1 R2 By using Equation 2.12 and Equation 2.13, the thin lens equation can be expressed as 1 1 1 þ ¼ so si f

ð2:14Þ

40

Optomechatronics

Equation 2.14 is called the Gaussian lens formula. In this formulation we have assumed that all light rays make small angles with the optical axis, which were termed paraxial rays. To help understand the formation of the focal point F, we use Figure 2.9, which illustrates how wave fronts of plane waves pass through the convex and concave lenses. It can be seen that the thicker portion of the lenses causes light to be delayed, thus resulting in convergence for a convex lens and divergence for a concave lens as shown in Figure 2.9a and Figure 2.9b, respectively. In the case of the convex lens, all rays that pass through the lens meet at the focal point F, whereas in the case of the concave lens all the rays that pass through diverge afterwards, as if all the rays appear to emanate from the focal point F: This focal point is called the second of image-side focus. Figure 2.10 summarizes the effect of positive and negative lenses on incident plane wave. The convex lens causes a divergent ray either to converge or diverge less rapidly. In particular, in a situation when a point source S is located at the focal point, the refracted ray becomes parallel, as shown in Figure 2.10d. A concave lens causes a convergent ray either to converge (not shown here) or to diverge (Figure 2.10b). It causes a divergent beam to become more divergent as shown in Figure 2.10c to e. When a convergent beam incident to the lens is directed toward the focal point F, it becomes parallel to the optical axis, as can be seen in Figure 2.10a. As we can see, a convex lens makes a convergent beam converge more rapidly or makes a divergent beam diverge with less speed or converge. In contrast, a concave lens behaves in exactly the opposite way. The output beam properties shown here are made only by a single lens with various angles of incident beam. The output beam properties may be made different depending on how we make the incoming beam incident to the lens. Also, they may be made variable depending on what types of lens or combination of lenses we use in order to manipulate a given incident beam. All of these concerns are involved with optical system design. The beam expander is a good illustration that we can combine two lenses in a proper way to make a beam expand with a desired magniﬁcation, as

plane wave

plane wave

F

(a) convex lens FIGURE 2.9 A plane wave transmitting through a lens.

F

(b) concave lens

Fundamentals of Optics

41 F

F

(a)

F

.

F

s

s

F

F

F

(b)

s’

s’

s

s’ F

s

(e)

F

.

s’

(c) s

(d)

F

F

s

F

F

F

s

s’ F

F

.

s’ F

F

s F

s’

F

concave lens

convex lens FIGURE 2.10 The effect of location of point light source.

shown in Figure 2.11. The ﬁgure shows two cases of beam expanding. The ﬁrst one uses two convex lenses and the other one uses one concave and one convex lens. In either case, the lenses have a common focal point. Depending on the focal length of the lenses, the beam diameter at the output is made different. By similar triangles, it is seen that D2 ¼

f2 D f1 1

ð2:15Þ

This implies that a desired output beam diameter D2 can be easily obtained by properly choosing those three variables.

L2

L1

L1 D1

D2

D1

D2 f1

(a)

f1

f2

L2

(b)

FIGURE 2.11 Transmissive beam expanding by combination of two lenses.

f2

42

Optomechatronics

An object is normally a collection of point sources. To ﬁnd its image, it is desirable to locate the image point corresponding to each point of the object. However, it is sufﬁcient to use a few chief rays in order to determine the object image. To illustrate this, ray diagrams for convex and concave lens chief rays are shown in Figure 2.12. It can be seen that the use of this diagram can facilitate the determination of the location, size, and orientation of an image produced by a lens. The size and location of an image is of particular interest, since they determine magniﬁcation. Following the sign convention, the transverse distance above the optical axis is considered positive, while the distance below the axis negative, as can be seen for the integrated image. It follows from the ﬁgure that the transverse magniﬁcation is expressed by MT ¼

hi s ¼2 i ho so

ð2:16Þ

where ho and hi are the transverse distance, height of the object, and that of its image, respectively. The minus sign for si accounts for an inverted real image as formed by a single thin lens. From Figure 2.12a we can see that in case of a convex lens, the sign and magnitude of the MT depend upon where

A ho

Fi Fo

B

A′

O

f

hi B′

f si

so (a) real image formed by a convex lens

A B′

ho B

Fo

so

A′

hi

O si

(b) virtual image formed by a concave lens FIGURE 2.12 Location and transverse lateral magniﬁcation of object image.

Fi

Fundamentals of Optics

43

an object to be image is located. When it is located within the range 1 . so . 2f , 21 , MT , 0: In case of a diverging lens, shown in Figure 2.12b, we can see MT . 0 for si , 0, since in this case all images are virtual and erect. They are always smaller than the object, and lie closer to the lens than the object. The discussed characteristics of the images of real objects are summarized in Table 2.2. When the object is located in front of a convex lens in the range 1 . so . 2f , MT becomes 21 , MT , 0: When it is located so ¼ 2f , it becomes MT ¼ 21: When it is located in the range f , so , 2f , the image is real and inverted and MT becomes magniﬁed MT , 21: Let us take some examples for thin lenses. When an object is located 6 cm in front of a convex lens having focal length 10 cm, the location and magnitude of the image to be formed can be calculated by using the formulas in Equation 2.14 and Equation 2.16, respectively. The image location is obtained by si ¼

6 £ 10 ¼ 215 6 2 10

which indicates that the image is virtual and lies to the left of the lens. The magniﬁcation of the image is obtained by MT ¼ 2

si ð215Þ ¼2 ¼ 2:5 so 6

which indicates that the image is erect. In a similar way, the location and size of the image of an object located 10 cm in front of a concave lens of focal length 6 cm can be obtained as follows: location image size

si ¼

10 £ ð26Þ 260 ¼ 10 2 ð26Þ 16

MT ¼ 2

si ð260=16Þ 3 ¼2 ¼ so 10 8

The above simple calculations show that we can vary magniﬁcation of an image formed by a single lens by changing its distance from an object. If the lens moves toward the object, by d lying within a range, the image will become larger, but if shifted away from the object, it will become TABLE 2.2 Image Characteristics of Real Objects Lens Convex

Concave

Object Location 1 . so . 2f so ¼ 2f f , so , 2f so , f Any

Image Characteristics

Remarks

Real inverted 21 , MT , 0 Real inverted MT ¼ 21 Real inverted MT , 21 Virtual erect MT . 1 Virtual erect 0 , MT , 1

si . 0 si . 0 si . 0 si , 0 si , 0

44

Optomechatronics

h f

f

h' f < s o < 2 f so - d

d

si

h h'

f

f so = 2f

d

h' s o = 2f

si

f

h

h'

f so + d

2f < s o <∞ si

FIGURE 2.13 Change of magniﬁcation and image plane by using a single lens.

smaller. This relation is illustrated in Figure 2.13. An object whose size is indicated by an arrow is located 2f away from the lens; f is the focal length of the lens. If the lens is shifted toward or away from the object by d, the image location si and magniﬁcation, M, can be determined by the lens equation as; shifted toward; si ¼

f ð2f 2 dÞ f ; M¼2 f 2d f 2d

shifted away from; si ¼

f ð2f þ dÞ f ; M¼2 f þd f þd

It is noticed that, as the lens moves closer to or away from the object, the image plane becomes further distant away or closer to from the object. So far we have discussed only a single thin lens. In many cases, a train of lenses is used to obtain a desired image of an object by modifying the ray path, and correcting the distortion or aberration magnitude of the image and

Fundamentals of Optics

45

object A

B F1

•

so1

F2

f1

•

L1

L2

O1

• O 2 F1

f2 d

si2 si1

RI

• F2

B' virtual object for lens 2 A'

so2

FIGURE 2.14 Formation of a real image by two convex lenses.

so on. To analyze the optical system involved with this, it is desirable to have a fundamental understanding of a two-lens combination ﬁrst. To this end, we will now derive the important parameters of the combination such as an equivalent focal length, the location of the image, and the transverse magniﬁcation. Let us consider two thin convex lenses having different focal lengths f1 and f2 , which are separated by distance d, as shown in Figure 2.14. Referring to the ﬁgure, we can determine the real image (RI) formed by two convex lenses. If no consideration is given to lens L2 the intermediate image, ! ! which is the real image ðA0 B0 Þ of an object ðAB Þ located far left, is formed by lens L1 and can be obtained by the lens formula given in Equation 2.14. si1 ¼

so1 f1 so1 2 f1

ð2:17Þ

The image is real and magniﬁed, since so1 . f1 and f1 . 0 as summarized in Table 2.2. This real image becomes a virtual object for lens L2, since the object is located to the right of the lens. The virtual object is located at so2 ¼ d 2 si1

ð2:18Þ

Utilizing the lens formula for L2 to obtain the image of the object and combining the resulting equation with Equation 2.17 and Equation 2.18, we have si2 ¼

so2 f2 ðso2 2 f2 Þ

ð2:19Þ

This result indicates that the distance of the image of the compound lens can be determined for a given set of parameters such as focal lengths of two lenses, separation distance of the two lenses and the distance of an object that

46

Optomechatronics

are given a priori. The size and orientation of the image can be obtained by total transverse magniﬁcation of the compound lens, which is the product of the magniﬁcation made by L1 and the magniﬁcation by L2; MT ¼ MT1 ·MT2

ð2:20Þ

Upon the use of Equation 2.16, MT becomes MT ¼

f1 si2 dðso1 2 f1 Þ 2 so1 f1

ð2:21Þ

Consider a two convex lens compound system has f1 ¼ 20 cm, f2 ¼ 20 cm, and their separation d ¼ 40 cm. We wish to compute the image of an object located 30 cm from the ﬁrst lens L1. Applying the lens formula L1 si1 ¼

30 £ 20 ¼ 60 cm 30 2 20

MT1 ¼ 2

60 ¼ 22 30

The intermediate image is real, 60 cm to the right of the ﬁrst lens, and inverted. The ﬁnal image formed by the second lens L2 can be obtained by using Equation 2.19. Since so2 ¼ d 2 si1 ¼ 30 2 60 ¼ 230, we have si2 ¼

so2 f2 ð230Þ £ 20 ¼ ¼ 12 so2 2 f2 ð230Þ 2 20

MT2 ¼ 2

12 ¼ þ0:4 ð230Þ

The size and orientation of this image can be obtained by multiplying MT1 and MT2 :Thus MT ¼ MT1 MT2 ¼ ð22Þ £ 0:4 ¼ 20:8 which indicates that the image is reduced and inverted.

Multiple Lenses and System Matrices In general, lenses are classiﬁed into two classes depending on whether they make a parallel bundle of light convergent or divergent, as shown in Figure 2.15. In each class, three types of lenses are illustrated. A converging (also called positive) lens is thicker in the center than the periphery, while a diverging (negative) lens is thinner in the center. A lens has two centers of

Fundamentals of Optics biconvex

plano-convex

(a) convex / positive lenses

47 positive meniscus

biconcave plano-concave negative meniscus

(b) concave / negative lenses

FIGURE 2.15 Various types of lenses.

curvatures, the two having sometimes two different radii. For example, one of the surfaces is plane, and the other has a ﬁnite radius, as shown in the case of plano-convex or plano-concave. There is no numerical limit where a thin lens and a thick lens are differentiable. In other words, whether a lens is thin or thick depends on the accuracy required for solving a given problem. For instance, a lens needs to be treated as thick, when its thickness along its optical axis cannot be ignored without leading to reviewed errors in analysis. There are many cases in which several lenses must be put together to constitute a compound lens that shapes the optical ray in a desired fashion. For example, a photographic lens consists of ﬁve or six lenses for focusing zooming ﬁlters, and so on. In this situation, the analytical matrix method is an efﬁcient and effective approach that facilitates the ray analysis. Ray tracing is undoubtedly an effective tool when a designer wishes to design or modify an optical system according to a given speciﬁcation. However, when an optical system has multiple lenses composed of various types, it is not at all easy to trace some relevant rays through the entire system to compute their ray paths. Let us consider a system of two convex lenses shown in Figure 2.16a, and derive the equivalent focal length of the system. Assume that a ray parallel to the optical axis is incident on lens 1 at a height h1 : The ray after lens 1 enters lens 2 at height h2 : Finally, the ray is focused at a resultant focal point feq : When a ray of light incident upon lens 1 is paralleled to the optical axis at a height h1 as shown in Figure 2.16a, the angle u1 between the refracted ray and the horizontal line parallel to the optical axis is given by

u1 ¼

h1 ¼ p1 h1 f1

ð2:22Þ

if u1 is assumed to be a small angle. In the above, p1 is called the optical power of the lens L1 and represented by 1=f1 : Likewise, when a ray of light parallel to the optical axis is incident upon lens L2 at height h2 as shown in

48

Optomechatronics lens L2

lens L1

h1

h2

θ1

θ2

f1

(a)

f2

(b)

L1

L2

H2

h1

(c)

θ1 h2

θ0∗

F2 feq

d

FIGURE 2.16 Equivalent focal length of a two-lens system.

Figure 2.16b, the angle u2 is given by

u2 ¼

h2 ¼ p2 h2 f2

ð2:23Þ

where p2 is the power of lens 2 and represented by 1=f2 : Let us return to the system of two lenses. After the ray is refracted in lens L1, it is refracted again in lens L2. Behind the second lens, the slope angle up0 is given by

up0 ¼ u1 þ u2 Therefore, the angle up0 can be obtained as follows:

up0 ¼ p1 h1 þ p2 h2

ð2:24Þ

With these results, we then consider two lenses together as a whole. If the focal length of the combined system is denoted by feq which can be measured from a principle plane H2 shown in the ﬁgure, angle up0 is expressed by

up0 ¼

h1 ¼ peq h1 feq

ð2:25Þ

Fundamentals of Optics

49

where peq is the equivalent power of the system. From Figure 2.16c, the angle u1 is obtained by geometry h1 2 h2 ¼ p1 h1 d

u1 ¼

ð2:26Þ

where d is the distance between the two lenses. From Equation 2.24 and Equation 2.26 we have peq ¼ p1 þ p2 2 p1 p2 d

ð2:27Þ

Since power peq is 1=feq , the above equation becomes feq ¼

f1 f2 f1 þ f2 2 d

ð2:28Þ

This feq is called the equivalent focal length of the system. We can see that we will have different equivalent focal lengths depending on the distance d. If two lenses are in contact, that is d ¼ 0, then the above result is given by feq ¼

f1 f2 f1 þ f2

ð2:29Þ

Let us take an example of determining the equivalent focal length of a twolens system. The front convex lens has focal length of 50 mm, and the rear concave lens a focal length of 25 mm. The lenses are separated in distance by 30 mm and then moved to another position, 40 mm. To determine the equivalent focal length, we use Equation 2.28, feq ¼

f1 f2 50 £ ð225Þ ¼ ¼ þ250 mm f1 þ f2 2 d 50 2 25 2 30

We consider another case when two positive lenses, L1 and L2, having an identical focal length, f are separated from each other with distance case (1) 14 f and pulled away from each other with distance case (2) f =2: We compute the feq , or the approximate power of the two cases and determine which one is the wider-angle lens. The use of Equation 2.28 yields: for case (1) d¼

f 4

feq ¼

f1 f2 4 ¼ f f1 þ f2 2 d 7

and for case (2) d¼

1 f 2

feq ¼

f1 f2 2 ¼ f f1 þ f2 2 d 3

50

Optomechatronics

Because case (1) has shorter focal length, the system separated 14 f apart is the wider-angle lens. The focal length in the above is the distance between the second principal plane H2 and the focal point F2 as shown in the ﬁgure. It is noted that this length is considerably much longer than the actual distance between the ﬁrst lens to the second. The System Matrices According to the matrix method, as long as paraxial approximation is made, changes in height and direction of ray can be expressed by linear equations. This makes it easy to use the matrix method to specify the properties of the compound optical system. This indicates that, given the position and size of an object, this approach would produce those of the image with a small computational effort. Let us consider a single ray that starts from P0 through P1 : Three frequently used matrices involved with progress of the ray are the translation, refraction, and reﬂection matrices. Figure 2.17a shows the ray translating from point P0 to P1 by ‘ from the elevation y1 making slope angle a1 relative to the optical axis. For the paraxial ray condition tan a0 < a0 , we have

P1

a0

P0

a1

y1

y0

(a) translation

a'

P

a'

qi a s

θr qt

y' V

P

R j j

R

C

C

a

y' V

n1 n2

(b) refraction at a spherical surface : convex FIGURE 2.17 Forming a matrix for ray tracing.

(c) reflection at a spherical surface : concave

Fundamentals of Optics

51

the following matrix equation representing translation "

#

y1

" ¼

a1

1

‘

0

1

#"

y0

# ð2:30Þ

a0

This represents the location of the ray when traveling from point ðy0 ; a0 Þ through point P1 in the homogeneous medium ðy1 ; a1 Þ: The refraction matrix can be derived when we consider the refraction of a ray traveling in a medium of refractive index n1 at a spherical surface of refractive index n2 as depicted in Figure 2.17b. Since refraction occurs at a point P, the elevation condition yields y0 ¼ y

ð2:31Þ

From the geometry, a 0 is given by

a0 ¼ u t 2

y R

ð2:32Þ

Snell’s law gives n1 ui ¼ n2 ut

ð2:33Þ

Combining Equation 2.31, Equation 2.32, and Equation 2.33 yields

a0 ¼

1 R

n1 n1 21 yþ a n2 n2

ð2:34Þ

Finally, the matrix relating the ray at ða; yÞ to the same ray at ða0 ; y0 Þ is obtained by "

y0

a

0

#

2

1

6 ¼ 4 1 n1 21 R n2

3 0 "y# 7 n1 5 a n2

ð2:35Þ

It is noted that surface, R is negative and y ¼ y 0 , then for a plane interface, R becomes inﬁnity. The reﬂection matrix can be derived in the same way as for the case of the refraction. Referring to Figure 2.17c, angle a 0 is given by

a0 ¼ u r þ

y R

52

Optomechatronics

It is noted here that by the sign convention R is minus for this concave lens. This leads to

a0 ¼ a þ

2y R

When the law of reﬂection at point P is applied, the above equation yields the reﬂective matrix; "

y0

#

a0

2

1 ¼4 2 R

3 0 "y# 5 1 a

ð2:36Þ

Even when an optical system consists of a series of lenses of several different kinds, these three matrix equations can be effectively used to track some major rays by this method. However, this analysis method is formidable to use if there are a number of rays of interest and a number of lens sets in the system. Computer Ray Tracing Computer ray tracing is nowadays a common practice to handle this kind of formidable job. Here, we will treat a very simple ray tracing problem to illustrate the effectiveness of the method. The system to be dealt with is to investigate the effect of variation of the positions of the zooming lenses on the radius of the output beam for a telephoto system as shown in Figure 2.18a. For numerical ray tracing, a computer software developed by the Optical Research Associates was used [8]. The system is composed of three lenses having initially 5 £ magniﬁcation, in which a collimated beam of radius

L1 L2

position 3

position 1

(a)

4

output beam

3 2 1

input beam

0 1

2

3 position

4

5

FIGURE 2.18 Beam expanding via three-lens conﬁguration.

distance(d), mm

beam radius, mm (b)

position 5

60

5

−1

5 mm

3.75 mm

2.5 mm

L3

50 30 20 distance : L2-L1

10 0

(c)

distance : L3-L2

40

1

2

3 4 position

5

Fundamentals of Optics

53

0.5 mm is incident at a concave lens. A variety of beam expansion can be obtained when the location of two lenses (L2, L3) while ﬁxing the position of lens L1 is changed. As can be seen from Figure 2.18b, the location change causes the refracted beam to leave the third lens with a beam of radius 2.5 mm shown in Figure 2.18c. Increasing further the separation between L2 and L3 from this point will yield larger radius of the output beam to 5 mm, that is, 10 £ magniﬁcation. The corresponding relative distance between L1 and L2 and that between L2 and L3 are plotted in Figure 2.18c.

Aperture Stops and Pupils Aperture plays an important role in optical systems. The aperture modiﬁes the effects of spherical aberration and distortion, and controls the brightness of image. The aperture stop is deﬁned to be the aperture, which limits the maximum amount of light that can be collected. In other words, it provides a measure of the light collecting ability. If an optical system consists of a single lens, the aperture stop is the lens itself. In a typical camera, the iris diaphragm is the aperture stop. In a telescope, the ﬁrst lens or objective lens is the aperture stop, since it determines the amount of light needed to form a ﬁnal image. Another important concept is the pupil that determines whether a given particular ray will pass through the entire optical system. The entrance pupil is the image of the aperture stop as seen from object space. In other words, it is the image of the stop through which we look into the entire optical system from the object. If we look into the optical system from the image space, another image of the aperture stop called the exit pupil can be seen to limit the rays forming the image: The object space is a space that contains all physical objects to the left of the lens, while the image space, the space to the right of the last optical element. The location and size of these three openings, the aperture stop, the entrance, and exit pupils have signiﬁcant effect on the light collecting and also on certain types of aberrations. Therefore, it is often necessary to shift the stop along the optical axis in order to improve performance of optical systems. Figure 2.19 illustrates the limitation of rays for various arrangements of convex lens and aperture stop. Figure 2.19a shows the case when the aperture stop (AS) is placed in front of the lens. It can be seen that the stop indeed determines the amount of incoming light, the maximum of which is limited by the marginal ray passing through the margin of the stop. In this case, the stop serves as an entrance pupil, and the exit pupil is real and located behind the lens. Another ray depicted in the ﬁgure is chief or principal ray, which is deﬁned to be any ray from an off-axis object point that passes through the center of the aperture stop. When the optical system is free from aberrations, the ray will also traverse the center of the entrance pupil and leave the center of the exit pupil. Figure 2.19b illustrates the system where the exit pupil is the image of the aperture stop itself and the entrance pupil is the image of the aperture.

54

Optomechatronics (AS) aperture stop marginal ray

f ray

chie

F

chie

f ray

(a)

exit pupil

entrance pupil

AS

chief ray F

chie

f ray

(b)

exit pupil

entrance pupil

FIGURE 2.19 Effects of aperture stop on formation of entrance and exit pupils.

In visual instruments, the observer’s eye is positioned at the center of the exit pupil. In this particular case, the pupil of the system itself is known to vary roughly within the range 2 – 8 mm, depending on the illumination condition. Therefore, the pupil of the instrument must ﬁt into this eye condition. The amount of light admitted through the aperture is very useful in deﬁning two useful parameters, f -number f =#, and numerical aperture, NA. If aperture diameter is denoted by D and focal length of an imaging lens is denoted by f, then the light power per unit area (irradiance) at the image plane is proportional to !2 D Ir / f since the image area is proportional to f : From this quantity, we can deﬁne f =# and NA. The quantity f =D is called focal ratio or relative aperture and deﬁned by f -number and thus f =# ¼

f D

If a lens has a focal length, 4.0 cm, and an aperture diameter 1.0 cm, f -number is 4: If for the same lens only aperture is decreased to 0.5 cm, f -number is 8. This indicates that the amount of light received at the image plane increases with smaller f =#:

Fundamentals of Optics

55

The numerical aperture (NA) is deﬁned by NA ¼ nt sin u where n is the refraction index of the medium in which the lens lies, and u is the half angle of the cone of light leaving towards the focusing area. When the lens is used in the air ðn ¼ 1Þ, we notice that the maximum value is unity. When u is small, NA is approximated by NA . tan u ¼ nt

D=2 f

It can be seen that NA represents the amount of the light power gathered in the image plane by the lens. NA is proportional to aperture diameter and inversely proportional to focal length. If the lens is in the air ðnt ¼ 1Þ and the object is located in the far ﬁeld, f =# and NA are related by f =# ¼

1 2NA

Numerical aperture and f -number are two different methods of deﬁning the same illumination characteristics of an optical system. The f -number is more conveniently applied to the case particularly when the object is in far ﬁeld, for instance, camera lenses and telescope objectives. The numerical aperture is useful for systems that work at ﬁnite conjugates ðso and si ﬁnite) in such a case as microscope objectives. We will return to these parameters in Chapters 6 and 7.

Aberration In the previous section, all rays participating in forming an image are assumed paraxial, that is, rays near to the optical axis or making a small angle with it. This would be a good approximation only for the ﬁrst-order theory for ray tracing. An exact ray tracing in actual case would cause departures from the predictions with the paraxial description. For example, it will no longer be generally true that all the rays leaving a point object will meet an image, as we shall see later. Such deviations from ideal Gaussian behavior are referred to as aberrations. For monochromatic aberrations, there are spherical aberration, coma, and curvature of ﬁeld, astigmatism, and distortion that deteriorate the image quality. Spherical aberration, astigmatism, and coma make the image unclear, while ﬁeld curvature and distortion deform the image. There is another important aberration resulting from

56

Optomechatronics

wavelength dependency of the imaging components of optical systems, but we will not treat this chromatic aberration. Spherical Aberration Spherical aberration is one of the most common aberrations in lenses. If all rays emanating from a point source S, not exclusive of rays reﬂected by a larger lens aperture are assumed to impinge on a convex lens, the image formed can be represented as the point P due to paraxial rays and the point P0 due to nonparaxial rays. This is shown in Figure 2.20. The axial distance between points P and P0 , PP0 is known as the longitudinal spherical aberration. In this case, P0 is to the left of the paraxial focus P and the aberration is positive. In contrast, when P0 is to the right of P, the aberration is considered to be negative. The height above the axis PT is called the transverse spherical aberration. The light rays incident in the intermediate zone of the lens are imaged between point P and P0 : Therefore, there exists no clear sharp image behind the lens because aberrations produce image degradation. If a screen labeled II is inserted between PP0 , we ﬁnd that the image is composed of a circular disk image constituting the cones of light rays produced from various overlapped axial image points. At some intermediate point between P and P0 we ﬁnd the best focus or the circle of least confusion having the smallest circular disk. This is generally the best place to observe the image. The amount of spherical aberration for ﬁxed focal length and aperture is inﬂuenced by both the object distance and the lens shape. This aberration of a lens can be reduced by inserting an aperture stop to block the light rays near the rim or by designing the surface curvatures of lenses properly. Coma Aberration Coma refers to the variation of magniﬁcation with aperture. When a bundle of rays is incident obliquely on a lens as shown in Figure 2.21, the rays

plane of the circle of least confusion j S

a j P

P′ T

longitudinal aberration FIGURE 2.20 Spherical aberration.

Fundamentals of Optics

57

lens

A

R1

hc

Rc R2

hp

B

image plane

FIGURE 2.21 Coma.

passing through the outer zones of the lens form a larger image than the rays through the center portion. If the rays through the outer zone denoted by R1 and R2 make an image at the intersection point A in the image plane, the coma of the lens, hcom , is deﬁned by hcom ¼ hp 2 hc where hp is the height of the image produced by the intersection of the rays R1 and R2 and hc is the height of the image produced by the ray passing through the center of lens. The image has a form which is something like the shape of a comet with its tail, therefore this aberration is named comet. Field Curvature Curvature of ﬁeld is the aberration of a lens that cannot transform a plane object into a precisely plane image. In other words, object points lying in a plane are imaged onto a curved surface when ﬁeld curvature is present. As shown in Figure 2.22, a plane object is imaged into a curved surface IIP, on which the object is imaged approximately as a plane only in the paraxial region. This curvature of ﬁeld is known as Petzval ﬁeld curvature. The Petzval surface for a convex lens is directed inward toward the object plane, whereas for a concave lens it is directed outward, away from that plane. This surface depends on the focal length of the individual lens components and on their refractive indices, as we might expect. Astigmatism Astigmatism occurs when an object point lies far distant from the optical axis and, therefore, the incident cone of rays strikes the lens asymmetrically. This aberration occurs mainly because off-axis object rays are in different axial sections of the lens, resulting in two slightly different focuses; tangential (meridional) and sagittal (radial). Let us consider Figure 2.23 to see how this

58

Optomechatronics

B′ .

lens

B

A′

A

i

o

object plane

P

image plane

FIGURE 2.22 Appearance of both the astigmatism and the ﬁeld curvature.

secondary image circle of least confusion

lens chief ray

A optical axis

C

primary image Ft sagittal plane

Fs meridional plane

D object point B FIGURE 2.23 Meridional and sagittal focuses of an oblique light ray: astigmatism.

arises. In the case of an axial object point, the cone of rays is symmetrical with respect to the spherical surface of a lens, arriving at a single focus in the image focal plane. However, when an incident cone of rays strikes obliquely, there occur two distinct focal lengths: one in the meridional plane and the other in the sagittial plane. These two planes are perpendicular to each other, meridional plane containing both the chief ray and the optical axis, and sagittal plane, containing the chief ray, is determined by the plane perpendicular to the meridional plane. As the beam enters the lens, its cross section is initially circular. However, as the beam proceeds to propagate, it becomes gradually elliptical, whose major axis lies in the sagittal plane. At the meridional focus Ft , known as the “primary image” the ellipse completely deforms into a line. Beyond this point, in a particular position, the image becomes circular with blur known as the circle of least

Fundamentals of Optics

59

confusion, where the major and minor axes of the ellipse are equal. When the beam moves farther from this point, its cross section again becomes gradually elliptical whose major axis this time lies in the meridional plane. At the sagittal focus Fs the ellipse deforms into a line in the meriodional plane called the “secondary image.” Distortion Distortion occurs because focal length and lateral magniﬁcation more or less vary with different distance from the optical axis. In this case, the lateral magniﬁcation Mt may be a function of the distance from the optical axis. This phenomenon can be explained by using Figure 2.24. Let us suppose that a small object ho lies in the axial region of a lens. Its image hi will then appear behind the lens. If the object is shifted upward farther from the optical axis as indicated in the ﬁgure, the image hi will be dependent upon where the object is placed in the vertical direction. The Mt will be changed according to the following formula; DMt ¼

M0t 2 Mt h 0 2 hi ¼ i hi Mt

ð2:37Þ

where DMt is the change in lateral magniﬁcation, Mt and M0t represent the lateral magniﬁcation of the object at two locations, and hi and h0i are the corresponding images at the same locations. If the lateral modiﬁcation increases with the distance from the axis, the distortion becomes positive, DMt . 0: By contrast, if it decreases with the off-axis distance, the distortion becomes DMt , 0: This phenomenon is well illustrated with a square grid as indicated in Figure 2.25. Pincushion distortion as shown in Figure 2.25b is the case for DMt . 0, positive. This causes each image point outward from the center. On the other hand, as shown in Figure 2.25c, barrel shape distortion

FIGURE 2.24 Change in magniﬁcation with the off-axis image plane.

60

Optomechatronics

(a) undistorted

(b) pincushion distortion

(c) barrel distortion

FIGURE 2.25 Distortion due to monochromatic aberrations.

(a) barrel distortion

(b) pincushion distortion FIGURE 2.26 The effect of aperture location on image formation.

occurs for DMt , 0: This causes each image point to move inward toward the center. When measurement of length, width, and dimension requires precision accuracy, DMt needs to be kept typically less than DMt , ^1%: Figure 2.26 illustrates the effect of the location of the aperture stop in distortion. When the aperture stop is inserted between the object and the lens, as shown in Figure 2.26a, barrel distortion occurs. As the aperture stop gets closer to the lens, the barrel distortion is less pronounced than that of the original distance. If the aperture stop is inserted behind the lens, as shown in Figure 2.26b, pincushion distortion occurs. It can be shown that the distortions

Fundamentals of Optics

61

mentioned in the above can be reduced if we add another identical lens behind the aperture stop (see problem P2.10).

Polarization A light wave is oscillating as a transverse electromagnetic wave. As mentioned in the beginning of this chapter, the directions of the electric and magnetic ﬁeld are always perpendicular to one another. Only the direction of the electric ﬁeld will sufﬁce to specify the direction of a light wave. Polarization of the wave is a process of characterizing the orientation of such wave oscillation in both time and space. This concept is widely utilized to the design of optical systems, for example, for the control of the amount and direction of light. Before getting into the subject of polarization in detail, let us consider some examples exhibiting such phenomena. Polarized light can be observed from the reﬂection process. Let us consider when ordinary unpolarized light is reﬂected from dielectric media like glass, water, plastic and so on, as shown in Figure 2.27. If the light is incident with up , other than 908 there will always be reﬂected light and refracted light. The characteristics of the reﬂected light indicate that the light is partially plane polarized in parallel to the plane of incidence as shown in Figure 2.27a. The degree of polarization depends on the incident angle of the incoming light. According to Brewster (1781 – 1868), only at a certain polarization angle that satisﬁes

up þ ur ¼ 908 unpolarized incident light

partially polarized reflected light

qp qp

unpolarized incident light

fully polarized reflected light

qp

qp interface

interface

p

p

qr

refracted light

(a) partial polarization FIGURE 2.27 Plane polarization by reﬂection.

qr

refracted light

(b) maximum polarization

62

Optomechatronics

will the reﬂected light be plane polarized, exhibiting the maximum amount of polarization at this angle, which is shown in Figure 2.27b. From Snell’s law, we can express this angle up in terms of the index of refraction n ð2:38Þ tan up ¼ t ni where ni is the index of refraction of incident medium and nt is that of the transmitting medium. For example, when the light is incident to the surface of a lake, reﬂection occurs with polarized light at an angle tan up ¼ 1:33 which gives up ¼ 53:18. When a polaroid sheet is placed in front of a propagation wave, only the component of light waves parallel to the transmission axis will pass through the sheet. This is because the device has the property that it transmits light waves in a speciﬁc direction without transmitting waves in other directions. The device exhibiting this behavior is called a polarizer. In order to investigate some details of this polarization phenomenon, let us consider when a light wave is oscillating, while remaining within a plane, which is called a “polarized plane.” Figure 2.28 depicts two cases showing y

Ey

E x

Ex

E

Ey

Ex

(a) linearly polarized

z propagation direction

ϕ =180°, 360°, Ex=Ey=const

y

x E Ey Ex z propagation direction (b) circularly polarized

ϕ =90°, 270°, Ex=Ey=const

FIGURE 2.28 Two perpendicular electromagnetic wave oscillations.

Fundamentals of Optics

63

such behavior: The elliptically polarized pattern, which is the most general case, is not shown here. The ﬁrst ﬁgure shows the case when a wave is linearly polarized. The orientation of the electric ﬁeld E is referenced to some axis perpendicular to the direction of propagation, and does not change with time as E oscillates from a maximum value in one direction to a maximum in the opposite direction. The electric wave oscillates in the plane. Since the wave is a vector quantity, it has amplitude E and direction at any instant of time which are given by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ E ¼ E2x þ E2y ð2:39Þ Each polarized vector is indicated in one quadrant plane shown as a dotted line. The phase difference between Ex and Ey is p or 2p. The second ﬁgure shows circular polarization whose constituent waves have equal amplitude ðEx ¼ Ey Þ and whose phase difference is p=2 or 3p=2: Figure 2.29 illustrates two different states of polarization of light waves. The ﬁrst ﬁgure denotes the case when unpolarized light encounters a polarizer whose transmission axis is in a vertical direction. Therefore, the output light will be polarized in a vertical direction. The second one shows addition of another polarizer whose transmission axis is horizontal. In this polarizer polarizer

.

.

(a) parallel orientation

.

(b) crossed orientation

incident light y

E θ

polarizer Ey = E cosq

x

z (c) Malus's law FIGURE 2.29 Dichroic crystals in two different positions and Malus’s law.

64

Optomechatronics

case, no light is transmitted through the analyzer. The third ﬁgure shows when light is polarized with angle u when the vertical transmission axis passes through a polarizer. The polarizer will transmit only the vertical component of the electric ﬁeld vector. Since light intensity is proportional to the square of amplitude, the irradiance of the transmitted light is obtained by I ¼ I0 cos2 u

ð2:40Þ

where I0 is the maximum intensity of the transmitted wave. This indicates that, when u ¼ 458, the transmitted irradiance becomes 12 I0 , which is half the irradiance of the initial light. This is known as Malus0 s law.

Coherence As to the phenomena of interference and diffraction to be discussed later, we will be concerned with what happens when two or more light waves superimpose each other in some region of space. The phenomena will occur differently depending upon how strongly they are correlated in both time and space. That is to say, when they are radiating perfectly in phase with each other, they will produce a strong response due to the addition of their individual amplitude. On the contrary, when they are not correlated, randomly in phase, their response will become very weak. In reality, these two extremes are not ideally attainable. This means that there are no such cases having a correlation value (coherence function) of either one or zero. Some currently-used laser beams can be approximated by a monochromatic wave having the value slightly less than 1 only if a certain condition is assumed. Partial correlation occurs when they are radiating with constant phase relationships. The coherence therefore is used to describe the correlation between phases of such radiating light waves. The coherence may be conveniently divided into two classiﬁcations; temporal and spatial. Temporal coherence refers to correlation in phase at the same location in space at a different time, while spatial coherence refers to correlation in phase at the same time at some different point in radiation space. Michelson interference is a typical case of exhibiting temporal coherence induced by path difference, while Young’s experiment using double slits is a typical case of exhibiting the spatial coherence caused by separation of two slits. The degree of coherence largely depends upon light path difference and the size of the light source. The temporal coherence is characterized by coherent time Dtc , which is the time interval, that is, time coherence length, during which the wave is coherent. If a light source has a long coherent time, Dtc , it has a high degree of temporal coherence. As mentioned before, the spatial coherence is the correlation in phase between some points in the radiation ﬁeld. Its effect arises when the light’s sources are extended within a ﬁnite spatial domain. It is characterized by spatial coherence length D ‘a over which the waves are coherent. Both coherence values, Dtc and D ‘a can be

Fundamentals of Optics

65

derived mathematically by considering the temporal correlation function and spatial interference between the interacting waves.

Interference Interference can be found from several natural phenomena such as soap bubbles or in oil ﬁlms on wet pavement and so on. Interference occurs when two or more monochromatic light sources are superimposed under some conditions to be discussed later. In a simple conﬁguration of producing interference, a single light source may be used to make light wave travel two different paths and then recombine. Interference between two such waves can be demonstrated where a pinhole point source is places near a plane mirror. This is called Lloyd’s single mirror arrangement. Under this arrangement, there are two light sources: One source is S and the other source is a virtual source S0 from which the reﬂected light appears to propagate in space. At a point in the region of a screen, the result of the interference between the wave from S and the wave S0 (see Problem P2.17), with the phase change that occurs upon reﬂection, will be observed. To consider a more general case of interference, let us consider two point sources S1 and S2 emitting monochromatic waves E1 and E2 in a homogeneous medium, as shown in Figure 2.30. For simplicity, we will assume that two waves have the same frequency, v and wavelength l: Let the two waves be represented by E1 ðr1 ; tÞ ¼ E01 cosðk1 r 1 2 vt þ w1 Þ

ð2:41Þ

E2 ðr2 ; tÞ ¼ E02 cosðk2 r 2 2 vt þ w2 Þ where E01 and E02 are the wave amplitude generated by S1 and that of S2, k1 and k2 are the wave numbers of the two waves, and r 1 and r 2 are spatial coordinates in space, respectively. Let the source separation w be much larger than l: According to the principle of superimposition, at an arbitrary point P in space, the two waves will intersect to produce the electric ﬁeld intensity EP : EP ¼ E1 þ E2 Then the irradiance, I, can be deﬁned by I ¼ 10 ckE2P l

ð2:42Þ

where 10 is the permittivity of free space (vacuum) and c is the speed of light, and kl implies time average of the quantity within the bracket. The irradiance is expressed by using Equation 2.42 I ¼ 10 ckE21 þ E22 þ 2E1 E2 l

ð2:43Þ

66

Optomechatronics P

E1

r1 r2

S1 S2 E2

(a) interaction of waves from two point light source irradiance (I) Imax

Imin 0

∆ϕ

(b) irradiance of interference fringe v.s. phase difference FIGURE 2.30 Interference of two coherent waves.

Taking the time average of the quantity in the bracket, we have I ¼ I1 þ I2 þ I12

ð2:44Þ

where I1 ¼ kE21 l, I2 ¼ kE22 l, I12 ¼ 2kE1 E2 l: In the above equation, I12 is called the “interference term”. Substituting the two waves E1 and E2 given in Equation 2.41 into the above equations and carrying out their time averages, we have I1 ¼

10 c 2 E ; 2 01

I2 ¼

10 c 2 E ; 2 02

I12 ¼ 10 cE01 E02 cos Dw

where, Dw, is expressed by the phase angle difference Dw ¼ ðr 1 2 r 2 Þk þ w1 2 w2

ð2:45Þ

where k ¼ k1 ¼ k2 : In the above equation, the ﬁrst term is due to the path difference while the second is due to the initial phase difference. Rewriting the irradiance at point P given in Equation 2.44 yields pﬃﬃﬃﬃﬃ I ¼ I1 þ I2 þ 2 I1 I2 cos Dw ð2:46Þ The above equation implies that the irradiance at an arbitrary point P in space is characterized by the path difference term, and the initial phase

Fundamentals of Optics

67

difference w1 2 w2 : If these values vary randomly, the sources S1 and S2 are mutually independent, i.e., incoherent as discussed previously. In this case, the time average of cos Dw becomes zero, which means that no steady state will be observed over a certain period of time. Therefore, cos Dw should not be zero and w1 2 w2 needs to be constant to make interference occur steadily over long period of time. Under this condition, the strength of interference depends on the path difference. In other words, it varies with the point given by r 1 2 r 2, Maximum irradiance occurs when cos Dw ¼ 1; pﬃﬃﬃﬃﬃ Imax ¼ I1 þ I2 þ 2 I1 I2 for Dw ¼ 2mp On the other hand, minimum irradiance occurs when cos Dw ¼ 21 pﬃﬃﬃﬃﬃ Imin ¼ I1 þ I2 2 2 I1 I2 for Dw ¼ ð2m þ 1Þp This relation is shown in Figure 2.30b. It is noticed that when I1 ¼ I2 ¼ I0 , then, Imax ¼ 4I0 and Imin ¼ 0: Young’s Experiment

S2

r2

q

d δ

z

s

Imin

S1

Imax

P r1

intensity (I)

The arrangement of Young’s experiment is schematically shown in Figure 2.31. Suppose that a plane light wave from a monochromatic source to the left of the ﬁgure is caused to pass through a narrow slit. According to Huygens’s principle, the propagation of a wave front can be constructed by considering each point on the wave front as a source of new spherical wavelets. Therefore S is regarded to be a new source of a spherical wave, and the spherical wave travels to two double slits S1 and S2. This generates two coherent secondary sources, which will spread out and, in turn, produce interference of two spherical waves. The ﬁgure shows a cross-sectional view of the wave fronts in a plane containing the sources S1 and S2. The solid lines

m = 2p m = 3p 2 m=p m= 1p 2

screen (a) schematic for double slit FIGURE 2.31 Double slit interference.

(b) irradiance v. s. distance from optical axis

68

Optomechatronics

indicate their crest, while the dotted lines indicate their valley. Therefore, the waves in solid lines will, when added together, give an intensity that is twice that of a single source. On the other hand, when one wave in the solid line is combined with the other in the dotted line, that is, two waves 1808out of phase are combined, the waves always cancel and therefore no light can be detected. Consider a point P on the screen and the intensity at that point. The intensity is obviously dependent on the distance S1 P and the distance S2 P: If S1 P ¼ S2 P, or S1 P differs from S2 P by an integral number of wave length, then the wave fronts will arrive at P in phase and will reinforce. From the ﬁgure, the path difference is given by

d ¼ r1 2 r2 ¼ d sin u

ð2:47Þ

where d is the distance between the slits. Depending upon the path difference d, the interference will be constructive or destructive: Constructive interference occurs when wave fronts add together to give a larger intensity value, while destructive interference occurs when wave fronts are out of phase. Equation 2.47 can be written as constructive;d ¼ S2 P 2 S1 P ¼ ml

1 l 2 This phenomenon is depicted in Figure 2.31b. Let us compute the position of the maxima in the direction of y-axis on the screen. Because u is small, from the trigonometric relationship, we have destructive; d ¼ S2 P 2 S1 P ¼ m þ

sin u . u .

y s

ð2:48Þ

where s is the distance from the slit to the screen and y is the distance from point P to the optical axis. Combining Equation 2.47 and Equation 2.48 yields y¼

s d d

This can be rewritten as s ym ¼ ml; m ¼ ^1; ^2… d

ð2:49Þ

which indicates that the maximum positions of fringes on the screen are determined by the distance between the slits, s, the spacing between two adjacent slit, d, light wave length l, and fringe order m, the spacing of the maximum intensity fringes can be obtained from Equation 2.49 ymþ1 2 ym ¼ Dy .

s l d

ð2:50Þ

where Dy is the distance between two adjacent fringe maxima. Large wavelength and screen distance causes large patterns while small spacing

Fundamentals of Optics

69

causes large fringe spacing. This equation will yield 2% or better for angles less than 208. Let us take example of calculating the location of the speciﬁc fringe order. For the given parameters, d ¼ 0:1 mm, s ¼ 1 m, and l ¼ 640 nm, the location of the ﬁrst order fringe is calculated to be y^1 ¼

1 £ 103 £ 0:64 £ 1023 ¼ ^6:4 mm 1 £ 1021

Thus, the ﬁrst peaks appear at 6.4 mm above and below the axis of m ¼ 0: Fabry-Perot Interferometer The Fabry-Perot interferometer, developed in 1899 by Fabry and Perot shortly after the Michelson interferometer, utilizes two parallel highly reﬂecting mirrors or a transparent plate with two reﬂecting surfaces within which an interference pattern occurs. This instrument is used for high-resolution spectrum analysis, high precision wave length measurement, and determination of such physical parameters as refractive indices. This interferometer principle is also widely used for optical ﬁber sensors to measure pressures or transparent ﬁlms of ﬂuid medium. The interferometer consists of two thick glass plates forming an air cavity between them. The interferometer conﬁguration can be made by a ﬁlm of transparent material, such as a coated layer on a glass substrate and metal oxide layer. The two intersurfaces are parallel; highly reﬂective surfaces, and polished ﬂat to within 1/20 to 1/100 of a wavelength. To keep high reﬂectivity they are coated with either a metallic (silver, gold, aluminum) layer or dielectric material depending upon the wavelength of light used. The plate can be ﬁxed or mechanically movable. When they are ﬁxed, the instrument is called an elatron. Consider the arrangement of the interferometer as shown in Figure 2.32a. Suppose that the interferometer having two parallel plates with distance, d, is illuminated by a narrow, monochromatic beam from a broad source. For simplicity only a single ray emitted from the source is depicted. As shown in the ﬁgure, the ray which impinges on the left plate with an incident angle of ui is transmitted through the cavity, and then hits the right plate. It is then multiply reﬂected within the cavity. All these coherent beams will eventually form a spot at point P: Any other rays parallel to this will be brought together at the same point P, which implies that the single beam generates multiple coherent beams in the interferometer. At the interface, the incident beam refracts with an angle ut : The optical path difference between the beams reﬂected and refracted then becomes

d ¼ 2nc d cos ut where nc is the refraction index of the medium in the cavity. When the two plates are separated in the air, then the path difference d between two

70

Optomechatronics

glass plate

P

collecting lens

qt

screen

d qi S source (a) multiple wave interference Er4 Er3

Et4

Er2

Et3

Er1

Et2 Et1

Eo (b) reflection and transmission from two parallel mirror surfaces FIGURE 2.32 Fabry-Perot interferometer.

successive parallel beams becomes

d ¼ 2d cos ut

ð2:51Þ

This indicates that there will be a constructive interference yielding maximum in the amplitude of the transmitted beam and a destructive interference yielding minimum amplitude. Fringes arising from refraction will appear satisfying maximum amplitude :

d ¼ ml

maximum amplitude :

d¼ mþ

1 l 2

ð2:52Þ

Combining Equation 2.51 and Equation 2.52, we can write ml ¼ 2d cos ut

ð2:53Þ

for maximum amplitude. Because l is very small as compared to d for a given refraction angle ut , we will see that number of fringe m will be very large, usually on the order of 10,000 to 100,000, depending on the plate separation. This number will increase as the plate distance increases, as can be seen from the above equation. The phase difference Dw due to path difference is

Fundamentals of Optics

71

given by Dw ¼

2p 2d cos ut l

ð2:54Þ

As long as the beams satisfy the above angle condition, the transmitted beams overlap at point P. Intensity at point P transmitted through the interferometer can be calculated by considering the set of parallel-reﬂected waves E1 ; E2 ; …EN as depicted in Figure 2.32b. Let the amplitude of the initial wave incident on the ﬁrst surface be E0 : In case of no energy absorption, the wave amplitude is decomposed into the amplitude of the reﬂected wave rr E0 and the amplitude of the transmitted wave rt E0 : The total amplitude of the transmitted waves arrived at point P, EP , becomes EP ¼ Et1 þ Et2 þ · · · þ EtN

ð2:55Þ

Taking into consideration of the phase difference with reference to the wave E1 , Equation 2.55 can be rewritten as 2iðN21ÞDw } EP ¼ r2t E0 {1 þ r2r e2i Dw þ r4r e2i2Dw þ · · · þ r2N r e

where rr and rt denote the reﬂection and transmission or refraction coefﬁcients, respectively. Since lr2r e2i Dw l , 1, the above expression becomes EP ¼ E0

r2t 1 2 r2r e2i Dw

ð2:56Þ

The intensity at point P is obtained by multiplying Equation 2.56 by its complex conjugate EpP : The resulting equation becomes 1 2 2 rE 1 p 2 t 0 I P ¼ EP EP ¼ 2 ð1 þ r4r Þ 2 2r2r cos Dw

ð2:57Þ

If the intensity of the incoming wave is denoted by I0 ¼ 12 E20 , and no energy absorption condition is assumed, Equation 2.57 is rewritten in the following from I0

IP ¼ 1þ4

rr 1 2 r2r

2

sin2

Dw 2

ð2:58Þ

where the relation, r2t ¼ 1 2 r2r , is used. The intensity transmitted is therefore a function of the plate separation d, the incident angle ui , wave length l, and the reﬂection coefﬁcient rr : As can be seen from Equation 2.58, intensity varies sharply with the reﬂection related factor in parenthesis called the

72

Optomechatronics 1

transmitted intensity (T)

rr = 0.3

rr=0.8 0

rr=0.9 2π phase difference (∆j)

0

4π

FIGURE 2.33 The transmitted intensity distribution.

coefﬁcient of ﬁnesse F. In terms of this coefﬁcient, IP is rewritten as T¼

IP ¼ I0

1 1 þ F sin2

Dw 2

ð2:59Þ

where T denotes the transmittance and F is expressed by F¼4

rr 1 2 r2r

2

The transmittance distribution function T is termed Airy function. The relationship between the transmission function and phase difference is plotted for various reﬂection coefﬁcients in Figure 2.33. A maximum intensity in transmission occurs when Dw ¼ 2mp, while a minimum occurs at Dw ¼ ðm þ 1=2Þð2pÞ: From this ﬁgure, we can see that a measure of the sharpness of the fringes becomes large for increasing rr value. In the case of rr ¼ 0:9 the irradiance drops off very rapidly on either side of the maximum value, showing a very sharp fringe. It is noted that the important parameters affecting the Dw are the plate separation, d, and refraction angle ut : The Fabry-Perot interferometer can be used to separate two wavelength components that appear in the incident light. Michelson Interferometer Optical path difference provides the cause of interference between waves. A typical interferometer that operates on this principle is the Michelson

Fundamentals of Optics

73

F

movable M2

B

(2) (1) light source S

A

(3) D

beam splitter (4) E (BS) detector or viewer

mirror (fixed)

C

M1

FIGURE 2.34 Michelson interferometer.

interferometer developed in 1880. This device is the best known and most versatile interferometric device. The basic arrangement is shown in Figure 2.34. It is composed of a coherent light from the source, a ﬁlm (slightly silver coated) beam splitter, and two retroreﬂecting mirrors, a ﬁxed mirror M1 and a movable mirror M2. When a monochromatic light source is incident to the beam splitter (BS), it is then split into two beams of equal amplitude. The reﬂected beam 1 and transmitted beam 2 continue to the retroﬂector M1 and M2, respectively. After being reﬂected back, their directions are reversed when those return to point D. Therefore, beam 4 includes rays that travel different optical paths: Beam 1 traveled to the ﬁxed M1 and is reﬂected back to D, whose path is ADCD, while beam 2 traveled to the movable M2 and is reﬂected back to D, whose path is ABAD. Now, let us suppose that M2 moves to a new position by ‘ from its initial position. In this case, the full path difference between the waves recombined at beam splitter will exhibit the wave interference phenomenon. The optical path difference d made by the two waves described by

d ¼ AFAD 2 ADCD ¼ 2‘ This indicates that if the mirror M2 is moved by ‘, the path difference d is given by two times of the displacement. In other worlds, if ‘ ¼ l=2 then d becomes l a beam wavelength. Since the amplitudes of the two waves are equal, the irradiance due to two beams creating the interference is given according to Equation 2.46 I ¼ 2I0 1 þ cos

4p‘ l

ð2:60Þ

74

Optomechatronics

where I0 is the source amplitude and l is the wavelength of the source. Bright fringes occur at 2mp ¼ ð4p‘Þ=l; m ¼ 0; ^1; ^2…, while dark fringes occur at ð2m þ 1Þp ¼ ð4p‘Þ=l, m ¼ 0; ^1; ^2…: As M2 (object to be measured) moves, its displacement can be measured simply by counting the number of maxima of light intensity obtained by the detector. Its velocity can also be measured by counting the number of maxima per unit time. Therefore, the output signal of the detector varies periodically with ‘, whose period is equal to l=2: This implies that, when a He –Ne laser is used as source and a displacement of an object is ‘ ¼ 2 mm, the Michelson interferometer would produce the number of fringes of the detector output signal; N¼

‘

l=2

¼

4 £ 1023 ¼ 6:3 £ 105 6:33 £ 1029

Diffraction During traveling, a light wave meets an obstacle such as a rectangular object placed in front and a small aperture on an opaque or a transparent screen, light spreading phenomenon occurs beyond the obstacle and as a result produces bright and dark fringes. This spreading of light in the neighborhood of such discontinuities is known as diffraction. Figure 2.35 illustrates the variation of the spreading phenomenon with increasing distance from a slit. In near ﬁeld, the image pattern due to diffraction somewhat resembles the geometrical shape of obstacles, which is known as Fresnel diffraction. In contrast, in far ﬁeld, the image pattern does considerably differ from the shape of obstacles due to its spread with traveling distance. This is called Fraunhoffer diffraction. We will deal with only Fraunhoffer diffraction in this section. Consider the arrangement shown in Figure 2.36 for observing Fraunhoffer diffraction. The slit (or aperture) shown in Figure 2.36a is coherently illuminated by means of a point source and a collimation lens, not shown here. A second lens is placed behind the slit and the plane of observation is the second focal place of it. The incident and diffracted wave fronts are therefore plane. The slit lies in the x 2 y plane and has width w and vertical length ‘ð‘ q lÞ: Due to a long slit little diffraction will occur in the direction of the vertical edge (y direction). Then, the problem to ﬁnd the wave intensity at point P is treated as a one-dimensional problem due to point sources across the width of the slit w as shown in Figure 2.36b. We consider each segment of slit dx as a source and calculate the contributions of all sources by integrating over the entire slit width. Then total wave amplitude arriving at P on a screen located at distance r from a point source is given by EP ¼ Ew

ðw=2 2w=2

eiðkr2vtÞ dx r

ð2:61Þ

Fundamentals of Optics

75

image plane fringe

slit

S FIGURE 2.35 Light wave spread due to an obstacle.

x

xs

y

lens +

−

2

slit 2

2

x

θ

2

w

O

o −

2

θ

z

R P

(a) slit geometry

-

screen

δ

2 slit

r θ R

z

o xp f

P

(b) a schematic of a single-slit diffraction cross section (top view)

FIGURE 2.36 Fraunhoffer diffraction.

where Ew is the source strength per unit width. Now let us consider the optical path difference d between two waves starting at x ¼ 0 and x ¼ w=2: From the geometry it is given by

d ¼ x sin u

76

Optomechatronics

where u is the angle which the OP makes with the horizontal optical axis. Then, the optical path length from the interval dx to the point P is given by rðxÞ ¼ R þ x sin u

ð2:62Þ

since r almost does not change due to this path difference, i.e., r . R: By substituting Equation 2.62 into Equation 2.61, the resulting equation becomes ðw=2 E E EP ¼ w eiðkR2vtÞ eikx sin u dx ¼ w R R 2w=2

sin

kw sin u 2 kw sin u 2

eiðkR2vtÞ

ð2:63Þ

Letting b ¼ ðkw=2Þ sin u, Equation 2.63 can be rewritten as Ew R

EP ¼

sin b iðkR2vtÞ e b

Since the irradiance I is determined by the time average of E2P i.e., I ¼ 10 ckE2P l, then I can be obtained by the above equation, IðuÞ ¼ or IðuÞ ¼

1 0 c Ew 2 R

2

1 0 c Ew 2 R

2

2

sin b b

ð2:64Þ

sinc2 b

The IðuÞ is the irradiance of the light diffracted in a given direction deﬁned by u. This also gives information on the intensity distribution of all light brought into the focal plane. It is noted that if we let the intensity at u ¼ 0 to be Ið0Þ, then the irradiance is simply expressed by IðuÞ ¼ Ið0Þsinc2 b where Ið0Þ ¼

1 0 c Ew 2 R

ð2:65Þ 2

:

The property of the irradiance pattern due to a single slit can be analyzed by examining sinc2 b function: The sinc2 b function becomes unity as b approaches zero, mathematically lim b!0

sin b !1 b

We can see that at this b value the irradiance becomes maximum as shown in Figure 2.38, that is,

b ¼ ðkw/2Þ sin u ¼ 0

Fundamentals of Optics

77 f(b) x = tan b

−3p

−2p

−p

0

p

2p

3p

x=b

b

FIGURE 2.37 Intersection points of the line x ¼ b and the curves x ¼ tan b:

The amplitude of the I(0) is observed to be proportional to the square of slit width w. The secondary maxima along with the above ﬁrst maxima can be obtained by letting dI=db ¼ 0, which yields the following transcendental equation; tan b ¼ b The b values satisfying the above equation are indicated in Figure 2.37. It can be seen that the points satisfying this equation lie between the minimum values but are not located at the midpoints between these values. At these points, the secondary maxima have rapidly diminishing intensity as can be seen in Figure 2.38. Table 2.3 lists the relative intensities of the ﬁrst three secondary maxima. The minimum values yielding IðuÞ ¼ 0 can be found from Equation 2.65. These occur at the points satisfying

b¼

kw sin u ¼ mp; 2

m ¼ ^1; ^2; … ^ n…

In terms of wavelength this can be rewritten as ml ¼ w sin u

ð2:66Þ

which can be clearly seen from the ﬁgure. The image drawn below the intensity curve reﬂects the intensity distribution along the screen axis xs in the focal plane.

78

Optomechatronics

1.0

0.7 0.5 0.3 0.1 b: sin q :

−3p

−2p

−p

0

p

−3l/w

−2l/w

−l/w

0

l/w

2p 2l/w

3p 3l/w

FIGURE 2.38 Irradiation pattern for a single-slit Fraunhoffer diffraction.

The image pattern consists of a central bright band and alternate bright and dark band on either side. The image within each band is very much blurred which is typical of diffraction of this kind. The angular width of the ﬁrst maximum can be determined from Equation 2.66: 2sin Du . 2Du ¼

2l w

ð2:67Þ

for a given l. This result implies that the central maximum will spread as the slit width is narrowed and the wavelength gets large. The spread of the central maximum in the far ﬁeld can be easily calculated from Equation 2.67.

TABLE 2.3 Relation Intensities of the Secondary Maxima of Diffraction Pattern of a Slit Maxima

b

Central max First max Second max Third max

0 1.43p 2.46p 3.47p

Relative Intensity 1 0.047 0.017 0.008

Fundamentals of Optics

79

Let us suppose that laser light with l ¼ 632 nm passes through a very narrow slit with w ¼ 0:1 mm. Then 2sin u ¼ 2

l 632 £ 1029 ¼ 1:264 £ 1022 ¼2£ w 1 £ 1024

Thus, angular width becomes 2u ¼ 0:7248: If the distance from the slit to screen is R ¼ 5 m, we can also calculate the width of the central maximum. 2Rl 2 £ 5 £ 632 £ 1029 ¼ 63:2 mm ¼ w 1 £ 1024

2xP ¼

Double Slits The diffraction pattern due to two parallel slits, each of which are separated by a distance apart d can be obtained in the same manner as that obtained for the single slit discussed in the above. Referring to Figure 2.31, the wave amplitude at point P can be obtained by integration of the following equation. For the coordinate system whose origin is located at the center of the ﬁrst slit, EP becomes E ðw=2 ixk sin u E ðdþðw=2Þ ixk sin u dx þ w dx ð2:68Þ e e EP ¼ w R 2w=2 R d2ðw=2Þ where R is the distance from the center of the slit separation to the screen, equal to s in the ﬁgure, w is the slit width, and slit separation, d. Introducing two variables associated with slit width w and slit separation d, b ¼ 12 kw sin u, a ¼ 12 kd sin u, the above diffraction integral can be evaluated as EP ¼

2Ew w sin b cos a R b

The corresponding intensity distribution function is now IðuÞ ¼ 4Ið0Þ

2

sin b b

cos2 a

ð2:69Þ

where Ið0Þ ¼

10 c E w w 2 R

2

Equation 2.69 implies that in the u ¼ 0 direction the double slit provides four times the maximum irradiance as compared with the single slit. The equation also tells us that the irradiance is just a product of the irradiances obtained for double slit interference found in Young’s experiment and

80

Optomechatronics

single slit diffraction. The diffraction related factor ðsin b=bÞ2 here constitutes an envelope for the interference fringes given by the factor cos2 a: To illustrate this, Equation 2.69 is plotted for the case of d ¼ 4w: In the Figure 2.39 ðsin b=bÞ2 and cos2 a are plotted together. As can be seen from Figure 2.39 the product of the sine and cosine factors can be interpreted as a modulation of the interference fringe pattern by a single slit diffraction envelope. As discussed in a single slit case, the envelope has a minimum at b ¼ mp ml ¼ w sin u;

m ¼ ^1; ^2…

ð2:70Þ

On the other hand, the interference maxima occur at

a ¼ np nl ¼ d sin u;

n ¼ 0; ^1; ^2…

ð2:71Þ

When the conditions Equation 2.70 and Equation 2.71 are simultaneously met at the same point in the pattern, the diffraction minima coincide with the interference maxima. In this case, there will be no light available at this point participating in the interference. This is called missing order which is given by the following condition; n w or a ¼ m

d¼

n b m

ð2:72Þ

cos2 a

sin2 b b2

1.0

0 b

:

sin q :

−16p

−12p

−8p

−4p

0

4p

8p

12p

16p

−4p

−3p

−2p

−p

0

p

2p

3p

4p

FIGURE 2.39 Interference and diffraction patterns due to double slit.

Fundamentals of Optics

81

For example, when d ¼ 4w the missing orders of interference are given for n ¼ 4m ¼ ^4; ^8; ^12…, as shown in Figure 2.40. The image shown below the curve reﬂects the characteristics of the diffraction curve, namely, and many black and white stripes corresponding to the curve pattern. In a limiting case when d is very large, i.e., d q w the ﬁrst missing order appears far from the center of the pattern. This case produces Young’s interference pattern for two slits. Bright fringes occur for a ¼ 0; ^p; ^2p, and so on. The angular separation between fringes is given by Da ¼ p, or approximately, 2Du .

4p 2l ¼ kd d

It is interesting to note that this is identical to the result of Young’s experiment given in Equation 2.49. Multiple Slits When an aperture consists of parallel multiple slits having a large number N of width w and center-to-center separation d, it is called a grating. In this case, the total wave amplitude arriving at point P in the observation plane can be calculated in a manner similar to that of the double slit. When we set up the origin of the coordinates at the center of the ﬁrst slit in this case, we have the irradiance at point P as " ðdþðw=2Þ ð2dþðw=2Þ Ew ðw=2 ikx sin u e dx þ eikx sin u dx þ eikx sin u dx EP ¼ R 2w=2 d2ðw=2Þ 2d2ðw=2Þ ð3dþðw=2Þ ððN21Þdþðw=2Þ þ eikx sin u dx· · · þ eikx sin u dx 3d2ðw=2Þ

ðN21Þd2ðw=2Þ

ð2:73Þ

where R is the same as deﬁned in Equation 2.68. When we consider only two terms in Equation 2.73, this reduces to the double slit case. If the contribution of the ‘th slit is denoted by E‘, adding all of these terms EP ¼

N21 X ‘20

E‘

and squaring this resultant amplitude, EP yields the following intensity distribution function; I ¼ Ið0Þ

sin b b

2

sin N a sin a

2

ð2:74Þ

where Ið0Þ is the irradiance in the direction u ¼ 0: It is noted that Ið0Þ ¼ N2 I0 : When N ¼ 1, and N ¼ 2, represent the case of single and double slit diffraction, respectively. Here, again, ðsin b=bÞ2 is the diffraction related term whereas ðsin N d=sin aÞ2 is related to the interference related term.

82

Optomechatronics I(θ) 4 I ( 0)

0

b:

−16p

−12p

−8p

−4p

0

4p

8p

12p

16p

sin q : −4p

−3p

−2p

−p

0

p

2p

3p

4p

FIGURE 2.40 A double slit pattern ða ¼ 4bÞ:

The interference pattern can be analyzed from the latter term as in the case of N ¼ 2: When a ¼ 0; ^p; ^2p; ^3p…, the term, which describes interference between slits, becomes maximum. According to L’Hospital’s rule for lim

a!mp

sin N a ¼ ^N; sin a

m ¼ 0; ^1; ^2; ^3…

If we call these maxima principle maxima to differentiate from others, those have a magnitude N2 : Because a ¼ ðkd=2Þsin u, it follows that for N q 2 the maxima occur at the u locations satisfying d sin u ¼ ml;

m ¼ 0; ^1; ^2; ^3…

ð2:75Þ

The examination of the interference term also indicates that there are minima and other maxima between the consecutive principle maxima. There are N 2 1 minima, which make sin N a ¼ 0; sin a

a¼^

p 2p ðN 2 1Þp ;^ ;… ;… N N N

ð2:76Þ

The secondary maxima occur between consecutive minima at ðN 2 2Þ points. These points can be approximated by noting that the term sin N a

Fundamentals of Optics

83

varies much faster than sin a: Therefore, the secondary maxima are determined at the following points when sin N a term has its largest value;

a¼^

3p 5p 7p ;^ ;^ ; ^· · · 2N 2N 2N

ð2:77Þ

In case of N ¼ 4, the primary and secondary maxima are shown along with the minima in Figure 2.41a. From this pattern, we can see that the principal wave patterns are much sharper than those of the double slit case are. In Figure 2.41b, the resultant intensity curve is plotted using Equation 2.74. It is clearly seen that the intensity ratio continues to decrease as due to wave diffraction. The device that utilizes multiple-slit diffraction is called the diffraction grating. In general, the incident wave fronts of light make an angle with the plane of grating, although the incident plane wave here is normal to the grating surface. N2

2 sin Na  sina 

irradiance, I



 α: β: sin q :

π

2π

3π

λ/d

2λ/d

3λ/d

I0

4π π 4λ/d

2 2 sin b  sin Na  b   sina 



sin b   b 



irradiance, I

2 2 sin b  sin Na  b   sina 



α: β: sin q :

2

π

2π

3π

λ/d

2λ/d

3λ/d

4π π 4λ/d

FIGURE 2.41 Interference and diffraction patterns due to multiple slits ðN ¼ 4; a ¼ 4bÞ:



(b)

2



(a)

sin Na  sina 

84

Optomechatronics

Circular Aperture We have discussed diffraction from a slit having width w much smaller than its length ‘. When both dimension of the slit are comparable, it becomes a rectangular aperture. As we can envision the pattern of its diffraction, it produces appreciable spreading over the plane ðx-y coordinates) of the screen. The spread of the pattern, of course, varies with the aperture dimension. Since the spread pattern can be determined in a manner similar to a single slit, we will not treat the details here. Diffraction from a circular aperture is different in its pattern from that of a rectangular or a square aperture. Consider a plane wave propagating in the z-direction incident on an opaque diffraction screen containing a circular aperture as shown in Figure 2.42. Then, the wave amplitude of the optical ﬁeld at point P due to source strength per unit area EA is expressed by E ð ð iyk sin u dA ð2:78Þ e EP ¼ A R A where the differential area is taken as dA ¼ s dy: Since s is related to qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ s ¼ 2 a 2 2 y2 where a is the radius of the aperture. The integral in Equation 2.78 becomes a ﬃ 2EA Z iyk sin u qﬃﬃﬃﬃﬃﬃﬃﬃﬃ EP ¼ e a2 2 y2 dy R

ð2:79Þ

a

y

dy

s y 0 a x

y′ q

r

p r′

R

z x′

FIGURE 2.42 Coordinate system for diffraction due to a circular aperture.

Fundamentals of Optics

85

If we introduce the quantities u and r deﬁned by u ¼ y=a and r ¼ ka sin u, respectively then Equation 2.79 can be rewritten as EP ¼

2EA a2 ðþ1 iru pﬃﬃﬃﬃﬃﬃﬃﬃ2ﬃ e 1 2 u du R 21

ð2:80Þ

The above integral has the value pJ1 ðrÞ=r where J1 ðrÞ is the Bessel function of the ﬁrst kind. Carrying out the integration and substituting the Bessel function, we have EP ¼

2EA a2 pJ1 ðrÞ R r

This implies that the wave amplitudes at point P and its vicinity decay as the distance departs from P, obeying the law of the ﬁrst order Bessel function. The irradiance for a circular aperture is obtained by IðuÞ ¼ Ið0Þ or IðuÞ ¼ Ið0Þ

2J1 ðrÞ r

2

2J1 ðka sin uÞ ka sin u

2

ð2:81Þ

where Ið0Þ is the irradiance at the center, i.e., u ¼ 0 and given by Ið0Þ ¼

E2A A2 R2

It is noted that the distance R is assumed constant and R q D where D is the aperture diameter, and A is the area of the aperture. The irradiance pattern due to Fraunhoffer diffraction given in Equation 2.81 is plotted in Figure 2.43a. It has an axial symmetry and a towering central maximum, which corresponds to a high irradiance. The image of the irradiance pattern corresponding to the distribution is plotted in Figure 2.43b and c. The central maximum is surrounded by a dark ring that is created by the ﬁrst zero of the function J1 ðka sin uÞ: This maximum spot is called Airy disk named after G. B. Airy (1801 – 1892). The central disk is surrounded by a dark ring, which is created by the ﬁrst zero of J1 ðrÞ: We ﬁnd that the ﬁrst zero J1 ðrÞ ¼ 0 gives r1 ¼ 3:83: The radius of the center of this dark ring, r1 that gives J1 ðka sin uÞ ¼ 0 is obtained by r1 ¼ 1:22

Rl 2a

ð2:82Þ

When light waves are transmitted through a lens, the lens acts as an aperture. If a lens of an identical size positioned within the aperture is focused on the screen, the focal length is approximately f . R: Then r1 can be rewritten in terms of aperture diameter as r1 . 1:22

fl D

ð2:83Þ

–10

–5

FIGURE 2.43 Irradiance due to a circular aperture.

(b)

–15 (a) 0

0

1.0

I (q ) I (0)

5

10

0.0175

15

I = I (0)(2J1(r)/r)2

(c)

r = (2π/λ)a sinq

86 Optomechatronics

Fundamentals of Optics

87

This equation indicates that the radius of the Airy disk is determined by the effective aperture of diameter D, focal length f, and wavelength of the incident light l. The radius of the Airy disk becomes small as the effective aperture becomes large but decreases as focal length gets large. The diffraction phenomenon can also easily illustrated by varying incident angle of the incoming beam to a thick lens. To illustrate this the Code V was used [8]. Figure 2.44 shows how the Airy disk changes with variation of incident beam angle when an image plane is held at the local plane. The lens has a focal length 2.6 mm and NA is 0.6. The smallest disk in the middle of the ﬁgure is obtained when the beam is perfectly collimated. In this case, the entrance pupil is 5.25 mm. However, as the incident beam is deviated with a slightly divergent angle as shown in the left ﬁgures, the Airy disk is seen to get larger in size and blurred in image. As the beam gets converged from the collimated case, the Airy disk again exhibits the same behavior as in the above. These phenomena are well depicted in the bottom ﬁgures where the image intensities of all six cases are shown. As we shall discuss in Chapter 7, this result implies that the collimator or other lenses should be accurately positioned in order to have accuracy of optical systems to satisfactory level. The resolution can be interpreted as the smallest resolvable separation between the central maximums (Airy disks) created by two equal-irradiance, incoherent, distant point sources separated apart by an angle u as depicted in Figure 2.45. If we introduce Du as an angular measure of the radius of the Airy disk, the disk at far ﬁeld, for each point source will be spread out over an angular half-width Du about its own center. The angular distance of the Airy disk at far ﬁeld is can be expressed by Du ¼

1:22l D

ð2:84Þ

where D is the diameter of the aperture. The equation in the above is of great practical importance since the result can be used to deﬁne the limit of resolution obtainable with the instruments containing various optical components. To explain this, let us consider the diffraction due to two point sources S1 and S2, shown in Figure 2.45. Those are formed through a single lens, which in this case is considered an aperture. If the separation angle between the two sources, u, is large enough, the formed images are well separated as seen from the ﬁgure. As S1 and S2 are brought together close to each other, the two image patterns gradually begin to overlap and ﬁnally get to the point where those cannot be resolvable. The limit condition is just resolvable and known as Rayleigh criterion. The limit of the resolvable angular separation of two point sources can be expressed as ðDuÞmin ¼

1:22l D

ð2:85Þ

Objective Le

1.00

ORA

Objective Lens that LGE Designed(01-03-2 Objective Le

FIGURE 2.44 Effect of the incident beam angle on image formation.

(c) intensity variation

(b) variation of Airy disk

MM

05-Feb-05

(a) beam configuration

Objective Lens that LGE Designed(01-03-2

1.00

ORA

MM

05-Feb-05

Objective Lens that LGE Designed(01-03-2 Objective Le

1.00

ORA

MM

05-Feb-05

Objective Lens that LGE Designed(01-03-2 Objective Le

1.00

ORA

MM

05-Feb-05

Objective Lens that LGE Designed(01-03-2

Objective Le

1.00

ORA

MM

05-Feb-05

88 Optomechatronics

Fundamentals of Optics

89

y′

S1 q

z′ = z

q

S2 lens image plane

(a)

qmin r1

(b) FIGURE 2.45 Diffraction limited image achievable by the minimum resolvable separation.

which is given in Equation 2.84. In terms of the center-to-center separation of the two images, ðDy0 Þmin , at the screen or at the focal plane of the lens, the limit of resolution is given by 1:22l ðDy 0 Þmin ¼ f ð2:86Þ D which is equal to r1 as shown in the ﬁgure. The diffraction-limited image is shown in Figure 2.46. In general, the resolving power of the imaging system is deﬁned as by either 1=ðDuÞmin or 1=ðDy 0 Þmin : The equation in the above indicates that the resolution can be increased by increasing the diameter of the lens or decreasing the wavelength of light. Therefore, in the ﬁne quality lens, if aberrations are assumed negligible the sharpness of Airy disk attainable by a point object is limited by diffraction. The spreading out of the image pattern due to the diffraction therefore represents the limit of the sharpness of image. As we shall see later, this criterion will be used throughout the book. Diffraction Grating The discussion made on diffraction previously can now be extended to the case of a large number of diffraction elements such as multiple slits, apertures, or obstacles. A periodic array of such elements is called a diffraction grating.

90

Optomechatronics

I2

I1

I1 + I2

(∆q)min

(a) q >> (∆q)min

(b) q ≈ (∆q)min

FIGURE 2.46 Images formed due to diffraction for two different angular separating of sources.

Previously, we have dealt with the case when the incident wave fronts of light make a right angle with the plane of grating surface but here will deal with an arbitrary incident angle. The common form of diffraction grating is an amplitude grating with a rectangular proﬁle of transmittance as shown in Figure 2.47. At its surfaces, light modulation occurs due to the alternate opaque and transparent regions. This can be obtained by vacuum deposition of many opaque (metallic) strips on a glass plate, which forms a periodic array of narrow slits of equal distance. If parallel light beam strikes this grating with an oblique incident angle, ui , diffracted beams from successive slits interfere constructively and destructively. Referring to the right handside of Figure 2.47b, the net path difference between waves from successive slits is given by

d1 2 d2 ¼ dðsin um 2 sin ui Þ

ð2:87Þ

mth order m= +1 m= +1 m= 0 diffracted light m= 0 m= −1 incident light m= −1

(a) diffraction geometry

(b) diffracted light

FIGURE 2.47 Diffraction of light by a transmission amplitude grating.

δ2

qi

qm

δ1 d

Fundamentals of Optics

91

R B

grating lens

R B

white 1 light

f

m = +2

m = +1

m=0 B R B R

m = −1

m = −2

focal plane FIGURE 2.48 Spectra produced by white light.

from which the grating equation can be expressed as dðsin um 2 sin ui Þ ¼ ml;

m ¼ 0; ^1; ^2…

ð2:88Þ

for all waves in phase. In Figure 2.47b only three diffracted waves ðm ¼ 0; ^1Þ are plotted for simplicity. We can see that the diffraction angle differs from diffraction order. When m ¼ 0, the diffraction angle of the zeroth order of interference becomes identical to that of the incident wave, i.e., um ¼ ui : At this direction, the diffracted wave produces the maximum irradiance for all wavelengths l, as we have seen in the previous sections. Higher orders, however, the direction of um varies with wavelength l. These properties are used to measure the wavelength of light and do spectral analysis. Figure 2.48 illustrates how white light can be decomposed into different colors having different wavelength by a diffraction grating. To achieve this a collecting lens is employed to focus each collimating beam of wavelengths to a point in the focal plane. R and B denote the red (R) and blue (B) ends of the light color spectrum. Figure 2.49 shows a reﬂection phase grating. The gratings have highly reﬂecting faces of periodically varying optical thickness. Those modify the phase of incident light while the amplitude remains unchanged. This is in contrast with the amplitude grating. As shown in the ﬁgure, optical path difference is expressed as given in Equation 2.87 and therefore, the same grating equation as given in Equation 2.88 can be applied.

92

Optomechatronics incident light

qi

m = −1 m=0

diffracted light m = −1 m = +1 m=0

d1 d d2 qm

mth order

m = +1

(a) diffracted lights

(b) diffraction geometry

FIGURE 2.49 Reﬂection phase grating.

Optical Fiber Transmission Ever since the concept of optical ﬁber has been used as a means of transmission of signal or data, they have been widely used as transmission media in optical measurement as well as optical communication. In addition, those are becoming extremely important for laser beam delivery for instrumentation and surgical applications. This is due to low, loss transmission, high capacity of carrying information and immunity to electromagnetic interference. An optical ﬁber is composed of two concentric dielectric cylinders; the core (inner cylinder) and the cladding (outer cylinder). The core having a refractive index nco is clad with the cladding of lower refractive index ncl i.e., nco . ncl : The optical ﬁber is made based on the phenomenon of total internal reﬂection within this ﬁber structural arrangement. Let us elaborate this principle in more detail. As shown in Figure 2.50, rays strike the interface of the core and cladding. Applying Snell’s law at the interface, we have nco sin uco ¼ ncl sin ucl

ð2:89Þ

From this relation we can easily see that, when ucl ¼ 908, the refracted ray travels along the boundary as shown in the middle of Figure 2.50a. The corresponding ucl that yields this phenomenon is called the critical angle uc as discussed previously. As indicated in the right hand side of Figure 2.50a, when uco . uc , all rays striking the interface will be totally reﬂected back into the core by means of many internal reﬂections. Thus, the ﬁber becomes a light guide, as shown in Figure 2.50b. This critical angle uc can be determined from

Fundamentals of Optics

93

θt

ncl nco

θi

θi θi

θc

when θi = θc critical angle

when θi < θc refraction + reflection

θi

θi

when θi > θc total internal refraction

(a) reflection and refraction at various ray incident angles ncl nco θi

θ > θc refracted ray

ni cladding

(b) total internal reflection of rays within an optical fiber FIGURE 2.50 Light propagation in optical ﬁber.

Equation 2.89

uc ¼ sin21

ncl nco

To yield this critical angle, there will be a range of an angle of incidence for a ray to enter the core. Let us determine the range of the incident angle. If the ﬁber is surrounded by a medium having a refractive index ni , Schnell’s law gives for some refractive angle u. ni sin ui ¼ nco sin u

ð2:90Þ

If it is assumed here that for this incident angle ui , the refracted ray hits the interface with angle u, as indicated in Figure 2.50b. The angle u to be greater than uc must satisfy the following condition cos u $

ncl nco

ð2:91Þ

From Equation 2.90 and Equation 2.91 we see that the condition for the occurrence of total internal reﬂection is limited by sﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ nco ncl 2 12 sin ui # ni nco

94

Optomechatronics

Since in air ni < 1:0 the maximum ui condition is given by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ð2:92Þ ½ui max ¼ sin21 n2co 2 n2cl qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ If the above quantity n2co 2 n2cl is deﬁned as the numerical aperture (NA) of the ﬁber, the above equation be rewritten as ½ui

max

¼ sin21 NA

For example, if nco and ncl are 1.51 and 1.49, respectively, then ½ui max of a ﬁber in air becomes approximately 14.188. Therefore, NA of the ﬁber can be an indication of the maximum angle of incidence. Fiber with a wide variety of numerical apertures ranging from 0.2 up to 1.0 is being used, depending on applications. Figure 2.51 depicts the three major types of ﬁber conﬁguration being used currently. The single mode index ﬁber depicted in Figure 2.51a has a very narrow core, typically less than 10 mm in diameter. Therefore, it can have only a single mode in which the rays travel parallel to the center through the ﬁber. The multimode step index ﬁber shown Figure 2.51b has a core of relatively large diameter, 50 to 200 mm, making it advantageous to launch light into

(a) single-mode step-index fiber

(b) multi-mode step-index fiber

(c) multi-mode graded-index fiber FIGURE 2.51 Major refraction index conﬁgurations of optical ﬁbers.

Fundamentals of Optics

95

ﬁbers using LED sources. In addition, it can be easily terminated and coupled. The multimode graded index ﬁber as shown in Figure 2.51c has a core diameter of 20 to 90 mm. The core employs a nonuniform refractive index, which has higher value at the center decreasing parabolically toward the interface. Due to this index variation, the rays smoothly spiral around the center axis instead of a zigzag path, as seen for the case of the multi mode index ﬁber. Power losses occur when light propagates through the ﬁber. It depends upon ﬁber material, scattering by impurities and defects for glass ﬁber and absorption for plastic ﬁber. The attenuation in optical power is usually deﬁned by Pl ¼ 10 log10

Wo Wi

ð2:93Þ

where Pl is expressed in dB, and Wi and Wo are the input and output power, respectively.

Gaussian Beam Optics In modern optics, lasers are the indispensable light source for measurement, communication, data recording, material processing and so forth. In Table 2.4, various lasers are compared with regard to the type, state, wavelength and power range. In most of their applications, it is necessary to shape the laser beam using optical elements. The laser for most cases operates and emits a beam in the lowest order transverse electromagnetic (TEM) mode. In this mode, the beam is a perfect plane wave and has a Gaussian wave front whose characteristics are radically symmetric and can be described by Gaussian function. Although in actual lasers it is not perfectly Gaussian, in Helium Neon (HeNe) and argon-ion lasers it is very close. Figure 2.52 illustrates

TABLE 2.4 Comparison Between Various Laser Sources Type

State

Wave Length (nm)

He–Ne Argon CO2 Semiconductor Excimer (KrF) Nd–YAG He–Cd Ruby

Gas Gas Gas Solid Gas Solid Gas Solid

632.8 488.0, 514.5 10600 850 –910 248 1047–1064 442, 325 694.3

Power Range 0.5– 50 mW 0.5– 10 W 0.5– 10 kW ,10 kW 1–300 W 1 mW– 10 kW 50–150 mW Pulse (,400 J) (0.2 —5 ms)

96

Optomechatronics

r

1 irradiance contour e2

r(z)

I0 e2

I0

r0

I0 e2

0

q

2r0 zr

z

FIGURE 2.52 Spot size variation with propagation distance: Gaussian beam.

the characteristics of Gaussian laser beam such as beam irradiance and divergence proﬁle. As can be seen from the ﬁgure, the beam has maximum beam intensity I0 at its center and decreases with distance from the axis. The beam diameter slightly differs depending upon how it is deﬁned. Three widely used deﬁnitions are: (1) full power beam diameter, (2) half power beam diameter, and (3) 1=e2 excluded power beam. The full power, beam diameter is the diameter of the laser beam core that contains the total beam power, except the power portion outside 8s which is eight times the standard deviation of the distribution. This corresponds to 99.97% of the total power. The half power beam diameter contains 50% of the total beam power but excluded the rest of the 50% power lying outside 2:35s: The 1=e2 excluded beam diameter is the diameter that contains the beam power exclusive of 13.5% ð1=e2 Þ of the total power. This power corresponds to 4s: This diameter is commonly adopted when referring to the diameter of a laser beam. The divergence angle u of a Gaussian laser beam as indicated in the ﬁgure is the angle by which the beam spreads from the center of the waist. This angle is essentially the angle of the asymptotic cone made by 1=e2 irradiance surface. Let us consider the geometry of this beam spread. If rðzÞ is denoted by the radius of the contour at the distance z from the beam waist, it can be expressed by " rðzÞ ¼ r0 1 þ

lz pr20

!2 # 12

ð2:94Þ

where r0 , called the radius of beam waist, is the radius of the irradiance contour at the location where the wave front is ﬂat. From Equation 2.94 we can see that r0 is the radius of the beam waist at z ¼ 0 and that rðzÞ is symmetrical about z ¼ 0: Also, it can be seen that at rðzÞ ¼ r0 the radius of curvature rðzÞ is inﬁnite. For large value of z, that is, at the far ﬁeld, the equation is largely determined by the term ðlz=pr0 Þ2 , since !2 lz q1 ð2:95Þ pr20

Fundamentals of Optics

97

Using the asymptotic line approaching z ¼ 0 and small angle assumption, we can obtain asymptotic angle u from Equation 2.95 tan21 u . u .

l pr0

ð2:96Þ

This angle is measured at a distance far away from the laser that is z q zr where zr stands for the Rayleigh range which is equal to pr20 =l: The range is deﬁned by the distance from the center of the waist to the propagated location at which the beam wave front has a minimum radius of curvature rmin ðzÞ on either side of the beam waist. Let us have an example to illustrate how much the laser beam will spread for a given HeNe laser having wavelength 632.8 nm and diameter 1.0 mm. From Equation 2.96, we have

u.

632:8 £ 1026 ¼ 4:03 £ 1024 rad pð0:5Þ

Once this is obtained, beam spread rðzÞ can be easily determined at any location of z. Equation 2.94 can be applied to the case when a Gaussian laser beam enters a lens with small divergence angle and is focused at its focal point. Since the lens is located in the far ﬁeld relative to the focus point as compared with the size of the small focused beam, the focused beam radius can be approximated by r0 ¼

lf pD

ð2:97Þ

where D is the aperture diameter of the lens.

Problems P2.1. Suppose that light wave is incident on the surface of a glass having refractive index nt ¼ 1:5 with an angle u ¼ 458, as shown in Figure P2.1.

45° qg

qa

FIGURE P2.1 A ray transmitting through a glass.

glass

98

Optomechatronics

(1) Determine the refractive angle ug : (2) What will be the transmittive angle ua ? P2.2. A meniscus convex lens made of a glass ðnt ¼ 1:5Þ has its radius of curvature of 28 cm at one side and 50 cm at the other side as shown in Figure P2.2a.

1.5m

1.5m

(a)

(b)

FIGURE P2.2 Meniscus convex lens.

(1) Determine the focal length of this lens. (2) If the same lens is ﬂipped over in respect to the direction of the incident beam as indicated in Figure P2.2b, determine the focal length. (3) In both cases, (1) and (2), determine the image point on the optical axis when an object is placed 1.5 m in front of the lens. P2.3. A bundle of parallel rays transmits through (a) convex lens (b) two lenses composed of a convex lens and a planar convex as indicated in Figure P2.3. Draw a ray diagram in each case in detail and discuss the results of the ray path. f and f 0 are the focal lengths of the two cases.

L1

L1 L2

f

(a)

(b)

f′

FIGURE P2.3 Ray diagram.

P2.4. Design a beam expander using any combination of two types of lenses (concave and convex). The expander is required to expand a beam of 1.5 cm to 9.0 cm. Assume that the absolute value of the focal length of the lens is less than 9.0 cm.

Fundamentals of Optics

99

P.2.5. There are several ways to measure the effective focal length of a lens. One way is to use a collimator. Draw a conﬁguration of this focal collimator, a focal length measurement system, and explain how it works in principle. P2.6. The beam expander is a frequently used device to expand beam diameter. The nonfocusing reﬂective beam expander is practical for most applications due to simplicity of its conﬁguration. Draw three such beam expanders and explain how they work. P2.7. The image size of an optical system can be conveniently described by a lateral magniﬁcation factor, m ¼ h0 =h, where h and h0 are the object height and image height, respectively. If so and si are the object and image distances from the lens, respectively, prove that so and si can be expressed by so ¼

1 21 f m

si ¼ ð1 2 mÞ f P2.8. Consider the astigmatism discussed in Figure 2.23. Suppose that rays focus as a line at fT and fs as shown in Figure P2.8. If the lens is assumed to have a tangential focus ð fT Þ 17.8 cm and a sagittal focus ð fs Þ, 19.2 cm, what will be the diameter of the circle of least confusion. The circle is assumed to be located at the point having an average of the optical powers of the two focal lengths. The diameter of the lens is D: lens

circle of least confusion

D fT

fs

FIGURE P2.8 Location of circle of least confusion.

P2.9. A telescope is an optical device that enlarges the apparent size of an object located at a distance away. Such a device can be conﬁgured by combining two thin lenses, L1 and L2, shown in Figure P2.9. The two lenses are separated by distance d, and f1 and f2 are the focal lengths of lenses L1 and L2, respectively. (1) Locate the focal point of each lens and draw optical paths by which light reaches the eye for the two cases shown in Figure P2.9a and b.

100

Optomechatronics L1

L1

L2

L2

eye

(a)

eye

(b)

d L1

d L3

L2

(c) FIGURE P2.9 Inverted and erected images in a telescope.

(2) In Figure P2.9a, the image obtained is inverted. In order to obtain the erected image, repeat the same problem (1) for an optical system shown in Figure P2.9c. In this case, what will be magniﬁcation? P2.10. Figure P2.10 shows an aperture placed in an optical system composed of two identical convex lenses. Discuss the effect of the aperture stop on distortion. lens

aperture stop FIGURE P2.10 The effect of aperture stop.

P2.11. Suppose we wish to grind the rim of the lens so that the line between the centers of curvature of two lens surfaces coincides with the mechanical axis, which is deﬁned by the ground edge of the lens. Given an accurate tubular tool on a rotating spindle where the lens can be fasten with wax or pitch, (1) conﬁgure an optical system for this grind system and (2) describe in detail how the system works.

Fundamentals of Optics

101

P2.12. Consider that an unpolarized light wave propagates through a polarizer with vertical transmission axis and then a second polarizer with its transmission axis making an angle of 608 with the vertical axis, as shown in Figure P2.12. What will be the polarization angle and the intensity of the transmitted light?

60°

unpolarized light

polarizer

polarizer

FIGURE P2.12 Polarizing light with a polarizer.

P2.13. Counting the number of fringes observed in a Michelson interferometer is used to measure a displacement of an object. When the object is displaced by 0.81 mm, a shift of 2000 fringes is observed. Compute the wavelength of the light source. P2.14. Figure P2.14 shows an arrangement of Fabry-Perot interference. The arrangement utilizes a collimation lens between the source and the interferometer. Discuss what types of fringes will occur. Give the reasons for this. d

S

source

collimating lens

interferometer focusing lens screen

FIGURE P2.14 Fabry-Perot interferometer.

P2.15. When an incident beam contains two wavelengths which are slightly different from each other, the transmitted irradiance of each fringe will be shown in Figure P2.15. If one wavelength l0 is 500 nm and the other differs only by Dl ¼ 0:0052 nm, what will be the spacing of the Fabry-Perot

102

Optomechatronics T

fringe 1 fringe 2

1.0

0.5

∆js ∆l l0

∆ϕ

FIGURE P2.15 Two overlapping wavelength components.

etalon when the mth order of one component coincides with the ðm þ 1Þth order of the other? P2.16. A Fabry-Perot etalon has a reﬂection coefﬁcient of rr ¼ 0:92 and a plate separation of 1.5 cm. The incident light has two slightly different wavelengths around 490 nm. Determine (1) coefﬁcient of ﬁnesse (2) maximum order of interference (3) minimum resolvable wavelength difference Dl: P2.17. The optical system shown in Figure P2.17 called “Lloyd’s mirror” consists of a point source S, a plane mirror M placed near the source, and image plane Si :

point source S

image plane Si

M mirror FIGURE P2.17 An optical system composed of point source S and a plane mirror M.

(1) Draw the relevant optical paths in order to describe the image in the image plane Si : (2) What type of image will appear in the plane Si : P2.18. Suppose that a parallel beam of 632.8 nm light propagates through a single slit. If the distance between the slit and screen is 1 m, determine the width of the slit in order to have the spread of the central maxima, 30 cm.

Fundamentals of Optics

103

P2.19. Suppose that a coherent light wave having the wavelength l ¼ 640 nm passes through double slits whose separation distance is 0.2 mm. The screen is located 1 m away from the slits. Calculate the width of the central maxima. P2.20. In double slit diffraction, the relationship between separation and width is given by d ¼ 5w: (1) Plot the interference term vs. a (2) Plot the diffraction term vs. b (3) Plot the irradiance vs. spatial angle, u P2.21. From Figure 2.40 and Figure 2.41, we can see that the waves appear much sharper for the case of multislits than that of double slits. Give a physical reason in some detail. pﬃﬃ P2.22. Show pthat ﬃﬃ at Rayleigh range zr the beam spreads by a factor of 2, that is, rðzr Þ ¼ 2r0 : P2.23. Suppose that a laser beam spreads out from the beam waist. Determine the beam radius at a location z ¼ 80 m for a beam of radius 0.3 mm laser diode l ¼ 810 nm. P2.24. Suppose that a HeNe laser of radius 5 mm enters a lens having focal length 50 mm. Compute an approximate focus spot diameter.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Andonovic, I. and Uttamchandani, D. Principle of Modern Optical Systems, Vol. 1. Artech House, Boston, MA, 1989. Bernacki, B.E. and Mansuripur, M. Causes offocus-error feedthrough in optical-disk systems: astigmatic and obscuration methods, Applied Optics, 33:5, 735–743, 1994. Fan, K.C., Chu, C.L. and Mou, J.I. Development of a low-cost autofocusing probe for proﬁle measurement, Measurement Science and Technology, 12:12, 2137–2146, 2001. Gasvik, K.J. Optical Metrology, 2nd Ed., Wiley, New York, 1996. Heavens, O.S. and Ditchburn, R.W. Insight into Optics, Wiley, New York, 1991. Hecht, E. Optics, 4th Ed., Addison Wesley, Reading, MA, 2001. Mansuripur, M. Classical Optics and its Applications, pp. 222– 239, Cambridge Press, Cambridge, 2002. Code V, Optical Research Associates, 2001. O’Shea, D.C. Elements of Modern Optical Design, pp. 24 – 75, Wiley, New York, 1985. Pedrotti, F.J. and Pedrotti, L.S. Introduction to Optics, pp. 43 – 348, Prentice Hall, Englewood Cliffs, NJ, 1992.

104 [11]

Optomechatronics

Pluta, M. Advanced light microscopy, Principles and Basic Properties, pp. 110 – 129, Vol. 1. Elsevier, Amsterdam, 1988. [12] Rossi, B. Optics, pp. 122– 149, Addison-Wesley, Reading, MA, 1957. [13] Smith, W.J. Practical Optical System Layout: and Use of Stock Lenses, McGraw-Hill Professional, New York, 1997.

3 Machine Vision: Visual Sensing and Image Processing CONTENTS Image Formation................................................................................................ 109 Imaging Devices................................................................................................. 112 Image Display: Interlacing ....................................................................... 115 Image Processing ............................................................................................... 115 Image Representation........................................................................................ 116 Binary Image............................................................................................... 117 Gray Scale Image ....................................................................................... 117 Histogram.................................................................................................... 118 Histogram Modiﬁcation............................................................................ 118 Image Filtering ................................................................................................... 121 Mean Filter .................................................................................................. 123 Median Filter .............................................................................................. 124 Image Segmentation .......................................................................................... 127 Thresholding............................................................................................... 127 Iterative Thresholding ............................................................................... 131 Region-Based Segmentation ..................................................................... 133 Edge Detection ................................................................................................... 136 Roberts Operator........................................................................................ 138 Sobel Operator............................................................................................ 140 Laplacian Operator .................................................................................... 142 Hough Transform .............................................................................................. 148 Camera Calibration............................................................................................ 152 Perspective Projection ............................................................................... 155 Problems.............................................................................................................. 165 References ........................................................................................................... 172

Visual sensing is prerequisite for understanding the structure of a 3D environment, which requires assessing the shape and position of objects. The sensing is to acquire the image of the environment through a visual or an 105

106

Optomechatronics

imaging system, usually a camera system. When the system sees the scene containing objects, the amount of light energy it acquires depends upon a number of factors, such as shape, optical properties of objects, background, and illumination conditions. Often, these factors are critical to determine the amount of light impinging on the object’s surface. Depending upon these factors, the camera system takes in a certain amount of light reﬂected and scattered from the scene through its optical system. The received light energy in general determines the intensity of light sensed by the camera, which is the physical quantity that the vision system handles to interpret the scene. In proportion to the intensity of the received light, the imaging hardware device of the camera transforms the physical quantity into an electrical signal. This electrical video signal is captured, processed, and digitized by the frame grabber in a discrete fashion so that a computer can understand the digitized values. After digitization, this signal is sent to a computer for image processing. Utilizing the acquired image, the image processing algorithms carry out several steps of image modiﬁcation and transformation with a view to extract the information necessary to understand the environments. This information includes a shape, a surface texture, a size of objects, location, orientation of objects, and distance from the vision sensor to objects. Based on this information, a so-called high level algorithm classiﬁes the objects into several groups of dissimilar objects and ﬁnally carries out the pattern recognition tasks, if necessary. Machine vision technique comprises a series of these tasks described above. The complete steps needed for image processing, analyzing, and understanding are depicted in Figure 3.1. Let us reiterate in more detail the image analysis part. The objective of the image analysis is to understand the nature of an image by acquiring and interpreting the information necessary to solve a variety of engineering problems. It is composed of several hierarchical steps: (1) (2) (3) (4)

Preprocessing Intermediate processing Feature extraction Scene understanding.

Preprocessing is a stage of selecting an interested region of the image and enhancing the acquired image to be analyzed by eliminating artifacts from the image, ﬁltering noise in the image, making image quantization such as thresholding, and so on. In many cases, the original image acquired usually contains noise, broken lines, blurs, low contrast, and geometric distortion caused by the lens. These unwanted defects make it difﬁcult to accurately extract high-level information such as points, edges, surfaces, and regions. It is, therefore, necessary to eliminate or reduce noise and blurring for the improvement in contrast and the correction of distortion. The processing of carrying out these tasks is called image preprocessing and should yield an improvement to the original image.

Machine Vision: Visual Sensing and Image Processing

107

FIGURE 3.1 Steps for visual information processing.

Since the image usually contains a great deal of pixel data, it is difﬁcult to directly use them to analyze the nature of the acquired scene. To this end, the task in the next stage concerns the extraction of the higher-level information mentioned above, based upon the results obtained by preprocessing. Utilizing the preprocessed image, the intermediate processing stage achieves edge detection, segmentation, and transformation of the image into another domain. Of course, the quality of preprocessing greatly affects that of this stage. In the step for feature extraction, selection of the features is carefully made by considering which information will be useful to best represent the characteristics of objects in the segmented image. The purpose of using the features of the image, rather than the raw image itself, is to make searching and acquiring the representative information easier for processing, thus reducing the amount of time for image analysis. There are many types of features used to extract information as listed in Table 3.1. The frequently used features are geometrical shape, texture, and color selection as clariﬁed in the table. Feature extraction is, therefore, the process

108

Optomechatronics

TABLE 3.1 Commonly Used Feature Sets for High Level Image Processing Feature Set Geometrical shape

Parameter

Description

Center of mass

Area moment

Perimeter

Width

h w

Vertex

Aspect ratio

b a

Principle axis

Elliptic ratio

b a

Circularity Gradient Symmetry Histogram

Orientation Measure for symmetry

Histogram shape (energy, entropy)

Color

Color difference

Color brightness

Texture

Power (spectrum)

Machine Vision: Visual Sensing and Image Processing

109

of acquiring hyper-level image impunity, which is application dependent. In the ﬁnal stage, these extracted features are utilized for the interpretation of the scene, which is an indispensable stage for inspection, measurement, and recognition of the objects. There are two types of the vision system: active and passive, which are needed to capture the image of a 3D environment. Active vision acquires a 3D image by projecting the light energy onto a scene. In contrast, the passive method utilizes images acquired in the visible region of light spectrum with no use of any external light projection. Due to this advantage, this method has been applied to various types of environments. In this chapter, we will treat the principle of an image formation, an imaging device or a sensor that acquires images, and some fundamental image processing techniques, which include ﬁltering, thresholding, and edge detection. Camera calibration will also be discussed, whose process enables us to interpret the acquired image output in terms of a real world coordinate system. Throughout this chapter, we will limit ourselves to a passive vision method, a monocular system, and a monochrome image.

Image Formation Light emits energy in the ultraviolet (UV), visible, and infrared (IR) portions of the electromagnetic spectrum. UV light is invisible with a very short wavelength in the range of 10 to 400 nm. Visible light is electromagnetic radiant energy with a short wavelength between 400 and 700 nm. In contrast, infrared light has a rather long wavelength between 0.7 and 100 mm. Brightness is related to the radiant ﬂux of a visible light emitted from a source. Depending on how particular objects in the scene to be imaged are illuminated and how they reﬂect light, the imaging system presents the image of different brightness. For a given illumination condition and scene, the imaging system will present an image with brightness distributed over the scene in a speciﬁed manner. The distribution of brightness contains such important information as shape, texture of surface, and color. In addition to these, it will also contain the information on the position and the orientation of the objects, which can be obtained by determining the geometric correspondence between points in the image and those in the scene. The brightness of the imaged surface depends upon optical reﬂectance characteristics of imaging surface which are very critical to acquiring images. They are, however, not so simple to analyze since the surfaces reﬂect light in many different ways according to their texture and wavelength. As illustrated in Figure 3.2, there are three typical types of reﬂectance pattern. They are the specular surface, which reﬂects light at an angle equal to the incident angle, the Lambertian surface, which uniformly scatters light in such a way that the light has consistent luminance scattered without

110

Optomechatronics specular diffuse

incident light

specular

diffuse

FIGURE 3.2 Radiance depending on surface condition.

dependency on the view angle, and ﬁnally, the specular diffuse surface, which scatters incident light with some directivity. When a reﬂected light contains specular properties to some extent, it sometimes requires special attention and, therefore, normally needs a special illumination condition and imaging methods. Another important parameter to imaging is the ﬁeld of view, since it determines the angle within which an imaging system can see. The angle is deﬁned by the angle of the cone of directions encompassed by the scene to be imaged, and depends upon the focal length of the system. Normally, a telephoto lens has a narrow ﬁeld of view due to its large focal length relative to the size of the object to be imaged, while an ordinary lens has relatively broader range of ﬁeld of view. Before discussing sensing and image processing, let us consider how the image of an object is formed within the image plane and discuss the geometric correspondence between points in the scene and points in the image. There are two projection models to relate the image to the scene: perspective and orthographic. In the perspective projection shown in Figure 3.3a, each point in the image corresponds to a particular direction from which a ray passes through a pinhole. Let P denote a point in the scene with camera coordinates ðxc ; yc ; zc Þ, and Pi denote its image described in the coordinates ðxi ; yi ; zi Þ: If f is the distance of the image plane from the pinhole, we have zi ¼ f : From the optical geometry, the relation between the two coordinate systems are given by xi y f ¼ i ¼ xc yc zc

ð3:1Þ

and from this we have xi ¼ fxc =zc

yi ¼ f yc =zc

ð3:2Þ

Machine Vision: Visual Sensing and Image Processing

111

image plane yi

xi

zi = f

xc

yc

Oi

object zc

P (xc, yc, zc) zc

Oc Pi(xi, yi)

(a) perspective model image plane yi

xi

Oi

xc

yc

P(xc, yc, zc) Oc

Qi(xi, yi)

object

zc Q(xc, yc, zc)

(b) orthographic model FIGURE 3.3 Image formation models.

This implies that as zc increases, the image point of the object point gets closer to the center of the image plane. Orthographic projection shown in Figure 3.3b involves the projections of two parallel lines to the optical axis on the image plane. Consider again a point Pc ðxc ; yc ; zc Þ in the object located at z ¼ zc : The perspective projection model given in Equation 3.1 can be rewritten as xi ¼ 2mxc

ð3:3Þ f where lateral magniﬁcation is deﬁned by m ¼ 2 : zc When the depth of the scene is small relative to the average distance of the surface from the camera lens, the magniﬁcation m is almost constant. In this case, we can simplify Equation 3.3 by normalizing the coordinate system with m ¼ 2 1, and thus, the orthographic projection is deﬁned by xi ¼ xc

yi ¼ 2myc

yi ¼ yc

ð3:4Þ

This equation implies that, unlike perspective projection, the orthographic projection is not sensitive to changes in depth and can ignore the depth information. This approximation can often be useful for the optical system whose focal length and camera distance are large to compare to the size of the object, as in the case for a microscope observing microparts.

112

Optomechatronics

Imaging Devices There are primarily three types of imaging sensors presently in use: a charge coupled device, a charge injection device, and a photo diode array. The most commonly used device is the charge coupled device (CCD) and, thus, we will describe it in brief. Since AT&T Bell Laboratories invented the CCDs in the late 1960s, the growth has been exploited in almost all application areas. They are currently being used as indispensable imaging detectors of visible, ultraviolet, and x-ray light, and have now become the central element in an imaging system. The CCD visual sensor is composed of an array of tiny photosensitive semiconductor devices called “pixels”. When light is incident on the devices, a charge is generated by absorption of photons and accumulated during a certain period of time. The amount of the charge accumulation is dependent on the intensity of light. The accumulated charge is stored and converted to an electrical signal to be used for image analysis when it is processed properly. To brieﬂy understand how CCDs work, let us consider a unit of CCD cell called “pixel” as shown in Figure 3.4. A semiconductor (p type) has two different states of electrons: One is where the electrons are free to move around in response to an electric ﬁeld and the other is where the electrons are prevented from moving around. When light enters the silicon surface and photons are absorbed, this causes electrons constrained in motion to move freely, leaving behind a hole, an occupied site. This generates charge, i.e., electron– hole pairs. The ﬁgure shows the state of the accumulated charges in a well by introducing an electric ﬁeld with a voltage þ V through an optically transparent electrode. This state is a result of charge creation by incident light. The amount of the charge here represents the integration of light. After accumulating the charge during a speciﬁed integration time, incident lights

transparent electrical contact

+V

e– – e– e e– – e– e e–

depletion region FIGURE 3.4 Generation of charge packet by photons.

p-type semiconductor

Machine Vision: Visual Sensing and Image Processing

113

the charge collected at each pixel needs to be transferred to the adjacent electrode for a sequential move to a readout port. Figure 3.5 illustrates a charge transfer in three phases, using seven electrodes and cycling three voltages (V ¼ 0, 5, and 10 V) on the rectangular grid of electrodes. In Figure 3.5a, the state of the charge due to CCD exposure to incident light is illustrated. A certain amount of charge is accumulated with the electrode voltage V ¼ 10 in the packets A3 and A6. When the voltages on the adjacent wells are increased from 0 to 10 V, the charges become shared between the electrodes E3 and E4 for the packet A3 and E6, and E7 for the packet A6, as shown in Figure 3.5b. Removing the voltages from E3 and E6, the charges in the packets A3 and A6 are completely removed and transferred to the respective adjacent electrodes as indicated in Figure 3.5c. The principle of the transfer of charge from pixel to pixel presented above can be applied to the case for the CCD array conﬁgured in Figure 3.6. The CCD system here contains a large pixel array, multiple vertical registers, and usually one horizontal shift register. The charge transfer ﬂow is sequenced V1 applied voltage V2 V3

0 5 10 E1

pixel electrode

E2

E3

E4

E5

E6

E7

0 A3

10

A6

packet

packet

(a) end of exposure V1 V2 V3

10 0 5

A1

A7

A4

(b) charge transfer V1 V2 V3

5 10 0

A2

A5

(c) end of transfer FIGURE 3.5 Charge transfer in a “three phase” CCD.

114

Optomechatronics CCD array configuration

pixel

vertical shift register

output stage

image data

Horizontal CCD Shift Register FIGURE 3.6 Charge transfer conﬁguration for read-out.

as follows. As indicated by dotted lines and arrows, the accumulated charges are transported down to the horizontal shift register by the vertical shift registers. The horizontal register collects one line at a time and moves each packet of charge in a serial manner far right to the output stage. This is called an interline transfer CCD method. Figure 3.7 shows a typical image signal pushed by shift registers. Image signal level is represented by a solid line ranged from 0 to þ 0.714 V, where þ 0.714 V indicates the white level while 0 V the black level. The horizontal synchronization intervals are denoted by the negative pulses with 10.9 m sec. The dotted lines before and after a pulse indicate the last pixel of a line and the ﬁrst pixel of the next line, respectively. + 0.714V

last pixel of line

white level

video signal 0 – 0.286V

blanking level

first pixel of line

horizontal sync interval 10.9 ms

active line time 52.95ms 1 video line 63.49ms

horizontal line timing FIGURE 3.7 Horizontal synchronization.

black level

Machine Vision: Visual Sensing and Image Processing

3 5

start of odd field frame 2 4

. . . .

477 479

. . . .

480 Active lines

retrace

in the interlaced frame

1

115

478 480 reset to even field frame

FIGURE 3.8 Interlaced image display.

Image Display: Interlacing Interlacing is a method of displaying an image signal by use of the alternate line ﬁeld structure required by the RS-170 format standard. According to the format standard which is administered by the Electronics Industries Association (EIA), the image on the monitor has 480 lines divided into two ﬁelds of 240 lines. As shown in Figure 3.8, they consist of a ﬁeld containing the odd numbered line depicted as solid lines and a ﬁeld containing even number lines depicted as dotted lines. Each ﬁeld is scanned in 1/60th of a second, so that a complete image can be scanned per 1/30 sec. The frequency of scanning is therefore 525 horizontal cycles per 1/30 sec. The time duration for a complete cycle is 63.5 m sec, the forward scan and return trace requiring 52.1 and 11.4 msec, respectively.

Image Processing The image signal sensed by CCD cells is transferred to the acquisition unit called a frame grabber. This captures video signals, buffers or stores the images, and simultaneously displays the image on an external monitor. Figure 3.9 is a simpliﬁed version of the grabber and shows its components. The major components are: a video multiplexer that takes in video signals up to N channels, an analogue to digital (A/D) converter that converts analogue

116

Optomechatronics

connected to PC/104 Plus

image grabber

CCD camera

video 1 video 2 …

. video n

video MUX

A-to-D converter amplifier

composite sync HSync VSync

On-board processing unit buffer memory

sync generator

clock syncronization

external trigger

BUS interface

internal clock PCI, ISA and PC104+ BUS Line

FIGURE 3.9 Architecture of an image grabber. Source: Datatranslation Co. Ltd (www.datx.com) and Meteor Co. Ltd (www.natrox.com).

video signal to digital signal, an onboard processing unit that processes various real-time tasks, a frame buffer that stores the image data and buffers its ﬂow, and the interfacing unit that allows to transfer the processed image data onto a host computer (CPU) through bus lines such as PCI, ISA, PC 104þ, and so on. We will discuss some of the characteristic features of a CPU, a multiplexer, an A/D converter, and the data bus later in Chapter 4.

Image Representation Brightness of the image acquired by the camera is deﬁned on every pixel within the CCD array, which is a function of spatial coordinates. To describe the brightness an image function is usually used, which is a mathematical representation of an image that a camera produces on a CCD cell. The function denoted by f ðx; yÞ deﬁnes the brightness of the gray level at the spatial location ðx; yÞ: In other words, the brightness of the image at the pixel point ðx; yÞ is represented by f ðx; yÞ: When digitized by the frame grabber, it can be represented by f ði; jÞ where i and j are the ith row and jth column of

Machine Vision: Visual Sensing and Image Processing

117

FIGURE 3.10 Gray image and binary image.

the 2D CCD array cell, respectively. Two types of image will be considered here: binary and gray scale. Binary Image The binary images have two-valued gray levels and are produced from the gray scale image by using a threshold technique to segment the image into regions. Let us consider a 256 £ 256 image of a single object in the ﬁeld of view of a camera, with a background having different brightness, as shown in Figure 3.10. The object and the background are seen to have approximately uniform brightness. Since, in this case, the object appears darker than the background does, it is easy to segment these into two regions, 0 for the object and 1 for the background. The image represented by this two-valued function is called binary image. Such an image can be obtained by thresholding the gray level image, which we will discuss later. If the pixel intensity is larger than that of the set threshold, it is regarded to possess “1”. On the other hand, if the intensity is below it, it is regarded to possess “0”. The binary image is easier to digitize and store than a full gray-level image, but some information is lost. Therefore this image is usually used for simple image analysis or analysis of simple image. Gray Scale Image In contrast to the binary information, gray scale images have a number of different brightness levels. The levels can be expressed by using eight bit/pixel which expresses 255 different levels. The levels often become

118

Optomechatronics

FIGURE 3.11 Various shapes of histogram.

a burden of computation. And they also have values in the range 0 to 63, or 0 to 1023, corresponding to six or ten bit digitization, respectively. Histogram Histogram representation of an image is a fundamental tool for image analysis and enhancement. The histogram is a plot of the relationship between gray scale value and the number of pixels at that value and, thus, it is a discrete function. As we shall see later, the shape and the magnitude of the histogram are indicative of the characteristic nature of the image, and they are very useful in determining a threshold. Figure 3.11 shows three different histograms where an image has a variety of different distributions over gray level. Shown in Figure 3.11a is the bimodal distribution in which two groups of gray level distributions are somewhat distinct. In this case, the whole image is usually divided into a binary image, that is, black “0” and white “1”. This is one of the simple forms of histogram, which makes it easy to do image analysis. Figure 3.11b shows a very narrow, clustered histogram, which is bad from the viewpoint of image quality. This is because the image has very low contrast, thus making it difﬁcult to discriminate different objects within the image. On the other hand, the histogram shown in Figure 3.11c indicates a spread of histogram over the wide range of gray values, which is good from the view point of image contrast. In this case, it is said to have a high contrast. Histogram Modiﬁcation When the contrast of an image is low due to poor lighting, incorrect set-up of the camera, or various other reasons, the image is not easily understandable. This is because the differences in the intensity between pixel values are small, so that objects and the background in the image cannot be discerned. In this case, it is necessary to improve the contrast of a poor contrast image.

max.

Ia

Ib

gray level

No. of pixels

(a)

min.

(b)

intensity after enhancement

No. of pixels

Machine Vision: Visual Sensing and Image Processing

119

Ib′ z′

Ia′ Ia Ia′

intensity

Ib′

gray level

z I b

z

intensity before enhancement

(c) Intensity transformation relationship

FIGURE 3.12 Contrast enhancement.

In general, there are two commonly adopted methods of modifying the contrast as shown in Figure 3.12. Let us assume that the original pixel intensities lie in the range Imin # I # Imax ; as shown in Figure 3.12a. This indicates that the image has most of its pixels in the range between Ia and Ib : The ﬁrst method shown in Figure 3.12b is histogram stretch which extracts a certain range of the intensity values of the interested pixels and modiﬁes them in such a way that a new intensity distribution can make the original image more visually discernable. Let us assume that we are interested in mapping the intensity level range of the original image shown in Figure 3.12a to a new intensity distribution shown in Figure 3.12b. Suppose that we wish to transform the original intensity distribution into a modiﬁed distribution. In order words, if a new intensity value z0 is mapped from the corresponding value z, intensity z Ia # z # Ib

!

intensity z0 I 0a # z0 # I 0b

To achieve this, a transformation formula for a histogram stretch can be derived from Figure 3.12c. z0 ¼

I 0b 2 I 0a ðz 2 Ia Þ þ I 0a Ib 2 Ia

ð3:5Þ

The second method called “histogram equalization” shown in Figure 3.13 transforms the original image such that an equal number of pixels is

Optomechatronics

No. of pixels

120

0

dr

r

gray level

No. of pixels

(a)

(b)

0

intensity

ds

s

gray level

FIGURE 3.13 Histogram equalization.

allocated throughout the intensity range of interest. The effect of the histogram equalization is to improve the contrast of an image. In other words, there will be an increase in brightness differences of pixel values near the intensity values at which there are many pixels. In contrast to this, there will be a decrease in brightness differences near the intensity values with few pixels. To illustrate the concept in brief, let us suppose that gray levels described in the original image shown in Figure 3.13a can be transformed in a new gray level distribution having an equal number of pixels within a certain gray value range as illustrated in Figure 3.13b. We wish to ﬁnd such transformation s ¼ TðrÞ where r and s are the variables representing gray level in the original image and that in the new image, respectively. If the probability density function Pr ðrÞ is assumed to represent the gray level pattern in the range ½r; r þ dr shown in Figure 3.13a, then the transformation will be made to obtain Pr ðsÞ in the range Pr ½s; s þ ds : Because the total number of pixels in the two images will remain equal, we have Pr ðsÞds ¼ Pr ðrÞdr This equation is a basic equation to the histogram equalization process which is essentially to make all the gray levels in the original image equally

Machine Vision: Visual Sensing and Image Processing

121

probable. By letting Pr ðsÞ be a constant C, we have ðs 0

C ds ¼

ðr 0

Pr ðrÞdr

The above equation leads us to obtain the range of the gray value having an equal probability s¼

1 ðr P ðxÞdx C 0 r

where x is the dummy variable. This is illustrated in Figure 3.13b. Other histogram modiﬁcations such as histogram stretching, shrink, and sifting can also be performed in a similar manner. Histogram shrink is the opposite of a histogram stretch and decreases image contrast by compressing the gray level. Histogram slide is to translate the histogram in order to make an image either darker or lighter, while retaining the relationship between gray level values.

Image Filtering As mentioned earlier, noise more or less occurs in the image, corrupting a true intensity variation. Filtering an image is to recover its intrinsic visual information when it contains unwanted noise. This can be done by transforming the image intensities in order to remove noise or by performing image enhancement. There are two types of ﬁltering depending upon whether the transformation is made in spatial or frequency domain: spatial domain ﬁlter and frequency domain ﬁlter. We will discuss brieﬂy only spatial ﬁlters such as mean ﬁlter and median ﬁlter, which are frequently used to remove noise. Before getting into further details of image analysis, it is very helpful to become familiar with the convolution concept in order to understand ﬁltering and edge-detecting procedures. Figure 3.14 indicates three cases of convolution masks. They show the 1 £ 2, 2 £ 1, 2 £ 2, and 3 £ 3 masks deﬁned in an image frame. In the ﬁgure, M are the masks operated on the speciﬁed pixels, which are also called detection operators. And mij indicates the weighting values placed on the respective pixels. As indicated for a (3 £ 3) mask in Figure 3.15, the convolution process proceeds as follows; Overlay the mask on the image frame, multiply the corresponding pixel value one by one, sum all these results, and place this result at the center of its mask, as shown in the ﬁgure. This process continues until it is completed with the ﬁnal pixel of the image. In a mathematical form, the output of the summation of the convolution results

122

Optomechatronics i 2x2 convolution mask

i

1x2 convolution mask Mx = m11 m12 j

M =

f(i,j ) f (i+1,j ) f(i,j+1)

j

f(i,j )

m11 m12 m21 m22

f(i+1,j )

f(i,j+1) f(i+1,j+1)

My = m11

2x1 convolution mask m 21 image frame

(a) 1×2 and 2×1 convolution masks

(b) 2×2 convolution mask

i

3x3 convolution mask

M= f(i-1,j-1) f(i,j-1) f(i+1,j-1) j

f(i-1,j)

f(i,j )

image frame

m11 m12 m 13 m21 m22 m 23 m31 m32 m 33

f(i+1,j)

f(i-1,j+1) f(i,j+1) f(i+1,j+1)

(c) 3×3 convolution mask FIGURE 3.14 Convolution masks.

3x3 mask

. mask center

FIGURE 3.15 The procedure of applying convolution mark.

image frame

Machine Vision: Visual Sensing and Image Processing

123

can be written as gði; jÞ ¼ f ði; jÞ p mði; jÞ ¼

p n X X

f ði 2 k; j 2 lÞmðk; lÞ

ð3:6Þ

k¼1 l¼1

where: p denotes the convolution of f ði; jÞ with mði; jÞ; gði; jÞ is the sum of results of the multiplication of coincident values, f ði; jÞ represents the input image, and mði; jÞ is called the convolution mask. For the case of a 3 £ 3 mask, the output gð2; 2Þ in the above is computed by gð2; 2Þ ¼ m11 f11 þ m12 f12 þ m13 f13 þ m21 f21 þ m22 f22 þ m23 f23 þ m31 f31 þ m32 f32 þ m33 f33

ð3:7Þ

This procedure is explicitly indicated in the ﬁgure. Referring to Equation 3.7, the convolution values at pixel ði; jÞ can be easily computed by the following manner; For 1 £ 2 and 2 £ 1 masks Mx ¼ m11 f ði; jÞ þ m12 f ði þ 1; jÞ;

My ¼ m11 f ði; jÞ þ m21 f ði; j þ 1Þ

ð3:8Þ

For 2 £ 2 masks M½ f ði; jÞ ¼ m11 f ði; jÞ þ m12 f ði þ 1; jÞ þ m21 f ði; j þ 1Þ þ m22 f ði þ 1; j þ 1Þ

ð3:9Þ

For a 3 £ 3 mask M½ f ði; jÞ ¼ m11 f ði 2 1; j 2 1Þ þ m12 f ði; j 2 1Þ þ m13 f ði þ 1; j 2 1Þ þ m21 f ði 2 1; jÞ þ m22 f ði; jÞ þ m23 f ði þ 1; jÞ þ m31 f ði 2 1; j þ 1Þ þ m32 f ði; j þ 1Þ þ m33 f ði þ 1; j þ 1Þ

ð3:10Þ

Mean Filter A mean ﬁlter is one of the simplest linear ﬁlters and is essentially an average ﬁlter that operates on neighboring pixels within an m £ m window. It replaces the center pixel with an average value of the image intensities of the pixels. A convolution mask travels through the image for computation and replacement of the computed average. This can be mathematically expressed by X ~ jÞ ¼ 1 fði; f ðk; lÞ N ðk;lÞ[w

ð3:11Þ

124

Optomechatronics

where N is the total number of pixel within the window W. For example, if a 3 £ 3 mask is given as 1 9

1 9

1 9

1 9

1 9

1 9

1 9

1 9

1 9

~ jÞ becomes then, fði; jþ1 iþ1 X X ~ jÞ ¼ 1 fði; f ðk; lÞ 9 k¼i21 l¼j21

ð3:12Þ

It is noted that the sum of the coefﬁcients of the mask is 1. Median Filter A median ﬁlter is effective in coping with impulse noise, while retaining original image details. This is because it does not consider any values which are signiﬁcantly different from the typical values in the convolution mask. To see how it works, consider Figure 3.16 where a mask (3 £ 3) of the ﬁlter is shown. It looks similar to the convolution mask. We can see that the process is not a weighted sum, but employs a nonlinear operation. In this method, all intensity values are sorted in an ascending order for pixels within the mask. Then the value of the middle pixel is selected as the new value for pixel ði; jÞ:

input image

filter window (3x3 case) 14 8 0 4 9 35 6 5 27

FIGURE 3.16 The concept of median ﬁltering.

output image ordered pixels 0 4 5 6 8 9 14 27 35

Machine Vision: Visual Sensing and Image Processing

125

This process is repeated in a successive fashion until all pixel operations are undertaken. The ﬁgure shows a 3 £ 3 ﬁlter window arriving at a designated location. Putting all the intensity values in ascending order and taking the middle value among them yields the intensity value, “9”. For a 3 £ 3 mask, ﬁltered output value G½f ði; jÞ can be written by G½f ði; jÞ ¼ Med{ f ði 2 1; j 2 1Þ; f ði; j 2 1Þ; f ði þ 1; j 2 1Þ; f ði 2 1; jÞ; f ði; jÞ; f ði þ 1; jÞ; f ði 2 1; j þ 1Þ; f ði; j þ 1Þ; f ði þ 1; j þ 1Þ}

ð3:13Þ

where Med{L} indicates the operator that takes the value of the middle pixel. Often 5 £ 5 and 9 £ 9 masks are used but their ﬁltering methods are the same as that of the above 3 £ 3 operator. Let us consider the original natural image composed of a human face and a bookshelf as shown in Figure 3.17a in order to see the performance of the median ﬁlter. It contains salt and pepper noise of 20% intentionally added to the original. A 3 £ 3 median ﬁlter is operated on it. As can be seen from Figure 3.17b, the noise is almost removed. The result indicates that this ﬁlter is very effective to impulsive noise. In order to obtain the above ﬁltered image by processing the median ﬁlter on this image, Matlab M-ﬁles are used. In the M-ﬁles we write Matlab-speciﬁc statements and execute a series of such written statements. Digital image processing methods using Matlab toolbox are well documented in Ref. [7]. Throughout this chapter we will use M-ﬁles to create the Matlab statements. The followings are the Matlab codes for obtaining median ﬁltering (Box 3.1).

FIGURE 3.17 Median ﬁltered image.

126

Optomechatronics

Box 3.1. FILTERING : MEDIAN FILTER Matlab source: Median Filter clear all % reading image File image ¼ imread(‘tao.bmp’); % add salt & pepper noise n Percent ¼ 20; [y, x ] ¼ size(image); nMaxHeight ¼ round(y p n Percent/100.0); nMaxWidth ¼ round(x p n Percent/100.0); for I ¼ 1:nMaxHeight, for J ¼ 1:nMaxWidth, cx ¼ round(rand(1) p (x 2 1)) þ 1; cy ¼ round(rand(1) p (y 2 1)) þ 1; aaa ¼ round(rand(1) p 255); if aaa . 128 image(cy,cx) ¼ 255; else image(cy,cx) ¼ 1; end end end % median ﬁltering for i ¼ 1:x for j ¼ 1:y if(i ¼ ¼ 1 ll j ¼ ¼ 1 ll i ¼ ¼ x ll j ¼ ¼ y) % boundary image_out( j,i) ¼ image( j,i); % not changed else for l ¼ 1:3 for k ¼ 1:3 window(l þ (k 2 1) p 3) ¼ image( j þ l 2 2,i þ k 2 2); end end %sorting for l ¼ 1:8 for k ¼ 2:9 if (window(l) . window(k)) temp ¼ window(k); window(k) ¼ window(l); window(l) ¼ temp; end end

Machine Vision: Visual Sensing and Image Processing

127

end image_out( j,i) ¼ window(5); % medain value end end end % showing image ﬁgure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_out);

Image Segmentation Thresholding Thresholding is the simplest and still one of the most effective methods to segment objects from their background. Segmentation is an image processing method that splits the image into a number of regions, each having a high level of uniformity in brightness or texture. It reduces a gray scale image to binary black or white pixels. In the binary image, objects appear as black ﬁgures on a white background, or as white ﬁgures on a black background. One method to extract the objects from the background is to use a thresholding method in which the object and the background pixels have gray levels grouped into two dominant modes. Any point ðx; yÞ having gray level greater than a threshold value, T; that is, f ðx; yÞ . T; belongs to an object point while the point for f ðx; yÞ , T is called a background point. In this case, we need only one threshold value to classify the two, but in other cases, multiple thresholds may be needed when the image contains brightness information rather complex so that it cannot be ideally made binary. In the case of an image containing only two principal brightness regions, a global thresholding is the simplest technique, which partitions the image histogram by using a single threshold, T. Consider the gray level histogram shown in Figure 3.18a, which corresponds to an image f ðx; yÞ: It is composed of light objects on a dark background, having the object and the background pixels grouped into two dominant modes in terms of gray level. The thresholding problem is to ﬁnd a threshold Tsuch that the two modes are well separated. If a thresholded image, gðx; yÞ; is deﬁned by ( gðx; yÞ ¼

1

if f ðx; yÞ . T

0

if f ðx; yÞ # T

) ð3:14Þ

then the pixels labeled 1 correspond to the objects composed of the object points, whereas the pixels labeled 0 correspond to the background composed

Optomechatronics

number of pixels (N)

number of pixels (N)

128

(a) single threshold

T

T1

intensity

T2

(b) multiple threshold

intensity

FIGURE 3.18 Idealized gray level histogram.

of the background points. By labeling the image pixel by pixel in this way, segmentation is achieved, depending on whether the gray level of the pixel is greater or less than the T value. To illustrate the threshold technique, two scenes with a 256 £ 256 image will be considered. As shown in Figure 3.19, clearly the binary image of the

histogram

number of pixels

10000 background 5000 object 0

0

50

(a) the image of a tree leaf and its corresponding histogram

T1=180

T2 = 150

(b) the processed images with two different thresholds FIGURE 3.19 Thresholding process using single thresholding value.

100 150 intensity

200

250

Machine Vision: Visual Sensing and Image Processing

129

tree leaf differs due to variation of the threshold value. Consider the image of a tree leaf whose corresponding histogram is shown in Figure 3.19a. From the histogram, the leaf is seen to roughly have the intensity range of 0 to 150, while that of the background is 150 to 255. Based on the thresholding operation given in Equation 3.14, the image is thresholded with two different values of T as shown in Figure 3.19b. It can be seen that when the image is thresholded at T ¼ 180; a clear binary image can be obtained. However, as T moves toward a smaller value, for example, 150, the leaf image starts to be deteriorated, showing a slight difference from that of the original. This is because, in this case, the pixels having intensity larger than 150 are classiﬁed to background (white region) rather than to part (black). A more complicated scene is the latter case, of which the original image is shown in Figure 3.20a. Thresholding at T ¼ 128 is used to obtain its binary image. It can be observed that, due to the similar gray level among the objects, the background, and the shadow within the scene, the binary image shown in Figure 3.20b does not exactly represent the original face image, but somehow retains some important features. The following is the Matlab source code for obtaining such image (Box 3.2). Returning to Figure 3.18b it shows two types of objects on a dark background, which is a more general case. In this case, we need to deﬁne two thresholds T1 and T2 as shown in the ﬁgure. The three modes separated by T1 and T2 are: object 1

if f ðx; yÞ . T2

object 2

if T1 , f ðx; yÞ , T2

background

if f ðx; yÞ , T1

FIGURE 3.20 Gray-to-binary image transformation.

ð3:15Þ

130

Optomechatronics

Box 3.2. (THRESHOLD ) Matlab source: Threshold clear all % reading image File image ¼ imread(‘tao.bmp’); [y,x ] ¼ size(image); threshold ¼ 128; % deﬁned by user % threshold for i ¼ 1:x for j ¼ 1:y if image( j,i) . 128 image_th( j,i) ¼ 255; else image_th( j,i) ¼ 0; end end end % showing image ﬁgure subplot(1,2,1), imshow(image) subplot(1,2,2), imshow(image_th)

The images having a multimode histogram of this type can be frequently found in most of the real scenes. For instance, when an IC chip is placed on a table, the gray level of this image may form three different distinct ranges; IC body, lead, and background. In this case, image analysis is more difﬁcult than the case of the distinct histogram distribution. Choice of threshold value in this case needs to be made depending upon the information to be extracted from the image. Figure 3.21 shows the images of various electronic parts and bolts. The objects include three chips with white characters engraved on their top surfaces and the image is observed to contain several shaded areas around their bodies. As can be observed from the ﬁgure, the contour and the shape of each object become clearer at T ¼ 50 than those of the other two threshold values ðT ¼ 100; 150Þ; and the shadow effect is small. In the case of the electronic chip having several leads of brighter color, the contours are hardly seen from the image. When T is increased to 100, the white characters become clearly recognized and the shadow effect becomes signiﬁcant. However, at T ¼ 150; the shadow signiﬁcantly inﬂuences the whole object image and, thus, the object and the shadows can not be distinguished easily. This example clearly shows a choice of threshold values should be appropriately made depending upon the objective of the given task.

Machine Vision: Visual Sensing and Image Processing

131

FIGURE 3.21 Thresholding of a variety of electronic parts having different imaging characteristics.

In general, multilevel thresholding becomes a difﬁcult task when the number of separation of regions of interest is large. An approach to this type of problems is to use a single, adaptive threshold. The difﬁculty also arises when an image is contaminated by some noise, which is frequently encountered in actual machine vision environments. In this case, optimal thresholding techniques can be used in such a way that the probability of the classiﬁcation error, that is, erroneously classifying an object point as a background point and vice versa, is minimized with respect to T. Iterative Thresholding The previous discussions reveal that there is a need for an optimal thresholding technique which yields the best binary image quality. Some of the techniques to achieve this objective as well as to work for two principal brightness regions include: (1) Iterative threshold method (2) Optimal thresholding using maximum likelihood. The iterative method does not require an estimated function of a histogram and, therefore, works for more general cases, whereas the second method requires an analytical function of the histogram for optimization. In the following, the algorithms for the method will be introduced, based upon

132

Optomechatronics

heuristic approach. Let us assume that an image to be segmented has two brightness regions. The algorithm begins then: (1) Choose an initially estimated value of the threshold value T. (2) Partition the image into two groups R1 and R2 using the T value. (3) Calculate the mean gray value fR1 ðx; yÞ and fR2 ðx; yÞ of the partitioned group R1 and R2 ; respectively. X f ðx; yÞ fR1 ðx; yÞ ¼

fR2 ðx; yÞ ¼

ðx;yÞ[R1

Pixel number of the R1 region X f ðx; yÞ

ð3:16Þ

ðx;yÞ[R2

Pixel number of the R2 region

(4) Choose a new threshold to have T¼

fR1 ðx; yÞ þ fR2 ðx; yÞ 2

ð3:17Þ

(5) Repeat step (2) through step (4) until the mean gray values fR1 and fR2 do not change as iteration proceeds. 2500 frequency

2000 1500 1000 500 0

(a) original image

T= 100

0

50

100 150 intensity (b) histogram of the image

T= 130.7

(c) binary images obtained from the iterative thresholding process FIGURE 3.22 The iterative thresholding process.

200

T= 147

250

Machine Vision: Visual Sensing and Image Processing

133

This algorithm discussed above is illustrated with the example of the electronic part and electric resistance. To begin with, we choose the initial value of T ¼ 100; which is indicated by a histogram in Figure 3.22a. According to Equation 3.16 and Equation 3.17 we can calculate fR1 ðx; yÞ and fR2 ðx; yÞ They are determined by fR1 ðx; yÞ ¼ 186:1

fR2 ðx; yÞ ¼ 75:2

ð3:18Þ

which enables us to obtain a new threshold value. We then use a new value Tð1Þ ¼ 130:7 and its image is indicated in Figure 3.22b for the next iteration. Repeating in this way, we obtain Tð2Þ ¼ 142; Tð3Þ ¼ 145; and ﬁnally a steady state T value, Tð4Þ ¼ 147; as shown in Figure 3.22c. It can be seen that, for this single and simple object, the convergence rate is very fast, reaching its optimal value only after four iteration steps. Region-Based Segmentation In the previous section, thresholding was used to partition an image into the regions based on the distribution of the pixel intensity. Another way of segmenting the image is to ﬁnd the regions directly. Region splitting and merging, and region growing belong to this segmentation category. Here, we will discuss a basic formulation based on splitting and merging technique. As shown in Figure 3.23, the image window contains seven different regions ðR1 ; R2 ; R3 ; R4 ; R5 ; R6 ; R7 Þ having different intensity color and texture. The purpose of this segmentation method is to divide the region into seven segments having identical properties. To generalize this concept, let us deﬁne R to represent the entire image region referring to the ﬁgure. Then, a segmentation is regarded as a process partitioning R into n subregions R1 ; R2 ; …; Rn : The conditions that must be satisﬁed during the process are then; ð1Þ

n

< Ri ¼ R

i¼1

ð2Þ Ri is a connected region; ði ¼ 1; 2; …nÞ: ð3Þ Ri > Rj ¼ f

;i and j; i – j:

ð4Þ PðRi Þ ¼ true;

i ¼ 1; 2; …n:

ð5Þ PðRi < Rj Þ ¼ false;

i–j

ð3:19Þ

In the above, f is the null set, and PðRi Þ is a logical predicate over the points in set Ri : Condition (1) implies that every pixel must be in a region while condition (2) indicates connectivity in that all points in a region must be connected. Condition (3) requires that there should be no jointed regions between regions Ri and Rj within R. In other words, they are disjointed between them. Condition (4) indicates that all points in Ri must have the

134

Optomechatronics

FIGURE 3.23 Seven divided different image regions.

same properties such as the intensity and the color. For instance, PðRi Þ ¼ True; if all pixels in Ri have an identical intensity. Condition (5) implies that the property of Ri must be different from that of Rj : Region splitting and merging is, therefore, to subdivide an image into a set of disjointed regions while satisfying the conditions given in the above. During this process, splitting and merging the regions are iteratively carried out. Consider an image frame R composed of four subregions, R1 ; R2 ; R3 ; R4 ; as shown in Figure 3.24. If R1 and R2 satisfy the conditions in Equation 3.19, where R3 and R4 do not, then R3 and R4 need to be segmented into eight subregions, each into four. Once again, these eight regions are segmented into smaller subregions. This segmentation process continues until all the subregions satisfy the conditions. To illustrate this basic concept, a binary image is depicted in the left hand side of Figure 3.25a, which consists of a single object (black) and the background (white). Within each region, the intensity will be assumed to be identical. Then, in the ﬁrst step, we divide the image into four quadrants, R1 ; R2 ; R3 ; and R4 : We see that two quadrants R2 and R4 satisfy the predicate while the other two do not; PðR1 Þ ¼ false; PðR2 Þ ¼ true; PðR3 Þ ¼ false; PðR4 Þ ¼ true

ð3:20Þ

Therefore, PðR2 Þ and PðR4 Þ are not changed. In the next step, we further divide two “false” regions into subquadrants as shown in the ﬁgure. The followings hold for R1: PðR11 < R13 Þ ¼ true; PðR14 Þ ¼ true

ð3:21Þ

Machine Vision: Visual Sensing and Image Processing

R1

R2

R3

R4

R31

R32

R41

R42

R33

R34

R43

R44

135

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

331 333

332 334

341 341

341 341

431 431

431 431

441 441

441 441

(a) partioning the image region R R

R1

R2

R31

R4

R3

R32

R33

R331 R332 R333 R334

R34

R41

R42

R43

R341 R342 R343 R344 R431 R432 R433 R434

R44

R441 R442 R443 R444

(b) tree representation FIGURE 3.24 Segmentation for the split and merge algorithm.

Combining Equation 3.20 and Equation 3.21, we obtain PðR2 < R4 < R11 < R13 Þ ¼ true for “0” intensity level PðR14 Þ ¼ true for “1” intensity level Following the same procedure shown in the above for the subquadrant R13 in Figure 3.25b, we obtain PðR2 < R4 < R11 < R13 < R31 < R33 < R34 Þ ¼ true for “0” intensity level PðR14 UR32 Þ ¼ true for “1” intensity level

ð3:22Þ

One more segmentation step yields the following ﬁnal conditions. PðR2 < R4 < R11 < R13 < R31 < R33 < R34 < R121 < R122 < R123 Þ ¼ true PðR14 < R32 < R124 Þ ¼ true

ð3:23Þ

The ﬁnal result, which combines all of these results, is obtained by merging the regions satisfying the conditions speciﬁed in Equation 3.19.

136

Optomechatronics Segmentation R1

R2

split

(a) R11

split

(b)

R4

R3 R12

R13

R14

R31

R32

R33

R34

split

R121

R122

R123

R124

FIGURE 3.25 Illustration of the region based segmentation.

Edge Detection In the previous section, we have discussed various methods of image segmentation, discriminating one region from another in the image of interest. Edge detection is one such method for analyzing and identifying image contents, and presents the most common approach for detecting discontinuities in gray level. In this section, we will consider the detection of edges and discuss its basic concepts, the processing steps associated with edge detection, and the edge detectors. As illustrated in Figure 3.26, an edge in the image is the boundary between two regions with relatively distinct gray level value. In other words, the gray level difference is distinct, exhibiting a signiﬁcant local change in the image intensity as shown in Figure 3.26a. The variation in the gray level f ðx; yÞ in the x-y image plane can be expressed as Gx and Gy in Figure 3.26b. As we go along from point A to point B through C in the image shown in Figure 3.26c, we can see that f ðx; yÞ slowly decreases to somewhere near point C. The gray level has a drastic change and afterwards it changes very little until it

Machine Vision: Visual Sensing and Image Processing

137

y

y

Gy

B

a Gx

C

C

A

G

x

(a) line A-B f(x,y)

x

(b) image f (x,y) G [ f ( x , y )]

A

C

B

(c) profile of a line A-B

A

C

B

(d) magnitude of first derivative

FIGURE 3.26 Edge characteristics.

hits point B. If f ðx; yÞ is differentiated with respect to spatial points along the line A – C – B, the magnitude of the derivative Gðx; yÞ is expressed as shown in Figure 3.26d. Clearly, the maximum value of Gðx; yÞ occurs at point C. This information provides a useful cue in edge detection, as we shall see later. In general, the edge varies with spatial location in the image. There are several standard edge proﬁles deﬁned depending on its shape: step, ramp line, and roof. Almost all general edge proﬁles are composed of these standard proﬁles. Along with these proﬁles, there usually occurs discontinuity in the image intensity. To mathematically characterize the edge, let us consider the image frame shown in Figure 3.26a of which intensity distribution is deﬁned by f ðx; yÞ: An edge is then characterized by the gradient of the image intensity having two components: magnitude and direction. The image gradient is deﬁned by " G½ f ðx; yÞ ¼

Gx Gy

#

2 6 6 ¼6 4

›f ›x ›f ›y

3 7 7 7 5

ð3:24Þ

The magnitude of the gradient is expressed by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ G½ f ðx; yÞ ¼ G2x þ G2y

ð3:25Þ

138

Optomechatronics

For simplicity of implementation, the gradient magnitude is approximated by addition of the gradient components as; G½ f ðx; yÞ ¼ lGx l þ lGy l

ð3:26Þ

where the direction of the gradient at location ðx; yÞ denoted by aðx; yÞ is given by

aðx; yÞ ¼ tan21

Gy Gx

ð3:27Þ

which is deﬁned with respect to the x axis. It is noted that the above Equation 3.26 and Equation 3.27 require computation of the partial derivatives ›f =›x; ›f =›y at every pixel location. However, the edge taken by CCD camera is discrete in nature and, thus, needs to be numerically approximated by different equations as follows: Gx . f ði þ 1; jÞ 2 f ði; jÞ

Gy . f ði; j þ 1Þ 2 f ði; jÞ

ð3:28Þ

where i and j refer to the ith pixel in x direction and jth pixel in negative direction, respectively. As already discussed, in order to calculate all gradient magnitude values within the image frame of interest, we locate the operator at the left uppermost corner as an initial point and calculate a gradient magnitude at that location. To obtain the next value, we move the mask to the next pixel location. In other words, one value is obtained at each point. This procedure is repeated until the computation is completed for all pixels located within the frame. Gradient operators are in fact the convolution masks that enable us to compute the gradient vector of a given pixel and thus detecting edges, examining small neighborhoods. Table 3.2 summarizes the operators being used for edge detection. The operator approximating the ﬁrst derivative (Roberts, Sobel) will be examined ﬁrst and then the second derivative operator (Laplacian). And then the operator called Laplacian of Gaussian (LoG) which combines a Gaussian ﬁlter and Laplacian operator will be treated. The edge operators discussed in the above have different characteristics responding differently to edge and noise. For instance, some operators may be robust to noise but miss some critical edges while others may be sensitive to noise although they detect most edges. This necessitates a performance metric for edge detection operator. One such metric called the Pratl Figure of Merit is frequently used for the comparison purpose, but we will not deal with the further details here. Roberts Operator The Roberts operator is a very simple gradient operator that uses a 2 £ 2 neighborhood of the current pixel. The operator is identical to that shown in

Machine Vision: Visual Sensing and Image Processing

139

TABLE 3.2 Properties of Various Edge Operators Detection

Typical Operator/Mask

Roberts

Gmx = Pwett

1

0

0

−1

Gmy =

−1 −1 −1

Gmx = Sobel

1

0 −1

0

1

0

0

Gmy = −1

0

1

1

1

1

−1

0

1

−1

0

1

Gmy = −2 0

2

Gmx = 0

0

0

1

2

1

−3 −3 5

−1

0 −1

∆ Robinson compass mask

−1

−3 5

0

0 −1

0

−1 0

1

5

or

−1

8

−1

−1 −1 −1 −2 −1 0

… G = −1 0 −1 m7

Gmo = −2 0 2 −1 0 15

0

1

m1 m2

Magnitude: maximum of convolution of eight major compass masks Direction: maximum value of eight compass mask Rotationally symmetric direction: sign of the result from two adjacent locations

−1 −1 −1

4 −1

qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Magnitude: m21 þ m22

1

−3 −3 −3

−3 −3 5

2=

0

qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Magnitude: m21 þ m2 m Direction: tan21 2 m1

Direction: tan21

… G = −3 0 5 m7

Gmo = −3 0 5

Laplacian

Edge point only

0 −1

0

−1 −2 −1

Kirsch compass mask

Edge Magnitude/Direction

2

Magnitude: maximum of convolution of eight major compass masks Direction: maximum value of eight compass mask

Figure 3.14b and is written by

Gmx =

1

0

0

−1

Gmy =

0

−1

1

0

ð3:29Þ

Using this operator, Equation 3.26, and Equation 3.28, the magnitude of the gradient value at pixel ði; jÞ is obtained by G½ f ði; jÞ ¼ lf ði; jÞ 2 f ði þ 1; j þ 1Þl þ lf ði; j þ 1Þ 2 f ði þ 1; jÞl

ð3:30Þ

Note that, due to its geometric conﬁguration, the above approximate difference values are assumed to be computed at the interpolated point

140

Optomechatronics

ði þ 1=2; j þ 1=2Þ: As can be observed, this operator is very sensitive to noise because only 2 £ 2 pixels are used to approximate the gradient. Sobel Operator This operator uses a 3 £ 3 mask that considers the neighborhood pixels for the gradient computation as shown in Figure 3.14c. It is expressed by

Gmx =

−1

0

1

−2

0

2

−1

0

1

Gmy =

1

2

1

0

0

0

−1

−2

−1

ð3:31Þ

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 85

85 255 255 255 255 255 255

255 255 255 255 255

85

41

41

255 255 255 255

85

41

0

0

41

255 255 255 85

85 255 255 255 255 255 85 25 5 255 255 255

41

0

0

0

0

41

255 255

85

41

0

0

0

0

0

0

85 255 255 255 41

85 255 255

255 255

41

0

0

0

0

0

0

0

0

41 255 255

255 255

85

85

85

85

85

85

85

85

85

85 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

(a) original mosaics mimicking a real image 0

0

0

0

0

0

0

0

0

0 -170 -170 170 170

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0 -170 -510 -510 -170 0

0

0

0

0

0

0

0

0

0 -170 -554-812 812 -554 -170 0

0

0

0

0

0

0

0

0 -170 -554 -683 -469 -469 -683 -554 -170 0

0

0

0

0 -170 -554-683 -425 -208 -208 -425 -683 -554 -170 0

0

0 -170 -554 -384 384 554 170

0

0 -170 -554 -683 -299 299 683 554 170

0

0

0

0

0

0 -170 -554- 683-425 -126 126 425 683 554 170

0

0

0

0

0

0

0

0 -170 -554-6 83-425 -167 -41 41 167 425 683 554 170

0

0 -170 -554-683 -425 -167 -41 -41 -167 -425 -683 -554 -170 0

0 -554 -853 -425 -167 -41

0

0

41 167 425 853 554

0

0 -214 -513 -425 -167 -41

0 -768 -894 -167 -41

0

0

0

0

41 167 85 768

0

0

0 -554 -595 -41

0

0

0

0

0

0

41 595 554

0

0

214 683 979 102010201020102010201020 979 683 214

0

0 -170 -170 0

0

0

0

0

0

0

0

0

0

170 510 680 680 680 680 680 680 680 680 510 170

0

0

0

0

0

0

0

0

0

0

0

0

0

0

170 170 0

0

Gx (b) the gradient values Gx and Gy FIGURE 3.27 Sobel operation of a series of mosaic images.

0

0

0

0

-41 -167 -425 -513 -214 0

44 173 299 340 340 340 340 299 173 44

0

0

0

0

0

Gy

0

0

0

0

0

0

0

0

0

Machine Vision: Visual Sensing and Image Processing

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0 0

0

0

0

0

240 537 537 240

0 0

240 783 898 898 783 240

0

0

0

0

0

0

141

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0 0

0

0

0

0

240 537 537 240

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

240 783 898 898 783 240

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

240 537 680 680 680 680 680 680 680 680 537 240

0

0

240 537 680 680 680 680 680 680 680 680 537 240

0

0

0

0

0

0

0

0

0

0

0

(c)

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

(d)

|G|

0

0

0

240 537 537 240

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

T= 500

0

240 783 898 898 783 240

0

0

0

0

240 537 537 240

240 783 898 898 783 240

240 783 965 556 556 965 783 240

240 783 965 601 243 243 601 965 783 240

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

240 783 965 601 236 57

57 236 601 965 783 240

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

593 995 601 236 57

0

57 236 601 995 593

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

768 895 240 301 340 340 340 340 301 240 895 768

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

593 905 979 102010201020102010201020 979 905 593

0

0

0

240 537 680 680 680 680 680 680 680 680 537 240

0

0

0

0 0

(e)

0

240 537 680 680 680 680 680 680 680 680 537 240 0

0

0

0

0

0

0

T= 800

0

0

0

0

0

(f)

0

0

0

0

0

0

0

0

0

0

0

0

0

0

T= 1000

FIGURE 3.27 Continued.

The magnitude of the gradient at the center pixel ði; jÞ is calculated in the following manner; Gx ¼ ð f13 þ 2f23 þ f33 Þ 2 ð f11 þ 2f21 þ f31 Þ Gy ¼ ð f11 þ 2f12 þ f13 Þ 2 ð f31 þ 2f32 þ f33 Þ

ð3:32Þ

where G values are easily computed by substituting Equation 3.32 into Equation 3.26. Some characteristics observed from this operator are as follows; First, the operator places an emphasis on pixels located closer to the center pixel. Second, due to this operator conﬁguration, it provides both differencing and smoothing effects. As a result, this operator is one of the most popularly used edge operators. We will consider an example of obtaining the edges of an input image composed of mosaics by using the Sobel operator for a simple image similar to the triangle in Figure 3.27. In the ﬁgure the number inside each

142

Optomechatronics

FIGURE 3.28 Sobel and LoG operated images.

pixel represents the image intensity of which gray level is in the range of 0 to 255. By application of the Sobel operator, we wish to obtain its edges when the threshold value of Gðx; yÞ is kept at three different values, 500, 800, and 1000. Utilizing Equation 3.32, Gx ðx; yÞ and Gy ðx; yÞ can be computed and the results are shown in the ﬁgure. Combining these two values yields G½ f ði; jÞ as indicated in the ﬁgure. The result shows that when T ¼ 500, at the edge pixels, the gradient values are the largest, as expected. We can proceed to the same computational procedure using Equation 3.32 and arrive at the results shown in the ﬁgure. It is noted that the threshold value, T, greatly inﬂuences the determination of the edge: At T ¼ 800, the edge shape of the triangle is reserved but, at T ¼ 1000, it is completely lost. The Sobel operation is also carried out on the face image considered previously in Figure 3.20a. Figure 3.28a shows the edgedetected results obtained at T ¼ 60: We can see that the detected edges obtained are composed of thick lines and appear to well represent the original image. The Matlab codes to detect the edges by the Sobel operation are presented in the below (Box 3.3). Laplacian Operator So far, we have discussed the gradient operators that use the ﬁrst derivative of the image intensity. The underlying principle of this method is that the peak of the ﬁrst derivative occurs at the edge. The Laplacian operator, however, uses the notion that, at edge points where the ﬁrst derivative becomes extreme, there will be zero crossing in the second derivative. To illustrate the concept, let us introduce the Laplacian of a 2D function f ðx; yÞ in

Machine Vision: Visual Sensing and Image Processing

143

Box 3.3. EDGE DETECTOR : SOBEL Matlab source: Sobel operation clear all % reading image File image ¼ imread(‘tao.bmp’); threshold ¼ 60; % deﬁned by user [y,x ] ¼ size(image); imgs ¼ double(image);% changing data type % Sobel operation for i ¼ 1:x for j ¼ 1:y if(i ¼ ¼ 1 ll j ¼ ¼ 1 ll i ¼ ¼ x ll j ¼ ¼ y) % boundary image_sobel( j,i) ¼ 255; % white else sx ¼ 2 (imgs( j 2 1,i 2 1) þ 2 p imgs( j,i 2 1) þ imgs( j þ 1,i 2 1)) þ imgs( j 2 1,i þ 1) þ 2 p imgs( j,i þ 1) þ imgs( j þ 1,i þ 1); sy ¼ 2 (imgs( j 2 1,i 2 1) þ 2 p imgs( j 2 1,i) þ imgs( j 2 1,i þ 1)) þ imgs( j þ 1,i 2 1) þ 2 p imgs( j þ 1,i) þ imgs( j þ 1,i þ 1); M ¼ sqrt(sx^2 þ sy^2); if(M . threshold) image_sobel( j,i) ¼ 0; % black else image_sobel( j,i) ¼ 255; % white end end end end % showing image ﬁgure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_sobel);

an image frame, which is a second order derivative. It is deﬁned as 72 f ðx; yÞ ¼

›2 f ›2 f þ ›x2 ›y2

ð3:33Þ

Consider a 1D edge proﬁle shown in Figure 3.29a of which intensity varies along x direction. Its ﬁrst and second derivatives are also shown in the ﬁgure. As can be observed in Figure 3.29b, the pixels having their ﬁrst derivative

144

Optomechatronics

above a threshold shown in a dotted line will be all regarded as the edge. This, however, gives a somewhat rough estimation of the edge. If the second derivative is utilized as shown in Figure 3.29c, only a local maximum of the ﬁrst derivative will be regarded as the edge because it becomes zero in this case. Therefore, all we need to do in the case of the Laplacian operator is to ﬁnd a zero crossing of the second derivative. This implies that ﬁnding the zero crossing position is much easier and more accurate than ﬁnding an extreme value of the ﬁrst derivative. This operator can also be expressed in a digitized numerical form by using different equations in the x and y directions,

›2 f ›Gx ¼ ¼ f ði þ 1; jÞ 2 2f ði; jÞ þ f ði 2 1; jÞ ›x ›x2

ð3:34Þ

›2 f ›Gy ¼ ¼ f ði; j þ 1Þ 2 2f ði; jÞ þ f ði; j 2 1Þ ›y ›y2

Adding these two equations according to the relation given in Equation 3.33 yields the Laplacian operator given in the following form

2=

0

1

0

1

−4

1

0

1

0

ð3:35Þ

∆

f (x, y)

(a)

x peak

f' (x, y)

(b)

a

b

threshold

x

f'' (x, y) zero crossing

(c) FIGURE 3.29 Laplacian operator ﬁnding zero crossing of f 00 (x,y).

x

Machine Vision: Visual Sensing and Image Processing

145

From the Laplacian operated image, we can ﬁnd the edge by ﬁnding the position of zero crossing. It can be described as IF { f ½i; j . 0 AND ðf ½i þ 1; j , 0

OR

f ½i; j þ 1 , 0Þ}

{ f ½i; j , 0 AND ðf ½i þ 1; j . 0

OR

f ½i; j þ 1 . 0Þ}

OR ð3:36Þ

THEN f ½i; j ¼ edge Although the Laplacian operator responds very sharply to variation in the image intensity, it is very sensitive to noise due to its second derivative action. Therefore, the edge detection by ﬁnding the zero crossing of the second derivative of the image intensity may yield erroneous results. To avoid this the Laplacian of Gaussian (LoG) ﬁlter, in which the Gaussian ﬁltering is combined together with the Laplacian operator, is popularly used to ﬁlter out noise before the edge enhancement. If the Gaussian ﬁlter is denoted by 2

2

gði; jÞ ¼ exp2ði þj Þ=2s

2

ð3:37Þ

Convoluting this equation with f ði; jÞ; we have ~ jÞ ¼ ½gði; jÞ p f ði; jÞ fði;

ð3:38Þ

The output of the LoG is then ~ jÞ ¼ 72 ½gði; jÞ p f ði; jÞ fði;

ð3:39Þ

The above notation implies the discrete version of the continuous function as shown in Equation 3.28. The derivative of convolution yields ~ jÞ ¼ ½72 gði; jÞ p f ði; jÞ fði; In the above the LoG is denoted by ! 2 2 2 i2 þ j2 2 2s2 2 7 gði; jÞ ¼ exp2ði þj Þ=2s 4 s

ð3:40Þ

ð3:41Þ

The edge detection using the LoG ﬁlter is summarized in the following three steps: (1) Smoothing operation by a Gaussian ﬁlter (2) Enhancing the edges using Laplacian (3) Detecting zero crossing. The same face image discussed in Figure 3.20a is used here to see the effect of the LoG ﬁltering. Its parameter has a mean value ¼ 0 and s ¼ 2.0.

146

Optomechatronics

The ﬁltered result is shown in Figure 3.28b. The image is slightly different from the image obtained by Sobel in that there are many thin lines appearing in the image. This is because the image produced by the LoG operator is the result of the edge detection together with the thinning operation. The Matlab codes for this ﬁlter are listed in the below (Box 3.4). Box 3.4. EDGE DETECTOR : LAPLACIAN

OF

GAUSSIAN

Matlab source: Laplacian of Gaussian Operation clear all % reading image ﬁle image ¼ imread(‘tao.bmp’); [y,x ] ¼ size(image); % image size % making a log ﬁlter mask N ¼ 13; % ﬁlter size sigma ¼ 2.0; % sigma half_ﬁlter_size ¼ round(N/2); siz ¼ (N 2 1)/2; std2 ¼ sigma^2; [xx,yy ] ¼ meshgrid(2 siz:siz, 2 siz:siz); arg ¼ 2 (xx. p xx þ yy. p yy)/(2 p std2); h ¼ exp(arg); h(h , eps p max(h(:))) ¼ 0; sumh ¼ sum(h(:)); if sumh , ¼ 0, h ¼ h/sumh; end; % now calculate Laplacian h1 ¼ h. p (x. p x þ y. p y 2 2 p std2)/(std2^2); op ¼ h1 2 sum(h1(:))/prod(N); % make the ﬁlter sum to zero op ¼ op 2 sum(op(:))/prod(size(op)); % make the op to % sum to zero imgs ¼ double(image);% changing data type % Laplacian operation for i ¼ 1:x for j ¼ 1:y if(i , half_ﬁlter_size ll j , half_ﬁlter_size ll i . x-half_ﬁlter_size-1 ll j . y-half_ﬁlter_size-1) % boundary imgs2( j,i) ¼ 255; % white else M ¼ 0; for k ¼ 1:N

Machine Vision: Visual Sensing and Image Processing

147

for l ¼ 1:N M ¼ M þ op(k,l) p imgs( j þ k-half_ﬁlter_size, i þ l-half_ﬁlter_size þ 2); end end imgs2( j,i) ¼ M; end end end m ¼ y 2 1; n ¼ x 2 1; % image size rr ¼ 2:m 2 1; cc ¼ 2:n 2 1; % The output edge map: e ¼ repmat(false, m, n); b ¼ repmat(false, m, n); b ¼ imgs2; thresh ¼ 0.15 p mean2(abs(b(rr,cc))); %thresh ¼ 1.0; % Look for the zero crossings: þ 2 , 2 þ and their transposes % We arbitrarily choose the edge to be the negative point [rx,cx ] ¼ ﬁnd(b(rr,cc) , 0 & b(rr,cc þ 1) . 0… &abs(b(rr,cc) 2 b(rr,cc þ 1)) . thresh); % [2 þ ] e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ ﬁnd(b(rr,cc 2 1) . 0 & b(rr,cc) , 0… &abs(b(rr,cc 2 1) 2 b(rr,cc)) . thresh); % [þ 2 ] e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ ﬁnd(b(rr,cc) , 0 & b(rr þ 1,cc) . 0… &abs(b(rr,cc) 2 b(rr þ 1,cc)) . thresh); % [2 þ ]0 e((rx þ 1) þ cx p m) ¼ 1; [rx,cx ] ¼ ﬁnd(b(rr 2 1,cc) . 0 & b(rr,cc) , 0… &abs(b(rr 2 1,cc) 2 b(rr,cc)) . thresh); % [þ 2 ]0 e((rx þ 1) þ cx p m) ¼ 1; % Most likely this covers all of the cases. Just check to see if there % are any points where the LoG was precisely zero: [rz,cz ] ¼ ﬁnd(b(rr,cc) ¼ ¼ 0); if , isempty(rz) % Look for the zero crossings: þ 0 2 , 2 0 þ and their %transposes % The edge lies on the Zero point zero ¼ (rz þ 1) þ cz p m; % Linear index for zero points zz ¼ ﬁnd(b(zero 2 1) , 0 & b(zero þ 1) . 0… &abs(b(zero 2 1) 2 b(zero þ 1)) . 2 p thresh); % [2 0 þ ]0 e(zero(zz)) ¼ 1; zz ¼ ﬁnd(b(zero 2 1) . 0 & b(zero þ 1) , 0… &abs(b(zero 2 1) 2 b(zero þ 1)) . 2 p thresh); % [þ 0 2 ]0

148

Optomechatronics

e(zero(zz)) ¼ 1; zz ¼ ﬁnd(b(zero 2 m) , 0 & b(zero þ m) . 0… &abs(b(zero 2 m) 2 b(zero þ m)) . 2 p thresh); % [2 0 þ ] e(zero(zz)) ¼ 1; zz ¼ ﬁnd(b(zero 2 m) . 0 & b(zero þ m) , 0… &abs(b(zero 2 m) 2 b(zero þ m)) . 2 p thresh); % [þ 0 2 ] e(zero(zz)) ¼ 1; end % normalization image_laplacian ¼ (1 2 e) p 255; % showing image ﬁgure subplot(1,2,1); imshow(image); subplot(1,2,2); imshow(image_laplacian);

Hough Transform The Hough transform, developed by Hough (1962) is an effective way of segmenting objects with known shape and size within an image. It is designed speciﬁcally to ﬁnd lines. An advantage of this approach is the robustness of segmentation and the result of the presence of imperfect data or noise. To introduce the underlying concepts of the transform, let us consider the simple problem of detecting a straight line in an image. Referring to Figure 3.30a, let us suppose that a straight line AB composed of inﬁnitely many points including Pi ¼ ðxi ; yi Þ: Let the general equation of the line passing through the point Pi be yi ¼ mxi þ c On the other hand, inﬁnitely

c

image space Pi+n

(xi+2, yi+2) (xi+1, yi+1) (xi,yi)

Pi

parameter m-c space …

(xi+n, yi+n) y = mx + c c = –mx + y

(a) multiple points on a line FIGURE 3.30 A line deﬁned in two different spaces.

x

c = –Xi+nm + Yi+n

…

y

c'

m'

c = –Xim + Yi c = –Xi+1m + Yi+1 c = –Xi+2m + Yi+2 m

(b) the corresponding points mapped into parameter space

Machine Vision: Visual Sensing and Image Processing

149

many lines will pass through ðxi ; yi Þ as long as it will satisfy the equation yi ¼ mxi þ c

ð3:42Þ

for some values of m and c. This implies that the above equation can also be deﬁned in the parameter space composed of the parameters m and c. For example, all the straight lines going through the point Pi can be represented in the m – c space by c ¼ 2xi m þ yi

ð3:43Þ

Similarly, straight lines going through the point Piþn can be described in the same space by c ¼ 2xiþn m þ yiþn

ð3:44Þ

If those two points were on the same line in the image, the two line equation will be intersected at a point denoted by a dot ðm0 ; c0 Þ as indicated in Figure 3.30b. This indicates that any straight line in the image is mapped to a single point in the m – c parameter space, and any part of this line is mapped into the same point. Lines of any direction in an image may pass through any of edge pixels. In some cases, if we consider the slope and the intercept value of those lines to be bounded within a certain range ðmmax ; mmin Þ and ðcmax ; cmin Þ, the parameter space may be digitized into a subdivision of the space, the so called accumulator array composed of the cells of a rectangular structure as shown in Figure 3.31. These cells clearly depict the relation between the image space (x, y) and the parameter space ðm; cÞ: A(0,L-1)

A(K-2, L-1)

c

A(K-1, L-1) cmax

… A(K-1, L-2)

…

…

… A(0,1) cmin

A(0,0) A(1,0) mmin

FIGURE 3.31 Discretization of parameter space (m, c).

A(K-1,0)

… mmax

m

150

Optomechatronics

A cell Aðm; cÞ has the parameter values m and c in the parameter space. For each image point ðxl ; yl Þ in the image plane, when the lines of the allowed directions pass through this pixel, their corresponding parameters m and c are determined by the equation c ¼ 2xl m þ yl , yielding different Aðm; cÞ values. If a line is present in the image in the form of the equation y ¼ m0 x þ c 0 , the value of the Aðm0 ; c 0 Þ will be increased many times. For instance, if the points composing a line are detected M times, the value in Aðm; cÞ will be M, which indicates that M points in the image plane lie on the line y ¼ m0 x þ c 0 : Therefore, lines existing in the image will produce large values of the appropriate cells. This results in local maxima in the accumulator space. In the following, the Hough algorithm for line detection is summarized: Step 1: Quantize the parameter space (m, c) within the range mmin # m # mmax and cmin # c # cmax : Step 2: Form an accumulator array Aðm; cÞ and initialize it to some value. Step 3: Increment the value of an accumulator array Aðm; cÞ in relation to the points composing lines in the image. It is noted that the accuracy of the colinearity of the points lying on a line is dependent upon the number of discrete cells in the ðm; cÞ plane. If, for every point ðxl ; yl Þ, the m axis is quantized into K number, then the number of the corresponding c values will be K. This requires nK computation for each cell of Aðm; cÞ when n points in the image are involved. It is also noted that the larger the K value, the lower the resolution in the image space. The next task is to select a threshold that serves as a criterion to decide whether the value contained in a speciﬁc block can be a line or not. For this, we then examine the quantized blocks that contain more points than the threshold value. The blocks that pass this criterion are marked as a line in the processed image. The Hough transform method suffers when the line approaches the vertical (as m ! 1 and c ! 1). One way to avoid this difﬁculty is to represent a line as x sin u þ y cos u ¼ r

ð3:45Þ

which is depicted in Figure 3.32a. Construction of the accumulator array, in this case, is identical to the previous method used for slope – intercept representation (m, c). Here, again, a straight line in the image plane is transformed to a single point in the ðr; uÞ plane as shown in Figure 3.32b. Suppose that L collinear points are lying on a line x cos ui þ y sin ui ¼ r . This yields L sinusoidal curves that intersect at ðri ; ui Þ in the parameter space. It is noted here that the range of u lies within þ 908 and 2 908, and ranges from 0 to N where N is the image size, N £ N. When u ¼ 0, a line is horizontal and has a positive r . Similarly, when u ¼ 908, the line is vertical and has a positive r in þ x direction and when u ¼ 2908, r has a negative x-intercept.

Machine Vision: Visual Sensing and Image Processing

ρ

y

151

A(0,L-1)

A(K-2, L-1) A(K-1, L-1) …

rmax

A(K-1, L-2) …

…

θ

…

ρ

x

(a) parameter space

rmin

A(0,1) A(0,0) A(1,0)

…

qmin

qmax

A(K-1,0) θ

(b) accumulation cell

FIGURE 3.32 Hough transform in (r, u) plane.

Hough transform to ðr; uÞ the plane follows the same procedure as discussed in the case of ðm; cÞ plane. The only difference is to use Equation 3.45 in order to determine r and u for each pair of ðx; yÞ in the image. Let us take two examples to understand how the transform actually works on the actual images. One example is made of a synthetic image composed of several polygons and the other is a real image of a camera placed upside down. Figure 3.33 illustrates the ﬁrst example image in which 11 line edges are obtained by an edge operator. To these line images, the Hough transform algorithm is applied and the transformed result is plotted in r – u space. If those lines were perfectly represented by a line, they should be plotted as points in the space. However, the ﬁgure shows several streaks instead of points, since the extracted line edges are not a thin line and contain some noise as a result of edge processing. In the last ﬁgure, the original lines are successfully reproduced by checking the cells in the accumulator that has the local maximum value. Figure 3.34 shows an image of a camera and its frame from which we wish to ﬁnd out some straight lines. It is noted from the extracted edges that the image contains several curved lines as well as straight lines. In addition, there appear several broken lines and noises. These lines and noises are all transformed into Hough r – u space. From this result, the straight lines are reproduced in the last ﬁgure. We can see that only a few lines are found from the transform although there appear many lines in the original input image. This is because the extracted edges contain noises and thick lines, in addition to circular arcs. The main advantage of using Hough transform lies in the fact that it is insensitive to missing parts of lines, image noise, and other parts of the image, exhibiting nonline characteristics. For example, a noisy or rough straight line will not yield a point in the parameter space but be transformed into a cluster of points. In this way, these are discriminated from straight lines in the image.

152

Optomechatronics

(b) extracted edges

(a) original image

50

50

100

100

150

150

200

200

250

250

300

300

350

350 50

100

150

200

250

(c) the result of Hough transformation

300

350

400

50 100 150 200 250 300 350 400

(d) detected lines

FIGURE 3.33 A Hough transformation of the image composed of polygon objects.

Camera Calibration In the pinhole camera model, a small hole is punched at the optical lens center in the camera coordinate system, through which some of the rays of light reﬂected by the object pass to form an inverted image of the object on the image plane. This is a camera perspective model as shown in Figure 3.35a, as discussed at the beginning of this chapter. Here, the camera lens is drawn in a dotted line to neglect its presence. A pinhole does not focus and, in fact, limits the entrance of incoming light, requiring long exposure time. No focusing required means that all objects are in focus, which, in turn, implies that the depth of ﬁeld of the camera is ideally unlimited. However, the pinhole to ﬁlm (image plane) distance affects the sharpness of the image and the ﬁeld of view. In actual cameras, the pinhole is sufﬁciently opened by using a converging lens to avoid the disadvantage of the pinhole model as shown in Figure 3.35b. However, the actual lens

Machine Vision: Visual Sensing and Image Processing

153

(b) extracted edges

(a) original image

50

50

100

100

150 150

200 250

200

300

250

350

50

100

150

200

250

300

(c) the result of Hough transformation

350

300

50

100

150

(d) detected lines

FIGURE 3.34 A Hough transformation of a real object image.

model causes focusing and aberration problems in reality. A difference from the previous model is that, in this model, the lens focuses and projects light rays from an object into the image plane, which is located in a speciﬁed distance from the camera lens. Disregarding aberration problem, let us consider the focus variation of the image plane. If an object is located at z ¼ s from the lens as shown in the ﬁgure, the following Gaussian lens formula holds 1 1 1 ¼ 0 þ f f s

ð3:46Þ

where f and f 0 are the focal length of the lens and the distance of the image plane from the center point of the lens, respectively. This f is called the effective focal length of the camera or camera constant. As we can see, when

154

Optomechatronics

virtual image object optical center

image sensor

zc real image real image

(a) pin hole model

optical lens

world point P

image plane zc image point P’

f

s

f’

(b) lens model FIGURE 3.35 The perspective camera model.

the object distance, s, varies from inﬁnity to some distance close to the lens, f 0 slightly deviates from the focal length, f : This deviation may cause an error of projection, which cannot be tolerated in the case of the accurate depth measurement at close range. In this case, this parameter must be accurately determined by a camera calibration technique. To analyze the imaging process with a real image rather than the inverted one, we will deal with a virtual image plane denoted in Figure 3.35a. For simplicity, we call this “image plane” in the sequel. The objective of the image analysis is then how we map the image of an object acquired in the image plane into the world coordinate system. Referring to Appendix A1, the mapping conveniently can be described in the following form. up ¼ ½Mp Xw

ð3:47Þ

where up is the image vector of the object described in the image pixel coordinates and is projected from the 3D world coordinates Xw, and Mp is the resulting projection matrix. This is obtained by a perspective model discussed at the beginning of this chapter.

Machine Vision: Visual Sensing and Image Processing

155

Perspective Projection A perspective transformation called the imaging transformation projects 3D points on to a plane. In other words, the transformation changes a 3D aggregate of the objects into a plane surface. To describe this in detail, we need to deﬁne four coordinate systems; With reference to Figure 3.36, the ﬁrst one is the world coordinate system, {Xw } where an object to be imaged in 3D scene is located, the second one is the camera coordinate system {Xc } of which origin is located at the optical lens center, the third is the pixel coordinate system, {up } denoted by the coordinates of the pixels in the digital image, and ﬁnally, the fourth is the image coordinate system {ui }; which describes the same image in a different manner from the one described in the pixel coordinate. As shown in Figure 3.37, the image coordinates ðui ; vi Þ are located at the center of the pixel coordinate system ðu0 ; v0 Þ: As shown in the ﬁgure, the relationship between the pixel coordinate and image coordinate systems is, therefore, given by up ¼ u0 þ ku ui

vp ¼ v0 þ k v vi

ð3:48Þ

where ku and kv are the inverse of the horizontal and the vertical effective pixel sizes, su and sv, respectively, ku and kv are expressed in units of pixel £ m 21 while su and sv are interpreted as the size in meters for the horizontal and vertical pixels, respectively. Utilizing these coordinate systems, the whole imaging process can be described in the following sequence: coordinate system : coordinate points :

{Xw } 2 3 Xw 6 7 6 Yw 7 4 5

!

{Xc } 2 3 Xc 6 7 6 Yc 7 4 5

!

Zw

Yc

Yw

Xw

Zw Ow world coordinates

{ui } "

!

Zc

Xc

Oc camera coordinate

!

oi

op

up

FIGURE 3.36 The coordinate system for image analysis.

vi

#

{up } "

!

up

#

ð3:49Þ

vp

object (Xc,Yc, Zc) world point: P

(ui,vi)

vi

vp

ui

!

ui

optical axis

virtual image plane

Zc

156

Optomechatronics vi

(uo,vo) o image coordinates

vp

pixel coordinates

ui

up

ku : 1/ su kv : 1/ sv su : horizontal effective pixel size

pixel

sv

su

sv : vertical effective pixel size FIGURE 3.37 Pixel and image coordinates.

The ﬁrst step to obtain the image interpretation is to obtain a relationship when a point ðXc ; Yc ; Zc Þ in the camera coordinate systems is projected onto the image coordinates ðui ; vi Þ: As indicated in Figure 3.36, under the assumption of the ideal projection, the transformation can be described by the projection equation ui ¼

f Xc ; Zc

vi ¼

f Yc Zc

ð3:50Þ

where f is the focal length of the camera. Note that these equations are nonlinear, since the depth variable Zc is at the denominator of each equation. These relationships can be utilized directly to avoid any complexity that arises from the nonlinearity. We may use alternatively homogeneous coordinates, which enable us to deal them in linear matrix from. As discussed in Appendix A1, the homogeneous coordinates of a point with respect to camera coordinates ðXc ; Yc ; Zc Þ are deﬁned as ðlXc ; lYc ; lZc ; lÞ, where l is a nonzero constant. Conversion of homogeneous coordinates back to Cartesian coordinates can be made by dividing the ﬁrst three homogeneous coordinates by l. Similarly, we can deﬁne the homogeneous coordinates of a point with the pixel coordinates ðup ; vp Þ by ðsui ; svi ; sÞ, where s is a nonzero constant. Then, the relationship in Equation 3.50 called

Machine Vision: Visual Sensing and Image Processing

157

perspective matrix can be conveniently expressed in linear form, 2 3 3 Xc 3 2 2 7 f 0 0 0 6 sui 6 7 76 Yc 7 7 6 6 6 svi 7 ¼ 6 0 f 0 0 76 7 ~c or u~ i ¼ ½Hc X 56 7 5 4 4 6 Zc 7 0 0 1 0 4 5 s 1

ð3:51Þ

~ c in the where u˜i and X˜c are the augmented coordinate vectors of u~ i and X homogenous coordinate space. Using the rigid body transformation, Xc can be related to the world coordinates by the equation 2

lXc

3

2

r11

r12

r13

r22

r23

TX

32

XW

3

2

r T1

76 7 6 6 7 6 TY 7 76 YW 7 6 r T2 76 7¼6 76 7 6 6 ZW 7 6 r T3 r32 r33 TZ 7 54 5 4 0 0 1 1 0T " # .. ~ c ¼ ½Hw X ~ w ¼ ·R· · · .· · ·T· · X ~w X . 0 .. 1 6 7 6 6 lY 7 6 r 6 c 7 6 21 6 7¼6 6 7 6 6 lZc 7 6 r31 4 5 4 0 l

TX

32

XW

3

76 7 76 7 TY 76 YW 7 76 7 76 7 6 ZW 7 TZ 7 54 5 1 1

or ð3:52Þ

where R denotes the 3D rotation matrix, T denotes the 3D translational vector, and ri (i ¼ 1,2,3) is a row vector. Finally, from the relationship given in Equation 3.52, the relationship between the image and the pixel coordinates in the homogeneous coordinate space, 32 3 3 2 2 ui ku 0 u 0 sup 76 7 7 6 6 6 svp 7 ¼ 6 0 kv v0 76 vi 7 or u~ p ¼ ½Hu u~ i ð3:53Þ 54 5 5 4 4 s

0

0

1

1

Therefore, the overall imaging process is described by the relationship, using Equation 3.50 through Equation 3.53. 32 3 2 T XW 2 32 3 r 1 TX 7 76 ku 0 u0 f 0 0 0 6 76 7 6 6 76 76 r T2 TY 76 YW 7 7 6 7 7 6 7 6 u~ p ¼ 6 0 k v 0 f 0 0 or v 0 54 4 56 T 7 76 ð3:54Þ 6 r 3 TZ 76 ZW 7 54 5 0 0 1 0 0 1 0 4 1 0T 1 ~W u~ p ¼ ½H X where [H] is the 3 £ 4 camera projection matrix, which is denoted by 3 2 au 0 u0 7 6 7 ð3:55Þ H ¼ ½C ½R ½T ¼ 6 4 0 av v0 5½R ½T 0

0

1

158

Optomechatronics

where au and av are the image scaling factors, given by au ¼ ku f and av ¼ kv f ; and aspect ratio is given by au =av ¼ ku =kv : The above relationship indicates that au and av are dependent upon the distance of the focal length and the sizes of the pixels in the horizontal and vertical directions, respectively. It is noted that the parameters au ; av ; u0 ; and v0 do not depend on the position and orientation of the camera, and are, therefore, called the intrinsic parameters. The above equation can be rewritten by 2

H11

H14

3

6 u~ p ¼ 6 4 H21

H12

H13

H22

H23

7 ~ H24 7 5Xw

H31

H32

H33

H34

ð3:56Þ

When the above 12 elements are scaled by the value H34 , we have 11 variables to determine. The resulting equation is obtained by 3

2

2

up 7 6 0 6 7 6 s0 6 4 vp 5 ¼ 4 H21

H012

H013

H022

H023

H031

H032

H033

1

H011

3 2 3 Xw 7 6 6Y 7 7 6 7 w 7 6 H024 7 7 56 6 Zw 7 5 4 1 1

H014

ð3:57Þ

where s0 is given by s=H34 : This yields the camera calibration equation from which the relationship between the pixel coordinates and the world coordinates can be established. To illustrate the relationship in more detail, let us consider a point, say ith point, in the world coordinates {X} and the corresponding point imaged in the pixel coordinates {up }: Use of Equation 3.57 leads to 3 H011 7 6 6 0 7 6 H12 7 7 6 7 6 6 0 7 6 H13 7 7 6 7 6 6 0 7 6 H14 7 7 6 7 6 6 0 36 H21 7 7 2 3 i i 7 6 2up Zw 6 u 7 76 0 7 4 p 5 56 H22 7 ¼ 7 6 vp 7 2vip Ziw 6 6 0 7 6 H23 7 7 6 7 6 6 0 7 6 H24 7 7 6 7 6 6 H0 7 6 31 7 7 6 7 6 6 H0 7 6 32 7 5 4 2

2 6 4

i Xw Yiw Ziw 1

0

0

0

0

0

0

i 0 2uip Xw 2uip Yiw

i i 0 Xw Yiw Ziw 1 2vip Xw 2vip Yiw

H033

ð3:58Þ

Machine Vision: Visual Sensing and Image Processing

159

In deriving the above equation, we have used the following relationship; s0 ¼ H031 Xw þ H032 Yw þ H033 Zw þ 1 which can be obtained from Equation 3.57. It needs to be pointed out that, at least, we need 12 equations like these that can be obtained by six calibration points. The equation can be written in a more compact form, Aq ¼ b

ð3:59Þ

where A represents the left-hand side matrix of the H vector and b represents the right-hand side of the equation. Solving for q yields q ¼ ½AT A

21

AT b ¼ Aþ b

ð3:60Þ

where Aþ is the pseudo inverse of A. Here, we have 11 unknowns for qs to be determined and, therefore, need at least six calibration points. Before we proceed to solve Equation 3.60, let us consider the transformation from a point in the image coordinates Pi to a point in world coordinates without a camera perspective model. Referring to Figure 3.38, this can be obtained by considering the absolute position vector of point Pw ðxw ; yw ; zw Þ with respect to the world coordinates, which is equivalent to point Pi ðui ; vi Þ: The position vector is expressed by rw ¼ rc þ ri þ rp

ð3:61Þ

where rw is the absolute position vector of point Pw expressed with respect to the world coordinates, rc is the position vector of the center of the camera with respect to the center of the world, ri is the position vector of the image coordinates with respect to the camera center Oc, and ﬁnally rp is the position vector of Pi with respect to the center of the image coordinates, Oi : The camera shown in the ﬁgure has usually two rotational motions denoted by the angles a and b, which are called, respectively, tilt and panning. The sequence of positioning point Pi in the image coordinates to the corresponding point Pw in the world coordinates is to obtain the camera transformation matrix in Equation 3.54. The ﬁrst translation gives 2

1

6 60 6 H1 ¼ 6 6 60 4 0

0 0 1 0 0 1 0 0

xw

3

7 yw 7 7 7 7 zw 7 5 1

ð3:62Þ

160

Optomechatronics Zc β

image coordinates {I}

pan-tilt device Xc α camera coordinates {C}

rp

ri Oi

Oc

Yc

Pi (ui ,vi ,0)

u

v

z optical axis

rc Zw

Ow Yw

Pw(xw, yw, zw)

Xw

world coordinates {W }

FIGURE 3.38 Geometry for camera coordinate transformation.

where xw, yw and zw are the positioning vectors of the center of the camera coordinates. The two rotations, tilting and panning, yield the following transformation: For panning (b) 2

cos b

6 6 sin b 6 H2 ¼ 6 6 6 0 4 0

2sin b

0

cos b

0

0

1

0

0

0

3

7 07 7 7 7 07 5 1

ð3:63Þ

Machine Vision: Visual Sensing and Image Processing

For tilting (a) 2 1 0 6 6 0 cos a 6 H3 ¼ 6 6 6 0 sin a 4 0

0

0 2sin a cos a 0

0

161

3

7 07 7 7 7 07 5 1

ð3:64Þ

The translation from the origin of the camera coordinates with respect to that of the image coordinate is expressed by 3 2 1 0 0 xi 7 6 60 1 0 y 7 6 i7 7 ð3:65Þ H4 ¼ 6 7 6 6 0 0 1 zi 7 5 4 0 0 0 1 where ðxi ; yi ; zi Þ is the position vector of the image coordinate center. As can be seen from the ﬁgure, the orientation of the image coordinates is different from that of the camera coordinates. Therefore, the frame needs to be rotated 908 about Zc axis, which yields 2 3 0 21 0 0 6 7 61 0 0 07 6 7 7 H5 ¼ 6 ð3:66Þ 6 7 60 0 1 07 4 5 0 0 0 1 Then, another rotation about YC yields 2 3 0 0 1 0 6 7 6 0 1 0 07 6 7 7 H6 ¼ 6 6 7 6 21 0 0 0 7 4 5 0 0 0 1

ð3:67Þ

If i rp is the position vector of Pi , the transformation of the Pi into the Pw is obtained by w

rp ¼ ½H1 ½H2 ½H3 ½H4 ½H5 ½H6 i rp

ð3:68Þ

This yields the relationship between Pw ðxw ; yw ; zw Þ in the world coordinates and Pi ðui ; vi ; 0Þ in the image plane. We have seen that a point deﬁned in the world coordinates (known) can be mapped into a point deﬁned in the image coordinates (measured) from Equation 3.68. To obtain the transformation matrix H, we need to solve Equation 3.58 or Hij s. However, H contains the intrinsic and the extrinsic parameters, which are

162

Optomechatronics

difﬁcult to obtain by a direct measurement. Furthermore, at least six points located in the world coordinates and the corresponding points in the image coordinates need to be exactly known. This necessitates a measurement technique called “camera calibration.” The calibration method requires geometrically known points called calibration marks. The choice of the marks depends upon the conditions that the camera calibration needs to be carried out, for example, the light conditions, the coloring of the mark and the accuracy of the calibration. The choice of the marks depends upon the conditions under which the camera calibration condition is carried out. Such conditions include the light conditions, the coloring of the mark, and the accuracy of the calibration. Figure 3.39 shows an actual geometrical conﬁguration set for the camera calibration. The calibration rig is composed of an object to be calibrated, a CCD camera, and an optical rail on which the camera can travel within a certain distance. The object contains an evenly spaced pattern of circles ﬁlled in white and is located in a known location along the Z axis from the camera position of which center is at Oc The ﬁrst step is to obtain the image of the pattern and calculate the locations of the center of the circle given in the pixel coordinates, based on the acquired image. Figure 3.40 shows two acquired images taken by the camera at two different locations at Zw ¼ 0 mm and Zw ¼ 250 mm. In this procedure, any information on the circle such as the features of the circle must be given for the calculation. Here, the centroids of the circles are used. As indicated in Figure 3.41, to determine this, we need to use the entire image processing procedures discussed in the previous section, selection of region of interest (ROI), image segmentation, and enhancement, detection, and determination of the mass center of each circle. Table 3.3 indicates the coordinate values of the centroids of the circles with respect to the world coordinate system.

Yw

Xw

Xc

camera

ical

optical center, Oc

opt Yc

FIGURE 3.39 Camera calibration set-up.

Zw

Zc

rail

Machine Vision: Visual Sensing and Image Processing

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

(a) image 1 ( Zw=0mm, “circle_0.raw ”)

163

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

(b) image 2 ( Zw=250mm, “circle_250.raw”)

FIGURE 3.40 The pattern images acquired at two different locations.

Utilizing the locations of the circles obtained in the pixel coordinate values and the world coordinate values, we wish to compute the intrinsic parameters and the calibration transformation matrix given in Equation 3.54. Plugging these data into Equation 3.54, we obtain the following calibration

input image

selection of ROI (region of interest)

segmentation by labeling

edge detection

mass center using edge of each circle

FIGURE 3.41 The procedure to determine the centers of the circle patterns.

164

Optomechatronics

TABLE 3.3 The Centroid of the Circles with respect to World Coordinate System Feature Point

X (mm)

Y (mm)

1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 1-10 1-11 1-12 1-13 1-14 1-15 1-16 1-17 1-18 1-19 1-20 1-21

2120 280 240 0 40 80 120 2120 280 240 0 40 80 120 2120 280 240 0 40 80 120

200 200 200 200 200 200 200 160 160 160 160 160 160 160 120 120 120 120 120 120 120

2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 2-10

280 240 0 40 80 280 240 0 40 80

160 160 160 160 160 120 120 120 120 120

matrix H. 2 0 H11 6 6 0 H¼6 6 H21 4 H031

H012

H013

H022

H023

H032

H033

H014

Z (mm) Feature Point Image 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Image 2 250 250 250 250 250 250 250 250 250 250

3

2

1:8813

7 6 7 6 0:0021 H024 7 7¼6 5 4 0 1

X (mm)

Y (mm)

1-22 1-23 1-24 1-25 1-26 1-27 1-28 1-29 1-30 1-31 1-32 1-33 1-34 1-35 1-36 1-37 1-38 1-39 1-40 1-41 1-42

2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120

80 80 80 80 80 80 80 40 40 40 40 40 40 40 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

2-11 2-12 2-13 2-14 2-15 2-16 2-17 2-18 2-19 2-20

2 80 2 40 0 40 80 2 80 2 40 0 40 80

80 80 80 80 80 40 40 40 40 40

250 250 250 250 250 250 250 250 250 250

0:0054

20:3737

21:8833 20:2707 0

20:0012

Z (mm)

321:0669

3

7 7 427:9406 7 5 1 ð3:69Þ

Machine Vision: Visual Sensing and Image Processing

165

Examination of Equation 3.57 indicates that this result enables us to relate a point Pp ðup ; vp Þ in the pixel coordinates to a point Pw ðxiw ; yiw ; ziw Þ in the world coordinates.

Problems P3.1. Figure P3.1 shows an image having a certain gray-level histogram. If we wish to modify the original histogram such that it is stretched over a certain range and that it slides with some offset from the original histogram, as shown in the ﬁgure, explain what kind of changes in the image will occur for both cases.

(a) stretching

(b) sliding FIGURE P3.1 Histogram modiﬁcation.

P3.2. Finding the connected components in an image is common practice in machine vision. A labeling algorithm ﬁnds all connected components in the image and assigns a label to all points in the same component. For an image shown in Figure P3.2, explain how the labeling algorithm can work to segment each individual component. P3.3. Binary image varies greatly depending on which threshold value is used. Figure P3.3 shows an image of a tree leaf. Obtain the binary images at T1 ¼ 150 and T2 ¼ 180 and explain the results by comparing the obtained binary images.

166

Optomechatronics

255 255 255 255 255 255 255 255 255 255 255 255 255 255 25 25 255 255 255 255 255 255 255 25 25 25 25 255 255 255 255 255 25 25 25 25 25 25 255 255 255 255 25 25 25 25 25 25 255 255 255 255 255 25 25 25 25 255 255 255 255 255 255 255 25 25 255 255 255 255 255 255 255 255 255 255 255 255 255 255 FIGURE P3.2 A mosaic image.

FIGURE P3.3 Image of a tree leaf.

P3.4. In Figure P3.4(a) and (b), two simple series of mosaics of 1s and 0s mimicking gray level image are given in an 8 £ 10 window. We wish to determine their edge using Robert and Laplacian operators. (1) Obtain the gradient vector Gx and Gy of the image (a), using a Robert operator and ﬁnd the edge, using T ¼ 220: (2) Obtain the Laplacian operation of the image (b) and ﬁnd the zerocrossing line from the result.

Machine Vision: Visual Sensing and Image Processing

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 25 25 25 25 255 255 255

167

20 20 20 20 20 80 80 80 80 80 20 20 20 20 20 80 80 80 80 80

255 255 255 25 10 10 25 255 255 255

20 20 20 20 20 80 80 80 80 80

255 255 255 25 10 10 25 255 255 255

20 20 20 20 20 80 80 80 80 80

255 255 255 25 25 25 25 255 255 255

20 20 20 20 20 20 20 20 20 20

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

(a)

20 20 20 20 20 20 20 20 20 20

(b)

FIGURE P3.4 A series of mosaic images.

P3.5. The image of an array resistance is shown in Figure P3.5. Use the Sobel operator to detect the edge at three different arbitrary values of threshold.

FIGURE P3.5 Image of an array resistance.

P3.6. Figure P3.6 contains two different noise levels. Obtain the ﬁltered images of the images using a median ﬁlter with mean ¼ 0.0 and standard deviation ¼ 2.0. P3.7. A line in the x-y coordinate frame is shown in Figure P3.7. Let points P1 and P2 be located on the line. (1) Explain why the polar coordinate representation of lines, r ¼ x cos u þ y sin u, is more suitable for line detection using the Hough transform than the standard line representation, y ¼ mx þ c: (2) Explain how the standard line representation y ¼ mx þ c can be converted to the polar coordinate representation, r ¼ x cos u þ y sin u:

168

Optomechatronics

FIGURE P3.6 Images of a pair of cutters with two different noise levels.

y

P2 P1

(−1,1+ 3) (− 2,1) O

x

FIGURE P3.7 A line in x-y coordinate frame.

(3) For the line shown in the ﬁgure, show the standard line representation and the polar coordinate representation of the line. P3.8. Let an image plane be with 300 £ 400 pixel size as shown in Figure P3.8. For extracting the line information from the image acquired in this image plane, a Hough transform is generally used. Here, we would like to get the speciﬁed resolution of the extracted line, and Dr ¼ 1 pixel. (1) Design the accumulator array AðuÞ: Assume that the image quantization effect is negligible. Determine m and c values where m and c are the horizontal and vertical sizes of the accumulator, respectively. (2) On this image plane, a line denoted by AB is acquired as shown in Figure P3.8. Which cell of the accumulator is corresponding to the line?

Machine Vision: Visual Sensing and Image Processing

169 q = ( 400, 300 )

vp

lA ( 100, 200 )

( 300, 100 ) B up FIGURE P3.8 A line in image coordinate frame.

(3) When this line is reconstructed from the corresponding accumulator cell, how much is the parametric error of the line equation? P3.9. A camera is located relatively to the world coordinate frame {W}, as shown in Figure P3.9. We wish to map a point in world coordinates on to the corresponding points in the image coordinates frame. (1) The transformation between the world coordinates and the camera coordinates can be expressed by using the euler angle (f,u and c)

Yc Xc Zc {C} Zw T = [a, b, c]

f = 90° q = 90°

Yw Xw

{W}

FIGURE P3.9 A conﬁguration of camera and world coordinate frames.

y = 90° T= [0, –100, –100]

170

Optomechatronics

and translation ða; b; cÞ and is given by 2

Xc

3

6 7 6Y 7 6 c7¼ 6 7 6 7 6 Zc 7 4 5 1

c

2 6 6 6 6 6 6 4

Tw

R … 0

32

Xw

3

76 7 .. Yw 7 . T7 76 6 7 76 7 76 .. 7 6 Z . …7 54 w 7 5 .. . 1 1

where c Tw is given by 2

cos f cos u cos c 2 sin f sin c 2cos f cos u sin c 2 sin f cos c cos f sin u a

3

7 6 6 sin f cos u cos c þ cos f sin c 2sin f cos u sin c þ cos f cos c sin f sin u b 7 5 4 2sin u cos c

sin u sin c

cos u

c

Let the intrinsic parameters of the camera be given by f ¼ 16 mm, ku ¼ kv ¼ 50 mm21, u0 ¼ 320, v0 ¼ 240. (1) Find the transformation matrix H in Equation 3.52. (2) If a point Pw is located at Xw ¼ ½200; 600; 200 in the world coordinates, what is the corresponding up in the pixel coordinates? ~w P3.10. Consider a perspective transformation having u~ p ¼ HX where 2

h11

h14

3

6 H¼6 4 h21

h12

h13

h22

h23

7 h24 7 5

h31

h32

h33

h34

and let q1 ¼ h11 h12 q4 ¼ h14 h24 h34 :

h13

q2 ¼ h21

h22

h23

q3 ¼ h31

h32

h33

(1) Show that u0 and v0 can be expressed as u0 ¼ q1 qT3 ; v0 ¼ q2 qT3 : qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ (2) Show that au ¼ q1 qT1 2 u20 av ¼ q2 qT2 2 v20 : P3.11. Consider the camera calibration conﬁguration presented in the section “Camera Calibration.” A pattern image is obtained as shown in Figure P3.11.

Machine Vision: Visual Sensing and Image Processing

171

FIGURE P3.11 A pattern image obtained a Zw ¼ 150 mm.

(1) Determine the coordinate values of the centroids of the circles from this image. (2) Compare this result with the one obtained by the camera calibration matrix. The absolute coordinate values of the circles are shown in Table P3.1. Discuss the error caused by calibration.

TABLE P3.1 World Coordinate Values of the Centers of the Circle Patterns Feature Point

X (mm)

Y (mm)

Z (mm)

Feature Point

X (mm)

Y (mm)

Z (mm)

2 120 2 80 2 40 0 40 80 120 2 120 2 80 2 40 0 40 80 120

80 80 80 80 80 80 80 40 40 40 40 40 40 40

150 150 150 150 150 150 150 150 150 150 150 150 150 150

Image 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 3-13 3-14

2120 280 240 0 40 80 120 2120 280 240 0 40 80 120

160 160 160 160 160 160 160 120 120 120 120 120 120 120

150 150 150 150 150 150 150 150 150 150 150 150 150 150

3-15 3-16 3-17 3-18 3-19 3-20 3-21 3-22 3-23 3-24 3-25 3-26 3-27 3-28

172

Optomechatronics

References [1] Ballard, D.H. and Brown, C.M. Computer Vision, Prentice Hall, Englewood Cliffs, NJ, 1982. [2] Bishop, R.H. The Mechatronics Handbook, CRC Press, Boca Raton, 2002. [3] Corke, P.I. Visual Control of Robots, Research Studies Press Ltd., Taunton, UK, 1996. [4] Datatranslation PCI Frame Grabber Product Catalog, Datatranslation Co. Ltd., http://www.datatranslation.com/top_products_pages/imaging-PCI.htm, 2005. [5] Fu, K.S., Gonzalez, R.C. and Lee, C.S.G. Robotics Control, Sensing, Vision, and Intelligence, McGraw Hill lnc., New York, 1987. [6] Gonzalez, R.C. and Woods, R.E. Digital Image Processing, Addison Wesley Publishing Co., Reading, MA, 1992. [7] Gonzalez, R.C., Woods, R.E. and Eddins, S.L. Digital Image Processing Using MATLAB, Prentice hall, Englewood Cliffs, NJ, 2004. [8] Horn, B.K.P. Robot Vision, The MIT Press, McGraw-Hill Book Co., 1986. [9] Jahne, B., Haupecker, H. and Geipler, P. Handbook of Computer Vision and Applications, System and Applications, Vol. 3. Academic Press, New York, 1999. [10] James, R. and Carstens, P.E. Automatic Control Systems and Components, Prentice Hall, Englewood Cliffs, NJ, 1990. [11] Jain, R., Kasturi, R. and Schunck, B.G. Machine Vision, McGraw-Hill, New York, 1995. [12] Metero Frame Grabber Product Catalog, Matrox Co. Ltd., http://www.matrox. com/imaging/products/frame_grabbers.cfm, 2005. [13] Petrou, M. and Bosdogianni, P. Image Processing: The Fundamentals, Wiley, New York, 1999. [14] Ridler, T.W. and Calvard, S. Picture thresholding using an iterative selection method, IEEE Transactions SMC, 8:8, 630–632, 1978. [15] Stadler, W. Analytical Robotics and Mechatronics, McGraw-Hill, New York, 1995. [16] Umbaugh, S.E. Computer Vision and Image Processing, Prentice Hall PTR, Upper Saddle, NJ, 1998. [17] Watson, D.M. Lab Manual, Astronomy 111, The University of Rochester, Rochester, NY, 2000. [18] Xie, M. Fundamentals of Robotics, World Scientiﬁc Publishing Co. Pte. Ltd., Singapore, 2003.

4 Mechatronic Elements for Optomechatronic Interface CONTENTS Sensors ................................................................................................................. 175 Capacitive Sensor....................................................................................... 176 Differential Transformer............................................................................ 177 Piezoelectric Sensors.................................................................................. 179 Pyroelectric Sensor..................................................................................... 185 Semiconductor Sensors; Light Detectors................................................ 186 Photodiode .................................................................................................. 189 Other Photodetectors................................................................................. 191 Photovoltaic Detectors ....................................................................... 192 Avalanche Photodiode....................................................................... 192 Signal Conditioning........................................................................................... 193 Operational Ampliﬁers ............................................................................. 193 Inverting Ampliﬁer ............................................................................ 193 Noninverting Ampliﬁer .................................................................... 194 Inverting Summing Ampliﬁer.......................................................... 195 Integrating Ampliﬁer......................................................................... 195 Differential Ampliﬁer......................................................................... 196 Comparator ......................................................................................... 196 Signal Processing Elements ...................................................................... 197 Filters .................................................................................................... 197 Digital-to-Analog Conversion .......................................................... 198 Analog-to-Digital Converters ........................................................... 199 Sample and Hold Module................................................................. 200 Multiplexer .......................................................................................... 201 Time Division Multiplexing.............................................................. 201 Wheatstone Bridge ............................................................................. 202 Isolator.................................................................................................. 203 Microcomputer System ............................................................................. 204 Microcomputer ................................................................................... 204 Input/Output Interface ..................................................................... 207

173

174

Optomechatronics

Microcontrollers.................................................................................. 207 Sampling of a Signal.................................................................................. 207 Actuators ............................................................................................................. 210 Electric Motors............................................................................................ 210 Piezoelectric Actuator ............................................................................... 212 Voice Coil Motor (VCM) ........................................................................... 216 Electrostatic Actuator ................................................................................ 218 Microactuators.................................................................................................... 220 Shape Memory Alloy (SMA) Actuator................................................... 221 Magnetostrictive Actuator ........................................................................ 222 Ionic Polymer Metal Composite (IPMC) Actuator............................... 223 Signal Display..................................................................................................... 224 Dynamic Systems and Control ........................................................................ 225 Dynamic Systems Modeling .................................................................... 226 Thermal Systems ................................................................................ 227 Spring-Mass-Damper System ........................................................... 227 Fluid System........................................................................................ 228 Optical Disk......................................................................................... 229 Dynamic Response of Dynamical Systems............................................ 230 System Transfer Function ......................................................................... 233 First Order System ............................................................................. 234 Second Order System......................................................................... 235 Higher Order System......................................................................... 236 Laplace Transforms Theorems ......................................................... 236 Open Loop vs. Feedback Control .................................................... 237 System Performance .................................................................................. 238 Basic Control Actions ................................................................................ 241 System Stability .......................................................................................... 244 Problems.............................................................................................................. 245 References ........................................................................................................... 252

As mentioned in the Introduction, mechatronics is an integrated discipline in which optical, mechanical, electrical, and computer technologies are embedded together. Therefore, this discipline encompasses a variety of technical ﬁelds ranging from mechanism design, sensor and measurement, signal conditioning and processing, drive and actuator, system control, microprocessor, and so on, as shown in Figure 1.11. These are considered to be the key areas of mechatronics. In this section, we will deal with some detailed fundamental methodologies involved with the technical ﬁelds, but, in particular, those that may be effectively combined with optical elements for optomechatronic integration. In the ﬁrst part, sensors and actuators are introduced. In recent years, a great deal of new materials and transduction methods have been developed. In addition, an abundance of micro sensors and actuators have

Mechatronic Elements for Optomechatronic Interface

175

appeared in recent years owing to development of microfabrication methods; these sensors and actuators are expected to grow at a very fast rate due to down-sizing of macroparts and wide applications of MEMS parts. We will discuss here some of these micro sensors and actuators in brief; the majority of these micro sensors and actuators here means the sensors and actuators that operate on orders of mm range or less, regardless of their physical size. Most sensors to be dealt with have a small range of measurement but high resolution such as capacitive and piezoelectric sensors. Other sensors based on semiconductor technology such as light detecting, piezoresistive are also treated because of the variety of their use in optomechatronic integration. Actuators to be discussed here cover a variety of actuating principles including piezoelectric, capacitive, electromagnetic, material phase transformation, magnetostrictive, and so on. Most of these actuators have a relatively small range of actuation. We will also discuss DC motor which is a rotary actuator and has been used in a variety of optomechatronic systems. We shall make use of some of these actuators in order to produce optomechatronic integration in the later part of this book. In the second part, we will discuss the components for signal conditioning such as operational ampliﬁers, ﬁlters, comparators, multiplexer, microcomputer, and then signal sampling via analog to digital (A/D) converter, sending-out output signal from microcomputer via digital to analog (D/A) converter. In the last part, we will ﬁnally discuss some of elementary concepts of system modeling, transfer function, system response, and basic feedback controllers.

Sensors Sensing is the most fundamental technique that senses the physical variables being measured. The sensor is a physical element that does this, and contains one or more transducers within it. The transducer converts or transforms one form of energy to another form in a detectable signal form: The energy includes various types of form-mechanical, electrical, optical, chemical, and so forth. In view of this, the transducer is an essential element of a sensor which needs to be contained within it. The transformed raw signal usually contains low amplitude, noise contaminated, narrow sensing range, nonlinearity, and so on. To make these raw signal forms conditioned into desirable forms, signal conditioning units are usually necessary. Some sensors contain these conditioning units as a part of their body but some are equipped with these units outside. The recent trend, however, due to downsizing of products, is that most of the necessary elements are put together within a single body. Semiconductor sensors belong to one of the groups that lead this trend. This trend can also be found from intelligent sensors in which dissimilar multiple sensors are

176

Optomechatronics

embedded together to measure a variety of different physical quantities. In this section, focusing piezoelectric sensors and light detecting sensors, we will discuss fundamental concepts of several sensors that are frequently used for optomechatronic systems. Capacitive Sensor When two isolated conductive objects or plates are connected to the positive poles of a battery, the plates will receive equal amounts of opposite charge. If the plates are disconnected from the battery, they will remain charged as long as they are in a vacuum. A pair of these plates called a capacitor has a capability of holding electrical charge whose capacitance depends on the magnitude of charge, q, and the potential difference between the plates, V. The capacitance of the parallel plate capacitor is given by q ð4:1Þ C¼ V In terms of the permittivity constant of free space, and the relative permittivity of dielectric material, we can write the capacitance between the plates as C¼

10 1r A d

ð4:2Þ

where 10 is the permittivity of free space, 1r is the relative permittivity of the dielectric material between two plates, A is the area of the plates, and d is the distance between the plates. The relative permittivity is often called the dielectric constant which is deﬁned by 1=10 ¼ 1r, 1 being deﬁned by the

TABLE 4.1 Dielectric Constants of Some Materials (T 5 258C) Material

1r

Vacuum Air Water Benzene Rubber (silicon) Plexiglas Polyesters Ceramic (alumina) Polyethylene Silicon resins Epoxy resins Nylon Lead nitrate

1 1.00054 78.5 2.28 3.2 3.12 3.22 to 4.3 4.5 to 8.4 2.26 3.85 3.65 3.50 37.7

Mechatronic Elements for Optomechatronic Interface

w

V

177

V

d x

(a) area A overlaps

d

(b) gap changes

V

d

dielectric material

(c) dielectric material moves FIGURE 4.1 Change of capacitance between two plates.

permittivity of the material. The dielectric constants of some materials are given in Table 4.1. As shown in Figure 4.1 there are three ways of achieving capacitance variation. In Figure 4.1a, the displacement of one of the plates changes the area of overlap. Figure 4.1b shows the case of varying the distance d between the plates, whereas in Figure 4.1c the displacement of the dielectric material causes a change in capacitance. As one example of such changes, let us suppose that the plate separation d is changed by the amount Dd, and then the resulting change in C will be given by DC ¼ 2

10 1r A Dd d þ Dd d

which indicates a nonlinear relationship between them. This type of sensor has several advantages: noncontact, high accuracy, and high resolution. In addition, stability is another advantage, because it is not inﬂuenced by pressure or temperature of the environment. Differential Transformer The differential transformer employing the principle of induction shown in Figure 4.2 provides an AC voltage output proportional to the displacement of the magnetic core passing through the windings. It is essentially a mutual inductance device which consists of three coils symmetrically spaced along an insulated circular tube. The center coil which is the primary coil is energized from an AC power source and the two identical end coils which are the secondary coils are connected together in series. The connection is made in such a way that its outputs are produced out of phase with each

178

Optomechatronics output voltage Ve

magnetic core secondary coil 2

input displacement

primary coil

secondary coil 1

input voltage i FIGURE 4.2 Schematic view of linear variable differential transformer (LVDT).

output voltage

other. When the magnetic core is positioned at the central location the voltage induced (electromotive force emf) in each of the secondary coils will be of the same magnitude, and thus the resulting net output is zero. When the core moves farther from the central position, there will be an increase in the output within a certain limit on either side of the null position. Figure 4.3 illustrates the characteristic curve of output voltage vs. core displacement. The phase difference between the outputs of the two regions is 1808, i.e., out of phase, as can be seen from the ﬁgure. Recalling Faraday’s law

A

O

A O

FIGURE 4.3 Typical output voltage of LVDT.

B

core displacement

B core displacement

Mechatronic Elements for Optomechatronic Interface

179

of induction, the voltage (emf) induced in a secondary coil due to change in current i ﬂowing in the primary coil is equal to Ve ¼ 2M

di dt

ð4:3Þ

where M is the coefﬁcient of mutual inductance between two coils. The minus sign indicates the direction of the induced voltage. Since there are two secondary coils in the vicinity of the primary coil, we have two induced emfs for the coils. Thus, if an input current i ¼ i0 sinvt ﬂows through the primary coil, each emf is given by Ve1 ¼ C1 sinðvt 2 wÞ

Ve2 ¼ C2 sinðvt 2 wÞ

where Ve1 and Ve2 denote the emf of the secondary coil one and two, respectively, the constants, C1 and C2, are determined by the position of the core relative to the secondary coils one and two, and w is the phase difference between the primary and secondary voltages. As can be seen from Figure 4.2, the two secondary emfs are connected in series, and therefore the output voltage becomes Ve1 2 Ve2 ¼ ðC1 2 C2 Þsinðvt 2 wÞ It is noticed that the sign and magnitude of ðC1 2 C2 Þ depend on where the magnetic core position is located at the instant of its motion. If C1 ¼ C2 , then the core is located at the null position. If C1 . C2 , the portion of the core positioned in the secondary coil 1 is greater than that in the coil 2, whereas C2 . C1 is the reverse case. This sensor has several advantages for displacement measurement. One of the advantages is noncontact, which eliminates friction and thus gives high resolution. Piezoelectric Sensors Certain materials become electrically polarized when they are subject to mechanical strain. This effect is known as the piezoelectric effect discovered by the French Curie brothers in 1880. The reverse effect is called the inverse piezo effect which is widely used in piezo actuators. Piezoelectric materials can be categorized into single crystals, ceramics, and polymers. Notable among the crystals materials are quartz and Rochelle salt. Barium titanate, lead zirconate titanate (PZT), and ZnO belong to ceramic materials, and polyvinylidene ﬂuoride (PVDF) is an example of a piezoelectric polymer. The piezoelectric effect stays up to the Curie point, since it is dependent upon the temperature of the material. Above that point, the ceramic material has a symmetric polycrystal structure, whereas below the point it has nonsymmetric crystal structure. When it is under the symmetric state, it cannot produce the piezoelectric effect since the centers of the positive and negative charge sites coincide. When it exhibits nonsymmetric structure due

180

Optomechatronics

V

(a) warmed up state

(b) placed in strong electric field V

FIGURE 4.4 Thermal poling of piezo material.

to a phase transformation of the material, the centers of the positive and negative charge sites no longer coincide. In this case, the material exhibits the piezoelectric effect. The Curie point depends on the material, for example, piezo ceramics (150 – 4008C). There are several different procedures for producing the piezoelectric effect called “polarizing” procedure. In the case of material such as PZT and barium titanate, the procedure starts with heating it to a temperature above the Curie point (1208C). This step makes some electric dipoles within the materials which were initially randomly oriented align in a desirable direction. Then, a high DC voltage is applied across the faces, and the material is cooled down while keeping the same electric ﬁeld across its thickness. Finally, when the electric ﬁeld is removed, its polarization stays permanent as long as the polarized material is kept below the Curie point. Figure 4.4 shows two states of the electric ﬁeld in the PZT developed in the course of poling, as indicated with arrows. In Figure 4.4a, the PZT is electrically neutral when no heat is applied, whereas the state of the PZT in Figure 4.4b is electrically charged in a certain direction shown with arrows due to shift of atoms inside the material, when electric ﬁeld is applied. This phenomenon leads to the development of electric charge on its surface in response to a mechanical deformation: one face of the material becomes positively charged and the opposite face negatively charged, resulting in an electric potential as shown in Figure 4.5a. In Table 4.2, various properties of piezoelectric materials at T ¼ 208C are listed. The charge developed is proportional to the applied force, F, and is expressed as q ¼ Sq F

ð4:4Þ

where Sq is the piezoelectric constant or charge sensitivity of the piezoelectric material and depends upon the orientation of the crystals material and the way in which the force is applied. The charge sensitivity coefﬁcient is listed in Table 4.3. To pick up an accumulated electric charge, conductive electrodes are applied to the crystal at the opposite sides of the material cut surface. From this point of view, the piezoelectric transducer is considered to be a parallel plate capacitor. The capacitance of the piezoelectric material

Mechatronic Elements for Optomechatronic Interface

181

force cable

A + + + + + + + + + +

piezoelectric transducer

electrodes

h

charge charge amplifier amplifier

_ _ _ _ _ _ _ _ _ _

(a)

(b)

force

Cf

Rf

i

Cp

Rp

−

Cc

i

+

(c) piezoelectric transducer cable

charge amplifier

Cp

Rp

Vo

(d)

FIGURE 4.5 Piezoelectric sensor connected to charge ampliﬁer.

between the plates is given by Cp ¼

10 1r A h

ð4:5Þ

where 1r is the relative permittivity of the piezoelectric material, 10 is the permittivity of the free space, and A and h are the area and thickness of the material, respectively. Since Cp ¼ q=V where V is the potential difference between the electrode plates, combining Equation 4.4 and Equation 4.5

TABLE 4.2 Various Properties of Piezoelectric Materials (T 5 208C)

Material Quartz Barium titanate PZT Polyvinylidene ﬂuoride

Density r, 103(kg/m3)

Young’s Modulus E, (109N/m)

Dielectric Constant 1r

Piezoelectric Constant Cp (pC/N)

1.78 5.7 7.5 1.78

77 110 830 0.3

4.5 1700 1200 12

2.3 78 110 d31 ¼ 20, d32 ¼ 2, d33 ¼ 30

182

Optomechatronics

TABLE 4.3 Typical Charge Sensitivities Sq of Piezoelectric Materials Material Quartz SiO2 Single crystal Barium titanate (BaTiO3) Ceramic, poled polycrystalline

Orientation

Sq (pC/N)

X-cut (length longitudinal) X-cut (thickness longitudinal), Y-cut (thickness shear) Parallel to polarization Perpendicular to polarization

2.2 22.0, 4.4 130 256

Source: Dally, J.W., et al. Instrumentation for engineering measurements, John Wiley and Sons Inc., 1984. Reprinted with permission of John Wiley & Sons, Inc.

leads to V¼

Sq h F 10 1r A

ð4:6Þ

The above equation implies that the developed electric voltage is proportional to the charge sensitivity, thickness, and applied force. The developed potential difference is usually very small, and thus it needs to be ampliﬁed by an ampliﬁer called a “charge ampliﬁer” as indicated in Figure 4.5b. When this sensor is connected via a cable to this ampliﬁer, the equivalent electric circuit of the piezoelectric transducer and the ampliﬁer circuit may be as shown in Figure 4.5c. The whole electric circuit can be modeled as a charge generator electrically connected to the capacitance Cp and the leakage resistance Rp , the cable represented as a single capacitance Cc , and the charge ampliﬁer composed of Cf and Rf . If the cable and the ampliﬁer circuits are neglected, the equivalent circuit can be modeled as shown in Figure 4.5d. A polyvinylidene ﬂuoride (PVDF) ﬁlm sensor shown in Figure 4.6 measures a force acting on the cantilever beam which in this case is a concentrated force. The force sensing principle of the sensor utilizes the same concept as that of the piezoelectric effect, as shown in Figure 4.6a. When the PVDF cantilever structure deforms as a result of the applied force, the deformation will produce an electrical charge due to the piezo effect. In general, the relationship between the behavior of the piezoelectric material and electrical behavior is known to exhibit a very complicated phenomenon. According to the property of the linear piezoelectric materials the following relationship holds Di ðxÞ ¼ dij sj ðxÞ þ 1ik Ek ðtÞ

ð4:7Þ

In the case of the PVDF ﬁlm sensor, Di is the amount of charge on the PVDF per unit area, Ek is the electric ﬁeld strength, dij is the piezoelectric coefﬁcient

Mechatronic Elements for Optomechatronic Interface

183

F electrode Vo

PVDF film electrode F

(a) force sensing principle PVDF layer A

F h

3 1

w 2

(b) PVDF sensor configuration FIGURE 4.6 PVDF-based piezoelectric force sensor.

of the material, 1ik is the permittivity of the PVDF, and sj ðxÞ is the applied stress. The double indices in dij indicate the direction of the generated charge (ﬁrst index i) and that of the applied stress (second index j). The double indices in 1ik indicate the direction of the generated charge (ﬁrst index i) due to the electric ﬁeld strength acting in the kth direction. It is noted that in the case of piezoelectric materials, the majority of the dielectric constants are zero except; 111, 122, 133, i.e. 1ik ¼ 0 when k – i. According to these properties, for the conﬁguration shown in Figure 4.6b Equation 4.7 can be expressed as D3 ðxÞ ¼ d31 s1 ðxÞ þ 133 E3 ðtÞ Then the charge generated in the PVDF can be determined by integrating the above equation ð q ¼ D3 ðxÞdA where A is the ﬁlm area. When the above integration is carried out by using

s1 ðxÞ ¼

hð‘ 2 xÞ FðtÞ 2I

induced due to a concentrated force, F(t), the charge developed for the PVDF conﬁguration is obtained by q¼

d31 Ah‘ FðtÞ þ 133 E3 ðtÞA 4I

ð4:8Þ

It can be observed that the developed charge is composed of the contribution due to the applied force and the induced electric ﬁeld. It is noted here that h denotes the ﬁlm thickness only, and I denotes the moment

184

Optomechatronics

of inertia which accounts for total thickness consisting of the electrode, ﬁlm, and surface coating. This charge generates a current dq=dt through the resistance of the PVDF Rp, so the output voltage Vo ðtÞ of the sensor can be determined by Vo ðtÞ ¼ Rp

dq dt

Because E3 ðtÞ ¼ 2

dVo ðtÞ V ðtÞ ¼2 o dh h

the above equation can be rewritten as Rp Cp

dVo ðtÞ dFðtÞ þ Vo ðtÞ ¼ Kp dt dt

ð4:9Þ

where Cp is the capacitance of the PVDF ﬁlm given by Cp ¼ the constant given by Kp ¼

133 A and Kp is h

Rp d31 Ah‘ 4I

Referring to Figure 4.5d, we can conﬁrm that the above equation represents the equivalent electrical circuit produced by PVDF ﬁlm. Piezoresistive sensors utilize materials that exhibit a change in resistance when subjected to an external force. A semiconductor gauge usually exhibits large piezoresistive effect. The common material is silicon doped with p-type or n-type material; the p-type yields a positive gauge factor whereas the n-type gives a negative one. Some details of this type of semiconductor material will be discussed in the next subsection. The strain gauges are attached usually to a beam or a diagram as shown in Figure 4.7a. Another type of piezoresistive force sensor shown in Figure 4.7b uses piezoresistive ceramic composite as a sensing element. As shown, it does not use any ﬂexible member in its conﬁguration but uses a bulk material which has both structural strength and force detection function. In recent years, piezoelectric sensors and actuators in micro scale have been well developed due to advancement of microfabrication methods such as deposition of thin ﬁlms and patterning techniques. For example, applied force piezoresistor piezoresistor

Si substrate

(a) beam-type FIGURE 4.7 Schematics of piezoresistive sensors.

applied force

electrode

(b) bulk-type

ceramic composite

electrode

Mechatronic Elements for Optomechatronic Interface

185

deposition of thin layers of polymers and ceramics over silicon is a commonly adopted method to obtain silicon based piezoelectric structure. This can be easily carried out by sputtering ion beam or vapor deposition methods. The structural shapes commonly used for deposition are rigid substrate, diaphragm, and two layer bimorph. Discussion of these methods will not be made further, for those involved with complex physical phenomena are beyond the scope of this book. Pyroelectric Sensor

polarization

The pyroelectric materials are crystalline materials capable of generating electric charge in response to heat ﬂow. This effect is very closely related to the piezoelectric effect discussed previously. In fact, many crystalline materials exhibit both pyroelectric and piezoelectric properties. When polarized materials are exposed to a temperature change, its polarization, which is the electric charge developed across the material, varies with temperature of the material. A typical relationship is shown in Figure 4.8a. Polarization decreases with increase in temperature and becomes zero or almost zero near the Curie temperature point. The reduction in polarization results in decrease in charge at the surface of material. As a result, a surplus of charge occurs at the surfaces because there are more charges retained at surface before temperature changes. The pyroelectric materials therefore act

temperature, T

(a) polarization of a pyroelectric material in response to heat flow heat flow out

electrodes

pyroelectric material

i

Cp

Rp

heat absorbing layer heat flow in

(b) sensor configuration FIGURE 4.8 Pyroelectric sensor.

(c) equivalent circuit

186

Optomechatronics

as a capacitor. When electrodes are deposited on opposite sides, as shown in Figure 4.8b, these thermally induced surplus charges can be collected by electrodes. The conﬁguration is the pyroelectric sensor that measures the change in the charge that occurs in response to change in temperature. From the relation shown in Figure 4.8a, the sensor will produce the corresponding charge changes Dq in response to the change in temperature DT in the following manner: Dq ¼ kq DT where kq is called pyroelectric charge coefﬁcient and exhibits a nonlinear function of temperature. From the sensor conﬁguration, we can see that the pyroelectric detector does not need any external excitation signal, but requires only an electric circuit that connects the electrodes. The equivalent electric circuit of a pyroelectric sensor is shown in Figure 4.8c. It consists of a capacitor Cp charged by the heat ﬂow induced excess charge and the resistance Rp of the input circuit. Semiconductor Sensors; Light Detectors Semiconductors are crystalline solid material and have several interesting properties: (1) their electrical properties lie in between those of conductor and insulator; roughly speaking, conductors have very low electrical resistivity (a pure metal), and isolators have very high resistivity. (2) their resistance decreases rapidly with increasing temperature, that is, they have a negative coefﬁcient of resistance. For example, silicon is found to have resistivity that decreases about 8%/8C. (3) In a pure state, they have very high resistance such that few electrons are available to carry electric current. Conductors that contain no impurities are called intrinsic semiconductors. However, when impurities are introduced to the pure semiconductor materials, they are called extrinsic semiconductors. The most interesting property of these impure semiconductors is that their conductivity changes, the amount of the change being dependent on the property of the impurity to be added. The impure materials added into the semiconductor materials are called dopants, and the process of adding impurities is called doping. In extrinsic semiconductors, there is current ﬂow within the materials under the action of an applied ﬁeld. In this situation, bound electrons arranged in energy bands are separated from particular atoms in the materials, and as a result, positively charged holes are formed and will move in the direction opposite to the ﬂow of electrons; a hole is a missing electron in one of the bands. Therefore, current ﬂow in semiconductor materials is due to the contribution of (1) positively charged holes and (2) negatively charged electrons. These two cases are illustrated in Figure 4.9. The p-type semiconductors are created due to an excess of holes. In this case, the contribution to current ﬂow is due to presence of holes. Boron, aluminum,

Mechatronic Elements for Optomechatronic Interface pure semiconductor

flow of holes flow of current

(a) p-type semiconductor

metal electrode

187 pure semiconductor

metal electrode

flow of electron flow of current

(b) n-type semiconductor

FIGURE 4.9 Two types of semiconductors.

gallium are the impurities that serve as p-dopants. In contrast, the n-type semiconductors are created excess of electrons in which current ﬂow is due to presence of electrons. As shown in the ﬁgure, it is noted that the current ﬂow in the same direction as that of whole ﬂow. Most semiconductor devices involve a junction at which p-type and n-type doping meet. In other words, these are placed in contact with one another. The resulting device is called a p –n junction. Through this junction, current ﬂows easily from the p-type to the n-type conductor, since positively charged holes easily enter the n-type conductor, whereas electrons (negatively charged) easily enter the p-type conductor. This process is known as diffusion process. In the case of the current ﬂow from the n-type to p-type, there is much greater resistance. There are two types of connection when a voltage source is applied to the junction; forward biased when the p-type is connected to the positive side of the source, and reverse biased when n-type is connected to the positive side, as shown in Figure 4.10a and Figure 4.10b. Figure 4.10c depicts the characteristics of a voltage –current for the p –n junction. In the forward biased connection, current does not ﬂow till a voltage reaches the threshold value Vth , but above the threshold voltage, the current rapidly increases with the voltage. In contrast to this trend, in reverse biased connection, the current is almost zero, reaching, 2i0 , which is called reverse saturation current. This small current ﬂow occurs due to the fact that there exist a small number of electrons in p-type and small number of holes in n-type. The saturation current exists within a wide range of reverse bias voltage. However, when the voltage increases slightly beyond a certain value called the breakdown voltage, Vbr, the junction of the semiconductor diode breaks down. As a result, the current increases rapidly without any further increase in voltage as shown in the ﬁgure. Combining the voltage vs. current relation, for both regions of applied voltage, the resulting curve looks as illustrated in Figure 4.11. It is noted that the curve depends upon the material properties and temperature. For instance, the reverse saturation current i0 is of the order of nanoampares for silicon but milliampares for germanian. The current equation in a diode that

(c)

current (i ), mA

Vth

voltage (V ), volt

metal electrode

FIGURE 4.10 Types of connections and their characteristics of voltage vs. current.

(a)

diode

n-type

(b)

Vbr

p-type

io

diode

current (i ), mA

p-type

metal electrode

voltage (V ), volt

n-type

188 Optomechatronics

189

current (i ), mA

Mechatronic Elements for Optomechatronic Interface

Vbr

−io

voltage (V ), volt

FIGURE 4.11 The characteristic curve of the voltage vs. current (silicon).

can represent the above relationship may be approximately expressed by i ¼ i0 exp

qV kT

21

ð4:10Þ

where V is the applied voltage, i0 is the saturation current, q is the charge of an electron, T is the temperature in degree Kelvin, and k is the Boltzman constant. Photodiode Semiconductor junctions are sensitive to light as well as heat. If a p – n junction is reversely biased, it has high resistance. However, when it is exposed to light, photons impinging the junction can excite bound electrons and create new pairs of electrons and holes on both sides of the junction. Consequently, these separate and ﬂow in opposite directions: electrons ﬂow toward the positive side of the voltage source, whereas holes ﬂow toward the negative side. This results in photocurrent, ip which is directly proportional to the irradiance of the incoming light, Ir ip ¼ CI Ir

ð4:11Þ

where CI is the proportionality constant depending on the area of the diode exposed to incoming light. Combining Equation 4.10 and Equation 4.11 yields i ¼ i0 exp

qV kT

2 1 2 ip

ð4:12Þ

190

Optomechatronics

dark current voltage (v), volt

Ir = 0 Ir = I1 Ir = I2

I1 < I2 < I3

current (i ), mA

Ir = I3

FIGURE 4.12 Voltage vs. current characteristics of a photodiode.

The characteristics of the voltage vs. current relation of the photodiode are shown in Figure 4.12. As can be seen, the photo current ip is much larger than the dark current i0 ðIr ¼ 0Þ and gets larger with increasing light irradiance or intensity. This indicates that if we operate the photodiode within the range of a reversed biased voltage we can make the output current directly proportional to the irradiance or intensity of incident light. Since total current is composed of photo current and dark current (reverse leakage), the dark current needs to be kept as small as possible to have high sensitivity of a photodiode. This can be done by cooling photodiodes to very low temperatures. A variety of photodiode detectors are available for converting a light signal to the corresponding electrical signal for position detection. One form of such detector types is multidiode units which are deposited on a common silicon substrate. There are two types of diode conﬁguration, two or four separate diodes. As shown in Figure 4.13, these are all separate identical diodes insulated from each other and produce a photocurrent in proportion to the area upon which the light beam is incident. A two-separate diode type can be used to determine one dimensional displacement whereas a guardant type can be used for measurement of two dimension displacement of the light beam. When the light beam is illuminated on an equal area of each diode, that is, the beam is centered, the output of each diode will be identical. This case yields i1 ¼ i2 for the two diode type, and ix1 ¼ ix2 ¼ iy1 ¼ iy2 for the quadrant and therefore no output will be produced out of the operational ampliﬁer. However, when the illumination is unequal, this will produce

Mechatronic Elements for Optomechatronic Interface

191

i1 OP amp

OP amp

ix

1

∆x

ix

2

i2

iy

∆y

1

iy

OP amp

2

(a) split cell detector

(b) quadrant detector

FIGURE 4.13 Photodiode detectors.

unequal output from each diode, which in turn will produce the output of the ampliﬁer. Other Photodetectors Photo detectors detect electromagnetic radiation in the spectral range from ultraviolet to infrared. These are categorized into two groups: photon detectors and thermal detectors. The photon detectors operate from the ultraviolet to mid infra spectral ranges, whereas thermal detectors separate in the spectral range of the mid and far infra. Photon detectors utilize the “photoelectric” principle that when light is incident on the surface of semiconductor materials the photonic energy is converted to kinetic energy of the electrons. There are three major mechanisms which produce this type of phenomenon. These include the photoemissive, photoconductive, and photovoltaic; photoemissive detectors produce an electrical signal in proportion to the incident light. As shown in Figure 4.14a, it consists of a cathode (negative), the emissive surface deposited on the inside of a glass tube, and an anode (positive) collecting surface. If a proper circuit is used, light incident on the cathode can cause electrons to be liberated and

light photoconductive layer incident light

e e e

electrode anode h

cathode

(a) photo emissive type FIGURE 4.14 Basic structure of photo detectors.

electrode

(b) photoconductor type

192

Optomechatronics

emitted. These emitted electrons reach the anode and make current ﬂow through the external circuit. When the output current is not sufﬁcient even at high intensity of light, adding successively higher electrodes called “dynodes” will provide substantial ampliﬁcation. This device is known as a photomultiplier tube. Photoconductive detectors made of some bulk semiconductors without a p – n junction sensitively respond to light. As shown in Figure 4.14b, these consist of a photoconductive thin layer fabricated from semiconductor materials such as cadmium selemium (CdSe), cadmium sulﬁde (CdS), lead telluride (PbTe), and an electrode set at both ends of the conductor. The electrical resistivity of these materials decreases when they are exposed to light. As discussed earlier, in photodiode due to the decrease in resistivity, the output current of the conductor can be quite large. The change in resistance DR is related to the change in conductance Ds; DR ¼

1 ‘ hDs w

where ‘, w and h are the length, width and thickness of the conductor, respectively. A photoconductor often uses a bridge circuit to detect the intensity of light illuminated on the n-type layer. The change in resistivity due to incident light can be effectively detected by this circuit. Photovoltaic Detectors These detectors supply the voltage or current without any external power source, when they are exposed to light. The detector commonly known as a solar cell consists of a sandwich of dissimilar materials such as an iron base coated with a thin layer of iron selenide. When the cell is exposed to light, a voltage is developed across the sandwitch. Semiconductor junction also belongs to one of such materials. A photodiode discussed in Figure 4.12 exhibits the photovoltaic effect, when it is operated in the region where i is negative and V is positive (fourth quadrant of the ﬁgure). In this region, the active area of the p –n type photodiode junction is illuminated without any external bias voltage, and a current is produced in proportion to the intensity of the incident light for essentially zero voltage across the diode. Avalanche Photodiode As already discussed in Figure 4.11, an avalanche photodiode operates in the region of near breakdown voltage Vbr : At this breakdown voltage, known as the Zener voltage, current rapidly increases with a small increase in reverse bias. In other words, at near Vbr, even a small change in the illumination on the photodiode causes a large change in the photocurrent.

Mechatronic Elements for Optomechatronic Interface

193

Signal Conditioning Operational Ampliﬁers An operational ampliﬁer (op-amp) is a complete ampliﬁer circuit supplied as an integrated circuit on a silicon chip and is a basic signal conditioning element. The ampliﬁer has an extremely high gain with typical value higher than 105, and therefore can be regarded as inﬁnite for the purpose of circuit analysis. Figure 4.15 shows the physical sizes of op-amps along with the schematic diagram of its internal circuit representation. We will limit our discussion here to the ideal op-amp which has the following characteristics; (1) high input impedance mega ohms to giga ohms, (2) low output impedance (of the order of 100 V) considered to be negligible, (3) extremely high gain (G ¼ 105 (100 dB) typical value). Figure 4.16 shows six different types of ampliﬁer circuits frequently used in instrumentation and signal conditioning. Inverting Ampliﬁer Since an operational ampliﬁer has a high intrinsic gain and very high input impedance, it is operated essentially with zero input current and voltage (ia ¼ 0, Va ¼ 0). Figure 4.16a shows the inverting mode of the op-amp which has negative feedback. At the junction of the two resistors we have i1 ¼

Vin 2 Va R1

inverting input noninverting input

output

(a) photograph of OP amplifiers

(b) operational amplifier circuit

FIGURE 4.15 Typical operational ampliﬁers. Source: National Semiconductor Corporation (www.nsc.com).

194

Optomechatronics

i2

Rf

Rf

R1 i1

Vin

a

a

Vout

Rf

R1

V1

Vout

(b) non-inverting amplifier

(a) inverting amplifier

V2

R1

Vin

C R

R2

Vout

(c) summing amplifier

Vin

Vout

(d) integrating amplifier Rf

Rf C

Vin

(e) differentiator

V1 V2 Vout

R1 R2

Vout

(f) differentiator

FIGURE 4.16 Basic operational ampliﬁer circuits.

i2 ¼

Va 2 Vout Rf

ð4:13Þ

where R1 and Rf are the input and feedback resistors, respectively. By Kirchhoff’s current law, i1 ¼ i2 þ ia Hence Vout R ¼2 f Vi R1

ð4:14Þ

The above relationship implies that the voltage gain of the op-amp is determined simply by the ratio of the input and output resistors.

Mechatronic Elements for Optomechatronic Interface

195

Noninverting Ampliﬁer This mode of op-amp operation causes a voltage of the same polarity to appear at the ampliﬁer output. The general use of this mode is schematically illustrated in Figure 4.16b. It has both inverting and noninverting inputs and ampliﬁes the voltage difference between the two inputs. Let us assume that current ﬂows through the inverting terminal input and feedback registers. The voltage at the junction “a” is then equal to Va ¼

R1 V R1 þ Rf out

Since there is no current ﬂow, i.e., ia . 0, this gives Va . Vin : Therefore, we have Vout R ¼1þ f Vin R1

ð4:15Þ

The closed loop gain is again determined by the ratio of resistors. The characteristic of this noninverting input is that the circuit input impedance is the input impedance of the ampliﬁer itself rather than the impedance denoted by ampliﬁer input resistor. Inverting Summing Ampliﬁer The objective of this mode is to obtain the output voltage in terms of the input voltages V1 and V2 : As in the case of the inverting ampliﬁer, the use of Kirchhoff’s law at the summing junction shown in Figure 4.16c yields i1 þ i2 ¼ ia þ i

ð4:16Þ

In terms of voltage drops, the above equation can be written as Vout V V ¼2 1 þ 2 Rf R1 R2 since ia ¼ 0: It is clear that this relationship can be extended to an arbitrary number of input ports. In the case of ﬁxed feedback resistance Rf , the gains of the individual input can be adjusted by varying the input resistances R1 and R2 : When R1 ¼ R2 ¼ Rs , the output of this summing mode given in the above equation is obtained by Vout ¼ 2

Rf ðV þ V2 Þ Rs 1

This relationship can be effectively used when we want to average the input voltages from multiple sensors. Integrating Ampliﬁer An integrating ampliﬁer is used to integrate the incoming input voltage and utilizes a capacitor as shown in Figure 4.16d. The expression for the output

196

Optomechatronics

voltage Vout can be obtained by considering the current ﬂow through the capacitor. Since the charge q ¼ CVout and i ¼ dq=dt, we have Vin dVout ¼ 2C R dt

ð4:17Þ

Solving for the output voltage Vout we have Vout ¼ 2

1 ðt V dt RC 0 in

The output is proportional to the integral of input voltage, Vin : Differential Ampliﬁer A differential ampliﬁer is used to obtain the difference between two input voltages. Two types of the differentiator are shown in Figure 4.16e and Figure 4.16f. The one shown in Figure 4.16e is similar to the integrating ampliﬁer except that the position of resistor and capacitor are interchanged. An expression for the output voltage Vout of this ampliﬁer mode can be developed by using Equation 4.17. In another type of the differentiator shown in Figure 4.16f, there is no current ﬂowing through the ampliﬁer. The voltage potential Va will then be given by Va R2 ¼ V2 R1 þ R2

ð4:18Þ

As with the previous ampliﬁers, the current ﬂowing through the feedback resistance Rf must be equal to that through the input resistance R1 : Thus V1 2 Va V 2 Vout ¼ a R1 R2

ð4:19Þ

Combining Equation 4.18 and Equation 4.19, we have Vout ¼

R2 ðV 2 V1 Þ R1 2

Comparator When a comparison between two signals is required, an operational ampliﬁer itself can be used as a comparator. As shown in Figure 4.17, a voltage input V1 is applied to the inverting input while another input V2 is applied to the non inverting input. Because of the high gain of the ampliﬁer, only a small difference signal of the order of 0.1 mV is required for the output to compare which of two voltages is the larger. The right hand side of the ﬁgure shows the output voltage vs. the input voltage difference. When non inverting input is greater than inverting input, then the output Vout swings from the negative 2Vs to the positive Vs , and vice versa.

Mechatronic Elements for Optomechatronic Interface

197 output Vs

V1 0

V2

input

−Vs FIGURE 4.17 Comparator.

Signal Processing Elements Filters Filters are used to inhibit the presence of a certain band of undesirable frequencies from a dynamic ﬁnal signal, permitting others to be transmitted, as illustrated in Figure 4.18a. The range of frequencies passed by a ﬁlter is a signal containing all frequency components

particular frequency components of interest

electronic filter

practical

0 fC (b) low pass

practical

fC1

(d) band pass FIGURE 4.18 Various signal ﬁlters.

f

ideal

signal magnitude 0

signal magnitude

ideal

fC2

f

ideal practical

fC 0 (c) high pass

signal magnitude

signal magnitude

(a) signal filtering

0

f

ideal practical

fC1

(e) band stop

fC2

f

198

Optomechatronics

called the pass band and the range not passed as the stop band. In actual practice, it is not possible to completely remove the unwanted frequency contents due to the dynamic characteristics of the signal. Figure 4.18b to Figure 4.18e show four different ideal ﬁlters classiﬁed according to the ranges of frequencies. A low pass ﬁlter is the one that transmits all frequencies from zero up to some speciﬁed frequency, that is, frequencies below a prescribed cut-off frequency. In contrast, a high pass ﬁlter transmits all frequencies from a cut-off frequency up to inﬁnity. A band pass ﬁlter transmits all frequencies within a speciﬁed band whereas a band stop ﬁlter rejects all frequencies within a particular band. As can be observed from the ﬁgure, the sharp cut-off of the ideal ﬁlter cannot be realized, because all ﬁlters exhibit a transition band over which the magnitude ratio decreases with the frequency. The rate of transition is known as the ﬁlter roll off. Digital-to-Analog Conversion The purpose for a digital-to-analog (D/A) conversion is to convert the information contained in a binary word to a DC output voltage, which is an analog signal. The signal represents the weighted sum of the nonzero bits in word. This indicates that the leftmost bit is the most signiﬁcant bit (MSB) having the maximum value which is twice the weight of the next less signiﬁcant bit, and so on. In general, the weighting scheme of an M bit word is given as below bit

M21

bit 2

bit 1

bit 0

2M21

···

22

21

20

2M21

···

4

2

1

Under this scheme, the leftmost bit, M 2 1 is known as the MSB, since its contribution to the numerical value of the word is the largest relative to the other bits. Bit 0 is known as LBS. The number of the digital word determines the resolution of the output voltage. Suppose that the word has M bits and expresses a full scale of an output voltage Vout : In this case, a change of one bit will cause the corresponding change in the output voltage Vout of Vout =2M : The above conversion relationship can be realized electronically in a number of ways one of which is shown in Figure 4.19. This is a simple DAC known as the weighted register DAC and employs an inverting summing ampliﬁer. In the ﬁgure Rf is the feedback resistance, R is the base resistance related to the input resistances, and bi ði ¼ 1; 2· · ·MÞ is 0 or 1. The bi is “off” when connected to the ground and “on” is when connected to VR : When all bis are on, DAC gives analog output voltage which is proportional to an input parallel digital signal; bM21 ; bM22 · · ·b2 ; b1 ; b0 : Summing all currents at the summing junction, we have i0 þ i1 þ · · · þ iM21 ¼ ia þ if

ð4:20Þ

Mechatronic Elements for Optomechatronic Interface VR

199

Rf bM-1

20 R

bM-2

21R

bM-3

22R

b1

2 M-2 R

b0

2 M-1 R

if Vout

Va ia

FIGURE 4.19 Digital-to-analog converter.

Equation 4.20 can be rewritten using Ohm’s law as bM21

VR 2 Va V 2V V 2V V 2 Vout þ bM22 R 1 a þ · · · þ b0 RM21 a ¼ ia þ a 0 Rf 2 R 2 R 2 R

Since Va ¼ 0, the output voltage is obtained by Vout ¼ 2

Rf VR R

bM21 b b1 b0 þ M22 þ · · · þ M22 þ M21 20 21 2 2

¼2

Rf V N R R

ð4:21Þ

where N is denoted by N¼

bM21 b b1 b0 þ M22 þ · · · þ M22 þ M21 20 21 2 2

ð4:22Þ

Equation 4.22 indicates that N is the binary number output to D/A converter and the output of the D/A converter Vout is proportional to N. In the above, bM21 is the MSB, bM22 the next signiﬁcant bit, and b0 is the least signiﬁcant bit (LSB). If M ¼ 8, there are 8 bits in the converters, switch number seven corresponds to MSB, while switch number 0 corresponds to LSB. When switch numbers one, six, and seven are closed, the bits in Equation 4.22 become 11000012, which corresponds to 9710. Analog-to-Digital Converters The analog-to-digital (A/D) converter is a device that samples an analog input voltage and encodes the sampled voltage as a binary word. A number of different types of A/D converters are available that offer a range of different performance speciﬁcations. The most common are successive approximation, ramp, dual ramp and ﬂash. The simplest form of A/D converter is the successive approximation. Figure 4.20 shows the converter circuit in which a D/A converter is used along with a comparator and a time and control unit. This circuit utilizes the output of a counter as the input to a D/A converter. The output of the

200

Optomechatronics clock n bit register

analog input

counter

DAC

parallel digital output FIGURE 4.20 Conﬁguration for successive approximation ADC.

counter is obtained by counting a sequence of pulses generated in a binary fashion by a clock. The D/A converter then converts this value into an analog value, which is compared with the input analog value by the comparator. When the output of the D/A converter equals or exceeds the input analog voltage, counting of the pulses from the clock is stopped by closing a timing and control unit (gate). The value of the counter is then the desired digital value. For example, initially the counter makes a guess that an input analog voltage is greater than or equal to a certain voltage, say, 5 V. The unit then sets MSB’s ﬂip-ﬂop to a logical 1 state. The binary word 1000 is fed to the D/A converter, which feeds 5.0 V to the comparator. If the comparator senses that the analog voltage is greater than the voltage from the D/A converter, the counter leaves the MSB’s ﬂip-ﬂop set to a logical 1 state. Because the ﬁrst guess is too small to represent the analog input, the counter then makes a second guess, setting the second bit to a 1. Sample and Hold Module The operation of analog-to-digital conversion takes some time. Therefore, it is necessary to hold an instantaneous value of analog input while conversion takes place. This is done by using a sample and hold device whose circuit is shown in Figure 4.21. The circuit consists of an electronic switch to take the

y(t)

Vin

C

(a) sample and hold circuit FIGURE 4.21 Sample/hold device.

original input signal

Vout

(b) sample and hold signal

t

Mechatronic Elements for Optomechatronic Interface

201

sample, a capacitor for the hold, and an operational voltage follower. The operation principle works, during sample and hold states as follows. In the sample state, if the switch is closed, the capacitor is charged with the applied analog input voltage. The voltage follower makes the output voltage follow the input voltage: in the hold state, if the switch is opened, the capacitor retains its charge. The ampliﬁer maintains the output voltage equal to the input voltage at the instant of time until the switch was opened. Therefore, the output voltage is held constant at the value of the input voltage at the instant of time until the switch is closed again. Multiplexer When sampling several input signals from a number of sources it is sometimes desirable to use a single A/D converter by switching one input signal to another by means of a multiplexer circuit such as that shown in Figure 4.22. The multiplexer is an electronic switching device that enables sampling of the input signal sequentially rather than in parallel. Multiplexing therefore is used to enable a single channel to be shared between a number of signal sources. Time Division Multiplexing Figure 4.23 shows a schematic diagram of a time division multiplexer with four channels, 0 1 2 3. When the four input signals are present, the multiplexer selects a signal as an input to a sample/hold device, and then it needs to know which signal is connected to the output line. Thus, each channel has its own binary address signal. For example, if the address signal is 10, the channel line two is switched on to the output line. In this manner, the multiplexer output has a series of sampled signals taken from different channels at different times, as illustrated in the ﬁgure. It is noted that the channels are addressed in the order, 0, 1, 2, and 3, and that DT is the sampling

V8 V7 V6 input signals V5 V4 V3 V2 V1

S/H 8-channel mux devices

FIGURE 4.22 Multiplexer with simultaneous sampling.

A/D

multiplexer switch

202

Optomechatronics

channels

0

multiplexer (MUX)

∆T

1

0 1 2 3 4

2

multiplexed signal

3 signal conditioners

∆T

solid state switch

address decoder

S/H

0 1 2 3 4

sample & hold signal

control signal

FIGURE 4.23 Time division multiplexing.

interval. When the multiplexed signals are fed to the sample hold device, it produces the signal types shown in the right hand side of the ﬁgure. Wheatstone Bridge The bridge circuit is the most common unit which can be used for obtaining the output of transducers. Of all possible conﬁgurations, the Wheatstone bridge is used to the greatest extent. It converts a resistance change in response to a voltage change. As shown in Figure 4.24, the bridge consists of four resistance arms with a constant electrical source (DC), and a meter M. Due to a constant supply voltage this type is called a constant voltage Wheatstone bridge. When the bridge is balanced, i.e., Vout is zero, no current may ﬂow through the meter, which gives iout ¼ 0: Then, the current i1 must be equal to i2 , whereas the current i3 must be equal to i4 : In addition, the potential difference across R1 must be equal to that across R3 , or i1 R1 ¼ i3 R3 and similarly, for the potential difference across R2 and R4 , i2 R2 ¼ i4 R4 :

B R1

R2 i2

i1

A

i3

R4

R3 D

Vs FIGURE 4.24 Constant-voltage Wheatstone bridge.

C

i4

M

Mechatronic Elements for Optomechatronic Interface

203

We therefore obtain the condition for balance R1 R ¼ 3 R2 R4 When this bridge undergoes change in one of the resistances, the balanced condition cannot be satisﬁed. This means that the output voltage Vout is no longer zero. The Vout under this condition can be obtained in the following manner: Since Vout ¼ VAB 2 VAD , the use of Ohm’s law yields Vout ¼ Vs

R1 R3 2 R1 þ R2 R3 þ R4

ð4:23Þ

If resistance R1 changes by an amount DR1 , Vout will accordingly change to Vout þ DVout : This relation can be written by Vout þ DVout Vs

8 > > <

DR1 R2 R3 1þ 2 R1 R1 R4 ¼ > DR R R3 1 2 > : 1þ þ 1þ R1 R1 R4

9 > > = > > ;

ð4:24Þ

When all resistances are initially equal and Vout ¼ 0, the relation in Equation 4.24 may be simpliﬁed as DVout DR1 =R ¼ Vs 4 þ 2ðDR1 =RÞ

ð4:25Þ

where R ¼ R1 ¼ R2 ¼ R3 ¼ R4 , which is a widely used form. Equation 4.25 shows that this type of resistance bridge exhibits nonlinear behavior, but can be considered to be linear, since usually DR1 is very small for most applications. Isolator When a circuit carrying high current or high voltage is connected to the next circuit, the connection may cause the possibility of damage. This can be protected in various ways. One means of such protection is to use an isolator between two circuits. The isolator is a circuit that completely isolates circuits, while keeping them electrically isolated from each other. Figure 4.25 shows a typical isolator circuit using an operational ampliﬁer, and optical signals. The noninverting ampliﬁer used here is a voltage follower discussed previously whose gain is unity. This implies that circuit 2 follows the output of circuit 1 which is the input to the op-amp. The electrical signal at the output of circuit 1 is converted into an optical signal by a light emitting diode (LED). The converted light signal is in turn converted again into the corresponding electrical signal at the output of circuit 2 by using a photo detector such as a photodiode or phototransistor.

204

Optomechatronics

photodiode LED circuit 2

circuit 1 Vin

Vout

light

optoisolator

(a) an isolator using op−amp

(b) optoisolator

FIGURE 4.25 Isolation via optoisolator.

Microcomputer System The microcomputer or microprocessor has become a vital and indispensable tool to solve almost all engineering problems, in practice, signal conditioning, modern complex control or automation problems. Furthermore, when it is interfaced with sensors, actuators, and artiﬁcial intelligent tools, it makes modern machines, processes or products or systems smart so that human intervention can be greatly reduced. This trend will be accelerated in the future, because computers will be equipped with smart software in addition to their capabilities interfacing with intelligent hardware tools and systems. In this section, we will brieﬂy discuss the basics of microcomputers in terms of hardware structure. Microcomputer Basically, most of the computers consist of the following essential units and circuitry: (1) (2) (3) (4)

the central processing unit (CPU) memory bus input and output interfaces and devices.

The CPU is a microprocessor, a semi conductor VLSI (very large scale integrated circuit) chip. The main objective is to recognize and carry out program instructions by fetching and decoding them. As shown in Figure 4.26, it consists of control units, an arithmetic and logic unit (ALU), registers accumulator, program counter, and other auxiliary units such as memory controller, interrupt controller, and so on. The control unit determines the timing and sequence of operations involving fetching a program instruction from memory and executing it. The ALU is a logic circuit which can perform arithmetic and logical operations such as addition,

Mechatronic Elements for Optomechatronic Interface

EPROM EPROM EEPROM EEPROM

address bus

205

control bus

RA M ROM

microprocessor ROM RAM

device select lines

I/O interfore unit

data in

data out

data bus

FIGURE 4.26 Schematic diagram of a micro-computer.

subtraction, and logical AND/OR. The microprocessor has several registers such as the status registers containing information on the latest processed results carried out in the ALU, the address register containing the address of data to be manipulated, the instruction register storing an instruction received from the data bus and some other registers. The accumulator is the data register where data to be sent to the ALU is temporarily stored. The program counter is a register which contains the address of the instruction to be executed next. The memory unit is used for several purposes such as storing the instructions simply in numbers and holding data to be processed, immediate answers and the ﬁnal results of a calculation. The size of the memory unit is speciﬁed by the number of storage locations available, for example, 1 k memory (10 bit address signal); 210 ¼ 1024, 64 k ¼ 216 ¼ 65536 locations. ROM (read-only memory) is a permanent memory where no data can be written, but the data can only be read. In this memory, programs that include computer operating systems and dedicated microprocessor applications are stored. RAM (random access memory) is the memory which information can be written into or read out of. It is used for temporary deposits and withdrawals of data or information required for the establishments of programs. The stored information is lost when the power supply is switched off. EPROM is the erasable and programmable ROM. The program is stored by applying voltages to the speciﬁed pins of the integrated circuit and producing a pattern of charged or uncharged cells. Erasing the pattern can be done by optical means using ultraviolet light. A bus is a transmitting media consisting of a group of lines. It ties the CPU memory and input – output (I/O) device together by three sets of wire lines. The ﬁrst wire set, data bus, is used to carry data between the CPU and the

206

Optomechatronics

memory or I/O channels. This bus also transports the instructions from the memory to the CPU with a view to make them executed. The data bus is bidirectional, allowing signals to ﬂow in and out from the CPU. The second set of wires is an address bus that carries a set of signals known as addresses. This address signal indicates which particular memory location is to be selected so that data may be transferred between the CPU and that location. In this situation, only the location is opened to the communication from the CPU. It is noted that on the address bus data always ﬂows from the CPU to the external devices. Most of the popular CPUs typically employ a 16 bit line address; having 64 K locations; 216 ¼ 65536, but some other types use 20 or 24 address lines. The third set is a control bus which carries a variety of timing and control signals out from the CPU. This bus also carries control signals generated by the control and timing units to the CPU, which synchronize and control each internal operation inside the computer. For example, read and write control signals generated by the CPU are used to indicate which way the data is to ﬂow across the data bus and when the data is to be transferred from memory to the CPU and vice versa. As shown in Table 4.4, the binary states of simple logic signals can be used to tell which mode is enabled. In and out control signals generated by the CPU specify whether an input or an output operation is to be performed. In addition, these specify when the addresses and data are valid for operation, as indicated in the table. The CPU interrupts the program being performed and begins execution of the Int signal when it receives an interrupt signal (Int). The functions of the TABLE 4.4 Control Signals for Some Modes of Operation Operation Mode Read 0 0 1 1

Write

Control Signal Function

0 1 0 1

No memory read/write operation requested Memory address and data are valid for write operation Memory address is valid for read operation Not assigned In/Out Control Signal

0 0 1 1

0 1 0 1

No I/O is to be operated I/O address and data are valid for sending out signals I/O address is valid for taking in external signals. Not assigned INT Control Signal

0 1

Not interrupt requested Interrupt requested

Mechatronic Elements for Optomechatronic Interface

207

interrupt signal are performed according to the operation mode, as shown in the table. It is noted that all of these signals are carried by the control bus. Input/Output Interface Input/output devices are connected to the CPU through three bus lines. This enables the CPU to interact with the devices to perform a basic operation or to send out the executed results. All of these interactions occur through the I/O ports whose pins are used for external connections of inputs and outputs. Ports are places where loading and unloading of data take place. The ports can be classiﬁed into input port or output port or programmable port depending on whether the signal is input or output. The peripheral (I/O) devices are keyboard, monitor, line printer, memory, A/D, and D/A. In control and instrumentation applications, the inputs may be signals from sensors. These signals are sampled via A/D for computer. In the case of instrumentation, the computed results may be used for analysis of the process being instrumented. In process control, the results will often be converted to analog signal via a D/A converter in order to meet the speciﬁcation required by actuators involved with control. Microcontrollers The microprocessors and microcontrollers are similar but as far as applications are concerned, these are slightly different in architecture. Microprocessors are primarily used for high speed computing applications. In contrast, the microcontroller is designed to satisfy the applications of signal processing, engineering measurements, industrial automation, system control and so forth. The microcontroller is a digital integrated circuit in which several functions are all brought together on a single chip which includes the CPU, ROM, RAM, ADC, DAC, and serial and parallel ports. From this point of view, the microcontroller is the integration of a microprocessor with memory I/O interface timer, and the converters (ADC and DAC). Sampling of a Signal Sampling of a continuous time signal is a basic process for computer assisted analysis, instrumentation, and control of processes and systems because of the discrete-time nature of the digital computer. Sampling a signal means to replace the signal by its values in a discrete set of points. If sampling is done at an equal time interval Dt, as indicated in Figure 4.27, then the sampling instants are equally spaced in time, therefore the kth sampling time is expressed by tk ¼ kDt;

k ¼ 0; 1; 2· · ·n

ð4:26Þ

If a time-varying continuous signal is represented by f ðtÞ, the sampled version of the signal may be expressed by f ðtk Þ: This is called a sampled

208

Optomechatronics

signal amplitude

f(t )

O 0 ∆t 2∆t 3∆t

k∆t (tk )

(t1) (t2) (t3)

time (t )

FIGURE 4.27 Sampling a continuous signal f ðtÞ:

sequence often referred to as a time series. In this case, f ðtk Þ is said to be sampled with a frequency of fs which is called sampling frequency or sampling rate; fs ¼

1 Dt

ð4:27Þ

amplitude (V)

amplitude (V)

amplitude (V)

amplitude (V)

To see the effect of the sampling frequency, let us consider a 0.16 Hz sine wave which is plotted versus time over time period 20 sec, as shown in 1 0 –1

0

5

10

15

20

5

10

15

20

5

10

15

20

10

15

20

(a) original signal (frequency: 0.16 HZ)

time (t ), sec

1 0 –1

0

time (t ), sec

(b) fs = 1 HZ

1 0 –1

0

(c) fs = 0.37 HZ

time (t ), sec

1 0 –1

0

5

(d) fs = 0.25 HZ

FIGURE 4.28 Sampling a sinusoidal signal with various frequencies.

time (t ), sec

Mechatronic Elements for Optomechatronic Interface

209

Figure 4.28a. This sine wave is sampled with three different frequencies and the results are shown in Figure 4.28b to Figure 4.28d: (b) fs ¼ 1 Hz (c) fs ¼ 0:37 Hz (d) fs ¼ 0:25 Hz: It is apparent that as the sampling frequency decreases the sampled signal loses the original shape of the continuous sine wave. If sampling frequency is very low, that is, sampling rate is too slow, then the sampled signal appears to be entirely different from the original, having a frequency much lower than that of the original. From this observation we can see that the frequency content of the original signal can be reconstructed accurately only when the sample frequency is higher than the twice the highest frequency contained in the analog signal. That is, fs . 2fmax

ð4:28Þ

where fs is the maximum frequency in the analog signal. This is known as Nyquist criterion. In terms of sampling time interval, Dt ,

1

ð4:29Þ

2fmax

In converting a continuous signal to discrete signal form Equation 4.28 provides a criterion for the minimum sampling frequency. Similarly, Equation 4.29 represents a criterion for the maximum time interval. Whenever a signal at a sample rate is less than 2fs , the resulting discrete signal will appear false differently from the original signal due to misinterpretation of the high frequency content of the original. This phenomenon is referred to as aliasing and the false frequency is called the aliasing frequency. Aliasing occurs when fa ,

1 f 2 s

ð4:30Þ

where fa is the aliasing frequency given by fa ¼ lfmax 2 nfs l

for integer n

ð4:31Þ

For example, suppose that a signal is composed of f ðtÞ ¼ A1 sin 2pð120Þt þ A2 sin 2p ð120Þt If this signal is sampled at a rate of 125 Hz, the frequency content of the resulting discrete signal would be fa ¼ l120 2 125l ¼ 5 Hz

210

Optomechatronics

Therefore, the sampled signal appears to be oscillating at frequency of 5 Hz. The aliasing problem can be prevented when a low-pass ﬁlter is placed before the signal enters ADC.

Actuators Electric Motors In view of operation principle, electric motors may be categorized into three basic types of actuators; DC, AC, and stepper motor. DC motors utilize DC current source to produce mechanical rotation, while AC motors consume alternating electrical power to generate such motion. In DC motor, depending upon what type of magnetic ﬁelds (permanent magnetic type or electromagnet wound ﬁeld type) is used, these are divided into two; permanent magnetic DC motor and electromagnetic type or wound ﬁeld DC motor. Stepper motors steps by a speciﬁed number of degrees according to each pulse the motor receives from its controller. We will here discuss DC motors only. Figure 4.29a depicts a schematic of a DC servomotor, in which the basic components are included. The stator has magnets which produce an electrical ﬁeld across the rotor. The rotor is wound with coils through which current is supplied through the brushes. The brush and commutator work together. They are in contact and rub against each other during rotation, as shown in Figure 4.29b. Due to this arrangement, the rotating commutator sends the current through the armature assembly so that current passes through the rotor. Two principles are involved in the motor motion: the motor law and generator law. In case of the motor law, Fleming’s left hand rule is applied to indicate the directions of the electric current. When all three ﬁngers are kept

rotor

stator magnet stator magnet

stator magnet

motor shaft

magnetic flux

coil

ia F

N

brush commutator

B

FIGURE 4.29 Electric DC motor.

stator magnet

ia

S

brush coil (electric conductor)

commutator ia

ia Va

(a) schematic of DC motor

F

(b) principle of DC motor rotation

Mechatronic Elements for Optomechatronic Interface

211

at right angles to each other, magnetic force (F) is indicated by the thumb, magnetic ﬂux (B) by the index ﬁnger, current ðia Þ by the middle ﬁnger. Because the current passing through the rotor coils (conductor) is in the magnetic ﬁeld generated by the stator, the rotor will receive the electromagnetic force called the “Lorentz force” which is given by F ¼ B‘c ia

ð4:32Þ

where B is the magnetic ﬂux density in tesla, ia is the armature current in amperes, and ‘c is the effective length of the coil in meters. The above equation converts electrical energy within a magnetic ﬁeld into mechanical energy so that a rotor can do work. On the other hand, the law of generator as mentioned describes a relation describing the effect of mechanical motion on electricity. The law states that when the rotor coils (conductor) are moving in the magnetic ﬁeld in the direction perpendicular to the speed v, it generates an electromotive force (emf) Ve in volts which is expressed by Ve ¼ B‘c v

ð4:33Þ

This Ve can cause the corresponding current to ﬂow in an external electric circuit. Based on the observation of these two laws, we can model the dynamics of the motor rotating in the magnetic ﬁeld by considering the torque acting on the rotor. T ¼ 2B‘c ia r ¼ kt ia where r is the radius of the rotor, and kt is the torque constant. The voltage generated due to the rotation of the rotor in the magnetic ﬁeld is expressed by Ve ¼ 2B‘c r

du du ¼ 2kb dt dt

ð4:34Þ

where du=dt is the angular velocity and kb is the back emf constant. Referring to Figure 4.30, we have for the electrical side La

dia du þ Ra ia ¼ Va 2 kb dt dt

ð4:35Þ

where Va is the supply voltage to the armature and for the mechanical side I

d2 u du þb ¼T dt dt2

ð4:36Þ

where I is the moment of inertia of the motor, b is the viscous damping coefﬁcient of the motor, and T is torque induced by a supply voltage. When all parameters are known, the angular speed and position of the motor can be determined from the above Equation 4.35 and Equation 4.36. As we shall see later, the motor parameters I, kb , and b determine the dynamic characteristics of the motor.

212

Optomechatronics Ra

La

Va

I

T

ia Ve

(a) electrical model

b

θ T

(b) mechanical model

FIGURE 4.30 Schematic diagram of an armature controlled DC motor.

Piezoelectric Actuator As mentioned earlier, a piezoelectric material deforms or produces a force when it is subject to an electrical ﬁeld. This concept is a basis of piezoelectric actuator. The relationship between electrical behavior and mechanical behavior has already been treated in piezoelectric transducer but we will brieﬂy discuss it here again to describe basic deformation modes. The relationship can be approximately described when piezoelectric materials are free of applied loads. It is given by S ¼ dij E

ð4:37Þ

where S is the strain in the piezo material, E is the electric ﬁeld strength, and dij is the coupling coefﬁcient between S and E, which is the piezoelectric charge constant. To get familiar with the physical meaning of the relation, let us consider two basic deformation modes as shown in Figure 4.31. The unchanged dimension of the piezoelectric material is shown in Figure 4.31a, whereas Figure 4.31b and Figure 4.31c, respectively, show the axially and transversely elongated states due to the electric ﬁeld E applied as indicated in the speciﬁed direction. According to the indicated directions of both electric ﬁeld and strain, dij in Equation 4.37 can be identiﬁed. The axial (thickness) mode has dij ¼ d33 , because in this case both directions are directed along axis 3. The transverse mode (longitudinal) has dij ¼ d31 , because the electric ﬁeld is in the direction of axis 3, while the strain direction is along axis 1.

electrode E 3

h 2

h+∆h 1

(a) original (unchanged)

(b) axial expansion

FIGURE 4.31 Piezoelectric effect: longitudinal and transverse modes.

electrode E _ +

+ _

+∆

(c) transverse expansion

Mechatronic Elements for Optomechatronic Interface

213 do di

z

+x −y

+y d

−x −x −y

(a) stack

(b) bimorph

(c) tube actuator

FIGURE 4.32 Bimorph conﬁguration.

The other deformation modes such as thickness shear and face shear can also be treated in this way. Generally, a single piezoelectric actuator employing the above conﬁguration produces a very small deformation, and therefore a variety of structural conﬁgurations are available, which yield relatively large deformations compared to that of single structure. Figure 4.32 indicates three such actuators which include stack, bimorph, and tube types. The stack type is composed of N axial piezo actuators and thus increases N times the deformation of single actuator. The bimorph actuator consists of two similar transverse actuators. It has a cantilever beam conﬁguration which has a large bending mode when subjected to transverse load. These actuators deform in opposite way and therefore produce a large bending. This bimorph actuator is of great importance in piezoelectric microactuator to be discussed later. The cylindrical hollow tube type has outer diameter do and inner diameter di : It is radially polarized and the outer electrode surface is divided into the quadrant. Each of two surfaces has an electrode; þ x and 2 x or þ y and 2 y. When two opposite electrodes are subject to opposite electric ﬁeld, it bends. Thus, these give two directional motions in both x and y directions. On the other hand, the z motion is generated when all electrodes are subjected to the same electric ﬁeld. Based on the linearized portion of the relationship given in Equation 4.37, Table 4.5 describes the deformation of various conﬁgurations of piezo actuators discussed in the above. The ﬁrst two denote the single actuator while the other three represent multiple actuators. In the table Vp is an applied voltage and dij is the coupling piezoelectric coefﬁcient. From the relations the applied force can be computed by considering the fact that the piezo generated static force Fp is proportional to the deformation. From Equation 4.6 the force is related to Fp ¼ CF VP where CF is the force sensitivity constant and given by 1 1A CF ¼ 0 r Sq h

ð4:38Þ

214

Optomechatronics

TABLE 4.5 The Displacement of the Piezoelectric as a Function of the Applied Voltage VP Conﬁguration

Mode

Deformation

Geometry electrode E

Axial

Longitudinal (axial)

_ +

D‘ ¼ d33 VP +∆ ‘

electrode E

Transversal

Transversal

Dz ¼

Stack

Longitudinal

D‘ ¼ nd33 VP

Transversal

L2 d ¼ 3 2 d31 VP h

h

d31 VP

_ +

z+∆z z

Bimorph

`

h

δ

do

di

Tube

Transversal

d¼

2L d V d0 2 di 31 P

+x −y

+y

−x −x −y

The relationship in Equation 4.38 shows that the actuation force is proportional to applied voltage. From the discussions we have made in the above it can be summarized that, when a piezoelectric material undergoes a change in geometry due to an applied voltage, the amount and direction of its deﬂection depend upon the type of piezo electric material and the geometry of the material. Due to this complexity involved, the piezo material exhibits unfavorable nonlinearity between strain and applied voltage, hysteresis, creep, and aging. To reduce or eliminate this inherent nonlinearity there are several methods developed to date. The most frequently used methods are table-loop-up based on feedback control compensation. In general, feedback compensation results in fairly accurate correction of nonlinearity, leaving only 1% nonlinearity. A general feedback control concept will be discussed shortly. So far, we have considered the static relationship between the deformation and electric ﬁeld under the assumption that the piezoelectric element will reach equilibrium after undergoing an initial deformation as given in Equation 4.37. However, the piezo element is essentially a moving element, so its motion will be governed by certain dynamic characteristics. The piezoelectric ceramics has inherently a spring element kp and an internal

Mechatronic Elements for Optomechatronic Interface electric load applied

215

electrode z z

Vp

bp

kp

Fp= CF (Vp - Ve )

piezoelectric ceramic

(a) electric load applied to a piezoelectric element

(b) equivalent mechanical model

FIGURE 4.33 Equivalent mechanical model of the piezoelectric actuator.

damping element, bp, and thus these need to be taken into account when its dynamic motion is considered in some particular cases. Figure 4.33a shows its equivalent dynamic model composed of a spring-mass-damper system. When an external load Fl is applied to a piezoelectric element, the dynamics of the element can be written as mp

d2 z dz þ bp þ kp z ¼ Fp þ Fl 2 dt dt

ð4:39Þ

where z is the displacement of the piezo actuator in the vertical direction, mp, bp and kp are the effective mass, damping, and elastic stiffness of the piezo ceramics, and Fp is the effective force developed due to the applied net input voltage to the element that can be obtained from the piezoelasticity theory. The ﬁrst term in the right-head side of Equation 4.39 results from the inverse piezo-effect Fp : Figure 4.33b indicates that although an input command voltage Vp is applied to the actuator, this does not produce the corresponding strain due to the “piezo effect” by the external load. Upon consideration of this effect, effective Fp can be written as Fp ¼ CF ðVp 2 Ve Þ where CF is a piezoelectric force sensitivity constant converting from electric voltage to force, Vp and Ve are the command input voltage and the voltage induced due to external load, respectively. The above Equation 4.39 can be rewritten as mp

d2 z dz þ kp z ¼ CF ðVp 2 Ve Þ þ Fl þ bp dt dt2

ð4:40Þ

This equation of motion describes the displacement of the actuator for a given input voltage, Vp , and external load, Fl. The natural frequency and damping ratio are determined by the system parameters mp, bp and kp. The displacement of the actuator is usually very small and usually ranges within a few mm.

216

Optomechatronics displacement x current i

coil force F

coil magnet yoke

+

R

L

Ve

V

N S

i

−

magnetic flux

(a) voice coil motor

(b) electrical circuit of the VCM leaf spring

yoke coil

base

moving part magnet

(c) actual system configuration FIGURE 4.34 Conﬁguration of voice coil motor.

Voice Coil Motor (VCM) The name “voice-coil motor” originates from the fact that the inventor got the idea for this motor from a speaker. Figure 4.34a illustrates a simple construction of a typical actuator system driven by a VCM, which can freely move in the direction parallel to a mounting rail. The motor is composed of a moving coil (bobbin), two magnets, and a spring-damper system that nulliﬁes the effect of any external disturbance and stabilizes the bobbin motion as well. For clarity, the spring and damper are not shown. The working principle of VCM is the same as that of the DC motor discussed previously and revisited here for better understanding: When a current ﬂows in the coil, this causes Lorentz force which moves the coil. The direction of motion is determined by the direction of the current in the coil and that of the magnetic ﬂow according to Flemming’s left hand rule. In more detail, the magnet M1 generates a magnetic ﬁeld in the direction of the arrow (x direction) as indicated in the ﬁgure. When the bobbin is placed within a magnetic ﬁeld with magnetic ﬁeld density B, force F in the x vertical direction is generated according to Fleming’s left hand rule. The force F is given by F ¼ nBi‘c

ð4:41Þ

where n is the number of coil turns, B is the magnetic ﬂux density in tesla, i is the current in ampere ﬂowing in the direction perpendicular to the paper and ‘c is the coil effective length in meters. Figure 4.34b shows the electrical

Mechatronic Elements for Optomechatronic Interface

217

circuit of the VCM which is written by L

di þ Ri ¼ V þ Ve dt

ð4:42Þ

where L and R are the inductance and resistance of the coil, respectively, V is the electrical voltage applied to the coil and Ve is the back emf of the bobbin, respectively. The inductance L can be obtained by Faraday’s and Ampere’s laws and is given by L¼

10 n2 Ac ‘c

where Ac is the cross-sectional area of the coil, and 10 is the permeability of the air. And the resistance R is deﬁned by R¼

r ‘c Ac

where r is the resistivity of the conductor. The back emf is given by Ve ¼ 2nB‘c

dx dx ¼ 2kb dt dt

ð4:43Þ

where dx=dt is the velocity of the bobbin in the upward direction. Rewriting Equation 4.42 by use of Equation 4.43, we have L

di dx þ Ri þ kb ¼V dt dt

ð4:44Þ

Therefore, when the bobbin is supported by a spring as shown in Figure 4.34c, the dynamic equation governing the motion of the bobbin can be written by m

d2 x dx þb þ kx ¼ F dt dt2

ð4:45Þ

where m is the mass of the bobbin, and b existing between the guide and bobbin and k are the damping coefﬁcient and the stiffness of the bobbin, respectively. The actuating force generated by the VCM can adjust the displacement of the bobbin to a desired accuracy, depending upon the choice of these values. We will return to VCM in more detail in Chapter 6. Typical magnetic conﬁgurations of linear VCM are shown in Figure 4.35. These are either cylindrical or rectangular in conﬁguration. They have some differences in such characteristics as gap ﬂux density, demagnetization effect, and leakage ﬂux between the magnets and other elements, shielding the leakage. For instance, the outer magnet long coil shown in Figure 4.35a has high leakage between the center yoke and the magnets, whereas inner magnet long coil shown in Figure 4.35b has low leakage factor due to its conﬁguration. Figure 4.35c has low leakage factor and coil inductance in comparison with that of the conﬁguration shown in Figure 4.35d which is the

218

Optomechatronics

N

side yoke

side yoke

S

coil center yoke

N S permanent magnet (a) outer magnet long coil

coil center yoke

N S

(b) inner magnet long coil

N

S

N

S

N

S

N

S

(c) steel plate pole

(d) enclosed magnet long coil

FIGURE 4.35 Conﬁguration of various voice coil motors.

enclosed magnet conﬁguration. This conﬁguration outperforms the previous ones in leakage and uniformity of magnetic ﬁeld along the path of the coil. Electrostatic Actuator This actuator utilizes an electrostatic force generated by two electrically charged parallel plates. The electrostatic energy stored in this actuator is expressed by U¼

1 CV 2 2

ð4:46Þ

where V is the potential difference between the electrodes and C is the capacitance between two electrodes given by C ¼ 10 1r

A d

ð4:47Þ

In the above, A is the area of opposing electrodes and d is the separation of the electrodes. Consideration of this relation has been already given in the capacitance sensor. Utilizing the relation in Equation 4.46, we can obtain the force acting in an arbitrary direction as Fr ðrÞ ¼

›UðrÞ ›r

ð4:48Þ

Let us elaborate the concept of the electrostatic actuator in two directions. Figure 4.36 illustrates two directional forces that can be actuated by this drive mechanism: (1) overlapped area driving force, (2) gap closing

Mechatronic Elements for Optomechatronic Interface

V

219

V z

z x

(a) overlapped area driving force

(b) gap closing force

kc

bc

V

zs

z

(c) dynamic model of the electrostatic actuator FIGURE 4.36 Electrostatic force actuator.

force. The area driving force shown in Figure 4.36a can be derived using Equation 4.48 and written as Fx ¼

10 1r wV 2 2z

ð4:49Þ

where V is the input voltage applied to the set of the plates, x is the overlap distance whose coordinate is shown in the ﬁgure, and w is the width of actuator. The gap closing force per gap is illustrated in Figure 4.36b. The force can be obtained in a similar way as derived in the case of the comb driving force, which can be derived as Fz ¼ 2

10 1r wV 2 2z2

ð4:50Þ

It is noted that the gap closing force increases drastically for a given input voltage V as z decreases. Equation 4.49 and Equation 4.50 present the relationship between the input voltage and the output for a speciﬁed gap z or overlap distance x. When this actuator gives an actuating force to a mechanical element, one of the capacitor upper plates moves in z or x directions [19]. Figure 4.36c illustrates the case when the capacitor generates a z-directional force while constrained to a spring kc and a damping element bc. Upon consideration of this dynamic conﬁguration, the equation governing the dynamics of the capacitor plate may be written as mc

d2 z dz 1o AV 2 þ k ðz 2 z Þ ¼ 2 þ b c 0 c dt dt2 2z2

ð4:51Þ

220

Optomechatronics

where A is the charge area of the capacitor and z is the gap between the two plates, which is measured downwards from the equilibrium position, z0. It is given by z ¼ z0 2 zs where z0 is the equilibrium position at which no spring force and capacitor charge occur, and zs is the displacement caused by the spring. This indicates that when z ¼ 0, the displacement of the plate, z, becomes zero. It is important to observe that the driving force appearing in the right hand side of the above equation is a function of displacement z. Examination of this equation shows that at equilibrium the electrostatic force pulling the plate down and the spring force must be equal. This implies that the net force Fn which is the difference between two forces must be zero, which is given by Fn ¼ 2

1AV 2 2 kc ðz 2 z0 Þ 2z2

In order to see the stability of this point, we now differentiate Fn with respect to z, we obtain dFn 1AV 2 2 kc ¼ dz z3

ð4:52Þ

The stability of this point must be dependent upon the sign of the above equation. In order for the capacitor plate to be at a stable equilibrium, the variation of the net force with respect to displacement z must be dFn =dz , 0, which leads to kc .

1AV 2 z3

ð4:53Þ

Detailed discussion on the behavior of the plate dynamics is left for a problem in Chapter 4. We will discuss the dynamic characteristics of this actuator in detail in Chapter 7.

Microactuators There are two categories of microactuators: one is the actuator that drives mechanical elements within micro range regardless of its physical size. The other is the actuator whose physical scale is limited to micro dimension. A number of these microactuators have been developed in the past. Microactuators to be discussed here will include only three actuators that operate on orders up to a few hundred mm. These actuators are made of shape memory alloy, magnetostrictive material, and ionic polymer metal composite. Table 4.6 compares a group of microactuators in terms of deformation range, frequency ranges, and force and so on.

Mechatronic Elements for Optomechatronic Interface

221

TABLE 4.6 Properties of Various Micro-Displacement Actuators Actuator Technology Piezoelectric (BM500) Shape memory (Nitinol) Magnetostrictive (Terfenol-D) Electrostrictive (PMN)

Typical Displacement

Force

Hysteresis

Frequency Range

100 mm (L ¼ 5 cm)

20 kN

8 , 15%

,30 kHz

500 mm (L ¼ 5 cm)

500 N

10 , 30%

,5 Hz

100 mm

1.1 kN

,10%

,4 kHz

65 mm

9 kN

1 , 4%

,1 kHz

Operating Temperature Range 220 , 2508C (Tc ¼ 3658C) up to 4008C (thermal actuation) up to 3008C (Tc ¼ 3808C) 0 , 308C (Tc)

Source: Prasad, E. Sensor Technology limited, 2002.

Shape Memory Alloy (SMA) Actuator SMAs such as titanium nickel alloy (TiNi) are smart materials, which possess thermoelastic martensitic transformation exhibiting shape recovery phenomenon when heated: The alloy has two main phases associated with the upper memory recovery, austenite (high temperature phase) and martensite (low temperature phase). Physically speaking, this means that, when a SMA is stretched from its undeformed state at some temperature below its transformation temperature (high temperature phase), it has the ability to return to the undeformed original shape upon heating. This characteristic results from a transformation in its atomic crystal structure. As shown in Figure 4.37, the martensite state possesses the twinned structure easily to deform while the austenite state has the very rigid cubic structure. This atomic structural change from the twinned martensite to the austenite causes the SMA to generate a large force which can be a very useful property as an actuator. Figure 4.38a depicts

crystal lattice of austenite

FIGURE 4.37 Phase transformation of shape memory alloys.

twinned crystal lattice of martensite

Optomechatronics

contraction,

∆

martensite (finish) austenite (start) cooling

∆T

heating

austenite (finish)

tensile force, F

222

cooling

∆T heating

martensite (start) Tm

temperature (T )

0

Ta

(a) contraction vs. temperature

temperature (T )

Ta

(b) tensile force vs. temperature

FIGURE 4.38 Typical transform hysteresis of shape memory alloys.

the physics involved with phase transformation and shows the relationship between the deformation and the temperature applied. As temperature rises, the deformed material in martensite phase starts to change its phase to austenite. At the temperature Ta , the phase of material is completely turned into austenite. At this phase, the material recovers its length (shape), whose deformation is reversed. It is noted that during this period, elongation is a slightly nonlinear function of temperature. When the material in austenite phase cools from the high temperature Ta , it does not follow up the relationship curve, but the lower curve, slightly deviated from the upper curve. This indicates it undergoes a typical hysteresis effect. As the temperature keeps decreasing, the phase of the material is completely changed into martensite, at temperature Tm : At this phase, the material reaches the deformed state again. This shape recovery property creates a wide range of application areas as an actuator. The drawbacks of this actuator are: nonlinearity due to the hysteresis and slow response. Some of these adverse effects may be partially compensated by applying some advanced control algorithms. In MEMS applications, a variety of methods depositing this SMA ﬁlm over silicon are used. In this case, actuation is due to the recovery of residual tensile stress in the ﬁlm. Figure 4.38b shows the relationship of tensile stress vs. temperature for TiNiCu ﬁlm material. As the ﬁlm material deposited at temperature Ta cools down, its phase is changed from austenite to martensite. Below temperature Tm , it can be seen that the thermal stress in the ﬁlm is almost relaxed. The reverse transformation occurs when it is heated from Tm to Ta and causes the ﬁlm to recover its original shape. Magnetostrictive Actuator Magnetostrictive materials found in 1970s transduce magnetic energy to mechanical energy when these are subjected to an electromagnetic ﬁeld

Mechatronic Elements for Optomechatronic Interface

223

as indicated in Figure 4.39a. These materials also generate electromagnetic ﬁelds when they are deformed by an external force. Therefore, magnetostrictive materials can be used for both actuation and sensing due to this bidirectional coupling. The advantage of this actuator is that it readily responds to signiﬁcantly lower voltage as compared with piezoelectric actuators (200 to 300 V). Magnetostrictive materials can be deposited over a silicon micromachined cantilever beam as shown in Figure 4.39b when they are used as a micro actuator. The actuator consists of a thin ﬁlm and a silicon beam. When it is subjected to a magnetic ﬁeld, the magnetostrictive ﬁlm expands and causes the beam to bend in the vertical direction. The deﬂection d is found to show a dependency of ﬁeld strength (telsa). Typical beam deﬂection [22] is found to be 100 mm at 0.01 telsa for a 10 mm thick terfenol-D ﬁlm over a silicon thickness 50 mm and length 20 mm. The recent applications of these materials can be found from a variety of applications: adaptive optics, high force linear motor, active vibration or noise control, and industrial and medical sonar and pump. Ionic Polymer Metal Composite (IPMC) Actuator The IPMC materials possess the susceptibility to interactions with externally applied electric ﬁelds and also to their own internal ﬁeld structure. Due to this property, when an electrical ﬁeld is applied, the hydrated cations in the materials move to negatively charged electrode side. As a result, the IPMC strip undergoes internal volume change and thus bends towards the anode, negatively charged side, as shown in Figure 4.40. The advantages of this actuator are light weight, relatively large displacement, low input voltage (4 to 7 V) and fast response (msec to sec). The deformation and actuation force are found to be dependent on the applied voltage and geometry of the material. This material can be used as

displacement

(a)

terfenol-D rod

field coil

terfenol-D layer

d (b)

Si

FIGURE 4.39 Conﬁguration of a magnetostrictive actuator.

displacement

224

Optomechatronics

+ – + – + – + – + –

V

+

+ + + + + + – – – – – + – – –

V

FIGURE 4.40 Behavior of an ionic polymer due to an applied electric ﬁeld.

a sensing element as well. When it is bent, the resulting internal stresses developed cause shifting of mobile charges, which can be detected by a low power ampliﬁer. This constitutes an IPMC sensor.

Signal Display The display element is usually the ﬁnal data representation element. There are a number of data presentation devices from simple pointer scale and recorders to very large scale display such as cathode-ray-tube (CRT) and plasma display panel (PDP). Here, a brief discussion will be made, being limited to optical or image display devices. As shown in Figure 4.41, optical displays units can be conveniently categorized into small scale alphanumeric displays and large displays that all work in the digital domain. As far as displaying principle is concerned, it is interesting to see that all of these can be grouped into three categories. The ﬁrst method is to use a light emitting source such as LED. The image optical intensity is directly controlled by controlling input current. The CRT as shown in Figure 4.41a uses this principle, in which deﬂected electrons controllable by a deﬂection system within the device interact with semiconductor material such as phosphors formed in dots. Plasma panels in Figure 4.41b are composed of an array of cells known as pixels. The electrodes between two glass substrates produce a gas in a plasma stage, which is made to react with phosphors in each pixel in discharge region. The resulting reaction causes each pixel to produce a desired image at the screen. The second one does not emit light but uses light incoming from other sources. In the liquid crystal device (LCD) depicted in Figure 4.41c, orientation of molecules remains not random but in certain directions, making the optical effect predominant in those directions. This property makes it feasible to modify the crystal structure and thus optical effects by applying electric ﬁelds to the LCD. The last display device is to use a signal manipulation technique which is totally different from the conventional display methods in the above. As shown in Figure 4.41d it is based on MEMS technology, and has two types

Mechatronic Elements for Optomechatronic Interface

screen

anode focusing

225

bus electrode

front panel

dielectric phosphor cathode ray tube

x-defection

back panel

y-defection

address electrode

(a) CRT

(b) PDP 7 segment electrode patterns

.. . . .. .

5 liquid crystal 6 7 material glass plate with electrode pattern

reflected light incident light

4

3

21

(c) LCD panel displaying numerals

mirror

(d) DMD schematic

FIGURE 4.41 Large scale display units.

of display, one utilizing optical switching and the other utilizing grating diffraction. The device using optical switching is called a digital mirror device (DMD). The device contains hundreds of millions of tiny mirrors which are embedded in one chip, and is based on micro-electro mechanical system technique (MEMS). Its main role is to steer a light beam into a screen (display unit) in order to project an image in a desired manner by a simple mirror switching action. The innovative concept lies in the fact that in contrast to (LCD) the color content of the image and intensity of the projected image can be ﬂexibly controlled by their tiny motion and switching time. Another device that belongs to this category utilizes grating diffraction. It is called grating light value (GLV) and is composed of a number of pixels which in turn are composed of six diffraction grating made deformable by an actuator. From the aspect of technology integration, DMD projector and GLV are of optomechatronic nature, while LCD is based on an optical-electrical (OE) combination having no moving or deﬂecting projection of light. We will discuss DMD and GLV in more detail in Chapter 7.

Dynamic Systems and Control Previously, we have seen a variety of physical systems from simple sensors to quite complicated complex systems. Many such systems involve

226

Optomechatronics

phenomena of more than one of the disciplines such as mechanical, electrical, optical and so on. In this case, we need to consider the physical interaction between the discipline involved, which adds more complexity such as nonlinearity, hysteresis and saturation to physical behaviors of the systems. Looking at the systems from a different view, many systems are considered to be static but some are inherently dynamic in nature. In many cases, static condition assumed for static systems often may not be valid due to some physical conditions involved, but may often need to be treated as dynamic systems, as we have already illustrated for the sensors and actuators treated in the previous sections. When it comes to analysis and control of these dynamical systems, characterization of their behaviors is ultimately necessary. To do this physical modeling we ﬁrst need to take into consideration the properties of physical variables involved and their interaction. Based on this modeling, a control system is to be designed next. Undoubtedly, the performance of the control largely depends on accuracy of the physical model and therefore, modeling needs to be done as accurately as possible. Since most of physical systems encountered in engineering are not ideally simple, some approximation is usually made to make them a linear system which makes controller design relatively much simpler than those not approximated. In this section, we will brieﬂy deal with system modeling, system transfer function, which represents the input – output relationship, and some elementary control techniques. Dynamic Systems Modeling Modeling refers to description of system dynamics in a mathematical form. It involves understanding of the physics involved and mathematical formulation of the physical behavior. Therefore, a fundamental step in building a dynamic model is writing the equations of motion for the system to be modeled. In doing this, we may often ﬁnd it difﬁcult to completely and accurately describe the system dynamics in detail due to the uncertainty and complexity associated with physical interactions between variables. In this situation, we resort to an identiﬁcation method with which the unknown or time varying parameters of the system can be identiﬁed either from experiments or simulations. There are a number of types of system that can be identiﬁed depending on how they behave. These are summarized in Figure 4.42. The lumped model uses lumped variables to represent the systems dynamics while the distributed system uses the partial differential equation in order to describe the system dynamics in both space and time without lumping them. Since the classiﬁcation given in the ﬁgure is fairly standard, one may easily ﬁnd the description of each model in any control-related books. The systems we will deal with belong to a lumped, continuous and deterministic model. We have already modeled the dynamics of various actuators and sensors. Therefore, we will not illustrate many actuator systems, but instead consider

Mechatronic Elements for Optomechatronic Interface

227

system model

lumped system

discrete system

linear system

deterministic system

time varying system

distributed system

continuous system

nonlinear system

stochastic system

time invariant system

FIGURE 4.42 Types of dynamical systems.

three distinct systems in nature: thermal, vibration and ﬂuid positioning system. Thermal Systems Consider a thermal system as shown in Figure 4.43a. It is assumed that the system at temperature T is exposed to the surrounding environment at temperature T0 : These temperatures are assumed to be uniform throughout the system. When this system has a heat energy inﬂow, heat energy ﬂow rate through it will be governed by q¼

1 ðT 2 T0 Þ Rt

where q is the heat energy ﬂow rate, and Rt is the thermal resistance. The net inﬂow into the substance within system causes variation of the temperature of the system. When T0 is constant, the relationship given above can be written as dT 1 ¼ q dt Ct where Ct is the thermal capacity. The above equation indicates that for a given rate of change of heat ﬂow or rate of energy storage q, if the thermal capacitance of the system is large, the rate of temperature change becomes low. The reciprocal of thermal resistance is called “heat conductance.” Spring-Mass-Damper System Many practical systems can be modeled as a spring-mass-damper system if some assumptions are made properly. Figure 4.43b illustrates one such system composed of a two-mass system. From the free-body diagram shown in the ﬁgure, we can model the system as m1

d2 x 1 dx1 dx2 ¼ 2b 2 dt dt dt2

2 k1 ðx1 2 x2 Þ þ F

ð4:54Þ

228

Optomechatronics flow in thermal system

valve (orifice)

heat inflow q1(t ) q2 (t ) heat outflow T(t ) T0(t )

(a) thermal system

piston Q A

x

(c) fluid system F x1

m11 k1

F

m m11

b m21 m

k2

k1 (x1–x2) b dx1– dx2 dt dt

x2

k1 (x1–x2) b dx1– dx2 dt dt

m m211 k2x2

(b) vibration system FIGURE 4.43 Various systems for dynamic modeling.

m2

d2 x 2 dx1 dx2 2 ¼b dt dt dt2

þ k1 ðx1 2 x2 Þ 2 k2 x2

It is important to note that the displacements x1 and x2 are taken from their equilibrium position that arises due to gravitational force. Therefore, this force was not included in the above model. Fluid System The ﬂuid ﬂow shown in Figure 4.43c is designed to move a piston in a desired fashion in the x-direction. The physical laws involved with this ﬂow are continuity, resistance and force equilibrium for the piston. When the ﬂuid ﬂows through the oriﬁce (valve) and enters into the empty area of the piston, due to ﬂow resistance at the valve, the mass ﬂow rate Q becomes Q¼

1 pﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ 1 pﬃﬃﬃﬃ ðP1 2 P2 Þ ¼ DP R R

ð4:55Þ

where R is the oriﬁce ﬂow resistance, and P1 and P2 are the ﬂuid pressures at both sides of the oriﬁce. The ﬂow should meet the continuity law Ardx ¼ Qdt

Mechatronic Elements for Optomechatronic Interface

229

where A is the piston area and r is the ﬂuid density. Due to this ﬂuid ﬂow, the equation governing the motion of the piston is obtained by ADP ¼ m

d2 x dx þb dt dt2

ð4:56Þ

where m and b are the mass and damping coefﬁcient of the piston, respectively. Examination of Equation 4.55 and Equation 4.56 signiﬁes that the model derived is highly nonlinear due to ﬂow through the oriﬁce. However, mass ﬂow rate can be linearized near the normal operation point, Q0 , DP0 , R0 , which will result in a linearized equation. Optical Disk The optical disk shown in Figure 4.44 has two main servo systems composed of a tracking system (radial) and an autofocusing system (vertical). The track following servo is divided into a ﬁne track servo and a coarse motion track servo which are schematically illustrated in the square box. The ﬁne servo system is mounted on the optical pickup and actuated by VCM. As can be seen from the ﬁgure, track-following accuracy is largely dependent on this ﬁne servo system. The operation principle is as follows: the laser beam coming out of the laser diode passes through the oblique glass plate (mirror) which is collected by the objective lens. Depending on the location of the lens in the vertical direction, the collected beam may or may not be focused onto the disk surface. The beam reﬂected from the disk pits travels back through the glass plate and ﬁnally is detected by the photodiode. The objective of the track following systems here is to position the laser spot in radial direction with sufﬁcient precision. We will discuss this in detail in Chapter 6.

x

Ω

optical disk

k

objective lens

fine motion unit

b

photo diode

laser diode mirror coarse motion

FIGURE 4.44 Track following system of an optical disk.

230

Optomechatronics

Let us consider the motion of the ﬁne servo system. It is composed of a mass, a damper, and a spring attached to the VCM. The equation of the motion of the ﬁne servo system is described by m

d2 x dx þb þ kx ¼ F dt dt2

ð4:57Þ

Here, m is composed of the mass of the lens and VCM, b is the damping coefﬁcient, and F is the force induced by disk run-out and imbalance of the disk. The unbalance force comes in due to the fact that the disk itself is not perfectly balanced. It should be noted that, although F is very small, this inﬂuences the overall accuracy of track following, since track following accuracy is required to be highly precise. Dynamic Response of Dynamical Systems Once the system dynamics is modeled in a mathematical form, we now need to investigate how the system will respond to a given input. As we have modeled a variety of dynamical systems in the previous sections, we know that the complexity may differ from system to system. The simplest system we have observed is a ﬁrst order system which is the case of the thermal system. And the next is a second order system, which the rest of the illustrative systems belong to. In this manner, dynamic systems may be in general represented by an nth order system whose dynamics is given by a0

dn x dn21 x dm u dm21 u þ a1 þ · · · þ a x ¼ b þ b þ · · · þ bm u n 0 1 n m dt dt dt1 dtm21

ð4:58Þ

where ai and bi are the system parameters and input parameters, respectively. Here, it is important to note that ai and bi may have two different types; time-invariant and time-variant. The two forms make a lot of difference in system analysis: The time invariant case is allowed to utilize the Laplace transformation method as we shall discuss later, whereas the latter case is not permitted, thus making system analysis much more difﬁcult. At this stage we now identify the given system dynamic model as an inputoutput model, which forms the basic of control system. By deﬁning u as an input and x as an output, we will start analyzing system response with a ﬁrst order system. Let us ﬁrst consider a ﬁrst order system which is described by a differential equation of the form a0

dx þ a1 x ¼ b1 u dt

where uðtÞ is an input, and a0 , a1 and b1 are constants. If we let a0 =a1 ¼ t and

Mechatronic Elements for Optomechatronic Interface

231

1.2 t=1

step response, x (t )

1.0 0.8

t=4

0.6

t=3 t=2

0.4 0.2 0.0

0

2

4

6 8 time (t), sec

10

12

14

FIGURE 4.45 Step response for a ﬁrst order system according to change of time constant.

b1 =a1 ¼ 1 for simplicity, then, this equation is modiﬁed as

t

dx þx¼u dt

ð4:59Þ

where t is called “time constant.” When a unit step input is applied to the system, i.e., u ¼ 1, ðt . 0Þ the steady state solution of this equation is obtained by xðtÞ ¼ ð1 2 e2t=t Þ

ð4:60Þ

which signiﬁes that for a unit step input and zero initial condition the ﬁnal value of xðtÞ reaches unity at a steady state, i.e., at t ¼ 1: The characteristics of the ﬁrst order system differ largely by time constant, t: When t ¼ t the response becomes xðtÞ ¼ ð1 2 e21 Þ: At this time xðtÞ reaches 0.63 of its steady state value as shown in Figure 4.45. In a time of 3t, xðtÞ rises to 95% of the steady state value. Therefore, it is important to note that time constant t can be a measurement indicating speed of the response. When t p 1, the response is very fast, but, when t is relatively large, the response gets sluggish. The second order system which has the most practical importance in real situations takes the form of the following; a0

d2 x dx þ a1 þ a2 x ¼ b1 u dt dt2

where a0 , a1 , a2 and b1 are constants. This can be modiﬁed as d2 x dx þ v2n x ¼ b0 v2n uðtÞ þ 2jvn 2 dt dt

ð4:61Þ

232

Optomechatronics

where 2jvn ¼ a1 =a0 , and a2 =a0 ¼ v2n , b1 =a0 ¼ b0 v2n : In the above, vn is the natural frequency of the system, and j is the damping ratio. The roots of the characteristic equation p2 þ 2jvn p þ v2n ¼ 0 are given by p1 ; p2 ¼ 2jvn ^ pﬃﬃﬃﬃﬃﬃﬃﬃ 2 vn j 2 1: Examination of the roots indicates that the equation can have three different roots depending upon j value; for underdamped case, j , 1, for critically damped case j ¼ 1, for overdamped case j . 1. When the roots j , 1 become complex conjugate, the response of the system becomes for a unit step input and zero initial conditions " ( )# j jvn t cos vd t þ pﬃﬃﬃﬃﬃﬃﬃﬃ sin vd t xðtÞ ¼ b0 1 2 e 1 2 j2 pﬃﬃﬃﬃﬃﬃﬃﬃ where the damped natural frequency is given by vd ¼ vn 1 2 j2 : The above equation reduces to " # e2jvn t xðtÞ ¼ b0 1 2 pﬃﬃﬃﬃﬃﬃﬃﬃ cosðvd t 2 wÞ ð4:62Þ 1 2 j2 j w ¼ tan21 pﬃﬃﬃﬃﬃﬃﬃﬃ 1 2 j2 Figure 4.46 illustrates typical responses for several values of j for b0 ¼ 1:0: As j becomes smaller, the response becomes more oscillatory with larger amplitude while speed of response gets faster. It is seen, however, that the maximum value of the response corresponding to the ﬁrst peak value is increasingly higher with decrease of j value. As we further decrease in j we can anticipate that a sinusoidal response will eventually appear when j ¼ 0: 1.6 x=0.25

1.4 x=0.55

step response, x (t)

1.2

x=0.75

1.0

x=0.10

0.8 0.6

x=0.15

0.4 0.2 0.0

0

2

4

6 8 time (wn t )

10

12

FIGURE 4.46 Step response for a second order system according to changes of j value.

14

Mechatronic Elements for Optomechatronic Interface

233

When j ¼ 1, the roots are equally at 2vn , and the response becomes xðtÞ ¼ b0 ½1 2 e2vn t 2 vn te2vn t

ð4:63Þ

The response shows no oscillation but approaches the steady state value with slower speed than that of the case j , 1: When j . 1, two roots are real and distinct, and the response becomes xðtÞ ¼ b0 ½1 þ Aep1 t þ Bep2 t

ð4:64Þ

where p1 and p2 are the roots of the system. In this case, the response gets sluggish with even slower speed than the case where j ¼ 1: System Transfer Function As we have seen previously, the response of the system dynamics relating an input to an output has been obtained by solving the differential equation analytically. When the system dynamics come to higher order, the solution method must resort to a numerical approach. Since an analytical solution tool is not easily available, this time domain approach makes it difﬁcult to understand the input –output relationship as well as the characteristics of system dynamics until a numerical solution comes out. To avoid this difﬁculty we usually transform a differential equation describing system dynamics into an algebraic equation in Laplace domain or s-domain. This transformation method is called Laplace transform which is an indispensable tool for control system analysis and design. This is because the s-domain transformation provides us with a transfer function that relates the input and output of a dynamic system. Once this is obtained in s-domain, we carry out the necessary calculation in algebraic form. To elucidate the concept in more detail, let us suppose that the input and output variables are transformed into Laplace domain (s-domain), respectively, as L½xðtÞ ¼ XðsÞ;

L½uðtÞ ¼ UðsÞ

where L denotes the Laplace transform of the variable in ½· : Utilizing the transformation, we may write the nth order dynamic equations in Equation 4.58, in a Laplace transformed form. The transfer function of the system describing the relationship between the input and output, can be written as GðsÞ ¼

XðsÞ UðsÞ

ð4:65Þ

where GðsÞ denotes the transfer function. It is important to notice that in transforming the differential equation all initial conditions are assumed to be zero. A block diagram representation GðsÞ is given in Figure 4.47. From this relationship, the response XðsÞ in s-domain can be easily obtained by XðsÞ ¼ GðsÞUðsÞ

ð4:66Þ

234

Optomechatronics G(s) 1 ts+1

U(s)

X(s)

(b) first order U(s)

G(s)

X(s) G(s)

(a) generic representation U(s)

wn2 s2

+ 2xwns + w n2

X(s)

(c) second order FIGURE 4.47 A block diagram representation of system transfer function.

if UðsÞ is given. Therefore, the important properties of system dynamics such as stability and characteristics of system response can be analyzed by manipulating the above algebraic equation in s. If necessary, we can easily go back to the time domain from the s-domain in order to do some analysis simply by inverse-transforming the above equation in s L21 ½XðsÞ ¼ L21 ½GðsÞUðsÞ ¼ L21 ½GðsÞ p L21 ½UðsÞ where the symbol p indicates the convolution operation. Let us ﬁnd out how we obtain these transfer functions by revisiting the differential equations we have dealt with previously. First Order System Consider again the dynamic system described by a ﬁrst order differential equation given in Equation 4.59. The equation is rewritten here for subsequent discussion

t

dx þx¼u dt

Laplace transform of this is given by

t sXðsÞ þ XðsÞ ¼ UðsÞ which for zero initial condition leads to the following transfer function; GðsÞ ¼

XðsÞ 1 ¼ UðsÞ ts þ 1

A block diagram representation is shown in Figure 4.47b. When UðsÞ is a unit step input, then XðsÞ is given by XðsÞ ¼

1 sðts þ 1Þ

Mechatronic Elements for Optomechatronic Interface

235

It can be shown that Laplace inverse transforming of this equation yields the same differential equation given in Equation 4.60. It is worthwhile to remembering that time constant t determines the speed of the system response, as already discussed. Second Order System Consider the second order system described by the equation given in Equation 4.61 which is rewritten as d2 x dx þ 2jvn þ v2n x ¼ b0 v2n uðtÞ 2 dt dt When we assume Laplace transforming of the above equation with all zero initial conditions we obtain s2 XðsÞ þ 2jvn sXðsÞ þ v2n XðsÞ ¼ b0 v2n UðsÞ from which the transfer function is obtained by GðsÞ ¼

XðsÞ b0 v2n ¼ 2 UðsÞ s þ 2jvn s þ v2n

ð4:67Þ

The block diagram of GðsÞ is described in Figure 4.47c for b0 ¼ 1: As we have already discussed, the roots of denominator of GðsÞ ¼ 0, that is, s2 þ 2jvn s þ v2n ¼ 0 yield three different cases, depending upon j value, as before. pﬃﬃﬃﬃﬃﬃﬃﬃ When j , 1; Two roots are at p1 , p2 ¼ 2jvn ^ jvn 1 2 j2 which are complex conjugates. By using these roots, Equation 4.67 can be rewritten as GðsÞ ¼

v2n ðs1 þ p1 Þðs þ p2 Þ

ð4:68Þ

When UðsÞ is a unit step input, XðsÞ is obtained by XðsÞ ¼

v2n pﬃﬃﬃﬃﬃﬃﬃﬃ pﬃﬃﬃﬃﬃﬃﬃﬃ s s þ jvn þ jvn 1 2 j2 s þ jvn 2 jvn 1 2 j2

By Laplace inverse transforming of this, we have exactly the same response equation given in Equation 4.62. When the system is critically damped, i.e., j ¼ 1 two roots are equally at p ¼ p1 ¼ p2 ¼ 2vn , GðsÞ in this case is given by GðsÞ ¼

v2n ðs þ pÞ2

from which XðsÞ is obtained for UðsÞ ¼ 1=s by XðsÞ ¼

b0 v2n sðs þ vn Þ2

The time domain response of this is identical to that given in Equation 4.63. When j . 1 two roots are distinct and real and located at

236

Optomechatronics

pﬃﬃﬃﬃﬃﬃﬃﬃ p1 ,p2 ¼ 2jvn ^ vn j2 2 1, we have GðsÞ ¼

b0 ·v2n sðs þ p1 Þðs þ p2 Þ

XðsÞ is then given by XðsÞ ¼

b0 ·v2n pﬃﬃﬃﬃﬃﬃﬃﬃ pﬃﬃﬃﬃﬃﬃﬃﬃ sðs þ jvn þ vn j2 2 1Þðs þ jvn 2 vn j2 2 1Þ

The Laplace inverse transform of this equation yields the same time domain response as given in Equation 4.64. Higher Order System When a system is described by the nth order differential equation given in Equation 4.58, a general form of its transfer function is represented by m Y

ðs þ z1 Þðs þ z2 Þ· · ·ðs þ zm Þ j¼1 ¼ n GðsÞ ¼ Y ðs þ p1 Þðs þ p2 Þ· · ·ðs þ pn Þ i¼1

ðs þ zj Þ ð4:69Þ ðs þ pi Þ

if the denominator and nominator are factored out by use of n poles, pi ði ¼ 1,2· · ·nÞ and m zeros zj ðj ¼ 1,2· · ·mÞ: The pi and zj may include the second pﬃﬃﬃﬃﬃﬃﬃﬃ order complex conjugate terms s þ jvn ^ vn j2 2 1: It is noticed that a system to be realizable needs to satisfy the condition n q m: The Laplace transform of this general equation can be made in a similar way to the case of the second order system. This is done by use of the partial fraction method which decomposes the transfer function into several simple ﬁrst or second order terms. Once the transfer function can be decomposed like this, then control system analysis and design will be much easier than handling the unfactored form. Laplace Transforms Theorems There are several theorems on the Laplace transform that are useful in the analysis and design of control systems. Here, we present them without proof. Final value theorem: If f ðtÞ and df ðtÞ=dt are Laplace transformable, and if f ðtÞ approaches a ﬁnite value as t ! 1, then lim f ðtÞ ¼ lim sFðsÞ

t!1

s!0

it is noted that when f ðtÞ is a sinusoidal function such as cos vt, and sin vt, this theorem is not valid, since for these functions lim f ðtÞ does not exist. t!1 This theorem is frequently used to determine the steady state error of the response of a control system.

Mechatronic Elements for Optomechatronic Interface

237

Initial value theorem: If f ðtÞ and df ðtÞ=dt are Laplace transformable and if lim sFðsÞ exists, then s!1 f ð0þ Þ ¼ lim sFðsÞ s!1

where t ¼ 0þ implies the positive side of time at t ¼ 0, and this deﬁnition together with t ¼ 02 is useful for a function having discontinuity at some instant of time. This theorem is useful to determine the initial value of the response of a control system at t ¼ 0þ , and thus the slope of the response at that time. The initial value theorem and ﬁnal value theorem enable us to predict the system behavior in the time domain without actually transforming Laplace transformed functions back to time functions. Open Loop vs. Feedback Control A system normally consists of a series of several transfer function elements as shown in Figure 4.48a. In this case, the block diagram is reduced to obtain the overall transfer function GðsÞ XðsÞ ¼ G1 ðsÞG2 ðsÞG3 ðsÞ ¼ GðsÞ UðsÞ

ð4:70Þ

This equation signiﬁes that once the desired output value is speciﬁed we can determine one-to-one correspondence between the input and output. For instance, when a desirable xd ðtÞ is given, we can provide the system with an input UðtÞ so as to make the response xðtÞ ! xd ðtÞ as time t goes to inﬁnity. This is true only when the system is not subject to external disturbance, which will be discussed shortly. This implies that this type of control is called open loop control and does not use any measurement information. In contrast to this, the feedback or closed loop control utilizes an instantaneous measurement information obtained by a sensor or sensors for feedback, as shown in Figure 4.48b. The overall system transfer function can

U(s)

G2(s)

G1(s)

G 3(s)

X(s)

Xd(s)

G(s) equivalent

(a) open loop control FIGURE 4.48 Open loop control vs. feedback control.

G(s)

X(s)

original

original

U(s)

+_

X(s)

Xd(s)

G(s) 1 + G(s) equivalent

(b) feedback control

X(s)

238

Optomechatronics

be obtained upon consideration of error variables which is deﬁned by EðsÞ ¼ Xd ðsÞ 2 XðsÞ Since XðsÞ=EðsÞ ¼ GðsÞ, we ﬁnally have XðsÞ GðsÞ ¼ Xd ðsÞ 1 þ GðsÞ

ð4:71Þ

Comparison of Equation 4.70 and Equation 4.71 enables us to examine two basic differences between the open loop control and closed loop control. The ﬁrst one is that the closed loop control is much less sensitive to the variation of system parameters. To illustrate this let GðsÞ be changed by DGðsÞ from its original function. Then open loop control :

XðsÞ ¼ ½GðsÞ þ DGðsÞ UðsÞ

closed loop control :

XðsÞ ¼

GðsÞ þ DGðsÞ X ðsÞ 1 þ GðsÞ þ DGðsÞ d

In the open loop system, the response XðsÞ is varied by the amount DGðsÞUðsÞ, while in closed loop control XðsÞ is varied approximately by DGðsÞ X ðsÞ 1 þ GðsÞ d Therefore, sensitivity to variation of system parameters is greatly reduced by a factor 1=ð1 þ GðsÞÞ: The second one is that the closed loop control is much less sensitive to external disturbance. To illustrate this let DðsÞ be the Laplace transform of a disturbance dðtÞ, as indicated Figure 4.49, XðsÞ given in Equation 4.70 is modiﬁed as open loop control :

XðsÞ ¼ GðsÞUðsÞ þ DðsÞ

closed loop control :

XðsÞ

GðsÞ 1 X ðsÞ þ DðsÞ 1 þ GðsÞ d 1 þ GðsÞ

We can see again that the effect of the disturbance can be drastically reduced with closed loop control. However, one thing to take into careful consideration is that the closed loop system often becomes unstable even if the system is open loop stable. Therefore, the closed loop control system design should take into consideration system stability. In conclusion, the effects of system variation and disturbance on system response can be controllable or even eliminated with some types of control action we will discuss later. System Performance As we have seen already, the system response of the second order system is largely dependent on damping ratio, j and natural frequency, vn : In other words, these system variables determine the response characteristics at both

Mechatronic Elements for Optomechatronic Interface

239 D(s)

(s)

+

G(s)

+

X(s)

open loop

D(s) X d(s)

+_

G(s)

+

+

X(s)

closed loop FIGURE 4.49 Inclusion of an external disturbance.

transient and steady states. This indicates that, once the desired characteristics are speciﬁed, we need to determine the corresponding system variables, j and vn according to the speciﬁcation. The problem is how we deﬁne such characteristics. In many practical cases, response characteristics of a dynamical or controlled system are speciﬁed in terms of time domain parameters that are indicative of system performance. Because the transient response of a practical system often exhibits damped oscillatory behavior before reaching state, we will deﬁne time domain parameters by using an oscillatory transient response curve we have observed from the second order system. In deﬁning such parameters, it is common to use a unit step input: in fact, there are several types of test signals such as impulse, unit step, ramp, parabolic, sinusoidal and random signal as shown in Figure 4.50. The choice of the test signal is dependent upon what type of response a system is required to produce. For example, if the system is a positioning system, the step type test signal may be useful to test its response. Referring to Figure 4.51 which shows a typical response curve of dynamic system for a unit step input, we may specify the time domain parameters commonly used for evaluating the response characteristics. For transient response: (1) rise time ðtr Þ : the time required for the response to rise from 10 to 90%, 5 to 95% or 0 to 100% of its ﬁnal value, (2) peak time ðtp Þ : the peak time is the time required for the response to reach the ﬁrst peak of the overshoot,

240

Optomechatronics

(1) impulse

(2) step

(4) parabolic

(3) ramp

(5) sinusoidal

(6) tracking

FIGURE 4.50 Various test signals to evaluate system response.

(3) maximum overshoot ðMp Þ : this quantity is deﬁned by Mp ¼

xðtp Þ 2 xð1Þ £ 100% xð1Þ

For steady state response:

1.2

Mp

response, x (t )

1.0

2%~5%

0.8 0.6 0.4 0.2 0.0 tr

tp

ts

FIGURE 4.51 Characterization of a system response for a unit step input.

time (t )

Mechatronic Elements for Optomechatronic Interface

241

(4) settling time ðts Þ : the time required for the response to reach and stay within a range of 2 to 5% of the ﬁnal value. (5) steady state error: the error which is the deviation from the ﬁnal value at steady state eð1Þ ¼ 1 2 xð1Þ According to this speciﬁcation, we may calculate the following time domain parameters for the second order system. tp ¼

pﬃﬃﬃﬃﬃ2 p 4 3 , Mp ¼ e2ðj 12j Þp , ts ¼ ð2% criterionÞ or ð5% criterionÞ ð4:72Þ vd jvn jvn

From this speciﬁcation, it is noted that all time domain parameters are a function of j and vn : Clearly, these parameters are a family of parameters that can characterize the system performance. As long as we keep all the parameters ðtp ,Mp ,ts Þ as small as possible, we can achieve the best desirable response, if there is no steady state error. The second order system given in Equation 4.67 has no steady state error if b0 ¼ 1: In general, steady state error appears in most control systems, although it is small. In some cases this small error may not be tolerable which requires careful design of a controller that may reduce or eliminate error altogether. One important thing to be noticed is that there is always a trade off between rise time ðtp Þ and maximum overshoot ðMp Þ because shorter rise time implies larger overshoot, and vice versa. A higher order system can also be speciﬁed using these parameters only in certain simple situations. Normally, we need to make an appropriate assumption for system analysis and design or to determine the parameters by obtaining the response numerically (Simulink). Basic Control Actions Most dynamical systems do not exhibit desirable response characteristics unless they are controlled properly. For instance, some systems may not satisfy the transient response requirements while some others may not satisfy the steady state response requirements. In severe cases, they may be inherently unstable for some range of operation. The effective way of correcting the unwanted response characteristics is to use a feedback control method. As we have already pointed out before, this method utilizes the information on the instantaneous state of systems, which is acquired by sensors. Based upon the sensor information, system actuators adjust their manipulating signal so as to reduce or eliminate the error that occurs at any instant of time and eventually the desired response. Figure 4.52 illustrates a generic form of a block diagram of a control system. It is composed of a controller, GC ðsÞ which produces a command signal to an actuator depending upon the error variable, an actuator that actuates the system,

242

Optomechatronics

according to the command signal from the controller, a dynamic system GP ðsÞ to be controlled and ﬁnally a sensor GS ðsÞ that measures the instantaneous state of the system. All of these elements are of paramount importance to obtain desired system performance, because each affects system response characteristics. In particular, sensors and actuators must be suitably chosen so that their dynamic characteristics meet the overall system speciﬁcation. In addition, the control signal should be properly generated by the controller, taking into account the dynamic characteristics of the system, actuators, sensors, and disturbances. There are a variety of controller types that may be categorized into several groups but here we will discuss in brief only three classical controllers; proportional (P) controller, proportional plus integral (PI) controller, proportional plus integral plus derivative (PID) controller. Upon examination of the three controllers listed above, it can be observed that combination of three basic actions enables those controllers to generate appropriate command signals. These are proportional (P), derivative (D), and integral (I) actions. Figure 4.53 shows those three basic actions. Proportional control action is based upon the proportional relationship between the controller output uðtÞ and the error signal eðtÞ which is described by uðtÞ ¼ kp eðtÞ

ð4:73Þ

or in Laplace-transformed quantities UðsÞ ¼ kp EðsÞ

ð4:74Þ

where kp is termed the proportional gain. This control action produces a signal that is linearly proportional to the error between the desired set value and the actual output value. Therefore, this type of action may be said to be one of the simplest control actions among the three. In integral action, the value of controller output uðtÞ is changed at a rate proportional to the integral

disturbance D(s) command signal

X d(s) desired value

+_

error controller E(s) Gc(s)

actuating signal actuator

U(s)

Ga(s)

Gs(s)

measured variable

sensor

FIGURE 4.52 A generic representation of a control system.

M(s)

+

+ Gp(s)

X(s) actual output

Mechatronic Elements for Optomechatronic Interface

desired value

Gc(s)

+_

243

Ga(s)

sensor signal

kp

+_

(a) proportional

+_

ki s

(b) integral

+_

kds

(c) derivative

FIGURE 4.53 Basic control actions.

of the error, whose relationship is given by ðt UðsÞ k uðtÞ ¼ ki eðtÞdt or ¼ i EðsÞ s 0

ð4:75Þ

where ki is the integral constant. This action provides a signal that is a function of all past values of error rather than just the current value. The action is effective in the steady state rather than in the transient state, because in the steady state, the error may be accumulated as time increases. The last type is the derivative action which has the form uðtÞ ¼ kd

de dt

or

UðsÞ ¼ kd s EðsÞ

ð4:76Þ

This control action depends upon the rate of change of the error. As a result, a controller with derivative action exhibits an anticipatory error. As can be observed, this action is effective during the transient period but not at steady state where not much variation in response occurs. It has the effect of stabilizing the response. The PI controller is composed of a proportional (P) action and integral action (I). The control signal is given by ðt k uðtÞ ¼ kp eðtÞ þ ki eðtÞdt ð4:77Þ or UðsÞ ¼ kp þ i EðsÞ s 0 As can be seen, the controller gains kp and ki play the role of a weighting factor on each term. If kp . ki , the proportional controller contributes to the command signal more than the integral part does. On the other hand, if ki . kp , the integral action contributes more. This controller may be effective when a system needs faster response and requires no steady state error due to constant disturbance. Electronic implementation of this controller in a single circuit is shown in Figure 4.54a. Refer to the basic circuits illustrated in Figure 4.16.

244

Optomechatronics

R2 R1

C1

R1

R3 R2

– +

C1

(a) PI controller

– +

(b) PD controller R2

C1

C2

– +

R1

(c) PID controller FIGURE 4.54 Electric implementation of three controllers.

The proportional plus derivative (PD) controller may be represented by uðtÞ ¼ kp eðtÞ þ kd

deðtÞ dt

or

UðsÞ ¼ ðkp þ kd sÞEðsÞ

ð4:78Þ

This controller may be effective when a controlled system requires proper amount of damping without too much loss of response speed. Figure 4.54b shows the circuit implementing this controller. The most general type of controller is basically a three mode controller which is the PID controller deﬁned by ðt deðtÞ or uðtÞ ¼ kp eðtÞ þ ki eðtÞdt þ kd dt 0 ð4:79Þ ki þ kd s EðsÞ UðsÞ ¼ kp þ s When a system to be controlled requires a proper amount of damping, a proper speed and no steady state error to constant external disturbances, this control action may be effective. Again, it is noticed that kp , ki and kd can be interpreted as the weighting factor of each action for the controller. For instance ki q kp , kd implies that integral action contributes greatly in comparison to the other two actions. Figure 4.54c shows a simple PID controller combined in a single ampliﬁer circuit. System Stability Stability of a system is a property that needs to be checked ﬁrst, because if a system is unstable, then the response characteristics we have discussed in the above is not meaningful at all. System stability can be checked by

Mechatronic Elements for Optomechatronic Interface

245

Im [s]

neurally stable Re[s] = 0; s = ± j ω stable region Re[s] < 0

unstable region Re[s] > 0 O

Re[s]

FIGURE 4.55 Stability in complex s-plane.

examining the denominator of GðsÞ given in Equation 4.69. The stability requires that all roots satisfying the characteristic equation (denominator set equal to zero) must lie in the left-half s plane as indicated in Figure 4.55. To be more speciﬁc, all roots that satisfy DðsÞ ¼ ðs þ p1 Þðs þ p2 Þ…ðs þ pn Þ ¼ 0

ð4:80Þ

must be strictly in the left hand side of the complex plane excluding jv axis. In other words, all pi s must have negative real part. If any one of pi s does not meet this requirement, the system is said to be neutrally stable or unstable. This stability check can be made by using the Routh Stability Criterion in Appendix A3. For a neutrally stable system, the roots lie in the jv axis and therefore its response exhibits a typical sinusoidal oscillation with constant amplitude but does not increase with time. For an unstable system, its response to any input will increase monotonically with time or oscillate with increasing amplitude. An important remark to be made here is that whether a linear system is stable or unstable is a property of the system itself, but not a property of external input, unless the input is dependent upon the system response.

Problems P4.1. Consider a capacitance sensor composed of two parallel plates separated by a dielectric material. Its original overlapping length is ‘: Suppose that the upper plate is moved by D‘ as shown in Figure P4.1. Using Equation 4.2, determine the sensitivity function deﬁned by S ¼ DC=D‘:

246

Optomechatronics moving w

fixed d

FIGURE P4.1 A schematic of a capacitance sensor with changing area.

P4.2. When a dielectric material between two plates moves by ‘ in the direction of the x axis, as shown in Figure P4.2, obtain the total capacitance of this sensor. 1r1 and 1r2 are the permittivity of the relative dielectric constants of material 1 and material 2, respectively. plate dielectric εr1

dielectric εr2

x

FIGURE P4.2 A capacitive sensor with dielectric area change.

P4.3. A piezoelectric sensor consists of a piezoelectric transducer, a connecting cable and an external circuit, as shown in Figure P4.3. Assume that it is modeled as a spring-mass-damper system having stiffness kp , damping coefﬁcient bp and mass mp :

applied force, F z

RL i

Cp

Cc

VL

piezoelectric transducer piezoelectric transducer

cable

load

FIGURE P4.3 Piezoelectric sensor conﬁguration.

Suppose that the cable is represented by a pure capacitance element Cc and that the external circuit is represented by a pure load RL : (1) Write down the equation of motion of the sensor in the z direction for a given force F.

Mechatronic Elements for Optomechatronic Interface

247

(2) If the transducer is modeled by a current generator with a capacitance Cp , write down the equation for the electric circuit shown in the ﬁgure. (3) Obtain the transfer function between the force F and the output voltage VL : P4.4. Suppose that there is a point light source (object) S distant from a detector (D)

object S FIGURE P4.4 A detector receiving a bundle of rays from an object surface.

detector, D, as shown in Figure P4.4. When high efﬁciency of sensing is required, suggest a method to increase the efﬁciency. R2

R1

C

Vin

Vout

FIGURE P4.5 Integral operation by op-amp.

P4.5. Shown in Figure P4.5 is an electronic circuit of the integral operation. Obtain the equation describing the input– output relationship. P4.6. In micro electro-mechanical-systems (MEMS) or micro actuator and sensors, clamped beam structure has a variety of interesting deformable beam h fixed beam

d

V

δ fixed

FIGURE P4.6 A capacitive clamped–clamped type actuator.

248

Optomechatronics

applications. One such application can be found from a capacitive type actuator as shown in Figure P4.6. The two beams are separated initially by d: The upper beam is deformable, while the lower beam is ﬁxed. If the electric ﬁeld V is applied to the deformable beam, determine the force per unit length acting on the beam element. In the ﬁgure, ‘, w and h are the length, width, and thickness of the beam, respectively. P4.7. A spring of mass m is made of shape memory alloy (SMA). The objective of the use of the SMA is to give the mass an oscillatory motion in a proper way (See Figure P4.7).

SMA spring

x m FIGURE P4.7 SMA actuated spring motion.

(1) Describe how the vibratory motion can be produced. (2) Based on this principle, express roughly the equation of the motion, taking into consideration the material hysteresis effect. P4.8. Figure P4.8 shows the use of a Wheatstone bridge to measure strain. When the deformation of the strain gauge occurs, it will cause the gauge resistance to change, DR: If the bridge has equal resistance R, show that Vout is determined by Vout ¼ ðVS Rf =RÞðDR=2Þ: In the ﬁgure Vs is the supply voltage and Rf is the feedback resistance of the ampliﬁer. P4.9. A galvanometer recorder utilizes a UV light as a light source, a mirror, and a moving strip of photosensitive paper. The mirror is attached to a moving coil assembly which is suspended by a torsional string and a viscous damper. The coil is moving in a magnetic ﬁeld under the same principle as that of the electric DC motor. While the paper is moving in the x direction, the mirror reﬂects an incident beam to a point Pðx; yÞ on the paper as depicted in Figure P4.9.

Mechatronic Elements for Optomechatronic Interface

249

VS Rf R1

R2 i1

i2

i3

i4

R+∆R

i5

i6 na Vout

R4

ng Rf

strain gauge

R+∆R

FIGURE P4.8 Bridge arrangement with a differential ampliﬁer for strain measurement.

(1) If the coil assembly has a moment of inertia I, and is suspended by a torsional spring, kt , and a viscous damper, b, write down the equation governing the mirror motion u: Assume that the magnetic ﬂux density is B, n is the number of coil turns, A is the cross sectional area of the magnet, and i is the current ﬂowing through the coil.

rotational damper

b

θ

mirror

photo sensitive paper

magnetic field B

magnet N tortional spring

P (x,y)

moving

S

i kt

x y S UV light source

FIGURE P4.9 Strip chart-galvanometer recorder.

250

Optomechatronics

(2) Suppose that the mirror is rotated by u from an initial angle uð0Þ ¼ 0, with zero angular velocity du=dtð0Þ ¼ 0: Write down the transfer function between the current iðtÞ and uðtÞ: Discuss the mirror motion in terms of natural frequency and damping ratio of the mirror system. (3) If the mirror is rotated by small amount Du from its initial angular position, what will be the relationship between y and Du ? Assume that the initial angle of incident beam is at u ¼ ui with respect to some reference angle. P4.10. A closed loop control system is shown in Figure P4.10. For control purposes, the microprocessor takes in the information obtained by a sensor whose signal is contaminated by noise as shown in the ﬁgure. Then the signal is sampled once within a period Dts and based upon this sampled signal, the controller generates a command signal to an actuator which drives the system. (1) Suppose that the signal of the output variable has a frequency range 10 Hz # fs # 50 Hz, and noise frequency ranges fn $ 60 Hz: What type of ﬁlter can be used to reduce noise? What kind of considerations should be given to design such a ﬁlter? (2) To ensure a good performance of the control system, what frequency of the control action is recommended? Assume that the control action ð fc Þ is carried out at least twice within one sampling frequency. P4.11. Consider a mirror driven by a gap closing actuator supported by a spring whose stiffness is k as depicted in Figure P4.11. When m is total mass of the mirror unit, z is the displacement from the equilibrium state in downward direction, V is the applied voltage to the actuator, and z0 is the original gap, the equation of motion of the mirror is the same as given Equation 4.51 in the text. (1) Discuss the behavior of the mirror motion depending upon the applied voltage. (2) Describe the behavior of the mirror motion when the mirror approaches the lower plate. (3) Show a block diagram of a feedback control system to maintain the mirror position at a desired position, and explain its control concept. P4.12. Consider the piezoelectric sensor treated in Problem P4.3. If the dynamics of the sensor is given such that its transfer function is described by GðsÞ ¼

VðsÞ tv2n s ¼ FðsÞ ðts þ 1Þðs2 þ 2jvn þ v2n Þ

A/D

controller

measured value

FIGURE P4.10 A signal processing and control for a digital control system.

(b) output signal with noise

O

amplitude

(a) feedback system

xd

desired value

sampled signal

sensor

D/A

actuator

system

time (t )

actuation signal

x( t )

actual output variable

Mechatronic Elements for Optomechatronic Interface 251

252

Optomechatronics

mirror

m leaf spring k

v

z actuator

FIGURE P4.11 A mirror driven by an electrostatic actuator.

(1) Discuss the effect of t on the system response as t decreases from a large value to a very small value. In the case where t # 1, what will be the type of response characteristics? (2) Plot the response of the sensor system to a unit step input of force FðsÞ ¼ 1=s, using the Simulink model in Matlab. The parameters are given in the followings: vn ¼ 1:6 £ 105 rad=sec, j ¼ 0:01, t ¼ 2:0 msec: (3) Determine the steady state error for (a) a unit step input force and (b) a unit ramp input force.

References [1]

Ando, T. Laser beam scanner for uniform halftones, Printing Technologies for Images, Gray Scales and Color, SPIE, Vol. 1458. 1991. [2] Auslander, D.M. and Kempf, C.J. Mechatronics — Mechanical System Interfacing. Prentice Hall, Englewood Cliffs, 1996. [3] Benech, P., Chamberod, E. and Monllor, C. Acceleration measurement using PVDF, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 43: 5, 838– 843, 1996. [4] Bentley, J.P. Principles of Measurement Systems, 4th ed. Prentice Hall, Englewood Cliffs, 2004. [5] Bolton, W. Mechatronics Electronic Control Systems and in Mechanical Engineering. Addison Wesley/Longman, Reading MA, London, 1999. [6] Bradley, D.A., Dawson, D., Burd, N.C. and Loader, A.J. Mechatronics. Chapman and Hall, London, 1991. [7] Dally, J.W., Riley, W.F., and McConnell, K.G. Instrumentation for Engineering Measurements. John Wiley and Sons Inc, New York, 1984. [8] Fraden, J. Handbook of Modern Sensor. American Institute of Physics, New York, 1993. [9] Fukuda, T., Hattori, S., Arai, F. and Matsuara, H. Optical servo systems using bimorph PLZT actuators, The American Society of Mechanical Engineering (ASME), 46, 13 – 19, 1993.

Mechatronic Elements for Optomechatronic Interface [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

[21] [22]

253

Fukuda, T. Performance improvement of optical actuator by double side irradiation, IEEE Transactions on Industrial Electronics, 42, 455– 461, 5 October, 1995. Gweon, D.G., Special Topics in Dynamics and Control, Lecture Note, MAE850, Korean Advanced Institute of Science and Technology, 2004. Kenjo, T. and Sugawara, A. Stepping Motors and Their Microprocessor Controls, 2nd ed. Oxford Science Publications, Oxford, 1995. Lin, R., Shape Memory Alloys and Their Application, http://www.standford. edu/, richlin1/sma/sma.html, 2005. Luxon, J.T., Parker, D.E., 2nd ed., Industrial Lasers and Applications. Prentice Hall, Englewood Cliffs, 1992. Mahalik, N.P. Mechatronics: Principles, Concepts and Applications. McGraw-Hill Inc, New York, 2003. Near, C.D., et al. Sensor Technology Limited (Product Information), SPIE, Vol. 1916. pp.396 – 404, Oxford, 1993. Pluta, M.. Advanced light Microscopy Principles and Properties. Elsevier, Amsterdam, 1988. Sashida, T. and Kenjo, T. An Introduction to Ultrasonic Motors. Oxford Science Publications, 1993. Senturia, S.D. Microsystem Design. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2001. Shen, Y., et al. A high sensitivity force sensor for microassembly: design and experiments advanced intelligent mechatronics, Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Kobe, Japan, Vol. 2. 2003, pp. 703– 708. Stadler, W. Analytical Robotics and Mechatronics. McGraw-Hill Inc, New York, 1995. Tabib-Azar, M. Microactuators. Kluwer Academic Publisher, Dordrecht, 1998.

5 Optomechatronic Integration CONTENTS Basic Considerations for Integration .............................................................. 256 Basic Functional Modules ................................................................................ 263 Signal Transformation ............................................................................... 263 Signal Manipulation .................................................................................. 264 Signal Sensing............................................................................................. 266 Signal Actuation ......................................................................................... 269 Signal Transmission ................................................................................... 270 Signal Display............................................................................................. 272 Optomechatronic Interface or Integration ..................................................... 273 Basic Two-Signal Integration.................................................................... 273 Fundamental Optomechatronic Integration: Illustrations................... 277 Generic Forms for Optomechatronic Interface ..................................... 279 Integrability................................................................................................. 288 Signal Flow.................................................................................................. 289 Integration-Generated Functionalities............................................................ 291 Problems.............................................................................................................. 294 References ........................................................................................................... 297

In the previous chapters we have reviewed some of the fundamentals of the two engineering ﬁelds, optical and mechatronic engineering. During the process of the review, we have seen that basic principles and concepts, and their relevant theories and tools, can be combined to interact with each other to create new concepts or technical elements. In fact, we have seen from the illustrations discussed in Chapter 1 that a variety of the integrated components and systems result from those interactions among optical, mechanical, and electrical technologies. Further, we have seen that such integrations can be derivable from certain principles of physical interaction. Judging from the observations, we can summarize the following characteristic points:

255

256

Optomechatronics

Hierarchical structure of integration: Optomechatronic integration is hierarchically composed of several levels of integration, so its architecture is composed of functional coordination level, structural integration level, and organization level. Fundamental functionalities: Functionality is a task-oriented functional entity required for a system to execute a given task speciﬁed by the designer. Such a functionality may be produced by combining several fundamental functional modules. Basic functional modules: The functional modules are the functionoriented modules required to produce a functionality in a speciﬁed manner. These functional modules are combined together to create the basic functionalities required by optomechatronic integration. Integration interface: There are certain laws for putting the functional modules together. The integration must consider the interfacing mechanism between the optical, mechanical, and electrical components. At the same time, it must take into consideration the signal ﬂow that represents physical phenomena that occur during the interaction involved with the integration. In this chapter, we will have a more detailed look into physical optomechatronic components and systems, in order to investigate their characteristics regarding the composition of their structure and the fundamental functionalities. To this end, we will consider the basic functional modules that create such functionalities, and study the signal ﬂow in order to facilitate the composition of the modules needed for functionality generation. Here, signal is meant in the broad sense by energy or information. Based upon the analysis of the functional modules, we will discuss the nature of integration in order to ﬁnd common modalities that might exist among various integrations of the three engineering ﬁelds. From these ﬁndings, we ﬁnally derive the optomechatronic integration and analyze the characteristics of the integration, which include integrability, integration structure, and information signal ﬂow. To better understand the underlying concept, we will illustrate a variety of examples of physical systems from the view point of integration.

Basic Considerations for Integration Looking into the details of optomechatronic technology from this viewpoint, we can see that it involves interaction among three engineering ﬁelds, optical, mechanical, and electrical. The interaction between the three signals will differ, depending upon the type of the integration and the strength of contribution of each technical element. The dependency will, therefore, determine the nature of optomechatronic integration. If we can formulate the relationship, the type, and strength of the integration vs. the characteristics

Optomechatronic Integration

257

of the integration, developing integration methodology will become easy for a given set of problems. However, as we might anticipate, it is an extremely difﬁcult task to formulate and analyze the relation even with a heuristic manner. Nevertheless, we will make an effort to dig out the underlying natures of the integration by introducing basic functional modules that can be the basis for the integration, and the analysis of the structure of the integration. As listed in Table 5.1, three engineering ﬁelds (O, E, M) have distinct physical variables originating from their own physical phenomena. Integration of these ﬁelds implies that these unique, distinct variables are interacting with each other to affect some of the interacting variables, or create new physical variables. As indicated in Figure 5.1, the unique optical variables include light energy, ray intensity, and radiation ﬂux, the mechanical variables include mechanical energy, motion, deformation, strain, and ﬂuid/heat ﬂow, and electrical variables include current, voltage, charge, and magnetic ﬂux. We will represent these unique variables in the following manner: mechanical variables as a mechanical signal, optical

TABLE 5.1 Signals and Basic Elements of Mechanical, Electrical, and Optical Engineering Mechanical Force ( f ) Velocity (v) Power ( p) (mechanical) Energy (E)

Electrical

Optical

Signal Current (i) Voltage (v) Power ( p) (electrical) Energy (E)

Ray (r) Intensity (I ) Irradiance (R) (radiant, luminous) Radiant energy (E )

Basic elements Mass (m) m

Capacitor (c)

Prism

Spring (k)

Coil (L)

Beam splitter

Damping (b)

Resistor (R)

Lens

Gear

Transformer

Cam

Alternator

Belt

Inductor

Bellows

Semiconductor p

Mirror N

S

Stop Aperture

n

Gratings

258

Optomechatronics

energy, ray, intensity, radiation flux ti

na

l

op

ca

sig

l

ical chan me

energy, current, voltage, magnetic flux charge

al ctric ele

signal

energy, motion, strain, fluid/heat flow

signal

FIGURE 5.1 Optical, mechanical, and electrical signals.

variables as an optical signal, and electrical variables as an electrical signal. The signals are expressed in the following form: optical signal ¼ O (energy, ray, intensity, radiation ﬂux…) mechanical signal ¼ M (energy, motion, ﬂuid ﬂow deformation, strain…) electrical signal ¼ E (energy, current, voltage, charge, magnetic ﬂux…) The interaction can occur with various modalities, depending largely upon: (1) which variables are interacting, (2) how many variables are interacting, (3) how and when they are interacting, and (4) ﬁnally, the sequence and the duration of the interaction. It will be a formidable, or even an impossible, job to formularize the integration modalities. Rather we will try to ﬁnd a relevant integration concept, taking into consideration the fundamental questions mentioned above. To start with, we will assume that all three signals contain six or fewer fundamental functional modules, as shown in Figure 5.2. These elements comprise transformation, manipulation, sensing, actuation, transmission, storage, and display of a signal. Let us discuss them one by one, in detail, to identify their characteristics. Signal transformation is the conversion of one signal form into another as a result of a change of physical elements. Signal manipulation is the action of diverting or modifying a signal. Sensing is the measurement of the state of a signal. Signal actuation is the control of a signal in order to maintain it at a certain value. Signal transmission is the action of transmitting a signal to a desired location. Finally, signal display is the presentation of a signal in a presentation element. Optomechatronic integration may often involve the combination of multiples of these functional modules, but, in a simple example, may

Optomechatronic Integration

259

FIGURE 5.2 Basic technological elements for optomechatronic technology.

involve only one single module. For instance, the optical disk shown in Figure 1.6c has several physical components that carry out their own tasks: an optically-encoded rotating disk, a tracking unit that searches a certain data location, a track following unit, and auto focusing and reading units. The rotating disk is a signal storage device that has a signal storage module (optically recorded element), and a signal transformation module (motor) that makes the disk rotate by an electric motor at a speciﬁed rotation speed. The tracking unit has a signal transforming component (moving optical head) and a sensing module (optical element) that measures the data track. The auto-focusing and reading unit has a signal control module and a sensing module as well. All of these modules involved here represent optomechatronic integration. Taking another example, the optical switching shown in Figure 1.6f has one functional module, “signal modulation,” which is achieved by optomechatronic integration composed of optical elements (light, mirror) and a mechatronic element (actuator) that actuates the mirror. From these two examples, it may be said that the optomechatronic integration, indeed, is necessary, not only for combining functional modules, but also for making each individual technical component work as desired. The principle of the optomechatronic integration concept departs from this concept. Figure 5.3 depicts a standard procedure which indicates the integration processes involved. According to the procedure, once the design speciﬁcation is given, the required functionalities are to be checked ﬁrst, in order for the desired tasks to be carried out by the system or device to be designed. The next thing to do is to identify all the functional modules that might be involved in producing the desired functionalities. Since all those

260

Optomechatronics

design specification

create the functionality

identify the functional modules

integration of optomechatronic components

system integration

verify the performance good

no

yes end FIGURE 5.3 The procedure of optomechatronic integration.

modules may not be feasible for the intended integration, optomechatronic integrability must be checked to investigate its feasibility. After the integrability is conﬁrmed, then optical, mechanical, and electrical hardware elements are to be integrated, so as to meet various requirements set by each individual module or the combined ones. The ﬁnal stage is to verify the performance of the integration. The steps for optomechatronic design are similar to those standard procedures for designing most engineered products or systems. However, there are several differences. In creating the functionalities, the optomechatronic designer can consider adding new functionalities that may not be realized by mechatronic design alone. Taking a simple example, when we wish to machine a precision mechanical part as speciﬁed, we may need to accurately measure its dimensions and surface roughness during the machining operation. This, however, may not be an easy task with the current mechatronic sensor technology. By utilizing optomechatronic concepts, this task becomes feasible by building a non-contacting optical sensor that can work with a rotating and translating machining element (mechatronic element). This implies that, in effect, the optical sensing device creates a new functionality, “optical feedback control,” for the machine

Optomechatronic Integration

261

system. When the optical device is combined with a control unit, the combined system constitutes a complete optical-based feedback control system, which drastically improves the quality of the machined part. In the stage of identifying which modules are necessary to achieve the required functionalities, there will be some difference between the two design methods. A simple difference is that the use of an optical signal may not require signal manipulation or processing (e.g., ﬁltering), while an electrical signal may. However, the major difference comes at the stage of the integration. As discussed earlier, the optomechatronic approach strives to ﬁnd the aggregated elements in order to achieve the required functionalities more effectively than the mechatronic approach alone can do. Let us consider a typical optomechatronic system, which is an opticalbased heat-treatment of the surface of a mechanical part, as shown in Figure 5.4. In the heat treatment, a moving mechanism (translating, or rotating, or a combination of both) for either a material or optical element is needed to cover the whole area to be heat treated — if we assume that the material is not heated all at once over the whole area. Under this conﬁguration, there may be ﬁve divided functionalities involved to achieve a desired quality of heat treatment, although they will not be explained in detail here. Those functionalities are light beam generation for heating the material, control of the actuator for precise light intensity, control of the stage motion in a desired manner (moving range, moving sequence), sensing or monitoring the heat-treated state of the material, and feedback control of this information to regulate the heat-treated state in a desired manner. In order to create these functionalities, we may consider the relevant functional modules, as illustrated in Figure 5.5. To produce an appropriately segmented mirror

mirror

4 kW CW CO2 Laser resonator laser beam lens

sensor fixture IR sensor

coated specimen A

c: >exp now controlled...

measurement & control hardened track XY-table

FIGURE 5.4 Laser surface hardening process.

262

Optomechatronics

light source generation

moving the material or optical head

sensing/control of the heat treated state

FIGURE 5.5 Identifying the functional modules necessary for a heat treatment process.

conditioned optical signal with required power, we ﬁrst need “signal transformation” that produces an optical signal generated by an electrical signal, optical signal modulation that converts the light beam into a desired form, and optical light control that focuses the beam in order to not only obtain high quality of the heat-treated workpiece, but also to prevent the waste of optical power needed for heat treatment. Note that optical beam control needs its own functional modules to carry out processing, manipulation, and control of the beam injected into the material surface. Another important component of this system is the moving mechanism. Because the moving element should be actuated and positioned as accurately as possible in a desirable manner, it needs a precision electricalto-mechanical actuator, a signal transformation module (which needs signal control), and sensing modules to produce accurate motion of either the material or optical head. Signal sensing and control of the treated material state, therefore, are the important functions of the system that affect treatment quality. The signal sensing has its own signal transformation part that can measure state variables, such as the treated quality, and the dimension of the treated zone. The control of the treated material state requires two signals to be regulated; one is the light power, and the other is the velocity of the moving element. In regulating these variables, the signal sensing part needs two individual sensing devices. The ﬁrst is the power measurement sensor, and the second is the velocity measurement sensor. It is noted that feedback control will not be classiﬁed as a basic functional

Optomechatronic Integration

263

module, but will be identiﬁed as a functionality having two functional modules; actuation and sensing.

Basic Functional Modules Signal Transformation In Chapter 1, we observed that a number of systems have signal or energy transformation from one form to another among optical, mechanical and electrical signals. For instance, a digital camera has three important components: CCD cell, auto-zooming by an electric servomotor, and autofocusing by an ultrasonic motor. The CCD cell transduces light information into an electrical signal as a sensor, the ultrasonic motor transduces electrical signal into a mechanical signal as an actuator (as the electric motor does). In actual optical, mechanical, and electrical systems, there are many such signal transformers that convert one signal to another, depending upon the causality between input and output. Table 5.2 conﬁgures such transforming elements that convert from one signal form to another. In this table, some of the elements transforming the same kind of signals are not included — for example, optical – optical (OO), electrical –electrical (EE), and mechanical– mechanical (MM) — although they are also important for optomechatronic integration. The ﬁrst transduction type, optical-to-electrical, denoted by the symbol ðTEO Þ, produces electricity from optical input, which can be derived from several phenomena, such as the photovoltaic phenomenon. The second type, TABLE 5.2 Basic Signal Transformations Transformation

Phenomenon

Symbol

Signal ﬂow

Typical Device

Photovoltaic, pyroelectric, photo emissive Emission of photon

TEO

O ! TEO ! E

Photo diode

E TO

E E ! TO !O

Optical-to-mechanical

Photovoltaic phase transformation

O TM

O O ! TM !M

Mechanical-to-optical

Triboluminiscent

M TO

M M ! TO !O

Light emitting diode Optical actuator, shape memory actuator

Electrical-to-mechanical

Electromagnetism, piezo electric Induction, piezo electric

E TM

E E ! TM !M

TEM

E M ! TM !E

Optical-to-electrical

Electrical-to-optical

Mechanical-to-electrical

Electric motor, piezo actuator Electric generator, piezo sensor

264

Optomechatronics

E Þ; is the reverse case of the ﬁrst. It produces light from electrical-to-optical ðTO electric input, and is popularly used for either light source or signal conversion for transmission. The third category of the transformation, which O Þ; yields mechanical signals out of optical is optical-to-mechanical ðTM signals. This conversion can be found from optical actuators which utilize some physical phenomena, such as inverse piezo-electric and, material M phase transformation. The fourth one, ðTO Þ mechanical-to-optical, can be found from smart luminiscent materials that generate light when they are E Þ is commonly used in most of subject to stress or friction. The ﬁfth one ðTM the mechatronic devices, including motion generating devices such as electric motors and piezo actuators. The last transformation type ðTEM Þ produces electrical signals that can be observed from mechanical signals from phenomena such as electric induction and piezo electricity. At this point, we can now consider a variety of signal transformation methods by combining some of the basic forms presented in the table. We notice that a number of signal transformations can be obtained as the result of such integration. For instance, suppose that we wish to combine the ﬁrst, E : The integration then yields: TEO and the ﬁfth, TM

optical ! TEO !electrical ! TME !mechanical which indicates that the optical input signal is transformed into the mechanical output signal. Mathematically, this can be expressed by: E O TEO þ TM ! TM

ð5:1Þ

An important observation we can make from this Equation is that, no matter what type of the intermediate transformation is involved (e.g., electrical signal), the ﬁnal result is written by the relation between input signal (optical) and output signal (mechanical). Avariety of cases that can illustrate this mode can be found from practical systems. One such example is to adjust an optical system based upon the measured information by a photo detector. Signal Manipulation In electrical signal processing, signal manipulation often needs to be converted into another form while retaining its original information, depending upon the application. The manipulation of optical beams is a process which a number of applications must utilize in order to obtain targeted outputs needed for processing transmission, sensing, or actuating communication via an optical signal. Steering, switching, alteration of amplitude and phase, change of wave form, ﬁltering within a certain frequency band, and noise reduction of optical signals are some of the examples that belong to optical beam manipulation. In this subsection, we will discuss the nature of beam manipulation from the viewpoint of the interaction between optical and mechatronic elements, by dividing the manipulation into two major types, beam modulation and scanning.

Optomechatronic Integration

265

In Table 5.3, various manipulation methods are summarized in the views of the basic operation principle. Almost all methods are operated based on optical manipulation. Mme oo , indicated in the table, denotes the manipulation of the optical beam by both mechanical and electrical elements. The “me” in the bottom of the symbol implies that manipulation is conducted by electrically driving the mechatronic elements. A variety of cases that can illustrate this mode can be found from practical systems. One such example is manipulation of an optical system by diverting its optical signal by means of a mechatronic element. Mm oo and Me oo imply that the manipulation is made by purely mechanical and electrical means, respectively. Let us consider two important methods that belong to signal manipulation in the sequel. (1) Optical signal modulation: In the optical engineering ﬁeld, the modulation method is also common practice in order to convert an optical signal into a form suitable for certain applications. The major need for optical modulation comes when we want to: (1) (2) (3) (4)

control the intensity and phase of a light beam. control the frequency and wave form of a light beam. impress information on to a carrier signal. have noise reduction.

The technical ﬁelds that frequently need modulation can be found among sensing, actuation, control, communication, display, and so on. The modulation can be made primarily by choppers, acousto-opto modulators, electrooptical modulators, photo-elastic modulators, and spatial light modulators. The mechanical optical chopper is typical of optomechatronic choppers,

TABLE 5.3 Types of Signal Manipulation Optical !

o

Mme o

! Optical

Optical !

o

Me o

! Optical

Manipulation type

Basic principle

Acousto-optical modulator Mechanical chopper Photo-elastic modulator Electro-optical modulator Mechanical beam scanner Acousto-optic scanner Electro-optical scanner Mechanical signal manipulation

Bragg deﬂection Beam chopping by blade Changes in refraction index Changes in refraction index Reﬂection Bragg deﬂection Reﬂection Electromechanical motion

Symbol Mme oo Mme oo Mme oo Me oo Mme oo Mme oo Me oo Me m m

Mme oo optical modulation by mechatronic principle, Me oo optical modulation by electrical method, Me m m mechanical signal manipulation by electrical means.

266

Optomechatronics

as illustrated in Figure 5.6a). In the ﬁgure, the optical beam (optical element) incorporates with the motion of a mechatronic unit (rotating chopper blade), and thereby provides a modulated beam as desired. (2) Optical scanning: Scanning is necessary when a light beam or ﬁeld of view needs to be directed to a certain point or area of an optical ﬁeld. To achieve scanning, there are two methods, reﬂective and refractive. In Figures 5.6b,c, these are shown schematically to illustrate the interaction of an optical element (beam) with a mechatronic unit (rotating mirror or prism). In the reﬂective method, mirrors are utilized, while lens and prisms are used in the refractive method. In terms of scan pattern, there are two classes of the method — ﬁxed pattern or random access. All of these methodologies involved with scanning affect the accuracy and time of scanning. There are three optical scanners most popularly used for practical applications: mechanical scanners, acoustooptical scanners, and electro-optical scanners. Upon examination of the signal manipulation methods shown in the ﬁgure, we can see that all manipulations are of optomechatronic nature. Signal Sensing The sensing module is the element that measures physical variables of the process or system being measured. In general, sensing modules operate with two different methods when classiﬁed, depending upon which modules are involved. One utilizes signal transformation, while the other uses signal modulation. In other words, sensing can be made by using either signal transformation or signal modulation, as listed in Table 5.2 and Table 5.3, respectively. Therefore, there may be a variety of sensing types classiﬁed into several different categories, depending upon the input –output signal form, and the physical phenomena used for sensing.

ψ

ω mirror

incident beam

ω

prism ω

incident beam light beam

reflected beam

refracted beam

(c) refractive scanning

2ω

(a) optical chopper

(b) reflective scanning

FIGURE 5.6 Optical scanning by optomechatronic principle.

(d) scan pattern

Optomechatronic Integration

267

Basically, all the signal transformation and manipulation modes can be used for a sensing element. Combinations of each individual module are also feasible for sensing. The problem is how effectively and accurately each form of sensing can be made. The sensing modules employing the transformation modules TEO , TEM and the modulation modules Mm oo , Me oo are popularly used in actual practice. The sensing module ToM can be used to measure a mechanical signal in terms of an optical signal. For example, when some structures are in fracture or mechanical strain, they produce optical signal. This phenomenon is called “triboluminiscience” as shown in the left hand side of Figure 5.7a. Another luminiscience that belongs to this transformation module is photo luminiscience as shown in the right hand side of Figure 5.7a. The ﬁgure shows a portion of a structural surface coated with pressure sensitive paint, which is essentially photoluminescence material. When this structure surface is stressed or strained, the intensity of light reﬂecting from the surface is varied, depending upon the pressure applied to E seems to be rare, but in the past has been the surface. The sensing module TM actively used in the control of mechatronic devices, or systems such as pneumatic or hydraulic systems.

TABLE 5.4 Signal Sensing and the Related Input – Output Variables Sensing Mode Optical !

TEO

Mechanical !

Electrical !

Phenomenon

! electrical

Photovoltaic, photo emissive

Photo diode storage tube

TEM

Deﬂection thermo effect, piezoelectric

Strain gauge, piezo sensor, pyro sensor

Mechanical motion

Fly ball governor

TME

! electrical

! mechanical

o

Optical ! Mm o ! optical o

Optical ! Me o ! optical e

Electrical ! Mo e ! electrical Optical !

TEO

Mechanical !

Typical Device

!

TEM

TOE !

! optical

TOE

! optical

Phase, intensity, wave, spectrum

—

—

Fabry-Perot sensor

—

—

Photo emissive —

Image intensiﬁer —

Some other transformation and manipulation modules and their combinations are feasible for sensing, which are not shown here due to space limitation.

268

Optomechatronics

Table 5.4 lists various modules of signal sensing whose principles are originated from signal transformation and modulation. Here, a sensing module measuring the same kind of signal is excluded, as was the case of the signal transformation. As far as transformation type sensing modules are concerned, there are six different types of sensing, whose symbols are identical to those of signal transformation. Among these the sensing E O not shown in the table may become an effective means of modules TO , TM sensing as opposed to the others TEO and TEM , when sensing involves special environments that require such conditions as being inﬂammable, being contamination-free to noise, and needing fast direct feedback. For instance, if a sensing environment does not permit the sensing module TEO to measure O may be a substitute for the measurement. In this the optical signal, then, TM M case, the module TE should also be replaced by the module Mm oo , which can measure a mechanical signal by modulation of the optical signal. A two-mode sensor module is also feasible by combining two transformation modules together, which was the case discussed in Table 5.4. One such combined sensor module can be seen in an image intensiﬁer, which E as shown in Table 5.4. Figure 5.7b has the signal ﬂow denoted by TEO ! TO illustrates the concept of a typical image intensiﬁer. The intensiﬁer takes a scene with dim incident light and produces a visible image. It consists of an emissive photo cathode surface which causes electrons to be emitted when receiving photons on to a surface, and a phosphor screen to receive the ampliﬁed image. In more detail, when photons produced by a dimly lit scene are incident on the cathode surface, the cathode causes electrons to be

λi

λe

emitted light

oxygen-sensitive probe molecules

fracture mechanical loading

structure

base coationg

structure

(a) single-mode transformation sensor photo cathode

induced surface corrugation phosphor screen

structure

photorefractive material light beam 1

collecting lens

(b) two-mode sensor

o

(c) optical actuator ; T M

FIGURE 5.7 Sensors and an actuator employing signal transformation.

light beam 2

Optomechatronic Integration

269

emitted. Under a certain voltage application, the electrostatic lens then focuses the generated electrons on to a phosphor screen. These photo electrons can excite a phosphor on the screen and intensify the illumination of the image. Therefore, the signal ﬂow occurring during the transformation process that produces this image ampliﬁcation can be written by: E O ! TEO ! TO !O

The other form of a two-mode sensor module is the case of signal ﬂow E : This module senses a mechanical motion by electrical means, TEM ! TO but converts the sensed electrical signal to the corresponding optical signal: E M ! TEM ! TO !O

ð5:2Þ

In this manner, some other multimode sensor modules can also be produced, with even more than two transformation modules. A vast number of sensors employ the modulation method. Here, we list only three kinds: mechanical modulation of optical signal, which is denoted by Mm oo , electrical modulation of optical signal denoted by Me oo, and optical modulation of electrical signal, denoted by Mo ee : The module Mm oo uses the modulation of phase, intensity, wave and spectrum of optical signal that may be caused by mechanical motion. Especially, optical ﬁber sensors which are popularly used today in various engineering ﬁelds adopt this concept, as we shall see later. The sensing module Me oo can be found from the sensors that utilize the interaction of the optical signal with the electro magnetic ﬁeld. The sensing module, Mo ee is the reverse case of the above modulator, and utilizes modulation of electrical signals by means of optical signals. Signal Actuation Actuating a signal to an arbitrary state is the most common form of the control technology, as we have seen already. As it might be confusing, a signal here means the physical variable to be actuated, and comprises three different types (optical, mechanical and electrical), as we may recall from the previous discussion. Signal actuation is deﬁned here by “actuation of mechanical signal,” to discriminate this type from other types of transformation and modulation modules. O A number of signal actuators employing TM have been developed. One such actuator is shown in Figure 5.7c. The proﬁle of a structural surface that may be altered optically is shown. This optically induced surface deformation employs the principle that photosensitive materials induce stresses and strains when they are exposed to light beam. As shown in the ﬁgure two light beams interfere and form a periodic change in the refractive index in a ﬁlm of a photosensitive material. Due to the changes in the index periodic deformation on the surface of the material results in, whose amplitude is within nm range. This periodic surface corrugations give rise to actuation and can be controllable by the optical interferometric light pattern.

270

Optomechatronics

E Þ is the most commonly-used The electrical-to-mechanical module ðTM actuation mode that actuates mechanical signal by electrical signal. On the other hand, the optical-to-mechanical module is the actuation of a mechanical signal by an optical signal. According to this deﬁnition, we can consider two important actuation modules of the transformation type; O E optical-to-mechanical TM and electrical-to-mechanical, TM : When it is necessary for signal actuation to become a remote operation, it involves transmission of a signal, either an optical or electrical. But when sending an electrical signal to a remote site may be not desirable due to noise contamination or a safety problem, an optical means may be a better solution for the transmission. Let us consider the case when an actuator to be operated is located at a remote site. The computer-generated electrical signal is converted into an optical signal, which is then transmitted in an optical signal form. The actuation module at the remote site is involved with the integration of the transformation of the second, the ﬁrst, and the ﬁfth modes listed in Table 5.2. The resulting transformation mode yields the following mathematical form: E O E E TO þ TRO O þ TE þ TM ! TM

ð5:3Þ

where transmission mode TRO O is included to express signal transmission explicitly. This equation implies that the ﬁnal transformation mode is operated in the form of the electrical-to-mechanical mode, which is the actuation module of transformation type. A typical example system of this case is an optically operated valve control of a pneumatic system located in a remote site. An electrical signal corresponding to a desired value of the pneumatic valve position is converted into the corresponding optical signal by a LED and is sent to a remote site. This transmitted optical signal is then converted into an electrical signal, which in turn, operates positioning of the servo valve. Depending upon the valve position, the pneumatic servo system is operated. We will treat this in more detail at the end of this chapter. Signal Transmission Signal transmission is a basic form of transporting information or data to or from one location or another. The distance between two locations may be very short, like the length within MEMs parts, short like that of small sensors, or very long like that of communication systems. It is mainly transmitted by means of optical or electrical signal modulation. The transmission should be immune from external noise, exhibit low attenuation, and safe from hazardous interference. For this reason, although application dependent, optical transmission has replaced many of the application areas dominated by electrical transmission. Table 5.5 shows three types of signal transmission, electrical through optical to electrical, optical to

Optomechatronic Integration

271

TABLE 5.5 Types of Signal Transmission Source Signal

Signal Transmission

Signal Flow

Electrical Optical Optical

Operation site ! remote site E Electrical ! optical–optical E ! Me ee ! TO ! TRO O !O o Optical–optical O ! Mme o ! O ! TRO O !O O Optical–mechanical O ! Mme oo ! O ! TRO O ! TM ! M

Electrical Optical Mechanical

Remote site ! operation site E Electrical ! optical–optical E ! Me ee ! TO ! TRO O !O E Optical ! optical O ! TO ! TRO ! O O E Mechanical –optical M ! TEM ! TO ! TRO O !O

optical, and optical to mechanical. The ﬁrst transmission case is useful when remote operation of mechatronic devices or systems is needed. A simpliﬁed optical data transmission system by optical signal is depicted in Figure 5.8. It consists of three major parts, transmitter, optical ﬁber, and receiver. The transmitter contains a laser or an LED, and appropriate control circuitry: The laser and LED here is a signal transformer that converts an electrical to an optical signal. In a case where the data source is optical, transmitting site

receiving site

laser diode

modulator

optical fiber

optical

(a) electrical-to-optical transmission transmitting site modulator

receiving site optical fiber

optical

optical switch

(b) optical-to-optical transmission transmitting site

receiving site optical

modulator optical switch

optical fiber

(c) optical-to-mechanical transmission FIGURE 5.8 Optical transmission conﬁguration.

mechanical elements

mechanical

272

Optomechatronics

the transforming unit is not necessary and the signal can be sent directly to the input optical ﬁber unit. When a signal from the signal source is fed into the transmitter, the signal is modulated by the transmitter. Then, the modulated light is launched into the ﬁber and transmitted to the receiving unit that retrieves the transported information or data. A light detector such as a photo diode (PD) takes the role of receiving signals. The received signal usually goes through a signal processing such as ampliﬁcation and noise ﬁltering. From the discussion above, it can be seen that in the remote operation shown in Figure 5.8a, signal is transformed from electrical to optical, modulated, transmitted, and transformed back to electrical at the receiving site, and then properly processed and conditioned according to applications. In terms of signal mathematics, the transmission procedure can be written: E E ! Me ee ! TO ! TRO O!O

ð5:4Þ

When a control operation at the sender site is needed, the resulting signal at the remote site is transmitted back to the sender side. In this case, the form of signal mathematics is propagated in a reverse direction as: E e O ˆ TRO O ˆ TO ˆ M e e ˆ E

ð5:5Þ

The second type is a direct optical-to-optical transmission as shown in Figure 5.8b. The input to the ﬁber is an optical signal modulated by an optical switch Mme oo , and the output signal is the optical signal type, which can be used for a variety of applications. It is noted that some means of providing an optical signal to the operation site is necessary. The signal ﬂow mathematics for this case is expressed by: O ! M meoo ! O ! TRO O!O

ð5:6Þ

The third type is optical-to-mechanical, as indicated in Figure 5.8c. In this case, the transmitted optical signal is directly interacted with mechanical elements at the optic – mechanical interface, thus producing a mechanical signal in a desired manner. The signal ﬂow mathematics is described by: O O ! M meee ! O ! TRO O ! TM ! M

ð5:7Þ

This type is a seemingly unachievable transmission type, but can be found from a practical example where the transmitted optical signal operates on a ﬂuid nozzle which controls ﬂuid ﬂow. This alters a deﬂection of the jet from the center position. Some of the details will be discussed in the last section of this Chapter. Signal Display The display element is usually the ﬁnal data representation element. A simple one is a bathroom scale that contains a typical signal display element of the LCD type. There are a number of data presentation devices, from simple pointer scales and recorders to very large scale display devices such

Optomechatronic Integration

273

small-scale alphanumeric

light-emitting diodes (LED)

liquid crystal display (LCD)

large-scale displays (electronic display)

liquid crystal cathod-ray display (LCD) Tube (CRT)

image MEMs based gas plasma discharge intensifier projector

plasma display panel

field emission display

electrolume scene

light emitting diode

FIGURE 5.9 Classiﬁcation of optical data presentation devices.

TABLE 5.6 Various Signal Display Modes Display Mode

Display Principle

Typical Device

E M ! TEM ! TO !O E E ! TEE ! TO !O E O ! TEO ! TO !O o O ! Mme o ! O O ! Me oo ! O

Photoemissive Photosemissive Photosemissive Optical switching Polarization

Weight indicator (scale) Cathode ray tube, LED, PDP Image intensiﬁer, image converter Digital micromirror, grating light valve (GCV) Liquid crystal display (LCD)

as cathode-ray-tubes (CRT) and plasma display panels (PDP), as shown in Figure 5.9. Since brief discussions on the principles of some of these display units have been made in Chapter 4, we will not discuss them here. As shown in the ﬁgure, optical display units can be conveniently categorized into small-scale alphanumeric displays and large displays that all work in a digital domain. As far as the displaying principle is concerned, it is interesting to see that all of these can be grouped into three categories as shown in Table 5.6. The ﬁrst method is to use a light emitting source like a light-emitting diode (LED). The second one is the polarization method which does not emit light, but uses light incoming from other sources. The last display method is to use a signal manipulation technique.

Optomechatronic Interface or Integration Basic Two-Signal Integration In the previous chapters, we have seen a variety of different types of integration. From the observations of such integrations, we realize that

274

Optomechatronics

to become an optomechatronic element requires an optical element (O), a mechanical element (M), and an electrical element (E) all to be properly integrated together while obeying certain laws. As we shall see later, not all types of the integrations made by these three elements will yield an optomechatronic nature. Integration exhibiting such a nature needs to meet certain conditions called optomechatronic integrability. Among all of the integrations, the basic types are the integration of two signal elements, namely, OE, EO, ME, EM, OM, and MO, if homogeneous integration such as EE, OO and MM are excluded. For example, the photo diode having the transformation TEO in Table 5.1 has the capability of transforming an optical to electrical signal, and does not have mechatronic elements in it. This is an optoelectronic integration that can be denoted by OE. The electric motor (ME) has the capability of transforming an electrical signal to a mechanical signal. And the optical actuator employing shape memory alloy (OM) has the capability of converting optical signals to mechanical. Similar arguments can also be applied to the rest of integrations, such as a laser diode (EO), electric generator (EM), and motion induced light generator (MO). The two signal elements can also be observed from some modulators and manipulators upon examination of the modules listed in Table 5.2 and Table 5.3. For instance, optical transmission itself does not interact with mechatronic elements in transmitting optical signals. It represents an optoelectronic element (OE). Because this two-signal element forms the basis of optomechatronic integration, let us take some more practical illustrations. Figure 5.10 depicts various forms of two-signal integrations of OE, EO, EM, ME, OM, and MO. The light emitting diode (LED) in Figure 5.10a emits electromagnetic radiation over a certain band wavelength when forward biased. This integration represents “EO,” the order of which identiﬁes the input– output relation. As discussed in “Signal Conditioning”, in Chapter 4, a light emitting diode (LED), like an electronic diode, directs electricity ﬂow in one direction. When electricity ﬂows from the cathode (positive lead) to the anode (negative lead), wire encapsulated in the LED body emits light, thus releasing photons. This component has a capability of transforming an electric signal to an optical signal. By using causality relation, this phenomenon can be expressed as electric signal as an input and optical signal as an output. The photo diode shown in Figure 5.10b is a photon detector, which is a semiconductor device in which electrons can be excited from the balance band to the conduction band when incident light hits the junction surface. This integration can be represented by OE, which means that an optical signal as an input produces electrical signal as an output. The two conversion elements in the above can be symbolically represented as: laser emitting diode:E þ O ! EO photo diode:O þ E ! OE Unless speciﬁed, we will describe this as OE.

Optomechatronic Integration

reflector

275 motion

electrode

illumination

reflector p type

negative positive

LED chip

n type

piezoelectric material

(b) photo diode

(a) light emitting diode

(c) piezoelectric transducer

motion core S

rotation

coil

arm

separation

S

motion

N

stator

(d) variable capacitance

(e) linear motion variable inductor

stator

rotor

(f) electric motor light

displacement piezoelectric material displacement

optical fiber

bevel painted on black

electrode field coil

deformation directio

terfenol-D rod

(g) magnetostrictive actuator

(h) piezoelectric actuator

diaphram

cavity

gas

light

(j) optical actuator

(i) moving fiber deflection

light

electrostatic bonding

optical fiber

shape memory alloy ( Ti Ni )

glass

light

(k) optical-based shape memory alloy

(l) optical sensor

FIGURE 5.10 Illustrations of various forms of two-signal integration.

Figure 5.10c depicts the phenomenon of piezoelectricity. When a piezoelectric material is subject to external force or pressure which is a mechanical signal, it produces electricity. Figure 5.10d shows the capacitance of an electric capacitor subjected to motion. One plate of the capacitor is made to move freely, while the other plate is ﬁxed. Thus, depending upon the motion applied, this device achieves modulation of electric signal due to change in the capacitance. Figure 5.10e illustrates a typical arrangement of a variable inductor that converts mechanical signal to electrical signal when a coil is subject to a mechanical motion of the core.

276

Optomechatronics

The piezoelectric transducer in the above exhibits the transformation of mechanical signal to electrical signal, while the other devices modulate their signal when they are provided with external mechanical motion. From the point of view of energy conversion, they convert one energy to a different energy form, which, in this case, is mechanical-to-electrical signal conversion. However, from the view point of a functional module, we can call this “signal transformation” rather than “signal modulation.” Symbolically, the above three conversion elements are then described by: M þ E ! ME

ð5:8Þ

Again, the ﬁrst element M denotes mechanical signal, while the second element E is electrical signal. The meaning of the equation implies a combination of mechanical signal generating elements and electrical signal generating elements, denoted by electromechanical or mechatronic elements. Figure 5.10f shows an electromagnetic motion generator when a coil wound element is exposed to an electromagnetic ﬁeld, and, subject to the change of the ﬁeld strength, the element rotates according to the applied electrical signal. The magnetostriction actuator shown in Figure 5.10g utilizes the material property that causes a material to change its length when subjected to an eletro-magnetic ﬁeld. Thus, this transforms electromagnetic signal to mechanical displacement signal. Another actuator similar to this type is the piezo-electric (PZT) actuator shown in Figure 5.10h. This device transforms electrical signal input to mechanical displacement output. The above elements are produced purely by combination of mechanical and electrical elements, and can be expressed by: E þ M ! ME

ð5:9Þ

Figure 5.10i, Figure 5.10j and Figure 5.10k illustrate optical actuators that transform heat generated by lightwaves to mechanical deformation. The deforming optical ﬁber works on the following principle. One end of an optical ﬁber is bevel cut and painted black. When the light illuminates at the other end of the ﬁber, it exhibits a photo-thermal phenomenon. The optical actuator transforms heat to mechanical signal indirectly, because it uses the expansion of gas to generate a mechanical deformation. The cavity is ﬁlled with gas that expands when heated with a light beam. This expansion in the deﬂection of the diaphragm generates vertical direction. Shape memory alloys such as Ti Ni exhibit thermoelastic martensitic transformation, which yields shape-recovery characteristics upon heating. This results in the generation of mechanical motion. The signal transformation for these types of actuation devices is expressed by: O þ M ! OM

ð5:10Þ

The MO integration shown in Figure 5.10l is an optical ﬁber sensor conﬁguration in which mechanical deformation provides changes in

Optomechatronic Integration

277

crystal mirror

photodetector sound undefracted

beam

light beam

diffracted piezoelectric actuation material

beam

(a) mechanical

(b) acousto-optical modulator

scanner

light

actuator motion

photo detector laser artifact

angular displacement light source

(c) optical chopper

deformable glass mirror

wave

PZT actuator stack electric leads

(e) grating sensor

laser

(d) optical encoder

(f) deformable mirror

atom or cell

(g) optical twizer

FIGURE 5.11 Illustrative examples of optomechatronic integration.

intensity of a light beam. This type of integration is described by: M þ O ! MO

ð5:11Þ

Fundamental Optomechatronic Integration: Illustrations As presented in Equation 5.8 through Equation 5.11, we have discussed three types of integration, including optical plus electrical (OE or EO), optical plus mechanical (OM or MO), and mechanical plus electrical (ME or EM). The OE (EO) is called optoelectronic integration. Similarly, the OM (MO) and ME (EM) are called optomechanical and mechatronic, respectively. When all of three basic elements, optical, mechanical, and electrical (O, M, E), are combined together, the combination may produce optomechatronic integration. The integration can be achieved with a variety of combinatorial types, which we have seen in Chapter 1. Figure 5.11 depicts various different types of optomechatronic integrations. The ﬁrst three devices are optical modulators. It can be seen that all of these devices do not change the form of the original input signal, but retain its optical signal form. They modulate direction, wave form, and amplitude, as we shall discuss in detail in Chapter 6. Figure 5.11a is a polygon scanner that diverts a light beam in a desired direction. This scanner consists of a light source (OE) and a rotating mirror (ME) which constitutes optomechatronic integration (OPME). An acousto-opto modulator, shown in Figure 5.11b, modulates the optical signal in such a way that it diverts a light beam in a certain direction by combining light waves with sound waves generated by excitation of a piezoelectric material. It is composed of a light source (OE) and an acoustic generator (ME), resulting in optomechatronic integration (OPME). Figure 5.11c shows an optical chopper, discussed already, that modulates a light beam

278

Optomechatronics

having a wave form and frequency different from the original signal. A light source (OE) and a rotating wheel (ME) constitutes an essential element of optomechatronic integration (OPME). Summarizing all of the above integration types, the modulators can be expressed by: OE þ ME ! OPME The sensors shown in Figure 5.11d,e exhibit all optomechatronic integrations. Shown in Figure 5.11d is an optical angle encoder, which is composed of a light source (OE), a rotating coded wheel (ME), and a separate photo sensor (OE) for each track. It has alternating transparent and opaque areas patterned on the rotating disk. A grating sensor shown in Figure 5.11e also runs on the optomechatronic principle [4]. It measures an artifact topology by using controlled motion of an optical grating device (ME), through the grating of which light (OE) is diffracted differently depending on the surface topology. When the conﬁguration in Figure 5.10l is combined with a photo sensor, the combined device becomes an optical ﬁber sensor based on the modulation principle. It modulates the amplitude of a light beam passing through the optical ﬁber (OE), which can be deformed by bimetal displacement due to temperature variation (M). This type of integration can be regarded as a signal transformation, since it converts a mechanical signal (bimetal strip displacement) into an optical signal (light beam amplitude). In summary, the integrated forms of the sensors discussed above are expressed in the following: optical encoder: OE þ ME þ OE ! OPME optical grating sensor: OE þ ME þ OE ! OPME optical ﬁber sensor: OE þ M ! OPME Two devices shown in Figure 5.11f,g are related to manipulation or actuation that can run on optomechatronic principles. The deformable mirror shown in Figure 5.11f is a mirror that can be locally deformable by multiple-stacked piezo actuators (ME). The state of deformation is controlled by translation of the actuators, depending on the distortion characteristics of incoming light waves (OE). The optical twizer shown in Figure 5.11g is a device that can directly manipulate atoms or nanometer objects by optical means. The motion of small nanoobjects (M) is controlled by the standing light wave generated by a laser light (OE). Therefore, this device can trap objects in the minima of the standing wave condition. From the integration viewpoint, two optical devices are expressed by: deformable mirror: OE þ ME ! OPME optical twizer: OE þ M ! OPME Figure 5.12 shows various types of optomechatronic-based manufacturing processes. Figure 5.12a is an inspection process of measuring the parts on a moving conveyor (ME) by using a CCD camera (OE). The laser welding system in Figure 5.12b is also based on the optomechatronic-based processing concept that the workpiece to be welded by the laser head (OE) is

Optomechatronic Integration

279 laser

camera

optical module/ wave guide

illumination

welding head parts conveyor

(a) inspection of moving parts

work piece

LED

lens

6 axis stage

motor

displacement

(b) laser welding

(c) optical packaging

FIGURE 5.12 Various types of optomechatronic integration used for manufacturing processes.

controlled according to the joining path plan (ME). The laser head is equipped with an autofocusing device, which is composed of a set of lenses (O) and an actuation mechanism (ME). This device itself, therefore, exhibits optomechatronic characteristics. An optical packaging process shown in Figure 5.12c uses a laser beam (OE) to align the photo diode by using a multiaxis adjustable stage (ME). Depending upon the degree of the alignment between the laser beam and the photo diode, the amount of light intensity detected by a photo sensor (OE) becomes different. The manufacturing processes considered above have the following integration form: vision-based inspection: ME þ OE ! OPME laser welding: OE þ O þ ME þ O þ ME ! OPME alignment process: OE þ ME þ OE ! OPME Based on the discussions we have made above, it is clear that optomechatronic features can be found from either one single functional module or more than two modules combined together to generate an output signal. In the case of a single module, optical and mechatronic signals within it are properly integrated to generate an output signal. When multiple functional modules are combined to generate an output signal, the condition for optomechatronic integration should be such that at least one module must exhibit the nature of one optical module, and one mechatronic module must be included for the combination. Generic Forms for Optomechatronic Interface The above observation enables us to deﬁne the basic forms of conﬁguration for optomechatronic integration. Figure 5.13 describes two such types, classiﬁed according to the change of the signal type between the input and output ports. Figure 5.13a indicates the case when the output signal has a different signal type as a result of integration. Most of the functional modules discussed in the previous section, except modulators, belong to this class.

280

Optomechatronics interface

integration module

input signal signal type :

modulating signal

interface

integration module

input signal

output signal

output signal

input signal = output signal

input signal ≠ output signal

(a) transformation type

(b) manipulation type

FIGURE 5.13 Basic types of optomechatronic integration.

This is called “transformation type” optomechatronic integration. In contrast, Figure 5.13b indicates that the type of output signal is not changed as a result of integration, retaining the same signal as that of the input signal, as can be seen from the modulators discussed in the previous section. This type is termed “modulation type” integration. Let us further examine details of each type of optomechatronic integration. As can be seen from Figure 5.14, the transformation type integration has basically three families of conﬁgurations. The thick line indicated in the ﬁgure denotes the interface between the input and output signals. The ﬁrst conﬁguration is the combination of a single-signal with one transformation module, which is shown in Figure 5.14a. When there appears no interacting external signal, a single signal transformer alone interface

O

E

EM

M

OM

O

O

O

M

E

M

MO

M

E

OE

E

ME

M

M

E

functional module

EO

E

O

O

(a) a signal with one transformation module O

TE O

OE

E

E

O

OM

EM

TO M

E

M

TM

TE

ME

EO

O

O

E

E

EM

M

TM

OM

TO M

M

M

TE

TO

MO

O

M

(b) combination of two transformation modules O E M

OPME OME

MO

O

O

O or M or E

(c) coupling between three different signals FIGURE 5.14 Interface for an optomechatronic transformation type integration.

ME

TE

OE

E

E

M

M

E

M

O

TM

E

TM

E

TO

EO

O

Optomechatronic Integration

281

cannot be of optomechatronic nature. The reason is that with such one transformer optomechatronic integration cannot be physically feasible, since an input signal cannot be transformed into another at the interface without an additional transforming unit. For instance, when an optical signal is present at the interface of a mechatronic transformer (ME, EM), the coupling result should produce either a mechanical or electrical signal, so that the transformer can accommodate an optical signal. However, an optical signal cannot be coupled with either one of the two signals, since there is no unit transforming the optical signal to either a mechanical or electrical signal. Therefore, an interacting signal to the transformer must be present that has a signal type different from those inputs and outputs involved with the transformation. In other words, it must occur under the presence of a signal type different from those of the transformer. Figure 5.14a shows all six feasible conﬁgurations of integration. The ﬁrst integration implies the case when an electrical-to-mechanical (E – M) transformation occurs under the inﬂuence of an optical signal. Due to the presence of an optical signal, the transformation may result in different characteristics. As we shall see later in “Optomechatronic Actuation” in Chapter 6, an interesting example of this case can be found in which the presence of an optical signal inﬂuences an electrostatic force between two capacitive plates. The last integration type in the ﬁgure describes an E –O transformation under the inﬂuence of a mechanical signal. Likewise, all the rest of the integrations shown in the ﬁgure can be physically interpreted. The second conﬁguration is the combination of two functional modules shown in Figure 5.14b. This is essentially a multiprocess signal-transforming element. Some examples of such transformations are illustrated. For example, optically driven motion of mechatronic elements belong to the conﬁguration shown in the ﬁrst ﬁgure. The transformation yields mechanical signal motion to a mechanical element by applying an electrical signal transformed from an optical signal. A practical case of this conﬁguration is a photodiode-driven PZT motion. Optical signal generation by an electrical signal, which in turn is produced by a mechanical signal, is another example of the integration shown in the last ﬁgure. A PZT driven laser diode well represents this conﬁguration. Typical examples that belong to this type of integration are given in Table 5.7. The mathematical forms of the transformation integrations given in Figure 5.14 are expressed by: E O TEO þ TM ! TM

E O E TO þ TM ! TM

M TO þ TEO ! TEM

O þ TEM ! TEO TM

E M E þ TO ! TO TM

E M TEM þ TO ! TO

The third conﬁguration is the combination of three single signal elements, denoted by OPME, as shown in Figure 5.14c. This conﬁguration produces an output, any one of the three optical, mechanical, or optical signals. The integration can be made in any order, depending upon the physical

282

Optomechatronics

TABLE 5.7 Symbols and Mathematics for Various Types of Integrations Type of Integration

Symbol

Integration Mathematics

Typical Phenomenon, Device

reflector

Electricalto-optical

EO

reflector

E!O

negative positive

E þ O ! OE

LED chip

illumination p type Optical-toelectrical

OE

O!E O þ E ! OE

n type electrode

Mechanical-toelectrical

ME

motion

M!E M þ E ! ME

piezoelectric material motion

Mechanical-toelectrical

ME

M!E

S

separation

M þ E ! ME

core Mechanical-toelectrical

ME

M!E M þ E ! ME

coil

arm motion

continued

Optomechatronic Integration

283

TABLE 5.7 CONTINUED Type of Integration

Symbol

Integration Mathematics

Typical Phenomenon, Device

rotation

Electrical-tomechanical

EM

E!M E þ M ! EM

S

N

stator

rotor

stator

displacement Electrical-tomechanical

EM

E!M E þ M ! EM

field coil terfenol-D rod displacement

Electrical-tomechanical

EM

piezoelectric material

E!M

electrode

E þ M ! EM

light

Optical-tomechanical

OM

O!M

optical fiber

bevel painted on black

O þ M ! OM

deformation direction diaphram

cavity

Optical-tomechanical

OM

O!M

electrostatic bonding

gas

O þ M ! OM

light

glass

continued

284

Optomechatronics

TABLE 5.7 CONTINUED Type of Integration

Symbol

Optical-tomechanical

OM

Integration Mathematics

Typical Phenomenon, Device

M!O M þ O ! MO

optical fiber

TABLE 5.8 Integration Type and Mathematic for Various Optomechatronic Integrations Integration Type

Symbol

Integration Mathematics

Typical Phenomenon Device, Processes

Object manipulation/actuation

mirror EO ! ME ! OH

OPME

OE ! ME ! OPME

beam

crystal

undefracted

EO ! M ! O

OE ! M ! OPME

diffracted beam

sound

actuation

EO ! ME ! O

piezoelectric material

OE þ ME ! OPME

light beam

continued

Optomechatronic Integration

285

TABLE 5.8 CONTINUED Integration Type

Symbol

O ! ME ! O

Integration Mathematics

Typical Phenomenon Device, Processes light

O þ ME þ O ! OPME

deformable glass mirror

PZT actuator stack

electric leads

OE ! ME

laser

OE þ ME ! OPME

wave

Sensing

EO ! ME ! OE

OPME

atomor cell photo detector

OE þ ME þ OE ! OPME

angular displacement

light source actuator motion

EO ! ME ! OE

OE þ ME þ OE ! OPME

det laser

artifact

Manufacturing system or process laser

EO ! ME ! O ! ME

OPME

OE þ O þ ME þ O þ ME ! OPME

welding head work piece

motor

displacement

continued

286

Optomechatronics

TABLE 5.8 CONTINUED Integration Type

Integration Mathematics

Symbol

Typical Phenomenon Device, Processes camera

EO ! ME ! OE

OE þ ME þ OE ! OPME

illumination

parts conveyor

EO ! ME ! OE

OE þ ME þ OE ! OPME

LED

optical module/wave guide 6 axis stage lens

phenomena involved. The mathematics of this integration is expressed by: O þ M þ E ! OPME In Table 5.8, typical devices that satisfy this relationship are summarized together with integration mathematics. The manipulation type shown in Figure 5.15 has basically six conﬁgurations. As we have already seen above, this type produces the same type of input signal as output, by utilizing a signal manipulator in the rectangular box. All manipulators are of the transformation type, operated on a twosignal integration element. The ﬁrst conﬁguration of the manipulation type shown in Figure 5.15a is a mechatronic type, since optical signal manipulation is achieved by mechatronic signal transformation. The second type in Figure 5.15b is an optomechanical type, whereas the last one is O

E O

EM

O

E

ME

E

M

M

M O

OM

E

O

(a) mechatronic type

E

MO

EO

M

O E

(b) optomechanical type

M

OE

(c) optoelectronic type

FIGURE 5.15 Interface for an optomechatronic modulation type integration.

M

Optomechatronic Integration

287

the optoelectronic type in Figure 5.15c. The mathematics involved with the manipulation-type integration may be expressed by: O ! Memoo ! O

E ! Momee ! E

m M ! Meom !M

O ! Mmeoo ! O

E ! Mmoee ! E

m M ! Moem !M

A physical example of the ﬁrst signal integration type can be observed from the optical modulator and mechanical beam scanner, as discussed in the “Basic Functional Modules” section. The other integration types may be conceivable, and may be found from some practical examples. In actual practice, optomechatronic integration can take place with the combination of more than two transformation modules. For example, the photo diode-actuated electromechanical device at a remote site belongs to this case. The mathematics of this integration can be given by: O E E E þ TRO TO O þ TE þ T M ! TM E denotes the signal transformation from electrical to optical It is noted that TO from a laser diode, TRO O denotes signal transmission to a remote site in the form of an optical signal, and ﬁnally, the signal transformation is back to a mechanical signal. When a piezoelectric sensor signal at a remote site is optically transmitted to a central control station, and then is transformed back to an electrical signal, the mathematics involved in this the integration can be expressed by: E O E þ TRO TEM þ TO O þ TE ! T M

The ﬁrst term indicates the transformation of a signal from mechanical to electrical by the piezoelectric sensor, the next two terms involve signal transmission, and the last term indicates the transmitted optical signal back to an electrical signal at the control station. Another integration type can take place when a signal transformer and manipulator are combined together. A combination of a signal transformation and a signal manipulator occurs when laser light impinges on to a deformable mirror. The resulting integration is a light distributor, whose mathematics is written by E TO þ Mmeoo ! O

where the ﬁrst term indicates a transformation for laser light and the second indicates the optical signal modulation by a mechatronic actuator. Another example of this category is an electric motor with an optical encoder. In this integration, the rotational motion of the motor is generated by a signal transformer, and then is measured by the encoder for feedback. The interface diagram for this interface cannot be seen in Figure 5.14 or Figure 5.15 because this interface requires two separate types of modules; one is

288

Optomechatronics

transformation and the other is modulation. In this case, the plus sign for mathematical expression can not be physically feasible, since the causality at the interface between two signal functional modules is not met. To this end, we use the multiplication sign to describe the mathematical expression as: E E ! TM £ Mmeoo ! O

Integrability So far, we have seen a number of different combinations conceivable for optomechatronic integration. However, it can be argued that not all integrations listed here can be expected to make the resulting device or system physically work. From this point of view, it appears that there need to be several considerations for devising a plausible interface between the integrated signals, to make such integrations physically realizable. Two important considerations can be made regarding the integrability. These are the interfaciability, and the output power requirement. The interfaciability refers to causality considerations, whereas output power requirement refers to the ability to produce the power that can be usable at the output side. Causality is the relationship between the signal at the input side and the signal at the receiving unit when they are interconnected at the interface. In other words, the input signal to the interface should be transformable, in order that the signal receiving unit can accommodate it. For example, if the integration shown in Figure 5.14a is considered, each input signal should be interfaciable to the port of each transforming unit. If this is not the case, the integration cannot possibly happen. The same arguments can be applied to the other integration cases shown in Figure 5.14b,c. The other consideration of integrability is the magnitude of energy or power attainable at the output port for a given input signal level. If the output magnitude is very small, the output signal may be affected by noise excitation, or require a very large input signal to attain a relatively large output signal. For a signal within some limited range, this kind of integration becomes problematic, although causality is met. Let us take one interface for an integration, for example, the second type O ! M, in Figure 5.14a. From what we have seen from the piezoelectric actuator, if the required power of the optical signal is too large to attain a desired mechanical signal, this integration module may not be appropriate or feasible. The same requirement on output power will be needed for the other two cases in Figure 5.14b,c. Photo diode-actuated mechatronic elements at remote sites are one such example that may need stringent power requirements. The transmitted optical signal is normally attenuated, and its transmitted optical power may not be large to provide sufﬁcient input to the photo diode. In other words, sufﬁcient power to drive the mechatronic element may not be attainable either due to attenuation during transmission, TRO O , or power loss during transformation. Another example can be found from the devices, such as optically driven mechatronic elements, acousto –optical modulators ðMmeoo Þ:,

Optomechatronic Integration

289

and mechanical scanners ðMmeoo Þ: As we will discuss in the next chapter, all of these devices will need to consider efﬁciency in their output power. Signal Flow An optomechatronic device or system may be full of functional modules or single signal elements. Figure 5.16 indicates the two generic conﬁgurations exhibiting an optomechatronic nature that can be made by optic – mechatronic interface. The arrow indicates the ﬂow of information from one module to another. The open loop structures shown in Figure 5.16a represents the case when the interfacing would occur in a series, that is, all modules are combined in series to generate an output signal. It has three basic forms, which are series, parallel, and series-parallel. The series structure has the conﬁguration of functional modules that are coupled in series such that the output signal of each module ﬂows from one to another. In the parallel structure, however, two modules are put together in parallel, such that their output signals are added together. The series-parallel structure is different from the two in that the divided signals from the proceeding module enter into the two different modules next to it. A device of this type is conceivable when an optical signal is split into two by a beam splitter. Then, each split signal enters into its own autofocusing functional module. The conﬁgurations shown in Figure 5.16b have a feedback structure in that the ﬁnal output is fed back to at least one of the modules within the system. Two feedback structures shown in Figure 5.16b differ from their feed forward (open) loop. The basic case has a single path, while the other form has a multiple path (in this instance, two paths). An example can be illustrated with a camera with autofocusing capability, as mentioned before. In this example, one of the relevant modules in the series has a feedback loop within the integration, as module

module

module

signal

module

+

module

signal

− module

+ + module module

signal

+

module +

module module

(a) open-loop configuration

module module module

(b) feedback configuration

FIGURE 5.16 Basic open loop and feedback conﬁgurations for signal ﬂow.

signal

290

Optomechatronics

transformer E

O Er

TEO

o me o

M

+_

E

Er +

_

Md

+

TME

M

O

transformer PSD sensor TEO

laser

E

transformer

(b) measurement of machined workpiece surface optical actuator

TOE

E

o meo

TME

TME

(a) camera

O

TOE

TOM

Mme oo

optical power

gripper

TMM

M

optical mirror gripper

UV light optical fiber iris (valve)

O M

T

optical actuator

TEM

mirror

mirror supporter

motor

sensor

(c) force feedback control of a two-fingered gripper FIGURE 5.17 Illustrations of the closed-loop system.

shown in Figure 5.17a. The optical signal (O) is continuously adjusted to make a feedback control of a lens location focused until a certain image can be focused to a satisfactory level. Another example is an optical power control which has been previously discussed in a laser-controlled machining system. In this example, focusing is made by adjusting the position of the optical system in the laser power system. The schematic diagram illustrated in Figure 5.17b shows an optical sensor-based control for mechatronic system, as discussed in “Optomechatronics: Deﬁnition and Fundamental Concept” in Chapter 1. Basically, it has three transforming elements and one modulator: The mechatronic E element denoted by TM has a signal transforming element (from electrical E has a transformation to mechanical signal); a laser diode denoted by TO (from electrical to optical signal); a photo sensor TEO has a signal transforming element (from optical to electrical signal). The module in the middle E represents the interaction with the mechatronic and optical elements. TM The modulation module, Mmeoo ; receives a laser beam produced by a laser E , and impinges it upon the surface of the mechatronic element (e.g., source TO rotating workpiece). The incident beam reﬂects differently and the reﬂected beam enters to the photo sensor, TEO depending on the machined surface texture of the workspace being machined. This signal is feedback to compare E with the desired electrical signal. Based on the difference signal, TM module generates an electrical command signal to a driving motor, which acts as a modulation module Mme oo .

Optomechatronic Integration

291

The cascade feedback form shown in Figure 5.17c has basically the same structure as that of the basic feedback form, except the cascade element in the open-loop path. One nature of this feedback form can be found in the case when two optical actuators actuate a mechanical element composed of two grippers which grasp an object with a desired force, as depicted in the ﬁgure. The gripping force signal generated by a force sensor can be used to detect the state of gripping, and fed back to the controller element of the controlling optical power coming from a light source. The operation principle of this optically-actuated two-ﬁgured gripper is as follows: A O E , actuates two optical actuators TM proper amount of optical power TO and then two grippers accordingly produce small mechanical deformations M Þ due to the gripping force. This force (mechanical signal, M) of an object ðTM is detected by a force sensor ðTEM Þ and fed back to an optical power controller force to compare with the desired mechanical signal ðMd Þ:

Integration-Generated Functionalities In the previous sections we have deﬁned basic functional modules that generate functionalities required for optomechatronic systems, discussed the integrability of the modules to create optomechatronic characteristics, and then studied the signal ﬂow graph that represents the ﬂow of information from input port to ﬁnal output port. All of these considerations are necessary to design a certain functionality speciﬁed for optomechatronic devices, processes, or systems. Our interest here is to know how they are generated by a single module, or by combining the functional modules. In this subsection, we will brieﬂy treat these subjects and illustrate how functional modules are used to produce the required functionalities for an optomechatronic device or system. Any engineering system that includes devices, machines, or processes must possess one or more functionalities in order to perform the desired tasks as speciﬁed by the designer. In essence, such a functionality is created by an individual module or combination of functional signal modules. Depending upon how many modules are involved, or how they are interacted with each other, the functionality can have different attributes. Various functionalities created in this way can be combined to make systems exhibit a certain functional behavior. In general, a system or device will need a variety of functionalities to perform a given task. The fundamental functionalities are the common functionalities that are frequently utilized everywhere. Let us consider here some of the fundamental functionalities listed in the Introduction. Figure 5.18 shows a variety of conﬁgurations to create a functionality. This conﬁguration is basically similar to those shown for basic conﬁguration in Figure 5.16. Figure 5.18a shows the case when a single module is needed to generate a functionality. The series conﬁguration in Figure 5.18b shows the case when signal ﬂow occurs in a forward direction,

292

Optomechatronics

functional module

input signal

functionality

(a) single module input signal

functional module

functional module

functional module

…

functionality

(b) series input signal

input signal

functional module

input signal + +

functionality

+

−

functional module

functional module

functional module

(c) parallel-series

functional module

(d) feedback

FIGURE 5.18 Various conﬁgurations to generate a functionality.

FIGURE 5.19 Functional modules for a pneumatic pressure control system.

functionality

Optomechatronic Integration

293

while the parallel series utilizes a scheme of signal processing in parallel, as shown in Figure 5.18c. Lastly, Figure 5.18d shows a feedback scheme to produce a functionality. Figure 5.19 illustrates the remote control of a pneumatic power system located in a remote site. This system is all optically operated, so that it uses all-optical operation modes, e.g., optical actuators and sensors. The pneumatic, ﬂapper-nozzle valve is controlled by an optical actuator, as considered in the previous discussion. The operation starts when the highpower optical signal transmitted through an optical ﬁber enters the optical actuator. The actuator, when activated, causes the ﬂexible membrane to move closer towards, or farther from, the nozzle, which accordingly yields a change in backpressure; closer means higher back pressure, while farther means lower backpressure. This change is ampliﬁed by a pneumatic gain block. The ampliﬁed pneumatic pressure operates a spool valve, and actuates a pneumatic cylinder connected to a system to be controlled. This control system, therefore, belongs to the feedback form shown in Figure 5.18d. Figure 5.20 illustrates the diffraction-based sensing concept that can be used for an atomic force microscope (AFM), which will be treated in Chapter 7. The sensing uses the principle that the variation of grating distance results in the variation of diffraction angle of a laser beam (ﬁrst order), as discussed in “Diffraction” in Chapter 2. The sensing tip is

actuator motion

photo detector

comb drive

laser

spring AFM tip

artifact

sample (a) diffraction grating sensor measurement function

input signal

+_

actuation module (comb drive)

measurement signal

sensor system

sensing module

(b) functional module diagram FIGURE 5.20 Functional modules for a diffraction grating sensor.

output distance

294

Optomechatronics

supported by a diffraction grating whose end is connected to a comb drive actuator. Since the grating element inherently possesses a spring element due to its material and structural composition, the grating element becomes compressed when the whole sensing unit is scanned through and encounters rough artifact. Note that the support element is ﬁxed at a constant height, as shown in the ﬁgure. When laser light is interfaced with this variable grating, it results in the angle variation of the diffracted beam, depending upon how the grating element is displaced.

Problems P5.1. The following are some single-stage transformation modules that interact with a single signal considered in Figure 5.14a. Illustrate a physical device or system corresponding to each case (Figure P5.1).

O

E

ME

M

E

O

O

OM

M

EM

E

M

FIGURE P5.1 Single –stage transformation modules.

P5.2. Illustrate a physical element, device, or system that works according to each of the following: two-stage module composed of transformation and/or modulation type as shown in Figure P5.2.

E

E

O

TO

O

TM

M

E

E

M

TE

E

o

Mme o

O

(2)

(1) M

O

TO

E

TO

(3)

O

O

O

TM

M

M

TE

E

(4)

FIGURE P5.2 Two–stage transformation modules.

P5.3. Repeat the same problem for the multi-stage modules composed of signal transformation and signal modulation as depicted in Figure P5.3.

Optomechatronic Integration

295 O

E

E

TO

O

o

Mme o

O

TE

TM

E

M

O

E

(1) O M

M

TE

E

M

E

TM

o

Mm o

O

TE

(2) FIGURE P5.3 Multi–stage modules composed of signal transformation and modulation.

P5.4. The following signal ﬂow graph indicates a feedback control system in which the position of an optical modulation type device ðMme oo Þ: is varied depending on the magnitude of electrical output of a transformation module ðTEO Þ: The transformation module receives the modulator’s optical output as its input as can be seen from Figure P5.4. (1) Describe a detailed operation principle of the feedback system. (2) Devise an optomechatronic system that obeys this signal ﬂow.

E

E

TO

O

o

Mme o

O

O

TE

E

E

TM

FIGURE P5.4 The signal ﬂow graph for an optomechatronic system.

P5.5. Consider a remote operation of the pneumatic pressure control system shown in Figure 5.19 and redrawn here for this problem in Figure P5.5. (1) Write down signal ﬂow mathematics for this system. (2) Draw a complete signal ﬂow graph describing remote operation of this system. Assume that the sender (control) site uses an LED for signal transformation. P5.6. Figure P5.6 shows an optically-ignited mechatronic system discussed in the Introduction. Repeat the same problem as given in P5.5 (1) and (2).

296

Optomechatronics

FIGURE P5.5 Remote operation of a pneumatic pressure control system.

laser path

sapphire window/case/seal projectile

collimator

propellant charge

FIGURE P5.6 Optically-ignited mechatronic weapon system.

P5.7. Figure P5.7 shows a remote operation of an inspection system, operated by a robot, also discussed in the Introduction. The transmission is carried out by an optical signal. (1) What type of transceiver in the control site do you recommend? (2) What type of transceiver in the remote site do you recommend?A transceiver contains a module for transmission and a module for receiving. control site

remote site

monitor

transceiver

transceiver fiber PC

FIGURE P5.7 Data transmission for remote operations.

sensor

Optomechatronic Integration

297

(3) Write down the signal ﬂow mathematics for this system for each. (4) Draw a complete signal ﬂow graph describing the remote operation of this system. Assume that there is no feedback operation at the control site, that is, one-way operation. P5.8. Laser welding process considered for discussion in “IntegrationGenerated Functionalities” in Chapter 5 is shown in Figure P5.8. (1) Describe in detail how it works to join two metal plates. (2) Based on this ﬁgure, draw a complete signal ﬂow graph to carry out the welding task. laser

welding head work piece motor displacement FIGURE P5.8 A laser welding system.

References [1] Cho, H.S. Characteristics of Optomechatronic Systems, Chapter 1, Optomechatronic Systems Handbook. CRC Press, 2002. [2] Cho, H.S. and Kim, M.Y., Optomechatronic technology: the characteristics and perspectives, special issue on optomechatronics: fusion of optical and mechatronic engineering, IEEE Transaction on Industrial Electronics, 52:4, 932– 943, 2005. [3] Hockaday, B.D. and Waters, J.P. Direct optical-to-mechanical actuation, Applied Optics, 29:31, 4629– 4632, 1990. [4] Kim, S.G., Optically sensed in-plane AFM tip with on-board actuator, Lecture No. 6.777, Design and Fabrication of Microelectromechanical Device, Final report of Design Project, MIT, 2002. [5] Liu, K. and Jones, B.E. Pressure sensors and actuators incorporating optical ﬁbre links, Sensors and Actuators, 17:314, 501– 507, 1989. [6] Ogata, K. Modern Control Engineering, International Edition. Prentice-Hall, Englewood Cliffs, NJ, 1970.

6 Basic Optomechatronic Functional Units CONTENTS Optomechatronic Actuation............................................................................. 301 Silicon Capacitive Actuator ...................................................................... 302 Optical Piezoelectric Actuator ................................................................. 304 Mathematical Model of the Photon-Induced Currents................ 305 Induced Strains of the PLZT ............................................................ 306 Photo-Thermal Actuator ........................................................................... 311 Optomechatronic Sensing................................................................................. 316 Optical Sensor............................................................................................. 316 Fabry-Perot Etalon.............................................................................. 318 Fiber Optic Sensors .................................................................................... 321 Automatic Optical Focusing ............................................................................ 326 Optical System Conﬁguration.................................................................. 327 Optical Resolution ..................................................................................... 328 Axial Resolution ................................................................................. 329 Feedback Control of the Objective Lens ................................................ 330 Effect of External Disturbance.......................................................... 336 Focus Measurement ........................................................................... 338 Acoustic-Opto Modulator ................................................................................ 339 Deﬂector ...................................................................................................... 341 Frequency Shifter ....................................................................................... 345 Tunable Wavelength Filtering .................................................................. 347 Efﬁciency of Modulation and Speed....................................................... 347 Optical Scanning ................................................................................................ 348 Galvanometer ............................................................................................. 349 Feedback Control of Galvanometer ........................................................ 356 Polygonal Scanner...................................................................................... 363 Correcting Scan Errors....................................................................... 367 Optical Switch .................................................................................................... 367 Thermally Actuated Mirror...................................................................... 369 Electrostatically Actuated Mirror Control ............................................. 371 Lens Controlled Switching ....................................................................... 375 Zoom Control ..................................................................................................... 377

299

300

Optomechatronics

Zooming Principle ..................................................................................... 377 Zoom Control Mechanism........................................................................ 383 Visual Autofocusing .......................................................................................... 386 Image Blurring............................................................................................ 386 Focus Measure............................................................................................ 389 Illumination Control.......................................................................................... 399 Illumination Methods................................................................................ 400 Illumination Control.................................................................................. 403 Illumination Quality Measure.................................................................. 408 Autofocusing with Illumination Control ............................................... 410 Visual (Optical) Information Feedback Control............................................ 411 Visual Feedback Control Architectures .................................................. 414 Fixed Camera Conﬁguration ............................................................ 417 Eye-In-Hand Conﬁguration .............................................................. 420 Feedback Controller Design ............................................................. 421 Optical Signal Transmission............................................................................. 428 Signal Transmission ................................................................................... 428 Power Transmission and Detection ........................................................ 431 Problems.............................................................................................................. 432 References ........................................................................................................... 443 In the previous chapter, we have seen that signal elements (mechanical, optical, and electrical) can be combined in a variety of ways to produce basic functional modules which include transformation, manipulation, transduction, actuation, transmission, storage, and display of the signal. These modules were often shown to be optomechatronic in nature, or may be formed by other natures such as optoelectronic, mechatronic, and so on, as we have discussed in Chapter 5. Due to the presence of optomechatronic interaction, the integration was found to create a variety of different types of functional modules, adding more attributes to the existing ones. Any engineering system that includes devices, machines, or processes must possess one or more functionalities in order to perform the desired tasks as speciﬁed by the designer. In essence, such a functionality is created by an individual module or combination of functional signal modules. Depending upon how many modules are involved or how they interact with each other, the functionality may have different attributes. The functionalities created in this way can be combined to make a system exhibit a certain functional behavior as speciﬁed by the designer. In general, a system or device will need a certain number of functionalities to perform a given task. But, in case the device carries out a number of different tasks, it will need a number of different functionalities. Generalizing this notion to the case of many systems or devices, we see that a number of functionalities may be frequently used for certain tasks. The fundamental functionalities are those functionalities that are common everywhere in optomechatronic-related engineering ﬁelds. In this chapter,

Basic Optomechatronic Functional Units

301

we will consider the fundamental functionalities produced by the modules that are the basis of creating optomechatronic properties. As mentioned brieﬂy in Introduction, these properties include: (1) optomechatronic actuation, (2) optomechatronic sensing, (3) optical autofocusing, (4) AO modulator, (5) optical scanning, (6) optical switching, (7) zoom control, (8) visual autofocusing, (9) illumination control, (10) visual servoing, and (11) optical signal transmission. Because these functionalities are frequently used in most optomechatronic systems, we will focus on understanding their basic concept, hardware conﬁguration, and important factors associated with their performance.

Optomechatronic Actuation The actuating function, one of the fundamental functions needed for actuating optomechatronic systems, is the actuation produced by a mechatronic actuator that drives optical systems, or by an optical drive that actuates mechatronic systems. The types of optomechatronic actuation can be grouped into three classes. The ﬁrst class is the optical actuators which employ the optically-driven actuation principle. In this case, the energy supplied for actuation comes from a light source, which provides displacement or force to moving mechanisms. A variety of this type of actuator has been developed for different applications. The second class is the mechatronic actuators with embedded optical units: Here, mechatronic actuators implies all non-optical actuators. An electrical motor with an optical encoder is a typical example. The third class encompasses the actuators that drive an optical or optomechatronic system. In this chapter, we will discuss some of the optically driven actuators. Optical actuators function when light is either directly or indirectly transformed into mechanical deformation that generates small displacements of micro- or nanometer scale. Depending upon the interaction of light energy with an absorbing surface, a variety of effects can occur. According to their nature, two types of optical actuations are considered here: photoelectric actuation and photo-thermal actuation. Photo-electric actuation utilizes the conversion of the variations in light intensity into the change in the electric power by means of a p –n junction of semiconductor devices, or piezoelectric material resulting from the generation of photo electrons. The other form of conversion uses the change in capacitance in a capacitor type of actuator conﬁguration. In addition to these types, optical actuators that belong to this photo-electric class use a variety of other conversion methods, including photo-conductivity modulation, photodiode, direct optical manipulation (optical twizer), and so on. In contrast to this principle, photo thermal actuation utilizes the conversion of the variation in light energy into the change in thermal energy.

302

Optomechatronics

Silicon Capacitive Actuator Silicon micro actuator is one of the photo-electric actuators that utilize the photon-generated electrons. This actuator uses change in the stored charges in a capacitor, which results in change in the electrostatic pressure acting on a cantilevered beam. This actuator therefore does not use piezoelectric or thermal effects for silicon microstructures. As shown in Figure 6.1a, the actuator is composed of a cantilever beam (Si) and a ground plate (Cu) on an insulating glass, which forms a parallel plate capacitor [33]. This parallelplate capacitor is given by a simple relation according to Equation 4.2 A d

C0 ¼ 10 1r

where A is the area of the cantilever facing the ground plate, 10 is the permittivity of the free space, 1r is the permittivity constant of the medium between the gap, which in this case is air, and d is the gap between the cantilever and the ground plate. If an electric voltage V0 is applied to the capacitor through a resistor R, the stored charge is obtained by q0 ¼ C0 V0

ð6:1Þ

Now, let us consider the optical actuator subject to the electrical potential V, whose gap is not constant due to the cantilevered beam being clamped at one end. The stored charge will then produce an electrostatic force, which in turn causes a deformation of the cantilever. Since the beam has a clamped-free condition at its ends, the deﬂection d at the tip can be described by

d ¼ kq q2

ð6:2Þ

where kq is a constant related to the geometry and material property of the cantilever. This relation can be derived from a simple beam theory from which the deﬂection of a cantilever beam is governed by d4 z 4 ¼ pðxÞ dx4

cantilever (Si) d

d

e ee

V

light ground plane(Cu) glass

R (a) actuator configuration

FIGURE 6.1 Semiconductor capacitive optical actuator.

total current (i). pA

ð6:3Þ

O

light intensity

DC bias voltage, Vb

(b) variation of total current with DC bias voltage

Basic Optomechatronic Functional Units

303

where z is the vertical displacement at the location x along the beam, and pðxÞ is the distributed loading due to the applied voltage. In the above equation, the cantilever has small deﬂection and the thickness h p ‘: Attention should be given in the above equation that, due to deﬂection, the electrostatic force is no longer uniform or constant, but depends on x. Nevertheless, we will assume for simplicity a constant loading condition. According to the electrostatic force theory, voltage causes a constant electrostatic force between the ground plate and the undeformed cantilever by the relation given by pðxÞ ¼

10 V 2 2d2

ð6:4Þ

as given in Equation 4.49. Substituting Equation 6.4 into Equation 6.3 and evaluating zðxÞ at the end of the cantilever, we have zðxÞlz¼l ¼ d <

1 0 ‘4 2 V 8d2

ð6:5Þ

It is found that this relation is valid for small d p d: This relates the displacement d to the applied voltage. If we neglect the changes in capacitance formed by the cantilever beam and ground plate, as the applied voltage increases, the displacement can be expressed by using the relation q < C0 V: Substituting this relation into Equation 6.5, we have the same expression as given in Equation 6.2, which relates the displacement at the cantilever tip to the stored charge q due to the applied voltage. This implies that the electrostatic ﬁeld provides the cantilevered optical actuator with a potential energy ﬁeld. When monochromatic light (photon) strikes the ground metal plate through the air gap, the photoelectrons are generated through the gap to the cantilever plate. These photo electrons reduce the charge on the capacitor from the q0 at the stressed condition by the following equation ðt ð6:6Þ qðtÞ ¼ q0 2 gAFa dt 0

where Fa is the optical power impinging on the area of the ground plate, A, and g is a constant related to photoelectron ﬂux generation. Examination of Equation 6.5 and Equation 6.6 indicates that generation of photoelectron current can control the cantilever beam position denoted by a dotted line shown in the ﬁgure. In more detail, this implies that the generated photoelectron, according to Equation 6.6, causes the stored charge to leak at a rate in proportion to its ﬂux. This results in conversion of the stored potential energy to kinetic energy. Therefore, the cantilever resumes its deformed state toward its unstressed position. To make this actuator continuously operate, it is necessary to charge it with a battery or other current source. Depending upon the magnitude of source current, the induced photocurrent will control the leakage of the charge through the gap, whose amount determines the direction

304

Optomechatronics

of the cantilever motion. The photoelectric actuation differs from an electrostatic actuation in that increase in optical ﬂux relaxes the cantilever to be open toward its undeformed state. Figure 6.1b illustrates how white light illumination on the ground plate creates the photo current with an increase of DC bias voltage, V. Total current includes the current induced due to optical power and that of the induced current due to bias voltage. The ﬁgure indicates a nonlinear relationship between the current and the voltage. This relationship differs from magnitude of light intensity, and shows that, as light intensity increases, total current increases accordingly. It is reported that for a bias voltage 6 V and optical power less than 0.1 mW/cm2 for 600 £ 50 £ 1 mm3 cantilever beam with a gap of 12 mm. Optical Piezoelectric Actuator When a piezoelectric material is irradiated by light (UV light) as shown in Figure 6.2, it exhibits two interesting phenomena: (1) photostrictive, (2) pyroelectric. Photostrictive effect is essentially photovoltaic effect caused by the electromotive force which comes from the electrons generated due to the irradiation by the light. Pyroelectric effect is due to a thermal electromotive force when the piezoelectric material such as lead zironate titanate (PLZT) experiences temperature change at the time of the light irradiation. The bimorph piezo element shown in the ﬁgure is a ceramic polycrystalline material made of a PZT material such as Pb (Zr, Ti)O3 and has a shape like a rectangular parallelogram [15 – 17]. As shown in the ﬁgure, two pieces of the PLZT element after polarization are glued so that each polarization bimorph PZLT light electrode light source

light

light

Bimorph PZLT (a) a bimorph PLZT beam element FIGURE 6.2 Optical piezoelectric actuator.

light source

bimorph PZLT

(b) deflection of PLZT when subjected to UV light

Basic Optomechatronic Functional Units

305

piezoelectric element photovoltaic effect

+ pyroelectric effect

+

mechanical deformation

inverse piezo electric effect

+ +

opto-thermic effect

thermal conduction

thermal deformation

FIGURE 6.3 The energy conversion processes to induce mechanical deformation.

direction is opposite to the others. Electrodes are installed at both ends of the PZT element. When this element is tested to investigate the time history of the displacement at the element tip, it is found that the major factor inﬂuencing the cause of displacement is the illumination intensity of UV light. The cause is mainly due to two combined effects, photostritive and thermal effects. Both of these effects convert electric energy into mechanical energy, which is in effect the inverse of piezoelectric effect, so that the actuator may deform from its original state. Therefore, the actuator deformation in total is composed of the contributions by those two effects and by pure thermal deformation. The energy conversion process is schematically illustrated in Figure 6.3. This optical actuator can be utilized for various applications such as optical switch, microvalve, and microrobot. Mathematical Model of the Photon-Induced Currents It is very interesting to see how these two photon-related effects are related to the PLZT deformation. In fact, the energy conversion mentioned in the section above is based upon the relationship. Therefore, it is necessary to obtain appropriate analytical models which can describe the two phenomena, rendering the relationship between induced current and optical energy. However, it is difﬁcult to identify the relationship analytically, since it is involved with physical phenomena occurring in various stages of energy conversion that are complex and nonlinear in nature. Nevertheless, we will model the conversion process based on Figure 6.3. For this modeling, we will assume that the conversion processes can be described as linear models, and that we can use some experimental results to obtain such mathematical models. Some experimental results [15 – 17] show that electromotive current ip ðtÞ due to photovoltaic effect may be expressed by the following ﬁrst order differential equation dip ðtÞ ¼ 2tp ip ðtÞ þ cp up dt

ð6:7Þ

306

Optomechatronics

where tp is the discharge time constant, cp is the conversion coefﬁcient involved with conversion from photonic energy to electric energy, and up is the irradiated light energy. The pyroelectric current it is governed by the temperature difference between both sides of the PLZT and can be expressed by it ðtÞ ¼ Ct

d {u ðtÞ 2 urm ðtÞ} dt ‘m

ð6:8Þ

where Ct is the coefﬁcient involved with conversion from thermal energy to electric energy, u‘m ðtÞ and urm ðtÞ are the average temperature at both sides of the PLZT due to irradiation, respectively. Induced Strains of the PLZT Once the two photon-induced currents are obtained, the induced strains of both sides of the PLZT can be explicitly expressed as a function of the electric charge by 1 a qðtÞ 2 q 1 1rp ðtÞ ¼ 2 aq qðtÞ 2

1‘p ðtÞ ¼

for the irradiated side

ð6:9Þ

for the back side

where aq is the coefﬁcient of strain conversion from the electric current to mechanical strain for the PLZT element, and qðtÞ is the electric charge produced by the light photonic energy. The electric charge is governed by dqðtÞ ¼ 2tq qðtÞ þ ½ip ðtÞ þ it ðtÞ dt

ð6:10Þ

where tq is the time constant of the electric charge process. On the other hand, the strain 1‘t and 1rt, due to thermal energy produced by the incident light can be expressed by 1‘t ðtÞ ¼ at u‘m ðtÞ

1rt ðtÞ ¼ at urm ðtÞ

ð6:11Þ

where at is the coefﬁcient of thermal expansion of the PLZT. Combining Equation 6.9 and Equation 6.11, we have total strains due to photovoltaic effect, pyroelectric effect, and thermal deformations 1‘ ðtÞ ¼

1 a qðtÞ þ at u‘m ðtÞ 2 q

1 1r ðtÞ ¼ 2 aq qðtÞ þ at urm ðtÞ 2

ð6:12Þ

From these strains, the displacement at the tip of a cantilever due to bending denoted by d‘ ðtÞ can be expressed by

d‘ ðtÞ ¼

3 L2 ½1 ðtÞ 2 1r ðtÞ 8 h ‘

ð6:13Þ

where L is the length of the optical piezoelectric element, h is the thickness of the same element, which can be easily obtained using the beam theory given in Appendix A2.

Basic Optomechatronic Functional Units

307

We are now able to consider control of the optical actuator, based upon the above derived model equations. Accurate control of the actuator is essential for microsystem positioning, because any error occurring from the control may be the order of positioning dimension. To determine the response of the system for a given controller, we need to determine all of the transfer functions. Some are neither simple nor linear. In fact, the relationship between the movement of the optical actuator and the intensity of the UV beam is nonlinear. The coefﬁcients related to photo pyroelectric and thermal phenomena, such as pizeoelectric constant, coefﬁcient of peizoelectric current, heat transfer coefﬁcient, heat capacity, and so on, exhibit highly nonlinear behavior. If the actuator system is operated at a certain nominal point, the relationship can be linearized near that point. The amount of UV light is controlled by a servomotor which can adjust the opening area of an optical ﬁber by rotating a light valve plate. The relationship between the DC motor and optical power of the UV beam is also highly nonlinear. For simplicity, we will assume all elements are linear. Figure 6.4a shows a servo-controlled optical actuator system in which an electric motor controls the position of the valve. The control system is composed of an optical actuator Gp ðsÞ consisting of two bimorph PLZT elements, a UV beam source Gl(s), a servomotor, Gm(s), a controller Gc ðsÞ, and a displacement sensor Gs ðsÞ: The control objective is to keep the position of the optical actuator xðtÞ as closely as possible to a desired location xd : In other words, the controller must be so designed that the error between xd and x(t), e(t), be driven to zero steady state with fast response. The open loop transfer function between the displacement at the tip dl ðsÞ and UV beam energy Up ðsÞ can be obtained by Laplace transforming both sides of Equation 6.7 through Equation 6.13. This can be written by

d ðsÞ 3 L2 ¼ Gp ðsÞ ¼ l 8 w Up ðsÞ

desired position

Xd

"

C p aq sCt aq þC ðs þ tp Þðs þ tq Þ ðs þ tq Þðs þ tt Þ

controller i Gc (s)

e + _

DC motor Gm (s)

d

# ð6:14Þ

UV radiater up optical actuator Gt (s) Gp (s)

X

sensor Gs (s)

(a) a closed loop optical control system Xd

e +_

controller Gc (s)

i

UV source/motor up Kl

optical actuator Gp (s)

(b) a simplified control system FIGURE 6.4 Block diagram of an optical actuator.

X

actual position

308

Optomechatronics

In deriving the above equation, the induced temperature difference given in Equation 6.8 is approximated by a ﬁrst-order lag-type transfer function, and C is a constant related to thermal phenomena. For simplicity of controller design, let us suppose that the dynamics of the servomotor-UV beam and sensor are fast enough to be neglected, as compared with that of the optical actuator itself. In this situation, the control system has two elements; plant to be controlled Gp ðsÞ and controller Gc ðsÞ: The control system block diagram shown in Figure 6.4a in this case can be reduced to Figure 6.4b to describe the simpliﬁed loop transfer function Go ðsÞ which is given by Go ðsÞ ¼

XðsÞ ¼ Kl Gp ðsÞ IðsÞ

ð6:15Þ

where dðsÞ is replaced by X(s), i is the current input to the UV radiator,Kl is the gain associated with the radiator and servomoter. Since Gp ðsÞ is given in Equation 6.14, we have Go ðsÞ ¼

3kl L2 aq ðb0 s2 þ b1 s þ b2 Þ 8w s3 þ a0 s2 þ a1 s þ a2

where the above coefﬁcients are b0 ¼ CCt aq , b1 ¼ Cp aq þ CCt aq tp , b2 ¼ Cp aq tt , a0 ¼ tp þ tq þ tt , a1 ¼ tp tq þ tp tt þ tq tt , a2 ¼ tp tq tt , and C ¼ 2:0 £ 1024 : Let the desired deformation at the tip of the PLZT be 20 mm, i.e., Xd ðsÞ ¼ 20 £ 1026 £ ð1=sÞ: With the open loop transfer function Go ðsÞ of the optical actuator system and a PID controller given in the form Gc ðsÞ ¼ kp þ

ki þ kd s s

ð6:16Þ

we can simulate the closed loop control system in order to see the control performance of the optical actuator, using the parameters listed in Table 6.1. All the transfer functions are put together and given to the Simulink model. Figure 6.5 shows the simulation results obtained for various controller gains. Figure 6.5a shows the effect of the proportional gain for a given set of the integral and derivative gains ki ¼ 50, kd ¼ 5. It shows that as the proportional gain decreases the response gets sluggish but exhibits higher overshoot, the trend of which appears unlike that of a standard second order system. The steady state is reached in approximately 5 sec. The effect of the integral gain can be seen in Figure 6.5b, from which we can see that as ki increases the response gets faster but has overshoot. The other two integral controller gains ki ¼ 100 and ki ¼ 150 show fast response, and, as expected, the responses exhibit larger overshoot as ki increases. Figure 6.6a exhibits the neutrally stable responses for a set of gain parameters, kp ¼ 0:7; ki ¼ 100; kd ¼ 5: However, when kp gain decreases slightly from the above value, the response gets unstable, as can be seen from Figure 6.6b.

Basic Optomechatronic Functional Units

309

TABLE 6.1 The Parameters Used for the PLZT Simulation Parameters

Symbol

Unit

tp tq tt L w Ct aq Cp kl

Natural discharge constants Natural discharge constants Natural discharge constants Length of the optical piezoelectric element Thickness of the same Pyroelectricity conversion constant Strain conversion coefﬁcient Conversion coefﬁcient Constant

Value

21

s s21 s21 m m CK21 C21 Cm2W21

1.5 0.5 0.075 2.5 £ 1022 1.8 £ 1022 7 £ 1029 1.5 £ 1024 4.2 £ 10212 650

Source: From Fukuda, T. et al. IEEE Transactions on Industrial Electronics, 42(5), 455–461, 1995. q 2005 IEEE.

25

displacement (y), μm

displacement (y), μm

This optical actuator has a variety of applications. Figure 6.7 illustrates an application to a gripper-like structure of a robotic manipulator [15]. The gripper has two bimorph actuators subject to incoming UV beam, while UV beam is transmitted through two optical ﬁbers. The beam transmitted irradiates the mirrors. In this system, a feedback controller can be employed in a similar manner to that discussed earlier to adjust the gripper position, and is schematically shown in Figure 6.7b. The UV beam is controlled by adjusting the iris of the beam radiators, whose movement is controlled by a DC servo motor. The displacement of the optical actuator is assumed to be measured by a non-contact displacement sensor. Referring to the ﬁgure, Gl ðsÞ is the transfer function between the controller input and the radiated input of the UV beam, Gop ðsÞ is the transfer function between the UV light and

20 kp = 50 kp = 100 kp = 150

15 10 5 0

0

1

2 3 time (t ), ms

4

5

(a) the effect of proportional controller gain; ki = 50, kd = 5

25 20 ki = 50 ki = 100 ki = 150

15 10 5 0

0

1

2 3 time (t ), ms

4

(b) the effect of integral controller gain; kp = 150, kd = 5

FIGURE 6.5 The controlled response with various controller gains.

5

Optomechatronics 40 35 30 25 20 15 10 5 0

displacement (y), mm

displacement (y), mm

310

0

2

4

6 8 10 time (t ), ms

12

100 80 60 40 20 0 –20 –40 –60 –80 0

14

(a) kp = 0.7, ki = 100, kd = 5

2

4 6 time (t ), ms

8

10

(b) kp = 0.01, ki = 100, kd = 5

FIGURE 6.6 Oscillatory and unstable responses of the optical actuator.

actuator movement, Gp ðsÞ is the gripper transfer function, Gc ðsÞ is the controller transfer function, and Gs ðsÞ is the sensor transfer function. The overall transfer function is obtained by Gc ðsÞGm ðsÞGl ðsÞGop ðsÞGp ðsÞ XðsÞ ¼ Xd ðsÞ 1 þ Gc ðsÞGm ðsÞGl ðsÞGop ðsÞGp ðsÞGs ðsÞ mirror

ð6:17Þ

UV light optical fiber

optical gripper

iris (valve)

mirror

mirror supporter

motor

(a) configuration of optical microgripper

Xd

controllerD/A converter e Gc (s) D/A + –

optical DC motor UV source actuator up f i Gm (s) Gop (s) Gl (s)

Gs (s) A/D converter

sensor

(b) control system block diagram FIGURE 6.7 Microprocessor-controlled optical gripper.

gripper Gp (s)

actual position X

Basic Optomechatronic Functional Units

311

If the same simpliﬁed system presented in Figure 6.4b is used and the gripper is approximated by a linear system Gp ðsÞ; we can carry out the same procedure as done previously. This gripper system can be controlled by a built-in microprocessor. In the digital control system as shown in Figure 6.7b, the measured displacement signal is taken via A/D to a microprocessor. Based on this system, the controller command signal is computed in the processor according to the control algorithm which, in this case, is a proportional control action. The computed values are sent to D/A, which converts the discrete values into the corresponding analogue. This analog signal is then used to drive a drive circuit which in turn actuates the DC motor that gives the motion of the DC motor. The displacement of the kind of gripper is found to achieve several hundred mm, although the magnitude is dependent upon the intensity of UV light, the geometry of the gripper and the PLZT material. Photo-Thermal Actuator This actuator employs an indirect means of transforming light energy into mechanical actuation. Typical arrangements of the photo-thermal actuator are illustrated in Figure 6.8. The actuator shown in Figure 6.8a is composed of a light power source, a closed miniature cell that contains air or gas, a light energy absorber, and a ﬂexible diaphragm that generates mechanical motion. The light coming out of an optical ﬁber is introduced through a trapped air

light shape memory alloy

optical fiber

diaphragm support

collimating lens

support spring

a light energy absorber glass cell back

(a) closed miniature cell

(b) shape memory alloy light

glass silicon diaphragm nozzle

(c) gas field cavity FIGURE 6.8 Photo-thermal actuators.

gas flow

electrostatic bonding spacer glass

312

Optomechatronics

hole in the left end of the cell. This light energy causes the absorber to heat, causing the air or gas trapped in the cell to expand, which in turn causes the membrane to move outward. Another optical manipulation shown in the right-hand side of Figure 6.8b utilizes the expansion of solids that can be generated due to phase-transformation, as can be found from shape memory alloys such as NiTi, Cu – Zn– Al and Cu – Al – Ni. As discussed in the section “Microactuators” in Chapter 4, the shape memory effect is a unique property of such alloys which have the ability to transform from their basic shape to a predetermined shape when heated above a critical “transformation temperature.” The material remembers its original shape when strained and deformed. The silicon diaphragm valve is a typical application of the indirect optical actuator. The valve is optically actuated and used to control the ﬂow of a gas as depicted in Figure 6.8c. When light heats up the gas ﬁlled in the cavity, it expands and actuates the diaphragm, thus adjusting the amount of gas or liquid ﬂowing through the nozzle. The diaphragm displacement can be modeled as a function of incident light intensity. Once this is modeled, a similar analysis can be performed to those given above. The conﬁguration of a micromachine employing the principle of Figure 6.8b is illustrated in Figure 6.9 [35,36]. The machine is composed of a body consisting of shape memory alloys and elastic spring and feet made of temperature sensitive ferrites, permanent magnets, and spacers which are assembled in a case separately. The body shown in Figure 6.9a has a capability to stretch when light is projected on to the surface of the shape memory alloy. On the other hand, when light is switched off, the body shrinks. In this way, the body stretches and shrinks by switching on and off a light beam. The foot jointed to the body has a capability of locking and moving freely. In interaction shape memory alloy ferrite

ferrite

magnet spacer

magnet spacer spring (a) system configuration temperature sensitive ferrite magnet spacer

light

A

B

gap floor

floor

(b) foot release-locking mechanism FIGURE 6.9 Optically actuated SMA walking machine. Source: Yoshizawa, T., et al., Proceedings of SPIE, 4190, 212–219, 2001.

Basic Optomechatronic Functional Units

313

with light, the foot is in the state of “free-to-move” or “locking,” as in Figure 6.9b. In a normal state without light (A), ferrite and a magnet are stuck together under a certain magnetic force. When light is projected on to the ferrite surface, a locking state is generated: The temperature of the surface rises and brings about a magnetization change at the Curie temperature. Under this condition, the attractive force between the ferrite and the permanent magnet becomes weakened, causing both the magnet and spacer to drop down on to the ﬂoor. This creates a locking motion state at the foot (B). When the light beam disappears, the state of the foot comes back to state (A). When the action of switching the projected light beam on and off is repeated, this kind of movement continues, moving the machine in the right direction, as shown in Figure 6.10. This light-activated machine can walk on a ﬂoor, and beneath a ceiling, climb a slope, and move on a curved surface. In any case, since the dynamic properties of the machine are important, it is necessary to minimize the size and weight of the machine. Size and weight are critical, especially when the machine climbs up a slope. In this situation, slipping may occur, causing a retarded motion which is certainly not desirable. As shown in Figure 6.11, when mass m1 moves while mass m2 sticks to the ﬂoor, the condition that eliminates slipping between the feet and the ﬂoor is obtained by considering the following two inequality conditions 2mF1 2 mðm1 þ mb Þg cos u þ F 2 ðm1 þ mb Þg sin u . 0

mF2 þ mm2 g cosu 2 F 2 m2 g sin u . 0

light on locking

1

B

A

2 (1) locking foot A

stretch locking

walking direction B

A

(2) stretching foot B B

A

locking

(3) locking foot B

shrink B

A

locking (4) shrinking foot A

d

displacement

FIGURE 6.10 Schematic of the walking process made within a cycle motion. Source: Yoshizawa, T., et al., Proceedings of SPIE, 4190, 212– 219, 2001.

314

Optomechatronics N2 light

mbg sinq

B A

m2g

q F1

m1g

displacement

mN2

m2g sinq

m1g sinq mN1

F

F

N1

F2 m2g cosq

mbg m g cosq b

m1g cosq

FIGURE 6.11 Force model of a micromachine climbing an indented slope. Source: Yoshizawa, T., et al., Proceedings of SPIE, 4564, 229–237, 2001.

from which, we have DF ¼ F1 2 F2 .

mt g sin u þ mb g cos u m

ð6:18Þ

where DF is the difference of maximum F1 and minimum sticking force F2, generated by the foot locking-release mechanism, mt is the total weight of the machine, mb is the weight of the body, m is the friction coefﬁcient, and u is the slope angle. This result indicates that the machine needs large DF, smaller mt, and large m to climb an inclined slope, the trend of which depends upon the angle of the inclination. Thus, the machine can move in various conditions (such as a horizontal plane, the ceiling, a vertical wall) and show little difference in speed. Typically, with a halogen lamp (100 V/50 W) the machines moves at an average speed of 4 mm/cycle horizontally, and climbs vertically at 2.9 mm per cycle. As observed from the above discussions, the amount of light beam determines the amount of shrink and stretch of the shape memory, which essentially controls the distance per one moving cycle. When this distance needs to change while during motion, a distance-control system can be built in such a way that the moving distance per movement cycle is kept as desired. Such a control system is illustrated in Figure 6.12. The system is composed of a light source whose intensity is controllable by a source,

desired foot step Sd

error + _

controller

u

light source

I

sensor FIGURE 6.12 A block diagram for the foot step control system.

SMA actuator

F

moving machine

actual foot step S

Basic Optomechatronic Functional Units

315

a shape memory actuator, and a moving machine. Suppose we set a desirable foot-moving distance according to a moving-path plan. When it starts to move, a sensor measuring its absolute position yields an output which is fed back to compare with the desired value. Based on the difference in value between the two, the controller generates a control command signal which activates a drive circuit of the light source. This control action adjusts the amount of light and its intensity necessary to actuate the shape memory alloy (SMA) part at a desired state. Accordingly, the phase transformed SMA yields a desired force, which then moves the machine by a desired distance for its next cycle. The third class of actuators driving optical or optomechatronic systems can be found from a variety of practical systems to be discussed in the later part of this chapter. Here, we introduce a deformable mirror concept for controlling the phase of light wave. A deformable mirror to correct the telescope image distortion caused by the air disturbance is schematically illustrated in Figure 6.13. The ﬁrst one shown in Figure 6.13a is a monolithic deformable mirror actuated by a PZT ceramic bulk with two-dimensionallyarrayed electrodes. Applying suitable electrical voltage on these electrodes forms a locally deformable mirror surface. A glass mirror actuated by needle shaped piezoelectric ceramic bodies arranged and embedded in a polymer matrix is another example of this class. In both cases, a desirable mirror surface contour is generated by applying suitable voltages on the electrodes, which can correct the wave front distortion. These adaptive characteristics of the mirror can control the phase of the light wave caused by the distortion. Deformable mirrors based on MEMS technology shown in Figure 6.13b are to correct the distorted wave front, which utilize a great member of a tiny

light deformable glass mirror

wave front

lenslet PZT actuator stack

electric leads original locations (a) deformable mirror

(b) lenslet array

FIGURE 6.13 Wave front correction by deformable mirror and lenslet array.

316

Optomechatronics

lenslet. This device provides smoother mirror surface as the number of lens in the array increases.

Optomechatronic Sensing Sensing is a basic functionality that optomechatronic systems possess. Depending upon whether optical or mechatronic elements participate in providing sensing function or not, optomechatronic sensors are classiﬁed into the following two groups. The ﬁrst group represents optical sensors whose sensing function is enabled by incorporating with mechatronic units. Most of the optical sensors belong to this class. The second group represents non-optical sensors integrated with optical units or elements belonging to the reverse case of the above example. In this case, sensors have optical units within their body to constitute a sensing element. Optical Sensor As an illustration of an optical sensor, consider an optical encoder which measures angular or linear motion of mechatronic systems in an optical

photo-electronic sensor encoder disc rotor of motor coil

bearing

photo-electronic sensor motor bearing

(a) DC-tacho / encoder assembly details LED mask wheel AB

phototransistors motor shaft

360˚

A B I 90˚

180˚

channel A

channel B channel I

(b) simplified schematic of an optical encoder (c) diagram of the output signals of a digital encoder

FIGURE 6.14 Optical encoder.

Basic Optomechatronic Functional Units

317

way by adopting the optical photoelectric principle. This sensor is based on the optomechatronic principle, which belongs to the ﬁrst class, because its measurement principle is based on the interaction of optical signal with mechanical signal (motion). As shown in Figure 6.14a, the rotating encoder is composed of a LED, a light source, a disk with mask, and a phototransistor (See Figure 6.14b). The LED emits light through the indexed code disk. Then, photo transistors react to the light input passed through the masks, yielding electrical outputs of a square wave or quasisine shaped signal, which is used for further processing, such as counting and positioning. The comparison between the phase of the outputs of channels A and B determines direction of rotation, as indicated in Figure 6.14c, which in this case shows counter-clockwise rotation of the shaft. The index channel I is used as reference point for determining the traveled angle of rotation. This is a typical conﬁguration of most electric servomotors with an embedded optical encoder that are being used currently. The principle of velocity feedback of a motor control using the optical encoder is as follows: When a microprocessor inputs a desired velocity command to the motor system, the motor starts to rotate with a certain angular speed. This motion is detected by an encoder whose signal, in turn, is utilized to calculate the speed by code converter. The computed speed is then fed back to a controller embedded within the microprocessor. The controller calculates the error between the desired and actual velocity, based on which controller command signal is given to the PWM circuit. The signal thus generated goes to the drive circuit, which ﬁnally alters the motor speed so as to eliminate the velocity error. The linear encoder utilizes the same principle as that of the optical encoder, based on translating grating patterns. As a moving member generates a shift of the moving grating pattern relative to a ﬁxed grating, a series of pulses is generated and counted to detect the position of the member. The interferometric dilatometer is a variation of Michelson type that is based upon opto-mechatronic interaction between the optical and mechatronic units. This type of measurement is necessary when an object to be measured is subject to temperature ﬂuctuations. As discussed in Chapter 2, in the section “Interference,” the Michelson interferometer utilizes the principle of the light intensity change with optical path length difference by every half wavelength, when two light beams with the same wavelength interfere. With this conventional interferometer, ﬂuctuations inevitably cause inaccuracy in measurement and undesirable decrease in sensitivity. The modiﬁed measurement system shown in Figure 6.15 employs a mechanical servo actuator made of PMN (lead magnesium niobate based ceramic) which has small hysteresis in the induced strain curve. This servo actuator generates appropriate motion so as to cancel the drift caused by the thermal ﬂuctuation or laser noise. The servo system is the key to improved stability and accuracy.

318

Optomechatronics object

mirror moving y'

C

laser source

mirror (s) actuator

O y

lens

d

half mirror

photo detector FIGURE 6.15 The interferometric dilatometer.

Fabry-Perot Etalon The Fabry-Parot etalon considered in Chapter 2 has a variety of different fringe characteristics controlled by varying one of the mirror separations, d, the index of refraction nt, the incident angle u: Due to this changeability of its

piezoelectric actuator

d mirror plate

medium

mirror plate

(a) configuration of Fabry-Perot Etalon desired x position d +_

controller

servo driver

piezoelectric actuator

Fabry Perot etalon

sensor (b) feedback control schematic for positioning of the Fabry-Perot etalon FIGURE 6.16 Precision positioning of mirrors with piezoelectric actuators.

x

actual position

Basic Optomechatronic Functional Units

319

parameters, and its special characteristics, the device has numerous applications in various ﬁelds. We will consider some of its characteristics under the situation that the mirror separation can be made changeable by piezoelectric actuator, as shown in Figure 6.16a. The etalon is composed of two piezoelectric actuators, two transparent mirror plates (or coated plates), and two connecting rods that link the plates. The actuators driven by the servo driver control the positions of two mirror plates according to a given desired mirror spacing. Figure 6.16b illustrates a control system that realizes this position control. The control system requires the precision positioning, and therefore, needs an accurate control. According to the control strategy, the mirror position error is fed back to the servo driver which drives the piezoelectric actuators. The actuators, in turn, actuate the mirror system composed of a lumped mass and a damping element that might exist in the actuators. This kind of control action continues until the mirror separation reaches a desired value. Based on the controllable mirror separation, we will discuss three characteristic features. First, it can enable us to calculate the unknown wavelength by moving the plates of the device and counting the number of interference fringes. As the plate moves to change the distance d, the fringe will move across the screen due to changes in constructive and destructive interference length. If the distance moved is accurately measured by precision sensors, and if the number can be counted by using a photodiode placed at the center of the fringes, we can calculate the wavelength of the incident light from Equation 2.53

l¼

2d m

for the incident angle normal to the plate surface. The second characteristic comes from the concept of resolvable power. So far, only a source with a single wavelength has been considered. When two wavelength components l0 and l1 ¼ l0 þ Dl; are contained in the source, superposition of two fringe systems would result. In this situation, if the wavelengths are very close, they may not be clearly distinguishable from each other. Therefore, it is necessary to establish the minimum resolvable difference in wavelength ðDlÞmin ; that is, the minimum discernable wavelength difference. In order to deﬁne this ﬁrst we normally deﬁne the closeness between the two peaks of the two circular fringes, as shown in Figure 6.17. Let P be the overlapping point. It is normally adopted to take this point which is not larger than the point yielding the full width at the half the maximum (FWHM) irradiance. This point can be found by considering the corresponding phase increment between two fringe maxima as shown in the ﬁgure. From the transmission equation considered in Equation 2.59, the FWHM DwFM satisﬁes DwFM 1 ¼ 2 sin21 pﬃﬃ 2 F

320

Optomechatronics T 1.0

T 1.0

FWHM

P ∆jFM 2

∆j ∆jmin

(a) resolving power

wavelength lo – Dl order (m +1)

lo m

lo + Dl (m –1)

wavelength

(b) frequency spectrum range

FIGURE 6.17 Characteristics of the Fabry-Parot etalon.

For large F, this may be rewritten by 4 Dwmin . pﬃﬃ F since Dwmin is nearly equal to DwFM. This value represents the smallest phase increment separating two resolvable fringes, and is related to equivalent minimum increments in wavelength ðDlÞmin : It can be shown that, if the resolving power is deﬁned by R¼

l ðDlÞmin

then it is given by p pﬃﬃ R¼m F 2 For small wavelength intervals, ðDlÞmin is given by: ðDlÞmin ¼

l2 pﬃﬃ p Fd cos u

ð6:19Þ

Therefore, to increase resolving power, it is necessary to increase the order of fringe as well as the coefﬁcient of ﬁnesse. The third characteristic of the Fabry-Perot etalon is free spectral range. This concept is important when the two wavelength components contained in the source become very different. Under this condition, the overlapped fringes will separate from each other. As the wave length difference increases, that is, when ^Dl gets large the (m)th order fringe for l0 will approach the (m þ 1)th and (m 2 1)th orders of fringe for the other neighboring wavelengths, l 2 Dl and l þ Dl, respectively. This separation range in wavelength is called the “free spectral range.” This can be easily

Basic Optomechatronic Functional Units

321

derived by ðDlÞfsr ¼

l m

and therefore ðDlÞfsr ¼

l2 2d

for normal incidence. We need to have this value as large as possible. One way to increase this value is to decrease the separation d, but this will decrease the resolving power, resulting in an increase in the ðDlÞmin as can be seen from Equation 6.19. Fiber Optic Sensors The characteristic of ﬁber optic sensors as distinct from other optical sensors is that they can easily interact with the mechanical elements in detecting motion. The interaction includes displacing one ﬁber relative to the other (shutter, grating), deforming a moving element with reﬂective surface placed in close proximity to ﬁbers (Y probe), and the small deformation of a ﬁber due to the small movement of an element in contact with the ﬁber (micro bending). As summarized in Table 6.2, there are a variety of optical sensors operated based on this optomechatronic interaction. Most of the sensors utilize the signal modulation discussed in Chapter 5, which includes intensity modulation, wavelength modulation, polarization modulation, and so on. The intensity modulation methods have a disadvantage in that they are susceptible to variations in source intensity or other noise. The wave length modulation method, however, is not dependent upon the undesirable variations.

TABLE 6.2 Fiber Optic Sensors Operated via Optomechatronic Principle Basic Measuring Principle and Methods Modulation Type Intensity

Wavelength Polarization Phase

Method

Motion Provider

Cutting, reﬂective, attenuation

Shutter grating, mechanical motion, mechanical motion Mechanical resonator Gratings Gratings Mechanical motion Mechanical motion Mechanical motion

Resonator Referencing Digital encoding Spectrum change Polarization change Phase delay

Physical Parameter to be Measured Temperature, pressure, displacement, force Force, pressure acceleration Pressure Displacement Displacement pressure Stress, pressure Displacement

322

Optomechatronics

The intensity modulation principles can be found from a number of optical sensors. The shutter modulation is the simplest conﬁguration of this kind – essentially an optical beam interruption device utilizing displacement of the shutter. For greater sensitivity, a pair of gratings can be situated between two ﬁbers, where one of the grating is held ﬁxed, while the other is moving due to an externally generated motion of mechanical elements. Deﬂection of a mechanical element (bimetallic strip) shown in Figure 6.18a utilizes the reﬂective property of strip. It consists of two optic ﬁbers, one as an input, the other as an output. In effect, it utilizes change in reﬂectance due to the deformation of a mirror-like surface. When any movement of the strip occurs, this causes change in light intensity at the sensor output. The concept of microbending utilizes bending-induced attenuation in ﬁber optic bundles, as can be seen from Figure 6.18b. Corrugations on the element when displaced due to externally provided load generate a periodic distortion on the ﬁber. When this occurs, the light beam within the ﬁber moves orthogonally instead of reﬂecting, and refracts the ﬁber wall. The amount of light received at the output is found to be dependent on the corrugation spacing, the core radius of the ﬁber, and the difference in dielectric index between the core and clad of the ﬁber deﬁned by ðnco 2 ncl Þ=ncl where nco and ncl are the refractive indices of the core and cladding, respectively. The ﬁber sensor employing the intensity modulation principle (cutting) shown in Figure 6.18c measures acoustic disturbance in the water. When a sound wave hits the diaphragm, the deformed diaphragm displaces the ﬁber on the left, thereby changing the amount of the light transmitted across the gap.

source mirror-like surface sensor

bimetallic strip

(a) reflection displacement

photodetector

light source gap

light optical fiber (b) microband force sensor

FIGURE 6.18 Intensity modulation optical sensors.

to photo detector

diaphragm sound wave

(c) optical fiber displacement

air water optic fiber

Basic Optomechatronic Functional Units

323 q

collimating lens input fibre

output fibre f

d

grating angular displacement sensor FIGURE 6.19 Wave length modulation-based optical sensor.

A typical sensor employing the wavelength modulation principle is shown in Figure 6.19. The ﬁgure shows a Littrow diffraction grating angular displacement sensor. It consists of a collimating lens and a diffraction grating composed of an array of grooves with spacing d when the grating is displaced by u: The sensor converts angular displacement, u into changes in wavelength of the input beam. Consider an input beam, with broad band radiation emerging from an input ﬁber located at the focal length f of the lens, is collimated into a parallel beam, and then incident on to the reﬂection grating at an angle u: As discussed in Chapter 2, the resulting diffraction pattern is that only light with wavelength l satisfying the following equation ml ¼ 2d sin u

m ¼ 1; 2…

ð6:20Þ

will be reﬂected back along its own path as indicated by an arrow. Since only the ﬁrst-order reﬂected beam is dominant, the above relationship can be rewritten for m ¼ 1 as

u ¼ sin21

l 2d

ð6:21Þ

This relates reﬂected wavelength, l; to grating angle, u: This sensor has a measurement range between 208 – 608; which depends upon the geometry of grating and optical arrangement. The principle of the Fabry-Perrot interferometer discussed in Chapter 2, in the section “Interference” can be applied to measure the pressure of liquid or gas mediums. This interferometric sensing concept utilizes the variation of the distance between the two parallel plates, that occurs due to the movement of one plate relative to the other in response to external pressure change [23, 24]. Figure 6.20a shows the conﬁguration of an interferometric Febry-Perrot pressure sensor. It consists of an F-P interferometer in which a cavity consists of an air gap between an optical ﬁber and a diaphragm, a multimode optical ﬁber carrying a single laser source light and a reﬂected light, and a thin silicon diaphragm capable of being responsive to the change

324

Optomechatronics glass tube

applied pressure

optical fiber

diaphragm (silicon)

internal mirror

phase change

(a) configuration of an optical pressure sensor

Pi Pr dielectric mirror

dielectric mirror

(b) Fabry-Perot interferometer

differential pressure (c) typical response of the pressure sensor

FIGURE 6.20 An optical ﬁber Fabry-Perot interferometer.

of external pressure. The interferometer has two mirrors coated with dielectric ﬁlm. The separation distance ‘ forms the cavity length of the interferometer. When pressure is applied to the diaphragm, the diaphragm is deﬂected, resulting in a change in the optical length of the interferometer shown in Figure 6.20b. This change causes light reﬂection off two surfaces to bounce back and forth between mirrors and thus interfere with each other. The interference can be either destructive or constructive, as discussed earlier. When two mirrors are assumed to be lossless and their reﬂectance R is much less than ‘, the reﬂected optical power Pr is given by Pr ¼ R{1 þ cos w} Pi

ð6:22Þ

where w represents the round-trip phase-change in the interferometer, which is given by w ¼ 4pn‘=l: In the above, Pi is the optical power of the incident beam, n is the reﬂective index of the material between the two mirrors, l is the optical wave length. The relation indicates that the ratio of the reﬂected to incident optical power is a function of the reﬂectance of two mirrors and the round-trip phase difference which depends on the optical length difference between the two mirrors. Now let us consider when an external pressure is applied to the sensor, which results in the deﬂection of the diaphragm. In this case, the optical ﬁber will experience longitudinal strain along the ﬁber axis, which in turn shifts

Basic Optomechatronic Functional Units

325

supporting base

light

bimorph cantilever emitting fiber h

receiving fiber light

d

FIGURE 6.21 Piezoelectric bimorph optical ﬁber sensor.

round-trip phase difference expressed by

dw ¼ 0:78

4pnðd ‘Þ l

ð6:23Þ

where d ‘ is the change of the cavity length of the interferometer sensor. The factor 0.78 is a correction value to account for the strain optic effect. This phase change is directly measured by the interferometer. Figure 6.20c shows a typical output of a sensor having the cavity length 1 mm, ﬁber diameter 1.25 mm and the reﬂectance of the mirror 3.5%. In response to an applied external pressure, we can see that the phase shift of the sensor output is proportional to the applied pressure. The relationship shows strong nonlinearity as applied pressure increases. Combining an optical ﬁber with a bimorph transducer provides a good means of measuring physical quantities [32]. Precision measurement of high voltage of an electric power system is an example, whose basic structure is shown in Figure 6.21. The operating principle is based on an intensity modulation. The sensor is composed of an emitting ﬁber, a receiving ﬁber, and a piezoelectric bimorph transducer. The input ﬁber is ﬁxed to a supporting plate, while the output (receiving) ﬁber is ﬁxed to one end of the bimorph transducer. The transducer is composed of piezoelectric plates metalized along their thickness. When a high voltage is applied to the transducer, its free end is deﬂected by d in the z direction, as discussed in Chapter 4, in the section “Actuators,” which is given by

d¼

3d31 ‘2 V 2 h

ð6:24Þ

where d31 is the piezoelectric coefﬁcient, V is the electric ﬁeld, and ‘ and h are the geometric dimension of the cantilever. As a result of the displacement, the amount of optical power transmitted from the emitting ﬁber to the receiving ﬁber is reduced. When the measured electric ﬁeld is AC, the bimorph beam will vibrate with the applied electric frequency. The natural frequency of the beam should be made much higher than that measured in order to avoid a resonance phenomenon. If the bimorph actuator of the size

326

Optomechatronics photodiode array

electronics

processor

light source

fibre-optic cable

scanner head

object (a) sensor configuration

(b) typical scanned images obtained by the optical scanner

FIGURE 6.22 Fibre-optic vision sensor.

3:5 mm £ 2:5 mm £ 0:6 mm having a free length of 28 mm, the resonance frequency is found to be 245 Hz. A visual sensor shown in Figure 6.22a whose motion is provided by a scanning unit, is a sensor embedded in a mechatronic unit, thus having a capability to measure an area range of interest [30]. It has two key components; coherent ﬁber-optic bundles for carrying light from an object to be imaged, and photodiodes to convert the light reﬂected from the measured object surface. The ﬁber optic cable can deliver light to illuminate the area of interest of the object. The scanner head has various ﬁber geometries for imaging, as well as illumination that can be used depending upon the geometry of object to be measured. A typical scanned image obtained for various objects under front lighting is shown in Figure 6.22b. The edges appear not to be sharp due to shadow effects and varying angles of reﬂectance at the edges.

Automatic Optical Focusing Autofocusing that automatically locates the focal position is a desirable attribute and an important concept in a number of optical systems. It produces clear, sharp images of objects by avoiding blurring of local imaging. For instance, in some types of microscopes, autofocusing makes possible the detection and identiﬁcation of atoms. For example, a confocal microscope employs the confocal principle in which a point of light source is always in focus with a point inside the sample, using a pinhole placed in front of the object. This confocal system can effectively see a clear image inside thick samples due to this arrangement in which its autofocusing adopts a single point illumination. In this way, it produces a clear image

Basic Optomechatronic Functional Units

327

of objects by locally imaging one point at a time, as we shall discuss in Chapter 7, in the section “Atomic Force Microscope.” In optical hard disks, it enables the optical head to accurately read and write the data by optical means. And in a laser printer, it also helps print better quality images. There are several factors that need to be taken into account to achieve auto focusing: The ﬁrst is to construct an appropriate optical system conﬁguration. The second is to compose a feedback system that controls the lens position or the object position. The third is to measure the image and ﬁnd a measure of defocus. All of these factors may differ from application to application. Therefore, an appropriate or optimal choice would be necessary when autofocusing is part of a designed system. In this section, we will deal with the optical system conﬁguration, the lens position control, and sensing problem. Optical System Conﬁguration There are two types of optical system arrangements for autofocusing, depending upon whether the optical system has pinholes or not. In confocal microscope systems, a typical arrangement is a pinhole in front of a detector. The role of the pinhole is that it blocks light from all parts of the object outside this focus. In the situation where there is a point source of light, a point focus

FIGURE 6.23 Basic conﬁguration of the autofocusing system.

328

Optomechatronics

of light inside or at the surface of the object, and a pinhole detector, all three are confocal with each other. Some autofocusing systems do not employ this pinhole system. Figure 6.23 illustrates the basic conﬁguration of the autofocusing system, and demonstrates its principle. When the object is located exactly in the focal plane of the objective, the focused light illuminates a single point on the object surface. The reﬂected beam spot is then exactly imaged on to a detector through a pinhole, which collects a large amount of the incident energy. A pinhole blocks light from all parts of the object outside this focus. In this situation, when the confocal system is located in the out-of-focus position as indicated in dotted lines, the image is focused at a position before or after the point detector. In these cases, the detector gets only a small amount of the incident energy — the larger the defocus distance, the weaker the signal strength. Therefore, the location of the objective lens system relative to an object under inspection is vital in determining the location of the imaged spot having high signal strength. This indicates that an accurate control of the system in the axial direction depending on the object shape is ultimately required, since a point image can be recorded only when the image is in the focal plane. In the case of a microscope, when this system is scanned over an interested region of the object, a map of 3D image information on the object can be accurately obtained. The brightness of the imaged spot partially depends on the amount of light gathered in by the objective lens of the autofocusing system. To describe this, the f-number is a useful parameter, when the object is located some distance away. However, when both the object and detector are located at a ﬁnite distance, as in the case of the confocal system, the numerical aperture denoted by NA is more useful. From the conﬁguration shown in Figure 6.23, NA is expressed by NA ¼ nt sin ua

ð6:25Þ

where nt is the refractive index of the immersing medium adjacent to the objective lens of the confocal system, and ua is the half-angle of the maximum light cone picked up by the lens, depending upon the aperture of the lens. A large value of NA implies a large lens aperture. This is true when the object lens pupil is uniformly ﬁlled with light. However, when a small beam enters through it, we do not get the beneﬁt. Therefore, in case of using a laser beam, we do not ﬁll the lens pupil with the light, since its proﬁle is generally Gaussian. Thus there is less light at the pupil edge. Optical Resolution Autofocusing directly concerns the axial resolution, but lateral resolution is also important as a performance factor. To this end we will brieﬂy review the lateral resolution, and then the resolution in axial direction. Lateral resolution is deﬁned by that in the plane of focus. As discussed in Chapter 2, in the section “Diffraction,” diffraction of light by an object and by

Basic Optomechatronic Functional Units

329

an objective lens determines image resolution. The lateral resolution is governed by diffraction, rather than limited by chromatic aberration, a spherical aberration. The image resolution for a given numerical aperture of the objective lens of an optical system is given by Dx ¼

0:61l NAobj

ð6:26Þ

where Dx is the minimum resolvable distance that the diffraction images of two adjacent points in the object can approach each other laterally before they merge. The above equation implies that the larger the numerical aperture, the higher the resolution. Axial Resolution Axial resolution is deﬁned by the minimum resolvable distance that the diffraction image of two points in the object can approach each other along the optical axis before they merge. Another way of deﬁning it is the radius of the ﬁrst minimum of the diffraction image produced by a light source point of the object. As derived for the lateral resolution, the wave optics can be used to obtain the minimum resolvable length Dz in the optical axis (objective lens) direction. This shows that the image of a point source produced by a diffraction-limited optical system is also periodic above and below the focal plane along the axial direction, as in the case of the in-focus plane. The difference between the lateral and axial resolutions is that the diffraction in the case of axial resolution occurs in 3D space. The minimum resolvable length along the axis of the objective lens is approximately twice as large as that in the plane of focus. In more detail, the distance from the center of the diffraction image to the ﬁrst minimum along the optical axis is approximately twice as far as it is to the ﬁrst minimum in the focus plane. This implies that the axial resolving power is approximately one half of the lateral one. The axial resolution is given by Dz ¼

2ln ðNAobj Þ2

ð6:27Þ

which shows that the resolution is inversely proportional to the square of the numerical aperture (NAobj). It is noted that the depth of ﬁeld of the autofocusing system is the depth of image measured along the optical axis, which appears to be in sharp focus by ﬁne focus adjustment. It is approximately equal to the axial resolution, but is found to be affected by several factors, such as the geometric spreading of the light beam near the focal plane that arises from a single point in the object and the magniﬁcation of the objective lens. Due to the use of different criterion for “in focus,” a number of expressions for the depth of ﬁeld are given in literature. One heuristic measure is to use a rule of thumb that the beam spot or image is regarded to be “in focus” if it is located in a region on either side of the focal (image) plane bounded by the depth of focus. The bound is given by ^ 10%

330

Optomechatronics

of the beam spot diameter, Dw at beam waist. This concept will be discussed in more detail later. Feedback Control of the Objective Lens In the previous section, we discussed brieﬂy the resolution in both lateral and axial directions. Now, with such resolution concepts established, we return to the control of an objective lens in the axial direction, so that the maximum intensity of light can be achieved. The performance of this control is critical to the precision of most of the opto-mechatronic systems such as an autofocusing system, a confocal microscope, an optical pickup device, AFM, inspection and measurement devices, and so on. Referring to Figure 6.24, image detection usually requires either high precision motion control of the objective lens or an object, and thus needs a high precision actuator for such control action. In the open-loop conﬁguration without feedback, the resolution can reach the order of nanometers, but the measurement range is limited by the linear range of the error signal. In addition, this conﬁguration is susceptible and sensitive to external disturbances, such as small vibrations and the reﬂective light from the object surface. Therefore, an autofocusing system with feedback is ultimately necessary for accurate measurements. A typical autofocusing system, as illustrated in the ﬁgure, controls the motion of the objective lens in the direction of optical axis so that it always places a focal point at the image plane. It consists of an actuator, a half mirror plate, a laser diode, and a quadrant photo sensor. A laser beam emanating photo sensor

mirror plate

laser source

objective lens actuator

object surface FIGURE 6.24 Schematic view of the optical autofocusing system.

Basic Optomechatronic Functional Units

zf

controller

actuator driver

Gc (s)

Gu (s)

error +_

331 disturbance D(s) focusing system actuator + + Ga (s) Gp (s)

z

Gs (s) sensor FIGURE 6.25 A schematic of the autofocusing system.

the laser source passes through a polarized beam splitter, a 14 l wave plate, and an objective lens, and ﬁnally focuses on the object surface. The beam reﬂected from the surface travels along the original path, passing through the same splitter, and then, ﬁnally, projects on the photodiode. The output of the diode is fed back to the controller to compute the error signal, from which a control command signal is produced. This command signal drives the actuator so that the error may be eliminated as fast as possible. The mirror plate plays both the roles of beam splitter and astigmatic lens. The mirror plate as a beam splitter transmits the beam converging from the objective lens to the four quadrants photo detector. In addition, the tilted plate can generate astigmatism along its principle axis. If this feedback control system is constructed, then it can be drawn as shown in Figure 6.25. We will discuss these control elements one by one. There are many actuators to use, but consideration must be given to several factors, such as accuracy, size, and weight. Table 6.3 compares several characteristics of various linear motion actuators which are frequently used to control high precision motion for various applications. The table shows that PZT is desirable in terms of accuracy and response time, but it may not be desirable from the point of view of moving range. This indicates that the actuator needs to be chosen depending on speciﬁcations imposed by the application. For autofocusing applications, voice coil motors and piezo-electric transducers are popularly

TABLE 6.3 Characteristics of Linear Motion Actuators Linear Motion Actuator Servo motor þ ball screw VCM PZT Friction drive Electrostatic actuator

Displacement Range Mm–m Mm–cm #100 mm mm–m #500 mm

Accuracy 0.1 mm– 10 nm ,50 nm ˚ ,A ,10 nm ,10 nm

Response Time .100 ms ,1 ms up to , ms .100 ms ,0.1 ms

332

Optomechatronics

used due to relatively simple structure, fast response, light weight and high precision in comparison with other actuators. Here, we will consider a voice coil motor (VCM) for focusing purposes. Figure 6.26a illustrates a typical autofocusing system driven by a VCM; an objective lens is installed on the VCM and can freely move in a direction perpendicular to the focal plane. The motor is composed of an objective lens on a moving coil (bobbin), two magnets, and a spring-damper system that nulliﬁes the effect of any external disturbance and also stabilizes the bobbin motion. In general, the actuator structure should satisfy the following mechanical requirements: (1) should be as small and light as possible. (2) should maintain the direction of laser beam during actuator motion. (3) should have high natural frequency of the moving bobbin. The magnets M1 and M2 generate magnetic ﬁeld in the direction of the arrow, as indicated in the ﬁgure. When the bobbin is placed within the magnetic ﬁeld with magnetic ﬁeld density B, force F in upward vertical direction is generated according to Fleming’s left hand rule. The force is given by F ¼ nBi ‘c

ð6:28Þ

coil for focusing

laser

FL

z

B SN

FR

coil

lens

m k

BS

PD

yoke

B NS

magnet

b

LD M1

(a)

M2

an objective lens installed on a voice coil motor

+ V

_ Rc

Lc

+

_

VbL

–

(b)

electrical circuit of the VCM

FIGURE 6.26 A conﬁguration of the confocal system.

+

VbR

i

Basic Optomechatronic Functional Units

333

where n is the number of coil turns, B is the magnetic ﬂux density in tesla, i is the current in amperes, ﬂowing in the direction perpendicular to the paper and ‘c is the coil effective length in meter. Figure 6.26b shows the electrical circuit of the VCM, which is written by L

di þ Ri ¼ V þ VbL þ VbR dt

ð6:29Þ

where L and R are the inductance and resistance of the coil, respectively, V is the electrical voltage applied to the coil and VbL and VbR are the back emf of the left and right hand side of the bobbin, respectively. The inductance L can be obtained by Faraday’s and Ampere’s laws, and is given by L¼

mn2 Ac ‘c

where Ac is the cross-sectional area of the coil, and m is the permeability. The resistance R is deﬁned by R¼

r ‘c Ac

where r is the resistivity of the conductor. The back emf is given by VbL ¼ VbR ¼ 2nB‘c

dz dz ¼ 2kb dt dt

ð6:30Þ

where dz=dt is the velocity of the bobbin in an upward direction. Rewriting Equation 6.29 by use of Equation 6.30, we have L

di dz þ Ri þ 2kb ¼V dt dt

ð6:31Þ

The dynamic equation governing the vertical motion of the bobbin can be written by m

d2 z dz þb þ kz ¼ F dt dt2

ð6:32Þ

where m is the total mass of the bobbin and lens unit, and b and k are the damping coefﬁcient and the stiffness of the bobbin, respectively. Laplace transforming of Equation 6.31 and Equation 6.32 and combining both, we can obtain the transfer function between vertical motion of the bobbin and electric input voltage to the coil. The transfer function is given by Gp ðsÞ ¼

ZðsÞ nB‘c ¼ VðsÞ Lms3 þ ðLb þ RmÞs2 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s þ Rk

ð6:33Þ

where Gp(s) here includes Ga(s). Let us consider when the above system is controlled by a feedback controller, which is shown in Figure 6.25. The objective of this control is to maintain a vertical displacement of the objective lens at the focal length. If the focal length is denoted by zf, then the controller should bring the system response always within some required

334

Optomechatronics

speciﬁcations. For simplicity, in Figure 6.25 we will assume D(s) ¼ 0, and neglect the dynamics of the sensor and driver, i.e., Gu ðsÞ ¼ 1 and Gs ðsÞ ¼ 1 or constant. If we use a PI controller, the controller transfer function is given by Gc ðsÞ ¼ kp 1 þ

ki s

From the block diagram, the overall closed loop transfer function is determined by G¼ ¼

Gc Gp ZðsÞ ¼ Zf ðsÞ 1 þ Gc Gp nB‘c ðkp s þ kp ki Þ Lms4 þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s2 þ ðRk þ nB‘c kp Þs þ nB‘c kp ki ð6:34Þ

The parameter used for the simulation is listed in Table 6.4. In order to investigate the effect of the controller parameter on the response, the system is simulated for various controller gain parameters, using the Simulink model. Figure 6.27 shows the system responses to a desired focal length of the objective lens, zf ¼ 1 mm. For a ﬁxed integral gain, ki ¼ 55, we can see from Figure 6.27a that, as kp increases, the speed of the response increases, but the responses have overshoot. From Figure 6.27b, similar trends can be found with the increase of integral gain for a ﬁxed proportional gain. When the two gains are chosen as, kp ¼ 500, ki ¼ 700, we see that the response amplitude increases with time, exhibiting instability of the lens motion. This leads us to analyze the stability of the control system. Let us consider the stability of the system, and determine the range of stability for various gain parameters. The characteristic equation of the

TABLE 6.4 The Parameters of the Confocal System Used for Simulation Parameters

Symbol

Unit

Value

Resistance Inductance Total mass bobbin and lens Damping coefﬁcient Stiffness Magnetic ﬂux density Number of coil turns Coil effective

R L m b k B n

V H kg N sec/m N/m wb/m2 Turn m

7.5 0.144 £ 1023 0.49 £ 1023 0.175 13.2264 0.454 123 0.01

‘c

Basic Optomechatronic Functional Units 1.2 lens displacement (z), mm

lens displacement (z), mm

1.2 1.0 kp=250 kp=500 kp=750

0.8 0.6 0.4 0.2 0.0

335

0

10

20 30 time (t ), ms

40

50

(a) the effect of proportional gain, kp ; ki = 55

lens displacement (z), mm

8

(c)

1.0 ki=35 ki=55 ki=75

0.8 0.6 0.4 0.2 0.0

0

10

20 30 time (t ), ms

40

50

(b) the effect of integral gain, ki ; kp = 500

kp=500, ki=750

6 4 2 0 −2 −4 −6 −8

0

100

200 300 time (t ), ms

400

500

the unstable response; kp = 500, ki = 750

FIGURE 6.27 The controlled responses of the autofocusing system.

transfer function is given by Lms4 þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s2 þ ðRk þ nB‘c kp Þs þ nB‘c kp ki ¼ 0 When we use the parameters in Table 6.4, the above equation can be written as 7:056 £ 1028 s4 þ 0:0037s3 þ 1:9381s2 þ ð99:198 þ 0:5584kp Þs þ 0:5584kp ki ¼ 0 ð6:35Þ According to the Routh stability criterion discussed in Appendix A3, we can determine the stability condition for the gains kp and ki from Equation 6.35 in the following manner: (1) Since all the coefﬁcients of the characteristic equation should be greater than zero, this yields 99:198 þ 0:5584 kp . 0; kp ki . 0: The above leads to 2177:6468 , kp , 0; ki , 0 or kp . 0; ki . 0 From these conditions, we have kp . 0; ki . 0:

336

Optomechatronics

(2) Arranging the coefﬁcients of the polynomial in rows and columns according to the following pattern yields s4

7:056 £ 1028

1:9381

0:5584kp ki

s3

0:0037

99:198 þ 0:5584kp

0

s2

A1

0:5584kp ki

s1

A2

0

s0

A3

0

We have the following parameter condition for stability 7:056 £ 1028

1:9381

0:0037

99:198 þ 0:5584kp

A1 ¼ 2

0:0037

¼ 1:9381 2 1:907 £ 1025 ð99:198 þ 0:5584kp Þ . 0 0:0037 99:198 þ 0:5584kp A2 ¼ 2

A1

A1 A1

A3 ¼ 2

0:5584kp ki

A2

¼ 0:0011kp ki þ 99:198 þ 0:5584kp . 0

0:5584kp ki 0 A2

¼ 0:5584kp ki . 0

To satisfy all of the above conditions, kp and ki should meet the following condition for stability 0 , kp , 1:82 £ 105 ; 0 , ki ,

9:018 £ 104 þ 507:6364 kp

Using this condition, the range of the stability limit (shaded area) is drawn in Figure 6.28. We see that kp is bounded for all values of ki, while ki is not for small kp value. Effect of External Disturbance Thus far, we did not consider the effect of external disturbance caused by thermal dilation or mechanical vibration that may cause unwanted position deviation. When disturbance, D(s), comes into the VCR, the overall output displacement of the VCR becomes (from Figure 6.25) ZðsÞ ¼

Gc ðsÞGp ðsÞ 1 Zf ðsÞ þ DðsÞ 1 þ Gp ðsÞGc ðsÞ 1 þ Gp ðsÞGc ðsÞ

ð6:36Þ

Basic Optomechatronic Functional Units

337

ki

750 507.64

55.0 0 500

1.82×105

kp

FIGURE 6.28 The area formed by kp and ki for system stability.

The displacement deviation DZðsÞ due to the external disturbance is given as DZðsÞ ¼

DðsÞ 1 þ Gp ðsÞGc ðsÞ

ð6:37Þ

It can be seen that the deviation can be remarkably diminished by a factor of 1=1 þ Gp ðsÞGc ðsÞ compared to the case without feedback. This observation implies that the open loop transfer function should be sufﬁciently larger than unity in order to minimize the displacement deviation. To investigate the effect of the feedback, let us assume that the disturbance DðsÞ is a step input type, i.e., DðsÞ ¼ 0:1=s Plugging this into Equation 6.37, we have 0:1 s DZðsÞ ¼ 1 þ Gp ðsÞGc ðsÞ For proportional controller, Gc ðsÞ ¼ kp ; we have 0:1 {Lms3 þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnBlc Þ2 }s þ Rk} s DZðsÞ ¼ Lms3 þ ðLb þ RmÞs2 þ {Lk þ Rb þ 2ðnBlc Þ2 }s þ ðRk þ kp nBlc Þ From the ﬁnal value theorem, the steady state error becomes lim sDZðsÞ ¼

Ds!0

0:1Rk Rk þ nB‘c kp

The results show that in order to reduce the steady state error, kp should be increased as large as possible, but the overshoot and stability must also

338

Optomechatronics

be taken into account. The case of using PI control yields DZðsÞ ¼

0:1{Lms3 þ ðLb þ RmÞs2 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s þ Rk} Lms þ ðLb þ RmÞs3 þ {Lk þ Rb þ 2ðnB‘c Þ2 }s2 þ ðRk þ nB‘c kp Þs þ nB‘c kp ki 4

Appling the ﬁnal value theorem again to the above equation, we have lim sDZðsÞ ¼ 0 s!0

which indicates that the integral control action eliminates the steady state error due to the step disturbance. Focus Measurement Accurate measurement of an optical head position (optical information detection lens system) is critical to its precision control. In the previous control system, it is assumed that the instantaneous location of the lens can be fed back to the controller for a feedback signal. Since, in most cases, focal position is not exactly known a priori, ﬁnding the error signal may often cause some difﬁculty. An appropriate criterion that can be used is the degree of defocus (or degree of focus) based on visual image measurement. This method, however, may not be fast enough to be used for precise control of the lens position, because most autofocusing applications, including optical disks, require very fast response for real-time control. This means that the calculation of the focus measure or error signal should be made well within the allowable control time step. A variety of sensors and the corresponding methods have been developed to detect focusing error. Here, we will introduce the astigmatism method, which employs a quadrant photo diode sensor as illustrated in Figure 6.29. The principle of a primary aberration known as astigmatism has been already discussed in the section “Aberration“, in Chapter 2. When at the instant an object plane is moved inside or outside the focal plane of the objective lens, light projected on the photo detector will produce an elliptical pattern shown in plane 1 or plane 3. When the object is exactly at the focal plane, it will have a circular shape shown in plane 2. In more detail, when an object is placed at the focal plane of the objective lens, the beam will scatter, symmetrically yielding a dark circle with a radius R as an output of the photo sensor, shown in the upper ﬁgure of the sensor output. As opposed to this, when the objective lens is out of focus, this causes the intensity pattern on the detector to become elliptical, as indicated in the ﬁgure. When the lens approaches toward the surface, the output will become elliptic with the major axis in the vertical line, as shown in the middle ﬁgure. On the other hand, when the objective lens moves farther away from the object surface, the output will become elliptic again, with the major axis in the horizontal line, bottom ﬁgure. Referring to Figure 3.29b, the four quadrant sensor shown in the ﬁgure exactly captures this astigmatism phenomenon. When ia, ib, ic, and id represent the output of the four quadrants, respectively, as shown in the

Basic Optomechatronic Functional Units

339

FIGURE 6.29 A schematic view of images formed by an astigmatic lens.

ﬁgure, the out-of-focus displacement of the lens Dz can be computed by the following equation Dz ¼ ðia þ ic Þ 2 ðib þ id Þ

ð6:38Þ

This Dz represents the error between the desired z(t), and actual values of the focal length, zf, and is measured at every instant of time. The computed error is used to calculate the controller signal, as indicated in the Figure 6.25.

Acoustic-Opto Modulator When an acoustic wave propagates in a transparent solid block of a medium such as fused quartz, it produces a periodic modulation of the index of refraction via the elasto-optical effect. All acousto-optic (AO) devices utilize the interaction of acoustic and optical waves within the medium. The devices are connected to various laser systems for information handing, optical signal processing, display and spatial and temporal modulation of coherent light. The AO interaction provides four basic properties that are frequently used in various devices. They are classiﬁed as follows: (1) Deﬂection: angular deviation of the light beam which is proportional to the acoustic frequency. (2) Frequency shifting: frequency shift of the deﬂected beam which is equal to plus or minus the acoustic frequency. (3) Tunable wave length ﬁltering: wave length selection from a large spectral

340

Optomechatronics

FIGURE 6.30 A schematic of acousto-optical modulator.

band source of the beam by changing the acoustic wave frequency. (4) Amplitude modulation: modulation of deﬂected beam intensity by varying acoustic power. In the AO cell, a piezoelectric transducer is vibrated when driven by a radio high-frequency RF signal. The vibration of the transducer generates an acoustic wave which passes acoustic waves through the quartz, as shown in Figure 6.30. The transducer is placed between two electrodes; the top electrode determines the active limits of the transducer, whereas the ground electrode is bonded to the crystal. The transducer thickness is usually selected to match the acoustic frequency to be generated. The shape of the electrode can be varied for impedance matching or to shape the acoustic wave, while the electrode length is chosen to yield the desired bandwidth and efﬁciency. An impedance matching circuit is necessary to adapt the AO cell to the impedance of the RF source (in general 50 V) in order to avoid power-returned loss. The RF frequency range covers 1 MHz to 1 GHz, and is band pass in nature. Actual products of AOD or AO modulator are shown in Figure 6.31. A variety of different AO materials are used depending on the wavelength, intensity and polarization of the beam, which in turn vary with the types of applications. For instance, tellurium oxide (TeO2) fused quartz or lithium niobate (Li Nb O3) is used for the visible and near-infra regions, while for the infrared region, germanium is frequently used. The material is antireﬂection-coated to reduce reﬂection from the optical surfaces. Table 6.5

Basic Optomechatronic Functional Units

341

FIGURE 6.31 Acousto-optical modulator products. Source: Brimrose Corporation of America, (www.brimrose.com).

show various properties of acousto-optic materials used for AOD. They include optimum optical range for AO applications, incident optical polarization refractive index at various light wavelengths, acoustic velocity, and so on. It is noted from the table that most of the materials are crystal or glass. Deﬂector Acousto-optical deﬂection utilizes an acousto-optic effect in which acoustooptic interaction provides a means of modifying the spatial position of an optical beam. This effect occurs when a variation in refractive index arises due to the propagation of an acoustic wave in a modulator medium, as shown in Figure 6.32. Here, as mentioned before, an acoustic wave is generated by electrically actuating mediums such as quartz. When a collimated beam with wavelength l is incident to a collective lens whose

TABLE 6.5 Various Properties of Acousto-Optic Materials Used for AOD

Material

Type

Ge Ge33Ass12Se55 TeO2 SiO2 (fused silica)

Crystal Glass Crystal Glass

Optimum Optical Range for AO Applications (mm)

Incident Optical Polarization

Acoustic Velocity (m/sec)

2.5 –11.0 1.1 –1.7 0.35–0.45 0.2 –2.2

Linear Unpolarized Linear– circular Linear

5500 2520 620 5960

342

Optomechatronics fc modulator medium

0-order undiffracted

Dw collimated beam

acoustic wave

λ

D

lens

Λ

piezoelectric transducer

1st order diffracted lens

FIGURE 6.32 Acousto-optical interaction.

focal length is fc, the collected beam encounters an acoustic wave having a wavelength, L. This is called an acousto-optic interaction. In order to make an effective interaction occur, it is necessary to position the modulator medium at the beam waist, Dw as shown in the ﬁgure. As a result of the interaction, a harmonic acoustic wave accompanies variation in the refractive index, which causes the input beam to deﬂect through a medium of alternating refractive index by the Bragg effect. Passing through this alternating medium has the same effect as passing through diffractive grating. Figure 6.33 illustrates in more detail how an optical beam can be deﬂected by this phenomenon. The ﬁgure shows rarefaction and compression of the medium, as a result of the interaction. By altering the amplitude and frequency of the acoustic wave, the spatial position, intensity, or frequency of an optical beam propagating through the acousto-optic medium can be varied. The acousto-optical modulator or deﬂection directly utilizes this principle. To derive the deﬂection angle, let us consider that a light beam of a single frequency is incident at an angle u to the plane of grating, which is the acoustic wave front. For acoustic waves of frequency fa, the wavelength L; and traveling at the speed of sound in a medium, va, the spacing between the planes of index of refraction variation is given by the relation va ¼ Lfa

ð6:39Þ

When the light beam interacts with this acoustically driven medium, it will be deﬂected to angle given by sin u ¼

ml 2L

ð6:40Þ

where m is the diffraction order, and l is the wave length of the light in air. As can be seen from the ﬁgure, use of the difference in beam path between the undiffracted and diffracted is made in the derivation of the above relation.

Basic Optomechatronic Functional Units

343 S m=1 first order

incident light beam θ

a = 2θ

Λ

m=0 zero order traveling acoustic waves

piezoelectric transducer FIGURE 6.33 Diffraction of optical beam in acoustic medium.

According to Heuggen’s principle, the ﬁrst-order diffracted beam ðm ¼ 1Þ shows the strongest among the diffracted beams, having the largest efﬁciency. In this case, the angle between a ﬁrst-order diffracted beam and the undiffracted beam is given by sin

a l lf ¼ a ¼ 2va 2 2L

ð6:41Þ

If a is assumed to be very small, the value is expressed by

a¼

lfa va

This indicates that a light beam can be diffracted by simply varying the frequency of acoustic wave. Acoustic-optical scanners use this relationship, in that the angle of deﬂection is in proportion to the sound frequency fa. For example, we consider the use of lithium niobate (LiNbO3) material as an AO medium. Suppose that we generate 250 MHz acoustic waves on the substrate whose acoustic velocity in the medium is 6.57 km/sec and refractive index is about 2.2. As a modulating light beam, a red-laser beam from He– Ne laser of wave length l ¼ 632:8 nm is used. Calculate the Bragg deﬂection angle, u, with respect to the plane of grating. The relation

344

Optomechatronics

in Equation 6.39 gives

L¼

va 6:57 £ 103 m=sec ¼ ¼ 3:285 £ 1025 m fa 2:00 £ 106 Hz

The Bragg deﬂection angle then becomes " # 29 m /2:2 21 ðl=nt Þ 21 632:8 £ 10 ¼ 0:58 a ¼ sin ¼ sin L 3:285 £ 1025 m This gives u ¼ a=2 ¼ 0:258: We can also compute the range of scan, if the maximum and minimum frequencies are known for scanning, respectively, max( fa) and min( fa). The swept angle range is obtained by the relation

a¼

l lDfa {maxð fa Þ 2 minð fa Þ} ¼ va va

ð6:42Þ

where Dfa denotes the acoustic frequency band width. Equation 6.42 provides the range of sound frequencies needed to produce a given scan angle. In acousto-optical deﬂectors, resolution and speed of the deﬂection are the most important parameters. The resolution is dependent on the beam divergence, and again, the given diffraction limited optics. It can be expressed by the number of resolvable spots, which is the scan angle range divided by the angular diffraction spread N¼

a D Dfa ¼ da va a g

where D is the beam diameter, and ag , is the factor related to the aperture geometry. As we shall see later, the term D=va is equal to the acoustic transient time across the optical aperture, tr : This indicates that the resolvable spot N is increased by decreasing the transient time tr for a given acoustic bandwidth Dfa : The increasing tr means that the speed of deﬂection decreases. Therefore, there is a tradeoff between resolution and speed of AO deﬂectors. When multiples of AO deﬂectors are combined, several applications can be considered for scanning. Figure 6.34 illustrates a typical application of multiple AO deﬂectors in a scanner. As mentioned before, the scanning area is limited by the scan angle, which is usually very small. When a helium – neon laser passes through a glass (tellurium oxide) interacting with acoustic wave having its velocity 0.62 km/sec and frequency 70 MHz, the scan range is roughly within 1.8 mm. This is a rather large scan angle that can be achieved by AOD, if Table 6.5 is examined. The AOD has several advantages in that (1) the scanning speed is much faster than that of a mechanical scanner, (2) it has no moving parts to wear out (3) random access is possible and (4) the accuracy and linearity is very high. On the other hand, it also has some disadvantages, such as small

Basic Optomechatronic Functional Units

345

mirror

laser source lens

X-Y deflector

lens

obeject

FIGURE 6.34 An AOD X-Y scanner.

deﬂection angle, low diffraction efﬁciency (about 50 , 70%), use of a single wavelength only due to dispersion of the refracting material, and required chromatic correction. The properties of AO, mechanical scanners and electro-optical scanners are compared in Table 6.6.

Frequency Shifter The acousto-optical device can also be used as a frequency shifter by modifying the frequency of the deﬂected light. The frequency of the diffracted beam is shifted as a result of the Doppler effect. When an observer and a light source have a relative velocity v, according to relativity theory, the

TABLE 6.6 Properties of Three Modulators Characteristics Transmittance Maximum frequency response (MHz) Rise time (nsec) Single wavelength Aperture Power (W)

Mechanical

Acousto-Optical

Electro-Optical

100 0.005 10000 No Large .10

80 50 25 Yes Small 5

70 100 10 Yes Small 20

346

Optomechatronics

observed frequency is given by 1 þ ðv=cÞ f ¼ f0 qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ 1 2 ðv2 =c2 Þ

ð6:43Þ

where f0 is the frequency if observer and source are at rest relative to each other. When v p c, the above equation may be simpliﬁed as Df v ¼ f0 c

ð6:44Þ

In this equation, the relative velocity is positive for relative motion in the same direction as each other, and negative for motion away from each other. In the acousto-optical modulator, we can easily show that the above equation can be rewritten as Df 2va sin u ¼ f0 c

ð6:45Þ

where f0 in this case is the frequency of the incident beam. In a simpliﬁed form, this can be written by Df0 ¼ ^fa

ð6:46Þ

As a consequence, the frequency of the diffracted light can be obtained by fd ¼ f0 ^ fa

ð6:47Þ

where fd is the frequency of the deﬂected beam. Figure 6.35 illustrates the above relations given in Equation 6.47. In the ﬁgure, the plus sign is for the case when the acoustic wave moves along the direction of the beam propagation, and the minus sign is taken when acoustic velocity moves against the propagation direction. Figure 6.36a shows the use of a prism to obtain the frequency shift twice that of the acoustic frequency. Figure 6.36b illustrates multiple use of AO cells with two different acoustic frequencies, which allows production of a variety of frequency shifts. The frequency shift is used for application to hetrodyne st

1 order f0 + fa

f0

0 order f0

q f0 fa RF frequencies (a) f = fo + fa FIGURE 6.35 Frequency shifts due to A-O interaction.

0 order

f0

f - fa fa RF frequencies

(b) f = fo - fa

st

1 order

Basic Optomechatronic Functional Units

f0 + 2fa

prism fa

f0

347 f0 + F f0 fo + fa1 + fa2

f0 + fa

f0

f0 + fa

fa1

fa RF frequencies (a) f = fo + 2fa

fa2 RF frequencies

(b) f = fo + fa1 + fa2

FIGURE 6.36 Frequency shifts for two different A-O-D arrangements.

detection, where precise phase information can be measured. A typical example is a laser Doppler velocimeter (or vibrometer). Tunable Wavelength Filtering According to the relation given in Equation 6.41, the angle of deﬂection of an acousto-optic deﬂector is proportional to the optical wavelength. Therefore, it is possible to extract a particular wavelength. This can be done by tuning the center wavelength of the ﬁlter passband, which in turn is changed by the frequency of the applied radio frequency signal. It can be shown that the increment in the spectral width can be deduced as Dl ¼

l va · D fa

ð6:48Þ

where D is the beam diameter. This indicates that low value of Dl gives high resolution, which is achieved by the increase of D. Efﬁciency of Modulation and Speed The important factors that affect the efﬁciency of the modulator are found to be the divergence angle ratio between the optical beam and acoustic beam, and the acoustic power. To preserve the interaction efﬁciency on all the bandwidths of the optical beam frequency, it is necessary to reach the Bragg conditions for all angles of the light beam. If dua and duo are denoted by the divergence angle of the acoustic beam and optical beam, respectively, it is known that the ratio duo =dua < 1 should be met in order to have the maximum interaction efﬁciency. The rise time of the AO modulator depends on the transit time of the acoustic wave propagation across the optical beam, and is proportional to the acoustic traveling time through laser beam. This is given by tr ¼ b

D va

ð6:49Þ

348

Optomechatronics

where b is the constant depending on the laser beam proﬁle. We can see that the only parameter to minimize tr is to reduce the beam diameter. Consequently, it is necessary to focus the incident light beam on the acoustic wave in order to reduce the beam diameter, and thus reduce rise time. As we shall see in the case of the scanners to be discussed in the next section, reducing light beam diameter causes degradation of the scanning resolution. A typical rise time for a 1 mm diameter laser is found to be approximately 0.15 msec.

Optical Scanning Optical scanning is a method of changing the direction of a light beam in a scheduled manner. A variety of applications can be found from laser printers to materials processing. In terms of scanning geometry, there are three categories. These include objective scanning, pre-objective scanning, and post-objective scanning, which are shown in Figure 6.37. Objective scanning is to move a lens to generate a desired path of beam focus, as shown in Figure 6.37a. Therefore, this scanning method does not require a separate scanner, but a scanning lens. Suppose that a collimated beam located a distance x from the optical axis of the lens is directed parallel

scanning lens

beam focus path x

x f

(a) objective scanning

(b) pre-objective scanning scanning device (mirror)

ϕ

FIGURE 6.37 Types of scanning geometries.

scanned surface

ϕ

scanning device (mirror)

collimating objective beam lens

(c) post-objective scanning

objective lens

light beam

L

L

f

Basic Optomechatronic Functional Units

349

to the optical axis of the lens. The beam will then be focused at the focal point of the lens having focal length f. If we want to generate a circular beam path, as indicated in the ﬁgure, we need to rotate about the beam axis exactly the same as the trajectory of the beam, as long as the beam remains within the lens aperture. The pre-objective scanning shown in Figure 6.37b utilizes an objective lens in the optical path after the beam is deﬂected by the scanning mirror. As can be seen from the pre-objective conﬁguration shown in the ﬁgure, the diameter of the focus lens must be much larger than the beam diameter in order to scan effectively. Also, an appropriate combination of lenses is needed to correct the curved scan ﬁeld. It is noted that, in this conﬁguration, the scan length is not linear with scan angle w: The postobjective conﬁguration shown in Figure 6.37c is different from the preobjective in that it uses an objective lens in the optical path before the beam reaches the scanner. The beam enters the objective lens and then is scanned by a scanner represented by a rotating mirror. Due to this arrangement, the scan length (L) is linear with the scan angle, w: since L is equal to R where R is the scan radius. The disadvantage of this conﬁguration is that the scan ﬁeld is curved considerably more than that of the pre-objective when the focal length of the lens is ﬁxed. To correct this curved focal plane, a dynamic focus system is used which employs a lens translator for focus correction in order to move an objective lens along its optical axis (as we shall see later). Basically, scanning in three dimensions is possible when scanners such as galvanometers and AO cells are appropriately arranged. There are several scanning devices that direct light onto objects with a preplanned rotation speed. There are three frequently used methods: (1) galvanometric scanning (2) acoustic-optical deﬂection method (3) spinning (polygon) mirror method. Galvanometric scanners use an electromagnetically-driven motor to steer a beam in a particular pattern over a certain angular range. AO scanners, as discussed in the previous section, and also known as acousto-optic beam deﬂectors, change the direction of the light by an acousto-optic effect — which occurs when a small variation in refractive index arises due to the acoustic wave propagation in the medium of collimated beam. Spinning mirrors have a number of plane-mirror facets parallel to and facing away from a rotation axis of a motor. They provide wide angular range, as much as 3608. Important factors affecting scanning performance include resolution, scan rates, range of scanning angle and pixel placement. In this section, we will consider galvanometers and polygon mirrors, which are widely used due to their high speed scanning. Galvanometer Galvanometric scanners use coated-plane mirrors to improve the reﬂex activity to deﬂect a laser beam. The scan head is operated via an optomechatronic principle, in that the optical device “mirror” conﬁgured with other optics is actuated by an electromagnetically driven motor.

350

Optomechatronics Y Y

X X

imaging objective (f - θ type) Y Y

scanners

X

scanners

imaging objective

X

lens translator

input beam

(a) pre-objective scan

(b) post-objective scan

FIGURE 6.38 Galvanometer scanning methods.

Typically, a pair of scanners is used to deﬂect the incident beam in two axes to create the x-y vector. Three-axis systems can be used when a z-axis stage for static ﬁeld focusing, or a z-axis galvanometer translator for dynamic focusing on tilted, stepped, or curved object surfaces are needed. As already discussed in the introductory part of optical scanning, there are two commonly used scanning conﬁgurations in galvanometer; pre-objective, and post-objective scanning [10]. The pre-objective conﬁguration, shown in Figure 6.38a, uses a ﬂat-ﬁeld scan lens called an f-u lens in the optical path after the beam is deﬂected by the scanning mirror. On the other hand, the post-objective conﬁguration shown in Figure 6.38b uses an objective lens in the optical path before the beam reaches the mirror. As can be seen from

image plane field curvature

cylindrical lens

image plane

toroidal lens

field curvature optical axis

(a) field curvature formed by toroidal FIGURE 6.39 Correction of ﬁeld curvature by f-u lens.

optical axis

(b) field curvature formed by cylindrical lens

Basic Optomechatronic Functional Units

351 y

X-Y pincushion

a +

x f - θ lenses motor

f-θ barrel

Y =

composite distortion

X motor collimated beam

FIGURE 6.40 Pre-objective scan distortion.

Figure 6.39, this lens consists of a positive toroidal lens and a negative cylindrical lens. The toroidal lens produces negative ﬁeld curvature, whereas the cylindrical lens produces positive ﬁeld curvature. Due to this integrated conﬁguration, a ﬂat scan ﬁeld can be achieved. In the preobjective conﬁguration, a beam expander designed to ﬁll the aperture of the objective lens transmits paralleled beams into the mirror of the scanner. Then, the deﬂected beam goes to that of the y-axis. As mentioned earlier, both scanning methods have distortion problem. In a pre-objective scan system, there occurs a distortion called “pincushion” produced by the scan, if a single objective lens is used. To compensate for this error, a ﬂat-ﬁeld objective lens called an f-u lens is predominantly used, as shown in Figure 6.40. The ﬁgure indicates that the barrel distortion due to the f-u lens is added into the pincushion distortion, resulting in much smaller composite distortion. If the f-u lens is simpliﬁed by a single lens, it produces a simple relationship between scan angle, u, and spot location, y 0 y0 ¼ fu which indicates a linear relation for a given focal length. Proportionality between the scanning angles u and the image height y0 ensures proportionality between the angular velocity of the deﬂector and the scanning speed in the image plane. The correction discussed in the above example produces exactly this relationship and therefore a spot location in the scanned ﬁeld proportional to the scan mirror angle. It is noted here that the entrance pupil of the f-u lens has to lie relatively far outside the region of the location of the deﬂecting unit. This provides sufﬁcient distance needed for the movement of the deﬂecting unit, as well as for the entry of the beam bundle.

352

Optomechatronics d Dw = 2 r 0

scanner

z

objective lens curved focal plane

2∆ z

FIGURE 6.41 Fixed focus objective lens of a post-objective scanner.

The post-objective scan system has two types of scanning, which includes a ﬁxed-focus objective lens system and a dynamic-focus objective lens system, as illustrated in Figure 6.41. A ﬁxed system is composed of a focused beam expander and a scanning unit. In this case, the focal length is constant, and thus, the deﬂected beam behind the deﬂector unit is incident to a curved focal plane. Let us discuss the ﬁxed-focus objective lens system ﬁrst. In Gaussian beam optics, as discussed in Chapter 2, “Gaussian Beam Optics,” the beam spreading can be described by " rðzÞ ¼ r0 1 þ

lz pr 20

!2 #1=2 ð6:50Þ

where z is the distance propagated from the surface of the rotating mirror surface, l is the beam wave length, r0 is the radius of the 1=e2 irradiance contour at the surface where the wave front is ﬂat, rðzÞ is the radius of the 1=e2 contour after the wave has propagated a distance z toward the image surface. Suppose that we choose the tolerance of the beam spot size in the scan ﬁxed to be within 5% of r0, that is, rðzÞ ¼ 1:05r0 from the above equation. From Figure 6.41, it can be seen that Dz satisfying rðzÞ ¼ 1:05r0 is given by Dz ¼

0:32pr20 l

In order to keep the scan range in focus, it is necessary to keep it within the depth of focus deﬁned by 2Dz: This means that the variation of focal plane due to the mirror rotation must be smaller than the depth of focus, which may be written by

d , 2Dz

ð6:51Þ

Basic Optomechatronic Functional Units

353

scanner input beam

motor

imaging objective lens

objective lens focal plane

lens translator

lens translator

FIGURE 6.42 Dynamic focus objective lens of a post-objective scanner.

where d is the deviation of the focal plane from the ﬂat one that can be obtained without mirror rotation. Like pre-objective systems, there are distortions generated in the scan ﬁeld: pincushion, and tangent error. To correct the error, electronic correction is normally used. When the scan ﬁeld exceeds the depth of focus given in Equation 6.51, a dynamic focus system shown in Figure 6.42, called a lens translator, needs to be employed to achieve a ﬂat-ﬁeld focus [10]. In this dynamic focus objective system, an objective lens is placed on a slider and translated dynamically in order to relay and focus the beam on to the scan ﬁeld at any instant. There are several conﬁgurations for achieving this objective. Figure 6.43 shows a basic, three thin-lens system for post-objective scanning. The small lens (element 1) located far left is a translating lens, which is actuated by a galvanometric actuator. Elements 2 and 3 can be either plano-convex or best-form signets oriented to minimize spherical aberrations. When a collimated beam enters a translating lens, as shown in the ﬁgure, the beam is expanded by a beam expander composed of elements 1 and 2 to ﬁll the aperture of the scanning

D1 f1

z1

1

f2

f 3 D3

2

3

f2

f1

z3

f3

zc

z 2

1

f2

3

z3

FIGURE 6.43 Basic three-element system for dynamic focus scanning.

f3

354

Optomechatronics

head. The focal length of element 3 is designed by considering the optical path length from the element to the target object scanned. This element usually is placed near to the scan head entrance pupil. On the other hand, the focal length of element 2 needs to be determined by considering the combined focal length of elements 2 and 3, f23, so that the designed focal length can yield an appropriate optical objective function. It is noted from the ﬁgure that the position of the translator z is not linear with the image focus which is denoted by zc : The translator position, focus change, and focal lengths of the three lenses are related by the two lens equations f1 ¼ f2

D1 D3

1 1 1 ¼ þ f23 f2 f3

1 1 1 ¼ þ f23 f2 2 z f3 þ zc

ð6:52Þ

where f23 is the focal length of the combined lens system composed of lens 2 and lens 3, z is the translator position, and zc is the scan position to be corrected. In order to control the position z, zc must be known by some means. However, zc is not known a priori, and thus needs to be measured in an on-line manner by an appropriate sensor. This will be discussed in the autofocusing section (“Visual Auto-Focusing”). When the maximum range of zc and z are known, we can determine the focal length f2 from the above design considerations. According to Equation 6.52, the focal lengths of elements 1 and 2 can be determined by the following equations f1 ¼ f2 D1 =D3 ;

f2 ¼

qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ 1 ðz þ z2 þ 4f3 ðf3 þ zc Þz=zc Þ 2

ð6:53Þ

The variation of the focus can also be derived for a two x-y mirror ﬂat ﬁeld scanning [19]. Let us consider the case when the incident beam enters into the x-mirror then the y-mirror, and ﬁnally hits the surface of the image plane, as illustrated in Figure 6.44. A coordinate system ðx; yÞ is attached to the image plane to describe the image point resulting from the scan angles ux and uy of the two mirrors. The x-mirror is located at point Ox while y mirror is located at Oy : The distance between two mirrors is dxy and the image plane Si is dim distant from the y-mirror surface. Referring to the ﬁgure, we consider an arbitrary point Pm located at ðxim ; yim Þ and a point A located at ðO,yim Þ in the target image plane. Point Pim was originally located at the center O of the image plane when ux ¼ 0, uy ¼ 0: But, due to rotation ux and uy, this point reaches to a point ðxim ; yim Þ: From the geometry yim is given by yim ¼ dim tan uy We need to know the x-coordinate value, xim to determine the focus at point Pim : To obtain this value, let us project the image plane on to the virtual image plane of the y mirror located as indicated in the lower part of the ﬁgure. xim is then expressed by xim ¼ Ox Atan ux

Basic Optomechatronic Functional Units

Ωx

355 x-mirror

y

Ox

A

incident beam

θx

dxy θy

O

y-mirror Ωy

Oy

dim

Pim (xim , yim)

x Σi image plane

O

x

A y Pim (xim , yim)

FIGURE 6.44 Focal length variation due to mirror rotation.

But Ox A is equal to Ox A ¼ dxy þ Oy A where Oy A is given by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Oy A ¼ d2im þ y2im Therefore, xim is rewritten by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ xim ¼ d2im þ y2im þ dxy tan ux

ð6:54Þ

Since Ox Pim is the focal length, we ﬁnally obtain the focus at the point ðxim ; yim Þ as sﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ 2 d2im þ y2im þ d2xy þ x2im fim ¼ From this equation, we can get the deviation of the focal length at ðxim ; yim Þ due to rotation which is given by Dfim ¼ fim 2 ðdxy þ dim Þ

ð6:55Þ

356

Optomechatronics

The Dfim expressed in Equation 6.55 is the amount of the correction needed to obtain the ﬂat ﬁeld scanning. It is noted that this focal length variation is a function of coordinate values in the image plane. In actual practice, drive motors and polygons produce combined errors in the directions of scanning, or at right angles to the scan. These errors result in various scanning anomalies, such as misallocation of initial scanning point, errors in scan line tracking, degraded resolution, and so on. The tracking errors are induced by errors in angular location of polygon facets, errors in facet ﬂatness in being parallel with the axis of rotation of the polygon, errors in fastening of the polygon to the motor shaft, and wobbling. In order to eliminate such tracking errors, dynamic motion, as well as fabrication of the polygon, needs to be carefully examined before it is actually used for scanning. Feedback Control of Galvanometer The foregoing study on optical scanning was made based on the assumption that the dynamics of galvanometers do not inﬂuence the optical path of light beams. However, their dynamics normally affect the scanning performance when they are not properly designed. In this regard, we need to analyze the dynamic characteristics of galvanometers. Galvanometric scanners utilize an electromagnetically driven motor to steer a beam. Figure 6.45 illustrates schematically a galvanometric optical scanning device (one-axis). The galvanometer is essentially an electromechanical transducer containing an optical unit, including mirrors. It consists of an amateur (a moving coil assembly), a position transducer, a driving electronics composed of ampliﬁer power supply and logics, a permanent magnet, and a rotating mirror. The mirror is coated with a thin ﬁlm material to enhance its reﬂectivity, and mounted on the moving assembly of the motor. There are three types of armatures: moving coil, moving iron, and moving magnet. Here, the assembly of a moving coil is mounted in a magnetic ﬁeld on a pivoted shaft, as shown in the ﬁgure, and its rotation is usually constrained. In general, the driver (amateur) needs to have the following ideal properties: (1) (2) (3) (4) (5)

a high torque to inertia ratio a linear relationship between torque, current, and angular position a dynamically balanced amateur immunity to all internal, external dynamic disturbances capability of low power consumption and good heat dissipation

In (1), the torque to inertia ratio affects the dynamic response of the galvanometer, with amateur of inertia driven by a motor capable of maximum torque. In (2), the linear relationship is static, and emphasizes a linearity between the above three parameters. To avoid any undesirable

Basic Optomechatronic Functional Units

Rg

357

Kt 2

ig

θ mirror

Rs

to the objective lens

+ vg

−

w N

S

permanent magnet ied viscous damping

b

laser

Kt 2

FIGURE 6.45 A schematic of galvanometric optical scanning device.

vibration, perfect balancing of the amateur is needed. Also, when the driver is dynamically perturbed, it should be insensitive to its disturbance effect. The moving coil drivers are simple in structure and low cost. Demand for higher performance requires low inertia, a well-balanced mirror and rotor construction, and good heat conduction. This moving, coil-based scanner is well suited to such applications as low-cost instruments, and vibrations and temperature protected environments. The moving iron drivers have a very high torque to inertia ratio, and therefore, are used in most high-speed scanners. They show nonlinearity between torque, current, and position relationships, but can be compensated by ﬂux path compensation. Moving magnet drivers are high performance optical scanners, and meet all the requirements listed in the above examples to acceptable degrees for most applications. These are used for low and medium speed scanners. The galvanometric scanner is a feedback control system that controls its rotation angle in a desired fashion, based on the information of a position sensor. The role of the transducer is to convert the angular position of the rotor into an electrical signal needed for a control electronic part to generate a control command signal. To be ideal, a sensor needs to have the following characteristics; (1) high resolution. (2) a high bandwidth. (3) a high signal to noise ratio. (4) decoupled from the electromagnetic driver. (5) insensitive to radial motions of the rotor. There are several sensors, typical of which are capacitive sensor and optical interferometric encoders. The optical encoders have low inertia and

358

Optomechatronics

high performance, while the capacitance sensors are commonly used due to low cost and simplicity. Galvanometers can position beams over a wide range of angular displacement (0 , 808) at speeds ranging from 1 to 100 msec. The positioning accuracy can be achieved up to micro radians. Because the scanning speed is primarily dependent on the dynamic characteristics of the moving optical mirror, we will analyze its dynamic properties in brief. As can be seen from the ﬁgure, when there is an input voltage to the electrical ﬁeld, a current-carrying conductor, a coil, experiences an electromagnetic force. This, in turn, generates a torque to the mechanical part (mirror and coil), causing it to rotate. Accordingly, the dynamic equations are composed of two governing equations, one for an electrical part and one for a mechanical part. The differential equation for the electrical ﬁeld due to an input voltage is given by ig ðRs þ Rg Þ þ Lg

dig du þ BN‘b ¼ Vg dt dt

ð6:56Þ

where Vg is the excitation voltage, ig is the current, Rg is the resistance of the coil and Rs is the signal source resistance, Lg is the coil inductance, B is ﬂux density, N is the number of turns on the coil, b is the width of the coil and ‘ is the coil length. The third term of the left-hand side of the equation denotes a back emf of the coil. The mechanical part consists of a mass (mirror and coils) whose motion is constrained by a tensional spring and built-in damping, either due to the reversed electromagnet effect alone, or combined with damping from enclosed viscous ﬂuid. The driving torque is assumed proportional to the input current. Then, the dynamics of the moving optical mirror is governed by I

d2 u du ðB‘wÞ2 du þ b þ k u þ ¼ Tg t dt dt Rf dt2

ð6:57Þ

where u is the angular displacement of the mirror, I is the moment of inertia of the mass, b is the viscous damping coefﬁcient, kt is the torsional spring constant, and Tg is the driving torque. The fourth term in the left hand side of the equation arises due to the fact that the frame on which the coil is wound is itself a conductor and, therefore, an electromagnetic torque 2Blwied is generated, where ied is the induced eddy current on the frame. If we neglect the effect of inductance within a certain operating frequency range of the galvanometer, we can easily have the resulting simpliﬁed equation expressed by d2 u du þ 2zvn þ v2n u ¼ Kv2n Vg 2 dt dt

ð6:58Þ

where K is the constant, vn is the natural frequency and z is the damping coefﬁcient. They are given by K¼

BN‘w rad=V kt ðRs þ Rg Þ

Basic Optomechatronic Functional Units

359

sﬃﬃﬃﬃ kt vn ¼ I

z¼

b þ ðB‘wÞ2 =Rf þ ðBN ‘wÞ2 =ðRs þ Rg Þ pﬃﬃﬃﬃ 2 Ikt

where Rf is the resistance of the coil frame. It can be seen from the above equations that increase in sensitivity results when we increase B, N, ‘ and w or decrease kt, Rg and Rs : Also, decrease in kt results in reduced natural frequency, thus making speed of response lower. The damping coefﬁcient shown in the above is composed of three factors: mechanical viscous damping b, electromagnetic damping proportional to du=dt; and back emf proportional to du=dt: Therefore, the inﬂuential parameters must be carefully designed in order to have a goal dynamic performance of the galvanometer. Inspection of Equation 6.58 shows that the response of the galvanometer has three different solutions, depending on z: As discussed in Chapter 4, “Dynamic Systems and Control,” these are: underdamped ðz , 1Þ; critical damped ðz ¼ 1Þ and overdamped ðz . 1Þ: Let us consider the underdamped case. When ðz , 1Þ, the dynamic response of the galvanometer is characterized by peak overshoot (Mp), rise time ðtp Þ and settling time (ts), as shown in Figure 6.46. The amount of overshoot depends on the damping ratio z of the galvanometer. When the damping ratio gets small, the overshoot gets large, and when large, the overshoot gets small. Certainly, we need to keep the overshoot as small as possible; however, for small damping, response time gets longer to arrive at steady state. This indicates that damping ratio should be designed in order that, within the permitted overshoot, the response be kept to a minimum. For example, if the accuracy required in a transient period is within ^ 5%, a damping ratio is chosen such that the percentages overshoot gives 1.05. Figure 6.47 illustrates this example: The response curve is tangent to the upper accuracy limit at point Pt, which is a peak. In this case, time required to 95% of the desired galvanometer rotation is given by 0.454 ðvn t=2pÞ: The damping ratio that meets this requirement is shown to be 0.690. When a galvanometer scans over a certain region in repeated fashion, a sinusoidal voltage must be given to it. In this case, ideally, the motion of the galvanometer needs to follow the input signal without distorting its amplitude and introducing phase shift. In reality, however, changes in amplitude and phase of the galvanometer motion will occur and will be a function of input frequency. The frequency response of the system is given for the magnitude ratio between the input and output

QðvÞ 1 ¼ GðvÞ ¼ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ KVg ðvÞ 2 2 ½1 2 ðv=vn Þ þ 2½zðv=vn Þ 2

ð6:59aÞ

360

Optomechatronics q mirror

mirror rotation angle (q)

underdamped: z <1

±2, ±5%

Mp

over damped: z >1 critically damped: z = 1

0

tr

ts

time (wnt /2π)

FIGURE 6.46 Response of a galvanometer to a step input.

response amplitude

tangency point Pt

1.05 1.03 1.0 0.97 0.95

upper limit

P

lower limit

0

0.2

0.4

0.6

0.8

1.0

time (wnt /2π) FIGURE 6.47 Accuracy limits superimposed on the response curves of a galvanometer.

1.2

Basic Optomechatronic Functional Units

361

and for the phase of GðvÞ which is the phase difference between the input and output ( ) 2½zðv=vn Þ 21 /fðvÞ ¼ tan ð6:59bÞ 1 2 ðv=vn Þ2 where GðvÞ is the transfer function between the input KVg ðvÞ and the output Q: In Figure 6.48, the relationship is plotted in terms of frequency ratio v=vn : The ratio QðvÞ/KVg ðvÞ is a measure of the optical mirror dynamic response. Examination of the ﬁgure indicates that the amplitude ratio is reasonably constant over a frequency range from DC to approximately 40% of the undamped natural frequency, vn of the galvanometer. Also, the ﬁgure shows

amplitude ratio |G(w)|

2.5 ζ =0.2

2.0 1.5

accuracy limit ± 5%

0.4

1.0

0.7 1.0 2.0

0.5 4.0 20.0

0.0 0

1

2 3 frequency ratio (w/wn)

4

5

0

phase

G(w)

–45

ζ=0.2

–90

0.4 0.7

–135 20.0 –180

0

1

4.0

2.0

2 3 frequency ratio (w/wn)

FIGURE 6.48 Amplitude ratio and phase vs. frequency; GðvÞ:

1.0

4

5

362

Optomechatronics

that damping ratio in the range of about 50 to 90% should be used to have reasonable bandwidth. Usually, the frequency response characteristics shown here do not yield a satisfactory response if any modiﬁcation or correction is not made. To obtain satisfactory results, we normally extend the range of acceptable frequency responses by feedback control gain adjustment. The assumptions for obtaining the dynamic model of the optical mirror are ideally made. In actual practice, the driving torque will be constant only when the coil turns are working in a magnetic ﬁeld and are moving in a ﬁxed direction relative to it. In addition, nonvisous damping is present to some extent in the form of ordinary spring hysteresis. As discussed in the above analysis, the system parameters such as damping and spring constant determine the characteristics of the galvanometer response. The open-loop system usually does not exhibit satisfactory response characteristics. To improve positioning accuracy and speed, a feedback control system can be considered, as shown in Figure 6.49. The control system is composed of a controller, power amp, a sensor and a mirror unit. We will consider time-domain response of the galvanometer response, but will not treat that of frequency domain. The performance of this position control system is investigated by using the dynamic equation presented in Equation 6.58. In order to simulate its controlled responses, we will use a PI controller and the parameters shown in Table 6.7. Figure 6.50a shows typical responses with various proportional gain parameters ðkp Þ. Those with integral gain ðki Þ ﬁxed at zero are shown in Figure 6.50b. As we can see, the galvanometer response gets faster, but becomes oscillatory as kp increases. With inclusion of an integral gain, the response becomes faster, with large overshoot, but has no steady state error, as expected. To see the effect of external disturbance such as noise, a step noise of magnitude 0:01=s was included, as shown in the Simulink model in Figure 6.51a. The results are shown in Figure 6.51b. The responses of the P control alone to this input noise show a large error

desired scanning angle + – (qd)

controller

driver

FIGURE 6.49 Schematic of a galvanometer control system.

mirror galvanometer

actual angle (q)

Basic Optomechatronic Functional Units

363

TABLE 6.7 The Parameters Used for the Galvanometer Simulation Parameters

Symbol

Unit

Flux density Number of turns on coil Length of coil Breadth of coil Resistance of coil Resistance of signal source Moment of inertia of moving parts axis of rotation Torsional spring constant Viscous damping coefﬁcient Resistance of frame (Ohm)

H N

Gauss

‘

m m V V g m2 N/m N m/(s radian) V

w Rg Rs I kt b Rf

Value 700 50 0.014 0.04 20 100 0.0015 100 0 1

Source: Ernest, O.D. Measurement Systems Application and Design, 4th Ed., McGraw-Hill, New York, 1990.

at steady state. However, the PI controller shows such error is eliminated due to the integral action at steady state. Polygonal Scanner This scanner is a simple mirror rotating on a motor shaft. The rotation of the mirror provides deﬂection of a ray bundle, which is then imaged by a lens and, ﬁnally, on to the image plane. Because the rotation can be made highspeed and repetitive, a polygonal scanner is used in a variety of applications in such areas as high-speed laser printers, facsimile, automated inspection displays, and so on. However, high-speed operations often cause wobbling and bearing wear, which affects the accuracy of scanning and the lifetime of the scanner. 1.4

1.2 1.0

kp=5.5

0.8

kp=2.5

0.6

kp=0.5

response (q), degree

response (q), degree

1.4

0.4 0.2 0.0 0.00

0.02

0.04

0.06

time (t ), sec (a) the effect of P gain; ki = 0 FIGURE 6.50 Responses of the galvanometer.

0.08

0.10

1.2 1.0

ki=50 ki=75 ki=100

0.8 0.6 0.4 0.2 0.0 0.00

0.02

0.04

0.06

time (t ), sec (b) the effect of I gain; kp= 0.5

0.08

0.10

364

Optomechatronics

Step Disturbance +−

++ Gain

Step

nu(s) de(s) PI Controller Transfer Fcn PI

Scope

(a) Simulink model 1.4

response (q), degree

1.2 kp=0.5 ki=50

1.0

kp=2.5 ki=0

0.8

kp=0.5 ki=0

0.6 0.4 0.2 0.0 0.00

0.02

0.04 0.06 time (t ), sec

0.08

0.10

(b) the effect of disturbance FIGURE 6.51 Responses of the galvanometer to a step disturbance.

Many of the important considerations for galvanometic scanning have been already treated in the earlier sections. Most of these considerations are also applied to polygonal scanning, for instance, pre-object and postobjective scanning methods, curved focal plane, and reduction of distortion by an f-u lens. Here, we will discuss resolution of scanning in some detail. There are two typical arrangements of collecting optical beams. In an optical arrangement shown in Figure 6.52a, the collimating beam impinges the scanner surface, and then converges at a focal point. The other arrangement, shown in Figure 6.52b, utilizes the collimating beam that illuminates the facets, then is reﬂected from the mirror and ﬁnally, focused on to an object surface. Both have the curved focal image surface. However, the collimated input beam has an advantage in that it illuminates the full area of facet, thus resulting in better resolution and minimization of modulation of the beam scanning angle.

Basic Optomechatronic Functional Units

365

polygon scanner

polygon scanner mirror

object Ω

Ω

lens

lens

object

(a) convergent input beam

(b) collimated input beam

FIGURE 6.52 Two different types of input beam for a polygon scanner.

The scanning resolution can be analyzed by applying Rayleigh criterion theory to derive the number of resolvable elements that can be produced by the scanner shown in Figure 6.53. In the ﬁgure, D is the beam width, or the width of the scanner mirror and Dwmin is the minimum resolvable deﬂection angle. If it is assumed that the aperture is uniformly illuminated, Equation 2.84 can be applied to obtain the minimum diffraction angle in the scan

Ω +

polygon mirror

D lens

scan plane FIGURE 6.53 Number of beam spot for a polygon mirror.

366

Optomechatronics

direction, which varies with the aperture shape. The angle is expressed by Dwmin ¼

1:22l D

ð6:60Þ

From this minimum resolvable deﬂection angle, we can now obtain the number of resolvable spots per scan which is given by w ð6:61Þ N¼ Dwmin where w is the total angle that can be scanned by polygonal mirror, and N is the number of spots. Let us consider how we can determine total angle w that a polygonal mirror can scan. As shown in the ﬁgure, the polygonal scanner has a multifaceted mirror which is driven by a driving servo motor as discussed for the case of galvanometers. The angle scanned by a polygonal mirror of M facets is written as

w¼

2ð360Þ 4p ðdegÞ or w ¼ ðradÞ M M

ð6:62Þ

The above result is due to the fact that M mirrors are on the circumference of a circle and that the scan angle is doubled by reﬂection. Equation 6.62 also indicates that the angle doubling affects the angular position of the focused spot at the focal plane and the scan length. Now, if this scan angle is related to a minimum resolvable angle for a circular aperture, the number of spot N can be obtained by N¼

w ¼ Dw

4p D 1:22 Ml

ð6:63Þ

This result implies that N is dependent upon the aperture diameter, the width of a facet D, the beam wavelength l and the scan angle w: The larger D, the number of beam spots gets larger, which indicates that the aperture needs to be fully illuminated over the whole facet. However, if M is increased within the diameter limit sufﬁcient to reﬂect the beam effectively, the resolution would become low, because this reduces the available scan w; as we can see form Equation 6.63. For instance, suppose that we wish to scan a collimated He-Ne laser beam having 8 mm diameter with a polygon mirror. If M ¼ 16; we should have a spot number 5.1093 £ 103. If we increase the face number M ¼ 40 the spot number N will increase 2.0437 £ 103. At this point it is noteworthy to realize that, once the number of facets, M and scan speed, vs are given, the scanner rotational speed, V is expressed by

V¼

60 vs rpm M

where vs is given in terms of line scans/sec. This relation shows that, as M increases, the rotation speed can be made to decrease. However, increasing M, as mentioned in the above, results in degrading the scan resolution.

Basic Optomechatronic Functional Units

objective scan lens

Ω

367

image plane

polygon mirror

beam collimator

position sensor _

AO deflector

+

desired scan position

control gain

laser source FIGURE 6.54 Optical feedback control system utilizing an AO deﬂector.

Correcting Scan Errors There are two methods for compensating scan errors, active and passive. The active method is used in the case when errors are not consistently repeatable. As we have already discussed in brief previously, one active method utilizes a dynamic focus system for correcting scan ﬁeld length. Another active method is to correct scan position error. In this case, active correction for beam angle (based upon the information of the position of the scanned beam) is made. On the contrary, the passive method utilizes additional optical components, such as cylindrical lenses in front of or after the polygon in the light path. Figure 6.54 illustrates an active microprocessor-based scheme to correct the scan error. It utilizes an acousto-optic deﬂector whose deﬂection angle is governed by acoustic wave frequency. It is placed in the beam path prior to a polygon mirror, and its deﬂection angle, which is the correction angle, is controlled based upon the measured information of the scanned beam position. This technique is effective to correct the facet errors in the range of arc minute, which is small. The tolerance of polygon fabrication is within this range.

Optical Switch In conventional mechanical ﬁber switches, ranges of macro-scale actuators have been adopted in order to control the switching action. Due to this actuator constraint, the structure becomes bulky, and the switching speed often becomes limited due to large mass. Recent advances

368

Optomechatronics mirrors

mirror input fiber

mirror 2

output fiber 1

4

3

output fiber input fiber

(a) on-off switch

(1 ×1 )switch

(b) cross-bar switch (2 × 2 switch)

(c) optical cross connector (n × n switch)

FIGURE 6.55 Conﬁgurations of micro-optical switches.

in micro-electro-mechanical systems (MEMS), however, have opened up many new possibilities for optomechatronic switches because movable structures, microactuators, and micro-optical elements can be monolithically integrated on the same substrate. The application of the MEMS is expected to improve the performance of switching devices. Figure 6.55 illustrates some of the architectures of micro-optomechatronic switches in terms of I/O port number. They include a simple on-off gate switch, a 2 £ 2 crossbar switch, and a n £ n optical cross connector. The switching system consists of a single mirror or multiple mirrors that steer the light beam, actuators that position the mirrors, collimating lenses, and input/output ﬁbers through which light comes. There are basically two ways of achieving switching as illustrated in Figure 6.56. In the left-hand side of the ﬁgure (Figure 6.56a) switching is achieved by controlling the position of the lenses. Contrary to this, the architectures shown in Figure 6.56b use the control of the position of mirrors for switching, as indicated by dotted lines. With this switching conﬁguration, many channels can be addressed, but here only two channels are drawn for simplicity. It is noted from this system that a telescope lens is normally employed to extend the beams on to the optical ﬁbers. In any conﬁguration, the motion of lens or mirror should be fast and smooth to avoid any undesirable phenomenon that might occur during optical signal transmission. A number of silicon-based actuators have been developed and implemented to realize movable microswitching structures. The commonly used actuators include a comb drive that produces electrostatic force between a pair of multi-ﬁngered rods and combs, thermal actuators that utilize thermal expansion, and microvibrators that use impact actuation to obtain large motion from resonant structures having small-displacement stepper motors that achieve large force. We will discuss some of the microoptical switches that employ the above actuators.

Basic Optomechatronic Functional Units

369 input 1

lens

optical fiber

optical fiber

M1

input 2

mirror array M2 output 1

M4 M3

(a) lens-controlled optical switch

output 2

(b) mirror-controlled optical switch

FIGURE 6.56 Typical methods of achieving optical switching.

Thermally Actuated Mirror The actuation method by heating utilizes the expansion phenomenon when a material is heated. The strain due to the thermal expansion of a metal cantilevered structure clamped at one end is given by elementary mechanics 1x ¼

DL ¼ a DT L

ð6:64Þ

where 1x is the axial strain along x-direction, L is the original length of the material, DL is the elongated length, a is the thermal expansion coefﬁcient and DT is the temperature difference in 8C. A variety of the thermal actuations utilizing this concept has been developed, and one of the popularly used conﬁgurations called bimorph or bimetal actuator is illustrated in Figure 6.57. It is composed of two base metals, the siliconoxide in the top having the thermal expansion coefﬁcient a ¼ 2:3 £ 1026 =8C and the aluminum in the bottom having the coefﬁcient a ¼ 25 £ 1026 =8C: Due to this signiﬁcant difference in a, the heated arm of the aluminum is expanded approximately 10 times larger than that expanded by silicon. This length difference causes the arm to move in an upward motion as indicated in the ﬁgure, which can be transformed into lateral movement. The thermal actuator is typically composed of thin-ﬁlm heaters and four counter parts of bimetal layers. It needs to be fabricated on a planar wafer surface before etching the ﬁnal micromechanical structures. Therefore, an additional process of depositing a thin-ﬁlm heater material is required in addition to bulk silicon micromachining. The actuators are found to produce large displacement and high force. A typical application of this actuator to an optical ﬁber switch is a thermally actuated bistable ﬁber switch, the top view of which is shown in Figure 6.58 [20]. It consists of two main actuators, one actuator for switching ﬁber and one actuator for optical ﬁber clamping, which is not shown here. The former is designed to move horizontally to displace an optical ﬁber, and the latter moves vertically to open and close the latching mechanism. Let us look into the switching mechanism in more detail by using a schematic shown in the ﬁgure. The actuator for switching action is a

370

Optomechatronics SiO2 Al (a) before heat applied

SiO2 Al

(b) after heat applied FIGURE 6.57 Thermal expansion of a bimorph.

U-shaped cantilever composed of two arms heated separately with thin-ﬁlm heaters. The heat is generated in thin-ﬁlm heaters on top of each cantilever arm. Due to the heated arm expansion, the upper arm causes the whole switching unit in the ﬁber to be switched from output ﬁber a to ﬁber b. The angular displacement of the actuator-causing switching motion can be obtained by a simple geometric relation shown in the ﬁgure tan u ¼

DL w

where w is the width of the arm and DL is the elongated length of the arm. If we assume a small angle and combine this equation with Equation 6.64, then, the resulting equation is written by

u¼

aLDT w

The angular displacement that can be achieved by this thermal actuator for a temperature increase can be easily calculated by the above equation. when U-shaped cantilever heated θ thin film heater

w

a

thin film heater

b L

input optical fiber

output optical fibers

FIGURE 6.58 Thermally actuated by optical ﬁber switch. Source: Hoffmann, M., et al. IEEE Journal on Selected Topics in Quantum Electronics, 5:1, 46–51, 1999 q2005 IEEE.

Basic Optomechatronic Functional Units

371

To illustrate this, let us calculate the angular displacement u for the Al-SiO2 bimorph. If DT is 5008C and the geometry for the actuator is given by L ¼ 200 mm, w ¼ 20 mm, we have

u¼

0:6 £ 1026 £ 200 £ 1026 £ 500 ¼ 0:003 rad ¼ 0:178 20 £ 1026

Depending upon the application, the shape, geometry, and dimension of the actuator can be designed differently. Electrostatically Actuated Mirror Control The comb drive mechanism has wide applications in actuating optical switches, and utilizes the electrostatic actuator. Figure 6.59a illustrates two directional forces that can be actuated by the comb drive mechanism; (1) comb driving horizontal force (2) gap closing force. The comb driving horizontal force per gap shown in Figure 6.59c can be derived by using Equation 4.51 and written as Fx ¼

10 V 2 w 2z

ð6:65Þ

where 10 is the permittivity of the air, V is the input voltage applied to the set of the plates, x is the overlap distance whose coordinate is shown in the ﬁgure, and w is the width of the comb structure. The gap closing force per

w V

V

z

z x

(a) comb driving force

(b) gap closing force

mirror

movable

mirror movable

fixed (anchored)

(c) comb driving actuator

FIGURE 6.59 Comb driving vs. gap closing.

(d) gap closing actuator

372

Optomechatronics

gap is illustrated in Figure 6.59b. In a similar way, as derived in the case of comb driving force, the force can be derived as Fz ¼ 2

10 V 2 w 2z2

ð6:66Þ

Note that the gap closing force increases drastically as gap, z, decreases. Equation 6.65 and Equation 6.66 imply that we can compute the input voltage V in order to obtain the desired gap zd for a given comb structure of width w. Ideally, this may be true, as long as the above theoretical relations hold. In case these relationships do not hold, a feedback control system may be necessary to obtain a desired force or a desired displacement of the comb drive. Figure 6.60 illustrates an example to help us understand a problem involved with a control input voltage generation for a horizontal comb drive system. Its structure is ﬁrmly anchored to the ground at its one end, and at the other end through four beam springs whose material have Young’s modulus E and Poisson’s ratio m: Each gap is kept constant at z0 , has width w and operates in the air. If the drive moves in the horizontal direction by x0 from an equilibrium state, the control input voltage V can be computed in the following procedure. Using Equation 6.65, we have the driving force for

_

V

+

x

k0

k0

k0

k0

h zo zo

spring comb drive

(a) comb drive structure

(c) comb drive dimension FIGURE 6.60 Horizontal comb drive mechanism.

L

(b) spring dimension

Fx

Basic Optomechatronic Functional Units

373

all gaps Fx ¼

710 wV 2 z0

ð6:67Þ

where Fx is the total force generated in the x direction. Due to the movement of the comb drive, the spring force acting in the x direction is given by Fx ¼ kx0

ð6:68Þ

where k is the equivalent total spring constant of the spring structure shown in the ﬁgure. The cantilever spring constant can be replaced by an equivalent spring constant, and can be derived by considering four cantilevered beams whose dimension is given in the ﬁgure. k ¼ k0 ¼

E wh3 1 2 m2 4L3

ð6:69Þ

where k0 is the equivalent spring constant of each plate. Note that this spring force must always be equated by the driving force Fx to stay in equilibrium. Equating Equation 6.68 and Equation 6.69, and solving for V, we have sﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Eh3 x0 z0 V¼ 2810 ð1 2 m2 ÞL3 The above relation implies that the voltage input is a complex function of the comb structure geometry, involving a number of parameters. Roughly speaking, it is in proportion to the gap z0 and the overlap x0 and inversely in proportion to the beam spring length L3 : A typical application of this comb drive can be seen in Figure 6.55b, where a mirror is actuated by an electrostatic comb drive actuator in order to accurately steer the light to a desired ﬁber. The conﬁguration describes a 2 £ 2 switch which uses a double side reﬂecting vertical mirror. The four ﬁbers are placed into grooves at 908, which are not shown here. When the mirror is out of the optical path, the light travels in a straight path. On the other hand, if it is in the cross state (that is, in the optical path) it is reﬂected by the mirror, creating two light paths, one from the ﬁber no.1 to no.4, the other from the ﬁber no.2 to no.3. It is noted that switching occurs due to the linear motion of the mirror as indicated by the arrow. As mentioned earlier, inserting and removing the mirror in the coupling gap needs to be made within the desirable switching time and with minimum switching loss. Also, it should not yield any jittering motion of the optical switch. Another application of the electrostatic drive is scratch drive actuator (SDA), whose schematic view is described in Figure 6.61a [4]. The actuator consists of three parts: plate, bushing, and substrate electrode. In essence, the plate has a bushing along one side, and forms a capacitance-like structure with the substrate. When a voltage is applied across the capacitance, an electrostatic force can be generated on the plate as given by Equation 6.66. Therefore, when a square pulse is applied periodically between the plate and

374

Optomechatronics plate bushing

(a) V

insulator

substrate

(b)

(c)

(d) dx FIGURE 6.61 A sequence of the motion of the scratch drive actuator (SDA). Source: Akiyama, T. et al. Journal of microelectromechanical systems, 6:1, 10–17, 1997. q2005 IEEE.

substrate, a periodic electrostatic force is generated which results in a forward step motion of the plate, in the substrate plane as shown in the ﬁgure. The detailed sequence of its operation is given as follows: When a square pulse starts to be applied between the plate and substrate, the plate is pulled down toward the substrate. Due to the bushing at the front, deformation of the plate is built, as shown in Figure 6.61b. It is noted that in this deformation, near the region of the bushing, the plate deforms as a triangular shape due to the pulling-down motion of the rest of the plate. As the pulse falls, the stored energy retaining the deformed plate starts to be released and the plate begins to regain its original shape as indicated in Figure 6.61c. During this restoring stage of the plate, its rear part moves forward as the bushing remains in contact with the insulator. When the pulse is applied again, the rear part of the plate quickly comes into contact with the insulator, since this part has a smaller separation distance from the insulator. Due to this deformation of the rear part, the front bushing is pushed forward and slides on the surface like a worm. When the voltage is removed, the plate and bushing return to their original position, but translate a small distance, denoted by dx in Figure 6.61d. This completes one cycle motion, dx, in the ﬁgure. Repeating this motion due to the applied pulse, the plate can move continuously in a substrate plane. The conﬁguration and velocity of the SDA motion is found to be dependent on the frequency of the applied pulses. Also, the force magnitude

0

375

applied voltage V1 > V2

applied voltage V1 > V2 force (F), μN

velocity (v ), μm/s

Basic Optomechatronic Functional Units

V1 V2

V2

0

frequency (fd), kHz

(a) the effect of frequency on velocity

V1

number of plate (N)

(b) the effect of number of plate on force

FIGURE 6.62 Velocity and force characteristics of a SDA actuator.

is found to be dependent on the number of plates. Figure 6.62 shows the characteristics of their relationship. As might be expected, the higher the driving frequency, the greater the velocity. It also shows that the velocity increases with the voltage applied, since a higher electrostatic force yields a larger pulling force, resulting in a larger step size. Force is found to increase with the number of plates, whose trend depends also on the applied voltage. Lens Controlled Switching A nonblocking n £ n micromechanical lens scanner for optical switching which employs control of the lens position shown in Figure 6.56a is schematically illustrated in Figure 6.63 [34]. It is composed of several tens of input and output ﬁber ports conﬁgured in an x-y stage. A refractive or diffractive microlens located in front of each ﬁber manipulates the light beam, thus sending the collimated and steered beam into a telescope which magniﬁes the scan angle. The resulting beam is ﬁnally transmitted to an

telescope 1

optical fiber matrix (input fiber port)

y

telescope 2

optical fiber matrix (output fiber port)

x z

2D beam switching device FIGURE 6.63 Non-blocking optical ﬁber matrix switch. Source: Hiroshi T., et al. MOEMS 99, pp. 165 –170, 1999.

2D beam switching device

376

Optomechatronics f

micro lens f

collimated beam scan angle ∆y/ f

collimated beam ∆y

optical fiber optical fiber actuator (y)

micro lens

(a) unchanged lens position

(b) changed lens position

FIGURE 6.64 Directing beam by lens motion control. Source: Hiroshi, T., et al. MOEMS 99, pp. 165– 170, 1999.

output ﬁber. Each lens is controlled by an actuator, depending on the beam switching requirement. The role of the microlens is to collimate the beam when the input ﬁber is placed at the focal length of the lens, as indicated in Figure 6.64a, and then to divert the beam to a certain desired angle, depending on the displacement of the lens in a lateral direction, as shown in Figure 6.64b. The ﬁgures show how the beam is collimated by the lens having a numerical aperture NA when an input ﬁber is located at the focal length, f. When this lens is pushed upward by Dy, the lateral lens motion bends the ray with an angle that satisﬁes the following condition

u ¼ sin21 ðDy=f Þ This small displacement of the lens is produced by a microactuator such as the electrostatic comb drive discussed previously. In order to obtain such necessary lens motion, a feedback control can be employed for the mirror switching. This one-dimensional lens motion control concept can be extended into two dimensions, a two-dimensional switching can also be achieved by actuating the lens in the x and y directions. Because this nonblocking matrix switch involves a n £ n switching, it requires a large scan angle which can be achieved by using a lens with a short focal length and a microactuator’s large travel distance. However, small focal length causes large spherical aberration. In addition, an electrostatic comb drive mechanism is limited to about 10 mm in traveling distance. Due to this reason, telescopes are used to widen the scan angle so that they can cover the region of the output port. We have seen several cases of mirror or lens control for switching action. In many cases of switching mirror or lens, there may be a need to accurately position mirrors or lenses to achieve effectiveness of light beam steering. To make such a precise control of the mirror position, a position control system can be built as shown in Figure 6.65. The control objective is to generate force, which can position it at a desired position xd : In the ﬁgure, V, is the control

Basic Optomechatronic Functional Units

377 x mirror

desired position

xd

+ −

controller

V

mirror position k

x

comb drive

sensor

FIGURE 6.65 Control of the position of a comb drive.

input voltage to be applied to the comb drive actuator. The performance of the control may be dependent on how reliably and accurately the displacement can be measured in a real time manner, since the amount of displacement to be measured is extremely small. To ﬁt this requirement, a noncontacting optical measurement method may be conceivable.

Zoom Control Maintaining the quality of the image produced by a lens throughout its zoom range is the purpose of the zooming control. A zoom lens can achieve this objective. It is an optical device which provides variable magniﬁcation without moving the image out of focus, by varying its focal length. In other words, the focal length can be varied continuously along the optical axis, the image position being maintained in a ﬁxed plane by either optical or mechanical means. This can be done by adding another lens that increases the size of the image by apparently increasing the focal length of the original lens. If the focal length is varied, but the image is not maintained, the system is called varifocal. The rate and direction of the lens motion are governed by in-out cams or slots cut into a rotary cylinder. Zooming Principle The zooming range of lenses is measured in its own X factor, called “zoom ratio.” It is given by X¼

longest focal length shortest focal length

For example, 35 to 70 mm is denoted as 2X and 80 , 200 mm is 2.5X. This is an indicative index that describes the power of zoom to increase the linear

378

Optomechatronics object

lens image screen

f

(a)

lens

Initial position

1.5 f

(b)

location P

FIGURE 6.66 Zooming concept by using a single lens.

dimensions of the image it produces. With a 2X zoom lens, the magniﬁcation of its image will be two times taller and two times wider than at its least magniﬁcation. The added lens is called a magniﬁer, and changes the magnifying effect with its distance from the original lens. To illustrate how the zooming principle discussed in the above example, let us consider a single lens arrangement shown in Figure 6.66a. When this lens with the focal length f moves forward to a location P at which the image distance is 1.5 f, the question is how large the magniﬁcation of the image will be. Assume that object distance is much larger than the image distance. To answer this question, we can draw the ray path through the pin-hole model of the thin lens as shown in Figure 6.66b. The important results we have here are: (1) the image obtained becomes enlarged by moving the lens by a factor of 1.5f/f ¼ 1.5, but only a portion of the object are seen and (2) the obtained image becomes somehow blurred due to the displacement of the lens, which changes the distance from the image plane. In order to have the whole object magniﬁed and in focus, we need an optical system exhibiting autofocusing as well as a variable focal length: The longer the focal length, the larger the magniﬁcation factor becomes. Therefore, it is necessary to have a compound lens system at least more than two when an appropriate magniﬁcation is required. The procedure for zooming and autofocusing is explained in the ﬂow chart shown in Figure 6.67. To explain the optical zooming principle in more detail, let us consider the system composed of two lenses, as shown in Figure 6.68. Suppose that the positions of two lenses L1, L2 in case (a) yield a focus point F at an image plane. If we move L2 (negative lens) backward, i.e., toward the

Basic Optomechatronic Functional Units

379

image acquisition blurred image autofocusing

fine image magnification ratio setting

lens position control

final focus adjustment

enlarged image

FIGURE 6.67 The zooming procedure.

image (focal) plane in case (b), the resulting focal length becomes shorter than the original, which means that it shifts forward, being deviated from the image plane. It is therefore necessary to make the shifted focal point back to the image plane. One way of achieving this is to move lens L1 (positive) backward, while tuning the position of lens L2 at the same position, so that the image of an object is focused at image plane (case (c)). In this system, the lens shifting the focal length is called “zooming lens,” or “variator,” whereas the lens correcting the focal point is called “correcting lens.” In case of analyzing multi-lens systems like the above system, it is effective to use the matrix method discussed in Chapter 2, “Multiple Lenses and System Matrices.” If f1 and f2 denote the focal length of the ﬁrst and the second lens for the zoom control system, respectively, the paraxial matrix is determined by 2

" M¼

s11

s12

s21

s22

#

d 61 2 f 6 2 ¼6 6 4 d

3 1 1 d 2 2 þ f1 f2 f1 f2 7 7 7 7 5 d 12 f1

ð6:70Þ

where d is the distance between them. From this matrix, we obtain the equivalent focal length, feq and the length, lF from the optical system to the

380

Optomechatronics zooming lens L2 correcting lens L1

image plane

F Σi d

(a)

zooming lens L2

correcting lens L1

image plane

. F Σi

(b) correcting lens

L1

zooming lens L2

F

image plane

Σi

(c) FIGURE 6.68 A schematic illustrating the zooming principle.

focal plane Si : They are determined by feq ¼

f1 f2 ; f1 þ f2 2 d

lF ¼

ð f1 2 dÞf2 f1 þ f2 2 d

ð6:71Þ

This result implies that by varying the distance d, the focal length of the equivalent system and the distance to image plane can be easily varied for the case of a two-lens zooming system. It is noticed that for this combination of convex and concave lenses, the increase in d results in shorter focal length and image plane distance. For example, suppose that focal lengths of the convex and concave lenses of the zooming system are given by 15 mm and 10 mm, respectively, and that initially the two are separated from each other d ¼ 10 mm. In this case, the paraxial matrix becomes from Equation 6.70 3 2 2 3 2 3 1 " # " # 1 1 6 2 2 30 7 s11 s12 7 6 1 10 7 1 0 6 1 2 15 7 6 ¼4 M¼ 7 ð6:72Þ 5 4 5¼6 4 1 5 s21 s22 10 1 0 1 0 1 10 3

Basic Optomechatronic Functional Units

381

From Equation 6.71, we have feq ¼

21 ¼ 30 mm; s12

lF ¼ s22 feq ¼

1 £ 30 ¼ 10 mm 3

When two lenses become closer, that is, they decrease to 8 mm, they are obtained by feq ¼

21 ¼ 50 mm; s12

lF ¼ s22 feq ¼ 23:3 mm

The results imply that, due to this change, both the equivalent focal length and the distance to the image plane increase, as might be expected. From Equation 6.71 we can obtain the sensitivity of a thin zoom lens with respect to separation distance. By differentiating with respect to d, we have dfeg f1 f2 ¼ dðdÞ ð f1 þ f2 2 dÞ2

dlF 2f22 ¼ dðdÞ ð f1 þ f2 2 dÞ2

ð6:73Þ

It is noted that the two sensitivity functions get relatively large when f1 and f2 are small. Zooming-in and zooming-out utilize this multi-lens operating system. In this system, a zoom lens is moved forward and backward in order to achieve a desired variable magniﬁcation. In most cases, it is composed of a group of lenses derived from a Taylor triplet, plus-minus-plus, and moves in a complex way. A three-lens case is illustrated in Figure 6.69. The front convex lens near to the object, Lf is not moving to provide a tight seal. When the concave Lm is in mid-position, the system has an image of a certain size, as shown in Figure 6.69b. However, when this concave lens moves forward (shown in Figure 6.69a), the image becomes smaller, as can be obtained by a wide angle lens system. On the other hand, when the lens moves backward as shown in Figure 6.69c, the image becomes larger due to the enlarged equivalent focal length — as can be seen by a telephoto lens system. It is noted from this ﬁgure that the equivalent focal length in each case is different. The focal length of case (c) is much longer than that of case (a), indicating that a desired portion of an object can be zoomed with a certain magniﬁcation. Let us consider one more zooming problem with a three-lens system. A software package for lens design and analyis has been used to analyze this zooming system. Figure 6.70 shows the computed relationship between lens position and the corresponding focal length when the ﬁrst convex lens is held ﬁxed far left at a constant location. Those relationships are denoted by a dotted line for the distance between two lenses L1 and L2, and a chain line for the distance between L1 and L3. To facilitate the calculation of the relative positions of the lenses, two curves are approximated using 4th order

382

Optomechatronics

object

Lf

object

Lf

object

Lf

Lm

image plane

Lb

(a) Lm

image plane

Lb

(b) Lm

image plane

Lb

(c) FIGURE 6.69 Zoom lens illustrating wide-angle position (a) and telephoto position (b).

f1 = 15

f2 = –10

f3 = 10

19 18

case 1

17

film plane

equivalent focal length (feq ), mm

20

16 15

case 2

14 13 12

case 3 18

16

14 12 10 8 6 distance from film (d), mm

FIGURE 6.70 Focal length variation due to lens movement.

4

2

0

Basic Optomechatronic Functional Units

383

polynomial as 2 3 dL2 ¼2411:29174þ112:76932feq 210:70851feq þ0:43622feq 26:51 4 £1023 feq

ð6:74Þ

2 3 4 dL3 ¼2468:11113þ121:31288feq 211:1073feq þ0:43888feq 26:39£1023 feq

where dL2 and dL3 are the positions of the lenses L2, and L3 from ﬁlm, respectively. When we use these curves, the magniﬁcation ratio can be speciﬁed by using the equivalent focal length, as long as two lenses are positioned at the two curves. Suppose we wish to change the initial lens conﬁguration of case 1 into that of case 3: case 1 has the equivalent focal length of 19 mm, while case 3 has that of 13 mm, as a result of a change of the conﬁguration. Since the equivalent focal lengths of the two cases are known, the positions of the two lenses can be easily determined from Equation 6.74. The positions of two lenses for case 1 are obtained as follows: the concave lens position of 9.19 mm and the convex lens 4.62 mm, while those for case 3, the concave lens, 17.41 mm and the convex lens, 13.53 mm. We therefore obtain the travel distance of each lens as ðconcavelensÞ¼17:4129:19¼8:22mm ðconvexlensÞ¼13:5324:62¼8:91mm We can see that, as a result of these two lens movements, the image ratio changes to M¼

feq1 13mm ¼ ¼0:68 feq3 19mm

where feq1 and feq3 denote the equivalent focal length of case 1 and case 3, respectively. Zoom Control Mechanism As we have already discussed using Figure 6.69 and Figure 6.70, providing zoom to obtain a desired magnitude of an object image requires adjustment of a focusing lens location. This indicates that from wide angle having the shortest focal length of the system to telephoto location having the longest focal length, lens Lb and lens Lf are required to simultaneously move with a certain relationship. Normally, the change in travel stroke of the two lenses is determined by the design of type and conﬁguration of the lenses. There are basically two methods of controlling lens travel length. One popular method is to use a cam driven mechanism for each lens motion, and the other is to drive each lens independently. In the cam method, two cam mechanisms are driven by one stepper or servo motor, as shown in Figure 6.71. The lens L1 is located and ﬁxed at the front (top), facing objects to be imaged. By a pin connection, the lens group, L2 is designed to move inside a set of three slots

384

Optomechatronics

1st lens (L1)

2nd lens (L2) cam barrel 3rd lens (L3)

motor

2nd lens (L2) 3rd lens (L3) FIGURE 6.71 Zoom lens control by a cam driven mechanism.

of the cam mechanism located right behind lens L1; whereas lens group L3 is designed to move inside another three cam slots located next to the previous cam-slot set. These slot-cams are all provided at the outermost lens barrel, which looks like a tube assembly. A slot-cam set (1) guides lens group L3. The barrel has a gear at its bottom periphery, as shown in the ﬁgure. When the motor starts to drive the barrel with this gear connection, the lens of each group moves within each cam curvature, rectilinearly moving along the optical axis of the system, as discussed above. The relationship between the curvatures of the two slot-cams is so designed that it produces exactly the same relationship between the relative displacement as the desired relationship between the lenses. With this cam mechanism, the desired variation of the lens position can be easily implemented. If the barrel is assumed to be designed to move in proportion to the increment of the focal length, the incremental rotation angle of the barrel necessary to move the lenses to the speciﬁed position can be easily obtained by Rb Du ¼ Dfeq

ð6:75Þ

where Dfeq is the increment in equivalent focal length due to zooming control, Rb and Du are the radius of the barrel, and the rotation angle of the

Basic Optomechatronic Functional Units lens 1 (fixed) lens 2

385

CCD lens 3 DSP

barrel 1

barrel 2

ultrasonic motor 1

ultrasonic motor 2

video signal processing unit

microprocessor (MPU) FIGURE 6.72 A schematic of microprocessor-based zoom control.

barrel provided by a motor, respectively. From this relation, if Rb ¼ 5 mm, then the angle increment that provides the necessary change in the equivalent focal length can be obtained for by Du ¼

13 2 19 ¼ 21:2 rad ¼ 268:758 5

As we may understand from examination of the slot-cam mechanism, this conﬁguration method may have some difﬁculty achieving the desired accuracy of fabrication and lens alignment. Also, ﬂexibility in zooming is not so high, due to the ﬁxed travel pattern of the lens. The second zoom control method is designed to reduce the above limitations. The structure shown in Figure 6.72 shows schematically the details of the zooming conﬁguration. The slot-cams are all removed, but, instead, the system utilizes two lens transport mechanisms driven by two ultrasonic motor drives. All the motion of the mechanisms is controlled by a microprocessor control unit (MPU). The motion data of the two lens groups are provided and preprogrammed in the MPU. From the signal generated by MPU based on these, each motor drives its own lens group accordingly: motor 1 drives lens barrel 1 containing the lens group L2, whereas motor 2 containing the lens group L3 drives lens barrel 2. At the other end of the system next to lens group 3, there is an image sensor (CCD) that feeds the information of focusing sharpness through a data signal processing (DSP) unit to the microprocessor in order to achieve autofocusing. The operation principle is as follows: When the system is set to a desired zoom position (i.e., either a telephoto or wide angle), two motors start to simultaneously drive each lens group according to the command signal sent by the MPU. Then, the image sensor located at the other end of the system next to lens

386

Optomechatronics

group L3 acquires the image of a scene, and feeds the information of focusing sharpness to the microprocessor in order to achieve autofocusing. Based upon this feedback information, each ultrasonic motor adjusts the position of its own lens, so that two lens groups move along the optical axis to desired locations. During the motion, the MPU registers the location of each lens group. Near the end of the motion, one lens group (say group 2) can move in ﬁne mode independently from the other group in order to achieve the desired focusing. The information on the focus sharpness at a certain location of the lens group is provided by the CCD sensor. As will be discussed in “Visual Autofocusing,” the focus sharpness can be represented by the focus measure, which is subject to image condition. This focus information is sent to the MPU, which, in turn, utilizes the value to calculate the instantaneous position of the lens groups.

Visual Autofocusing Localization of the focal position of an optical system is a task critical to imaging for many optical systems, including microscopes, optical disks, and so on. The localization is normally carried out by a technique called autofocusing. This is an important procedure to obtain a sharp image, which is ultimately required for dimensional measurements, inspection, object tracking, and visual servoing. For instance, object distance from a focused lens position based on depth-from-focus is useful for measuring industrial parts or products, which usually have varying depth within their own geometry. Electronic cameras can be also automatically adjusted, such that they give a best-focused image by searching for the lens position. This is also important for lighting a microscope, when sharpness, robustness to noise, and speed are ultimately required. Besides these applications, another important application is the well-known approach to ﬁnd the distance of an object from a certain reference position. All applications involve accurate positioning of either optical lens or image detector or both. Therefore, accurate control of their position is critical. In most cases, servo actuators are used, such as servo motor, ultrasonic motor, and piezoactuator. Image Blurring Let us consider the underlying concept of the visual autofocusing problem. Image information in a simple camera is shown in Figure 6.73a. As shown, suppose that an object (tree) is placed in front at a distance so from a lens having a focal length f. If the image detector is located at distance si to the right of the lens, a point source p on the object will arrive at p0 on the image plane. At this plane, we can see a clear, inverted image of the tree. As discussed in Chapter 2, “Lenses,” the focused image plane depends upon the

Basic Optomechatronic Functional Units

p object

387 image detector

lens

z f s0

p′

si

∑i

(a) image formation in a convex lens (in-focus) image detector

lens

p

z

f so

si ′

(b) image formation in a convex lens (out-focus)

si

p′ p ′′

Db ∑i

FIGURE 6.73 Defocused image formation of an object.

distance of the object, and the focal length of the lens, f. They are related by the formula given in Equation 2.14 1 1 1 þ ¼ so si f

ð6:76Þ

The equation indicates that, as the image plane moves away from the lens, the object point should move closer to the lens in order to get a clear image. When the image plane moves further, i.e., displaced from Si to S 0 i, as shown in Figure 6.73b, the image will be obtained at p00 but will become defocused at that position of the image plane. In this case, the above relationship (Equation 6.76) does not hold, because the image plane is not in focus, giving rise to a blurred image. Another case of causing blur occurs when the lens moves closer toward the image plane or the object moves towards the lens. In any of two cases, Equation 6.76 does not hold, thereby causing a blur of a ﬁnite size. We can see from the ﬁgure that the point p on the object becomes blurred on the image detector, displaced with a blur circle of diameter Db : According to optics, the blurred image of p00 has the same shape as that of the aperture of the lens, but is scaled by a certain scale factor. In the other method, we can determine blur diameter Db by using geometric and wave optics. Let us consider a simpliﬁed version of Figure 6.73b, in order to discuss the size of

388

Optomechatronics object plane Σ'o Σo δ1 δ2

image plane Σi lens Dbg p'

D ′bg

s

so

FIGURE 6.74 Depth of ﬁeld.

blur. As shown in Figure 6.74, the blur is composed of two contributions, blur due to geometry of the optics and that caused by diffraction limit. When an object is located at the in-focus plane So, its conjugate image point is at a focus point p in the image plane. When it is moved out of So to So0 , the image will be formed in front of the focused image plane Si : Therefore, in the image plane, a blurred image will be formed with a size denoted by mD 0bg where m is the magniﬁcation. In this case, a point on S0o will be projected on a point on So, thus making a projected disc of ﬁnite size. By letting the displaced distance of the object be denoted by d ¼ s 2 so, we can determine the corresponding the blur size mDbg in the image plane. It can be shown that the defocused blur Dbg can be obtained from the geometric relation by Dbg ¼

2mdðNAÞ n

where NA and n are the numerical aperture and refraction of the optics involved, respectively. In addition, the blur due to diffraction at the lens aperture will occur. The diameter of the blur which is equal to Airy disc is then Dbd ¼

1:22lm ðNAÞ

where l is the wavelength of light. Combing the two contributions, the diameter of the resultant blur can be written as Db ¼ Dbg þ Dbd

ð6:77Þ

This is a theoretically determined amount of blur, but in reality, the blur will not be exactly given like this. Therefore, if we wish to determine its exact value, we must resort to an appropriate measurement method. Normally, this is done by evaluating a measure indicative of whether the measure image is in state of focus or not.

Basic Optomechatronic Functional Units

389

Focus Measure The blurred image can be corrected by searching for the lens position that yields the best focused image. There are two methodologies: One is to control either the position of the lens or the position of the detector relative to each other. The other is controlling the camera position while keeping the relative position at a constant distance. Then, the problem is how to control either one of them so that autofocusing can be achieved with high speed and accuracy, so as to yield the best image of an unknown object in the environment with noise. There are two processes needed to achieve autofocusing. The ﬁrst one is to determine a focus measure which reﬂects the degree of focus correctly, as shown in Figure 6.75. The focus value must be simple enough to compute in a fast real-time manner for on-line applications. However, the problem is not so simple, because, in general, image signals contain high frequency components which make the focus value noise-sensitive and scene-dependent. Here, noise includes gray-level noise, excluding the noise coming from the non-front paralleled surfaces. The ﬁgure shows how focusing space is related to the focus measure, and indicates a focus measure as function of lens position. The problem is to ﬁnd the position zf, where the focus measure is maximum within some adjustable range zmim # z # zmax : As discussed before, this range is called “depth of focus.” We will develop an algorithm which ﬁnds the best focusing position lens

search space

zf z

zmin

zmax without noise

focus measure (F)

with noise

0 FIGURE 6.75 Search space and its corresponding focus measure.

lens position (z)

390

Optomechatronics

within the focus range, by instantaneously controlling the distance of the focus lens. To do this, we need to establish a reliable focus measure. There are several measures that can be used for autofocusing. Some of these include; (1) gray level (2) gradient magnitude (3) square of gradient magnitude (Laplacian operator) (4) median ﬁlter. These focus measures are expressed in the form of their total summation or variance total summation : FT ¼

XX m

Fðm; nÞ

n

1 XX ½Fðm; nÞ 2 Fðm; nÞ variance : FV ¼ mn m n

2

ð6:78Þ

where FT denotes the sum of the measure values of Fðm; nÞ deﬁned within the window of interest having of m £ n pixels, FV is the sum of the variance values deﬁned within the window, and Fðm; nÞ is the statistical mean of Fðm; nÞ: In the above, FT and FV values are usually sensitive to noise, and depend upon edge directivity and type of scene. Therefore, the choice of edge operators introduced in Chapter 3, “Image Processing” needs to be carefully made in consideration of image contents. We will deal with four different Fðm; nÞ values to determine FV in the calculations below. (1) FV based on intensity level variance: Let f ðm; nÞ be the pixel intensity value within a window of the image obtained at a certain focusing time. The image is then normalized by subtracting the mean gray value from the gray level of each pixel Fðm; nÞ ¼ f ðm; nÞ 2 fðm; nÞ where fðm; nÞ is a mean gray level value deﬁned in the same window. (2) FV based on the variance of the Sobel operator: If Gmx and Gmy are the Sobel operator with the convolution kernels as previously deﬁned in Chapter 3, “Image Processing,” Fðx; yÞ can be obtained by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ð6:79Þ Fðm; nÞ ¼ {G x p f ðm; nÞ}2 þ {G y p f ðm; nÞ}2 where p denotes the convolution integral of the quantities in parentheses. With this value obtained, the focus measure based upon the Sobel operator is deﬁned as FV, the variance of the whole Fðm; nÞ within m £ n image window. (3) FV based on Laplacian operator: The Laplacian utilizes two second partial derivatives with respect to horizontal and vertical directions. The Fðx; yÞ value is expressed as Fðx; yÞ ¼ 72 f ðx; yÞ ¼

›2 f ðx; yÞ ›2 f ðx; yÞ þ ›x2 ›y2

ð6:80Þ

The operator 72 was given by Equation 3.34 in Chapter 3, “Image Processing,” and its variance measure FV can be computed by Equation 6.78. (4) Variance FV based on weighted median ﬁlter:

Basic Optomechatronic Functional Units

391

It has been pointed out in Chapter 3, in the section “Image Processing” that, in a noisy image, a weighted median ﬁlter can not only extract high frequency components from the edges image but also eliminate impulsive noise. As we might recall, the weighted median ﬁlter is given by qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Wðm; nÞ ¼ w2n þ w2m 1 1 Med{Im23;n ; Im22;n ; Im21;n } þ Med{Im21;n ; Im;n ; Imþ1;n } 4 2 1 2 Med{Imþ1;n ; Imþ2;n ; Imþ3;n } 4 1 1 wm ¼ 2 Med{Im;n23 ; Im;n22 ; Im;n21 } þ Med{Im;n21 ; Im;n ; Im;nþ1 } 4 2 1 2 Med{Im;nþ1 ; Im;nþ2 ; Im;nþ3 } 4 wn ¼ 2

where Med{·} is a median operator. The focus measure in this case is deﬁned by the variance term Fðm; nÞ ¼ Wðm; nÞ

ð6:81Þ

Now, let us consider some practical examples to understand the underlying concept on how the focus measure can be used for autofocusing by evaluating the methods presented above. To investigate this, we need to set up a zooming unit and an autofocusing unit as indicated in Figure 6.76a. It consists of a camera with zoom lenses mounted on a movable stage, an illuminating system, a light source and pattern for a camera calibration, and a beam splitter. By use of this hardware system, a focus measuring system is constructed for autofocusing, as shown in Figure 6.76b. The system is aimed to compute a focus measure, (F) of an image at various locations of the stage. light source object / pattern camera lens CCD

pattern

beam splitter

zoom lens

motor

camera

stage

illuminator

z

frame grabber

microprocessor F value focusing

object

(a) a schematic diagram

of a visual autofocusing system

(b) block diagram of representing the flow of autofocusing

FIGURE 6.76 Finding the relationship between the focus measure and camera position.

392

Optomechatronics

address bus data bus control bus

3 x 8 bit I/O bus

CPU

S/W (manual/auto) manual adjuster

chip set

PCI bus

EPROM

RAM

AGP bus

ROM

I/O Board

power supply motor

motor driver

motion controller

camera motor

frame grabber graphic device

PC

object monitor

stage

linear motion stage

FIGURE 6.77 PC-based controller architecture: autofocusing system & zoom control system.

Figure 6.77 shows the interface between the personal computer and the hardware executing illumination control, zoom control and camera position control. If we look into the interfacing part between the personal computer and the relevant hardware involved with this illumination control system, we can see that there are three major parts. The ﬁrst part is a motion controller that provides command signals to 2-axis servo drivers through a DA converter, and controls the position of the zoom lens and that of the focus lens. The second part is an illumination controller that provides control current to the 3-layered ring illumination system through the DA converter. In digital I/O, 8 bits per ring is allocated to represent the intensity of each ring, in total 24 bits. The third part is frame grabbers, which capture the image of a scene. The rest of the parts include ADC, DAC ﬂag control, and multiplexer.

FIGURE 6.78 Three image patterns used for autofocusing.

Basic Optomechatronic Functional Units

Wx

393

Wx = 150

Wy = 150

Wy

focus measure window

filtered image

Wx

Wy

focus measure calculation

filtered image FIGURE 6.79 The size of window within image frame.

Three different image patterns are considered here, as shown in Figure 6.78a: a stripe pattern, a rectangular block array, and a human face. At each instant of time, the image is acquired by a frame grabber. This acquired image is processed by the edge processing methods. The image is captured by a CCD area camera (640 £ 480). A window size 150 £ 150 within this image is used as illustrated in Figure 6.79, in order to calculate the variance value of F value. This value is used to represent the focus measure at that position. The set of the focus measures is computed utilizing the equations given in Equation 6.78 through Equation 6.81. The focus value is repeatedly determined by varying the camera position incrementally, Dz ¼ 1 mm: Figure 6.80 shows variation of the computed measure with camera position. For the stripe pattern, the weighted median ﬁlter and Sobel operator show much better performance than the Laplacian operator. A similar trend can be seen for the square matrix array. For a more general image like a human face, the Sobel operator has a distinct peak value at a certain lens location, which is proven to be a reliable measure for focusing. The median ﬁlter shows a peak which is not so distinct as compared to that of the Sobel operator. For the Laplacian operator, the value almost does not appreciably change with lens location for all three patterns. This may be attributable to some noise included in the patterns. In summary, for the stripe and matrix patterns, the Sobel operator shows better performance

394

Optomechatronics

20

Sobel

15 10 5 0 –20

WMedian

Laplacian

–10 0 10 distance (z), mm

20

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

(a) stripe pattern

4

16 focus measure

focus measure

25

12

3

Sobel

8

2

WMedian

1

4 Laplacian

0 –20

0 –10 0 10 distance (z), mm

20

(b) matrix pattern 0.8

focus measure

8 6 4 2 0 –20

0.6

WMedian

0.4

Sobel Laplacian

–10 0 10 distance (z), mm

0.2 20

0

(c) human face FIGURE 6.80 Focus measure variation with camera position.

outside the near-focus region, while the weighted median ﬁlter yields better within the in-focus near region. The next phase for executing autofocusing is to ﬁnd the best focused position of the camera in an on-line fashion. In other words, we are interested in ﬁnding the position automatically. To solve this problem, we will employ the hill-climbing algorithm which is frequently used in the optimization method. The algorithm is essentially a search method, which determines the direction of the lens movement in order to search for the maximum value of the focus measure. Figure 6.81 illustrates schematically the concept of the algorithm in detail. The algorithm starts with search mode to determine the direction of the initial search. Let us suppose we can move the camera (lens) incrementally with distance Dz within the range zmin # z # zmax and calculate the corresponding focus value at each location. If the focus lens is assumed to be initially located at z ¼ zi ; where zi denotes the ith location of the focus position, we can compute Fðzi Þ and Fðzi þ Dzi Þ; the focus measure at the initial position and z ¼ zi þ Dz position, respectively. The incremental distance, Dz is called a hopping distance. If Fðzi þ DzÞ . Fðzi Þ; the search direction is the same as increasing Dz: Thus, we need to move the lens to a new position, z ¼ zi þ Dzi : If not, we move it backward, and the new position

Basic Optomechatronic Functional Units

395 z*

focus measure ( F(z) )

zfocus df db

F( zi + Dz ) F( zi )

… zi

zi + Dz

zp-1

zp

zp+1

camera position (z)

FIGURE 6.81 Finding the best focus measure by the hill climbing method.

will be at z ¼ zi 2 Dz: This can be written mathematically as ziþ1 ¼ zi þ Dz sgn{Fðzi þ 1Þ 2 Fðzi Þ}

for

Dz . 0

ð6:82Þ

In this manner, the search operation continues until it reaches around the peak zp < zfocus at a time step tp : At this instant, the F value meets the following condition Fðzp21 Þ , Fðzp Þ . Fðzpþ1 Þ

ð6:83Þ

Because this zp is not an exact peak point, it is necessary to reﬁne the focus position. The reﬁned peak can be determined by " # db 2 df Dz z ¼ zp þ £ 2 db þ df where db ¼ Fðzp Þ 2 Fðzp21 Þ; and df ¼ Fðzp Þ 2 Fðzpþ1 Þ: When the peak estimation is roughly completed at z ¼ zp in the above, hill climbing algorithm is performed again, with a small hopping distance dz for further ﬁne tuning. The search starts with zpþj and, j ¼ 1 and continues until it hits the same condition as expressed in Equation 6.83, with p replaced by j. At this point, the equation is evaluated with dz replacing Dz and zp by zp þ N: N is here the integer value that gives z as an estimated peak. The ﬂow chart of describing this procedure is depicted in Figure 6.82. To practice the hill climbing method, let us assume that a focus measure for a human face reveals a Gaussian distribution with respect to camera position z as follows FðzÞ ¼

1 1 exp 2 ðz 2 zÞ2 ; z ¼ 3:5; s ¼ 1:0 2ps 2s 2

396

Optomechatronics

start i = 0, dz = ∆z

focus measure Fi

zi+1 = zi + dz focus measure F i+1 dz = dz

Fi+1 > Fi no

dz = -1 × dz

no

yes

Fi-1 < Fi > Fi+1 ( i > 1)

yes peak estimation z *focus yes

dz = ∆z no end

FIGURE 6.82 The ﬂow chart of the hill climbing algorithm ﬂow.

which is shown in Figure 6.83. When the camera is initially located at z(0) ¼ z0 ¼ 2 2.6, the problem is to ﬁnd the peak measure, and the best focused position zfocus, using the hill climbing method with the searching parameters Dz ¼ 3:0; dz ¼ 1:0: Let us begin with the initial position in order to ﬁnd the initial value of F value at this location z0 ¼ 22:6 ! Fðz0 Þ ¼

1 1 exp 2 6:12 ¼ 1:32 £ 1029 2p 2

This gives the initial value. We then proceed to obtain the F value at z1 z1 ¼ z0 þ Dz ¼ 0:4 ! Fðz1 Þ ¼

1 1 exp 2 3:12 ¼ 1:30 £ 1023 2p 2

397

F

Basic Optomechatronic Functional Units

0

∆ FIGURE 6.83 Focus measure of human face image vs. camera position.

Since Fðz1 Þ . Fðz0 Þ; update z as z þ Dz z2 ¼ z1 þ Dz ¼ 3:4 ! Fðz2 Þ ¼

1 1 exp 2 0:12 ¼ 1:58 £ 1021 2p 2

Also, Fðz2 Þ . Fðz1 Þ; let us update z as z þ Dz n o 1 z3 ¼ z2 þ Dz ¼ 6:4 ! Fðz3 Þ ¼ exp 2 12 2:92 ¼ 2:37 £ 1023 2p At this time, we meet the peak condition, Fðz1 Þ , Fðz2 Þ . Fðz3 Þ; and make the ﬁrst peak estimation " p

z ¼ z2 þ

db 2 df Dz £ 2 db þ df

#

" ¼ 3:4 þ

db 2 df 3 £ 2 db þ df

# ¼ 3:405135

since db ¼ Fðz2 Þ 2 Fðz1 Þ ¼ 0:157058 and df ¼ Fðz2 Þ 2 Fðz3 Þ ¼ 0:155986, we start again hill climbing method with dz ¼ 1:0 z4 ¼ zp ¼ 3:405135 ! Fðz4 Þ ¼

1 1 exp 2 ð20:095Þ2 ¼ 1:58 £ 1021 2p 2

z5 ¼ z4 þ dz ¼ 4:405135 ! Fðz5 Þ ¼

1 1 exp 2 0:9052 ¼ 1:06 £ 1021 2p 2

Since Fðz5 Þ , Fðz4 Þ; update z as z 2 dz z6 ¼ z5 2 dz ¼ 3:405135 ! Fðz6 Þ ¼ Fðz4 Þ ¼ 1:584 £ 1021

398

Optomechatronics

Since Fðz6 Þ . Fðz5 Þ; update again z as z 2 dz z7 ¼ z6 2 dz ¼ 2:405135 ! Fðz7 Þ ¼

1 1 exp 2 ð21:095Þ2 ¼ 8:74 £ 1022 2p 2

We meet the peak condition again, Fðz5 Þ , Fðz6 Þ . Fðz7 Þ; and make the peak estimation for ﬁne motion: " # " # db 2 df dz db 2 df 21 p ¼ 3:405135 þ ¼ 3:478876 z ¼ z6 þ £ £ 2 db þ df 2 db þ df since db ¼ Fðz6 Þ 2 Fðz5 Þ ¼ 0:052778 and df ¼ Fðz6 Þ 2 Fðz7 Þ ¼ 0:071039: Now, the searching procedure is completed, and the ﬁnal focused position is estimated to be z8 ¼ zfocus ¼ 3:478876: Let us consider an autofocusing problem for a stripe pattern in order to implement the hill climbing method. To carry out autofocusing a system conﬁguration shown in Figure 6.76a is used in which a camera is moving instead of an optical lens. The image is taken by a 640 £ 480 area camera, and the window size is chosen to be 150 £ 150 pixels. Camera motion for searching is given in two different modes: The incremental movement for coarse searching, Dz is 3 mm, and for ﬁne search operation dz is selected to be 1 mm. The variance of gradient magnitude (Sobel operator) value given in

FIGURE 6.84 Finding the focused position using the hill climbing method.

Basic Optomechatronic Functional Units

399

Equation 6.79 is used as the focus measure (F). The procedure to compute FV value at each search position is the same as described previously. In Figure 6.84a, the instantaneous camera position against search time is plotted, while the corresponding FV value is plotted in Figure 6.84b. In Figure 6.84c, the image variation is illustrated for various camera locations z ¼ 19.5, 13.5, 5.4, and 5.9 mm. The effort of focusing starts from z(0) ¼ 19.5 mm. As we can see, when the camera position is moved to the next stop z ¼ 22.5 mm, where FV value decreases slightly to 7. This indicates that searching is made in the opposite direction away from the object. Now, if the searching is made in sequences toward the other direction closer to the object, FV value obtained shows a monotonic increase from the previous value. In this way, searching continues until it reaches a near-maximum value FV ¼ 35 at z ¼ 7 mm. Further increase from this movement toward the object results in a slight decrease in the FV value at z(7). From this moment, the ﬁne tuning method is applied by changing the increment in z from Dz to dz: Finally, the search ends with Fv ¼ 37:5 at z ¼ 5.9 mm. Figure 6.84c clearly depicts the change of the pattern image with a different camera position. As can be seen, the best focused image is obtained at x ¼ 5.9 mm.

Illumination Control A number of visual and optical methodologies, including Moire´, structured light projection, and stereo vision, has been developed to obtain such information as the shape dimension, and the pose of objects, which are needed for various applications. It has been discovered, however, that the optical visual information obtained is vulnerable to external illumination, and the surface reﬂection properties and shape of objects. The image varies according to different reﬂective, absorptive, and transmissive characteristics, depending on material properties and surface characteristics of the objects. Also, they are affected by the directivity of illumination angle and the types of illumination, such as backlight, direct or indirect lighting, which are problem-dependent. Therefore, it is most important to design an illumination system that is appropriate to the object to be inspected. In actual cases, objects have a variety of different characteristics in terms of geometrical shape, surface roughness, and reﬂection properties. This implies that an illumination system must be controllable and adaptable to changes in measuring conditions and environments, and thus needs to be restructured based on the optomechatronic principle. In designing an optical or visual measurement, this principle becomes an important consideration. This is because illumination control has an enhancing effect on accuracy, reliability, and processing time. In this section, we will ﬁrst treat several considerations

400

Optomechatronics

needed for illumination control and various illumination methods. Then, actual methodologies for control of illumination will be treated in brief by illustrating the systems that are capable of adjusting a directivity and intensity of illumination.

Illumination Methods There are several factors that need to be considered for illumination. First, lighting directivity is important to obtain the maximum contrast between features, e.g., between an object and its background. Optical properties of an object surface produce different patterns of radiance, which are diffuse, specular diffuse, and specular. When specularity is unwanted for a detector, then polarization ﬁlers, which employ the principle of polarizing light discussed in Chapter 2, “Aberration,” may be used to reduce the effects of specular reﬂection. Secondly, the spectral distribution of the power from the light source must be taken into consideration for overall sensitivity and discrimination of the system. Figure 6.85 shows the spectral emission of some common illuminators in a vision system. As can be seen from the ﬁgure, an LED light source (for example) has a narrow spectral band. Therefore, it can be effectively used as a light source for illuminating a scene with background light whose spectral content is outside this band. In case of the broad band background light, an optical band pass ﬁlter may be used to detect the scene. Longer wavelengths of infrared light are not much affected by the presence of small particles. Therefore, this light source is particularly effective when objects to be detected are surrounded by small particles such as dust or chemical ingredients. Ultraviolet light can be used to identify the presence of ﬂuorescent materials; for instance, when cracks in objects to be detected are sprayed with ﬂuorescent dye.

relative intensity (I)

1 0.8

fluorescent LED

0.6

halogen light He-Ne laser(632.8nm, ∆l = 10–6nm) laser diode tungsten lamp Si blue sensitive photodiode

0.4 0.2 400

600

800

1000

wave length (λ), nm

FIGURE 6.85 Spectrum of various light illuminators.

1200

1400

Basic Optomechatronic Functional Units

401

Thirdly, the diffusive property and size of the source affect the illumination conditions. Diffuse lighting produces rather uniform intensity over the wide illumination area of interest, while small lighting sources produce sharp shadows. Diffusion of light sources can be easily achieved by placing diffusing glasses such as mylar or opalescent glass in front of the sources. The diffusion material serves to enlarge the effective area of the source. Small sources can be produced by optical components such as mirrors, Fresnel lenses, and ﬁber optics, whose types are dependent upon how they are combined. For instance, to produce a parallel beam, a point light source can be placed at the focal point of a parabolic mirror. To make a focused beam, the source can be put at one focal point of an ellipsoidal mirror. Figure 6.86 depicts various commonly used forms of illumination methods. Figure 6.86a illustrates two types of backlighting. The one on the left is diffuse backlighting, which provides a silhouette image for the target object to be measured and a white image for the background. This can be done by illuminating with respect to the sensing lens. The directional back lighting uses a collimator, which produces all the beams traveling parallel to detector or camera

detector or camera

object

object

lens light source

light source

(a) black lighting detector or camera

incident light

detector or camera diffusing glass

diffusing glass

light source object

(b) directional lighting FIGURE 6.86 Various types of illumination.

object

(c) diffuse front lighting

402

Optomechatronics detector or camera

detector or camera

light source ring

polarizing filter

object

object

(d) fiber optic ring lighting

(e) polarized lighting detector or camera

transparent object

ring

(f) dark field illumination FIGURE 6.86 Continued.

one another. This is effectively used to produce sharply deﬁned shadows on objects, which can be created by collimated light. Figure 6.86b shows a directional lighting whose angle of incidence of the light is arbitrary placed to the object surface. We can see that, as the angle of incidence increases, a larger portion of the incident light is reﬂected from the ﬂat surface of the test plane. In this case, the visible contrast between the test object and the ﬂat test plane gets reduced. Diffuse front light is illustrated in Figure 6.86c. As mentioned before, diffusion of a light source can be provided by either passive or active diffusing glass in front of a light source. Due to the arrangement, this lighting method eliminates shadows and provides uniform illumination conditions. Fiber optic ring-lighting is illustrated in Figure 6.86d. A ring-lighting device consists of several concentric rings having an array of optical ﬁbers within each ring. Due to its conﬁguration, the ring lights reduce specular reﬂections from shiny surfaces and provide shadow-free lighting. This method is very useful for highly reﬂective objects. Polarized lighting is shown in Figure 6.86e. This method reduces specular reﬂections from object surfaces from any angle by using a polarizing ﬁlter placed in front of the camera: Specularly reﬂected light is usually highly polarized, and the polarized components of the light can be greatly attenuated when the polarizer is at right angles to the other. Figure 6.86f shows a dark ﬁeld illumination conﬁguration that can be effectively utilized for many objects not visible by diffuse backlighting. In this method, only the

Basic Optomechatronic Functional Units

403

light scattered or reﬂected by the object will be directed into the optical axis of the camera or detector. Therefore, we can improve image contrast, since, by dark ﬁeld illumination, the edges of a transparent object appear bright against a dark background. Illumination Control Figure 6.87 shows a simple ring-type illumination that can adjust the directivity and diffusivity of light. The directivity control is achieved by switching on the LEDs in an appropriate layer. Clearly, depending upon the LED location, the incident angle of light varies as illustrated for three cases of incident angles in the ﬁgure. Diffusivity of light is controlled by placing a special glass in front of each LED, which changes the degree of its transparency when it gets a certain electric current. Now, suppose that a rectangular object of white color is mounted on a table of black color that effectively absorbs light. Both surfaces are assumed to be lambertian such that they scatter light in a random direction. Under this conﬁguration, there will be a certain incident angle with which the camera can better see an object. From the discussions in Chapter 3, we can see that the incident angle of illumination and the angle of detection are crucial factors to acquisition of an image of good quality. A detailed consideration of this problem is left for exercise at the end of this chapter. With this preliminary knowledge in mind, we will now discuss some of the illumination control systems. Figure 6.88 illustrates an illumination control system, whose conﬁguration is changeable, depending upon the measurement condition. The system can vary the illumination angle to a desired degree by producing a special oblique lighting, and can control intensity as well. The system consists of a lighting unit, an optical mirror system and a unit movable in a vertical direction, carrying the lighting and optical units. The lighting unit has a concentric ring light structure either with LED lights or ﬁber guided lights. The whole ring is essentially a quadrant ring system, which can arbitrarily combine illuminations from four camera ring illuminator

LED object (block) FIGURE 6.87 Three different illumination angles produced by the ring-type illuminator.

404

Optomechatronics CCD camera

motor for rotating turret pattern projector lenses on turret

front

mirror

mirror

quadrant RGB LED ring lights toroidal mirror

parabolic mirror object

(a) optical system configuration

left

lens

quadrant ring l ights

section 1

elevation stage

section 3

section 4

right

toroidal mirror

section 2

4 parabolic mirrors back

(b) bottom view of quadrant RGB ring lights

FIGURE 6.88 A schematic of a variable illumination system.

different directions. By the optical-path guiding system composed of curved mirrors, illumination incident angle is allowed to be selectable as the different rings are illuminated. It is also noted that segmenting a certain part of the rings allows the light to be directional, which is especially suitable for some object features that lie in a particular direction. More segmentation implies more detailed control of the angle of illumination. The optical mirror system composed of four mirrors shown in Figure 6.88a has the role of providing the beam steering that produces a desired angle of incidence. They are drawn here as a separate unit for laser scanning proﬁling, but, for visual measurement, the mirror system has a continuous, smooth surface (such as a parabolic mirror and a toroidal mirror), depending on the illumination applications depicted in Figure 6.88b. The illumination incident angle can also be controlled by lowering or lifting the illumination unit, including lighting together with optical units, as shown in Figure 6.89. Incident angles 308, 608, and 808 are illustrated for better understanding. To generate each condition, it can be seen that the distance between the quadrant ring light with the ﬁxed toroidal mirror and the parabolic mirror is varied from d1, through d2, and d3. It makes the optical rays incident from toroidal mirror to parabolic mirror experience a different incident angle in each case. In this arrangement, if it satisﬁes d1 . d2 . d3, then u1 , u2 , u3, which are the corresponding incident angles to the object. In other words, relatively larger distance makes a larger incident angle to the parabolic mirror, which, in turn, results in a smaller incident angle to the workpiece to be measured.

Basic Optomechatronic Functional Units

quadrant RGB LED ring lights

405

toroidal mirror parabola mirror

d1 d2 d3 30°

60°

(a) using high slope of mirror

80°

(b) using middle slope of mirror

(c) using low slope of mirror

FIGURE 6.89 Illumination angle variation by vertical motion of the optical unit.

The principle of the illumination control discussed in the above example can be applied to the case when higher resolution of an image is required. Consider an optical system that focuses incident light to an object plane, as shown in Figure 6.90. The system is composed of a lighting system (not shown here), a diffraction grating pattern, and a collecting lens. When the pattern is illuminated by a light source as indicated by an arrow, according to diffraction theory, the diffraction angle u is given by sin u ¼ l=w light

light a

grating m = –1

m = +1 m = –1

m=0

q

grating m = +1

q m=0

lens

object plane

(a) a = 0 FIGURE 6.90 Enhancing the image by illumination control.

lens

object plane

(b) a ≠ 0

406

Optomechatronics

light source pattern lens

beam spliter

camera

zoom Zoom

LED illuminations layer 1

motor

stage

layer 2

z

layer 3 object

backlight

variable diffuser

x-y positioning stage

FIGURE 6.91 A ﬂexible vision system with variable illuminators and zooming and autofocusing capabilities.

where l is the wave length of the illuminating light and w is the period of the slit of the pattern. When w is small, the m ¼ ^1 order beam cannot reach the aperture of the lens and only m ¼ 0 order beam will be transmitted, as indicated in Figure 6.90a. This condition deteriorates the image focusing, because the m ¼ þ1 or 2 1 order light beams do not contribute to the image formation. As indicated by the arrow in Figure 6.90b, if illuminating angle a is made oblique, e.g., a – 0, either the m ¼ þ1 or 2 1 order beam will be transmitted through the aperture in addition to m ¼ 0 order beam. In this case, m ¼ þ1 will not be transmitted through the lens. This obviously helps obtain brighter, sharper image quality. This concept is being used for the photomasking process in photolithography. In evaluating illumination quality, it is necessary to take images with a desired magniﬁcation and sharpness. For this purpose, referring to Figure 6.91, a ﬂexible system employs a zoom control system and an autofocusing system, to maximize detector coverage for objects of a wide range of sizes, and to maximize the sharpness of the image as well. The system consists of LED light sources, controllable diffusers, a pattern projector, a zoom control unit, an autofocusing camera, and an x-y stage. The main light source is a three-layered ring illumination made of a number of LEDs. A backlight panel mounted on the x-y stage is employed here as a sub-lighting unit to further control the illumination. A controllable diffuser glass is mounted in front of each LED layer. It is an active glass device whose transparency is electrically adjustable, so that it plays a role of controlling the diffusing level of the light emitted from the source, and of changing the spatial distribution of the light. Zoom control is provided by the control of

Basic Optomechatronic Functional Units

407

40

–40

–60 –60

0

–20

–60

–40

0 –4

0

–2

0

20

–40 –20 0 20 40 Y(mm)

–40

20

X(mm)

20

X( 0 mm –20 )

m Y(

–20

40

0

–2

40

) mm Y(

(b) case 2 : all diffusers off FIGURE 6.92 The distribution of the incident light according to diffuser control.

0

0.25 0.2 0.15 0.1 0.05 0 60

0

–4

40

m)

20

(a) case 1 : all diffusers on

–60

20

40

–40

0

40

20

0 X( mm –20 )

60

60

z (mA)

–40 –20 0 20 40 Y(mm)

0.25 0.2 0.15 0.1 0.05

60

z (mA)

the position of the lens whose unit is included in the camera; the auto focusing is achieved by adjusting the location of the camera with respect to the object. To this end, a motion control is provided for the motor in order to move the camera assembly along the optical axis in such a way that the object of interest can be located within the depth of the focus of the lens optics used at the particular magniﬁcation. Once this is done with the electrical motor system with linear slides and scales to keep track of zooming position, autofocusing is performed to tune the sharpness of images. The autofocusing utilizes a pattern projection method in which a pattern is generated and projected on to an object surface by a LCD projector. The pattern can be differently designed, depending on the type of applications. The role of the x-y table keeps the object of interest within the ﬁeld of view of the camera. Using this conﬁguration, illumination characteristics are varied according to a number of combinations of adjusting light intensity and diffuse levels of intensity of each LED layer. In order to investigate quality of the illumination with this system conﬁguration, the performance of the autofocusing unit is evaluated for various conditions in the LED lighting system. The effect of the diffuser on uniformity of illumination is investigated for a simple object geometry, hemisphere. To do this a hemisphere is placed in the center of the x-y table, and the illumination intensity is measured on a visual inspection plane using a photo detector. Figure 6.92 shows the actual intensity distribution of an incident illumination on a surface according to the

X(mm)

408

Optomechatronics

diffuser

J-lead

J-lead

J-lead

J-lead

back bac k llight i g ht

(a) backlight

(b) direct light (layer 1)

(c) direct + indirect (d) diffused indirect light (layer 1,2,3)

(layer 1,2,3 + diffuser)

FIGURE 6.93 Images obtained by variable lighting conditions.

controllable diffuser. The intensity for case 1 is lower, but its distribution spreads wider than that of case 2. This indicates that when diffusers are on, the light intensity is more uniformly distributed over the space of interest. The effect of LED layers is shown in Figure 6.93. The ﬁgure depicts several images of an electronic chip under different lighting conditions, which are typical illuminations widely used in visual inspection. To detect the boundary of an object, a silhouette image by backlight, as shown in Figure 6.93a, is found to be useful. To detect leads on the chip, on the other hand, LED lights are used. Figures 6.93b to d show the images changed according to the differently applied illumination conditions. Three different conditions for the angles of incidence are applied. When we use only the top layer, it works as coaxial light. Thus, the ﬂat and specular parts of the leads are highlighted in the image, as shown in Figure 6.93b. By using all LED layers together, we obtain a brighter image, shown in Figure 6.93c. However, there are high lighted spots, specular reﬂected, on leads. Finally, the use of the diffused LED lights make LED light uniformly illuminated in all directions, although overall intensities of the resulting image are somewhat reduced, as shown in Figure 6.93d. Illumination Quality Measure As seen from the above illustrations, the adjustment of illumination is problem dependent, for the criterion of the image quality should be changed according to which information is important in the image for a particular purpose. For instance, the body or the leads of a chip could often be utilized for chip recognition. To detect the leads, illumination needs to be adjusted so that the reﬂected light on the body is not imaged, and vice versa. In this

Basic Optomechatronic Functional Units

409

application, adjustment of illumination can be made based on the criterion of how well the obtained image represents the local information of interest. Although there may be several criteria to determine illumination quality, we will consider here one heuristic method that can be used for image segmentation. Let the region R be a region of interest, with m £ n pixels within the image. Using a threshold value, the gray-level image of the region R can be divided into two sets of pixels, which are a set of pixels, A, with higher intensity values than the threshold value, T and the other set of pixels, B, with lower intensity values. This condition is represented mathematically as follows A ¼ {Ii lIi . T for i [ R};

B ¼ {Ii lIi # T for i [ R}

ð6:84Þ

where Ii denotes the intensity value of the ith pixel in the region R, and T is the speciﬁed threshold value. One way to represent the quality of the binarized image obtained in Equation 6.84 is to use the average intensity value and the standard deviation of the pixel intensities in the two regions. The quality index can be expressed by combining the two parameters as r ð6:85Þ uimg ¼ ðIA 2 IB Þ þ sA sB where IA and sA are the average and the standard deviation of the pixel intensities in the set A, respectively, IB and sB are the same quantities related to the set B, and r denotes a scale factor. The ﬁrst term indicates the difference in intensity between the two areas within the image, while the second denotes how uniformly distributed the intensity value of each region is. The threshold value here is determined by an iterative threshold selection algorithm introduced in Chapter 3, “Image Processing,” and the scale factor is set to a desired value depending on which term is of larger importance. It is noted that this quality criterion rates high value when the areas segmented by binarization are far from each other in terms of intensity, and also when the intensity distribution of each region is uniform, having a low standard deviation. In order to see the effectiveness of this evaluation method, illumination quality is investigated for three kinds of chips, capacitor, small ontline integrated circuit (SOIC), and rectangular chip, as shown in Figure 6.94. The purpose of the investigation is to ﬁnd the best illumination condition that can discriminate one feature region from the other within an image. Finding the illumination condition is changed by controlling the conﬁguration of the illumination system, shown in Figure 6.91. For instance, LED-on in layer 1 implies all LEDS in the layer turned on, while diffuser-off of layer 2 implies all diffusers in the layer to be turned off. In this way, various illumination conditions are produced to investigate the quality. Under each condition, the index value given in Equation 6.85 is computed. Figure 6.94 represents the best results obtained from Equation 6.85 when illumination and diffusing units are properly adjusted. The imaging results reveal that in this

410

Optomechatronics

SOIC

capacitor

RC-chip

FIGURE 6.94 Illumination adjustment results.

illumination system each chip has its own conﬁguration of the illumination rendering a high quality image. The condition yielding the best result is listed in Table 6.8. Autofocusing with Illumination Control In the electronics industry, locating, inspecting, and recognizing surfacemount device packages of various sizes and types are ultimately important to obtain good quality PCB. For instance, in particular applications such as microchip assembly, the illumination condition is required to be varied for better image acquisition according to the types of chips under inspection. In such situations, if the autofocusing system is used with the illumination control system, the combined system can signiﬁcantly enhance image quality. To illustrate this example, the integrated system used above is implemented to inspect electronic parts. We will consider a rectangular chip, a plastic-leaded chip carrier (PLCC), and SOIC. Figure 6.95(a) TABLE 6.8 Optimal Illumination Conﬁguration for Chip Inspection Layer 1

Capacitor SOIC Rectangular chip

Layer 2

Layer 3

LED

Diffuser

LED

Diffuser

LED

Diffuser

ON ON ON

OFF OFF ON

ON ON ON

ON ON ON

ON OFF OFF

ON OFF OFF

Basic Optomechatronic Functional Units

411

loading chips

Backlight LED lights LED lights Diffuser

object segmentiaton (number of object = N) k =1 Object 1

autofocusing Autofocusi ng Object 2

illumination adjustment chip Chipinspection inspection if k=N yes

no

k=k+1

Object 3

end

(a) the overall procedure of chip inspection

(b) segmentation , autofocusing, and inspection

FIGURE 6.95 Illumination control system with autofocusing capability.

illustrates the overall procedure of chip recognition, which is composed of four steps as follows: The ﬁrst step is the segmentation of the objects in the camera view, in which only the backlight is utilized. The backlight image is then binarized, and the objects are segmented by a blob analysis. Once the positions and sizes of the objects are identiﬁed through the blob analysis, each region is windowed and processed independently, as shown in the ﬁgure. The second step is adjusting the focus of each chip in each window, under the situation that all of the LED layers are lighted. In this procedure, each chip is independently focused sequentially. The third step is adjusting the LEDs and diffuser units to achieve a high quality image, which varies with the type of chip. In this application, inspection of chips is conducted to obtain the leads information, such as the number, the size, and the pitch of the leads. Since the leads are all made of specular reﬂected materials, illumination is adjusted here so that the leads are dominantly imaged. After the procedure of autofocusing and illumination adjustment is completed for the image, chip inspection is performed as the ﬁnal step. This procedure is sequentially conducted for all objects appearing in the image. The picture shown in Figure 6.95b shows the segmentation and autofocusing results for an image of the three chips. The focused results indicate an image quality satisfactory to ﬁnd the required information on the leads of each chip.

Visual (Optical) Information Feedback Control Visual or optical information is being widely used in most of mechatronic processes or systems. Among its various applications, feedback control of

412

Optomechatronics

visual (optical) information is becoming essential for precision positioning, monitoring and control, inspection, and pattern recognition, as illustrated in Table 6.9. For instance, current robotic systems needed for a variety of placing, manipulating, and tracking of objects or workpieces are also in need of such a technique, but they have inherent limitations in that they operate on an open loop kinematic chain for executing such tasks with respect to the work environment. That is, they cannot utilize accurate information for execution, mainly due to a lack of capability of sensing the position and orientation of the end point of the robotic system with respect to the object. To overcome this problem, visual sensing is being widely used, since it is known to be the most efﬁcient and useful sensing method among all other available sensors developed to date. It mimics the human sense of vision, and provides such characteristics as wide sensing range, fast measurement, and noncontact sensing. Accordingly, for the task execution utilizing robotic systems, visual servoing or feedback has received increasing attention ever since its inception in the early 1980s. The concept of the camera motion-based visual feedback control is illustrated by using a schematic drawing in Figure 6.96. Suppose that we want to grasp an object by using a hand-held camera system. It is assumed that a target (object) image viewed from the camera is given as shown in the ﬁgure. The objective of the visual feedback is to control the motion of the camera system such that the camera image obtained at the current position eventually becomes the target image at the ﬁnal stage. The problem is how to control the camera motion based on the visual information obtained from the camera. In this sense, visual feedback is the fusing technology which combines the results from high speed image processing and mechanical dynamic motion control. A visual feedback system can be categorized into two groups depending on camera conﬁguration, and also two groups depending on control architecture. In camera conﬁguration, these are two modes; the one ﬁxed in the work space and the other mounted on the end effector, as shown in

TABLE 6.9 Practical Applications of Visual Servoing Application area

Role of Visual Servoing

Assembly of macro parts Assembly of MEMs parts Packaging of MEMs parts Object tracking Object manipulation Optical packaging Parts inspection

Parts guide and positioning Parts guide and positioning Part guide and positioning Object recognition and motion tracking Part recognition and grasping/handling Part recognition/alignment Part positioning, recognition

Characteristics Accuracy Accuracy Speed Accuracy Accuracy, speed Accuracy, speed

Basic Optomechatronic Functional Units

413 camera desired camera frame XC

moving

u

YC camera

ZC

current camera frame

ZC

YC

u

v

desired camera image

XC

v

object

current camera image FIGURE 6.96 The camera motion-based visual feedback control.

Figure 6.97a,b, respectively. When a camera is ﬁxed in the workspace as shown in Figure 6.97a, the camera is related to the coordinates system of the robot by R TC and to the object (work piece) C TO : In this case, the relation between the pose of the camera and that of the robot’s end effector is image

image

camera {C}

OC

{E} end-effector

OC

{C} camera

robot{R}

robot {R} {W}

(a) fixed camera system

object

{W}

(b) eye-in hand system

FIGURE 6.97 Two typical system conﬁgurations for visual feedback.

{E} end-effector

object

414

Optomechatronics

unknown and varying with time. Here, E T represents a coordinate’s frame with respect to the frame E, which is shown in the ﬁgure. In the eye-in hand conﬁguration (which is the case of the camera mounted on the robot’s end effector), shown in Figure 6.97b, the relationship between the pose of the camera and that of the end effector is known, and often constant. If this relationship is represented by the pose expressed in the coordinates of the camera relative to the end effector, it can be expressed by E TC ; which indicates the location of the origin of camera frame C with respect to the end effector’s frame E. It is noted here that the target image is independent of the motion of the robotic system. Visual Feedback Control Architectures As schematically shown in Figure 6.98, there are two classes of viewing the control architecture: position-based control and image-based control. This is grouped by the view point at which visual information is utilized for the control. The position-based control uses the information of the object pose with respect to the camera. The pose information is obtained from the image acquired at every instant from the camera. We then relate those values to the pose of the object in the world coordinates by using the coordinate transformation discussed in Chapter 3 in the section of “Camera camera

wX

d

+ _

cartesian controller joint controllers amplifiers

wXˆ

robot

joint angle sensors pose estimation

image feature extraction

video

(a) position-based control camera

+

_

feature space controller joint controllers amplifiers

robot

joint angle sensors image feature extraction

(b) image-based control FIGURE 6.98 Two architectures for visual feedback control method.

video

Basic Optomechatronic Functional Units

start

start

input desired image & extract desired features

input desired pose capture current image from camera extract features of current image estimate current pose

415

capture current image from camera command calculated velocity by control law

extract features of current image

command calculated velocity by control law calculate image Jacobian

calcuate feature errors calculate pose error pose error e < δ (user defined)

no

yes end

(a) position-based visual servoing

feature errors, e < δ (user defined)

no

yes end

(b) image-based visual servoing

FIGURE 6.99 Flow charts for two visual servoing methods.

Calibration”. This transformation is carried out at every instant of control. To do this, we extract object features associated with the geometry of the object from the image. Once the visual pose information denoted by w X^ as shown in the ﬁgure is obtained, this is compared with a desired pose value w Xd to determine an error deﬁned in the pose space (World coordinates). The error thus obtained is fed back to the World controller, which, in turn, actuates the motion of a robotic system. This control action is repeated until the error is eliminated, or reaches the desired value. In contrast to this, the image-based control method uses the prespeciﬁed image features of the object for computing control input without computing the position of the object. This reduces a great deal of computational time and simpliﬁes control methodology. The ﬂow charts shown in the ﬁgure 6.99 clearly compare the two control schemes. Feature parameters of an image in the image plane include point, lines, corners, the area of its projected surface, the periphery of its edge, and so on. The choice of these parameters inﬂuences the accuracy and speed of visual feedback. To illustrate how the visual feedback control can be accomplished in actual practice, we will consider the image-based visual feedback control. In this control, it is necessary to relate the variation of the feature values obtained in the image to the motion of a robotic system (manipulator), in order to generate the control signal at any instant of time. Therefore, determining the velocity of a robotic system is required to move the robotic system in order to obtain the desired value of the feature values f deﬁned in either pixel or image coordinate discussed in Chapter 3, “Camera Calibration.” To this end, let us deﬁne image Jacobian, J deﬁned by ›f/›r,

416

Optomechatronics

which is interpreted as a sensitivity matrix of the feature values, f with respect to position of the end-effector deﬁned in task (world) space, r. This Jacobian can be used to relate the motion of the robotic systems to the variation of feature in camera image with respect to time. The relation can be expressed as df dr ¼ JðrÞ dt dt where: J is a k £ l dimension and given by 3 2 ›f 1 ›f 1 ··· 6 ›r ›r l 7 7 6 1 7 6 6 . ›f ðrÞ .. .. 7 7 JðrÞ ¼ ¼6 . 6 . . . 7 ›r 7 6 7 6 4 ›f k ›f k 5 ··· ›r 1 ›r l

ð6:86Þ

ð6:87Þ

and l and k are the dimension of the task space and that of the feature space, respectively. In this relationship, the rate of feature variation df =dt and the velocity of the robot motion dr=dt are respectively deﬁned by 3 2 du1 6 dt 7 7 6 7 6 6 dv1 7 7 6 6 dt 7 7 6 2 3 7 6 6 . 7 VX 6 .. 7 7 6 7 6 7 6 6 VY 7 6 du 7 7 6 j 7 6 6 7 7 6 6V 7 6 dt 7 dr 6 Z7 df 7; 7 ¼6 ¼6 7 6 6 7 dt 6 dvj 7 dt 6 vX 7 7 6 6 7 6 dt 7 7 6 7 6 6 vY 7 7 6 5 4 6 . 7 6 .. 7 vZ 7 6 7 6 6 du 7 k 7 6 7 6 6 dt 7 7 6 4 dv 5 k

dt

It is noted here that the feature velocity vector is speciﬁed for the number of the feature points, k/2, and dr=dt denotes the translational and rotational velocities of the end effectors with respect to camera coordinates. The relationship expressed in Equation 6.87 indicates that, if the change in the pose of the robotic system is given, the change in the image feature can be obtained accordingly, and vice versa. For the visual feedback control, therefore, we need to determine the rate of the motion dr=dt at every instant,

Basic Optomechatronic Functional Units

417

camera OC C C

WE

end-effector {E} O E robot {R}

W C

Pj

point P C

E

VE

camera {C}

fixed frame

PE

{C} W

j

WC

C

Pj

PC

robot {R}

Pj

OC

W

VC point P j

{W}

{W}

(a) coordinates systems for fixed camera

object

(b) coordinates systems for the eye-in hand system

FIGURE 6.100 Coordinate system for the visual feedback control.

so as to make the current feature values rapidly approach the desired value of the given feature. To illustrate the implementation of this underlying concept, we will consider two camera conﬁgurations: (1) ﬁxed camera system. (2) eye-inhand system. We will deal with the ﬁxed camera system ﬁrst. Fixed Camera Conﬁguration Let us suppose that the end-effector is moving with transnational velocity V E and angular velocity C VE which are deﬁned with respect to the camera frame. Referring to Figure 6.100a, they are described by C

C

V E ¼ ½VX ; VY ; VZ

and

C

VE ¼ ½vX ; vY ; vZ

Let Pj be a point attached to the end-effector frame expressed by C

Pj ¼ ½C Xj ; C Yj ; C Zj

T

¼ ½XC ; YC ; ZC

T

From the ﬁgure, it can be seen that the displacement vector C Pj with respect to camera frame is given by C

Pj ¼ C PE þ C TE E Pj

ð6:88Þ

where E Pj denotes the displacement vector of point Pj with respect to endeffector frame {E}, C TE is the translation matrix of the end-effector frame {E} with respect to camera frame {C}, and C PE is the displacement vector of the origin OE of end-effector frame {E} with respect to camera frame {C}. Differentiating both sides of Equation 6.88, we have dC P j ¼ C V E þ C VE £ C Pj dt where C VE is the rotation of the end effector with respect to the camera coordinate.

418

Optomechatronics

This relationship enables us to obtain the translational velocity of a point at the end-effector with respect to the camera coordinate system which is given by 3 2 dXC 3 6 dt 7 2 7 6 vY ZC 2 vZ YC þ VX 7 6 6 dYC 7 6 7 7 ¼ 6 vZ XC 2 vX ZC þ VY 7 6 ð6:89Þ 7 4 6 5 6 dt 7 7 6 vX YC 2 vY XC þ VZ 4 dZ 5 C dt These values will now need to be related to those measured in the image coordinate systems. To derive the relation, we will consider the coordinate systems shown in Figure 6.101a. Suppose that a point M is positioned in the camera coordinates, as shown in the ﬁgure. And let a point m positioned at the coordinates ðui ; vi Þ be the corresponding projected point of M in the image coordinates. In Chapter 3, “Camera Calibration” we have assumed the image coordinates to camera coordinates YC

XC

(CR E,CPE)

vi

f

XE OC

ZE

ui

Oi

ZC

m

OE

M(Xc,Yc,Zc)

end-effector coordinates YE

(a) end-effector coordinates viewed from camera coordinates vp

vi m

v0

oi

θ op

u0

image coordinates ui pixel coordinates up

(b) image vs. normal coordinates FIGURE 6.101 Relating the image motion to the end-effector motion.

Basic Optomechatronic Functional Units

419

be orthogonal in deriving the relationship between the pixel coordinate and image coordinates are frames. But, in more general cases of CCD pixels, since the pixel coordinates are skewed due to fabrication limitation, they are often not orthogonal each other. The pixel coordinates ðup ; vp Þ are skewed with angle u; and are related to the normalized image coordinates system expressed by ðui ; vi Þ at a point Oi, as shown in Figure 6.101b. From the geometric relationship between the two, the transformation equation was determined by u~ p ¼ ½Hu~ u~ i

ð6:90Þ

where u~ p is the augmented pixel coordinate frame, u~ i is the augmented image coordinate frame, and ½Hu~ is the transformation matrix, which is slightly different from the ½Hu~ deﬁned in Equation 3.53. If ½Hu~ is substituted, Equation 6.90 becomes 32 3 2 3 2 ku 2ku cot u u0 ui up 76 7 6 7 6 6 7 6 vp 7 ¼ 6 0 ð6:91Þ kv =sin u v0 7 5 4 vi 5 4 5 4 1

0

0

1

1

where ku and kv are the scaling factors given in Chapter 3, “Perspective Projection.” To express the relationship in the camera frame in term of the normalized image coordinates, we need to use the perspective projection equations as previously deﬁned in the same section mentioned above, " # " # ui XC f ¼ ð6:92Þ ZC YC vi where f is the focal length of the objective lens of the camera, as shown in the ﬁgure. Substituting Equation 6.92 into Equation 6.89, we obtain 3 2 3 2 vi ZC dXC v v Z 2 þ V Y C Z X 7 f 6 dt 7 6 7 6 7 6 7 6 7 6 7 6 dYC 7 6 ui ZC 7 6 7¼6 v 2 v Z þ V ð6:93Þ 6 Z X C Y 7 6 7 6 f 7 6 dt 7 6 7 6 7 7 4 dZ 5 6 4 ZC 5 C ðvi vX 2 ui vY Þ þ VZ dt f This means that once an image is deﬁned and acquired in the image plane, it can be easily expressed in the normalized coordinates. If we use the following relationships ZC

dXC dZC 2 XC dt dt Z2C

dui d X ¼ f C dt dt ZC

¼f

dvi d Y ¼ f C dx dt ZC

dYC dZC ZC 2 YC dt dt ¼f Z2C

and ð6:94Þ

420

Optomechatronics

the motion of the point m in image coordinates is expressed in terms of the motion variable, C V E and C VE of point M deﬁned in camera coordinates system. Substituting Equation 6.94 into Equation 6.93 we obtain the following equation 3 2 VX 7 36 6 VY 7 3 2 f 2 2 2 7 6 u u v f þ u i dui 7 0 2 i 2 i i 2vi 76 7 6 6 dt 7 6 V f ZC ZC f 76 Z 7 7 6 6 7 ð6:95Þ 7 6 6 7¼ 6 7 76 4 dvi 5 6 56 vX 7 4 f vi f 2 þ v2i ui vi 7 6 0 2 2 ui 6 7 dt ZC ZC f f 6 vY 7 5 4

vZ This relation implies that if the rate of the feature vector motion is known, the motion of the end effector can be speciﬁed with respect to camera coordinates, and vice versa. Eye-In-Hand Conﬁguration Suppose that a camera mounted on the end-effector moves with the velocities W

VC ¼ ½vX ; vY ; vZ

W

V C ¼ ½VX ; VY ; VZ

where W VC and W V C are deﬁned, respectively, by the angular velocity and transnational velocity of the camera/end-effector with respect to the world coordinate in task space. Let Pj be a point attached to the end-effector. The velocity of the point Pj with respect to the camera frame is expressed by dC Pj ¼ 2W V C 2 W VC £ C Pj dt

ð6:96Þ

In a way similar to that used for the case of the ﬁxed camera system, we obtain the following relationship between the velocity of the camera motion and the feature change obtained in image coordinates 3 2 VX 7 36 6 VY 7 3 2 2 2 2 7 6 f u u v 2f 2 u i i i i dui 7 2 0 vi 7 6 7 6 6 dt 7 6 V ZC ZC f f 76 Z 7 7 6 6 7 7 ð6:97Þ 6 6 7¼ 6 76 7 4 dvi 5 6 56 vX 7 4 f vi f 2 þ v2i ui vi 7 6 2 2ui 6 0 2 7 dt ZC ZC f f 6 vY 7 5 4

vZ

Basic Optomechatronic Functional Units

421

Up to this point, we considered only a single feature point for visual feedback control. In actual cases, we need to use many different feature points to characterize the object to be handled. The above expression can now be extended to more general cases. If the number of feature points is k/2, the above relationships become: 3 f u1 u1 v1 f 2 þ u21 2 0 2 v 6 du1 1 7 72 3 6 ZC1 ZC1 f f 7 V 6 dt 7 6 X 7 7 6 6 76 7 7 6 6 2 2 76 7 7 6 6 f v f þ v u v 1 1 1 1 6 dv1 7 6 0 2 2 2u1 7 VY 7 76 7 6 6 7 ZC1 ZC1 f f 76 6 dt 7 6 6 76 7 7 6 6 76 VZ 7 7 6 6 7 6 . 7 6 .. .. .. .. .. .. 7 6 7 6 . 7¼6 7 . . . . . . 7 76 6 . 7 6 7 76 7 6 6 6 vX 7 7 7 6 6 6 7 2 2 7 6 duk=2 7 6 f þ u 7 6 uk=2 uk=2 vk=2 f 7 7 6 6 k=2 7 7 62 6 2 vk=2 76 0 v 6 Y 74 7 6 dt 7 6 ZC;k=2 ZC;k=2 f f 5 7 7 6 6 7 7 6 6 7 vZ 4 dvk=2 5 6 7 6 vk=2 f 2 þ v2k=2 uk=2 vk=2 f 5 4 0 2 2 2uk=2 dt f f ZC;k=2 ZC;k=2 2

3

2

ð6:98Þ for the eye-in-hand conﬁguration. This derivation completes relating the camera motion velocity to the feature change in image. Feedback Controller Design Now with the known relationship between the motion velocity and the image feature change, we can determine the end-effector velocity at any control instant that can lead to reaching a desired image feature vector. Let the velocity vector in the right-hand side of Equation 6.98 be denoted by u, which is regarded to be the control command inputs to the motion controller of joint servo motors. Assuming that there exists a pseudo inverse J þ, u can be written from Equation 6.98 by u ¼ Jþ ðrÞ

df dt

ð6:99Þ

where Jþ is given by Jþ ¼ ðJT JÞ21 JT

ð6:100Þ

and f represents the feature points in the image coordinate. If the controller employs a proportional control law, for simplicity here, u in Equation 6.99 can be expressed by u ¼ K p Jþ ðrÞð f d 2 f Þ

ð6:101Þ

422

Optomechatronics

where f d is the desired value of f, f d 2 f denotes the error in features, and K p is the proportional controller gain. Here, we will concentrate on point features for visual servoing. The remaining part of this section will be devoted to conﬁrming the above visual feedback concept by illustrating how to solve actual problems. To this end, let us consider a visual feedback system for servoing a camera towards a square block shown in Figure 6.102a. The right hand side of the ﬁgure shows the current image of the four feature points of an object obtained at a certain camera location, while the ﬁnal image of the corresponding feature obtained at another camera location is shown in the left-hand side of the ﬁgure. The feature points here denote the characteristic point that can best represent the pose of the object to be tracked. At the desired location, we will assume that the camera sees the feature points ð f1 ; f2 ; f3 ; f4 Þ deﬁned in the desired camera coordinate systems {D} as depicted in the ﬁgure. Notice that the image plane is denoted by the pixel coordinates ðup ; vp Þ; and that ðu0 ; v0 Þ is desired camera frame

u v

Ycr

{D} f3d

f4d

XD YD

f2d

current camera frame {cr} u Xcr f3c

ZD

f1d Zw

desired camera image

v

Zcr

f2c

Yw

f3 f2

f1

Xw {W} feature point

camera camera frame

XD

YD

up

virtual image plane vp

{D}

(u0, v0 )

f3

f2

f4 ZD

f1

(b) feature points defined in the camera coordinate system

FIGURE 6.102 Visual feedback control for a square block.

f1c

current camera image

f4

(a) current image vs. desired image

f4c

{W}

Basic Optomechatronic Functional Units

423

the center point of the image coordinates. The problem is to ﬁnd a control input signal which drives the joint motors holding the camera module so as to locate the camera at the desired position. The camera intrinsic parameters and the feature points are given, respectively, by intrinsic parameters: f ¼ 0:016; av ¼ 1955; av ¼ 1940; u0 ¼ 320; v0 ¼ 240; u ¼ 908 desired feature points in world coordinate frame: f1 ðX1 ; Y1 ; Z1 Þ ¼ ½ 0:015 20:015 0 T ;

f2 ðX2 ; Y2 ; Z2 Þ ¼ ½ 20:015 20:015 0 T ;

f3 ðX3 ; Y3 ; Z3 Þ ¼ ½ 20:015 0:015 0 T ;

f4 ðX4 ; Y4 ; Z4 Þ ¼ ½ 0:015 0:015 0

T

In addition, transformation of the current camera frame with respect to the desired camera frame, that is, the two coordinate frames will be assumed to be at some control instant related by a rotation Rða; b; gÞ which is given by 32 3 2 32 cosa 2sina 0 1 0 0 cosb 0 sinb 6 76 7 76 6 7 76 Rða; b; gÞ ¼ 6 1 0 7 54 0 cosg 2sing 5 ð6:102Þ 4 sina cosa 0 54 0 0

0

1

2sinb 0 cosb

0 sing

cosg

For roll: g ¼ 28; pitch: b ¼ 38; yaw: a ¼ 158; and translation T ¼ ½0:01; 0:01; 20:03 : The transformation between two coordinate frames is determined by 2 3 0:9646 20:2569 0:0596 0:01 6 7 6 0:2585 0:9658 20:0202 0:01 7 6 7 D 7 Tcr ¼ 6 ð6:103Þ 6 7 6 20:0523 0:0349 7 0:9980 20:03 4 5 0 0 0 1 In actual servoing, this information is determined at every instant of time. According to the coordinates shown in the ﬁgure, transformation of the world coordinate frame with respect to the desired camera frame gives 2

1

0

0

0

3

6 7 6 0 21 0 7 0 6 7 D 7 TW ¼ 6 6 7 6 0 0 21 0:3 7 4 5 0 0 0 1 It is noted that the ZD axis (optical axis) of the desired frame coincides with the zw axis of the world frame, facing against each other. Given the condition in the above formula, we need to transform the feature points viewed by the desired frame and by the current frame into those seen in the image coordinate frame in order to determine the Jacobian given in Equation 6.87. To do this, we need to follow the following procedures step by step.

424

Optomechatronics

(1) Determine the camera perspective matrix: In the case of the skewed pixel coordinate system, C matrix in the perspective model given in Equation 3.55 is modiﬁed as 2

au

6 6 ½C ¼ 6 0 4

0

au cot u u0 av v0 sin u 0 1

0

3

7 07 7 5

ð6:104Þ

0

where u is the skew angle. Plugging the given parameter values into the above Equation 6.104 yields, we have 2

1955

6 ½C ¼ 6 4 0

0

320

1940

240

0

1

0

0

3

7 07 5 0

(2) Determine the coordinate values of the desired image feature points in the pixel coordinates: Let the values be denoted by ½p1d ; p2d ; p3d ; p4d . Utilizing the above perspective matrix obtained and the homogeneous transform matrix of the world coordinate frame with respect to the desired camera frame, DTW, we can compute the coordinate values of the desired frame in the pixel coordinates in the following manner s1 p1d ¼ CD TW f1 ¼ CD TW ½ 0:015 s1 p1d ¼ s1 ½pT1d l1

T

¼ ½ 125:32

20:015

101:1

0:3

0

1

T

¼ ½ 125:32

101:1

T

0:3

T

s1 ¼ 0:3 where s1 ¼ Zc is the scale factor in homogeneous coordinates. This leads to obtaining p1d ¼ ½ 417:75

337

T

Similarly, s2 p2d ¼ CD TW f2 ¼ CD TW ½ 20:015 s2 p2d ¼ s2 ½pT2d l1

T

¼ ½ 66:675

101:1

20:015 0:3

0

1

T

T

¼ ½ 66:675

¼ ½ 66:675

101:1

0:3

T

s2 ¼ 0:3 s3 p3d ¼ CD TW f3 ¼ CD TW ½ 20:015 0:015

0

1

42:9

0:3

T

T

Basic Optomechatronic Functional Units

s3 p3d ¼ s3 ½pT3d l1

T

¼ ½ 66:675 42:9

425 T

0:3

s3 ¼ 0:3 s4 p4d ¼ CD TW f4 ¼ CD TW ½ 0:015 s4 p4d ¼ s4 ½pT4d l1

T

0:015

¼ ½ 125:32 42:9

0 1

T

¼ ½ 125:32

42:9

0:3

T

T

0:3

s4 ¼ 0:3 Therefore, the feature points in the desired image are expressed in the pixel corrdinates by p1d ¼ ½ 417:75

337 T ; p2d ¼ ½ 222:25

337 T ;

p3d ¼ ½ 222:25

143 T ; p4d ¼ ½ 417:75

143

T

(3) In order to determine the Jacobian matrix, we further need to transform the feature points f deﬁned in the world frame into the pixel points in the current image frame by using the homogeneous transformation between the desired camera frame and the current camera frame given in above Equation 6.103. If the transformation from the desired to the current frame 21 , four corresponding points expressed in the current is denoted by DTcr camera frame can be obtained. Since, 21 D s01 p1c ¼ CD Tcr TW f 1 21 D ¼ CD Tcr TW ½0:015 20:015 0 1

¼ C½0:0112 0:0151 0:330 1 ¼ ½ 83:6 108:3 0:330

T

T

T

Hence, p1c ¼ ½ 253:75 328:65 T : Similarly, 21 D s02 p2c ¼ CD Tcr TW f 2 21 D ¼ CD Tcr TW ½20:015 20:015 0 1

¼ ½ 26:48 122:83 0:328 p2c ¼ ½ 80:78 374:75

T

T

T

¼ C½20:0401 0:0228 0:328 1

426

Optomechatronics

21 D s03 p3c ¼ CD Tcr TW f3 21 D ¼ CD Tcr TW ½20:015 0:015 0 1

¼ ½11:513 66:766 0:328 p3c ¼ ½35:06 203:33

T

¼ C½20:0479 20:00621 0:328 1

T

T

21 D s04 p4c ¼ CD Tcr TW f4 21 D ¼ CD Tcr TW ½0:015 0:015 20:3 1

¼ ½68:657 52:242 0:330 p4c ¼ ½207:95 153:23

T

¼ C½20:0189 20:0139 0:330 1

T

T

In summary, in terms of the pixel coordinates the feature points can be expressed as p1c ¼ ½ 253:75 328:65 T ;

p2c ¼ ½80:78 374:75

T

p3c ¼ ½ 35:06 203:33 T ;

p4c ¼ ½207:95 153:23

T

And the depth of ﬁeld of each feature point deﬁned in the current frame is Z1 ¼ 0:330;

Z2 ¼ 0:328;

Z3 ¼ 0:328;

Z4 ¼ 0:330

Finally, these feature points need to be expressed in terms of the image coordinate frame, because the image Jacobian is expressed in that frame. They are obtained as ½u1c v1c ¼ ½ 20:542 £ 1023 0:731 £ 1023 ; ½u2c v2c ¼ ½ 22:0 £ 1023 1:1 £ 1023 ; ½u3c v3c ¼ ½ 22:3 £ 1023 0:3 £ 1023 ; ½u4c v4c ¼ ½ 20:917 £ 1023 20:716 £ 1023

Basic Optomechatronic Functional Units

427

Therefore, we ﬁnally obtain J in Equation 6.97, which can be calculated by 2

f 0 6 6 Z1 6 6 6 f 6 0 6 6 Z 1 6 6 6 f 6 0 6 6 Z2 6 6 6 f 6 6 0 6 Z 2 6 [J ¼ 6 6 f 6 6 6 Z3 0 6 6 6 f 6 6 0 6 Z 3 6 6 6 6 f 6 6 Z4 0 6 6 6 f 4 0 Z4 2

u 2 1c Z1

u v 2 1c 1c f

2

v1c f 2 þ v21c 2 f Z1

2

u2c Z2

2

v2c f 2 þ v22c 2 Z2 f

2

u3c Z3

2

v3c f 2 þ v23c 2 Z3 f

2

u4c Z4

2

v4c f 2 þ v24c 2 Z4 f

2

2

2

u2c v2c f

u3c v3c f

u4c v4c f

3 f 2 þ u21c 2v1c 7 7 f 7 7 7 u1c v1c u1c 7 7 7 f 7 7 2 2 7 f þ u2c 7 2v2c 7 7 f 7 7 7 u2c v2c 7 u2c 7 7 f 7 7 7 2 2 f þ u3c 7 2v3c 7 7 f 7 7 7 u3c v3c 7 u3c 7 7 f 7 7 7 2 2 f þ u4c 7 2v4c 7 7 f 7 7 7 u4c v4c 5 u4c f

4:85 £ 1022

0

1:64 £ 1023

0

4:85 £ 1022

2:17 £ 1023

6 6 6 0 4:85 £ 1022 22:22 £ 1023 6 6 6 4:89 £ 1022 0 6:10 £ 1023 6 6 6 0 4:89 £ 1022 23:35 £ 1023 6 ¼6 6 6 4:89 £ 1022 0 7:01 £ 1023 6 6 6 0 4:89 £ 1022 9:15 £ 1024 6 6 6 6 4:85 £ 1022 0 2:80 £ 1023 4

2:48 £ 1025

1:60 £ 1022

27:31 £ 1024

3

7 7 21:60 £ 1022 22:48 £ 1025 25:42 £ 1024 7 7 7 1:38 £ 1024 1:63 £ 1022 21:10 £ 1023 7 7 7 22 24 23 7 21:61 £ 10 21:38 £ 10 22:00 £ 10 7 7 7 24:31 £ 1025 1:63 £ 1022 3:00 £ 1024 7 7 7 21:60 £ 1022 4:31 £ 1025 22:3 £ 1023 7 7 7 25 22 24 7 7 24:10 £ 10 1:61 £ 10 7:16 £ 10 5 22 25 24 21:60 £ 10 4:10 £ 10 29:17 £ 10

If we deﬁne error by e ¼ f d 2 f in the image coordinates, determine the velocity command inputs to the six axis motion controllers at this particular servoing instant. To compute the error values we need to express the features obtained in the pixel coordinates in terms of the image coordinate values. These are obtained by ½ u1d

v1d ¼ ½ 8:0 £ 1024

½ u2d

v2d ¼ ½ 28:0 £ 1024

8:0 £ 1024 ; 8:0 £ 1024

428

Optomechatronics

½ u3d

v3d ¼ ½ 28:0 £ 1024

½ u4d

v4d ¼ ½ 8:0 £ 1024

28:0 £ 1024 ; 28:0 £ 1024

Once these feature values deﬁned in the image coordinates are obtained, we can determine the errors in the feature values, by 2 3 u1d 2 u1c 6 7 6 7 .. e ¼ fd 2 f ¼ 6 7 . 4 5 v4d 2 v4c Then the velocity command inputs are computed from Equation 6.101 u ¼ K p Jþ e ¼ K p ðJT JÞ21 JT e ¼ ½ 0:267

0:135

0:00309

0:394 20:711

0:312

T

which completes the computation of the control inputs that are required to drive the six-motor joint mechanisms.

Optical Signal Transmission As mentioned previously, the advantage of ﬁber optic transmission is that it provides rapid transmission of data and immunity to noise . In addition, optical ﬁbers are compact and ﬂexible. For this reason, optical data transmission is widely used when data/signals obtained from sensors area subject to external noise, when the amount of data needed to be sent is vast, or when an operation is operated at remote sites. Operation of systems at a remote site is ubiquitous, nowadays. Remote operation of a machine, visual servoing of a robot operated in a remote site, internet-based monitoring, inspection, and control are some of such developments. Signal Transmission A transmission system utilizes several techniques of modulation in transmitting optical signal through a ﬁber. Figure 6.103 shows a schematic of the intensity modulation based system. It consists of an LED(s) for emitting a light signal of data, and an LED(r) for emitting another light signal of reference for correcting errors occurring during transmission. In addition, it has a pair of current drives for two LEDs, and an optical multiplexer. A receiver has a photodetector (PD(s)) for detecting a light signal of data, a photodetector (PD(r)) for the light signal of reference, a pair of current ampliﬁers for two photo detectors, an optical demultiplexer, and an analog multiplexer.

Basic Optomechatronic Functional Units

429

transmitter Vi

current drive

LED(s)

receiver

optical multiplexer Vr

current drive

LED(r)

Pi(s)

Po(s)

Po(r)

optical fiber

PD(s)

optical demultiplexer Pi(r)

PD(r)

current amplifier

Vo(s)

analog multiplexer current amplifier

Vo

Vo(r)

FIGURE 6.103 Conﬁguration of the optical ﬁber transmission system.

The transceiver system simultaneously transmits the light signal of the data and another signal of the reference from the transmitter to the receiver. In the transmitter, the voltages Vi and Vr the input and reference voltages, respectively, are converted to the forward currents of LED(s) and LED(r) by their respective current drives. As a result, the optical output powers are generated from the LEDs, which, in turn, are multiplexed by the optical multiplexer. At the receiver side, the signals and reference lights whose powers are P(s) and P(r), respectively, are demultiplexed through the optical demultiplexer, then fed to PD(s) and PD(r) respectively. The photocurrents of the diodes are converted to electrical voltages Vo ðsÞ and Vo ðrÞ through the current ampliﬁers, respectively, which can be made in proportion to the P(s) and P(r) values. An optical transceiver containing the transmitter and receiver that can perform the operation explained in the above signal transmission is shown in Figure 6.104. Its role is to launch an optical signal emitted from an LD into the optical ﬁber and to receive transmitted signal from the ﬁber by a PD. It consists of a laser diode(LD), photodiode beam spliter and three lenses. The three lenses are used to collimate or focus the laser beam, while the beam splitter is used to couple lenses and other optical devices from or to optical ﬁber. It can be seen that the laser beam from the LD to the ﬁber and the transmitted signal input to the PD from the ﬁber pass through the beam splitter based on confocal scheme. Laser diode (LD) package has a laser diode, an embedded photodiode, and two lenses. In this package, the optical power is controlled by a feedback loop which utilizes the power measurement out of the photodiode. In the transmission system shown in Figure 6.103, Vo needs to be equal to Vi by adjusting the gains of the current drives and ampliﬁers when the optical ﬁber is kept straight. However, when the ﬁber is bent, their relationship is found to be given by Vo ¼

T1 ðl1 ; r1 ; u1 Þ V T2 ðl2 ; r2 ; u2 Þ i

where T1 and T2 are the transmittances of the optical powers passing through a pair of optical powers at wavelength l1 and l2 ; respectively. The transmittances are found to be also dependent upon the bending radius r and

430

Optomechatronics

beam splitter

LD laser diode package

optical fiber PD

lens

lens lens PD

FIGURE 6.104 Conﬁguration of the optical transmitter.

bending angle, u. At the optical connector, coupling loss should be minimized, and distortion of optical signal should also be minimized during the transmission. Figure 6.105 illustrates a full-duplex ﬁber optic transmission link for a remote operation of a group of machines (robots) controlled by several servo motors. The system carries feed-forward and feedback digital signals over a distance of many meters between the control site and operation site. The control site executes a control algorithm with controller gain dedicated to each motor. At the remote site, this control command is executed via motor power switching circuits, D/A buffer circuits for feedback and feedforward control, and A/D buffer circuits for feedback signals from the angular displacement sensor. Each motor control circuit in the control site is connected to a control computer which provides scheduling of the machine operation, programs the controller gains, and provides the high-level control commands. The inputs to each control circuit from the computer are the sign of the commanded motor current and pulse-width modulation representing the magnitude of the command motor current. This information is sent out through the transmitters, the optical ﬁbers, and

sign magnitude channel 1

channel n

control site

bidirectional interface circuit

transmitter

receiver

receiver

transmitter

machine (robot) sign magnitude channel 1 …

…

central control computer

digital motor control circuits

bidirectional interface circuit

channel n optical link

FIGURE 6.105 An optical transmission link for remote operation.

operation site

power and buffer circuits … channel 1 channel n

motor

…

Basic Optomechatronic Functional Units

431

the receivers, and then ﬁnally input to the power and buffer circuits. The ﬁber optic link has two optical ﬁbers for each motor control; one for feedforward, one for feedback. At both ends, they are connected to identical bidirectional interface circuit boards, each of which contains a transmitter and a receiver. The transceiver A is connected to transceiver B through an optical ﬁber (for example, a few hundred meters core diameter 50 mm and cladding diameter 125 mm). Power Transmission and Detection In use of optical ﬁbers, we need to consider the efﬁciency of the coupling between optical source and ﬁber, since the transmitted power is affected by the focusing system and the geometry of coupling. Figure 6.106 illustrates two cases of coupling; from source to ﬁber, and from ﬁber to detector. Therefore, two coupling efﬁciencies will be taken into consideration. Here, it will be assumed that an optical system focuses the light beam in such a way that the incident angle is equal to half the cone of acceptance of the ﬁber, which ensures internal reﬂection. When the coupling of source to ﬁber shown in Figure 6.106a is considered, it is possible to compute an approximate value of the coupling coefﬁcient Csf. The coefﬁcient is deﬁned as W ðlÞ Csf ¼ i ð6:105Þ SðlÞ where Wi ðlÞ is the power incident on the entire surface of the ﬁber per unit wave length, and SðlÞ represents the power emitted from the source per unit wave length at l[7]. The geometry of rays incident to the ﬁber indicates that rays with the cone of acceptance will be transmitted and internally refracted through the core, if Rco is smaller than the source radius Rs : In other words, the Rs is essentially smaller than the aperture of the focusing lens. The effective source area becomes Aeff ¼ pR2a ¼ pd2i tan2 ½ui

max

collecting lens

from source Rs

[q i]max

coupling unit detector fiber

fiber Rco

RD

Rco

[q i]max di

(a) coupling between source and fiber FIGURE 6.106 Coupling of source-ﬁber and ﬁber detector.

[q i]max

(b) coupling between fiber and detector

432

Optomechatronics

where Ra is the effective aperture of the lens, di is the distance between lens and ﬁber, equal to f , and ½ui max is the maximum angle of acceptance. Then, Wi ðlÞ is expressed by Wi ðlÞ ¼ Aeff DuSðlÞ < p2 R2co tan2 ½ui

max SðlÞ

ð6:106Þ

where Du is the solid angle which the ﬁber core sustends. From Equation 6.106, the coupling coefﬁcient is obtained approximately by Csf ¼

Wi ðlÞ < p2 R2co ðNAÞ2 SðlÞ

ð6:107Þ

where NA ¼ sin½ui max . The coupling of ﬁber to detector can be obtained in a similar way; the coefﬁcient CfD is simply CfD ¼

WD ðlÞ ¼ 1 for RD . Rco Wf ðlÞ

ð6:108Þ

where RD is the radius of the detector, and Wf ðlÞ and WD ðlÞ are the power leaving the optical ﬁber and the power transmitted to the detector. In this situation, all the power leaving the ﬁber transmits through the optical unit of the detector. In case RD , Rco, the coefﬁcient can also be computed in the same way as obtained in the case of source-to-ﬁber transmission. Light transmitted by a ﬁber attenuates optical power, and also loses some part of information due to several distortion mechanisms. They include model distortion, material distortion, and waveguide dispersion. The modal distortion occurs because rays take many different travel paths, thus reaching the output end in different times. When an input of a square wave is sent into a ﬁber, this distortion mechanism causes broadening of pulse. Material distortion occurs, because the refractive index is a function of wave length. There are some ﬁbers that exhibit this phenomenon. Light normally contains several wavelengths, unless it is perfectly monochromatic, and thus, different wavelengths yield different speeds. Waveguide dispersion is a small effect that can occur, resulting from the variation of the refractive index with wavelength, even in the absence of material dispersions. Clearly, depending on the applications, we need to have a careful choice of ﬁber, and of light source, in order to reduce some loss of information being sent.

Problems P6.1. Consider a robot gripper driven by a PLZT optical actuator control problem shown in Figure P6.1. Gs ; and Guv ðsÞ and Ga ðsÞ are assumed to be

Basic Optomechatronic Functional Units

433

given by Guv ðsÞ ¼ 1

Gs ðsÞ ¼ 1

and

Ga ðsÞ ¼

2 3kl L2 aq ðb0 s þ b1 s þ b2 Þ 3 8w s þ a0 s2 þ a1 s þ a2

All parameters are given in Table 6.1. When a proportional controller is used, Gc ðsÞ ¼ kp ; and Gp ðsÞ is neglected then (1) What will be the steady state error to a step position, xd ðsÞ ¼1/s? (2) If an error is present for this input, how you can eliminate it? (3) In the above problems (1) and (2), the gripper dynamics Gp ðsÞ is neglected. If this cannot be neglected, what type of Gp ðsÞ will it be? If this Gp ðsÞ is included in the transfer function, what will be the effect of the inclusion?

xd

controller

UV radiator and motor

optical actuator

gripper

Gc(s)

Guv(s)

Ga(s)

Gp(s)

x

Gs(s)

FIGURE P6.1 The control system block diagram of the optically operated gripper.

P6.2. Consider the case when mass m2 moves the slope shown in Figure P6.2, while mass m1 sticks to the ﬂoor. Determine the condition for no slip between mass m2 and ﬂoor. moving SMA actuator m2 fixed m1 q FIGURE P6.2

P6.3. The conﬁguration of the optical actuator employing photothermal effect, shown in Figure P6.3, uses the optical ﬁbers forming three cantilevers whose legs are joined to an equilateral triangle body, as shown in Figure P6.3. The bottom of each leg is cut in a bevel shape and colored black so that it can absorb light to convert it to heat when light enters. If the light is on, this photothermal effect occurring at the bottom of the leg makes the leg bend

434

Optomechatronics light

body

moving fiber optical fiber with black color coated motion direction FIGURE P6.3 Schematic of the optical actuator moving in a straight line.

(deformed). Then, due to this deformed state, the leg stretches forward. Explain in detail how a straight line movement and two-dimensional movement can be generated by appropriately applying light to the ﬁbers. P6.4. Consider the dynamic model at stage (2) shown in Figure 6.10. At this stage, the moving machine has one real foot in the state of locking, while its body stretches with the front foot free to move due to temperature rise in the shadowed area. (1) Draw a free body diagram in this case, as shown in the case of stage (4) of Figure 6.11. (2) Derive the condition of no-slip condition for the climbing-up motion of the machine. P6.5. If Bragg deﬂection angle 0:258 is required, what frequency of an acoustic wave needs to be generated? A red laser beam is used for an optical scanning purpose. The acousto-optic material is a doped glass listed in Table 6.4. P6.6. An acousto-optic crystal deﬂects the beam from the angle ui to ud : Suppose that the width of the aperture is 2 mm, and that the velocity of the acoustic wave in the crystal is 700 m/s. Determine the rise-time required to deﬂect the beam by acoustic wave. Assume that the beam fully ﬁlls the aperture of the crystal.

Basic Optomechatronic Functional Units

435

P6.7. Figure P6.7 shows a two-dimensional scanning method for reading or writing. It is composed of a single post-objective galvanometer scanner, an objective lens (not shown), and a moving medium. (1) Discuss a method that can achieve a ﬂat scan ﬁeld without using any additional optical systems. (2) Consider the case when we employ an active method which uses a z translator shown in Figure 6.42. We wish to correct the deviation of the scan focus length in an on-line fashion. Draw a control system block diagram using Figure P6.7, and explain the operation principle. In (2), devise a sensor that can measure the scan deviation at each instant of scanning. motor

scanner

beam from objective lens

moving

FIGURE P6.7 A conﬁguration of two-dimensional scanning with one scanner.

P6.8. An x – y galvanometer scans through the whole region of a scan ﬁeld (80 £ 80 mm area), as shown in Figure P6.8. Suppose that two mirrors are

80 A 80 y-axis scanner

−A −80

(0,0)

−80

FIGURE P6.8 Geometry of a galvanometer scanning.

400

mm

436

Optomechatronics

separated by 20 mm. The distance from y-axis mirror to scan plane is 400 mm. Compute the deviation of the focus along the line of AA0 at an interval of 10 mm. P6.9. Derive the simpliﬁed equation given in Equation 6.58, obtained by neglecting the effect of inductance of the electric circuit d2 u du þ 2zvn þ v2n u ¼ Kv2n eg 2 dt dt P6.10. Show that the damping ratio used in a galvanometer is 0.690 when the response curve intersects the magnitude u=us ¼ 0:95 at the time t ¼ 0:454ðvn t=2pÞ and meets the tangency point pt at upper accuracy limit, as shown in Figure 6.47. P6.11. Consider a single lens arrangement shown in Figure P6.11. If this lens with focal length f moves forward to a location P at which the image distance is 2f, what will be the magniﬁcation of the image size? Assume that object distance is much larger than the image distance. (1) Explain the results by drawing the ray path through the pin-hole model of the thin lens. (2) Describe the defocusing phenomenon when only a single lens is used for zooming.

object

lens

image screen

f initial position lens

2f location P FIGURE P6.11 Magniﬁcation change using a single lens.

Basic Optomechatronic Functional Units

437

P6.12. Suppose that a plus lens with a focal length f1 ¼ 15 is placed in front of the minus lens with a focal length f2 ¼ 210 shown in the Figure P6.12. They are located initially at P1 and M1, as shown in the ﬁgure. When the minus lens gradually moves backward to two other locations M2, M3, and the plus lens moves to two other locations P2, P3 from their initial position, the resulting focal length variations are shown in the ﬁgure. Note that the focusing points correspond to the ﬁlm plane for all cases. (1) Explain the magniﬁed image obtained at the images plane in case 2 and case 3 as compared with case 1. (2) In order to make the focal length of the imaging system 40mm, design the positions of the plus and minus lenses based on the ﬁgure. Explain the image magniﬁcation by comparing with the case of initial settings.

d =7.5

70 60

case1 P1

50

M1 d =10

40 30

case 2 P2

20 10

film plane

equivalent focal length (fequ), mm

80

M2

d =14

case 3 P3 50

40

30 20 distance from film (d), mm

M3 10

0

FIGURE P6.12 Focal length variation due to the lens movement in a two-component zoom.

P6.13. Consider a cam-driven mechanism for zoom control system. The relation between the lens position and the equivalent focal length are found to be as given in Table P6.13. We wish to achieve feq ¼ 15 mm from initial feq ¼ 20 mm: (1) Determine a function form (4th order polynomial in term of feq ) that describes the data given in the table. (2) Compute the distance of lens movement L2 and L3 from their initial positions.

438

Optomechatronics

(3) Express the above relation in terms of the rotation angle u of the cam mechanism. Assume the radius of barrel is R (Figure P6.13). TABLE P6.13 The Parameter Values Used for Autozooming d1,2 (mm)

d1,3 (mm)

fequ (mm)

17.33 16.33 15.33 14.33 13.33 12.33 11.33 10.33 9.33 6.33

13.46 12.70 11.87 10.96 10 6.96 7.85 6.62 5.20 3.33

12.98 13.52 14.02 14.51 15 15.52 16.12 16.87 17.96 20

radius: r

motor

f cam profiledL2

cam profiledL3

CCD

q fixed 1st lens

2nd lens

dL2

3rd lens dL3

radius: R M:1 reduction gear

image plane

FIGURE P6.13 A cam-driven system for auto-zooming.

P6.14. Consider an autofocusing system whose conﬁguration is given in Figure P6.14. Suppose that a camera is autofocusing an image of a stripe pattern shown in Figure P6.14. The camera moves by an incremental distance dz ¼ 1 mm ranging from z ¼ 0 mm to z ¼ 8 mm:

Basic Optomechatronic Functional Units

439 z=0 z=1 z=2 z=3 z=4 z= 5 z=6 z=7 z=8 100 pixels 100 pixels

480 pixels

640 pixels FIGURE P6.14 The image of the stripe pattern obtained at various locations.

(1) The focus measure in each image is deﬁned for the area within the window, as indicated, 100 £ 100 pixels. Determine the focus measure for the image obtained at each location. The measure is computed based on the following three operators. (1) Sobel operator, (2) Laplacian, (3) Weighted median. (2) Compare the results of the three methods by plotting the relationship of the focus value vs. camera location, and discuss the differences between them. P6.15. Suppose that a focus measure vs. camera position for a given human face is plotted in Figure P6.15. The distribution of the measure is assumed to be given by

FðzÞ ¼

1 1 exp 2 2 ðz 2 zÞ2 2ps 2s

The exact peak value of the measure needs to be determined by the hill climbing method, starting from zð0Þ ¼ 24:0: Assume that the parameters are at s ¼ 1:0; z ¼ 3:5; z0 ¼ 24:0; Dz ¼ 3:0; dz ¼ 1:0: (1) Show all the procedures needed to determine the peak measure. (2) Plot the camera position vs. time, and the corresponding measure vs. time.

Optomechatronics

focus measure F(z)

440

z

z0

F(z) =

−4.0

1 exp 2ps

1 (z-z)2 2s 2

3.5 camera position (z)

s = 1.0, z = 3.5, z0 = −4.0, ∆z = 3.0, δz = 1.0

FIGURE P6.15 Focus measure of human face image vs. camera position.

P6.16. Consider the illumination concept illustrated in Figure P6.16. Based on the concept, (1) discuss the effect of surface normal, incident angle, and the optical axis of the detection system – and then the relation between these three factors. (2) when the normal surface of an object is to be measured, how we can design an appropriate illumination system. camera ring illuminator

LED

(a) small incident angle

object (block)

(b) middle incident angle

(c) large incident angle

FIGURE P6.16 Three different illumination angles produced by a ring-type illuminator.

P6.17. Figure P6.17 shows a three-layer illumination system to detect the image of a PCB which is composed of several jointed areas of electronic parts, marks, and a background. The PCB to be imaged is shown in the ﬁgure.

Basic Optomechatronic Functional Units

441

camera

Contact part

top illumination middle illumination bottom illumination

Board mark

PCB board

(a) vision system for imaging

(b) PCB board to be inspected

FIGURE P6.17 Illumination system for detecting a PCB.

(1) What color of illumination light will be effective to detect the jointed areas and marks when the color of the joints is approximately similar to that of gold, while marks are in white. The camera used here is monocolor. (2) When the illumination ring in the bottom layer is on, what can we expect the image to look like? (3) When the top layer is on, discuss the characteristics of the image that can be detected. P6.18. Eleven fundamental functionalities have been discussed in Chapter 5. Following the methods illustrated in the chapter, draw each of the signal ﬂow diagrams for the systems shown in Figure 6.16, Figure 6.24, Figure 6.34, Figure 6.42 and Figure 6.65. P6.19. Figure P6.19 shows a scene of visual servoing that drives an electric chip on a 6-axis stage to be located at a desired position with a desired pose. Therefore, the objective of this visual control is to move it from the current image in (a) to the desired image (b), as shown in Figure P6.19c. (1) Determine four feature points(corner points) of the electronic chip shown in Figure P6.19a by using Hough transform presented in Chapter 3, “Perspective Projection.” (2) Determine the image Jacobian at the current location by obtaining the feature points in the current image frame. Assume that the camera intrinsic parameters are given by: au ¼ 1955, av ¼ 1940, u0 ¼ 358, v0 ¼ 292, u ¼ 908, f ¼ 0.016. The chip is located 0.3 m away from the camera, along the optical axis of the camera. For simplicity,

442

Optomechatronics

assume that at this instant all the four corner points are located at the same vertical position z ¼ 0:3m. (3) From the image Jacobian obtained in the above question, determine the velocity command input u ¼ K p Jþ e where Kp is the identity gain matrix and J þ is the pseudo inverse of J and e is the error vector.

camera ring illumination

rectangular chip

y X

wX

wy

(b) desired image

wz Z 6 axis stage

(a) current image

desired

experimental rig

current

(c) current image error FIGURE P6.19 Visual servoing for an electronic chip.

P6.20. An acousto-optic deﬂector is used to diffract the incident light beam at a speciﬁed angle, as will be seen in the next chapter. The diffraction angle determines which photo-diode within the array will receive the diffracted beam as shown in Figure P6.20. The output of the diode will then enter the ampliﬁer. lens

AOD

laser source

lens PD array

signal conditioning

q

to actuator amplifier

FIGURE P6.20 AO diffraction-based signal actuation.

Basic Optomechatronic Functional Units

443

(1) How many signal transforming modules are in the system? Explain them in detail. (2) How many signal modulation modules are in the system? Explain them in detail. (3) Draw a signal ﬂow graph of this system, and give it a physical explanation. P6.21. Figure P6.21 shows an optical disk that works on the optomechatronic principle. (1) Explain its operation principle. (2) Repeat the same problems (1), (2) and (3) given in P6.20. disk photo-detector lens laser half-mirror

mirror

FIGURE P6.21 Optical storage disk.

P6.22. Figure P6.22 shows a digital camera. (1) Explain how it works. (2) Repeat the same problems (1), (2) and (3) given in P6.20. pento-penta prism view finder aperture

imaging plane

shutter zoom lens

focusing sensor

FIGURE P6.22 A camera.

References [1] [2]

Acousto-Optics A.A Sa, www.a-a.fr, 2005. Acousto-Optic Report, Crystal Technology, Inc., http://www.crystaltechnology. com/AOMO_app_notes.pdf, 2005.

444

Optomechatronics

[3] Acousto-Optic X-Y Scanning System, Electro-Optical Products Corp., http:// www.eopc.com/dtsxy100_dtsxy250_dtsxy400.html, 2005. [4] Akiyama, T., Collard, D., and Fujita, H. Scratch drive actuator with mechanical links for self-assembly of three-dimensional MEMS, Journal of Microelectromechanical Systems, 6:1, 10 – 17, 1997. [5] AO Application Notes, AA Opto-electronic, http://www.a-a.fr/doc/ AOApplicationnotes.pdf, 2005. [6] Atkins, R.A., Gardner, J.H., Gibler, W.N., Lee, C.E., Okland, M.D., Spears, M.O., Swenson, V.P., Taylor, H.F., McCoy, J.J. and Beshoun, G. Fiberoptic pressure sensors for internal combustion engines, Applied Optics, 33:7, 1315– 1320, 1994. [7] Bentley, J.P. Principles of Measurement Systems, 3rd Ed. Longmans, New York, 1995. [8] Bernacki, B.E. and Mansuripur, M. Causes of focus-error feedthrough in optical-disk systems: astigmatic and obscuration methods, Applied Optics, 33:5, 735– 743, 1994. [9] Choi, K.S., Lee, J.S., and Ko, S.J. New autofocusing technique using the frequency selective weighted median ﬁlter for video cameras, IEEE Transactions on Consumer Electronics, 45:3, 820– 827, 1999. [10] Ehrmann, J. S., Optics for vector scanning, Beam Defection and Scanning Technologies, SPIE Vol. 1454, pp. 245-255, 1991. [11] Ernest, O.D. Measurement Systems Application and Design, 4th ed. McGraw-Hill, New York, 1990. [12] Fan, K.C., Chu, C.L., and Mou, J.I. Development of a low-cost autofocusing probe for proﬁle measurement, Measurement Science and Technology, 12:12, 2137– 2146, 2001. [13] Feddema, J.T. and Simon, R.W. CAD-driven microassembly and visual servoing, Proceedings of IEEE International Conference on Robotics and Automation, 2, 1212– 1219, 1998. [14] Fujita, H. and Toshiyoshi, H. Micro-optical devices, Handbook of Microlithography, Micromachining, and Microfabrication, Vol. 2, P.Rai-Choudhury, ed.,.1997. [15] Fukuda, T., Hattori, S., Arai, F., Matsuura, H., Hiramatsu, T., Ikeda, Y. and Maekawa, A. Characteristics of optical actuator-servomechanisms using bimorph optical piezo-electric actuator, Proceedings of IEEE International Conference on Robotics and Automation, 2, 618– 623, 1993. [16] Fukuda, T., Hattori, S., Arai, F. and Matsuura, H., Optical servo systems using bimorph PLZT actuators, Micromechanical Systems, 46, 13 – 20, 1993. [17] Fukuda, T., Hattori, S., Arai, F. and Nakamura, H. Performance improvement of optical actuator by double side irradiation, IEEE Transactions on Industrial Electronics, 42:5, 455–461, 1995. [18] Haralick, R.M. Computer and Robot Vision Volume I and II. Addison Wesley, Reading, MA, 1993. [19] Harshall, G.H. Optical Scanning. Book News, Inc, Portland, 1991. [20] Hoffmann, M., Kopka, P., and Voges, E. All-silicon bistable micromechanical ﬁber switch based on advanced bulk micromachining, IEEE Journal on Selected Topics in Quantum Electronics, 5:1, 46 – 51, 1999. [21] Hutchinson, S., Hager, G.D., and Corke, P.I. A tutorial on visual servo control, IEEE Transactions on Robotics and Automation, 12:5, 651– 670, 1996. [22] Introduction to Acousto-Optics, Brimrose Corporation of America, http:==www.brimrose.com, 2005.

Basic Optomechatronic Functional Units [23]

445

Joung, J., Kim, K., Kim, I.S. and Park, J. High-sensitivity ﬁber optic Fabry-Perot pressure sensors for medical applications, Proceedings of SPIE-Optical Engineering for Sensing and Nanotechnology(ICOSN 2001), 4416, 432– 435, 2001. [24] Kao, T.W. and Taylor, H.F. High-sensitivity intrinsic ﬁber-optic Fabry – Perot pressure sensor, Optics Letters, 21:8, 615– 617, 1996. [25] Matsuba, Y., Otani, Y., and Yoshizawa, T. Two-dimensional movement of photothermal actuator composed of optical ﬁbers, Proceedings of SPIE, 4902, 78– 82, 2002. [26] Mitutoyo Catalog No. E4181, E4142-359, E4242-361, and E4214-361, Mitutoyo Corp., http://www.mitutoyo.com/, 2005. [27] Nussbaum, A. Optical System Design. Prentice Hall, Englewood Cliffs, NJ, 1997. [28] Otani, Y., Matsuba, Y., and Yoshizawa, T. Photothermal actuator composed of optical ﬁbers, Proceedings of SPIE, 4564, 216– 219, 2001. [29] Pech-Pacheco, J., Cristobal, G., Chamorro-Martinez, J. and FernandezValdivia, J. Diatom autofocusing in brightﬁeld microscopy: a comparative study, Proceedings of International Conference on Pattern Recognition, 15:3, 314– 317, 2000. [30] Pugh, A.. Robot Sensors, Tacktile and Non-Vision, Vol. 2. Springer-Verlag, Berlin, 1986. [31] Subbarao, M. and Tyan, J.K. Selecting the optimal focus measure for autofocusing and depth-from-focus, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:8, 864– 870, 1998. [32] Sun, F., Xiao, G., Zhang, Z. and Grover, C.P. Piezoelectric bimorph optical-ﬁber sensor, Applied Optics, 43:9, 1922–1925, 2004. [33] Tabib-Azar, M. and Leane, J.S. Direct optical control for a silicon microactuator, Sensors and Actuators, 21:1/3, 229–235, 1990. [34] Toshiyoshi, H., Su, J.G.D., LaCosse, J. and Wu, M.C. Micromechanical lens scanners for ﬁber optic switches, Proceedings of the Third International Conference on Micro Opto Electro Mechanical Systems, 165– 170, 1999. [35] Yoshizawa, T., Hayashi, D., and Otani, Y. Optical driving of a miniature machine composed of temperature-sensitive ferrite and shape memory alloy, Proceedings of SPIE, 4190, 212–219, 2001. [36] Yoshizawa, T., Vsui, T., Yomamoto, M. and Mayashi, D. Miniaturized machine moving in a pipe using photothermal effect, Proceedings of SPIE, 4902, 110 – 115, 2002. [37] Zhang, J.H. and Cai, L. An autofocusing measurement system with a piezoelectric translator, IEEE/ASME Transactions on Mechatronics, 2:3, 213– 216, 1997.

7 Optomechatronic Systems in Practice CONTENTS Laser Printer ....................................................................................................... 448 System Conﬁguration and Units ............................................................. 451 The Laser Optical Elements.............................................................. 452 Photoconductor................................................................................... 452 Toner Charging Development System ............................................ 452 Hot Roll Fuser ..................................................................................... 454 Black Density Control System .......................................................... 455 Printing Performance Speciﬁcation......................................................... 455 Laser Source ........................................................................................ 456 Acousto-Optical Deﬂector................................................................. 457 The Optical System.................................................................................... 458 System Conﬁguration ........................................................................ 458 Resolution of Optical Devices.................................................................. 459 Line Scanner: Polygon Mirror .......................................................... 459 Feedback Control of Beam Focus ............................................................ 463 Aperture Control ................................................................................ 465 Optical Storage Disk.......................................................................................... 469 System Conﬁguration................................................................................ 470 Optical System for Focusing .................................................................... 472 Beam Tracking ............................................................................................ 477 Disk Servo Control System....................................................................... 478 Atomic Force Microscope ................................................................................. 484 Measurement Principle ............................................................................. 484 Cantilever Deﬂection ......................................................................... 493 Optical Measurement of Cantilever Deﬂection .................................... 496 Optics for Tip Deﬂection Measurement ......................................... 497 Control of the Scanning Motion .............................................................. 501 Confocal Scanning Microscope........................................................................ 510 Measurement Principle ............................................................................. 510 Beam Scanning ........................................................................................... 513 Nipkow Disk............................................................................................... 518 System Resolution...................................................................................... 520

447

448

Optomechatronics

Focus Measure and Control ..................................................................... 521 Projection Television.......................................................................................... 524 Digital Micromirror Device ...................................................................... 525 Architecture of a DMD Cell.............................................................. 526 Light Pulse Width Modulation ........................................................ 532 Grating Light Valve Display .................................................................... 535 Light Diffraction by Deformable Grating Modulator................... 536 Deﬂection Control of Microbeam Structures ................................. 538 Visual Tracking System..................................................................................... 540 Image Processing ....................................................................................... 541 Feature Extraction ...................................................................................... 544 Visual Tracking of a Moving Object ....................................................... 548 Zoom Lens Control ............................................................................ 549 Problems.............................................................................................................. 558 References ........................................................................................................... 563

In the previous chapter, we derived a variety of functional components that are characteristics of optomechatronic nature. These components can be identiﬁed as a fundamental common tool that can realize optomechatronic integration and in actual practice are becoming the essential parts of many optomechatronic systems. In fact, this emerging trend can be observed in a number of engineering products and systems. In this chapter, we will illustrate several optomechatronic systems in which the optomechatronic components are playing essential roles in performing key functional tasks. The systems to be dealt with here include laser printers, optical storage disks, atomic force microscopes (AFM), confocal microscopes, digital micromirror devices, and visual tracking systems.

Laser Printer Basic operation principles of a laser printer combine the technology of a photocopier with laser light scanning [22]. The printer has several advantages over an inkjet printer. It can write with much greater speed than an inkjet printer and it can draw more precisely due to adoption of an optical system with an unvarying diameter laser beam and optical units with high level of precision and reliability. Another advantage is that its maintenance cost is much lower than that of the inkjet, because toner powder is cheap and lasts a long time. The laser printer uses an electrophotographic process which is operated with a variety of optical and mechatronic units as shown in Figure 7.1. The process develops a latent image on a photoconductive rotating drum when a laser beam from a spinning polygon mirror scans across it.

Optomechatronic Systems in Practice

449

FIGURE 7.1 A schematic of a laser printer.

This image is then coated with toner particles and transferred on to a sheet of paper as it rolls in contact with the drum surface. Due to the interaction of optical units with mechatronic ones, printing quality is affected by how well each unit is interfaced with the other. For instance, even if a laser beam is shaped with appropriate units, the output beam leaving the spinning mirror will be seriously affected if it were to wobble while rotating. Also, the variation of the drum rotation will create an image artifact, called banding, when the shaped output beam hits the surface of the drum. Therefore, interaction problems including those just mentioned are among those that need to be solved for the purpose of ensuring print quality with higher resolution and speed. The basic principle of the printing is static electricity which is an electrical charge built up on an insulated object, such as a balloon. To use this principle, the printer as shown in the ﬁgure consists of a rotating metal drum coated with a highly photoconductive material, a charging corona wire charging the drum, a developer roller that coats the drum with toner particles, a densitometer that measures the developed blackness of a pattern on the area of the drum, and a transfer corona wire keeping the paper from clinging to the drum. Figure 7.2 illustrates some of the details of the printing process. The drum assembly is initially negatively charged and can conduct electricity only when it is exposed to light photons. When the printer

450

Optomechatronics hot roll

roll

stationary magnet drum

incoming paper

. . .. ..............

roller

optical switch

.

shield . wire

to

drum

un

.. ........ .... ar .. ........... .. ge d ar ea

ch

ground

ne

positively charged

drum

scanner

g ch ati ar vel ge y d

paper with image

ne ..... ........... .... r a..... ....... .. tta.. .. ch ed

.

.

.

.

FIGURE 7.2 Laser printing process.

receives the image in digital form from the computer, the laser light hits the desired parts of the drum surface according to the switching action of the optical switch and the scanning action of the polygon mirror shown in the ﬁgure. These control actions are directed by the printer’s microelectronics. As soon as the scanned light irradiates the designated surface area of the drum, the illuminated area begins to conduct electricity and is discharged as a result. The drum is then covered with a pattern of negatively charged areas (those not exposed by light) and uncharged areas (those exposed by light). After the pattern is set, the developer roller coats the drum with negatively charged black toner particles. They cling to the uncharged area of the drum but not to the negatively charged background that was not exposed to the light. Once the powder pattern is in place the drum rolls over a blank sheet of paper with a positive charge given by the transfer corona wire. The positively charged paper attracts the negatively charged toner particles and pulls them away from the drum as indicated in the ﬁgure. Here, it is necessary to keep the rotation speed of the paper at the drum speed in order to make the paper pick up the pattern exactly. As the paper passes through the fuser rollers composed of a pair of heated rollers, the plastic toner powder melts and fuses permanently with the ﬁbers in the paper. Finally, the fuser rolls the paper

Optomechatronic Systems in Practice

451

to the output tray. It is noted here that, because the fuser also heats up, it is important for the paper to pass the rollers very quickly. The discharge lamp shown in the ﬁgure takes the roll of erasing the electrical image of the entire photoreceptor surface by illuminating bright light on to the surface. System Conﬁguration and Units As already explained above, the laser printer is of complex optomechatronic nature, because generating the laser pattern and forming an electrostatic image from the pattern involve optomechatronic processes in which optical and mechatronic units are interacted with each other. These interactions are provided by the acousto-optical deﬂector (AOD), the polygonal scanner and a rotating drum and its drive unit, as shown in the system control block diagram shown in Figure 7.3. According to the logic circuits, the computer sends the basic information to the printer concerning the printing format such as text, graphics, and margin sizes. The information to be used to set up the pattern for modulating the laser beam is created in the microprocessor. In response to the generated digital signal pattern, the AOD deﬂects the beam either into the optical path to the drum or away from it. When the laser is “on,” the scanner transmits the beam on to the surface of the photo-sensitive rotating drum resulting in a dark spot. The scanning action is performed along the axis of the drum, i.e., (x direction). When one-line scanning for a line is completed, rotation of the drum (y direction) makes the scanner start another line. This results in a raster scan of the drum surface.

motor servo

fuser roller

microprocessor

paper densitometer

motor servo

photoconductor Phot oconductor drum drum f - q lens developer station laser

A-O beam encoder acousto-optical deflector deflector (AOD) FIGURE 7.3 Control system schematic of a laser printer.

polygon mirror

452

Optomechatronics

The Laser Optical Elements The optical system consists of a laser source, a movable mirror, a scanning mechanism, and a series of lenses including a f-u lens. The system receives the page data which makes up the text and images one horizontal line at a time. They come in digital form (tiny dots). According to the data, the laser source emits a pulse of light for every dot to be printed and no pulse for every dot of empty space. Referring to Figure 7.1 and Figure 7.3, the laser beam enters the AOD modulator and the modulated beam is collimated by a collimating lens, which is again shaped ready to enter the mirror facet of the rotating polygon deﬂector. Because this causes aberrations such as pincushion, a f-u lens is normally placed in front of the electro-photographic drum. The role of the beam scanner is to distribute the laser beam spatially in the horizontal line on to the drum. As the spinning mirror moves, it illuminates the beam through the f-u lens, making the beam move across the drum. It can be seen that the optical assembly scans only one horizontal line for each move. After each scan, a printer mechanism moves the drum up by a notch for preparation of the next line drawing and then a new scanning command is detected by a scan start detector. A print-engine computer synchronizes all of these. Photoconductor The conductor drum contains a charged layer photoconductive element that bears the latent image generated by the light; charge transforming layer (typically 10 to 30 mm), and charge generation layer (typically 0.5 to 5 mm). As indicated in Figure 7.4a, initially the photo drum charges to a high voltage level which is uniform over its surface. When laser light is impinged on the charge generation layer, this generates the photoconduction process in the layers by which the electron-hole pairs are produced. In the process shown in Figure 7.4b, electrons exit at the electrode, while holes are injected into the charge transporting layer. These holes are transported across the layer to the free surface of the photoconductor and neutralize the negative charge to create the latent image. Therefore, the surface of the drum enters the development section with a light exposed area and nonexposed area having approximately 0 and 500 V, respectively. Toner Charging Development System The development section contains a toner hopper and a developer roller, as shown in Figure 7.5. The toner is essentially an electrically charged powder, which is composed of two main ingredients, pigment and plastic. The use of it makes the laser printer distinct from an ink-jet printer. The advantage of the toner over the liquid ink is that it ﬁrmly sticks to the ﬁbers of almost any type of paper. The toner powder is supplied by a developer roller from the toner hopper as shown in the ﬁgure. The developer is covered with small positively charged magnetic beads, which are attached to a rotating metal

substrate

FIGURE 7.4 Cross sectional view of a photoconductive drum.

(a) corona charge electron hole pair

electrode

charge generation layer

charge transporting layer

light

(b) hole transport neutralization

substrate

latent image

electrode

charged photoconductive surface

Optomechatronic Systems in Practice 453

454

Optomechatronics

f -q lens photoconductive drum

carrier particles with toner W photo conductor (drum)

stationary magnets

developer roller magnetic brush

paper developing region FIGURE 7.5 A schematic of the development system for toner charging.

roller. The developer carries the beads with toner particles into the developing zone. Depending upon which zone light is absorbed, the transported toner particles are transferred. In this way the toner powder is provided to the drum while the roller rotates through the hopper. Hot Roll Fuser In the transfer process, during which the toner attached to the drum is transferred to the paper, the toner particles are weakly attached to the paper. It is therefore necessary to ﬁx them in place to the paper surface permanently. This process is called the fusing process and has two types of fusing, contact and noncontact. The contact type requires fusing by touching the toner slider of the paper, while the noncontact one does not touch. Figure 7.6 illustrates a typical contact type hot roller fuser which is composed of a hot roller heated by a lamp, a backup roller (pressure roller) applying pressure to the hot roller, and a thin layer coated with elastomer used for high thermal stability and low surface energies. The low energies employed here yield minimum toner sticking. It can be seen that fusing quality mainly depends upon the temperature (T) of the interface between the paper and toner, the pressure ( p) applied to the toner parties, and time (t) in the nip zone.

Optomechatronic Systems in Practice

455 elastomer core coating

hot roller

nip zone pressure roller backup roller W

FIGURE 7.6 Contact-type hot roll fuser system.

Black Density Control System An important unit at this stage of printing process is the one which controls the blackness of the pattern printed on an area of the drum. A reﬂective densitometer measures the blackness of the developed area and this information is sent to the microprocessor. The processor then compares with the desired blackness and, if any deviation occurs, it adjusts appropriate development parameters to regulate the proper developed density. Printing Performance Speciﬁcation There are several factors that affect the printing quality of the laser printer. Three primary factors that represent the print quality are resolution of spot size and waiting speed of the printer and laser power consumption. We will examine some of these factors that determine quality. Let us begin our discussion with a series of characters that can be formed on the drum surface. Suppose that the scanning system scans the drum in such way that a character is formed within n £ m dot matrix as illustrated in Figure 7.7. If the printer has a printing rate of Nc characters per second, it must produce Ps dots per second Ps ¼ n £ m £ N c

ð7:1Þ

To ensure printing image quality proper overlap between adjacent dots is necessary, as can be seen from the H character in the ﬁgure. To this end we need to specify the horizontal separation and the vertical separation between dots. The horizontal separation ‘h is determined by the modulation speed

456

Optomechatronics

FIGURE 7.7 A laser scanned H character.

of the AOD, but the vertical separation ‘v is determined by the positioning accuracy of the drum controlled by a rotation motor. The writing resolution can be also easily determined in terms of number of dots. For example, if the printer is assumed to print the entire width of an A4 page (210 £ 297 mm), with ‘h ¼ 30 mm, ‘v ¼ 40 mm, the required number of dots per scan in horizontal direction is given by N¼

210 ¼ 7000 dots 0:03

As we shall see later, the number of dots per scan is related to the geometry of the polygon scanner and the diameter of the beam spot entering into the mirror facet of the scanner. The above discussion makes it necessary to analyze the optical system of the printer in order to determine the design parameters that meet the speciﬁcations required for the resolution, speed, and exposure laser level of the printer. We will deal with this topic in the subsequent subsections. Laser Source The laser needs to have long working life span and low power consumption while its wavelength should be short to make the drum photosensitive. Nowadays, due to advancement of diode laser technology, laser diodes are

Optomechatronic Systems in Practice

457

more conveniently chosen over the He– Ne laser as a printer source in view of cost and size, but we will work with the He– Ne laser to become familiar with the operation of the AOD. Provided the speciﬁcations are of the required resolution, and the printing speed is that given in Equation 7.1, we can calculate the laser power consumption level. If the printer has a resolution of 25 dots per mm, that is, dot size is given by Dspot ¼ 2rspot ¼ 40 mm where Dspot and rspot are the diameter and radius of the dot, respectively. The dot area denoted by Ad approximately becomes Ad ¼ 40 mm £ 40 mm ¼ 1600 mm2 Now, consider the case when the printer prints 36,000 characters per second each of which is composed of 18 £ 25 dot matrices. If the energy level of a laser is given by 1.6 mJ/cm2, laser exposures per dot is given by 2.56 £ 1025 mJ. Therefore, the laser power level P‘ becomes P‘ ¼ ð2:56 £ 1025 mJÞ £ 1:62 £ 107 =sec ¼ 0:41 mW Acousto-Optical Deﬂector Equation 7.1 determines the time for one dot to be printed, which is the reciprocal of Ps : Let us suppose that the time for printing one dot would be 40 nsec. This indicates that if an AOD cell is used, the rise time must be shorter than 40 nsec. According to Equation 6.41, the laser beam diameter to be modulated inside the modulator affects the rise time as Tr ¼ b

Ds va

where Tr is the rise time of the AOD, Ds is the beam diameter illuminating a facet of the polygon, and va is the velocity of the sound. If b is unity, and the time for printing one dot is required to be less than 40 nsec, then Ds ¼ 40 £ 1029 sec £ 6 £ 106 mm=sec ¼ 240 mm in the case of using fused quartz as a modulator. In early laser printers adopting a hard-sealed He– Ne laser with a beam diameter 1 mm, the beam must be compressed by a factor of four or ﬁve in order to obtain the 240 mm diameter by a beam compressor as depicted in Figure 7.1. We shall see later that when a laser beam compressed for modulation enters the polygonal scanner, its beam diameter needs to be expanded by a beam expander to increase the writing resolution whose expansion ratio depends upon the area of the scanner facet.

458

Optomechatronics

The Optical System System Conﬁguration Figure 7.8 shows a complete arrangement of the optical system redrawn for better understanding of the optical beam ﬂow from the laser source to the photoconductive drum surface. The optical system shown here is so composed that the effects of some undesirable factors affecting print quality are reduced. In general, the optical system without compensation has several factors that critically affect print quality. If the major ones are listed here: aberration (geometrical and wavefront), focusing, and beam spot divergence. Aberration causes ﬁeld curvature on the scan plane, defocusing causes blurring effects and beam divergence affects resolution of imaging. Unless all of these are solved, good printing quality cannot be guaranteed. We will brieﬂy discuss some of the correction methods that can drastically reduce these unfavorable optical effects. Correcting the ﬁeld curvature is done with an f-u lens as discussed in Chapter 6, “Optical Scanning.” To reiterate, here we will consider a lens assembly which consists of the combination of several lenses whose conﬁgurations depend on performance requirements, some have only two, for instance, a toroidal lens, and a cylindrical lens. Referring to Figure 6.39, the cylindrical lens having a concave surface forms the scan line bent outward from the photodrum surface. In this situation we can use this lens to ﬂatten out the scan line along the surface. Figure 7.9 illustrates the principle of correction by drawing the scan lines with and without correction. As can be seen from the ﬁgure, the solid line bent inward towards the lens denotes the focus line of the image obtained by a single convex lens (toroidal surface), while the one bent outward from the drum surface represents the result obtained by a concave cylindrical surface. Combining these two curves together, a solid line can be obtained, which is the result of the correction by the f-u lens. This result shows that when two lenses are combined, the focus shift is greatly reduced, otherwise it remains large with a single toroidal convex lens as indicated in the ﬁgure. Further correction needs to be made for higher demands of printing cylindrical toroidal concave surface surface

aperture

photo drum

polygon mirror laser source

collimator

cylindrical lens

f - q lens

image plane

FIGURE 7.8 Optical arrangement for the laser printing system. Source: Donald C. O’Shea, Elements of Modern Optical Design, John Wiley & Sons, 1985. Reprinted with permission of John Wiley & Sons, Inc.

Optomechatronic Systems in Practice

459

corrected lens inward

scan position

image plane

outward

cylindrical concave lens without correction (toroidal convex lens)

lens

−1.0

0 (normalized)

1.0

drum

FIGURE 7.9 Correction of ﬁeld curvature using a single cylindrical concave lens.

performance. In this case several more lenses may be added in order to eliminate the focus shift. Resolution of Optical Devices A scanned beam impinged on the surface of the photoconductive drum is the result of the modulation by an external modulator, such as AOD, and then reﬂection from the surface of the rotating polygonal mirror. Therefore, printing quality depends largely upon the angular resolution of the modulator as well as the scanner. To analyze this in detail, let us consider the limit of resolution discussed in Chapter 2, “Diffraction,” that is, the minimum achievable angular resolution of a circular aperture of a certain dimension when a beam passes through it. Line Scanner: Polygon Mirror A collimated beam passing through the modulator is now reﬂected from the polygonal mirror facet and the reﬂected beam is focused toward the drum. Resolution of the printer adopting a polygonal mirror is largely determined by the parameters of the scanner, as noted from the polygonal scanner in Chapter 6, “Optical Scanning.” The number of resolvable spots can be obtained if we consider the diffraction-limited resolution determined by a circular aperture. For the scanner having the width of the deﬂected beam, Ds and Ms facets, this is determined to be Nr ¼ 12:6

Ds lMs

ð7:2Þ

460

Optomechatronics

When the scanner is chosen such that for Ds ¼ 1:5 cm, Ms ¼ 36 and l ¼ 633 nm we will have the spot number Nr ¼ 8300 spots. It is noted here that, if the resolvable length in the scan direction Dy0min in Equation 2.86 is used, we can evaluate the printing resolution in terms of the scan-spot size. The scan spot radius denoted by rspot is expressed in terms of f number ð f =#Þ of the system including the scanner and the focusing lens rspot ¼ 1:22lð f =#Þ

ð7:3Þ

It is noted that this formula is valid for a circular aperture and not strictly correct for a rectangular aperture like a polygon mirror. Since the error is very small, we will use this to calculate the spot as spot size radius. Consider the case when we scan a page of 28 cm with a 36-facet scanner having a polygon face-to-scan plane distance d as 800 mm. If the facet width Ds is given to be 8.0 mm, we can calculate the system f =# in the following manner f =# ¼

800 ¼ 100 8:0

Using this f number and Equation 7.3, we have for spot radius rspot ¼ ð1:22Þð6:33 £ 1027 Þð100Þ ¼ 77:2 mm if a He –Ne laser is used. So far, we have discussed the resolution of the beam scanning, but it is equally important to keep the accuracy of the beam spot position, because the beam accuracy determines the position of the dot along the scan line. The defects that cause its positioning error are facet-to-facet angular error and tilting angle between the facet surface and a line parallel to the rotation axis. We can calculate the allowable facet-to-facet error d, if tolerable spot positioning error is given a priori. If the error is 1/2 of the spot radius, the positioning error equation is given by tan d ¼

rspot 2d

ð7:4Þ

where d is the distance between the facet of the scanner and the scan plane which is equal to focal length f. Substituting Equation 7.4 into Equation 7.3, we have for small angle error

d¼

0:61lð f =#Þ d

ð7:5Þ

To obtain the actual tolerable angle error, however, the d in the above must be halved, if the fact is used that angles are doubled upon reﬂection from the scanner surface. When this angular error is difﬁcult to achieve, it needs to be achieved in special production devices. Let us take an example to illustrate tolerance concept. Suppose we have a scanner with the facet width Ds of 6 mm and polygon-to-scan plane

Optomechatronic Systems in Practice

461

polygon mirror untilted

ϕ

tilted

photoreceptor

laser beam

scanned position

ϕ cylindrical lens (L1) cylindrical lens (L2) spherical lens (L3) FIGURE 7.10 A schematic of an optical system correcting the deviated beam due to tilting angle error of the polygon mirror.

distance of 720 mm. The f =# is then f =# ¼

720 ¼ 120 6:0

We then calculate the error at the 50% intensity points of the spot from Equation 7.5

d¼

ð0:61Þð6:33 £ 1027 Þð120Þ ¼ 64 mrad 7:2 £ 1021

When the tilt angle w deﬁned in Figure 7.10 is not tolerable, it is necessary to correct the deviation by some optical means. One such optical modiﬁcation method is depicted where one cylindrical lens L2 and one spherical lens L3 are employed as shown in the ﬁgure which is redrawn from Figure 7.8. The idea is that the ray focused at the focal point of cylindrical lens L1 will be out of optical axis due to facet tilt, if the lenses L2 and L3 are not inserted in between L1 and the photo receptor. With the cylindrical lens L2 positioned at the focal distance from the facet of the mirror it will collimate again the deﬂected beam parallel to the optical axis of the spherical lens L3 : Finally, the spherical lens focuses this collimated beam parallel to its optical axis on to the scan surface of the photoconductor. As discussed in Chapter 6, in the section “Optical Scanning,” there are two methods of scanning depending upon where the scanner and the objective lens are to be arranged; preobjective scanning and postobjective scanning, as shown in Figure 6.37. As mentioned earlier, postobjective scanning is attractive due to a simple lens conﬁguration, but it has a disadvantage that the focal plane is curved due to the variation of beam deﬂection angle at the scanner. In laser printer preobjective scanning is widely used to avoid this, which brings a ﬂat focal plane but requires a complex assembly of various types of lens such as f-u lens as shown in Figure 7.1, adding more complexity in the lens design. Here, the scanning optics needs to be determined by the resolution and overall size of the system. The resolution of the scanning system is largely dependent on the spot size of the focused beam which is, in turn, determined

462

Optomechatronics polygon mirror

spherical lens photoconductive drum

Ds f 2θ

Dw z

laser beam df depth of field FIGURE 7.11 Optical geometry for focused spot.

by such parameters as focal length of the lens, the beam diameter, and the laser wavelength, as discussed previously. The spot diameter at the beam waist Dw indicated in Figure 7.11 is given by Dw ¼ 2:44lð f =#Þ

ð7:6Þ

If the beam spot is located in a region on either side of the focal plane bounded by the depth of focus, the beam spot is regarded to be in focus. Let the depth of focus be denoted by df which is given by df ¼ 2lð f =#Þ2 This is shown in the expanded portion of the ﬁgure, and the bound is given by ^ 10% of Dw : Therefore, it is necessary to locate the photoconductor drum within the depth of focus df : It is noted that within df ; the beam diameter Dw is a function of the distance along the optical axis z, discussed in “Gaussian beam” in Chapter 2. In choosing an f number for optics design, we face a trade-off between the spot size and the depth of ﬁeld. If a large df is used, then a curved focal plane can be utilized to scan the surface of a photoconducting drum, since the whole scan area in this case is considered to be in focus, as indicated in the ﬁgure. If, however, df is chosen to be too large, this makes the f number large, resulting in a large spot size. This harms the resolution of the scanning. To illustrate this let us assume that a desired beam spot size diameter is 45 mm whose beam wavelength is l ¼ 633 nm. When a facet-to-drum surface distance is 860 mm, it is possible to calculate the acceptable range from the mirror facet within which the photoconductor drum can be located to be focused. From Equation 7.6, f =# is determined by f =# ¼ Dw =2:44l ¼ 29

Optomechatronic Systems in Practice

463

From this, the depth of ﬁeld is obtained by df ¼ 2 £ 6:33 £ 1027 £ ð29Þ2 ¼ 1:06 mm Therefore, the range is obtained to be (430 ^ 1.06) mm from the facet surface. Feedback Control of Beam Focus In the foregoing discussions, the arrangement of the optical system is assumed to be ideally laid out such that the laser spot is always exactly focused on to the image surface of the printer if there is no lens aberration. When focused, the diameter of the laser spot at the focal plane is typically maintained at, say, 40 mm possessing a uniform optical density. In practical situations, however, maintaining such ideal conditions is difﬁcult due to variations in ambient temperature, vibration of the rotating mirror, and so on. The resulting defocus produces blurry spots in image formation, and, thus, image quality is not well preserved. To solve this defocusing problem a feedback control system that utilizes the instantaneous contrast value is shown in Figure 7.12 [12]. The system consists of a focus detector, an adjusting mechanism for the collimating lens, a contrast measurement circuit, and a feedback controller. As discussed previously, all of these elements can be conﬁgured in a microprocessor control. Here the control objective is to adjust the position of the collimating lens so that the optical system can have its focal plane at the photoelectric drum. A difﬁculty with the control is how the focus measure is chosen and measured in real time.

f - q lens

imaging surface

polygon mirror

photo conductor drum

collimator focus control mechanism focus controller

laser source

contrast measurement

FIGURE 7.12 Autofocusing control of the scanning system.

focus detector

464

Optomechatronics

We have already discussed an astigmatic method for measurement suitable for an autofocusing system treated in Chapter 6, “Automatic Optical Focussing.” In that case, the measurement was made to obtain the information for focusing conditions of the laser spot and this was used as a feedback signal. This method is not suitable when the laser spot is not stationary and moves with high velocity. In laser printers, the laser spot moves at a very high velocity, as high as 1 km/sec. We therefore need a measurement technique that is not required to track a moving spot. Figure 7.13 illustrates a measurement method which employs a focus detector composed of a grid ﬁlter and a light detector, which are placed adjacent to the start point of the scanning at the photoconductor drum. The grid ﬁlter consists of alternating transparent and nontransparent parts as shown in the ﬁgure. The measurement principle is that the light intensity of the laser beam transmitted though the grid ﬁlter varies according to the position of the laser spot. If the laser spot is in focus and moves in the direction of scanning, the rate of the transmitted light becomes high when it is at the transparent part of the ﬁlter. When it is at the nontransparent part, the leakage of the light through the adjacent part is small: Ideally the leakage should be zero if the spot size is within the range of the nontransparent part of the ﬁlter. The reverse phenomenon is true, when the laser spot is out of focus and accordingly its size is large in the direction of scanning. As deﬁned for visual auto focusing in the Chapter 6 section “Visual Auto Focusing,” a focus measure needs to be deﬁned for this control to judge the focusing quality. A measure of contrast value can be utilized which is given by Cr ¼

ðImax 2 Imin Þ ðImax þ Imin Þ

FIGURE 7.13 A sensor detecting the focus state and its sensing characteristics.

ð7:7Þ

Optomechatronic Systems in Practice

465

where Imax and Imin are the maximum and minimum values of the sensor signal, respectively. A typical relationship between the spot size variation and this contrast value shows that the contrast value does not decrease up to a certain diameter of the laser spot, but thereafter sharply decreases when the diameter of the spot increases. From Equation 7.7 it seems to be desirable to choose a spot size which makes the Cr value as large as possible. When a desired Cr value is predetermined for feedback control, the controller then adjusts the Cr value during printing time so as to maintain the Cr always at the desired value. In other words, the control objective is to have laser spots focused at the drum surface in order to eliminate any defocused image on the drum. The servo motor achieves this by moving the collimating lens in an appropriate position. A block diagram of the control system that can realize this control concept is shown in Figure 7.14. To achieve this, the position of a collimator is made controllable by using a servo motor. As illustrated in Figure 7.14a, depending on the position of the lens, the reﬂected beam will direct differently toward the scanning mirror. When the lens is located at the focal point O of the collimator, the beam will be ideally collimated, thus producing parallel beam leading to the scanner. However, when its position is shifted out of the point O either point A or B, the reﬂected beam will not be exactly collimated, eventually causing the defocusing or blurring at the drum image plane. Figure 7.14b shows a focus control system that maintains the focus measure at a desire value. When the collimator is initially located at a certain position, the optical system will have a defocused image whose contrast is detected by a contrast sensor. The detector output is computed, which then enters the comparator. The error computed is used to generate the controller signal, which actuates a servo motor. The servo motor will in turn control the collimating lens to correct the defocused amount. This feedback control action will be repeated until the controller produces a steady state desired value of the focus measure. The response characteristics of this control system are illustrated in Figure 7.14c. It is illustrated that depending upon the collimating lens position, the degree of focus of the image contained in the circle is changed. The ﬁgure clearly shows that as the collimating lens approaches its desired position, the focused level of the image increases, ﬁnally approaching at a focused state. Aperture Control When light source has the variation of divergence angle, even if small, this will be one of source of quality deteriorations since this results in spot size variation. As illustrated in Figure 7.8, one way to correct such defect is to use a beam aperture which can control the beam width entering the f-u lens, thereby keeping the spot size as constant as possible, e.g., between 40 and 60 mm. The relationship between beam aperture size ðra Þ and radius of scan spot ðrspot Þ is shown in Figure 7.15. This ﬁgure indicates that small spot size can be achieved with relatively small divergence angle of laser beam and

(c)

O

f

A

servo motor

O

focus measure

desired position

controller

FIGURE 7.14 Feedback control of the collimating lens position.

(b)

desired focus position

(a)

laser source

collimating lens position

B

collimator

collimator lens

bundle of ray

time (t)

contrast sensor

optical system

actual focus position

466 Optomechatronics

radius of spot (rspot)

Optomechatronic Systems in Practice

467

increasing beam divergence angle

beam aperture size (ra) FIGURE 7.15 Radius of spot vs. beam aperture size.

large size of beam aperture ðra Þ: Clearly, there will be a trade-off between these variables that satisﬁes the performance speciﬁcation of the optical system of the laser printer. Up to this point we have not taken into account uniformity of the density of laser spot between adjacent lines. In real situation, the printed laser dot does not have equal intensity throughout the area, but rather has an intensity distribution, as discussed in Chapter 2, “Gaussian Beam Optics.” To have a uniformity of printing the overlap is necessary between adjacent dots composing a line. This is also true with the laser dots between neighboring lines. The intensity distribution of a laser spot impinged on the drum surface along a scanned line (x direction) is shown in Figure 7.16a. When a laser beam coming out of the rotating mirror scans through an entire line of the photoconductive drum, a simpliﬁed distribution of the transferred energy to the scanned line can be depicted as shown in the ﬁgure. Similarly, the printed laser area between the adjacent lines is regarded as the overlap of the spots. Therefore, the degree of the overlap between adjacent dot lines determines the uniformity of intensity distribution throughout the printed region. Typical overlaps of laser dots for four successive printed lines are illustrated in Figure 7.16b. The problem is how we can determine the degree of the overlap between successive lines along y direction by obtaining a criterion that can be expressed by Fmax and Fmin denoted as the maximum beam power and the minimum beam power of the laser spot, respectively. Let us determine the beam power distribution Fðx; yÞ to obtain the criterion. From Figure 7.16a, if a is the aperture of the collective lens of the laser printer and r 0 is the radial distance of point P from the center of the Airy disk, then, according to Fraunhoffer diffraction theory discussed in Chapter 2, “Diffraction,” the irradiance distribution within the spot is given by Iðr 0 Þ ¼ IðoÞ

2J1 ðkar 0 =RÞ kar 0 =R

2

ð7:8Þ

x0

x' x

FIGURE 7.16 Beam energy consideration for printing quality.

(a) transferred energy distribution along a printed line

y0

moving laser spot y y'

photoelectric drum

y

transferred energy x distribution

beam energy

∆y

line spacing

y

(b) laser beam energy distribution in an overlapped scanning mode

Φmin

Φmax

Φ total

468 Optomechatronics

Optomechatronic Systems in Practice

469

where Iðr 0 Þ is the irradiance incident on point P, IðoÞ is that on the center of the Airy disk, J1, is the Bessel function of the ﬁrst kind, k is the wave number, and R is the distance between the aperture and the image plane. In the above, IðoÞ is given by pa2 R2 where Ia is the irradiance at the aperture of the lens. To calculate the energy distribution, let us assume that the laser spot travels with its velocity long the drum axis, x as indicated in the ﬁgure. Equation 7.8 can then be rewritten as

IðoÞ ¼ Ia

Iðr 0 Þ Iðx 0 ; y 0 Þ Iðx 2 x0 ; y 2 y0 Þ ¼ ¼ IðoÞ IðoÞ IðoÞ

ð7:9Þ

where x0 and y0 are the center coordinates of the moving laser spot with velocity ux : If the moving coordinate x0 is denoted by x0 ¼ uxt, Equation 7.9 can be rewritten as Iðr 0 Þ Iðx 2 ux t; y 2 y0 Þ ¼ IðoÞ IðoÞ

ð7:10Þ

Then the energy Fðx; yÞ per unit area impinged at an arbitrary point ðx; yÞ in a scan line can be calculated from ðþ1 ð t1 Fðx; yÞ ¼ IðoÞIðx 2 ux t; y 2 y0 Þdt < IðoÞIðx 2 ux t; y 2 y0 Þdt 21

0

where t1 ¼ w=ux and w is the width of the drum. Knowing the energy density function Fðx; yÞ at a point (x,y), we can deﬁne a spot energy criterion function F by F¼

Fmax 2 Fmin Fmax þ Fmin

ð7:11Þ

In Figure 7.16b Fmax and Fmin values are determined by using

Ftotal ¼ Fðx; 0Þ þ Fðx; DyÞ þ Fðx; 2DyÞ þ Fðx; 3DyÞ þ Fðx; 4DyÞ By choosing F value in an appropriate way, say F , 0.1, we can now determine a desired line spacing Dy: Based on the observation of these results, it can be said that there appears to be an optimal line spacing Dy to satisfy a given F criterion. It is noted here that the velocity of the moving spot is controlled by adjusting the scanning speed while the line spacing is controlled by adjusting the drum velocity.

Optical Storage Disk Ever since the optical disk was envisioned in 1958 by an engineer named D.P. Gregg, nowadays it has become the most popular means of

470

Optomechatronics

information storage and retrieval. During recent years, a variety of optical disks or optical disk systems have been developed and introduced successfully into the market due to the diversiﬁed areas of their applications such as computer peripherals as a storage device, and audio/video consumer applications. These include compact disk readonly memory (CD-ROM), compact disk rewritable (CD-RW), digital versatile disk (DVD) and minidisks (MDs). Among these devices, DVDs and recently developed advanced storage magneto optical disks (ASMO) have greatly upgraded the recording capacity and transmission time. The DVD has several different types of recording layer; single-sided single layer, double-sided single layer, single-sided double layer, and double-sided double layer. All of these types have high information recording capacity, the largest capacity being 17 GB for the case of double-sided and double layer type. In this section we will treat an optical disk system and discuss its ability to correctly retrieve the recorded data with a designated speed. To this end we will introduce a general description of the disk conﬁguration about the recorded state of data, and the operation principles of data retrieval. Since the retrieval processes are involved with seeking the target track during data access and positioning of a laser spot on the track and in focus, the control functions will be described in some detail together with their drive architectures. System Conﬁguration In an optical disk, large quantities of data are recorded in the form of digital signals along concentric spiral tracks. Recording such signal forms is physically realized by making a sequence of pits (indentations) on the disk surface along each spiral shaped track, as shown in Figure 7.17a. The disk consists of an information stored layer, composed of pits and lands, a polycarbonate substrate, label layer, and a protective layer. The pits are small areas having contrast with respect to the surrounding mirror surface. This makes the intensity of the light reﬂected from the pits differ from that reﬂected from the mirror surface. The variations of the optical signal contain the information to regain the video, audio, or data signal. The dimensions of the pit (h p,wp, ‘p) and the distance between tracks pd, and the distance between the adjacent pits are all important to determine recording density. The CD has typically pd ¼ 1:6 mm, and the length of the smallest pits 0.83 mm, while DVD have 0.74 and 0.4 mm, respectively. Table 7.1 shows some speciﬁcation on pit dimension, laser wavelength and rotation speed for various optical disks. It is noted that the CD and DVD have high rotation speed, that is, high data transmission, as compared with the other two devices. When retrieving the data recorded in this form correctly, it is essential to make the laser spot follow the track accurately and at the same time focus the spot on to the surface of the pit, while the disk is rotating.

FIGURE 7.17 Conﬁguration of optical disk.

Optomechatronic Systems in Practice 471

472

Optomechatronics

TABLE 7.1 Some Speciﬁcations of Various Optical Storage Disks Device Video disk Compact disk Disk memory Double video disk

Pit Dimension (mm) wp ¼ 0:4; hp ¼ 0:1; ‘p ¼ 0:4 – 2:0 wp ¼ 0:5; hp ¼ 0:15; ‘p ¼ 0:8 – 3:0 — wp ¼ 0:28; hp ¼ 0:12; ‘p ¼ 0:4 – 2:05

Disk Thickness (mm)

Laser Wave Length (nm)

Rotation Speed (rpm)

2.4

l ¼ 780

210 –480

1.2

l ¼ 780

6000 –7000

2.4–30 1.2

l ¼ 685 l ¼ 635 – 650

300 –600 2400 –5600

In order to achieve these tasks, the optical disk system has the following major parts: (1) Optical shaping and focusing system. (2) Optical pickup and positioning system. (3) Servo system for disk rotation. The system conﬁguration of the optical shaping and focusing system is shown in Figure 7.17b. The optical system contained in the pickup is composed of a laser diode as a light source, a diffraction grating, a collimator lens, a polarizing beam splitter, and a l=4 plate and objective and cylindrical lenses as beam shaping as well as data retrieving units, respectively, voice coil motor (VCM) as an actuator for focusing the objective lens in the direction normal to the disk surface, a four quadrant photodiode integrated chip as a feedback sensor for autofocusing, and two single photodiodes as a feedback sensor for track following. The grating unit and two single photodiodes here are used for the laser spot to follow the series of pits that can be achieved by accurately positioning the optical pickup in the radial direction. We will focus our discussions on the operation principles of shaping, retrieving, and sensing of the optical system, and then discuss about the focusing servo system and the track following servo system. Optical System for Focusing Referring to Figure 7.17b this system consists of a semiconductor laser, a beam splitter, a collimator lens, an objective lens, a condenser lens and ﬁnally a quadrant photodiode IC. The principle of signal detection during the data retrieving process can be described as follows. A laser beam is emitted from the source whose diameter is very small, only approximately 2 mm. In the beam focusing stage this beam is transmitted through a beam splitter and keeps spread out to a certain diameter along

Optomechatronic Systems in Practice

473

the optical axis, as we might recall the beam spreading discussed in Chapter 2, “Gaussian Beam Optics.” The diameter is shaped by a collimator which makes the spread beam into a parallel beam having a desired diameter D. This collimated beam enters the objective lens which then sharply focuses the refracted beam on to the track surface. When the focused beam hits the mirror surface of the optical disk, the beam reﬂects from its surface and will return toward the objective lens, thus generating a strong reﬂective signal. However, when the beam impinges upon a pit, the amount of the reﬂected light will be reduced due to light scattering as a result of the interaction between the beam and the indented surface. When the beam hits a pit, reﬂective diffraction occurs, causing a phase change in the beam. This results in light scattering. The difference between the reﬂected light from the disk mirror surface and the light scattered from the pit makes it possible to discriminate the two states, thus enabling us to read back the data from the stored disk. One important thing to be noted is that, since the detected light signal is dependent upon the diffraction limit, it is important to keep the numerical aperture NA to an appropriate value according to Chapter 2, “Diffraction.” This is shown in Figure 7.18. If we express the laser spot diameter Dw as the Airy disk diameter at full width at half maximum (FWHM) then Dw is given by Dw ¼

l 2NA

ð7:12Þ

where l is the wavelength of the laser and NA is the numerical aperture of the optical lens system. It is observed that Dw can be made small by making NA large. However, it increases with larger wavelength of the spot. Therefore, it is necessary to compensate for the larger wavelength of the laser by increasing NA in order to keep the spot diameter constant. As already discussed in the previous chapter, the consequence of a larger NA causes depth of focus or vertical tolerance to decrease according to the following relationship df ¼

l 2ðNAÞ2

ð7:13Þ

Clearly, the depth of focus is sharply decreased when NA increases. To illustrate this let us take an example for the case of a semiconductor laser AlGaAs. Since NA is typically 0.47 for this laser, we have for the beam spot diameter

Dw ¼

l 0:78 mm ¼ 0:83 mm ¼ 2 £ 0:47 2NA

f

Dw q

D

df

FIGURE 7.18 Principles of beam focusing and tracking.

(a) beam waist and Airy disk

objective lens

optical disk

Airy disk

FWHM

(b) three diffracted beams

laser beam

optical disk

diffraction grating

objective lens

recorded layer

474 Optomechatronics

Optomechatronic Systems in Practice

475

and for the depth of ﬁeld df ¼

l 0:78 mm ¼ ¼ 1:8 mm 2 2ðNAÞ 2 £ ð0:47Þ2

Once these are determined, we will now be able to determine the diameter and focal length of the objective lens, D and f, respectively, shown in the ﬁgure that satisfy the error speciﬁcation. Another method to express the laser spot diameter is to use Gaussian laser beam waist as discussed in Chapter 2, “Gaussian Beam Optics.” If in this optical system the same approximations such as far ﬁeld lens location and very small divergence angle of the laser beam incident to the lens can be used, then Dw can be approximated by Dw ¼

l pNA

For the same semiconductor laser with an identical NA, we have Dw ¼

0:78 mm ¼ 0:53 mm 3:14 £ 0:47

which is smaller than the diameter obtained in the above. Returning to Figure 7.17b, detection of optical signal coming from the disk surface utilizes an optical system that consists of a circular lens, a cylindrical lens, and a photodiode sensing system. As we recall from the diffraction theory discussed in Chapter 2, “Diffraction,” the intensity of the diffracted light is dependent upon the depth of pit. It can be shown that the maximum intensity of the diffracted light can be obtained when the depth is l=4; i.e., hp ¼

l 4nt

ð7:14Þ

where n is the refraction index of the disk (glass). Since nt ¼ 1:55 for the disk (glass), we have for the pit’s depth hp ¼

0:78 mm ¼ 0:126 mm 4 £ 1:55

We can see that this value is comparable with those of the values listed for various disks in Table 7.1. Most of disks are speciﬁed to have the pit’s depth to be less than this value in case of adopting the split photodiode due to track following problem. The relationship given in Equation 7.13 will be discussed in detail under “Confocal Scanning Microscope”. Table 7.2 summarizes the focus detection method with their measurement principles. The detection system shown in Figure 7.19 utilizes astigmatic phenomenon much of which have been already discussed in Chapter 2, “Aberration”, and Chapter 6, “Automatic Optical Focusing”. For completeness of discussion the detection principle is shown here again. According to the principle, when the disk surface is in focus, the zeroth-order diffraction

c

d

B

(a) detection of beam focus

quadrant (focus)

b

a

FIGURE 7.19 Detection of beam focus and tracking error signals.

single photodiode

A

tracking error

single photodiode

(b) principle of three beam method

error amplitude

time

476 Optomechatronics

Optomechatronic Systems in Practice

477

TABLE 7.2 Focus Error Detection Methods Type of Beam

Principle

Focus Error Detection

Converging Beam

Detection of beam shape Beam density variation Symmetry of beam position Beam collimation

Astigmatism, Beam size Edge, Edge prism Off-axis beam, Skew beam Critical angle

Collimated Beam

becomes dominant the shape of the output beam being circular. In this case, the resultant output signal VF will be zero. However, when the pit surface is out of focus, the shape of the output becomes elliptic. Thus, the resultant signal will not be zero. When the lens is positioned closer than the focal distance the shape becomes elliptic in vertical direction, indicating that ia þ ic . ib þ id : On the other hand, when the lens is situated farther from the focal position, the shape becomes elliptic in horizontal direction, which indicates that ia þ ic , ib þ id : In summary, the detector output varies, depending on the focused state. Deﬁning VF ¼ ðia þ ic Þ 2 ðib þ id Þ; we have 8 .0 when too close > > < VF ¼ 0 when focused > > : ,0 when too far The relationship between the output voltage and the lens distance from the focal position has already been shown in Figure 7.19a. Beam Tracking Tracking the series of pits is necessary, since there are several external disturbances coming into the optical disk such as: (1) (2) (3) (4) (5)

Warping, misalignment, and tilt of the disk. Thickness variation of the disk. Optical misalignment Spherical aberration of lens. Lens spot noise.

Most of these noises affect the tracking performance. Therefore, this renders it necessary to detect the tracking error and to compensate it by some means. There are several ways of the error detection as listed in Table 7.3. The ﬁrst one is to use the variation of intensity of the diffracted beam from the pits. The second is to oscillate the beam across the pits and detect the resulting phase difference between the oscillation signal and the detected signal. The third one is to use the positions of three beams diffracted from

478

Optomechatronics

TABLE 7.3 Tracking Error Detection Methods Number of Beam One beam Three beam

Principle

Tracking Error Detection

Diffraction from pits Wobbling Diffraction from grating

Push-pull, Heterodyne Beam wobbling, Dish wobbling Three-beam

the track surface. The three beams are generated by a diffraction grating as can be seen from Figure 7.18. We will discuss here only the three-beam method in brief. Referring to Figure 7.18b, a diffraction grating is placed in front of the laser diode. The laser beam is then separated into three components of diffracted light, m ¼ 0th order, and m ¼ ^ 1st order. The higher orders are not shown here for simplicity. The zeroth order having strongest intensity is directed toward the pits on these tracks which carry the information to be read. The other two ﬁrst-order beams are much weaker in intensity than the zeroth and are projected on either side of the track. These three beams reﬂect or diffract back from the same areas and are projected on the surface of the photosensors. Figure 7.19 illustrates how three beams are projected on the track. In the ﬁgure the gray color circles denote the three light beams. The central beam is the zeroth order which is the major beam, while the other two beams offset from the central beam are the ^ 1st beams. To receive these, a sensor system is employed, which consists of a quadrant photodetector and two single photodetectors, as shown in the ﬁgure. The quadrant detector is used for detecting the major beam as already discussed in Figure 7.19a, while the other two single photodetectors A and B are indicative of the focused state measure the positions of the two ^ 1-order beams on either side of the track. Figure 7.19b shows how the tracking servo signal varies with time, depending on the positions of 1 , the signal which is the the two beams. When the track position is in state W difference between the outputs from A and B becomes negative; but when 3 , the signal becomes positive. The state W 2 indicates that the it is in state W output is zero, signifying that the track is being read correctly. Disk Servo Control System Realization of reading high density data places extreme demands on the performance of the focus, tracking, and access servo systems. In other words, in order to keep the laser spot following the recorded layer of the disk with high speed while keeping the spot in focus it is necessary to have precise control of the motion of the pickup head within a required accuracy speciﬁcation. There are several servo loops involved with this retrieval

Optomechatronic Systems in Practice

479

process. These include control of the laser spot position, autofocusing of the laser spot, and control of disk rotational motion. In the case of recording, there are two more servo systems such as laser power control and disk tilt motion control, but here we will consider only those of the retrieving case. Figure 7.20 depicts the architecture of the servo system. There are four servo systems needed for operating an optical disk. The ﬁrst system is to precisely position the beam spot on the focus plane which is the disk information recorded layer. As indicated in the ﬁgure, this spot positioning is made by the control of the vertical movement of the objective lens. This guarantees that the focal point of the laser beam is precisely on the recorded layer. The actuator that satisﬁes the above requirements popularly adopts the VCM. The second servo system is to keep the spot following the recorded track on the disk. This can be done by the control of the ﬁne movement of the pick-up unit in radial direction. An independent VCM actuator is adopted to avoid any possible motion coupling that might occur between focus and track directions. The third one is to control the radial position of the pickup unit in coarse mode when accessing data from another track. This servo performs large displacements and slowly follows the ﬁne actuator movement while tracking. The fourth one in the ﬁgure is to control the speed of the disk motor such that the optical pickup follows the track at a constant linear velocity. To achieve the functional requirements mentioned above the actuator should satisfy the following mechanical requirements; (1) light weight (2) independent movement in two directions; focus and track (3) high natural frequencies in disk vibration. In addition to these physical requirements, the servo systems should have their controllers be robust to noise and attenuate external disturbances such as eccentricity of the disk and spindle, axial tilt, and so on while preserving dynamic stability. Therefore, controller design that can achieve this objective has been a great concern among researchers. Since this design requires somewhat advanced control techniques we will not discuss them here. We will consider here a very simple control of a ﬁne radial movement of the pickup unit for track following. Due to disk rotation and periodicity of pits on the track, the characteristics of external noises, if any, are normally described in frequency domain. To this end we will discuss the controlled characteristics in time and frequency domains. A general control block diagram is depicted in Figure 7.21. The control system consists of a pickup unit that includes the optical lenses and components, a laser spot measurement sensor, a compensator (controller), a servo drive, and a VCM actuator. The external disturbances entering the system and the measurement noise are also included in the diagram. However, we will not treat here such disturbance and noise, since some advanced knowledge is required to analyze the frequency response of such a control system. Rather, we will consider only the pickup system for simplicity.

motor

FIGURE 7.20 Conﬁguration of the optical disk servo system.

recorded layer

W

actuator

PD

optical disk

data in

speed control

sledge control

focus control

tracking control

signal processing

actuator

LD

actuator

data memory

m processor

program memory

audio output

480 Optomechatronics

I/O port

+_

compensator

FIGURE 7.21 A block diagram of the ﬁne track servo control system.

desired track position voice coil motor

measurement noise

tracking sensor

fine track servo driver

pickup optical unit

disturbance

actual track position

Optomechatronic Systems in Practice 481

482

Optomechatronics

Suppose that the dynamics of the VCM and optical pickup unit for ﬁne movement are characterized by the equation given in Equation 4.57. Let its transfer function be given by Gps ðsÞ ¼

334 £ 103 s2 þ 577s þ 3:34 £ 105

ð7:15Þ

When a simple controller such as PID is employed, the controller transfer function is given by Gc ðsÞ ¼ kp þ ki

1 þ kd s s

where kp , ki, and kd are the proportional, integral, and derivative controller gains, respectively. If the dynamics of the spot measurement sensor and that of the VCM driver for track following are neglected, the control system becomes a unity feedback. By combining Equation 7.15 with the above controller equation, we have the following open loop transfer function; Gop ¼

334 £ 103 ðkd s2 þ kp s þ ki Þ sðs2 þ 577s þ 3:34 £ 105 Þ

For this open loop transfer function, we will consider two cases of closed loop control; case 1, proportional control only and case 2, PID control. Figure 7.22a plots the frequency and time domain responses of the P controlled system. In the ﬁgure, bandwidth is deﬁned by the range of frequencies over which the closed loop magnitude ratio exceeds 2 3 dB where dB is deﬁned as 20 log10 k for some quantity k. First of all, we can see that the system is always stable. When kp ¼ 100, it is noted that the bandwidth of the control system is greatly increased as compared with that obtained without feedback. It reaches almost more than 1 kHz without expenditure of the system phase, as expected. This implies that the speed of response as well as the system resonant frequency increases. The time domain ﬁgure shows typical underdamped oscillatory response characteristics of a second-order control system. When a PID controller with arbitrary chosen gain parameters ﬁxed at kp ¼ 100, ki ¼ 100, kd ¼ 0:5 is used, as shown in Figure 7.22b, the increase in system bandwidth is pronounced reaching more than 10 kHz. The magnitude and phase plots show that the system gets more stable. In this case, no signiﬁcant oscillation can be seen from the time domain response. One important thing to be noted here is that in practice we cannot design the control system response characteristics without consideration of the effect of external disturbance and measurement noise. This is because one of the important objectives is to attenuate such noises while improving

magnitude (20log|G|), dB

phase ( G), degree

0.0

0.5

1.0

1.5

2.0

−180

−135

−90

−45

0

−150

−100

−50

0

10

−1

10−1

10

0

100

5

10

1

101

15 10 time (t), msec

3

103

10 10 frequency (f ), Hza

2

102

10

4

104

20

10

5

105

FIGURE 7.22 Frequency and time domain responses of the track following system.

(a) bode diagram with P controller; kp = 100

amplitude (y), μm

25

10

6

106

magnitude (20log|G|), dB phase ( G), degree 0.0

0.5

1.0

1.5

2.0

−180

−135

−90

−45

0

−150

−100

−50

0

0

10−1

10−1

100

100

5

101

101

(b) bode diagram with PID controller; kp = 100, ki = 100, kd = 0.5

amplitude (y), μm

0

103

10

15 time (t), msec

102 103 frequency (f ), Hza

102

104

104

20

105

105

25

106

106

Optomechatronic Systems in Practice 483

484

Optomechatronics

system accuracy and stability. Here, what has been considered was only the ﬁne tracking system for controller design.

Atomic Force Microscope The surface topology can be probed with atomic level accuracy using several microscopic measurement principles. One device achieving such probing is the scanning probe microscope (SPM) which can measure individual atom sizes as small as a carbon atom (approximately 0.25 nm) and as large as the cross section of a human hair (approximately 80 mm). There are two popularly used types of SPM in imaging surface features such as shape and dimension; atomic force microscope (AFM) and scanning tunneling microscope (STM). STM has appeared ﬁrst among all SPM devices and utilizes the current ﬂow occurring due to the quantum tunneling phenomenon between a probe tip and the surface of a sample to be measured. Since this can be used only for dielectric materials, it has limited applications, which is a drawback of this device. On the other hand, AFM can be used for nonconducting material as well, because its measuring concept employs the interactive atomic force between a probe tip and a sample surface. Besides, the AFM can work well in ﬂuid if tip material and sample preparation are carefully chosen, and thus is successfully utilized for bioresearch applications. Table 7.4 summarizes resolution, imaging type, and imaging principle of various microscopes. Measurement Principle AFM was introduced by Binnig, Quate, and Gerber in 1986 as a topographic imaging method for conductors and insulators. The AFM employs the essence of typical optomechatronic characteristics in that it fully utilizes the fusion of optical and mechatronic technologies. In principle, the AFM operates measuring the atomic force, attractive or repulsive, induced between a deformable mechanical element (probe) and the surface of a sample. The measurement is made optically to detect such a force. As schematically illustrated in Figure 7.23a, the system possesses a probe, a laser source, and the optic elements; a detector, and a scanning system. The scanning system consists of a z direction scanner and a two-dimensional (2D) x-y scanner. Each of them is capable of scanning in two modes; course motion and ultra precision motion, thus having two different translators. The scanning system moves the probe in a raster scan fashion by dragging its tip over the sample surface of interest, for the measurement needs to be made over a certain region of a sample. Let us examine in more detail the AFM system in order to study its measurement principle by utilizing Figure 7.23b. It is composed of several optomechatronic components; a cantilever probe, a laser source; and optical elements, a position-sensitive detector (PSD), a piezoelement actuator for

Light interference 3D

— 3D

Transmitted light 2D

Electron emission 2D

Interatomic force 3D

All

All

Vacuum

Vacuum or air Tunneling current 3D

Far-ﬁeld Non-contact

Near-ﬁeld Non-contact

Far-ﬁeld Non-contact

All

.500 nm (for visible light) Far-ﬁeld Non-contact

Near-ﬁeld Non-contact

.12 nm

Near-ﬁeld Contact or non-contact All

.5 nm

.0.1 nm

Phase measuring Interferometry

.0.1 nm

SNOM

STM

AFM

SEM

Optical Proﬁlometry Confocal Scanning Microscope

SEM, Scanning Electron Microscope; OM, Optical Microscopy; SNOM, Optical Scanning Near-ﬁeld Microscopy. Source: Sitti, M., Survey of nanomanipulation systems, IEEE-Nano 2001, 75–80, 2001. q 2005 IEEE.

Visual object size (resolution) Imaging type Interaction with object Imaging environment Imaging principle Imaging dimension

Microscope Properties

TABLE 7.4 Characteristics of Various Microscopes

Light-matter interaction 2D

Air or liquid

Far-ﬁeld Non-contact

.100 nm

OM

Optomechatronic Systems in Practice 485

detector

FIGURE 7.23 Atomic force microscope.

display unit

computer

piezoelectric X-Y scanner

coarse X-Y motion translator

sample

probe

laser source

(a) overview of the system structure

coarse z-motion translator

piezoelectric z-scanner

y xy

z axis piezoactuator

x-y piezoelectric stage

AFM tip

(b) measurement configuration

sample

z

AFM cantilever

laser diode position sensitive detector

486 Optomechatronics

Optomechatronic Systems in Practice

487

vertical axis and an x-y servoing stage for scan motion. Basically, the AFM is a system positioning an atomically sharp conducting tip in a direction vertical to the surface of an object (sample) using a piezoelectric actuator in response to the induced atomic force. A certain amount of force is induced, when the gap between the probe tip and the sample surface is kept within a speciﬁed region. The atomic force between the atoms is called Van der Waals force. This force causes the deﬂection of the cantilever whose displacement is dependent upon the gap between the tip and sample surface. As the cantilever bends due to change of the gap, the position of the laser beam reﬂected from the cantilever surface shifts according to the deﬂection. In other words, the laser incident on the top surface of the cantilever is deﬂected to a certain point at the photodiode sensor, different from the spot when the cantilever is undeﬂected. The detected shift amount is used to drive the piezoactuator so as to make the laser spot to a state of the undeﬂected cantilever, which ensures a constant, desired, interatomic force. The instantaneous displacement of the probe (cantilever) driven by the piezoactuator is then registered, which gives the z directional topology of the sample at the speciﬁed x-y location. When this procedure is repeated for the whole sample region of interest by scanning motion of the AFM system, the recorded history of the cantilever motion depicts the topographic data of a measured sample. To understand in more detail the interactive force between the atoms brought together let us examine the Van der Waals curve, Fvd, shown in Figure 7.24a. The ﬁgure illustrates the relationship between interatomic force vs. distance curve. The distance here indicates tip to sample separation as shown in the ﬁgure. When the separation is large, the attraction force is very weak. However, the atoms come closer together, i.e., z ¼ zp the attraction force gradually increases, which is the negative force ð2Fvd Þ shown in the ﬁgure. However, when z , z p, the attraction increases very sharply until their electron clouds begin to repel each other electrostatically. As the interatomic separation continues to decrease, the repulsion progressively decreases the attraction and the interatomic force ﬁnally becomes zero when ˚ . When the atoms are in contact, that is, the the separation approaches 2 A AFM tip makes physical contact with the sample, the Van der Waals force becomes positive. This region is called the “contact” state within which the slope of the cantilever pushes the tip against the sample and the cantilever deforms due to the repulsive force. As can be seen from the ﬁgure, the interaction force for the cantileversample system is highly nonlinear with the gap distance. In general, the interaction force is found to be expressed for the interaction between a sphere representing the probe and planar surface representing the sample; Fvd ¼ 2

A B þ 8 z2 z

ð7:16Þ

where the values of A and B are 4.6625 £ 10227 kg m3/sec2 and 1.133 £ 10279 kg m9/sec2, respectively, z is the separation distance between the tip

FIGURE 7.24 Van der Waals force.

atomic force, (nN) Fvd

non-contact

z∗

attractive force

distance (z) (tip-to-sample separation)

repulsive force

(a) inter-atomic force vs. tip displacement

0

contact

intermittentcontact

(b) geometry of the cantilever

stage

object

z

(cantilever support) piezoactuator

Fvd

AFM tip

w

h probe tip

cantilever

cantilever

z

488 Optomechatronics

Optomechatronic Systems in Practice

489

and the surface of the sample. The resulting force in general shape has been already illustrated in Figure 7.24a, and the force is typically ranged within 0.01 to 10 nN. The force magnitude depends greatly upon the separation distance. This magnitude is a critical parameter for operation of AFM, since it is utilized as feedback information. Several methods of operation depending upon the force magnitude have been developed but we will consider here two operation modes; contact mode and noncontact mode and intermittentcontact mode. When the force magnitude is large, contact mode is advantageous since the probe output can be highly sensitive. However, when force is small, its sensitivity becomes really small, due to small cantilever deﬂection, which is difﬁcult to measure. In this case, use of a dynamic noncontact mode is advantageous. In the static contact mode, the force translates into a deﬂection of the cantilever as depicted by Equation 7.17.

d¼

F keff

ð7:17Þ

where the effective stiffness is given by keff ¼

Ewh3 4‘ 3

In the above equation, E is the Young’s modulus of the cantilever, and w and h are the width and thickness of the cantilever, respectively, as shown in Figure 7.24b. As can be seen from Equation 7.17 the cantilever needs to have a small value for keff in order to have large force sensitivity, requiring that the cantilever is much softer than the bond between bulk atoms in the tip and sample. The stiffness range is somewhere keff ¼ 0.01 to 1.5 N/m, although it varies with the material of the sample. For this stiffness range the force varies within the order of 1 to 10 nN. The sensitivity of the deﬂection is limited by the mechanical vibration, pointing stability of laser, and noise which normally comes from thermal oscillation of the cantilever and other electronic circuits. In the contact mode, the z-axis scanning motion accompanies the dynamic motion of the cantilever much more ﬂexible than the z scanner. In this situation, it is appropriate to consider only the cantilever motion and it is desirable to control its motion by actuating the z-axis piezostage in a vertical direction. The control objective is to keep the inter atomic force Fvd at a desired value, i.e., the separation distance at a constant value which is nearly zero. To achieve this, it is necessary to construct a force displacement control based on the PSD signal. Figure 7.25 shows its control conﬁguration which is composed of a controller, a piezodriver, and a piezoelectric actuator and AFM system. Since the stiffness of the z scanner supporting the cantilever is much higher than that of the cantilever itself, we will neglect the scanner dynamics. Therefore, it will sufﬁce to consider only the motion of the

490

Optomechatronics probe

sample surface contact mode

desired force zd (z + za) (force) motion

controller

driver laser

piezo actuator

keff

b

photo detector z

m

za

artifact

cantilever

piezoactuator

zd

FIGURE 7.25 The AFM tip control in contact mode.

cantilever. The equation governing its motion is given by m

d2 z dz þb þ keff z ¼ Fp þ Fvd 2 dt dt

ð7:18Þ

where z is the vertical displacement of the cantilever from an arbitrary position, Fvd is the Van der Waals force, Fp is the control force to drive the piezo stage, and m, b and keff the mass, the damping coefﬁcient, and the effective stiffness of the cantilever, respectively. When the sample artifact to be scanned has a varying height za along the scan directions, the control scheme is aimed at maintaining the desired gap (force) at a nearly-zero value zd between the probe tip and the surface of the artifact (contact mode) by controlling the displacement of the cantilever. When the separation becomes larger than that of the contact mode, an operation mode of AFM called the “noncontact AFM” is utilized. In this mode, the spacing between the tip and the sample is on the order ˚ , and the interatomic force becomes attractive. The of 1/10 , 1/100 A order of the attractive force is very small, generally about 0.1 , 0.01 nN, and therefore measurement is not easy, furthermore, it is susceptible to noise. This method is suitable for measuring soft and fragile samples, since the tip does not touch them. To overcome the difﬁculty in the measurement of a small force, a vibrating cantilever technique is adopted in which an AFM cantilever is vibrated near the surface of a sample. This method has two basic modes; amplitude

Optomechatronic Systems in Practice

491

modulation (AM) and frequency modulation (FM). In AM mode, the actuator is driven by ﬁxed amplitude at a ﬁxed frequency. When the AFM tip approaches the sample surface, interaction causes a change in both amplitude and the phase of the cantilever. These changes are used as a feedback signal to drive the actuator. In FM mode, however, the cantilever mounted in the z-axis piezoactuator is kept oscillating with a constant amplitude at its resonant frequency by an oscillator control ampliﬁer. Let us consider how this vibration is related to measuring the sample topology. As in the contact case, we will assume that the cantilever vibrating in the air can be modeled as a damped harmonic oscillator. If the displacement from the equilibrium position of the tip is denoted by z, as shown in Figurez 7.26, the equation of motion of the vibrating cantilever is governed by m

d2 z dz þb þ keff z ¼ Fo sin vo 2 dt dt

ð7:19Þ

where b is the damping coefﬁcient, Fo is the magnitude of actuating force, vo is the operating frequency, which is slightly off from the natural frequency, vn : In this forced oscillation, the steady state motion of the cantilever is described by z ¼ Ao sinðvo t þ wo Þ

ð7:20Þ

probe F0 sin w0 t

piezoactuator (support)

noncontact mode

keff

vibration magnitude (A)

b

with interaction

A

z

m

za

artifact

∆A no interaction between tip and sample ∆w w' w n

wo

frequency (w)

FIGURE 7.26 Variation of natural frequency due to Van der Waals force.

cantilever

492

Optomechatronics

The vibration amplitude and phase at steady state are given by " # Fo =mv2n 2jvo =vn 21 Ao ¼ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ; wo ¼ tan 1 2 ðvo =vn Þ2 ðv2 2 v2 Þ2 þ ð2jv =v Þ2 n

o

o

ð7:21Þ

n

where from Equation 7.19, the natural frequency vn and damping ratio, j, respectively, are given by sﬃﬃﬃﬃﬃﬃ k b ð7:22Þ vn ¼ 2p eff ; j ¼ pﬃﬃﬃﬃ m 2 mk Now, let us consider the case when the tip approaches the surface. In this case, the stiffness keff varies with the separation z, because the Van der Waal’s force (attraction) changes with z as can be seen from Equation 7.16. Under this consideration, keff of the cantilever varies according to 0 keff ¼ keff 2

›Fvd ›z

ð7:23Þ

where k0eff is a new stiffness of the cantilever. Therefore, natural frequency, vn, varies according to the relationship given in Equation 7.23 and now new frequency becomes v0n : This phenomenon is plotted in Figure 7.26. Due to this change, the motion of the cantilever becomes changed and is obtained from Equation 7.19 and Equation 7.23 as follows: 0

z ¼ A0o sinðvo t þ w0o Þ þ At e2jvn t sinðv0n t þ wt Þ

ð7:24Þ

where A0o and w0o are the new steady state amplitude and phase, respectively, and At and wt are those of transient oscillation. It is noted that in the noncontact AFM regime the force gradient becomes greater than zero, and, therefore, the keff becomes smaller than the cantilever stiffness k0 : The resulting new frequency v0n gets smaller than vn as indicated in Figure 7.26. This implies that the change in resonant frequency of a cantilever can be used as a measure of change in force gradient. This measure reﬂects change in the tip-to-surface separation representing sample topology. In this mode, the piezoelectric actuator drives the z motion stage holding the cantilever in such a way that the resonant frequency or vibrational amplitude of the cantilever be kept constant. Keeping the resonant frequency or amplitude constant implies that the AFM also keeps the average tip-to-sample distance constant. The case of controlling the oscillation frequency is shown in Figure 7.27. The cantilever here is kept at its resonant frequency vn or slightly off the resonance. According to Equation 7.23 if any frequency change is detected due to tip-to-sample distance variation the controller will try to bring the frequency back to its original value. Here, the ampliﬁer has an automatic gain control circuit which keeps the vibration amplitude at a constant level. Any change in oscillation frequency is detected by an analog FM demodulator circuit or any other methods such as frequency counter, tunable analogue FM detector, and gated timer. These methods are found to normally measure a frequency shift

Optomechatronic Systems in Practice

493

oscillator control

photo detector FM demodulator

servo driver

laser

cantilever z stage (piezoactuator) sample

FIGURE 7.27 Schematic of the FM measurement system and control.

of 0.01 Hz at 50 kHz with 75 Hz bandwidth. This noncontact method can only be applied in certain cases, since the force is very small as compared with that of the contact mode. Therefore, the measurement is more susceptible to noise such as (1) thermal expansion of the cantilever and (2) oscillation control ampliﬁer. Intermittent contact mode is in principle similar to noncontacting mode. The difference is that in this mode the probe approaches closer to the sample surface during vibration. Cantilever Deﬂection The cantilever and the imaging probe are critical to the determination of lateral resolution and force to be applied to the sample surface. Therefore, care needs to be taken in designing the shape and dimension of the cantilever and tip. Stiffness is determined by the shape, size, and material of the cantilever, while those of the probe affect resolution of AFM. The stiffness normally ranges from 1/1000 to 1 N/m and a sharp probe tip has ˚ when a Rayleigh resolution a range of lateral resolution from 10 to 20 A criterion is employed. They are fabricated monolithically using silicon or silicon nitride by photolithography. Two types are frequently used; (1) rectangular beam and (2) V-shaped, as depicted in Figure 7.28a and b. The rectangular beam has higher resonant frequency than that of the V-shaped, but has a disadvantage in that it is susceptible to vertical force. This deteriorates image quality in some situations. In contrast, the V-shaped is robust to vertical force variation and is widely used in actual systems. Its dimension is dependent on application but ranges in length from 100 to

494

Optomechatronics w

b

1 2

w

(a) rectangular beam

(b) V- shaped

noncontact/ tapping mode

force modulation

super sharp silicon

high aspect ratio

contact mode

silicon nitride

diamond-coated electrical AFM probes

(c) AFM tips Source: Products of AFM tips, Nanoworld Co. Ltd FIGURE 7.28 Two frequently-used cantilevers and types of probe tip.

200 mm, width 10 to 40 mm and thickness 0.3 to 2 mm. Various types of probe tip are shown in Figure 7.28c. Let us consider the deﬂection of a cantilever of rectangular shape and its measurement. The conﬁguration of the deﬂected AFM is shown in Figure 7.29. The cantilever reﬂects the incident laser beam toward a position-sensitive detector (PSD). When the cantilever is not deﬂected, the reﬂected beam hits point o0 ; the center of the PSD. However, when it is deﬂected from its original neutral axis with displacement z, the beam changes its position from o0 to A, on the detector whose distance is denoted by d 0y : This displacement of the laser spot can be derived from the momentcurvature relation of a beam. It is assumed that the cantilever beam has bending modulus EI and length ‘ and is subjected to a concentrated force pz at the tip which is the interatomic force. This force can be decomposed into two components Px and Pz as shown in the ﬁgure. Here, Px is a friction force and Pz is the reaction force in vertical direction. The differential equation governing the beam displacement zðxÞ is expressed by use of the Structural Beam Deﬂection in Appendix A2.

EI

d2 z ¼ Mb dx2

ð7:25Þ

FIGURE 7.29 Optical measurement of the cantilever deﬂection.

vertical motion

z cantilever

laser

Pz

s Px

y' o'

x'

z θ

B

z'

A

x

PSD detector

o' detector

A

y'

δ y’

x'

Optomechatronic Systems in Practice 495

496

Optomechatronics

where E is the Young’s modulus of the beam, I is the area moment of inertia about the x-axis, and Mb is the bending moment acting at the arbitrary cross section within 0 # x # ‘: The Mb is given by Mb ¼ Px x þ Pz ð‘ 2 xÞ If the friction force is negligible, then Px ¼ 0: According to Appendix A2, the deﬂection and deﬂection angle at the tip in this case can be determined respectively by ztip ¼ Pz

‘3

3EI

u ¼ tan21

3 z 2‘ tip

ð7:26Þ

Once ztip can be calculated or known, the angle of cantilever deﬂection at the tip can be determined according to this relationship. The deformation of the tip itself, however, is negligibly small and therefore dy0 variation in the detector due to the contribution of the probe tip deformation can be neglected. With this assumption the PSD signal corresponding to the cantilever deﬂection can be easily determined by

dy0 ; r tan 2u

ð7:27Þ

where r is the distance from beam reﬂection point B on the top surface of the cantilever to the center o0 of the PSD. The deﬂection angle is doubled due to the geometry. Let us consider an AFM design problem. Suppose that the AFM system has resolution of 10 nm having a constant cross section and length ‘ ¼ 100 mm. In addition, the PSD has 5 mm resolution. For this we wish to determine the distance between the cantilever and the PSD, r. We will assume the condition that there is negligible friction between the tip and the sample surface, that is px ø 0 during the x-y scan motion. From Equation 7.26, the angular deﬂection u is determined to be

u ¼ tan21

3 z ¼ 1:5 £ 1024 rad 2‘ tip

Because the PSD resolution is 5 mm, we have from Equation 7.27, r¼

dy 0 ¼ 1:7 £ 1022 m tan 2u

This is the minimum distance r that can yield the speciﬁcation given above. Optical Measurement of Cantilever Deﬂection The accuracy of measurement depends upon various system parameters, such as optics, probe geometry, including tip shape and scanning method, in addition to z scanning and x-y scanning motions. The most inﬂuential factor is how to accurately measure the cantilever deﬂection. In this section, we will now elaborate upon some aspects of the optical measurement problem.

Optomechatronic Systems in Practice

497

Optics for Tip Deﬂection Measurement The tiny movement of the cantilever tip can be measured by several techniques, including laser feedback detection, interferometry, and optical beam deﬂection. The optical deﬂection method is widely used because of its simplicity and reliability in detection. This method utilizes a very simple optical arrangement composed of a laser source, beam shaping lens, and a position sensitive photosensor, split-cell type. The cantilever displacement can be measured by detecting the deﬂection of a laser beam reﬂected from the backside of the cantilever. The measurement resolution is limited by a diffraction limit of the laser spot size at the PSD. Figure 7.30 illustrates three basic conﬁgurations of the optical measuring system. In Figure 7.30a, a collimated Gaussian laser beam is incident on the back reﬂective side of the cantilever having a width w. The beam reﬂected from the ﬂat surface is focused by a beam collecting lens having a focal length f, and impinged upon the PSD. In this arrangement, the beam diameter denoted by Ds is assumed to be smaller than w, i.e., Ds , w and therefore, almost all light is reﬂected by the cantilever surface. In addition, the cantilever PSD laser source collimating lens

f

beam collecting lens

Ds

cantilever probe

(a) Ds < laser source

laser source lens PSD

lens

PSD x cantilever probe

(b) Ds ≈

cantilever probe

(c) Ds >

FIGURE 7.30 Optical arrangements for measuring the cantilever deﬂection.

498

Optomechatronics

bending phenomenon is assumed to be neglected, as it was in the previous section, which allows the cantilever to rotate around its base as a rigid body. Then, the tip displacement z will cause a change Ddps in the position of the intensity distribution at the split-cell detector as shown in Figure 7.31. Subtracting the two signals obtained at the two segments of the detector by using an op-amp, shown in the ﬁgure, yields the amount of tip displacement. If the angular displacement due to rotation causing tip displacement is assumed to be very small, the relationship between Ddps and z is given by z¼

wDdps 2f

ð7:28Þ

where f is the focal length of the beam collecting lens. From this equation the cantilever deﬂection can be determined from the sensor signal output. In deriving the concept of using a collimating laser beam as an incident light beam, it is assumed that almost all light is reﬂected from the surface of the cantilever, so that as large as possible optical power is transmitted to the sensor surface without deteriorating sensing resolution having the same Gaussian distribution as the incident beam. This method is only plausible when the cantilever has a large reﬂecting surface. When the beam diameter Ds is larger than or equal to w, which is the case of micro fabricated cantilever coated with gold, the collimated beam method is no longer suitable for tip detection, since a beam diameter of this small size cannot stay collimated over a wide range. The conﬁgurations shown in Figure 7.30b and c represent these two cases, adopting two different focusing methods; (b) a beam focused at the

i1 i0

intensity of the detected beam

i2

∆ dPS

FIGURE 7.31 Location of the beam incident to the detector.

position of the detected beam

Optomechatronic Systems in Practice

499 detector laser

y'

A

x' o'

z'

cantilever x probe tip z

Fp

kν

za

artifact FIGURE 7.32 Schematic representation of the simpliﬁed AFM system.

cantilever and (c) a beam focused at the detector. Determining the beam displacement Ddps at the detector in these cases will need to take into consideration of some geometric factors pertaining to the relation between laser beam diameter and cantilever beam width. We will not seek the analysis further here. The measuring performance of the AFM can be evaluated by constructing a feedback control system that can respond to change in surface topology of a sample. This AFM dynamic model shown in Figure 7.32 represents the contact mode and the Van der Waals force is modeled as a spring force whose spring constant is kv : The aim of the control is to follow the topology of a sample as accurately as possible in such a way that the Van der Waals force Fvd is kept at a desired value regardless of the artifact height. In order to investigate the performance of the AFM control system the z motion control of the cantilever is simulated. Using Equation 7.18, the control system is shown Figure 7.33a. A simulation for a PID control system is made for a 10 nm step change of artifact. The Simulink model and the system parameters are shown in Figure 7.33b, and Table 7.5, respectively. In this simulation a Gaussian white noise of amplitude 0.1% of the desired amplitude is included. The responses shown in Figure 7.34 indicate that, as the proportional gain increases, noise effect becomes pronounced which is obviously undesirable to maintain a “near” contact of the AFM tip with sample surface. When variation of I action is made for a ﬁxed kp and kd , we see that the speed of response increases with integral gain ki and ki affects steady state of the response. The effect of the derivative gain is that with slight increase in the gain the amplitude of the response is seen to become slightly ampliﬁed. When AFM is required to follow an arbitrary surface topology of the sample shown in the top part of Figure 7.35, the trend of AFM response for kp ¼ 0:1; and ki ¼ 3 is found to follow the

+ _

error

FIGURE 7.33 Feedback control of an AFM system.

(b) Simulink model

desired output

+−

K-

artifact

z-motion stage

sensor

Fp

measurement signal

++

piezo actuator

measurement Gain

artifact profile

PID Controller

PID

Gain

K-

PID controller

(a) Z motion control of the cantilever

(desired force) zd

surface profile ( za )

x' = Ax+Bu y = Cx+Du State-Space AFM model

cantilever system

grating stretch

Height

(actual force) z + za

500 Optomechatronics

Optomechatronic Systems in Practice

501

TABLE 7.5 The Parameters Used for the AFM Simulation Parameters

Symbol

Unit

Value

Mass Damping coefﬁcient Spring constant of cantilever Stiffness of the tip-artifact interface Width of cantilever Height of cantilever Length of cantilever Elastic modulus

m b k ki w h L E

kg Ns/m N/m N/m mm mm mm GPa

1.3271 £ 10211 9.8427 £ 1029 0.073 0.1 45 1.5 450 176

Source: Rutzel, S., et al. Proceedings, Mathematical, physical, and engineering sciences, 459:2036, 1925~1948, 2003.

desired surface with some lag. This needs to be improved by making an appropriate change in AFM parameters or feedback controller design. Since with this gain combination noise effect is pronounced, it seems to be necessary to reduce kp gain and increase ki gain, while setting kd gains to zero. Control of the Scanning Motion The performance of the AFM is largely dependent upon that of the ultraprecision positioning mechanism. The mechanism is required to accurately move in x-y and z-axes without interaction. In particular, the z-axis needs to be accurately controlled, such that the probe should not generate higher force applied to a sample than required when the AFM is in operation. In addition, the mechanism needs to be free from cross-coupling effects in xyz motion and unnecessary, parasitic motion. Furthermore, it needs to have no friction or geometric assembly error due to bolting. The scanning motion of the cantilever probe over the surface of samples is generated by two sets of dual positioning mechanisms in z and x-y motions, as shown in Figure 7.23a. The ﬁrst set consists of a coarse z-motion translator and a coarse x-y-motion translator stage. These coarse positioning mechanisms position the cantilever probe in the sample area so that the probe gets ready to measure the force between the tip and the sample surface of interest. The other set consists of a z-axis precision positioning mechanism and a precision x-y stage. The z-stage mechanism moves the probe in the direction normal to the sample surface, while the x-y stage moves it over the surface for scanning. The two sets of mechanisms are independently controlled, that is the coarse mechanisms initialize the point of scanning by their own precision scanners. The x-y scanner can scan from

502

Optomechatronics

TABLE 7.6 Details of AFM Components Component Course motion translator (z) Course motion translator ðx-yÞ Fine motion translator (z) Fine motion translator ðx-yÞ Cantilever tip Feedback control unit (z) Signal generator ðx-yÞ Computer (CPU)

Optical unit

Beam detector

Role

Characteristics

Move AFM head in z

5-statge step motor motion range (z) Motion range in x-y plane Contact/noncontact

Move AFM head in x-y Move the cantilever tip to the surface Move the x-ypiezo stage in a raster scan motion Measure the force between the tip and the sample Maintain a constant deﬂection of the cantilever in z direction Describe the motion of the probe in x-y plane Setting the scanning parameters, scan size, scan speed, signal processing, and visualizing Transmit the information of the probe deﬂection via light reﬂection Detect the variation of the cantilever deﬂection while scanning

— — — Raster scan motion —

Laser diode, objective lens, single (multiple) mirror Photo-detector, CCD camera

˚ to a few hundred mm, while the z scanner can measure the height a few A from sub angstrom to 10 mm. The two sets are very similar in the concept of positioning control, although they employ different types of actuators. Table 7.6 summarizes the components and their role in the AFM system. Therefore, we will discuss in brief only the precision mechanisms actuated by piezoelectric elements. Two typical types of actuators being used for AFM scanning are shown in Figure 7.36, layered (laminated) type (a) and tube type (b) which have been already shown in Figure 4.36a and Figure 4.36c. Since the displacement of a typical piezoelectric material is very small, approximately nm/V, many layers are stacked up to achieve a desirable displacement. For example, if 200 layers and 100 V are used, then 20 mm can be obtained. The actuator of a cylindrical hollow tube type gives two directional motions in x and y directions. On the other hand, the z motion is generated when all electrodes are subjected to the same electric ﬁeld. Here, we will consider the case of the layered actuator for the control motion.

0

5

10

0

5

0.1

0.1

0.1

time (t), ms

FIGURE 7.34 Response of the AFM for various controller parameters.

ki = 3, kd = 0

0.0

0.0

0.0

0.2

0.3

0.3

0.3

kp = 1x10–2

kp = 1x10–1

0.2

0.2

kp = 1x10–3

(c)

0.1

0.1

kp = 1×10–1, ki = 3

0.0

40 20 0 –20 –40

0.1

ki = 1

time (t), ms

time (t), ms

ki = 3

ki = 5

kp = 1×10–3, kd = 0

0.0 60

–40

40 20 0 –20

60

(b)

0 0.0

2

4

6

8

10 measurement signal, nm measurement signal, nm

10

40 20 0 –20

(a)

measurement signal, nm

12

kd = 0

kd = 1X10–6

0.2

0.2

0.2

0.3

0.3

0.3

Optomechatronic Systems in Practice 503

Optomechatronics

error (e), nm measurement signal, nm

artifact (za), nm

504 20 15 10 5 0 –5 0

2

4

6

0

2

4

6

0

2

4

6

20 15 10 5 0 –5 60 40 20 0 –20 –40 time (t), ms

FIGURE 7.35 The response of the AFM system response to the varying artifacts; kp ¼ 1 £ 1021 , ki ¼ 3, kd ¼ 0:

Figure 7.37 shows a typical monolithic piezoelectric scanning stage of the AFM which can translate in x-y directions. The ﬁgure indicates only the x-y stage, but the z-motion scanning mechanism may be made identical in its conﬁguration. The moving mechanism for each direction consists of symmetrically placed eight-plate springs, which is a double compound linear spring mechanism. The material of the stage is made of duralium A . . . ..

z ...

...

do di

VP Fl

z

A

Vρ

h

(a) layered actuator

(b) tube actuator

FIGURE 7.36 Two typical piezoelectric actuators used for precision scanning motion.

Optomechatronic Systems in Practice

505

PZT for x-motion

PZT for y-motion

FIGURE 7.37 The conﬁguration of the x-y positioning stage.

which is a light but hard material, and its spring element is normally fabricated by a wire-cutting method. The stage is driven by two piezoelectric actuators located in each axis. To analyze the details of the stage motion, we will consider one directional spring mechanism only (x-axis) driven by one piezoactuator, because the two mechanisms are identical. For this purpose, y-axis mechanism shown in the previous ﬁgure is omitted here as shown in Figure 7.38a. The x-stage model consists of three main moving bodies (m1, m2, m3) whose motions are constrained by ﬂexural plate springs in linear translation motion. When force Fp is applied to the main body m3 as indicated in the ﬁgure, the applied force in general causes the translation and rotational motions of body m1, body m2, and body m3 : However, the static analysis of this mechanism shows that the double compound mechanism adopted here does not generate the rotational motion of body 3 which is a parasitic motion for x-axis scanning. The static force analysis can be easily carried out using elementary solid mechanics and can be found in literature. We will therefore consider only translational motion of the stage. If the structural damping is neglected, the x-stage dynamic model can be schematically represented by an equivalent spring – mass mechanism composed of three masses and springs without dampers. To possess a dynamic motion of high performance the x stage needs to have high natural frequency, while satisfying a given motion range requirement. The dynamic model of the x-directional stage is shown in Figure 7.38b, while Figure 7.39 depicts free-body diagrams for the dynamics model shown in Figure 7.38b. The dynamic equations can be easily obtained by considering

F

FIGURE 7.38 A dynamic model of the x-directional stage.

(a) x-directional model

PZT actuator

flexure spring

intermediate body, m2

x3

main moving body, m3

intermediate body, m1

m1 k2

x3 m3

Fp k2

R

(c) piezoelectric actuator

Vi

C

(b) spring-mass-damper model

k1

x1 m2

Vo

k3

x2

506 Optomechatronics

Optomechatronic Systems in Practice

507 Fp

kx1

m1

k(x3 – x1) k(x2 – x3)

m2

x1

−kx2

k(x3 – x1)

x2

m3

k(x2 – x3) x3

FIGURE 7.39 The free body diagram shown in Figure 7.38b.

a free body. The dynamic equation can be written as; m1 x€ 1 ¼ 2kx1 þ kðx3 2 x1 Þ;

m2 x€ 2 ¼ 2kðx2 2 x3 Þ 2 kx2 ;

m3 x€ 3 ¼ 2kðx3 2 x1 Þ þ kðx2 2 x3 Þ þ Fp

ð7:29Þ

where all stiffnesses are assumed to be identical for all masses; k1 ¼ k2 ¼ k3 ¼ k, and Fp is the force generated by piezoactuator due to the applied electric voltage. In this dynamic model, the piezoactuator is considered to be the force generating device having no dynamics. In other words, the dynamics of the piezoelectric actuator is neglected, because its dynamics is very fast compared to that of the stage. Therefore, we can model it only as a force generating capacitance element as shown in Figure 7.38c. Based on this observation, the induced force and applied voltage are related by Fp ¼ Fcf Vo

ð7:30Þ

where Fcf is a constant related to geometry and physical properties of the piezomaterial, and Vo is the voltage applied to the electrodes. The capacitance model of the piezoactuator with a driving circuit is given by relating the input voltage to driving circuit Vi, and the output voltage Vo; RC

dVo þ Vo ¼ Vi dt

ð7:31Þ

where R is the electrical resistance of the driving circuit and C is the capacitance of the piezoactuator. Laplace transforming the above equations and obtaining a transfer function between the piezoelectric displacement and the applied force, we have ðm1 s2 þ 2kÞX1 ðsÞ ¼ kX3 ðsÞ;

ðm2 s2 þ 2kÞX2 ðsÞ ¼ kX3 ðsÞ;

ðm3 s2 þ 2kÞX3 ðsÞ ¼ kðX1 ðsÞ þ X2 ðsÞÞ þ Fp ðsÞ

508

Optomechatronics

The transfer function between the displacement m3 and the piezoactuator force Fp is expressed by GðsÞ ¼

X3 ðsÞ m1 s2 þ 2k ¼ Fp ðsÞ m1 m3 s4 þ 2kðm1 þ m3 Þs2 þ 2k2

for the case of m1 ¼ m2 : From Equation 7.30 and Equation 7.31, we have Fp ðsÞ ¼ Fcf Vo ðsÞ;

Vo ðsÞ 1 ¼ Vi ðsÞ RCs þ 1

ð7:32Þ

On the other hand, if we use a PID controller, we have the transfer function expressed by Gc ðsÞ ¼ kp þ ki

1 þ kd s; s

UðsÞ ¼ Gc ðsÞEðsÞ

In addition, due to saturation and nonlinearity the voltage input Vi is related to the controller output U by Vi ¼ satðUÞ where satðUÞ 8 Umax > > < Vi ¼ U > > : Umin

is deﬁned by if U $ Umax if Umin , U , Umax if U # Umin

Combining all of the elements mentioned above, we can draw a complete block diagram for a feedback control system as illustrated in Figure 7.40. The controller utilizes an error signal eðtÞ ¼ x3d ðtÞ 2 x3 ðtÞ, the difference between the reference and the feedback signals of the displacement in the x-axis. The aim of the control objective is to eliminate this error, that is, to position the stage in x direction to a desired location with smaller overshoot but faster rise time without any steady state error. From observation of the control system block diagram, we can see that the system has nonlinearity with saturation. To analyze this, we need to utilize a controller such as a describing function method, adaptive control, and so on. However, we will apply a simple linear PID controller for simplicity in order to see the scanning motion control. With this PID controller we now determine

x3d

e

GC (s) controller

U

Vi

Umin

Vi UmaxU

FIGURE 7.40 Schematic of the x-axis position control system.

Ga (s) driver

VO

Fcf

Fp

G (s) stage

x3

Optomechatronic Systems in Practice

509

TABLE 7.7 The Parameters Used for the x-Stage Simulation of an AFM Parameters

Symbol

Unit

Value

Force coefﬁcient The ﬁrst mass The second mass The third mass Stiffness Resistance of drive Capacitance of piezoactuator Saturation voltage

Fcf m1 m2 m3 k R C Vsat

Nm/V kg kg kg N/m V F V

30 £ 1023 8.68 £ 1023 8.68 £ 1023 1.28 £ 1021 5 £ 1025 6.7 £ 1023 4.8 £ 1029 100

the response characteristics for a variety of controller gain parameters. The parameters used for this simulation are listed in Table 7.7 and the Simulink model is shown in Figure 7.41. Figure 7.42 represents the response of the stage for a step displacement of 5 mm. Figure 7.42a shows the response for three different proportional gains at a ﬁxed set of integral and derivative gains, ki ¼ 5 £ 103 , and kd ¼ 5 £ 1021 : It can be seen that, as the proportional gain increases, the response becomes oscillatory, but the steady state error is seen to rapidly decrease. The effect of the integral gain is shown in Figure 7.42b for a ﬁxed set of gains, kp ¼ 2 £ 103 and kd ¼ 5 £ 1021 : As the integral gain increases, the response becomes very fast without overshoot. In general the x-axis response shows a satisfactory response in both speed and steady state characteristics. In this analysis, the motion coupling due to the monolithic stage mechanism is not considered. In the actual stage, the x-stage motion will be somewhat inﬂuenced by y-direction motion due to the characteristic property of the monolithic stage mechanism. However, the degree of the y-stage inﬂuence is found to be very small in terms of its amplitude due to the design for decoupling between axes. The same trend is found from the of y-stage motion. Although, the stage motion in the

target Xr

e

1000000 Gain1

PID Controller

Vi

U Saturation

1 π R C.s+1 Actuator

Scope2

Scope1

FIGURE 7.41 Simulink model for the x-stage simulation.

Vo

30 e-3 Gain2

F

2

π

M1.s +2 K den (s) System

X3

510

Optomechatronics

5

4

displacement (x), μm

displacement (x), μm

5

kp = 3x103 kp = 2x103 kp = 1x103

3 2

ki = 5x104 ki = 5x103 ki = 5x10

3 2 1

1 0

4

0

1

time (t ), ms

2

3

(a) effect of the proportional gain, ki = 5×103, kd = 5×10-1

0

0

1

time (t ), ms

2

3

(b) effect of the integral gain, kp= 2×103, kd = 5×10-1

FIGURE 7.42 The response of the x-stage for various gain parameters.

z direction is not analyzed here, it is found to exhibit motion characteristics similar to that of the x-stage. The z-motion stage that moves the cantilever in the direction vertical to the sample surface during the x-y scan motion should also exhibit extremely high accuracy motion in response to the cantilever deﬂection.

Confocal Scanning Microscope The concept of confocal microscopy was ﬁrst proposed by Minsky in the 1950’s in an effort to improve the limited resolution of conventional microscopes by reducing the amount of scattered light from the sample being inspected that has been observed from conventional microscopes. However, due to lack of suitable lasers and precision stages the technique could not be practically realized until the early 1980s. Confocal microscopy is now a well-established technique measuring high resolution surfaces of the topography of objects in biomedical applications, material science, semiconductor quality monitoring and control, forensic applications, and so forth. Measurement Principle As shown in Figure 7.43a, the microscope is composed of a point-wise light source (usually a laser), a precision scanning mechanism, a light detector with a pinhole sensing the reﬂected light from a specimen (object), and the necessary optical units.

Optomechatronic Systems in Practice

511

laser source lens lens condenser lens objective lens

collective lens

source

pinhole

sample plane

objective lens sample

(a) confocal (reflective type)

image plane

focal plane out of focus

(b) conventional

FIGURE 7.43 Comparison between confocal and conventional microscopes.

Due to the basic difference in its optical arrangement a confocal microscope yields image “characteristics” different from those that can be obtainable by a standard microscope. The operation principle with the confocal microscope is as follows. When the object to be imaged moves out of the focal plane, the resulting defocused image will be very weak in intensity and beyond some locations it will disappear rather than blurring. This is because the light reﬂected from the object out of focus is defocused at the pinhole located in front of a detector and (dotted line), therefore not much light will pass through it, thus giving no appreciable contribution to imaging at the detector. It is noted here that the image intensity in this case decreases as the object image is defocused. Due to this property, a confocal microscope tends to yield sharper images than a standard microscope does. By contrast, in a conventional optical microscope as shown in Figure 7.43b, the whole area of the sample, including the focal plane, is simultaneously illuminated by an extended source through a condenser lens full-ﬁeld illumination. Therefore, the information from each illuminated point in the object is simultaneously imaged on to a screen or the retina of the eye. Due to this conﬁguration, much of the light illuminated from the regions of the objective plane above and below the selective focal plane are simultaneously collected by the objective lens. This causes an out-of-focus blur to form the image. However, as we recall the principle of autofocusing treated in Chapter 6, “Automatic Optical Focusing,” the blurring can be eliminated or reduced by keeping the position of the image plane at a focal plane. There are two types of confocal system; the reﬂective type and the transmission type shown in Figure 7.44. In both models, one objective lens is used for illumination, while one collector lens is used for detection. The transmission geometry is particularly useful for imaging transparent objects

512

Optomechatronics axial response from a mirror objective beam splitter

source .

z object

scanned

objective

collector scanned

collector

point detector

point source specimen plane

(a) transimitive type

pinhole detector

(b) reflective type

FIGURE 7.44 Two basic types of the confocal system.

in a highly scattering medium. For weakly scattering objects, the threedimensional (3D) imaging properties are not much improved, because of strong unscattered light in transmission. Its 3D point spread function is found to be almost identical with nonfocal imaging. Due to this, the majority of confocal systems operate in the reﬂection mode under the same operation principle but with a slightly varied conﬁguration. The reﬂective type, as mentioned above, produces a strong signal only at the in-focus axial position of the object, producing a very weak one at the defocused position. It is noted that the property of this reﬂective type is identical to that of the transmission mode if an object is placed in the focal plane. Although confocal concept has been already explained, let us elucidate it again for a better understanding of this subject. Figure 7.45 depicts a simpliﬁed schematic to explain the principle of confocal detection. The basic steps employed are to project a focused beam of light into a specimen, to scan a beam of light over the specimen, and at the same time detect the light reﬂected from its surface by a light detector device with a pinhole or single-mode optical ﬁber. In more detail, (1) the laser beam enters a collimating optical lens from the left-hand side. (2) The collimated light is ﬁrst transmitted to the right by a beam (solid line). (3) This transmitted light is then focused on to a spot by a focusing lens. Depending on the location of the object to be measured, the beam is focused in the focal plane of the objective (location b), positively (location c) and negatively focused (location a). (4) The reﬂected light travels back through the objective lens, enters the beam splitter where the light directs down towards a collecting lens. The collective lens then focuses the light on to the detector aperture. The ray paths indicated in

Optomechatronic Systems in Practice

lens

513 out of focus (c) in focus (b) out of focus (a)

beam splitter objective lens

laser

p´

object p

p´´

x-y stage

z stage

collecting lens d´´

(a) (b) (c)

d d´

pinhole detector

FIGURE 7.45 A simpliﬁed schematic to explain the principle of confocal detection.

dotted and dashed lines show how lights from an out-of-focus object are propagated though the optical system. The ﬁgure shows three focal points illuminated according to the object location in an axial direction (z direction) (a), (b), (c). The focused point, d in the pinhole is due to the focused point, p at the in-focus object position (b). Likewise, the defocused point p0 at the location (c) forms the defocused point d0 in the detector, while the defocused point p00 at the location (a), the defocused point d.00 The detector signal reaches its maximum if the object is at the focal point of the objective lens. If the illumining spot is defocused a larger area is illuminated with a much weaker intensity. This property implies that by changing the focal plane in the direction of the optical axis while carrying out successive x-y scans, a 3D image of a sample can be recorded as a result of the confocal rejection of out-of-focus signal. In the case of a thick sample, its 3D image can be recorded with slicing of the sample, since the reﬂection system can record a series of sections of the image at different depths of the sample and thereby build up complete 3D information of it in a 3D world. By use of this confocal scanning, height information of 3D samples whose variations are in the order of 1 nm can be imaged. In addition, oblique sections at any angle can also be accurately measured. Beam Scanning As in the case of AFM, the scanning mechanism is a basic device needed to build up the image point by point. The mechanism consists of a z-motion scanning stage that maintains a constant objective-sample separation and an

514

Optomechatronics transmitive sample pinhole

light source

D y

light source

x

detector

(a) sample scanning

pinhole

A'

D'

A

D

light source

planar detector array planar lenslet array

detector

(b) beam scanning

(c) multiple simultaneous beam scanning

z pinholes

FIGURE 7.46 Types of scanning in confocal microscopes.

x-y stage that performs an x-y raster pattern scanning. All three elements, sample, objective, and laser beam shown in Figure 7.46 can be scanned. Objective scanning shown in Figure 7.46a scans samples relative to a ﬁxed, stationary focus spot. It has a relatively low speed of scanning due to the large inertia of its mechanical conﬁguration and requires relatively high positioning accuracy. This scanning method has difﬁculty maintaining uniform illumination across the ﬁeld of view (FOV), and therefore, has not been popularly used. Due to the limitations associated with the above method, the beam scanning method has been widely adopted for the majority of commercial uses. This scanning method scans the laser beam relative to a stationary object. It deﬂects the beam by some scanning mechanism such as in two different schemes, point-by-point (Figure 7.46b) and multiple simultaneous scanning (Figure 7.46c), schematically shown in the ﬁgure. The point-by-point method is beam scanning which utilizes one illumination pinhole and one detection hole. It employs a 2D vector scanner for the x and y directions. When the illumination pinhole moves from point A to A0 , accordingly this causes the confocal image to move image point D to D0 upward as can be seen from the right-hand side of the ﬁgure. The scanning system should cover the pupil size of the objective lens of the microscope and angle of view, and meet requirements such as scanning resolution, accuracy and speed. The scanned optical quality depends not only upon the type of scanner, but also on scanning optics that needs to be matched with the microscope. As discussed previously, the scanning resolution here again is limited by diffraction theory, and the larger aperture size of the scanner will increase the resolution. As scanning devices galvanometer mirrors and acousto-optic

Optomechatronic Systems in Practice

515

laser source

A-O scanner deflected beam

beam splitter galvanometer x-y mirrors

(b) acousto-optic scanner aperture

detector

object

rotating mirror

(a) rotating x-y mirrors A-O scanner

(c) hybrid scanner FIGURE 7.47 Schematic diagram of the x-y scanner used for confocal microscopes.

cells are used, while a polygon mirror is rarely used since it cannot provide satisfactory scanning accuracy and reliability due to wobbling while rotating and fabrication defects. There are several ways to achieve this scanning which includes methods as illustrated in Figure 7.47, viz., the use of (1) rotating mirror, (2) acousto-optic deﬂector, (3) hybrid scanner, and (4) a single-mode optical ﬁber in replace of the pinhole. Although these methods are based upon point by point scanning, they are very fast in scanning. The ﬁgure shows a schematic overview illustrating three beam scanning schemes. In the case of the galvanometer mirror (Figure 7.47a), as mentioned previously, the laser beam passing through an illuminating aperture is reﬂected from a beam splitter. The reﬂected beam then impinges upon the mirror surfaces of the two scanning galvanometers; the scanning optics projects the deﬂecting point of the scanner on the pupil of the objective lens and the deﬂected beam resulting from the scanning motion is focused and then illuminates the sample through the objective lens. The reﬂected light returns through the microscope along the same ray-path as the projection light. The light passed back is transmitted by the beam splitter and ﬁnally imaged on to the detector aperture. This operating principle is also applied to the case of adopting acousto-optic scanners and hybrid methods. When the sample image of some large area needs to be formed simultaneously, an array of a large numbers of pinholes would be required for both pinholes of illumination and detection. This method is called a “tandem scanning optical microscope” (TSOM) and uses a large number of pinholes for real-time simultaneous scanning of a certain area of the object. We will return to this method in the next subsection.

516

Optomechatronics objective lens objective lens detector

rotating mirror sample detector

rotating mirror

light source

sample

light source

(a) objective lens with partially filled with light d1

(b) objective lens with completely filled with light

d2

d3 sample

detector rotating mirror

objective lens auxiliary lens

light source

(c) auxiliary lens system FIGURE 7.48 Two types of beam scanning using a rotating mirror.

Figure 7.48 shows two typical types of rotating mirror based-beam scanning. If the beam is deﬂected by a mirror and is incident on an objective lens, the lens may be either partially or completely ﬁlled with light. If the lens with partially ﬁlled with light is used, as in Figure 7.48a, the numerical aperture is smaller than that in case of using a full aperture, this results in loss of resolution. If the lens is fully covered with light as shown in Figure 7.48b, a large portion of the light will be lost. These drawbacks can be resolved by using auxiliary lenses as shown in Figure 7.48c. The rotating mirror must be placed in a zero-deﬂection plane, a distance d1 in front of the auxiliary lens where d1 ¼ ðd2 þ d3 Þ

d2 d3

ð7:33Þ

where d2 is the focal length of the auxiliary lens. The auxiliary lens here contributes to the complete ﬁlling of the objective lens without a loss of light. The schematic conﬁguration of a confocal scanning microscope that has two-axis scanning mirrors is shown in Figure 7.49. It is basically the same conﬁguration as shown in Figure 7.48c in that it utilizes an auxiliary lens. This conﬁguration satisﬁes the above condition in Equation 7.33. A confocal system that employs an acoustic-optical deﬂector (AOD) for beam steering is shown in Figure 7.50. The laser light irradiating from a laser diode is expanded through a beam expander. When the expanded parallel

Optomechatronic Systems in Practice

517 auxiliary lens

motor objective lens

sample

x scanning mirror

z stage

y scanning mirror motor beam splitter

detector pinhole lens lens

laser source FIGURE 7.49 Confocal laser scanning microscope with a two-axis rotating mirror scanner.

beam enters the AOD, the parallel beam is deﬂected with a certain angle depending on the voltage applied to a piezoelectric transducer. If the deﬂected angle by the AOD is given by ud, then sin ud ¼

lfa 2va

ð7:34Þ

tube lens

scan lens AOD

beam wave splitter plate

y z

laser collector

f1

f2

sample objective

collective

photo diode

FIGURE 7.50 Confocal scanning microscope using AOD.

518

Optomechatronics

where va is the sound velocity and fa is the frequency of sound. This deﬂected beam goes through a scan lens and becomes parallel again by a tube lens. In this condition, it follows that if the focal lengths of the scan and tube lenses are given as f1 and f2 , respectively, then the magniﬁcation ratio should meet the following condition M¼

f2 f1

ð7:35Þ

This parallel beam then enters an objective lens, which eventually is focused on a point within a specimen. It should be noted that between the tube and objective lenses there are a polarized beam splitter and a 14 l wave plate, which makes it possible to discriminate the refractive beam from the incident beam. This is because the wave plate makes the beam refracted from the object polarized into a right angle to that of the light from the source. Therefore, all the light refracted from the object is transmitted by the beam splitter, then enters a collective lens and is eventually focused on the photodiode through a slit. The role of the scan lens and the tube lens is to expand and collimate the beam so that the objective lens aperture is ﬁlled with the beam. To fulﬁll this condition, the diameters of the two beams passing through the lenses should satisfy M¼

Dt Ds

where the size of the beam incident to the scan lens is Ds and the incident to the tube lens is Dt , which is another way of expressing M. The role of the AOD here is to make the position of the focused beam vary along the vertical axis (y-axis), as can be seen from the coordinate system attached to the object. This phenomenon results from the fact that AOD deﬂects the incident beam to a desired angle with which the deﬂected beam is incident to the objective lens, thus making the resulting beam focused on a point on the vertical axis different from that which can be obtained without any deﬂection. Again, the deﬂection angle of the AOD ud , is determined by Equation 7.34. In order to vary the scanning position we simply change the frequency of the acoustic wave or the RF according to Equation 7.34. Although, in the above, scanning is explained only in one dimension, the same principle can be applied to the case of scanning, the x-y plane in 2D. It is noted that the intensity of the beam collected on the sensor depends on how accurately focused the beam is on the object. This necessitates the need of the feedback control of the object position in an axial direction. Nipkow Disk The operating principle of TSOM may be easily understood in more detail from Figure 7.51. The system employs the Nipkow disk which is an opaque disk containing many thousands of pinholes drilled or etched in spiral patterns. The use of many interlacing spiral patterns is to provide

Optomechatronic Systems in Practice Nipkow disk light source

519

eyepiece

W

perforated pinholes collective lens mirror

mirror

mirror mirror

beam splitter objective object

(a) optical arrangement

(b) Nipkow disk made of thousands of pin holes

FIGURE 7.51 Tandem scanning optical microscope (TSOM).

the confocal optical microscope with the raster scan. Typically, the diameter of the pinhole is of the order of a few tens of mm and spaced at ten pinhole diameters. The pinhole spacing is made to ensure that there is no interaction between the images on the object formed by the individual pinholes. As shown in the ﬁgure, the optical arrangement is made in such a way that a collective lens focuses the incoming light on to the pinholes on a portion of the rotating disk. This beam from the source illuminates and passes through several thousands holes located in the left-hand side of the ﬁgure. The light then travels via several optics and again is focused by a collective lens. Each point of light reﬂected or scattered by the sample is focused by the same lens. The focused light returns through a conjugate set of pinholes located on the opposite side of the disk. This light can be seen in the eyepiece. From the discussions on the confocal principle shown in Figure 7.45, it is noted that only the light reﬂected from the focused region of the object returns through the pinholes via the optical agreement, while the light from the defocused part of the object does not. In effect, this is equivalent to achieving several thousand confocal microscopes all returning in parallel in a real-time manner. It is apparent that performance of the TSOM depends upon the diameter ðdÞ spacing of the pinholes ðsÞ; and rotation speed of the disk ðVÞ: The pinhole size certainly inﬂuences the range (depth) resolution while its spacing affects the interaction between the light reﬂected from the object through the individual holes. Too close spacing will make the reﬂected lights interfere with each other. The rotation speed inﬂuences the image capturing rate.

520

Optomechatronics

Too slow rotation speed will not yield beneﬁts of processing the images at high speed. The performance inﬂuenced by these factors is further limited by the mechanical precision required for disk fabrication, alignment between the input and output pinholes, optical complexity which requires a large number of optical components and low light efﬁciency. The stage carrying the sample needs to be operated at high positioning accuracy and speed to have high resolution and high measurement throughput. Positioning accuracy is determined by the accuracy of the constructed stages and that of the position controller. The accuracy of the stage itself is determined by fabrication accuracy and its alignment with guide systems such as lead screw and bearing. The control accuracy is dependent upon the performance of the servo actuator and its control algorithm. Since performance of the control accuracy requires very high precision, the control algorithm must be able to compensate for all the uncertainty and nonlinearity involved with the stage, servo system, and alignment system. It also must take into consideration the external loads, such as vibration from the stage itself, as well as from the ground. Since the stage control is illustrated for the case of AFM in the previous section, the discussion of the control system for the scan stage will not be treated again here. System Resolution Much of the resolution of the focused system is discussed in the section in Chapter 6, “Automatic Optical Focusing.” The important fact is that lateral and axial resolutions are limited by diffraction phenomena. As we may recall, the minimum resolvable length in the lateral direction perpendicular to the optical axis is given by Dx ¼

0:61l NAObj

for an objective lens of a confocal microscope having a numerical aperture NAObj, and a wavelength of l of the light beam. On the other hand, the resolution of axial direction is found to be expressed by using the para-axial approximation as Dz ¼

2ln ðNAObj Þ2

where n is the refractive index of the immersing medium adjacent to the objective lens. Another way of describing the axial resolution is to use the sharpness of the axial response of the image. Based on a high aperture theory for a plane refractive object, axial intensity is found to vary with normalized axial position as shown in Figure 7.52a. It shows the dependency of the axial response of the confocal system of the dry objective lens. In the ﬁgure, the vertical axis shows normalized intensity (I) while the horizontal axis

FWHM

–2

–1

0

1

normalized axial position

521

normalized resolution

Optomechatronic Systems in Practice

2 z /l

(a) plane object

axial resolution (plane)

1st Airy disc normalized radius (rn)

(b) axial resolution vs. circular aperture

FIGURE 7.52 Axial resolution of the confocal system.

indicates the axial position normalized with respect to the wavelength of the illuminating light. The measure of the axial resolution is indicated by the value of FWHM in the ﬁgure. The FWHM is the full width intensity at which the light power becomes one half of the maximum power. The smaller value of FWHM indicates a shaper response curve. It is found that, as NA becomes higher, the response from a planar object gets sharper, which indicates higher axial resolution. That is, the width of the response decreases with the increase of NA. However, the signal intensity which is the area integrated under the curve becomes smaller with higher NA. Figure 7.52b depicts the normalized confocal axial resolution for circular confocal aperture size. The normalized radius, rn is expressed by rn ¼ ð2pa=lÞsin u where a is the radius of the aperture. As can be seen from the ﬁgure, the normalized axial resolution drops sharply near the Airy disk indicated by a vertical line as the confocal aperture increases. Since signal intensity increases with the aperture radius, selection of the confocal aperture therefore requires a trade-off between resolution and intensity in the axial direction. From this point of view an optimal aperture size of the pinhole needs to be properly determined. Focus Measure and Control We have dealt with the autofocusing problem in the previous chapter, in the sections “Automatic Optical Focusing” and “Visual Autofocusing,” in which autofocusing has been achieved based on feedback control of various focus measures. In optical autofocusing discussed in Chapter 6, “Automatic Optical Focusing,” the principle of aberration called “astigmation” was employed in determining a focus measure. In “Visual Autofocusing” in Chapter 6, we

522

Optomechatronics

discussed several focus measures for an online focus control that utilizes the characteristics of the image intensity of the object to be focused. These can be basically utilized for this confocal microscope unless the time for computing the focus measure based on measurement is considerably long or resolution of the detector such as CCD camera and photodetector is relatively low in comparison to that required by the system. Here, we will brieﬂy discuss some other factors associated with visual autofocusing that have not been discussed previously. The focus measures denoted in Equation 6.78 to Equation 6.79 are not directly comparable, since they are not of the same scale in the same quantity. To this end, focus assessment is needed in order to determine which focus measure is the best among a variety. The so-called “progressive sharpness” is one such assessment determined by Ff ¼ Ak

3 X k¼1

ldi 2 dj lk

for

i–j

ð7:36Þ

where di and dj are the two intersecting distances giving an equal focus measure in the central lobe of a focus measure, k represents the index denoting the equidistant focus level from the maximum value, and A is the normalized constant. For example, if the maximum value is assigned 1.0 at k0 ¼ 0, the focus measures are 0.75 at k1 ¼ 1, 0.5 at k2 ¼ 2, 0.25 at k3 ¼ 3, and so on. Another assessment is to evaluate the robustness of the measure relative to noise. The assessment measure can be determined by Ff ¼

DF=Dz F

ð7:37Þ

where F is the focus measure deﬁned in Equation 6.79, and Dz is the distance from the sample. These measures are usually computed within a predeﬁned window size. Focus control is the problem in adjusting the sample distance. The controller will drive the stage in the direction required to maximize the value. This is equivalent to ﬁnd the best location at each instant of time which gives the maximum value of the measure. In order to achieve this, the control system needs to move the stage and then evaluate whether the focus measure increases or decreases can determine this. The hill climbing method is one of the effective methods as we have used before. Figure 7.53 illustrates one typical method of focus control for the confocal microscope in order to obtain the desired focus. It utilized a focus value of the image obtained from the sample plane. Based on the error occurring between the desired and actual values, the controller computes the drive signal in order to position the z-stage (optical axis) at a desirable location. When PID control is adopted, the control drive signal utilizes the same control law given previously. Here, eðtÞ is given by eðtÞ ¼ Ffd 2 Ff ðtÞ

Optomechatronic Systems in Practice

523

beam splitter

z

laser

piezoactuator

object

driver

pinhole

controller focus value

detector

–

+ desired focus value

FIGURE 7.53 Focus control of the confocal system based on the focus measure value.

where Ffd is the desired focus value and Ff ðtÞ is the value obtained at time t, either from Equation 7.36 or Equation 7.37. If Ffd is evaluated and given prior to positioning control, the control signal can be carried out within each sampling period. The positioning step size for each control step is different for the two control strategies; rough positioning and ﬁne positioning. In rough positioning, the actuation mechanism moves the stage in large increments, when far from focus, while the ﬁne mechanism moves it in much smaller steps when close to the focus point. In ﬁne positioning, the increment of the moving z-stage at each step should be far less than submicron, , mm. Figure 7.54 depicts a typical image of a sample obtained by this principle

2.10 [μm] A′

0.90 775.00

750.00

1.50

0.30 –0.30

A 0.00

–0.90 0.0

FIGURE 7.54 Surface topology measured by a confocal microscope. Source: Company presentation, Nano Focus AG Co. Ltd.

155

310

465

620

775 [μm]

524

Optomechatronics

where an autofocusing control scheme is utilized. It shows an nm scanning accuracy.

Projection Television MEMs-based projection displays stem from the principle of optomechatronics in that the controller and microdevices are interacted with incoming light according to the input video schedule of light modulation. The interaction occurs when light is steered as desired by means of the control of micromirrors. There are two MEMs-based approaches in the control of light for projection display; reﬂective display and diffractive display. The reﬂective type shown in Figure 7.55a utilizes a large 2D array of micromirrors fabricated monolithically over a single integrated circuit that reﬂect light according to their controlled surface angle. The mirrors in the array are all controlled by individual electro-static actuators to various positions. The diffractive type shown in Figure 7.55b, on the other hand, uses a linear array of diffraction gratings which is a much smaller device than the reﬂective 2D arrays. One pixel of the image is composed of six deformable beams and all beams are actuated by the undivided electrostatic actuators (Here only four beams are shown). When actuated, incoming light diffracts at a certain speciﬁed diffraction angle, otherwise stays reﬂective without diffraction as shown in Figure 7.55b. Let us now study some of the details of silicon nitride beam

A

1

mirror electrode

yoke address pad

2

3

4

A'

address pad

silicon dioxide supports

to projector θ

θ

θ

mirror

to projector

θ deflected beam state solid beam array

(a) digital micro mirror device FIGURE 7.55 Optomechatronic MEMS-based projection display.

(b) grating light value

Optomechatronic Systems in Practice

525

the projectors such as operation principle, system conﬁguration, and control of the dynamics of the mirrors and beams that interact with light. Digital Micromirror Device The digital micromirror device (DMD) is a device that makes projection displays by converting white light illumination into full-color images. This is achieved by using spatial light modulators with independently addressable micromirrors. The device was invented by Texas Instruments in 1987, which is the outgrowth of work that began a decade earlier on micromechanical, analog light modulators [26]. Figure 7.56 shows a picture illustrating the conﬁguration of the DMD display system. It consists of a display light processing (DLP) board, a DMD, a light source, a RGB color ﬁlter, optical units, a projection lens, and a screen. The board (chip) contains tiny micromirrors, memory, and processor. The device is a reﬂective spatial modulator composed of an array of rotatable monolithically fabricated aluminum micromirrors, and has anywhere from 800 to more than 1,000,000 mirrors, depending on the size of the array. These mirrors work as pixels in display as can be seen from Figure 7.57. The mirrors can be rotated within ^ 108 by mechanical stops which are digitally operated in two modes, on or off. The ^ 108 rotation angles are converted into high contrast brightness variations by use of a dark-ﬁeld projection system.

FIGURE 7.56 A picture of the conﬁguration of the DMD display system. Source: DLP projector overview, Texas instrument Co. Ltd.

526

Optomechatronics

pixels on screen

projection lens l ight absorber

light source

–10

+10 DMD micro mirrors

FIGURE 7.57 Operation of two micromirrors.

The working principle of the DMD display can be described with the aid of Figure 7.56 and Figure 7.57. When white light from a source enters, a condenser lens collects the light from a source which is forced down onto a color wheel ﬁlter. This rotating wheel combines the red, green, and blue video signals in sequence, depending on the content of the coded video data generated by the chip. These signals are sent to the array of mirrors. When a mirror is rotated to ^ 108 “on state,” it reﬂects the incoming light into the pupil of the projection lens. The lens then produces an enlarged image of each DMD mirror on a projection screen. In this case, the mirror appears bright at the projection screen. A mirror rotated to 2 108 or “off” state reﬂects the incoming light to a light absorber, which indicates that the light misses the pupil of the projection lens. Contrary to the previous case, the mirror appears dark at the projection screen. In more detail, when light is incident to the two mirrors, the left mirror directs it through the projection lens and on to the screen, generating one square white image. The right mirror, however, does not direct the light through the lens, projecting it to a light absorber shown in the right-hand side of the ﬁgure. This case produces a square dark image on the designated part of the screen. Architecture of a DMD Cell Figure 7.58 illustrates an exploded view of a structure of a DMD mirror basically composed of a mirror, a yoke and hinge mechanism, a metal threelayer and CMOS static RAM cell. The mirror is connected to an underlying yoke by a support post and suspended over an air gap by two mechanically

Optomechatronic Systems in Practice

527

mirror

landing tips mirror address electrode

torsion hinge yoke

mirror−10dug mirror+10dug

yoke address electrode bias/reset bus

landing sites

hinge

yoke

spring tip

CMOS substrate

CMOS memory

(a) exploded view

(b) assembled view

FIGURE 7.58 The structure of a DMD mirror. Source: Van Kessel, P.F. et al. Proceedings of IEEE, 86:8, 1998, q 2005 IEEE.

operated torsional hinges. This enables the yoke to freely rotate within a speciﬁed angle which is in this case 2108 # u # 108: The limit of rotation is achieved by a mechanical stop, the landing tips of the yokes make a stop at the landing sites of the metal three layer. Underlying the mirror are a pair of address electrodes, mirror address and yoke address, which are connected to the CMOS memory layer. The mirror and yoke are electrically and mechanically connected to a bias reset bus built in the metal three layer, one of the three motorization layers. This bus interconnects the mirror and yoke to a bond pad at the chip parameter. A simpliﬁed cross-sectional view of the assembled structure of the DMD mirror is shown in Figure 7.59a. The rotational motion of the mirror, u, is provided by the torque electrostatically generated by voltage control. This torque acts upon the DMD unit cell composed of the mirror, the substructure including yoke, hinge unit, and electrode. The mirror is electrically grounded through its support post which is electrically connected when external voltage is applied. This voltage is called bias voltage. When a voltage is applied to the electrode in the right-hand side, due to the induced electrostatic force, the mirror tilts toward the right landing site (not shown here), until it reaches the mechanical limit. This results in a þ 108 “on” state, inclined at 108 to the horizontal plane. When a voltage is applied to the left side of the electrode, the mirror reaches another mechanical limit, yielding a tilting angle 2 108 “off” state. As mentioned above, the motion of the DMD exhibits binary behavior by choosing either one of the two stable positions, þ 108 or 2 108, depending on the magnitude of the applied voltage ðV1 ; V2 Þ to the electrodes of both sides and bias voltage Vb applied to the mirror. The tilt motion should be generated in a desired fashion for switching and therefore there needs to

528

Optomechatronics incident light

reflected light

mirror address electrode

address electrode address pad

yoke

address pad

(a) geometry of the DMD incident light reflected light

mirror mirror

mirror

(b) An illustration of mirror rotation in sequence FIGURE 7.59 Geometry and rotational motion of the DMD.

be a strategic (logical) operation of the three voltages to achieve such motion. The method involved with the operation of applying the three voltages will be brieﬂy described by use of Figure 7.59b. Three ﬁgures illustrate a series of sequential motion, composed of three different states of tilt motion. Let the voltage V1 and V2 be applied at the left-hand electrode and right-hand electrode, respectively. Whenever they satisfy the following condition V1 2 Vb , V2 2 Vb the tilting motion will begin in a clockwise direction. When at some instant the applied voltage is such that V1 2 Vb p V2 2 Vb the mirror will continue to tilt rapidly. Finally, when the voltage difference between the two is decreased, V1 2 Vb , V2 2 Vb the mirror will have a much smoother motion than before to be ready to land at the right-hand side landing site. Based on the foregoing discussions, we can see that there may be a variety of logic for applying the voltages V1 ; V2 ; and Vb in order to produce an electrode torque that can generate the desired tilting motion. The electrostatic force generating this torque is found to primarily depend on the voltage difference across the air gap between the mirror and its

Optomechatronic Systems in Practice

529

electrode, V, the air gap at a given element area, h and 10 is the permittivity of air. The electrostatic pressure, that is, the inﬁnitesimal electrostatic force per unit area exerted on an area dA of the mirror or yoke is given by dF 1 V ¼ 10 dA 2 h

2

ð7:38Þ

where 10 is given by 8.54 £ 10212 F/m. From Figure 7.60, the electrostatic torque acting on the mirror is written as ð dF dA ð7:39Þ t¼ x dA Substitution of Equation 7.38 into Equation 7.39 yields,

t¼

ððð 1 V x10 h A 2

2

dA

ð7:40Þ

where x is the distance from the rotation axis to the applied inﬁnitesimal force, and y is the coordinate perpendicular to the x-axis. If the width of the electrode is assumed to be uniform, and further, tilting angle u is assumed to be small, Equation 7.40 can be rewritten as ( ) ð‘ x1 w V2 0 t¼ dx ð7:41Þ 2 ðho 2 xuÞ2 0 where w and ‘ are the width and length of the address electrode, respectively, and ho is the original gap between the mirror and the electrode. The above relationship implies that, even if V is held constant, the electrostatic force is varied due to rotation of the mirror. According to this phenomenon, when a mirror approaches the land site of the þ 108 side, it experiences much larger torque than a mirror in other positions except the other region, 2 108 side, as can be seen from Equation 7.41. To obtain a good quality of projection image the mirrors should not provide any undesirable motion to the optical path. This means that the angular motion of the mirrors u must have good dynamic characteristics ∆F electrostatic force

x

mirror

x

θ h0

address electrode yoke address pad FIGURE 7.60 Geometry to calculate the electrostatic torque.

address pad

address electrode

530

Optomechatronics

when they are switched from one state to the other. The motion should be fast, exhibiting small overshoot, no steady-state error, and no oscillation at the steady state. We therefore need to consider the mirror dynamics for the optical system design. If the mirror system has parameters associated with dynamics such that the moment of inertia is I, tensional spring constant is kt , damping inherently possessed by the microstructure is b, the dynamic equation governing the tilting motion of the mirror (one pixel) is given by [11] I

d2 u du þ kt u ¼ t þb dt dt2

ð7:42Þ

Here, it is assumed that there exist no other inputs except torque. Equation 7.42 represents a nonlinear dynamic equation. Because the dynamics is rather complex due to t, we will approximate t by expanding it into a Taylor series expansion about u0 : The torque in Equation 7.41 then can be rewritten as;

t ðV; uÞ ¼ t ðV; u0 Þ þ þ

dt ðV; uÞ dt

d2 t ðV; uÞ dt2

u¼u0

u¼u0

ðu 2 u0 Þ

ðu 2 u0 Þ2 þ high order terms in ðu 2 u0 Þ

when u0 ¼ 0; we can express the above equation as

t ðV; uÞ ¼

1 1 0 w ‘2 V 2 1 10 w‘3 V 2 3 1 0 w ‘4 V 2 2 u þ u þ ··· þ 4 3 8 h20 h30 h40

ð7:43Þ

If we substitute this equation into Equation 7.42 and retain up to the ﬁrst order term in u, then the resulting equation is expressed by I

d2 u du þ kt u ¼ t^ ðV; uÞ þb 2 dt dt

ð7:44Þ

where t^ ðV; uÞ is given by

t^ ðV; uÞ <

1 10 w‘2 V 2 1 10 w‘3 V 2 þ u 4 3 h20 h30

It is noted that the torque t^ ðV; uÞ contains the product term V 2 u, which cannot be separable, thus exhibiting a strong nonlinearity. Using this simpliﬁed system dynamics, we will illustrate how this mirror rotation will behave according to a certain control action. Although the control system is highly nonlinear we will consider here a simple PID control. The error is deﬁned by e ¼ ud 2 u where ud is the desired mirror angle, in this case, þ 58 or 2 58. Let us simulate this by using Simulink. By letting x1 ¼ u, x2 ¼ du=dt, Equation 7.44 can be put into the following

Optomechatronic Systems in Practice

+– Step

PID PID Controller

KGain

531

DMD_system S-Function

Scope Output To Workspace

FIGURE 7.61 Simulink model for DMD simulation.

state space model, 1 0 0 1 dx1 x2 B dt C C B C _ ¼B X C ¼ @ kt B b t^ ðV; u Þ A @ dx2 A 2 x1 2 x2 þ I I I dt The Simulink model simulating the above equation is depicted in Figure 7.61, and the parameters used for simulation are given in Table 7.8. It shows typical simulation results obtained with various controller gains. In Figure 7.62a the response of the proportional control action is shown for the two proportional gains while keeping ki ¼ 25, kd ¼ 4: For this gain set, it shows a smooth transition to a steadystate value without any oscillation. Not like the phenomenon observed by a linear system, the steady state error appears approximately the same, regardless of variation of the proportional controller gain. The other response is the result of the gain set kp ¼ 85, ki ¼ 25, kd ¼ 1 £ 1022 : It can be seen that the response becomes oscillatory during transient time. The results of the integral control in Figure 7.62b show no appreciable change with its gain variation, and have improved steady state response, yielding no appreciable errors. When the derivative gain is applied, the result shows signiﬁcant difference in response speed, as can be

TABLE 7.8 The Parameters Used for the DMD Simulation Parameters

Symbol

Unit

Value

Moment of inertia Damping coefﬁcient Spring constant Gap between micromirror and electrode Electrode width Electrode height Permittivity

I b kt h0 w l 10

g m2 N m/(sec rad) mN m/rad mm mm mm F/m

355 £ 10212 170 £ 10212 10.5 18 190 190 8.854 £ 10212

Source: Lee, J.B., Introduction to MEMS, Lecture Note, EE7382, The University of Texas at Dallas, TX, 2004 (http://www.utdallas.edu/~jblee/EENU821/).

Optomechatronics

10 8

kp=85 kp=105 kp=85, ki=25, kd=0.01

6 4 2 0

0.0

0.1 time (t), μsec

6 mirror angle (θ), degree

mirror angle (θ), degree

532

5

3 2 1 0

0.2

(a) ki=25, kd=4

ki=5 ki=25 ki=45

4

0.0

0.1 time (t ), μsec

(b) kp=85, kd=4

5 4

kd=1 kd=4 kd=7

3 2 1

mirror angle (θ), degree

mirror angle (θ), degree

6

0 0.0

0.2

desired mirror angle system response

5

0

−5 0.1 time (t), μsec

0.2

(c) kp=85, ki=25

0

2

4 6 time (t), μsec

8

10

(d) kp=85, ki=25, kd=4

FIGURE 7.62 Response of the DMD for various controller parameters.

seen from Figure 7.62c. Figure 7.62d shows the performance of the DMD control in response to a series of switching actions, which is considered to be a fast and smooth response. It is noted that the desired torque to be input to the mirror can be computed based on incoming video data dictated by the memory on the DLP board. The torque command determines how the mirror needs to be optically switched on or off. Light Pulse Width Modulation Unlike most LCD based light modulators, the DMD is a binary device which modulates the reﬂected light intensity in a discrete fashion. Due to switching action of the mirror, it reﬂects light either into or out of the pupil of the projection lens to produce short-duration digital light pulses. These light pulses can be produced in a time sequential manner, such that human eye cannot discern whether they are discrete or analogue. Let us examine whether a DMD device can project light within this time limit. The rise time of the light pulse as the light enters the projection lens pupil is found to

Optomechatronic Systems in Practice

533

be less than 2 msec. The time for the mirror to settle and electromechanically latch is 16 msec, the time for video display is 16.67 msec, the time for a clear of the content of the memory cells is 12 msec, and ﬁnally a reset time to turn all the mirrors off. It can be seen that total of these times is not beyond 20 msec, which is interpreted as an analog image by human eyes. The arrangement describing the DMD-based light projecting principle is shown in Figure 7.63. The system has eight or ten bits color depth per primary color (RGB) which allows the projector to produce a gray ramp having 256 or 1024 discrete intensity values, respectively. To produce these digital light pulses binary-weighted pulse modulation is utilized. In this technique, the formatter of the DMD receives electrical words representing gray levels of brightness as its input, converts these into pulse-width modulated (PWM) light intensity values, and then projects these optical words on to the screen. Here, PWM intensity values imply that the intensity value is produced in proportion to the pulse width. The ﬁgure shows some details of the PWM technique for a two-bit word (22 or four gray levels) in a form of pixel image. These words are sent to the formatter and coded with PWM, according to the intensity level shown in the electrical words table. Each digit in the word represents a time duration for light to be on or off (1 or 0). This means that the mirrors allocated with 1 are on for a certain period of time. The time duration has two types, 20 and 21. The shortest interval (20) is

R er att

m

for

G

B

0

0

0

0

1

2

1

2

3

1

1

1

0

1

2

2

2

3

2

2

2

0

1

2

3

3

3

color pulse modulation chart

B G R

projection lens

2, (2,

(1,

DMD shaping lens

optics

) 2, 3

) 2, 3 (0,

(

, 2) 1, 1

(

, 2) 0, 1

B R

G

color filter

FIGURE 7.63 Light intensity modulation.

3)

) 1, 3 (2,

condensing lens

light source

(2,

( 1,

) 0, 3

) 0, 2

0, (0,

1)

screen

534

Optomechatronics

called least signiﬁcant bit and the longest interval (21) is called most signiﬁcant bit. The possible gray levels produced by all combinations of bits in the two-bit word are four equally spaced gray levels (0, 1/3, 2/3, 3/3), as shown in a grid form for each primary color. This relationship is shown for some combination of the RGB colors in a 3 £ 3 image screen. For instance, the ﬁrst grid in the upper left corner of the screen (represented by 0, 0, 1) implies that the only color appearing in the right bottom corner of the screen is blue, having time duration 1, where as the grid right below the ﬁrst (represented by 1, 0, 2) indicates ﬁrst, red of duration “1”, no green and then blue of duration “2.” It is noted here that the color display is done in a sequential manner for a 60 Hz video refresh rate, indicating that all bits of three colors need to be displayed within 16.67 msec. The overall system performance including contrast, brightness and uniformity of the image on screen is dependent upon a number of design considerations such as DMD’s dynamic and optical characteristics, illumination method, optical system conﬁguration, layout efﬁciency of optical elements, and system electronics. To discuss them all it requires more detailed analysis of optical design, DMD static and dynamic properties, color generation technique, system electronics, and so on. These topics are beyond the scope of this book, so we will not treat them here. However, it would be instructive to discuss the design of a simpliﬁed optical system of the projector system shown in Figure 7.63. Let us consider a practical problem for designing an optical system when a DMD is used to project a light beam on to a screen. As shown in Figure 7.64,

ih

iv

Airy disk oh

mirror ov screen

projection lens D

mirror array

ov

iv

s2 FIGURE 7.64 A DMD-based projection system.

s1

Optomechatronic Systems in Practice

535

the DMD projection system consists of a DMD, a projection lens, and a screen. The DMD has a dimension of oh £ ov and an nh £ nv number of micromirror arrays, while the screen has a dimension of ih £ iv : The lens has a focal length of f and an aperture diameter D. As shown in the ﬁgure, the projected image may have discontinuity due to the discrete characteristics of the mirror array, although the interval of discontinuity is very small. To compensate this possible defect, let us suppose that we can use Airy disks to ﬁll the discontinuity. The design problem is then to determine the lens parameters f and D, for a given Airy disk spot radius rspot ; a given wavelength l, the distance from the projector lens to the screen s1 , and a projector having a magniﬁcation ratio M. Having an equal ratio nh : nv ¼ ih : iv ¼ oh : ov , we will treat this system for simplicity as a one-dimensional (1D) optical projection system. The aperture diameter D is simply given by D¼

1:22ls1 rspot

ð7:45Þ

To determine the focal length of the lens, ﬁrst we consider the magniﬁcation factor expressed by s ð7:46Þ M¼ 1 s2 from which, the distance between the DMD and the projection lens, s2 can be determined. With s2 determined, we can obtain the focal length from the lens formula by s1 s2 f ¼ ð7:47Þ ðs1 þ s2 Þ This completes the design of the optical system of the DMD. When D and f are chosen in this manner, Airy disks may effectively ﬁll the blank discontinuity zone that may occur between pixel images. Grating Light Valve Display As mentioned from the beginning of this section, the diffraction projector called “grating light valve (GLV)” shown in Figure 7.55b, utilizes diffraction phenomenon observed from the grating surface of solid beams. Figure 7.65 depicts the basic principle of the GLV. When a voltage is not applied between the silicon beam and the substrate, the solid beam is not deﬂected. In this case, the light reﬂected from the surfaces of all the beams will be directed back to the light source. As depicted in Figure 7.65a in this case, no light will be projected on to the surface of the projector. However, when alternate beams are electrostatically actuated, the electrostatic force pulls the beams down and therefore light for a single pixel composed of six solid beams shown in Figure 7.65b breaks up into multiple diffraction orders upon reﬂection from the grating. Peak diffraction intensity will then occur with the ﬁrst order m ¼ ^1, which we shall see later in this section. Accordingly

536

Optomechatronics

incident

reflected

light

screen

(a) for undeflected beams

projection lens mirror

m= –1

m= 0

m= +1

diffracted

light diffracted light

diffracted grating

(b) for deflected beams

(c) simplified geometry of the optical arrangement

FIGURE 7.65 Illustration of the operating principle of the grating light value (GLV) display.

the diffracted light of peak amplitude is introduced to a collective lens, which collects, focuses, and projects the light on to a screen, as shown in Figure 7.65c. As noted from the preceding discussions, the important parameters associated with the solid silicon beam that inﬂuence light projection quality are the beam deﬂection, its width, and the pitch between beams, for they affect the intensity of the diffracted light and its directivity. A brief analysis will be made to relate these parameters to those optical parameters. Light Diffraction by Deformable Grating Modulator The deformed state of the silicon beams is represented as being ﬂat but, in fact, exhibits a speciﬁed curvature along its neutral axis compared to its width. When the beam is loaded with a uniform pressure across its whole length, it will be assumed that the beam is displaced with its maximum amplitude along the entire span [21]. To illustrate this, the cross-sectional view of the deﬂected beams are depicted in Figure 7.66. According to diffraction grating theory, discussed in Chapter 2, “Diffraction,” the amplitude of a wave at a point P with coordinates zs and y; Eðzs ; yÞ is given by Eðzs ; yÞ ¼

ð3w 23w

ei½kr þ dwð‘Þ d‘ r

ð7:48Þ

where w is the width of one solid beam, r is the distance between a point light source located at S00 , and a disturbance point P located at a long distance from the beams (mirrors). In the above r can be approximated by r < r0 2 ‘ sin u, where u is deﬁned by the angle between the line zs S and

Optomechatronic Systems in Practice

537 z

y

P(zs,y)

zs

screen light

diffracted light r

ro

θ undeformed

S" d

S S'

d

w deformed

FIGURE 7.66 Construction for determining irradiance for the diffracted light by phase grating.

PS. Here, r0 is the distance from S to P in the screen, i.e., the value of r when dwð‘Þ represents the phase change resulting from the light path length increase due to the displaced mirror. From the geometry shown in the ﬁgure the path length increase is given by

‘ ¼ 0: The

d ¼ dð1 þ cos uÞ Accordingly, the resulting phase change dwð‘Þ is expressed by

dwð‘Þ ¼ kdð1 þ cos uÞ

ð7:49Þ

where k is the wave number of the incident light. Now, if the phase change due to path difference is taken into account using Equation 7.49, Equation 7.48 can be rewritten as ð3w ð7:50Þ Eðzs ; yÞ ¼ eikr0 e2ik‘ sin u eidwð‘Þ d‘ 23w

Without loss of light power the intensity Iðzs ; yÞ is assumed to be collected and focused on to the screen by the lens as shown in Figure 7.65c. Making use of the fact that the pixel is composed of three identical pairs of solid beams, it can be shown that the integrated value of Eðzs ; yÞ can be approximated by the intensity of the diffracted light that reaches the maximum. For no beam deﬂection ðd ¼ 0Þ, this gives a minimum value, which is zero. In this case, light reﬂected from the mirror surface returns to its source, thus no light is projected on to the screen. A subsequent analysis in brief will show an important design aspect that mechanical motion of one quarter of a wavelength is sufﬁcient to provide the switching needed for a micromachined diffraction grating.

538

Optomechatronics

Deﬂection Control of Microbeam Structures The foregoing discussions on diffraction grating conditions indicate that an appropriate design procedure is needed to determine a suitable microstructure to meet the required deﬂection. To determine this we will use the Euler beam equation given in Appendix A2. In this analysis, it will be assumed that the structure has very small deﬂection. Substituting the approximated r value and Equation 7.49 into Equation 7.50, we obtain the following light wave amplitude [21]; o n Eðzs ; yÞ / eikr0 eikw sin u=2 eikdð1 þ cos uÞ þ e2ikw sin u=2 {1 þ 2 cosð2kw sin uÞ}

sin

kw sin u 2 kw sin u 2

ð7:51Þ

The last expression of the equation governs the diffraction pattern as seen from that obtained by a narrow aperture or slit as we have discussed in Chapter 2, “Diffraction.” The determination of the amplitude equation is left as an exercise problem. According to this, peak intensities occur at kw sin u ¼ mp;

m ¼ ^1; ^2; …; ^n

ð7:52Þ

The maximum occurs at the ﬁrst diffraction order m ¼ ^1: The other orders are composed of much weaker bands of diffracted light, which can be made outside the pupil of the collective lens shown in Figure 7.65c. If this value mp is substituted into the amplitude equation, we can obtain the condition for the projected light amplitude Eðzs ; yÞ as being the maximum value. It can be shown that the condition is given by cos 2kd ¼ 21

ð7:53Þ

The above condition indicates that, if d ¼ l=4, the amplitude of the diffracted light becomes a maximum value. It is noticed that this result has already been mentioned in the diffraction grating for the case of optical pits discussed in “Optical Storage Disk” in this chapter. Here again the result tells us that we need to design the geometry and material of the solid beam that can yield the required beam deﬂection at its midpoint. Let us design the dimension of the clamped-clamped beam shown in Figure 7.55b by determining the maximum deﬂection at its midpoint. The equation governing the small deﬂection of the microbeam structure under residual axial stress condition is rewritten by use of the Structural Beam Deﬂection in Appendix A2. EI

d4 z d2 z s wh 2 ¼p r dx4 dx2

ð7:54Þ

Optomechatronic Systems in Practice

539

where z is the beam displacement, E is the Young’s modulus, I is the moment of inertia of the beam cross section with respect to its neutral axis, x is the coordinate deﬁned along the beam longitudinal direction, sr is the residual stress in the beam, and p is the distributed load per unit length. In the above, w and h are the width and thickness of the beam, respectively. It is now necessary to determine the distributed load on the beam. Referring to the ﬁgure and utilizing the results obtained from the electrostatic actuator discussed in Chapter 4, “Actuators,” the load may be expressed by p¼

1AV 2 1wV 2 ¼ 2d2 L 2d2

ð7:55Þ

where V is the electrical voltage applied to the microbeam structure, A is the beam area of the structure facing the substrate, d is the original gap between the microbeam and substrate, and L is the length of the beam. If we neglect the dynamics of the microbeam structure, we can relate the deformation of the structure to the induced electrostatic force. Under the clamped boundary condition at both ends, z can be obtained by solving Equation 7.54 as z¼

p L{cosh ke ðx 2 L=2Þ 2 coshðke L=2Þ} xðL 2 xÞ þ 2s ke sinhðke L=2Þ

ð7:56Þ

where s and ke are given by

s ¼ sr wh sﬃﬃﬃﬃﬃﬃﬃﬃ 12s ke ¼ Ewh3 From this equation we can determine the maximum deﬂection that occurs at the center of the beam, x ¼ L=2 zlx¼L=2 ¼ zmax ¼

pL L coshðke L=2Þ 2 1 22 4s 2 ke sinhðke L=2Þ

ð7:57Þ

This maximum deﬂection expressed in Equation 7.57 can be used for the diffraction grating as if whole beam were deformed with this displacement. That is, d ¼ zmax

ð7:58Þ

This result reﬂects the fact that as illustrated in Figure 7.55b incident light irradiates approximately central part of the beams whose displacement is maximum. The above relationship enables us to design the microbeam structure for GLV. In summary, if the input voltage, the material properties, and the geometry of the microstructure are given, the diffraction grating distance

540

desired intensity

Optomechatronics

error +_

controller

V electrostatic Fs actuator

silicon beam

d

GLV system

actual intensity

photo sensor

FIGURE 7.67 A system block diagram for intensity control of the GLV.

can be determined from Equation 7.57. This equation is useful since it gives information on the relationship between the grating distance d and the input voltage, which ﬁnally lead us to determine the intensity of the diffracted light. Typical dimensions of the microbeam structure are reported to be , 200 mm for length, and 3 mm for width. The fact that the intensity of the diffracted light depends upon the grating distance leads to a very interesting control problem, when it is necessary to adjust the intensity for a given light source strength depending upon the incoming video information. This may be achieved by controlling the grating distance at a desired level in a closedloop manner. The closed-loop scheme requires a reliable light detecting sensor (PSD) that can measure the instantaneous intensity of the diffracted light. A feasible closed-loop control system is illustrated in Figure 7.67. The error between the desired and actual intensities is used to generate the grating distance, until it reaches a desired intensity value. If the controller is a PI type, then the control command signal is given by ðt V ¼ kp ðId 2 IÞ þ ki ðId 2 IÞdt 0

where Id and I are the desired and actual intensity, respectively, and kp and ki are the proportional and integral controller gains, respectively. This voltage signal drives the electrostatic actuator to vary the beam deﬂection at its midpoint. According to the actuator force Fs the microbeam again deforms at a certain displacement from which light diffracts. This newly diffracted light is collected by a lens and is again directed toward the screen.

Visual Tracking System Online video tracking of moving objects is now becoming a very important task in many industrial, robotic, and security systems. For example, in security areas, the tracking system should be capable of tracking and zooming an unexpected object, and recording a series of the extracted images. In Chapter 6, “Visual (Optical) Information Feedback Control,”

Optomechatronic Systems in Practice

541

we treated a visual servoing that utilizes the features of an object image in order to reduce the servoing time. The objects in that case were stationary with respect to time, which makes it relatively easier to extract the features for use of servoing. In tracking situations, however target objects are usually unknown in advance and furthermore moving with respect to time. This situation makes it rather difﬁcult to immediately choose and obtain the features of an object to be followed. In some cases, this will be more difﬁcult, when image recognition needs to be carried out additionally in real time while tracking. In this section, we will discuss a fundamental tracking concept based upon the feature extraction of the object images and treat a very simple case of tracking. The tracking system consists of a visual sensor, a visual-data processor, a moving mechanism such as a robotic manipulator, a pan-tilt mechanism, a 3D motion table, and a servo control unit that drives the mechanism. The sensory unit carries out the acquisition, processing, and interpretation of the information needed for deriving control signals to drive the tracking mechanism. In the method that will be discussed below, we will consider only a simple case, assuming that there is only one object to be followed in the environment. This implies that the vision system tracks only one object, while uncertain stationary objects, if any, are treated as background scene. To determine the moving object parameters such as motion and shape information, we need to go through three main stages, image processing, object information analysis based on the acquired image, and noise detection and ﬁltering. In this section, noise detection and ﬁltering will not be treated in order to focus only on the basic concept of tracking. Image Processing In the image processing stage, the processing needs to be carried out in order to determine such time-dependent information as the location of the object center, object shape, the direction of optical axes and focal length, and the object motion data such as moving velocity. To this end a series of images are taken by a vision camera sensor with time interval Dt; t ¼ t0 ; …tn21 ; tn ; tnþ1 …; where n is the nth sampling instant of taking image. Let us suppose that at the particular instants, the images of a particular scene containing a moving target object are obtained at two consecutive times t ¼ tn ; tnþ1 : The two images then go through edge operation by using one of the edge detection methodologies introduced in Chapter 3. In doing this it is very important to choose an appropriate edge operator which makes it suitable to extract the desired features for the acquired image. Let the extracted edge images Iðtn Þ and Iðtnþ1 Þ be taken at t ¼ tn and t ¼ tnþ1 , respectively. If temporal differentiation is operated on these two edge images, they are divided into the positive and negative in intensity levels.

542

Optomechatronics

2 This results in object edge images, Gþ u and Gu at time tn and is described by ( 1 if Gðx; y; tn Þ 2 Gðx; y; tnþ1 Þ . 0 þ Gu ðx; y; tn Þ ¼ 0 otherwise ð7:59Þ ( 1 if Gðx; y; tn Þ 2 Gðx; y; tnþ1 Þ , 0 2 Gu ðx; y; tn Þ ¼ 0 otherwise

Figure 7.68 depicts a scene image in the upper ﬁgure and the corresponding edge images in the lower ﬁgure. It shows the 2D edge images in which one object (human) is moving, while the other (block) is stationary. The value 100 in the image denotes the edge strength. Let Iðtn Þ and Iðtnþ1 Þ be the edge intensities obtained at t ¼ tn and tnþ1 , respectively. If Iðtn Þ is subtracted from Iðtnþ1 Þ, we can get the result shown in the right hand side of the ﬁgure. According to Equation 7.59, the edge strength þ 100 is denoted as “1” in the open parenthesis while 2 100 is also denoted as “1”. The rest of the scene area is assigned “0.” This kind of image processing can be done without difﬁculty in a real tracking situation, when a camera is mounted on a robot or mechatronic system that can move in space with some degree of freedom, for instance, six degrees of freedom composed of three translational and three rotational, as discussed in Chapter 6, “Visual Tracking System”. A pan-tilt device holding a CCD video camera is one of the frequently-used moving mechanisms. Figure 7.69 shows a 2D pan-tilt mechanism tracking a scene which contains a moving toy car and two blocks. As can be seen from the ﬁgure, the car is moving toward a triangular block, while a rectangular block is stationary. The processed images of the scene taken at two different times tn and tnþ1

t = tn

t = tn+1

(a) scene image

100

100

–

Gu

+

Gu 100

–100

I (tn)

I (tn +1)

(b) the extracted edges FIGURE 7.68 Operation on the images of a moving scene acquired at two consecutive times.

Optomechatronic Systems in Practice

543

FIGURE 7.69 Pan-tilt device for tracking a moving image.

by using the moving device are shown in Figure 7.70. In this image tracking, the FOV of the camera is assumed to be kept constant. Using this image the edge operation is made by Sobel operator and the edge strength Iðtn Þ is subtracted from Iðtnþ1 Þ: The result is shown in the right-hand side of the ﬁgure. We notice that the result contains noisy images due to shadow and some other factors which often makes it difﬁcult to discern the object image from the background.

FIGURE 7.70 Edge operation on the images taken at two consectutive times.

544

Optomechatronics

Feature Extraction Once these edge images are obtained, extracting the information from these edges relevant to the moving object is the next step for tracking. The important information includes the center location and orientation of the object at two consecutive times, and the size of the object. The change in its center location can be used to calculate the object velocity, whose information is then used for calculating the control algorithm. Spatial distribution of image intensity in the image Gðtn Þ is important information to obtain concerning the orientation of the object t ¼ tn : The second moment of inertia of the image of the object is useful to determine this, from which the principal axes can be obtained. Figure 7.71 and Figure 7.72 illustrate how to calculate the center and the second moment of inertia of the moving target object, respectively. Since the center location (centroid) of an object in image can be calculated as arithmetic means of the object-occupied area, it is described in x-y coordinates in the image as X X mx ðtn Þ ¼ i £ Gði;j; tn Þ=Mn ; my ðtn Þ ¼ j £ Gði; j; tn Þ=Mn ð7:60Þ ði;jÞ[W

ði;jÞ[W

where W is the image ﬁeld deﬁned at t ¼ tn , mx ðtn Þ and my ðtn Þ represent the centroid at the instant tn in two coordinates, respectively, and Mðtn Þ is

moment of inertia

object movement detection + Go –100 y

+

G0

A=

y

(x, y ) x

center Axx Axy Ayx Ayy

Axx = 1 ∑ (xi – x )2 N i Axy = Ayx = Ayy =

+

100

− Go x

(x i, y i)

−

+

1 ∑ ( xi – x ) ( yi – y ) N i

1 2 ∑ ( yi – y ) N i

FIGURE 7.71 Center of mass and the second moment of inertia.

x=

1 x N ∑i i

y=

1 y N ∑i i

Optomechatronic Systems in Practice

545

principal axes analysis e2

object rotation e2

e1 +

y

Go–

y

Go– x

+ θ

e1

y

x

θ = tan–1 (e1 / e1 )

x

Axx Axy A = A A = e1 e2 yx yy

l1 0

0 l2

e1 e2

T

where l1 > l2 FIGURE 7.72 Determining the orientation of an object.

the number of pixels given by X Mðtn Þ ¼ Gði; j;tn Þ

ð7:61Þ

ði;jÞ[W

In the above, i and j indicate the pixel at the location of the ith column and jth row in the image, respectively. When Equation 7.59 and Equation 7.60 2 are used, the centroid of the object edge images, Gþ u ðx;y;tn Þ and Gu ðx;y;tn Þ can be obtained. Once these are obtained, we now compute the shift between the two locations obtained at t ¼ tnþ1 , and t ¼ tn , in order to determine the object velocity at t ¼ tn : 2 Dxi ðtn Þ ¼ mþ x ðtn Þ 2 mx ðtn Þ

2 Dyi ðtn Þ ¼ mþ y ðtn Þ 2 my ðtn Þ

ð7:62Þ

where xi and yi are the x and y coordinates, respectively, in the image coordinate system. As illustrated in Figure 7.71, the orientation of an object can be determined by using the second moments of the object inertia which are indicative of the mass distribution. We ﬁrst determine them as X X Axx ¼ {i 2 mx ðtn Þ}2 Gði; j; tn Þ Ayy ¼ {i 2 my ðtn Þ}2 Gði; j; tn Þ ði;jÞ[W

Axy ¼ Ayx ¼

ði;jÞ[W

X ði;jÞ[W

2

2

{i 2 mx ðtn Þ} {j 2 my ðtn Þ} Gði; j; tn Þ

ð7:63Þ

With these second moments of inertia of the object image, we form a matrix to obtain its principal axes in the x and y directions as follows; " # Axx Axy A¼ Axy Ayy

546

Optomechatronics

where in the above the symmetric relationship Axy ¼ Ayx is used. Eigenvalue analysis for the eigenvalue problem Ax ¼ lx is well known to determine such principal axes. The eigenvalue analysis leads us to obtain the eigenvectors ½e1 ; e2 ; and the scalar eigenvalues l1 and l2 , from which the principal axes of the object can be computed by y

u1 ¼ tan21 ðe1 =ex1 Þ

y

u2 ¼ tan21 ðe2 =ex2 Þ

ð7:64Þ y

y

where e 1 , e x1 are the components of the ﬁrst eigenvector e1 and e2 ; ex2 are those of the second eigenvector e2 : If l1 . l2 , u1 becomes the major axis of the object, whereas u2 is the minor. Let us practice determining the centroid and the principle axes of an object image obtained at a certain time. An image of an ellipsoid (object) is composed of 11 £ 11 pixels as shown in Figure 7.73a. Assume that its edge processed image is obtained as given in Figure 7.73b and that the edge intensity map is represented as in Figure 7.73c. To calculate the geometric

(a) image

(b) edge j

10 9 8 7 6 5 4 3 2 1 0

0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 1 1 1 0 0 1

0 0 0 0 1 1 0 0 0 1 0 2

0 0 0 1 0 0 0 0 0 1 0 3

0 0 1 0 0 0 0 0 0 1 0 4

0 1 0 0 0 0 0 0 0 1 0 5

0 1 0 0 0 0 0 0 1 0 0 6

0 1 0 0 0 0 0 0 1 0 0 7

0 1 0 0 0 0 1 1 0 0 0 8

0 0 1 1 1 1 0 0 0 0 0 9

(c) edge intensity image FIGURE 7.73 Determining the centroid and principle axes of an ellipsoid.

0 0 0 0 0 0 0 0 0 0 0 10

i

Optomechatronic Systems in Practice

547

centroid of nonzero pixels, we use Equation 7.60; mx ¼

10 X 10 1 X i £ Gði; jÞ Mn j¼0 i¼0

¼ ð1 £ 3 þ 2 £ 3 þ 3 £ 2 þ 4 £ 2 þ 5 £ 2 þ 6 £ 2 þ 7 £ 2 þ 8 £ 3 þ 9 £ 4Þ=23 ¼ 119=23 ¼ 5:17 my ¼

10 X 10 1 X j £ Gði; jÞ Mn j¼0 i¼0

¼ ð1 £ 4 þ 2 £ 3 þ 3 £ 2 þ 4 £ 2 þ 5 £ 2 þ 6 £ 2 þ 7 £ 2 þ 8 £ 2 þ 9 £ 4Þ=23 ¼ 112=23 ¼ 4:87 The use of Equation 7.63 leads us to obtain the second moments of inertia, Axx ; Ayy ; Axy : Axx ¼

10 X 10 X j¼0 i¼0

ði 2 mx Þ2 £ Gði; jÞ ¼ ð1 2 5:17Þ2 £ 3 þ ð2 2 5:17Þ2 £ 3

þ ð3 2 5:17Þ2 £ 2 þ ð4 2 5:17Þ2 £ 2 þ ð5 2 5:17Þ2 £ 2 þ ð6 2 5:17Þ2 £ 2 þ ð7 2 5:17Þ2 £ 2 þ ð8 2 5:17Þ2 £ 3 þ ð9 2 5:17Þ2 £ 4 ¼ 185:3 Ayy ¼

10 X 10 X j¼0 i¼0

ðj 2 my Þ2 £ Gði; jÞ

¼ ð1 2 4:87Þ2 £ 4 þ ð2 2 4:87Þ2 £ 3 þ ð3 2 4:87Þ2 £ 2 þ ð4 2 4:87Þ2 £ 2 þ ð5 2 4:87Þ2 £ 2 þ ð6 2 4:87Þ2 £ 2 þ ð7 2 4:87Þ2 £ 2 þ ð8 2 4:87Þ2 £ 2 þ ð9 2 4:87Þ2 £ 4 ¼ 192:61 Axy ¼

10 X 10 X j¼0 i¼0

ðj 2 my Þði 2 mx Þ £ Gði; jÞ

¼ ð1 2 4:87Þ{ð2 2 5:17Þ þ ð3 2 5:17Þ þ ð4 2 5:17Þ þ ð5 2 5:17Þ} þ ð2 2 4:87Þ{ð1 2 5:17Þ þ ð6 2 5:17Þ þ ð7 2 5:17Þ} þ ð3 2 4:87Þ{ð1 2 5:17Þ þ ð8 2 5:17Þ} þ ð4 2 4:87Þ{ð1 2 5:17Þ þ ð8 2 5:17Þ} þ ð5 2 4:87Þ{ð2 2 5:17Þ þ ð9 2 5:17Þ} þ ð6 2 4:87Þ{ð2 2 5:17Þ þ ð9 2 5:17Þ} þ ð7 2 4:87Þ{ð3 2 5:17Þ þ ð9 2 5:17Þ} þ ð8 2 4:87Þ{ð4 2 5:17Þ þ ð9 2 5:17Þ} þ ð9 2 4:87Þ{ð5 2 5:17Þ þ ð6 2 5:17Þ þ ð7 2 5:17Þ þ ð8 2 5:17Þ} ¼ 68:52 "

Finally, by forming matrix A # 185:3 68:52 68:52 192:61

548

Optomechatronics

we obtain its eigenvectors which determine the principal axes of the object. The eigenvectors are given by; e1 ¼ 0:69 0:73 T ;

e2 ¼ 0:73 20:69

T

where T denotes the transpose of the row vectors. From Equation 7.62, the magnitude of the velocity v, and direction of the target object in the image coordinates u can be determined as qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ qﬃﬃﬃﬃﬃﬃﬃﬃﬃ Dxðtn Þ2 þ Dyðtn Þ2 ð7:65Þ vðtn Þ ¼ or vðtn Þ ¼ v2x þ v2y Dt

u ¼ tan21

Dy Dx

where Dt ¼ tnþ1 2 tn and vx and vy are the velocity of the object at t ¼ tn in the x and y directions, respectively. Equation 7.62 and Equation 7.65 provide the instantaneous current location, velocity, and the moving vector direction of the target. These quantities, instantaneously obtained at t ¼ tn by the camera, are fedback to the motion controller which controls the camera position at that time. Visual Tracking of a Moving Object The visual object tracking consists of three modes; the zoom and focusing mode, the estimation of object motion based on the processed image, and the control of visual sensor. Normally, these modes are executed in sequence, that is, keeping the target object within the FOV, estimation of the object position/velocity, and then control of the camera servoing unit to direct optical axis of the camera toward the object center position. As discussed in Chapter 6, in “Zoom Control” and “Visual Auto Focusing”, the zoom control allows us to instantaneously varying the object size in consideration of object motion. On the other hand, the control of focal length f ðtn Þ of the focusing lens enables the sensor to obtain a well-focused object by changing its resolution in image. In this subsection, we will discuss zooming control in brief and then will consider a visual tracking problem. In the previous subsection, we have estimated the object motion by determining the object center location, and the orientation and size of the object in the image processing stage. Based on this image information, we further need to compute how we control the camera position and orientation so that the camera always keeps track of an object of interest. This necessitates design of the tracking controller. For example, if a camera is mounted on a pan-tilt device, the angle upan ðtn Þ, utilt ðtn Þ at t ¼ tn , which determines the angle pointing toward the centroid of the object provides the data needed to determine the feedback input to the pant-tilt control mechanism. The camera rotation of the pan upan ðtn Þ and tilt utilt ðtn Þ angles centering the object on the image can be related to Dxi and Dyi in the image coordinate.

Optomechatronic Systems in Practice

549

The two angles at t ¼ tn are given by

upan ðtn Þ ¼ f {Dxðtn Þ; Dyi ðtn Þf ðtn Þ}

utilt ðtn Þ ¼ f {Dxi ðtn Þ; Dyi ðtn Þ; f ðtn Þ}

ð7:66Þ

It is noted that the f ðtn Þ varies at every sampling period, Dt: We will derive the above relationship in the next. Zoom Lens Control

focus length (f ), mm

The control of focal length to instantaneously zoom into or out from the object with a view to obtain a desired object size largely depends on the estimated size of the object. This is because the resolution of the tracked object in the scene is being controlled in accordance with the desired size. Special care must be taken when a small object to be tracked is in high or random motion. In this case, it is necessary to track it with wide zoom range in order not to lose the object from the FOV. A simple zooming principle was treated in Chapter 6, “Zoom Control” using a camera optical system consisting of three lenses, two of which are objective and one is the convex. Here we will not apply its principle in detail, but instead will use a typical relationship between the control signal and focus length illustrated in Figure 7.74. The relationship shown here somehow exempliﬁes a nonlinear relationship in that as the control signal gets higher the focal length drastically increases, showing high sensitivity to the control signal. In this case we assume that the relationship is calibrated a priori for a chosen FOV. Based on the above discussions the zoom control signal can be written as a function of the current focal length, f, the estimated object size A, and the derivative of the focal length with respect to control signal, df =dt: If the current control signal is denoted by i, the control signal can be derived from

control current (i ), ampere FIGURE 7.74 Focal length vs. control current.

550

Optomechatronics

the ﬁgure as Diðtn Þ ¼

df di

21 t¼tn

Df ðtn Þ ¼

df ðiðtn ÞÞ diðtn Þ

21

f ðtn Þ

Ap 21 Aðtn Þ

ð7:67Þ

where the derivative of f ðtn Þ with respect to the control current can be approximated by df ðiÞ di

t¼tn

<

f ðtn Þ 2 f ðtn21 Þ iðtn Þ 2 iðtn21 Þ

ð7:68Þ

and Ap is the desired object size to be focused. Equation 7.67 is the equation that generates the control current keeping the instantaneous object area Aðtn Þ seen in the image at a desired size Ap : To illustrate the concept of visual control, we will consider a car slowly moving on a plane ðXw ; Yw Þ deﬁned in the world coordinates. Under the tracking situation shown in Figure 7.75 the car leaves initial points and follows the speciﬁed path which is terminated at point G. The tracking system consists of a camera, a pan-tilt moving device ﬁxed in a frame and a tracking controller. The objective of the visual tracking is to have the image of the car positioned always in the same position and orientation in the image frame, regardless of the change of its position and heading angle as shown Figure 7.76. In this visual tracking we will not consider the z coming and auto-focusing and therefore assume that they are always carried out automatically before the image acquisition stage. Under this condition we need to take care of camera positioning control only. In other words, we assume that zoom and focusing are appropriately controlled to a desired state, so that the camera does not need instantaneous adjustments of their values. Let us begin our discussion by deﬁning the coordinate systems. The desired position and orientation are indicated in gray while those of the currently imaged are depicted in dotted lines. Let ðxdi ; ydi ; i ¼ 1; 2Þ deﬁned in image frame denote the desired coordinates which consist of a mass center ðxd1 ; yd1 Þ and a point determining the car orientation, ðxd2 ; yd2 Þ: Also, let the corresponding current coordinates of the car be ðxi1 ; yi1 Þ and ðxi2 ; yi2 Þ; respectively. Then, the instantaneous errors at the two points associated with the moving car can be deﬁned by " # " # ðxd1 2 xi1 Þ ðxd2 2 xi2 Þ e1 ¼ and e2 ¼ ð7:69Þ ðyd1 2 yi1 Þ ðyd2 2 yi2 Þ By deﬁning the instantaneous error like this, we can now apply the visual servoing algorithm discussed in Chapter 6, “Visual (Optical) Information Feedback Control.” To derive a servoing signal controling the motion of the camera rotating in two axes, we will consider the camera coordinate system for tracking as depicted in Figure 7.77. The coordinates are deﬁned in ðxc ; yc ; zc Þ: In the ﬁgure, vcx and vcz are the angular velocity of camera rotation in x-axis, and that in y-axis, respectively, these are the control

Optomechatronic Systems in Practice

551

xc panning tilting

rolling

yc

t n-1

tn

Yw

(X

zc

t n-2

G

mg

,Y

mg

t =t 2

)

X

t =t 1

w

(X

wo

,Y

wo

S

)

car

FIGURE 7.75 Tracking a car traveling on a plane ðXw ; Yw Þ:

variables to make the camera track the moving car at each control time. The feature points, Pc1 and Pc2 attached to the car are those deﬁned in the principal direction of the car which will be assumed to be known. When the camera frame is rotating with vcx , vcy and vcz , the velocity of point Pc in the camera coordinate is described by x_ c ¼ zc vcy 2 yc vcz

y_ c ¼ xc vcz 2 zc vcx

z_ c ¼ yc vcx 2 xc vcy

ð7:70Þ

According to the perspective projection, point ðxc ; yc Þ deﬁned in the camera coordinates is related to a point projected in the image coordinates by xi ¼

f x zc c

yi ¼

f y zc c

ð7:71Þ

Differentiating both sides of Equation 7.71, we have x_ i ¼ f

x_ c zc 2 xc z_ c z2c

y_ i ¼ f

y_ c zc 2 yc z_ c z2c

ð7:72Þ

552

Optomechatronics

,y ) Ppd1d1 ((xx d1 d1 , y d1 d1 )

Pd2 ( x d2 , yd2 )

yi

e2

Pi2 ( x i2 , yi2 )

principal axis

e1 xi

q

Pi1 ( x i1 , y i1) )

FIGURE 7.76 Desired and current image of a car in image coordinates.

wcy ( tilting ) yC wcx ( panning ) wcz ( rolling )

xC zC

Pc2 = (xc2 , yc2 , zc2)

FIGURE 7.77 Camera coordinate system for tracking.

Pc1 = (xc1 , yc1 , zc1)

Optomechatronic Systems in Practice

553

Substituting Equation 7.72 into Equation 7.70 yields to express the relationship between the velocity of the image and that of the camera coordinates. 3 2 3 2 xi yi f 2 þ x2i " # 6 2 2yi 7 vcx x_ i f f 76 7 6 76 vcy 7 ¼6 74 5 6 2 2 5 4 2f 2 yi y_ i xi yi xi vcz f f This expression can be applied for the two feature points in the image coordinates; 3 2 xi1 yi1 f 2 þ x2i1 2 2y i1 7 6 f f 7 2 3 6 7 6 72 6 x_ i1 2 2 3 7 6 2f 2 yi1 x y i1 i1 6 7 6 7 vcx x i1 7 6 y_ 7 6 f f 6 i1 7 6 76 7 6 7¼6 76 vcy 7 6 7 6 74 5 2 2 6 x_ i2 7 6 7 f þ xi2 4 5 6 2 xi2 yi2 7 v 2y i2 7 6 cz f f 7 6 y_ i2 7 6 7 6 5 4 2f 2 2 y2i2 xi2 yi2 xi2 f f Let the above matrix be noted by J 3 2 xi1 yi1 f 2 þ x2i1 2y 2 i1 7 6 f f 7 6 7 6 7 6 7 6 2f 2 2 y2i1 x y i1 i1 7 6 x i1 7 6 f f 7 6 7 J¼6 7 6 2 2 7 6 x y f þ x i2 i2 i2 7 6 2 2y i2 7 6 f f 7 6 7 6 7 6 5 4 2f 2 2 y2i2 xi2 yi2 xi2 f f

ð7:73Þ

where J is called the image Jacobian or interaction matrix. This matrix transforms the velocities of two points deﬁned in the camera frame into those of the corresponding points in image frame. 2 3 x_ i1 2 3 6 7 vcx 6 y_ 7 6 7 6 7 6 vcy 7 ¼ J þ 6 i1 7 4 5 6 7 6 x_ i2 7 4 5 vcz y_ i2 where J þ denotes pseudo inverse and is expressed by J þ ¼ ðJ T ·JÞ21 J T :

554

Optomechatronics

Finally, we have the expression for vcx, vcy and vcz using the error values deﬁned in Equation 7.69. 2

2

3

xd1 2 xi1

3

7 6 vcx 6y 2 y 7 6 d1 6 7 i1 7 þ 7 6 vcy 7 ¼ KP J 6 7 6 4 5 6 xd2 2 xi2 7 5 4 vcz yd2 2 yi2

ð7:74Þ

where KP denotes the diagonal proportional gain. The above equation implies that control signals for vcx, vcy and vcz can be produced, once we know the instantaneous error, because J þ is instantaneously calculated from Equation 7.73. In each tracking period, a series of servoing action take place repeatedly to track the car in the image plane in a speciﬁed position and orientation. Therefore, the controlling time period needs to be much faster than the speed of the moving car. Car tracking by means of visual servoing discussed above can be simulated if the tracking condition is given. As shown in Figure 7.78a, the objective of the tracking is to move a camera to follow a car running on a plane ðXw ; Yw Þ along a half circle path. The camera is positioned at the origin of the camera coordinate system (10m, 10m, 10m) and freely rotates with respect to the three camera axes, xc ; yc ; and zc : The car starts to travel from the origin S and reaches the terminal point G as depicted in Figure 7.78b. The velocity V of the car along the path is 0.033 m/s. The coordinates of two feature points on the car in the world coordinate denoted by Pw1 ðXw1 ; Yw1 Þ and Pw2 ðXw2 ; Yw2 Þ can be calculated using

camera holder (10,10,10) ωcx ωcy yc ω cz xc zc

10 8

Pw2

Yw (m)

Yw Zw Xw S

Pw1

20m

G

6 4 2

10m

(a) parameter setting in simulation FIGURE 7.78 Simulation condition for a car tracking.

0

S 0

5

10 Xw (m)

15

G 20

(b) trajectory of car in word coordinates

Optomechatronic Systems in Practice

555

following equations: Xw1 ðkÞ ¼ R 2 R cos

V kDt R

Xw2 ðkÞ ¼ Xw1 ðkÞ þ d sin

V kDt R

Yw1 ðkÞ ¼ R sin

V kDt R

Yw2 ðkÞ ¼ Yw1 ðkÞ þ d cos

V kDt R

ð7:75Þ

where R is the radius of the trajectory, which equals to 10 m, and d is the distance between Pw1 and Pw2 ; which equals to 0.1 m. At the initial state, the coordinates of Pw1 and Pw2 are equal to (0m, 0m) and (0m, 0.1m), respectively. In the above equation, Dt denotes sampling time which for this simulation is taken as 0.03 sec. The tracking procedure discussed below describes the coordinate transformation, zooming, error computation, and angular velocity control input at each servoing instant. Step 1: Transform the feature points of the car in world coordinates to those in the camera coordinates as follows: Pc ¼ R C TW Pw

ð7:76Þ

where Pw is the vector representing the feature points of the car deﬁned in the world coordinates, whereas Pc is the vector representing those transformed in the camera coordinates, and C TW is the transformation matrix of the world coordinate frame with respect to the camera coordinates. The matrix C TW is deﬁned by C

TW ¼ Rða; b; gÞTðx; y; zÞ

where Rða; b; gÞ is the rotation matrix deﬁned by 2

cos a

2sin a

0

6 6 sin a cos a 0 6 Rða; b; gÞ ¼ 6 6 6 0 0 1 4 0 0 0 2 1 0 0 6 6 0 cos g 2sin g 6 6 6 6 0 sin g cos g 4 0

0

0

0

32

cos b

76 6 07 76 0 76 76 6 07 54 2sin b 1 0 3 0 7 07 7 7 7 07 5 1

0

sin b

1

0

0

cos b

0

0

0

3

7 07 7 7 7 07 5 1

556

Optomechatronics

and the translation 2 1 0 6 60 1 6 Tðx; y; zÞ ¼ 6 6 60 0 4 0

0

matrix, Tðx; y; zÞ; is given by 3 0 x 7 0 y7 7 7 7 1 z7 5 0

1

In the above equations, a, b, g are the rotation angles with respect to the axes zc ; yc ; and xc ; respectively, and x, y, z is the translation along Xw ; Yw ; Zw ; respectively. In our simulation x equals to 2 10, y equals to 2 10, and z equals to 2 10. Step 2: Transform the feature points Pc into those in the image coordinates by using Equation 7.71. " # f f xc P ¼ ð7:77Þ Pi ¼ zc c z c yc Step 3: Obtain the instantaneous focal length f ðkÞ at the kth servoing instant in order to keep the size of the car image at a constant value. f ðkÞ ¼ f ðk 2 1Þ

Dd12 Di12 ðk 2 1Þ

where Dd12 is the desired distance between the two desired feature points Pd1 and Pd2 and Di12 is the distance varying with time between the two current feature points Pi1 and Pi2 : They are deﬁned by following equations: qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Dd12 ¼ ðxd1 2 xd2 Þ2 þ ðyd1 2 yd2 Þ2 qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ Di12 ðkÞ ¼ ðxi1 ðkÞ 2 xi2 ðkÞÞ2 þ ðyi1 ðkÞ 2 yi2 ðkÞÞ2 where points Pd1 ðxd1 ; yd1 Þ and Pd2 ðxd2 ; yd2 Þ are the desired positions of the two feature points in the image plane. Step 4: Calculate the errors in the image between the desired position and current position, and orientation error of the car. They are deﬁned by 2 3 2 3 2 3 2 3 e1x e2x xd1 2 xi1 xd2 2 xi2 5; e2 ¼ 4 5 ¼ 4 5 and e1 ¼ 4 5 ¼ 4 e1y e2y yd1 2 yi1 yd2 2 yi2 ðyi1 2 yi2 Þ ð7:78Þ u ¼ tan21 ðxi1 2 xi2 Þ In this simulation, Pd1 ¼ (0mm, 0mm), Pd2 ¼ (0.74mm, 0mm), and u ¼ 20:12011 rad are used. The initial e1x, e1y, and u is 2 0.0038 m, 2 0.0032 m, and 2 0.0244 rad, respectively, which are calculated by Equation 7.75 through Equation 7.78 when the car is at the start point.

Optomechatronic Systems in Practice

557

Step 5: Calculate the image Jacobian matrix deﬁned in Equation 7.73. Step 6: Determine the velocity control input signal, vcx ; vcy ; and vcz from Equation 7.74. The proportional control gain used herein is given by 2 3 10 0 0 6 7 7 Kp ¼ 6 4 0 10 0 5 0

0

10

Step 7: Calculate the instantaneous angular positions, a, b, g of the camera motion based on the following equation:

gðk þ 1Þ ¼ gðkÞ þ vcx Dt bðk þ 1Þ ¼ bðkÞ þ vcy Dt aðk þ 1Þ ¼ aðkÞ þ vcz Dt

3

3

2

2

1

1

e2y(m) x10–3

e2x(m) x10–3

where Dt denotes the step size in time. In this simulation, Dt equals to 0.03 sec, gð0Þ equals to 0.7724 rad, bð0Þ equals to 0.6455 rad, and að0Þ equals to 1.0302 rad. This completes one visual step for tracking the car.

0 –1 –2

–2 –3

–3 –4

0 –1

0

50

100 150 iteration number

200

–4

0

50

100 150 iteration number

(a) position error in the image plane 10

θ (rad) x10–3

5 0 –5 –10 –15 –20 –25 0

50

(b) orientation error in the image plane FIGURE 7.79 The results for car tracking.

100 150 iteration number

200

200

558

Optomechatronics

Step 8: Go to step 1 for a new servoing step. Figure 7.79 depicts the simulation results for the position error and orientation error in the image plane. The ðe1x ; e1y Þ is the position error of the car center point with respect to the origin of the desired car coordinates. It can be seen that the error values exhibit an overshoot, but ﬁnally, reaches the desired path. The oscillation persists approximately until 1.5 sec. The orientation error shows a similar trend as that observed from the position error of the car center point.

Problems P7.1. Consider a 36-facet scan mirror used for a laser printer. If the scanner has a scan ﬁeld angle (total scan angle) of 0.25 rad, compute the diameter of the beam illuminating a facet. P7.2. We wish to measure the surface topology of a sample using an AFM whose cantilever dimension is shown in Figure P7.2. If the deﬂection of the cantilever is obtained to be 10 nm, determine the displacement of the laser spot in the PSD dy0 during the x-y scan. The distance between point B on the sample surface and the PSD is given by r ¼ 20 mm. Assume there is no friction between the sample surface and the tip. The parameters related to the geometry and shape of the cantilever is given by ‘1 ¼ 50 mm, ‘2 ¼ 100 mm, a1 ¼ 5 mm, a2 ¼ 10 mm, h ¼ 1 mm.

y' PSD

laser y'

A

x'

A

y’

o' z'

r y

PSD

x

B cantilever

Px

1

q

z

e

x'

x

Py

2

l22 w

w

b(x)

w 0< − x<

1

beam cross-section

FIGURE P7.2 The cantilever geometry of an AFM.

l11

h

h

1

< − x<

a1 2

a2

Optomechatronic Systems in Practice

559

P7.3. In Figure 7.44c, an auxiliary lens is additionally used to completely ﬁll the objective lens with light while losing almost no light power. To achieve this, the rotating mirror must be placed in front of the auxiliary lens. Prove that the mirror should be located at the distance d1 from the auxiliary lens, as given in Equation 7.29 written below d1 ¼ ðd2 þ d3 Þ

d2 d3

P7.4. In a confocal system, a pinhole takes an important role as an aperture either in the detector unit or the light source unit or both. Figure P7.4 shows two extreme cases of the pinhole size through which light passes towards an objective lens, too small in size and too large in size. Discuss the effect of pinhole size (ra , radius of the pinhole) in terms of entrance pupil of the lens, numerical aperture (NAobj), loss of beam power, and axial resolution. laser beam

laser beam

pinhole 2ra

2ra objective lens

object FIGURE P7.4 Effect of pinhole size.

P7.5. Consider the case of an AFM which operates in contact mode. The parameters of the AFM system are identical to those listed in Table P7.5. If the height of an artifact is given in Figure P7.5, the system should follow this artifact under a constant force. Plot the result of the controller response of the AFM tip for kp ¼ 0:1 and ki ¼ 5 £ 102 : P7.6. In a DMD projector, the DMD switching motion is dependent upon how the bias voltage, Vb and address voltages, V1 and V2 are applied to the mirror and address electrodes, respectively. Here V1 is the address voltage in the left-handside, while V2 is that of the right-hand side. In Figure P7.6, a sequence of mirror rotation from state (1) to state (5) is shown for a period from a state in motion transition to landing at the right-hand side. Explain how these three voltages mentioned in the above can be applied to induce

560

Optomechatronics

TABLE P7.5 The Parameters Used for the AFM Simulation Parameters

Symbol

Unit

Value

Mass Damping coefﬁcient Spring constant of cantilever Stiffness of the tip-artifact interface Width of cantilever Height of cantilever Length of cantilever Elastic modulus

m b k ki w h L E

kg Nsec/m N/m N/m mm mm mm GPa

7.1088 £ 10212 1.5633 £ 1027 4.2974 3 13 6.5 300 130

artifact height(za), nm

Source: Rutzel, S., Proceedings, Mathematical, physical, and engineering sciences, 459:2036, 1925–1948, 2003.

10

5

0

0

1

2

3 time(t ), ms

4

5

FIGURE P7.5 Artifact input to AFM.

(1) V1

Vb

(4)

V2

(2) V1

Vb

Vb

Vb

V2

(5)

V2 (3)

V1

V1

V2

FIGURE P7.6 An illustration of a switching sequence.

V1

Vb

V2

Optomechatronic Systems in Practice

561

the necessary switching motion. Assume that V1 , 10 V, V2 , 10 V and Vb , 25 V. P7.7. Consider the design problem illustrated in Figure 7.64. Suppose that the system parameters are given as follows;

l ¼ 70 nm;

M ¼ 40; or ¼ 15 mm;

ih ¼ 90 cm;

nh ¼ 1280;

iv ¼ 60 cm

nv ¼ 960

oh ¼ 20 mm;

s1 ¼ 2 m

and the spot size of the Airy disk is required to be one sixth of the mirror pitch, as indicated in Figure P7.7. Determine the distance between the DMD and the projection lens, s2 , the aperture diameter, D, and the focal length of lens, f. ih

p iv

Airy disk mirror screen FIGURE P7.7 Airy disk projected in the projector.

P7.8. Using the wave amplitude given in Equation 7.51, Eðzs ; yÞ / eikr0 {eikw sin u=2 eikdð1 þ cos uÞ þ e2ikw sin u=2 } kw sin u sin 2 £ {1 þ 2 cosð2kw sin uÞ} kw sin u 2 determine the amplitude of the diffracted light at a point Pðzs ; yÞ: Show that peak values occur at kw sin u ¼ mp

m ¼ ^1; ^2; …; ^n:

P7.9. A GLV is composed of six identical structural beams. As shown in Figure P7.9 each beam has the following geometrical dimensions and tension due to thin ﬁlm deposit, L ¼ 120 mm;

h ¼ 0:5 mm;

E ¼ 250 GPa

sr ¼ 400 MPa

w ¼ 2:5 mm;

h0 ¼ 1 mm;

and the wave length of light l ¼ 633 nm. Compute the applied voltage necessary to have the maximum amplitude of the diffracted lightwave.

562

Optomechatronics structural beam V

subs

trate

w h h0

L

FIGURE P7.9 Geometry of a structural beam.

P7.10. Consider the intensity control system for a GLV shown in Figure 7.67. Describe the sequence of operation of the whole GLV system in detail including the optical system, electrostatic actuator, and sensor when this control concept is applied to the system. P7.11. Consider that a person is moving in a security area which contains only a square box in the left-hand corner shown in Figure 7.68a. The box is assumed to be stationary. In this situation, (1) obtain the edge images of the scene area at t ¼ t1 , and t ¼ t2 : þ (2) plot the G2 u ðtn Þ and Gu ðtn Þ and discuss the result. Solving this problem is to prove the results presented in Figure 7.68b. P7.12. We wish to track object B which is moving within a scene from time tn to tnþ1 as shown in Figure P7.12. In the scene object A is stationary while object B is moving toward A. If there is no noise, we can obtain the þ object images at time t ¼ tn , G2 u ðtn Þ and Gu ðtn Þ, utilizing edge detection results obtained at each time tn and tnþ1 : Draw the edge intensities, Iðtn Þ and þ Iðtnþ1 Þ and then obtain G2 u ðtn Þ and Gu ðtn Þ:

B

B

A

A

scene at t =tn

FIGURE P7.12 The scenes captured at two consecutive times.

scene at t = tn+1

Optomechatronic Systems in Practice

563

P7.13. From these G2 ðtn Þ and Gþ ðtn Þ obtained in Problem 7.12, (1) compute the center location and second moments of inertia of object B at t ¼ tn and t ¼ tnþ1 : (2) formulate the eigenvalue problem to obtain the object pose, and then determine the eigenvalues and eigenvectors of the problem. What is the angle of the motion of the object B from t ¼ tn to t ¼ tnþ1 ?

References [1] Akkermans, T.H. and Stan, S.G. Digital servo IC for optical disc drives, Control Engineering Practice, 9:11, 1245– 1253, 2001. [2] Ando, T. Laser beam scanner for uniform halftones, Printing Technologies for Image, Gray Scale, and Color, SPIE, 1458, 128– 132, 1991. [3] Bining, G. and Quate, C.F. Atomic force micro-scope, Physical Review Letter, 56, 930– 933, 1986, March. [4] Bouwhuis, W., Breat, J., Huijser, A. and Pasman, J. Principles Optical Discovery Systems. Adam Hilger Ltd, Bristol, UK, 1986. [5] Conn, P.M. Confocal Microscopy. Academic Press, London, 1999, (Methods in Enzymology, 307). [6] Corle, T.R. and Kino, G.S. Confocal Scanning Optical Microscopy and Related Imaging Systems. Academic Press, London, 1996. [7] Fan, K.C., Chu, C.L., and Mou, J.I. Development of a low-cost autofocusing probe for proﬁle measurement, Institute of Physics Publishing, Measurement Science Technology, 12, 2137– 2146, 2001. [8] Hsu, S.H. and Fu, L.C. Robust output high-gain feedback controllers for the atomic force microscope under high data sampling rate, International Conference on Control Applications, Proceeding of the 1999 IEEE, August, 1999. [9] Kim, S.G., Optically sensed in-plane AFM tip with on-board actuator, Lecture number 6.777, Design and Fabrication of Microelectromechanical Device, Final Report of Design Project, MIT, 2002. [10] Lee, J.B. Introduction to MEMS, Lecture Note, EE7382. The University of Texas, Dallas, TX, 2004. [11] Liao, K.M., Wang, Y.C., Yeh, C.H., and Chen, R. Closed-loop adaptive control for torsional micromirrors, Proceedings of SPIE, 5346, 184– 192, 2004. [12] Maruyama, T., Hisada, T. and Ariki, Y. A laser scanning optical system for high-resolution laser printer, Optical Hard Copy and Printing Systems, SPIE, 1254, 54 – 65, 1990. [13] Min, G. Principle of Three Dimensional Imaging in Confocal Microscope. World Scientiﬁc, Singapore, 1996. [14] Nano Focus AG Company Presentation, 2005. Nano Focus AG Co. Ltd., www. nanofocus-ag.com. [15] O’Shea, D.C. Elements of Modern Optical Design, Wiley, New York, 1985. [16] Pawley, J.B. Handbook of Biological Confocal Microscopy. Plenum Press, New York, 1995, (Revised Edition).

564 [17] [18] [19] [20]

[21] [22] [23] [24] [25] [26] [27]

Optomechatronics Razzaghi, M. High resolution high speed laser recording and printing using low speed polygons, Printing Technologies for Image, Gray Scale, and Color, SPIE, 1458, 145– 153, 1991. Rembe, C., Muller, L., Muller, R.C. and Home, R.T. Full three-dimensional motion characterization of a gimballed electrostatic microactuator, IEEE 39th Annual International Reliability Physics Symposium, Orlando, Florida, 2001. Rutzel, S., Lee, S.I., and Raman, A. Nonlinear dynamics of atomic-forcemicroscope probes driven in Lennard-Jones potentials, Proceedings, Mathematical, physical and engineering sciences, 459:2036, 1925 –1948, 2003. Seki, K. and Ishii, K. Robust design for dynamic performance: optical pick-up example, Proceedings of The 1997 ASME Design Engineering Technical Conferences and Design Automation Conference, 97-DETC/DAC3978, pp. 1 – 10, California, USA, 1997. Senturia, S.D. Microsystem Design. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2001. Sharp, J.H. The Laser Printer (LASERS & ELECTRO-OPTIC SYSTEMS 4), Lecture Note, University of Glasgow, UK, 2004. Shim, J.Y. Three-axis nanopositioning slider design, analysis and control, Ph.D. Thesis, Department of Mechanical Engineering, KAIST, Korea, 2004. Shiraishi, M. Optical-Based In-Process Monitoring and Control, Part IV: OptoMechatronic Systems Control, Opto-Mechatronic Systems Handbook. CRC Press, Boca Raton, FL, 2002. Sitti, M. Survey of nanomanipulation systems, IEEE-Nanotechnology Conference 2001, 75 – 80, 2001. Van Kessel, P.F., Hombeck, L.J., Meier, R.E. and Douglass, M.R., A MEMSbased projection display, Proceeding of the IEEE, 86:8, August, 1998. Wilson, T. and Sheppard, C. Theory and Practice of Scanning Optical Microscopy. Academic Press, London, 1984.

Appendix A1 Some Considerations of Kinematics and Homogeneous Transformation Some Considerations of Kinematics We will consider a rigid body motion in two- and three-dimensional spaces as depicted in Figure A1.1. Let us ﬁrst consider the planar motion of a rigid body shown in Figure A1.1. The motion is composed of a translation and a rotation motion. Suppose that frame xB -yB is a rotating coordinate system attached to the rigid body. If rC is denoted by the position vector of a ﬁxed P with respect to OC ; and rO is the position vector of the center OB of the rigid body with respect to OC ; then the combined motion of point P is described by rC ¼ rO þ rB

ðA1:1Þ

where rB is the position vector of P relative to OB : The velocity of P is then obtained by differentiating both sides of Equation A1.1, giving the equation drC drO ðtÞ dr ðtÞ ¼ þ B dt dt dt

ðA1:2Þ

The last term of the right-hand side of the above equation is due to the translation of P within the rigid body with respect to OB : When P is ﬁxed in the body , which is translating through space, the above velocity equation can be rewritten as drC drB ¼ dt dt

ðA1:3Þ

since rB is constant. Next, let us consider rotation of the rigid body in a two-dimensional case as shown in Figure A1.2. Assume there is no translation and only pure rotation. In this case, we assume that the origins OC and OB coincide. When the coordinates frame ðxB -yB Þ rotates about the origin in a counterclockwise direction, we wish to express the xB -yB coordinates in terms 565

566

Optomechatronics yB yC P

xB

rB ω

OB

rC rO

OC

xC

FIGURE A1.1 The planar motion of a rigid body.

of the xC -yC coordinates. From the geometry shown in the ﬁgure, the unit vectors of the rotating frame rotated by u with reference to xC -yC is expressed by iC ¼ cos u iB 2 sin u jB

jC ¼ sin u iB þ cos u jB

ðA1:4Þ

yC P

yB

xB

rC

rB

θ

OC = O B zC FIGURE A1.2 Rotation of a coordinate system ðxB -yB Þ in a plane.

xC

Some Considerations of Kinematics and Homogeneous Transformation

567

where iC and jC are the unit vectors of the xC -yC coordinates, and iB and jB are those of the xB -yB coordinates. Utilizing the above relationship, we obtain the following expression between the two coordinates xC -yC and xB -yB such that #" # " # " xB xC cos u 2sin u ðA1:5Þ ¼ yB yC sin u cos u The matrix shown in the above is called a rotation matrix. The rotation velocity of the rigid body can be obtained by differentiating Equation A1.5 and setting dxB/dt ¼ dyB/dt ¼ 0, since P has no relative velocity with respect to the xB -yB coordinates. Carrying out the differentiation leads to the following relation 3 2 dxC " #" # 6 dt 7 xB du sin u cos u 7 6 ðA1:6Þ 7¼2 6 4 dyC 5 dt 2cos u sin u yB dt Rewriting the above equation as a vector expression, we have drC du du ¼ 3 xB iB þ 3 yB j B dt dt dt and then ﬁnally drC ¼ v 3 rB dt where v ¼ du=dtðkB Þ ¼ du=dtðkC Þ is used, and kB and kC are the unit vectors normal to the plane x-y. Combining the translation and rotation of a rigid body motion in the two-dimensional case, we can express the resulting position vector drC drO ðtÞ ¼ þ v 3 rB dt dt

ðA1:7Þ

where v is the angular velocity of the xB -yB : For the three-dimensional case, shown in Figure A1.3, we can carry out a similar procedure to obtain the rotation matrix and a position vector. Referring to Figure A1.3, let {C} be a frame ﬁxed at OC ; {B} be a frame ﬁxed at OB translating with a rigid body without rotating, and {B0 } be a frame ﬁxed at OB0 rotating with the rigid body. To obtain the rotational velocity, we assume that the coordinate frames {B} and {B0 } coincide initially. When frame {B0 } rotates from frame {B} by u, its unit vectors are expressed by iB ¼ cos u iB0 2 sin u jB0 þ 0 kB0 kB ¼ 0 u iB0 2 0 u jB0 þ 1 kB0

jB ¼ sin u iB0 þ cos u jB0 þ 0 kB0

ðA1:8Þ

568

Optomechatronics yB´

zB zB´

yB

P

zC

rB

q

OB = OB´ rC

q rO xB {B}

xB´ {B´}

yC

OC {C}

xC FIGURE A1.3 A rigid motion in three-dimensional space.

Therefore, we obtain the following relationship between two coordinates; 2 3 2 32 3 cos u 2sin u 0 xB0 xB 6 7 6 76 7 6 yB 7 ¼ 6 sin u cos u 0 76 yB0 7 ðA1:9Þ 4 5 4 54 5 zB

0

0

1

zB0

Carrying out differentiation of the above equation, we can see that the resulting rotational velocity becomes drB du ¼ k 0 3 ðxB0 iB0 þ yB0 jB0 þ zB0 kB0 Þ ¼ v 3 rB dt dt B

ðA1:10Þ

Combining this with the translation of the rigid body we have the following position vector of point P, drC dr dr ¼ O þ B dt dt dt

ðA1:11Þ

where drO =dt is the translation of the rigid body in three-dimensional space, drB /dt, in Equation A1.10. The rotational velocity yields the same equation as given in the two-dimensional case, since we had made the same assumption to derive the rotational speed.

Some Considerations of Kinematics and Homogeneous Transformation

569

Homogeneous Transformation As discussed previously, the rigid body motion in general consists of translation and rotation motions. To represent these in a single matrix form, we normally use homogeneous transformation, which is particularly useful in the manipulation of image matrices or coordinate transformation for computer vision. Before we elucidate the concept of the transformation, let us consider a pure rotational motion ﬁrst. In this case, the rotation matrix describing the orientation of a frame {xB -yB -zB : ½B } can be determined in a systematic way as shown in Figure A1.4. To start with, the frame is assumed to be initially coincident with a reference frame, {xC -yC -zC : ½C }: If CB Rðg; b; aÞ denotes a resultant rotation matrix due to rotations RðxC ; gÞ about xC axis, RðyC ; bÞ about yC axis, and RðzC ; aÞ about zC axis, they are called, roll, pitch, yaw, respectively. The deﬁnitions of these rotations are depicted in the ﬁgure. If the order of the rotations is roll, pitch, and then yaw, the resulting transformation is given by C B Rðg; b; aÞ

¼ Rðz; aÞ Rðy; bÞ Rðx; gÞ 32 2 cos b cos a 2sin a 0 76 6 76 ¼6 4 sin a cos a 0 54 0 0

0

1 0 6 6 0 cos g 4 0

zB

sin g

2sin b

1

2

0

sin b

1

0

0

cos b

3 7 7 5

3

0

7 2sin g 7 5

ðA1:12Þ

cos g zC

zB

zC

zC = zB yB

yB β

g

g OC

OC β xC

xC

(a)

yC

x –axis : roll

(b)

yC = yB

α xC

xB y-axis : pitch

FIGURE A1.4 Rotation about ﬁxed axes in three-dimensional space.

(c)

α OC

xB z –axis : yaw

yC

570

Optomechatronics

Carrying out multiplication of the above equation, we have C B Rðg; b; aÞ

2

cos a cos b cos a sin b sin g 2 sin a cos g cos a sin b cos g þ sin a sin g

3

7 6 7 6 ¼ 6 sin a cos b sin a sin b sin g þ cos a cos g sin a sin b cos g 2 cos a sin g 7 5 4 2sin b

cos b sin g

cos b cos g ðA1:13Þ

Let us take some examples of this rotation transformation. Consider the rotation of a point deﬁned in the coordinate frame {C} by an angle 908 about zC and then 908 about yC . Then 2

0

6 Rðy; 908Þ Rðz; 908Þ ¼ 6 4 0

0 1

32

0 21 0

3

2

0 0 1

3

76 6 1 07 54 1

0

7 7 6 7 6 07 5 ¼ 41 0 05

0

0

1

21 0 0

0 1 0

Suppose that the point P is given by ½4i þ 2j 2 3k : The above two rotations of this vector will result in 2

23

3

2

0 0 1

32

4

3

6 7 6 76 7 6 4 7 ¼ 6 1 0 0 76 2 7 4 5 4 54 5 2

0 1 0

23

When we reverse the order of the rotation, we can notice that Rðz; 908Þ Rðy; 908Þ – Rðy; 908Þ Rðz; 908Þ Now, let us consider general case of a coordinate transformation involving rotation and translation. The transformation equation can be rewritten in a 4 £ 4 matrix form as 2

3

2

6 7 6 6 r 7 6 CR 6 C7 6 B 6 7 6 6 7 6 6 7¼6 6 7 6 6 7 6 6 · · · 7 6 · · ·· · · 4 5 4 0 1

32 .. . .. . .. .

3

76 7 76 r 7 76 B 7 76 7 76 7 76 7 76 7 76 7 6 7 · · ·· · · 7 54 · · · 5 1 1 C B Tr

ðA1:14Þ

where CB R is the 3 £ 3 rotation matrix of frame {B} with respect to {C}, and CB T is the 3 £ 1 translation vector of {B} with respect to {C}. This 4 £ 4 matrix is called homogeneous transformation matrix H. The above equation can then

Some Considerations of Kinematics and Homogeneous Transformation

571

be expressed with H as 2 3 2 3 6 rB 7 6 rC 7 6 7 6 7 6 7 6 7 6 7 ¼ H6 7 6 7 6 7 4···5 4···5 1 1

ðA1:15Þ

The transformation H, corresponding to a translation by a vector 2i þ 10j þ 4k; is expressed by 3 2 1 0 0 2 7 6 6 0 1 0 10 7 7 6 7 H¼6 7 6 60 0 1 4 7 5 4 0 0 0 1 If a vector 4i 2 3j þ 6k is 2 3 2 .. 667 61 0 0 . 6 7 6 6 7 7 6 0 1 0 ... 6 7 6 6 7¼6 6 10 7 6 0 0 1 ... 6 7 6 4···5 4··· ··· ··· ··· . 0 0 0 .. 1

translated by the above H, then 3 32 4 2 76 7 7 76 6 7 7 23 10 76 7 76 6 7 6 7 7 4 76 7 · · · 54 · · · 5 1 1

The transformation H, corresponding to a rotation about x-axis by an angle g, is given by 3 2 1 0 0 0 7 6 6 0 cos g 2sin g 0 7 7 6 7 ðA1:16Þ Hðx; gÞ ¼ 6 7 6 6 0 sin g cos g 0 7 5 4 0 0 0 1 In a similar way, Hðy; bÞ and Hðz; aÞ can be expressed using Equation A1.14. Let us suppose that a vector 4i þ 12j 2 3k is rotated by an angle 908 about the x-axis. In this case, the transform is obtained from Equation A1.16 as follows 2 3 2 32 3 4 1 0 0 0 4 6 7 6 76 7 6 3 7 6 0 0 21 0 76 12 7 6 7 6 76 7 6 7¼6 76 7 6 7 6 76 7 6 12 7 6 0 1 0 0 76 23 7 4 5 4 54 5 1 0 0 0 1 1 We now combine a series of rotations R(z,908), R(y,908) with a translation 2i þ 5j 2 4k: The combined homogeneous transformation matrix

572

Optomechatronics

is expressed by

2

1 0 0

2

32

0 0 1 0

3

2

0 0 1

2

3

6 76 7 6 7 6 0 1 0 5 76 1 0 0 0 7 6 1 0 0 5 7 6 76 7 6 7 76 7¼6 7 Tð2; 5; 24ÞRðy; 908ÞRðz; 908Þ ¼ 6 6 76 7 6 7 6 0 0 1 24 76 0 1 0 0 7 6 0 1 0 24 7 4 54 5 4 5 0 0 0 1 0 0 0 1 0 0 0 1 With the vector above becomes 3 2 2 0 0 0 7 6 6 6 0 7 61 0 7 6 6 7¼6 6 7 6 6 6 21 7 6 0 1 5 4 4 1

2 5i þ 3j 2 2k, the transformation matrix expression in the 1

2

32

25

3

7 76 7 6 5 7 76 3 7 7 76 7 76 6 22 7 0 24 7 5 54 1 0 0 0 1 0

Stretching spreads the points uniformly by a factor sx along the x axis, along the y axis by a factor sy, and along the z axis by a factor sz : The matrix for this transformation is 3 2 sx 0 0 0 7 6 6 0 s 0 07 7 6 y 7 ST ¼ 6 7 6 6 0 0 sz 0 7 5 4 0 0 0 1 Scaling is stretching points out along the coordinate directions by the same factor s. The scaling matrix is expressed by 3 2 s 0 0 0 7 6 60 s 0 07 7 6 7 S¼6 6 7 60 0 s 07 5 4 0 0 0 1 When a vector point given by ai þ bj þ ck is scaled by s, the matrix is obtained by 2 3 2 32 3 sa s 0 0 0 a 6 7 6 76 7 6 sb 7 6 0 s 0 0 76 b 7 6 7 6 76 7 6 7¼6 76 7 6 7 6 76 7 6 sc 7 6 0 0 s 0 76 c 7 4 5 4 54 5 1 0 0 0 1 1

Appendix A2 Structural Beam Deﬂection When a structural beam is subjected to lateral loads acting transversely to the longitudinal axis, the loads cause the beam to bend. When there is no applied load to the beam, it will not be deﬂected, its longitudinal, neutral axis being in line with the x-axis, as shown in Figure A2.1. Now, suppose that an initially undeformed beam segment A – B is bent downward due to the applied bending moment M, as shown in the ﬁgure. Let us derive vðxÞ, the transverse displacement along the x direction due to this moment. If r is deﬁned as the radius of curvature, then from the geometry of the bent beam segment A – B, we have

rdu ¼ ds

or

1 du ¼ r ds

ðA2:1Þ

where du is the small angle between the normals, AO and BO, and ds is the distance along the curve between the normals. The reciprocal of r in the above equation is called curvature which is deﬁned by 1=r: For small deﬂection, i.e., small u, ds is approximately given by ds . dx

ðA2:2Þ

The transverse displacement vðxÞ is described by tan u . u ¼

dv dx

ðA2:3Þ

Combining Equation A2.2 and Equation A2.3, we obtain du d2 v ¼ dx dx2

ðA2:4Þ

From Hook’s law, the curvature 1=r is related to the bending moment M by 1 du M ¼ ¼2 EI r dx

ðA2:5Þ

if the beam deﬂection remains within the elastic limit. In the above equation E is Young’s modulus of elasticity, I is the moment of inertia of the beam, and 573

574

Optomechatronics x

dx v

M A

x

q

v + dv

ds

q + dq B

M

r dq

O FIGURE A2.1 Geometry of a deﬂected beam segment subjected to a bending moment.

EI is called ﬂexural rigidity. Minus sign in the above is due to the sign convention; “ þ ” sign assigned to bending moment acting on the beam upward, while “ 2 ” sign to the downward one. Substituting Equation A2.5 into Equation A2.4, we ﬁnally have d2 v M ¼2 EI dx2

ðA2:6Þ

This is the fundamental equation governing the beam deﬂection. Once the bending moment M and beam geometry are given, the transverse displacement of the beam along the x direction can be obtained from this equation. Now, let us consider a cantilever beam, whose supports are clamped at one end and the other end is free as shown in Figure A2.2, is subjected to an external load. The beam is subjected to a uniform load of strength p. In this case, the bending moment becomes M¼2

pð‘ 2 xÞ2 2

Then, substituting this equation into Equation A2.6, we obtain EI

d2 v pð‘ 2 xÞ2 ¼ 2 2 dx2

ðA2:7Þ

Structural Beam Deﬂection

575 P

d q

FIGURE A2.2 A cantilever subjected to a uniform load of intensity p.

Integrating this equation once we get dv pð‘ 2 xÞ3 ¼2 þ C1 dx 6EI

ðA2:8Þ

The boundary condition dv=dx ¼ 0 at x ¼ 0 determines the constant C1 , and substitution of C1 into Equation A2.8 results in dv px 2 ¼ ðx 2 3‘x þ 3‘2 Þ dx 6EI Integration of the above equation yields v¼

px2 2 ðx 2 4‘x þ 6‘2 Þ þ C2 24EI

Again, using the boundary condition, vð0Þ ¼ 0; we obtain C2 ¼ 0: Finally, the equation describing the beam deﬂection is obtained by v¼

px2 2 ðx 2 4‘x þ 6‘2 Þ 24EI

ðA2:9Þ

Therefore, the deﬂection d at the free end can be easily found from the above equation and is obtained by plugging x ¼ ‘ in the above equation. vðxÞlx¼‘ ¼ d ¼

p ‘4 ; 8EI

uðxÞlx¼‘ ¼ u ¼

p ‘3 6EI

ðA2:10Þ

In a similar way, the deﬂection equation of a beam supported under various boundary conditions can be easily obtained. Table A2.1 summarizes the deﬂection ðdÞ and angle of rotation ðuÞ of a beam under various supporting conditions.

576

Optomechatronics

TABLE A2.1 Deﬂections and Slopes of Beams in Various Support Conditions Beam-Deﬂection Conditions A

Deﬂection Equation

Load Uniform

B d

v¼

px2 2 ðx 2 4‘x þ 6‘2 Þ 24EI

Deﬂection and Slopes

q

Triangular

v¼

po x2 ð2x3 þ 5‘x2 120‘EI 2 10‘2 þ 10‘3 Þ

p‘4 8EI

d ¼ vð‘Þ ¼ u¼

dv p‘3 ð‘Þ ¼ dx 6EI

d¼

p‘3 30EI

u¼

p‘3 24EI

d¼

p‘3 3EI

u¼

p‘2 2EI

d¼

p‘4 384EI

P

Concentrated

v¼

px2 ð3‘ 2 xÞ 6EI

P

Uniform a

v¼

px2 ðx 2 ‘Þ2 24EI

b

ua ¼ ub ¼ 0

P

Uniform a

v¼

px ðx3 2 2‘x2 þ ‘3 Þ 24EI

d¼

b

ua ¼ P

Concentrated a

2

2

b

v¼

px ð24x2 þ 3‘2 Þ 48EI 0#x#

‘

2

d¼

5p‘4 48EI p‘3 ¼ ub 24EI p‘3 48EI

ua ¼ ub ¼

p‘2 16EI

Appendix A3 Routh Stability Criterion

Stability is the most important characteristic that needs to be analyzed for control systems. This is because if a control system becomes unstable, analyzing its controlled performance becomes meaningless. There are several ways of checking system stability but we will conﬁne ourselves to the Routh stability criterion. Let us consider a closed-loop control system shown in Figure A3.1, whose transfer function is given by GðsÞ ¼

XðsÞ b sm þ b1 sm21 þ · · · þ bm21 s þ bm ¼ 0 n Xd ðsÞ a0 s þ a1 sn21 þ · · · þ an21 s þ an

ðA3:1Þ

where ai’s and bi’s are constant and n $ m: The Routh stability criterion enables us to ﬁnd the stability condition under which the system is stable, namely, when the system becomes unstable and how many closed-loop poles lie in the right-half s-plane. The advantage of using the criterion is that it does not require factoring of the characteristic polynomial of the denominator of GðsÞ in order to ﬁnd the closed-loop poles. If we write the characteristic polynomial equation

DðsÞ ¼ a0 sn þ a1 sn21 þ · · · þ an21 s þ an ¼ 0

ðA3:2Þ

the ﬁrst stability condition is that the above equation must have all a0i s . 0 for a0 – 0: This also means that all ai ’s must have the same sign for a0 – 0: If this condition is met, then we need to proceed ordering the coefﬁcients of the characteristic equation into an array called the Routh array composed of the following rows and columns sn

n21

s sn22 sn23 .. . s2 s1 s0

a0 a1 b1 c1 .. .

d1 e1 f1

a2 a3 b2 c2 .. .

a4 a5 b3 c3

a6 a7 b4 c4

··· ··· ··· ···

d2

577

578

Optomechatronics

X d (s)

+

−

G o (s)

X(s)

(a) A closed loop system

X d (s)

G (s)

X(s)

(b) The equivalent loop transfer function FIGURE A3.1 Transfer function of a closed loop system.

The coefﬁcients in the above arrays are given by a a 2 a0 a3 b1 ¼ 1 2 a1 a1 a4 2 a0 a5 b2 ¼ a1 a1 a6 2 a0 a7 b3 ¼ a1 .. . This process continues until we get a zero for the last coefﬁcient in the third row. Similarly, the coefﬁcients of the remaining rows the 4th, 5th, … and nth are determined in the following way: Here, only the 4th row is shown for illustration. c1 ¼

b1 a3 2 a1 b2 b1

c2 ¼

b1 a5 2 a1 b3 b1

c3 ¼

b1 a7 2 a1 b4 b1 .. .

This process continues until the nth row is completed. In the process of developing the Routh array the missing terms are replaced by zeros. The Routh stability criterion is stated as below: The necessary and sufﬁcient condition for a control system to be stable is that each term of the ﬁrst column of Routh array be positive if a0 . 0:

Routh Stability Criterion

579

If this condition is not met, some of the roots of the characteristic equation given in Equation A3.2 lie in the right-half of the s-plane, implying the system is unstable. The number of such roots is equal to the number of changes in sign of the coefﬁcients of the ﬁrst column of the array. Consider the forth order-system with the transfer function CðsÞ 1 ¼ RðsÞ sðs2 þ s þ 1Þðs þ 2Þ þ 1 The characteristic equation is given by s4 þ 3s3 þ 3s2 þ 2s þ 1 ¼ 0 Then, the Routh array is constructed below s4

1

3

1

3

3 7 3 5 7

2

0

s

s2 s1 s0

1

1

Since the terms in the ﬁrst column is all positive, the system is stable. Let us take another example for stability test. If the transfer function is given by CðsÞ 2 ¼ RðsÞ sðs2 þ s þ 1Þðs þ 2Þ þ 2 Since this system has the characteristic equation. s4 þ 3s3 þ 3s2 þ 2s þ 2 ¼ 0 the Routh array is given by, s4

1

3

2

3

3 7 3 24 7

2

0

s

s2 s1 s0

1

2

Examination of the ﬁrst column of the Routh array show that there are two changes in sign. Therefore, the system under consideration is unstable having two poles in the right half of the s-plane.

Index

A aberrations, 55– 61, 458 AC see alternating current accumulators, 205 acoustic disturbance measurement, 322 acousto-optical deflectors (AOD), 451– 2, 457, 515–18 acousto-optical modulators, 277, 339– 48 acousto-optic interaction, 342 acousto-optic materials, 340– 1 acousto-optic scanners, 515 active vision systems, 109 actuation, 13, 14, 22, 24 actuators, 210– 20 see also individual types A/D see analog-to-digital address buses, 206 address electrodes, 527 advanced storage magneto optical disks (ASMO), 470 AFM see atomic force microscopes Airy disks, 85–9, 467, 469, 473, 474 Airy function, 72 aliasing frequency, 209 –10 alternating current (AC) motors, 210 ALU see arithmetic and logic units aluminum, 369 AM see amplitude modulation Ampere’s law, 217, 333 amplitude grating, 90– 1 amplitude modulation (AM), 340, 490– 1 amplitude ratio, 361 analog-to-digital (A/D) conversion, 199 – 200 angles

angular resolution, 459 arbitrary incident, 90 Bragg deflection, 343 – 4 critical, 36, 92 – 3 of illumination, 403 –5 incident, 87, 88, 90, 404 optical encoders, 277 tolerable error, 460 – 1 anti-reflection coating, 340 AOD see acousto-optical deflectors apertures aperture stops, 53 – 5, 60 – 1 circular, 84 – 9 laser printers, 465, 467 –9 slits, 67 – 8, 74 –83 arbitrary incident angles, 90 architectures digital micromirrors, 526 – 32 information feedback control, 414 – 28 area driving force, 219 arithmetic and logic units (ALU), 204 –5 armatures, 356 – 7 ASMO see advanced storage magneto optical disks astigmatism, 55, 57 – 9, 338 – 9, 475, 477 atomic attractive forces, 20 atomic force microscopes (AFM), 10, 11, 293, 484 – 510 cantilever deflection, 493 – 501 cantilever tips, 497 –501 component details, 502 measurement principle, 484 – 96 optical measurements, 496 – 501 scanning motion control, 501 – 10 atomic repulsive forces, 20 austenite state, 221 – 2

581

Index

582 autofocusing systems, 326– 39 configuration, 327– 8 illumination control, 406– 7, 410– 11 objective lens feedback control, 330– 9 optical discs, 229– 30 optical resolutions, 328– 30 visual, 386– 99 automatic optical focusing, 326– 39 avalanche photodiodes, 192 axial configurations, 212, 214 axial resolution, 328– 30, 520– 1

bimorph piezo elements, 304, 325 – 6 binary images, 117 black density control systems, 455 blurring and correction, 386 – 99 Bragg deflection angle, 343 – 4 Bragg effect, 342 breakdown voltage, 187 bridge circuits, 202 – 3 brightness, 109, 116 bulk-type piezoresistive sensors, 184 buses, microcomputers, 205 – 6

B

C

background thresholding, 127– 31 backlighting, 401– 2, 408 barium titanate, 181, 182 barrel distortion, 59– 60, 351 basic control actions, 241– 4 basic functional modules, 263– 73 beams cantilever, 574– 5 collimating, 477 convergent, 40, 477 deflection, 342, 343, 494, 538 –40, 573 – 6 diffraction, 342, 343 divergent, 40 Euler, 538 expanding, 40, 41 lasers, 95 – 7, 463– 6 manipulation, 264–6 microbeam structures, 538– 40 beam-type piezoresistive sensors, 184 polygon scanners, 364– 5 rotating-mirrors, 516 scanning, 459– 60, 513– 18 splitters, 331, 518 spot position, 460, 462 spreading, 74, 75, 96– 7, 473 structural beam deflection, 494, 573– 6 tracking, 477– 8 Bessel functions, 85, 469 bias voltage, 527 biconcave lenses, 47 biconvex lenses, 47 bimetal actuators, 369, 370 bimetallic strips, 322 bimorph actuators, 213, 214, 369, 370 bimorph configurations, 213, 214

calibration cameras, 152 – 65 marks, 162 rigs setup, 162 cam driven mechanisms, 383 – 5 cameras calibration, 152 – 65 components/operation, 9 – 11 coordinates, 155 – 62 evolution, 8 Hough transform example, 151, 153 motion-based visual feedback control, 412, 413 perspective model, 152, 154 rig setup, 162 transformation geometry, 160 cantilever beams, 574 –5 cantilever deflection, 493 – 501 cantilever probes, 484 – 510 cantilever tip deflection, 497 – 501 capacitance/capacitors, 176 – 7, 274, 275, 409 – 10 capacitive actuators, 302 – 4 capacitive sensors, 176 –7, 357 –8 cars, visual tracking, 550 – 7 cascade feedback, 290 cathode-ray-tubes (CRT), 224, 225 causality, 288 CCDs see charged coupled devices CD-ROM storage device, 470 CD-RW storage device, 470 central processing units (CPU), 204 – 7 ceramic materials, 179, 180 characteristics of optomechatronic technology, 16 – 20 charge amplifiers, 181, 182

Index charged coupled devices (CCDs), 4, 112– 14 charge sensitivity coefficients, 180, 182 charge transfer, 113– 14 chip inspection, 409– 11 chip mounters evolution, 6 – 8 circle of least confusion, 58 circular apertures, 84– 9 closed-loop systems, 237– 9, 289, 290 closed miniature cell photo-thermal actuators, 311 –12 CMM see coordinate measuring machines coaxial light, 408 coherence, 64– 5 collimating beams, 477 color, feature extraction, 108 coma aberrations, 55– 7 comb driving, 368, 371– 5 compact disk read-only memory (CD-ROM), 470 compact disk rewritable (CD-RW), 470 comparators, 196– 7 compensating scan errors, 367 composite distortion, 351 compound lens systems, 47, 378– 83 computer ray tracing, 52– 3 computing, optical technology, 14, 15 concave lenses, 36, 37 image characteristics, 43 plane waves, 40 types, 47 see also lenses conductor drums, 452– 4 confocal detection principle, 512– 13 confocal system, 326– 7, 332 confocus scanning microscopes, 326– 8, 485, 510–24 beam scanning, 513– 18 conventional microscopes comparison, 511 focus measure and control, 521– 4 measurement principles, 510–13 Nipkow disks, 518– 20 resolution, 520– 1 conjugate points, 37 constructive interference, 68 contact-type hot roller fuser, 454– 5 continuity law, 228– 9 contrast enhancement, 118– 21

583 control buses, 206 disk servo/optical storage disks, 478 – 84 dynamical systems, 241 – 4 laser printers, 451 – 5 signals, 206 systems, 451 – 5, 478 – 84 see also individual forms controllers optical information feedback design, 24 – 5, 260 –1, 411 – 28 types, 242 –4 visual information feedback design, 24 – 5, 260 –1, 411 – 28 conventional optical microscopes, 511 convergent beams, 40, 477 convex lenses, 36, 37 image characteristics, 43 plane waves, 40 types, 47 see also lenses conveyors, 278 convolution kernels, 390 convolution masks, 121 – 5 coordinate measuring machines (CMM), 6 –8 coordinate systems, 155 – 64 corrugations, 322 CPU see central processing units critical angle, 36, 92 – 3 crossbar switches, 368 CRT see cathode-ray-tubes Curie point, 179 – 80 curvature of field, 55, 57

D D/A see digital-to-analog damping, 215, 358, 359, 362 dark current, 190 dark field illumination, 402 – 3 data buses, 205 – 6 display, 14, 15 devices, 224 – 5 fundamental functions, 25 – 6 memory, 13 – 15 retrieval, 22, 25 storage, 13 – 15

584 discs evolution, 8 fundamental functions, 22, 25 systems, 470 switching, 22, 25 transmission, 14– 15, 22, 25, 27, 270 – 2 see also signal DC see direct current definition of optomechatronics, 8 – 20 deflection acousto-optical deflectors, 451– 2, 457, 515 – 18 acousto-optical modulators, 277, 339 – 48 beams, 342, 343, 494, 538– 40, 573– 6 Bragg, 343– 4 cantilever, 493– 510 microbeam structures, 538– 40 optical beam, 342, 343 structural beam, 494, 573–6 defocusing, 463–5, 511 deformable grating modulators, 536 – 7 deformable mirrors, 277, 278, 315 deformation modes, 212– 15 depth of field, 388 depth of focus, 389– 90 derivative control actions, 242– 4 design steps, 260– 1 destructive interference, 68 detection confocal detection principle, 512– 13 edge detection, 136– 48 focus error, 475, 477 Hough algorithm, 150 optical signal transmission, 431– 2 tracking error, 476– 8 dichroic crystals, 63 dielectric constants, 176– 7 dielectric cylinders, 92 differential amplifiers, 194, 196 differential transformers, 177– 9 diffraction, 74– 92 deformable grating modulators, 536 – 7 diffraction-limited images, 89 diffractive display, 524 Fraunhoffer, 74, 75, 467– 8 Fresnel, 74 gratings, 81– 3, 89– 92, 225, 293, 536– 7

Index Littrow grating angular displacement, 323 optical beam, 342, 343 slits, 74– 83 diffuse backlighting, 401 diffuse front lighting, 401, 402 diffuse light properties, 401 diffuser control, 403, 407 – 8 diffuse surfaces, 35, 110 diffusion, 401 digital cameras, 263 digital encoders, 316 – 17 digital micromirror devices (DMD), 25 – 6, 524 – 35 architecture, 526 – 32 light pulse width modulation, 532 – 5 performance, 534 projection system, 534 – 5 signal display, 225 digital-to-analog (D/A) conversion, 198 – 9 digital versatile disks (DVDs), 470 diodes laser technology, 456 – 7, 472 light-emitting, 400, 403, 406 – 9, 411 control, 272, 274 – 5 optical fiber transmission system, 428 – 9 photodiodes, 189 – 92, 274, 275 direct current (DC) motors, 210 – 12 direct current (DC)-tacho/encoder assemblies, 316 directional backlighting, 401 –2 directional lighting, 400, 401 – 2 directivity control, 400, 403 disk servo control system, 478 – 84 display devices, 224 – 5 distortion, 55, 59 – 61, 351, 432 divergent beams, 40 DMDs see digital micromirror devices dopants, 186 – 7 doping procedure, 186 Doppler effect, 345, 347 double-slit diffraction, 79 – 81 double-slit interference, 67 – 8 duralium, 504 – 5 DVDs, 470 dynamic-focus objective lens, 353 dynamic-focus scanning, 353 – 4

Index dynamic systems, 225– 45 basic control actions, 241– 4 dynamic response, 230– 3 modeling, 226– 30

585 Euler beam, 538 exit pupils, 53 – 4 external disturbance, 336 – 8 extrinsic semiconductors, 186 eye-in-hand systems, 413, 414, 420 – 1

E F edge detection, 136– 48 edge images, 542– 3 edge operators, 390 edge processing, 393 edge profile types, 137 eigenvalue analysis, 546– 8 electrical engineering unique variables, 257 – 8 electrical modulation, 269, 270 electrical signals representation, 258 electrical-to-electrical signal transformation, 263 electrical-to-mechanical actuation, 269, 270 electrical-to-mechanical integrations, 283 electrical-to-mechanical signal transformation, 263, 264 electrical-to-optical integrations, 282 electrical-to-optical signal transformation, 263, 264 electrical-to-optical signal transmission, 271 electrical transmission, 270, 271 electric fields, 32 electric motors, 210– 12, 275 electromagnetic motion generators, 276 electromagnetic radiation classification, 32 electromagnetic spectrum, 32 electromagnetic waves, 32– 3 electromotive current, 305– 6 electromotive force (emf), 178– 9 electronic revolution, 4 –8 electrons, 112 electrostatic actuators, 218–20, 371–5 electrostatically actuated mirrors, 371– 5 electrostatic force, 303, 528– 9 electrostrictive actuators, 221 emf see electromotive force entrance pupils, 53– 4 erasable and programmable read-only memory (EPROM), 205

Fabry-Pe´rot etalon, 318 – 21 Fabry-Pe´rot interferometer, 69 – 72, 323 –5 false frequency, 209 Faraday’s law, 178 – 9, 217, 333 feature extraction, 107 – 9, 544 – 8 feedback control atomic force microscopes, 499 – 501, 508 – 10 beam focus, 463 – 6 functionalities, 291 –2 fundamental functions, 24 – 5 galvanometers, 356 – 63 laser printers, 463 – 6 objective lenses, 330 – 9 open loop controls, 237 – 8 optical information, 24 – 5, 260 – 1, 411– 28 sensory systems, 23, 26 signal flow, 289, 290 visual optical information, 24 – 5, 260– 1, 411– 28 fiber-detector coupling, 431 fiber optics see optical fibers field curvature, 55, 57 field curvature correction, 458 – 9 field of view, 110 filter roll off, 198 filters Gaussian filters, 145 – 8 grid filters, 464 image filtering, 121 – 7 mean filters, 123 –4 median filters, 124 – 7, 390 – 1, 393 signal processing elements, 197 – 8 tunable wavelength, 339 – 40, 347 weighted median filter, 390 – 1, 393 fixed camera systems, 412 – 14, 417 – 20 fixed-focus objective lenses, 352 –3 fixed-focus scanning, 352 – 3 Fleming’s left hand rule, 210 – 11, 216, 332 flexible vision system, 406 – 7 flexural rigidity, 574

Index

586 fluid systems, 228– 9 FM see frequency modulation f-numbers, 54– 5, 328 focal length mirror rotation, 354, 355 thin lenses, 39– 40 two-lens system, 47– 50 zoom control, 379– 83 focal point formation, 40 focal ratio, 54 focus, 36 control, 521– 4 detecting sensors, 464– 5 error, 338 – 9, 475, 477 feedback control, 463– 9 length, 549–50 measurement, 338–9, 389– 99, 521– 4 shift, 458 –9 spot diameter, 462 see also autofocusing; confocus scanning focusing error detection, 338– 9, 475, 477 focusing systems, 471– 7 foot locking-release mechanisms, 312– 14 frame grabbers, 115– 16, 392– 3 Fraunhoffer diffraction, 74, 75, 467–8 free spectral range, 320– 1 frequency modulation (FM), 491 frequency response, 359– 62, 482, 483 frequency shifting, 23, 26, 339, 345– 7 Fresnel diffraction, 74 Fresnel lenses, 401 front lighting, 401, 402 full-duplex fiber optic transmission, 430 – 1 functionalities, integration-generated, 291 – 3 functional units acoustic-opto modulators, 339– 48 automatic optical focusing, 326– 39 basic, 263– 73 illumination control, 399– 411 optical information feedback control, 411 – 28 optical scanning, 348– 67 optical signal transmission, 428– 32 optical switches, 367–77 optomechatronic actuation, 301–16 optomechatronic sensing, 316– 26 visual autofocusing, 386– 99

visual information feedback control, 411 – 28 zoom control, 377 – 86 fusing process, 454

G galvanometer mirrors, 515 galvanometers, 349 – 63 feedback control, 356 – 63 simulation parameters, 362 – 3 galvanometric scanners, 349 – 63 gap closing actuators, 371 gap closing force, 218 – 19, 371 – 2 gas filled cavity photo-thermal actuators, 311, 312 Gaussian filters, 145 – 8 Gaussian laser beam optics, 95 –7 Gaussian lens formula, 40, 153 Gaussian optics, 38, 95 –7 generator law, 210, 211 geometric optics, 33 geometric shape, 108 germanium, 340, 341 GLV see grating light value grabbers, 115 – 16, 392 – 3 graded-index fibers, 94, 95 gradient edge detection, 137 – 42 magnitude, 137 – 8, 390, 398 –9 operators, 138 –42 grating angular displacement sensor, 323 grating diffraction, 81 – 3, 89 – 92, 225, 293, 536 – 7 grating light value (GLV), 225, 524, 535 – 40 grating sensors, 277 – 8, 293 gray level, 390 gray scale images, 117 – 18 grid filters, 464 grippers, 290, 309 – 11

H hardening process, 261 – 3 heat conductance, 227 heat treatment, 261 –3 helium-neon lasers, 95, 97 higher order systems, 236

Index hill-climbing method, 394–9, 522 histograms, 108, 118– 21 historical background, 4 – 8 homogeneous transformation, 569– 72 Hook’s law, 573 hopping distance, 394– 5 horizontal comb drive system, 372– 3 Hough algorithm/transform, 148– 53 Huygen’s principle, 343 hybrid scanners, 515

I illumination autofocusing, 406– 7, 410– 11 control, 21, 22, 399– 411 fundamental functions, 21, 22 illumination angle, 403– 5 methods, 400–3 optical technology functions, 13, 14 quality measure, 408– 10 requirements, 399– 400 types, 401– 3 visual autofocusing, 392 see also lighting image analysis, 106– 9 image-based visual feedback control, 414 – 17 blurring and correction, 386– 99 characteristics, 43 coordinates, 155– 62 display, 115 enhancement, 405– 6 filtering, 121– 7 formation, 109– 11 grabbers, 115– 16, 392– 3 gradient definition, 137 intensifiers, 268 intermediate processing, 107 interpretation stage, 109 Jacobian, 415– 16, 421, 423, 425– 8, 553 – 4, 556 point, 38 point location, 42 preprocessing, 106 processing, 115– 16, 541– 3 representation, 116–21 segmentation, 127– 36, 148– 52

587 imaging devices, 112 – 15, 485 principles, 485 types, 485 incident angles, 87, 88, 404 incident light distribution, 407 – 8 induced strains, 306 – 11 inductors, 274, 275 inertia, 19, 356, 357, 544 – 8 infrared light, 400 input beams, 364 – 5 input/output interfaces, 207 Int see interrupt signal integral control actions, 242 – 4 integratability, 260, 287 – 8 integrating amplifiers, 194 – 6 integration basic considerations, 256 – 63 integration-generated functionalities, 291 – 3 optomechatronic, 255 – 97 intensity modulation principles, 321, 322 interaction, unique variables, 258 interfaceability, 288 interfaces, 173 – 252, 273 – 90 interference, 65 – 74 interferometers, 317, 318, 485 encoders, 357 – 8 Fabry-Pe´rot interferometer, 69 –72, 323 – 5 Michelson, 72 – 4, 317 interferometric dilatometers, 317, 318 interferometric sensing, 323 interlaced image display, 115 interline transfer charged couple devices, 114 intermediate processing stage, 107 interrupt signal (Int), 206 – 7 intrinsic semiconductors, 186 inverse piezo effect, 179 inverting amplifiers, 193 – 4 inverting summing amplifiers, 194, 195 ionic polymer metal composite (IPCM) actuators, 223 – 4 irradiance patterns circular apertures, 85 –6 double-slit diffraction, 81, 82 interference, 65 – 7 single-slit diffraction, 76 – 8

588 isolators, 203– 4 iterative thresholding, 131– 3

J Jacobians, 415– 16, 421, 423, 425–8, 553 – 4, 556

K kinematic considerations, 565– 9 Kirchhoff’s current law, 194 Kirsch compass mask, 139

L Lambertian surfaces, 109– 10 laminated actuators, 502, 504 Laplace inverse transformation, 235– 6 Laplace transformation, 230, 233– 7, 333 Laplacian of Gaussian (LoG) operators, 138, 142, 145–8 Laplacian operators, 138, 139, 142– 8, 390, 393 laser beams, 95– 7, 463– 6 laser diodes, 456– 7, 472 laser Doppler velocimeters, 347 laser printers, 448– 69 acousto-optical deflectors, 451–2, 457 aperture control, 465, 467– 9 banding, 449 beam focus feedback, 463– 6 black density control system, 455 control system, 451– 5 densitometers, 455 developer rollers, 452, 454 drum assemblies, 449– 50 fuser roller, 450– 1 hot roll fusers, 454– 5 laser source, 456– 7 line scanners, 459– 63 optical elements, 452 optical resolution, 459– 63 optical system, 458– 9 performance, 455–7 photoconductive drums, 452– 4 polygon mirrors, 459–63 printing process, 448– 51 resolution, 459– 63 system configuration, 451– 5, 458– 9

Index toner charging development systems, 452, 454 toner hopper, 452, 454 units, 451 – 5 laser sources, 95, 456 – 7 laser surface hardening, 27, 261 – 3 laser welding, 278 – 9 lateral resolution, 328 – 9 layered actuators, 502, 504 LCD see liquid crystal device lead zirconate titanate (PLZT) elements, 179 – 81, 304 – 6 induced strains, 306 –11 stimulation parameters, 309 least significant bit, 534 LED see light-emitting diodes lens-controlled optical switching, 368, 369, 375 – 7 lenses camera calibration, 152 – 4 classification, 46 – 7 compound, 47, 378 – 83 concave image characteristics, 43 plane waves, 40 types, 47 convex image characteristics, 43 plane wave, 40 types, 47 Fresnel lenses, 401 Gaussian lens formula, 40, 153 lens maker’s formula, 39 lens translators, 353 objective lenses, 330 – 9, 352 – 3 optic fundamentals, 36 – 53 telephoto lens, 381, 382 thin-lens equations, 38– 40 toroidal lenses, 350, 351 zoom, 18, 378, 379, 381 – 3, 549 – 57 lenslet arrays, 315 – 16 L’Hospital’s rule, 82 light detectors, 186 –9 intensity, 533 pulse width modulation, 532 –5 scattering, 473 sources, 401 spreading phenomenon, 74, 75, 96 – 7, 473

Index light-emitting diodes (LED), 272, 274– 5 illumination control, 400, 403, 406– 9, 411 optical fiber transmission system, 428 – 9 lighting, 400– 2 see also illumination linear motion actuators, 331 linear motion variable inductors, 274, 275 linear variable differential transformers (LVDT), 178 line detection, 150 line scanners, 459– 63 liquid crystal device (LCD), 224, 225 lithium niobate, 340, 343 lithography tools, 6 Littrow diffraction grating angular displacement sensor, 323 Lloyd’s single mirror, 65 LoG see Laplacian of Gaussian logic units, 204– 5 longitudinal spherical aberrations, 56 Lorentz force, 211, 216 lumped models, 226 LVDT see linear variable differential transformers

M machines evolution, 2 machine vision, 105– 71 magnetic fields, 32 magnetostrictive actuators, 221– 3, 275, 276 magnification, 42– 6, 59, 378– 83 Malus’s Law, 64 manipulating signals, 264– 6 manipulation type optomechatronic integration, 279–81, 286 martensite state, 221– 2 materials distortion/optical signal transmission, 432 piezoelectric, 179– 85, 213, 214, 304– 11 processing, 23, 27 property variations, 14, 15 mathematical models, 305– 6 Matlab M-files, 125 maximum overshoot, 240, 241 MDs see minidisks

589 mean filters, 123 – 4 measurement principles, confocus scanning microscopes, 510 – 13 mechanical engineering unique variables, 257 – 8 mechanical modulation, 269 mechanical scanners, 277 mechanical signals representation, 257 –8 mechanical-to-electrical integrations, 282 mechanical-to-electrical signal transformation, 263, 264 mechanical-to-mechanical signal transformation, 263 mechanical-to-optical signal transformation, 263, 264 mechatronic actuators, 301 mechatronic configuration, 2 – 3 mechatronic elements, 15 –16, 173 –252 mechatronics historical background, 4 –6 median filters, 124 – 7, 390 –1, 393 MEMs see micro-electro-mechanical systems meniscus lenses, 47 meridional focus, 58 – 9 Michelson interferometer, 72 –4, 317 microactuators, 220 – 4 microband force sensors, 322 microbeam structures, 538 – 40 microcomputer systems, 204 – 10 microcontrollers, 207 micro-electro-mechanical systems (MEMs), 4– 6, 12, 19, 524 –40 microfabrication, 184 –5 microfactory inspection, 10, 12 micromirror devices, 25 –6, 225, 524 – 35 micro-optomechatronic switches, 368 microprocessors, 207 historical background, 4 industrial evolution stimulant, 3 optical grippers, 310 – 11 zoom control, 385 – 6 microscopes, 326, 484 – 510, 515, 518 – 20 minidisks (MDs), 470 mini-robots, 11 – 12 mirrors digital micromirror devices, 25 – 6, 225, 524 – 35 optical switches, 368, 369 rotating, 515 – 17

Index

590 rotation/focal length variation, 354, 355 spinning polygon, 349, 363–7 thermally actuated, 369– 71 missing orders of interference, 80– 1 modal distortion, 432 modulation acousto-optical modulators, 277, 339 – 48 amplitude, 340, 490– 1 efficiency, 347–8 electrical, 269, 270 frequency, 491 intensity principles, 321, 322 light pulse width, 532– 5 mechanical, 269 optical, 265– 6, 270 shutter, 322 signal, 259, 265– 6, 269, 270 wavelength principles, 321, 323 moments of inertia, 544– 8 monolithic circuits, 4 monolithic deformable mirrors, 315 most significant bit (MSB), 198– 200, 534 motion control, 22, 25, 392, 407 rigid bodies, 565 motor law, 210– 11 moving coil drivers, 356– 7 moving iron drivers, 357 moving magnet drivers, 357 MSB see most significant bit multi-mirror servo system, 25 multiple acoustic-opto deflectors, 344 multiple beam simultaneous scanning, 514 multiple lenses, 46– 53 multiple-slit diffraction, 81– 3 multiplexers, 201– 2 mylar glass, 401

N NA see numerical aperture negative meniscus lenses, 47 Nipkow disks, 518– 20 nonblocking optical fiber matrix switches, 375– 7 noninverting amplifiers, 194, 195

n-type semiconductors, 186 –7 numerical aperture (NA), 54 – 5 nxn optical switching systems, 10

O objective lenses, 330 – 9, 352 – 3 objective scanning, 348 – 50, 352 – 3, 461, 514 object orientation, 545 oblique lighting, 403 ODD see optical disc drives OM see optical microscopes online video tracking, 540 – 57 on-off gate switches, 368 opalescent glass, 401 op-amps, 193 – 7 open-loop configurations, 288 – 9 open-loop controls, 237 – 8 operational amplifiers (op-amp), 193 – 7 optical actuators, 275, 276, 301, 304 – 11 optical angle encoders, 277 optical beam deflection, 342, 343 optical beam diffraction, 342, 343 optical choppers, 266, 277 optical cross connector switches, 368 optical data transmission, 27, 270 – 2 optical device resolution, 459 – 63 optical disc drives (ODD), 10, 11 optical discs, 10, 11, 229 – 30 optical display units, 272 –3 optical encoders, 316 – 17, 357 – 8 optical engineering unique variables, 257 – 8 optical feedback controls, 260 – 1 optical fibers displacement, 322 fiber-detector coupling, 431 graded-index fibers, 94, 95 microfactory inspection, 10, 12 piezoelectric sensors, 325 – 6 ring lighting, 402 sensors, 11, 276, 321 – 6 step-index fibers, 94 –5 switching, 369, 370, 375 – 7 transmission, 92 – 5, 428 – 32 vision sensors, 326 optical information feedback control, 24– 5, 260– 1, 411 – 28

Index optical interferometric encoders, 357– 8 optical laser printers, 459– 63 optically actuated SMA walking machine, 312– 15 optical measurement, 496– 501 optical microscopes (OM), 485 optical modulation, 265– 6, 270 optical packaging, 278, 279 optical pattern recognition, 23, 26– 7 optical pick-up devices, 6 optical piezoelectric actuators, 304– 11 optical power, 47 optical property variation, 26 optical resolution, 328– 30, 459– 63 optical scanning, 24, 266, 348–67 optical scanning near-field microscopes (SNOM), 485 optical sensors, 316–21 optical sensory feedback, 10, 11 optical signal representation, 258 optical signal transmission, 27, 92– 5, 270 – 2, 428– 32 optical storage disks, 469– 84 beam tracking, 477– 8 components/operation, 10, 11 disk servo control system, 478– 84 optical system for focusing, 472– 7 specifications, 470, 472 system configuration, 470–2 optical switches, 10, 12, 259, 367– 77 optical systems configuration, 327– 8 focusing, 471– 7 laser printers, 458– 9 optical-to-electrical integrations, 282 optical-to-electrical signal transformation, 263, 264 optical-to-mechanical actuation mode, 269 optical-to-mechanical integrations, 283 – 4 optical-to-mechanical signal transformation, 263, 264 optical-to-mechanical signal transmission, 271, 272 optical-to-optical signal transformation, 263 optical-to-optical signal transmission, 271, 272 optical transceiver systems, 429

591 optical twizers, 277, 278 optic fundamentals, 31 – 103 optoelectronics, 4 – 6, 274 optoisolators, 204 optomechatronics actuation, 301 – 16 definition, 6, 8 – 20 fundamental concepts, 8 – 27 generic interfaces, 279 – 87 integration, 16 –20, 255 –97 interaction, 16, 17 interfaces, 173 – 252, 273 – 90 mechatronic interfaces, 173 – 252 sensing, 316 – 26 system characteristics, 16 – 20 orientation of objects, 545 orthographic projection models, 110, 111 out-of-focus displacement, 339 output interfaces, 207 overlapped area driving force, 218 – 19

P panning, 159 – 60 pan-tilt devices, 542 – 3 parallel open-loop configurations, 289 parallel series configuration, 291, 292 paraxial matrices, 379 –81 paraxial rays, 38 pass bands, 197, 198 passive vision systems, 109 PCB see printed circuit boards PD see proportional plus derivative PDP see plasma display panels peak times, systems performance, 239 performance digital micromirror devices, 534 laser printers, 455 – 7 systems, 2, 3, 238 – 41 permittivity, 176 – 7 permittivity of free space, 176, 181 perspective camera model, 152, 154 perspective matrices, 157 – 62 perspective projection, 110 – 11, 155 – 65 Petzval field curvature, 57 phase measuring interferometry, 485 photoconductive detectors, 191, 192 photoconductive drums, 452 – 4 photocurrent, 189 – 90, 192

592 photodiodes, 189– 92, 274, 275 photo-electric actuators, 301– 4 photoemissive detectors, 191–2 photon detectors, 191– 2, 274 photon-induced currents, 305–6 photons, 112 photostrictive actuators, 17, 18 photostrictive effects, 304– 5 photo-thermal actuation, 301, 311– 16 photo transistors, 317 photovoltaic detectors, 191, 192 photovoltaic effect, 304, 305 PI see proportional plus integral PID see proportional plus integral plus derivative piezoelectric actuators, 212– 15, 221, 275, 276, 304 – 11, 504 bimorph optical fiber sensors, 325– 6 effect, 179, 180 sensors, 179– 85, 325– 6 transducers, 274, 275, 331– 2 piezoresistive sensors, 184– 5 pincushion distortion, 59, 60, 351 pinhole cameras, 152, 154 pinhole systems, 327–8 pipe-welding process, 10, 12, 25 pitch rotation, 569 pits and lands, 470– 5 pixels, 112 – 14, 155– 64, 224 plane polarization, 61– 2 plane waves, 40 plano-concave lenses, 47 plano-convex lenses, 47 plasma display panels (PDP), 224, 225 plastic-leaded chip carriers (PLCC) inspection, 410– 11 pneumatic pressure control, 292 p-n junctions, 184 polarization, 61– 4 oscillation, 62– 3 piezoelectric effect, 180 polarized beam splitter, 518 polarized lighting, 402 polarizers, 62 pyroelectric sensors, 185 signal display, 272 polygonal scanners, 277, 349, 363– 7 polygon mirrors, 349, 363–7, 459– 63 polygons, 151, 152

Index polyvinylidene fluoride (PVDF), 179, 181 – 4 ports, input/output, 207 position-based visual feedback control, 414 – 15 position control systems, 362 – 3 positive meniscus lenses, 47 post-objective scanning, 348 – 50, 352 – 3, 461 power consumption, 455 measurement sensors, 262 optical, 47 resolvable, 319 – 20 transmission, 431 – 2 pre-objective scanning, 348 – 50, 461 preprocessing stages, 106 pressure sensors, 323 – 5 printed circuit boards (PCB), 6 – 8 printers, 8 see also laser printers probes, 484 – 510 program counters, 205 progressive sharpness, 522 projection digital micromirror device, 524 – 35 grating light value display, 524, 535 – 40 television, 524 – 40 tilt perspective, 159 – 61 projector evolution, 7, 8 propagation of electromagnetic waves, 32 – 3 proportional controllers, 242 – 4 proportional plus derivative (PD) controllers, 242 – 4 proportional plus integral (PI) controllers, 242 – 4 proportional plus integral plus derivative (PID) controllers, 242 – 4 p-type semiconductors, 186 – 7 pulse-width modulated (PWM) light intensity, 533 pupils, 53 – 5 PVDF see polyvinylidene fluoride Pwett operator, 139 PWM see pulse-width modulated pyroelectric effect, 304 pyroelectric sensors, 185 – 6 PZT see lead zirconate titanate

Index

Q quadrant photodiode detectors, 191 quadrant ring system, 403– 5 quality measure, 408– 10 quartz, 181, 182, 340, 341

R radiation classification, 32 random access memory (RAM), 205 rapid prototyping (RP), 10, 12 raster scans, 451, 484 ray diagrams/tracing, 34– 5, 37– 45, 50 – 3 Rayleigh criterion, 87, 365, 493 Rayleigh range, 97 read-only memory (ROM), 205 rectangular chips inspection, 409– 11 reflection laws of, 33– 6 matrix, 50– 2 optical fiber transmission, 92– 3 patterns, 109– 10 phase grating, 91– 2 plane polarization, 61– 2 reflective display, 524 reflective scanning, 266 reflective type confocal systems, 511 – 12 refraction indices, 35– 6 laws of, 33– 6 matrix, 50, 51 refractive scanning, 266 spherical surfaces, 37– 46 region-based segmentation, 133– 6 registers, 205 relative aperture, 54 relative permittivity, 176– 7, 181 remote operation, 27, 269– 70, 428, 430 resolution axial, 328– 30, 520– 1 beam scanning, 459– 60 confocus scanning microscopes, 520 – 1 illumination control, 405– 6 laser printers, 459– 63 lateral, 328– 9 microscopes, 485

593 optical, 328 – 30, 459 – 63 scanning, 364 – 6, 459 – 60 spot size, 455 writing, 456 resolvable power, 319 – 20 response curves, 359, 360 rigid bodies homogeneous transformation, 569 – 72 motion, 565 rotation, 565 – 8 velocity, 567 – 8 ring lighting, 402 – 5 rise time, 239, 241 Roberts operator, 138 – 9 Robinson compass mask, 139 robots, 11 – 12 roll rotation, 569 ROM see read-only memory rotating discs, 259 rotating mirrors, 515 –7 rotation homogeneous transformation, 569 – 72 matrices, 567 mirrors, 354, 355 rigid bodies, 565 – 7 velocity, 567 – 8 Routh arrays, 577 – 9 Routh stability criterion, 245, 335, 577 – 9

S sagittal focus, 58 – 9 sample and hold modules, 200 – 1 sampled sequences, 207 – 8 sampling frequencies, 208 – 10 sampling signals, 207 –10 scale issues/scaling, 18 – 20, 572 scanning, 266, 345 acousto-optic scanners, 515 atomic force microscopes, 501 – 10 beams, 459 – 60, 513 – 18 confocus scanning microscopes, 326 – 8, 485, 510 – 24 error correction, 367 focus, 353 – 4 galvanometric scanners, 349 – 63 line scanners, 459 – 63

594 microscopes, 484– 510, 515, 518– 20 motion control, 501 –10 objective, 348–50, 352–3, 461, 514 optical, 348– 67 patterns, 266 polygonal scanners, 277, 349, 363– 7 resolution, 364– 6, 459– 60 scan-spot size, 460 scanning electron microscopes (SEM), 485 scanning probe microscopes (SPM), 484 scanning tunneling microscopes (STM), 19, 484, 485 scene interpretation, 109 scratch drive actuators (SDA), 373– 5 screen projection, 534– 5 SDA see scratch drive actuators second of image-side focus, 40 second moments of inertia, 544– 8 segmentation, 127 –36, 148 –52 SEM see scanning electron microscope semiconductive capacitive optical actuators, 302 semiconductors, 186– 9, 302 semiconductor sensors, 186– 9 sensing definition, 258 functional units, 316– 26 fundamental functions, 22, 24 interferometric, 323 optical technology functions, 13, 14 optomechatronics, 316– 26 signal, 263, 266– 9 visual, 105– 71 sensors, 175–92 optical fibers, 11, 276, 321– 6 see also individual types sensory feedback-based optical system control, 23, 26 series configuration, 291– 2 series open-loop configurations, 289 series-parallel open-loop configurations, 289 servo-controlled optical actuators, 307– 8 servomotors, 17, 18 servo systems, 229– 30 settling time, 241 shadow-free lighting, 402 shape memory alloys (SMA), 205, 275, 276

Index actuators, 221 –2 photo-thermal actuators, 311, 312 walking machine, 312 – 15 shift registers, 113– 14 shutter modulation, 322 signal actuation, 258, 269 – 70 conditioning, 175, 193 – 210 display, 224 –5, 258, 272 – 3 flow, 288 – 90 manipulation, 258, 264 – 6 modulation, 259, 265 – 6, 269, 270 processing elements, 197 – 204 sampling, 207 – 10 sensing, 263, 266 – 9 storage, optical technology functions, 13 – 15 transformation, 258, 263 – 4 transmission, 27, 92 – 5, 258, 270 – 2, 428 – 32 see also data silhouette images, 401, 408 silicon actuators, 368 capacitive actuators, 302 – 4 diaphragm valves, 312 dioxide, 524 doping, 184 micro actuators, 302 nitride, 524 oxide, 369 Simulink model, 308, 334, 499, 509, 530 – 1 single lens zoom control, 378 single-mode transformation sensors, 268 single-slit diffraction, 74 – 9 sinusoidal signals, 208 – 9 size issues, 18 – 20, 455, 460 SLA see stereo-lithography apparatus slit diffraction, 74 –83 slit interference, 67 – 8, 81 slopes, beams, 576 slot-cams, 384 – 5 SMA see shape memory alloy small outline integrated circuits (SOIC), 409 – 11 SMD see surface mounting devices Snell’s law, 35, 37, 62, 92, 93 Sobel operator, 139 – 43, 146, 390, 393 – 4, 398 – 9, 543

Index SOIC see small outline integrated circuits solar cells, 192 source-fiber coupling, 431 spatial coherence, 64– 5 spectral distribution, 400 specular diffuse surfaces, 110 specular reflection, 35, 400, 402 specular surfaces, 109– 10 speed, 32 – 3, 347–8, 455 spherical aberrations, 55, 56 spherical surfaces, 37– 46 spinning polygon mirrors, 349, 363–7 split cell photodiode detectors, 191 SPM see scanning probe microscope spot diameter, 462, 473, 475 spot positioning, 479 spot size, 455, 460 spread, Gaussian beams, 96– 7 spreading phenomenon, 74, 75, 96–7, 473 spring-mass-damper systems, 215, 227 – 8 stability issues, 244– 5, 335, 577– 9 stack configurations, 213, 214 static electricity, 449 static force analysis, 505 steady state error, 241 steady state responses, 240– 1 step-index fibers, 94– 5 step input, 359, 360 stepper motors, 210 stereo-lithography apparatus (SLA), 12 STM see scanning tunneling microscope stop bands, 197, 198 storage systems, 470 stretching, 572 structural beam deflection, 494, 538– 9, 573 – 6 successive approximation, 199– 200 summing amplifiers, 194, 195 surface hardening, 27 surface mounting devices (SMD), 6, 7 switches/switching data, 22, 25 micro-optomechatronic switches, 368 on-off gate switches, 368 optical, 10, 367–77 optical fibers, 369, 370, 375– 7

595 system system system system system

evolution, 2 matrices, 50 – 2 resolution, 520 – 1 stability, 244 – 5 transfer functions, 233 – 8

T tandem scanning optical microscopes (TSOM), 515, 518 – 20 Taylor series, 530 Taylor triplet, 381 telephoto lenses, 381, 382 television, 524 –40 tellurium oxide, 340, 341, 344 TEM see transverse electromagnetic mode temporal coherence, 64 – 5 test signals, 239 – 40 texture, 108 thermal detectors, 191 thermal expansion, 369 – 70 thermally actuated bistable fiber switches, 369, 370 thermally actuated mirrors, 369 – 71 thermal poling, 180 thermal resistance, 227 thermal systems, 227, 228 thin-film heaters, 369, 370 thin-lens equations, 38 –40 three beam methods, 476 – 8 three-lens systems, 381 –3 thresholding, 127 – 33 tilt digital micromirrors, 527 – 30 perspective projection, 159 – 61 time division multiplexing, 201 – 2 time domain parameters, 239 – 40 tolerable angle error, 460 – 1 toroidal lenses, 350, 351 torque to inertia ratio, 356, 357 total internal reflection, 92 – 4 tracking beams, 477 – 8 error detection, 476 – 8 optical discs, 229 – 30, 473, 474, 477 – 8 storage disks, 473, 474, 477 – 8 units, 259 visual, 540 – 57 transceiver systems, 429

Index

596 transducers, 175, 274, 275, 331– 2 transfer functions, 233–8 transformation homogeneous, 569– 72 signal, 263– 4 transformation type optomechatronic integration, 279– 81 transient responses, 239– 40 translation matrix, 50– 1 transmission amplitude grating, 90– 1 confocal systems microscopes, 511 – 12 data, 14 – 15, 22, 25, 27, 270– 2 electrical transmission, 270, 271 full-duplex fiber optic, 430– 1 optical fibers, 92– 5, 428– 32 optical signal, 27, 92– 5, 270–2, 428– 32 power, 431– 2 signal, 27, 92– 5, 258, 270– 2, 428– 32 transversal configurations, 212, 214 transverse electromagnetic mode (TEM), 95 transverse magnification, 42– 3, 46 TSOM see tandem scanning optical microscope tube actuators, 502, 504 tube configurations, 213, 214 tunable wavelength filtering, 339– 40, 347 twizers, 277, 278 two-axis rotating mirror scanners, 516, 517 two-lens combinations/systems, 45, 47 – 50 two-mode sensor modules, 268– 9 two-signal integration, 273– 6

U ultrasonic motor drives, 385– 6 ultraviolet light, 400

V value, systems/machines evolution, 2, 3 Van der Waals force, 487, 488, 491, 492, 499 variable capacitance, 275 variable inductors, 274, 275

variators (zooming lens), 379 VCM see voice coil motors velocimeters, 347 velocity feedback, 317 measurement sensors, 262 rigid bodies, 567 – 8 vibrometers, 347 virtual image plane, 154 vision-guided micropositioning systems, 10 – 12 vision-guided precision robots, 11 – 12 vision sensors, 326 visual autofocusing, 386 – 99 visual information feedback control, 24– 5, 260– 1, 411 – 28 architectures, 414 – 28 controller design, 421 – 8 eye-in-hand systems, 413, 414, 420 – 1 fixed camera systems, 412 – 14, 417 – 20 fundamental functions, 24 – 5 visual information processing, 105 – 9 visual sensing, 105 – 71 visual sensors, 326 visual servoing, 25, 412 visual tracking, 540 – 57 feature extraction, 544 – 8 image processing, 541 – 3 moving objects, 548 – 57 zoom lens control, 549 –57 voice coil motors (VCM), 216 – 18, 229, 230, 331 – 6 voltage, 187, 192, 325 – 6 volume force, 19

W waiting speed, 455 walking machines, 312 – 15 washing machines, 6, 10, 11, 25 wave front correction, 315 waveguide dispersion, 432 wave length frequency shifters, 23, 26 wavelength modulation, 321, 323 weapons ignition, 17, 18 weighted median filter, 390 – 1, 393 welding, 10, 12, 25 Wheatstone bridges, 202 – 3 white light, 91 wide angle lens systems, 381, 382

Index world coordinate system, 155– 64 writing resolution, 456

X x-directional staging, 505, 506 x-stage simulation, 509– 10 x-y scanners, 345, 484– 510

Y yaw rotation, 569 yoke, 526 – 7 Young’s experiment, 67– 9, 81 Young’s interference patterns, 81 Young’s modulus, 489, 496, 539, 573

597

Z Zener voltage, 192 zoom control, 377 –86 illumination control, 406 – 7 mechanism, 383 – 6 visual tracking systems, 549 – 57 zooming principles, 377 – 83 zooming-in, 381 zooming lens, 379 zooming-out, 381 zooming principles, 377 – 83 zoom lenses, 18, 378, 379, 381 – 3, 549 – 57 zoom ratio, 377 z scanners, 484 – 510