Multidisciplinary Research in Control: The Mohammed Dahleh Symposium 2002 (Lecture Notes in Control and Information Sciences)

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari

289

Springer Berlin Heidelberg NewYork Hong Kong London Milan Paris Tokyo

Laura Giarr´e, Bassam Bamieh (Eds.)

Multidisciplinary Research in Control The Mohammed Dahleh Symposium 2002 With 63 Figures

13

Series Advisory Board A. Bensoussan · P. Fleming · M.J. Grimble · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · J.N. Tsitsiklis

Editors Prof. Laura Giarr´e Dip. di Ingegneria dell’Automazione e dei Sistemi (DIAS) Universit`a di Palermo Viale delle Scienze 90128 Palermo, Italy [email protected] Prof. Bassam Bamieh Dept. of Mechanical and Environmental Engineering University of California at Santa Barbara (UCSB) 93106 Santa Barbara, CA, USA [email protected]

ISSN 0170-8643 ISBN 3-540-00917-5

Springer-Verlag Berlin Heidelberg New York

Cataloging-in-Publication Data applied for Bibliographic information published by Die Deutsche Bibliothek Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in the Internet at . This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution act under German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science + Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2003 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by the authors. Final processing by PTP-Berlin Protago-TeX-Production GmbH, Berlin Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper 62/3020Yu - 5 4 3 2 1 0

Preface

This volume is not only dedicated but mainly inspired by our unforgettable friend and colleague Mohammed Dahleh whose memory is indelible. True to his intellectual legacy, the first Mohammed Dahleh Symposium (http://www.engineering.ucsb.edu/˜ mdsymp) was organized in Santa Barbara on February 8-9, 2002, encouraging the participation of researchers from a broad range of disciplines, giving presentations on new emerging research areas of key significance. These new areas share in common the fact that system dynamics and control theory provide the appropriate framework for the understanding of the core phenomenon in each area, and at the same time, provide many of the tools necessary for their application to relevant technologies. The importance of these emerging areas in the current research agenda in science and technology creates a unique opportunity to drastically extend the scope and impact of dynamics and control methods far beyond their traditional areas of application in engineering. This volume, originating from the Symposium to honor Mohammed’s legacy, reflects the multidisciplinary nature of the new research domains that have been addressed, and it includes researchers with expertise in various disciplines within engineering, physics, biology, and applied mathematics. Mohammed promoted free thinking and broad education and his view of the role of university and research is briefly summarized in this quote: First of all the main mission of the university is teaching. Teaching in the sense of analysis, critical thinking, and creativity. To achieve this goal successfully, I believe, that there must be research that is not bound completely by the desires and requirements of the society. It should draw inspiration from life, in all its forms, and in all its complexity. That’s why in a university people study poetry,literary criticism, cosmology, and yes even pornography. The product of the university is the educated, analytical student, and not a transistor radio. Maybe a transistor radio can come out, and indeed it did, from a university research environment, but that is not the intended product it was just a nice byproduct. The volume consists of two parts. The first part presents Mohammed Dahleh biography, and his legacy to the control community. The second part comprises the main contributions of the symposium divided in the following topics: Control in Networks and Communications, Quantum Control, Micro and Nano-scale Dynamics and Control, Connections with Biology, Control and Identification. Santa Barbara December 2002

Bassam Bamieh Laura Giarr´e

Contents

Part I. Mohammed Dahleh Legacy Legacy of Mohammed Dahleh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Munther A. Dahleh 1 Childhood (1961-1979) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 College Years (1979-1987) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 Faculty at TAMU 1987-1990 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 The Decade at the University of California at Santa Barbara, 19912000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5 Mohammed’s Sickness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 6 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Part II. Control in Networks and Communications A Control Theoretical Look at Internet Congestion Control . . Fernando Paganini, John Doyle, Steven H. Low 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Control design with linear scalable stability . . . . . . . . . . . . . . . . . . . . . 4 Nonlinear laws and the equilibrium structure . . . . . . . . . . . . . . . . . . . . 5 Signaling requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Indelible Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nicola Elia 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Problem Set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Channels as Model Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Mean Square Closed Loop Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Minimal Channel Quality for Means Square Stability . . . . . . . . . . . . . 6 Mean Square Stability Robustness to Structured Bounded Variance Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Special Cases with only One Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Minimal Channel Quality for the Stabilization of a Pendubot . . . . . . 9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17 17 18 22 27 29 30 30 33 33 35 36 37 38 39 41 42 43 44 45

IV

Contents

Optimal Control Design under Structural and Communication Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Petros G. Voulgaris 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Specific Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Controller Parametrization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Optimal Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Conclusions and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47 47 48 54 56 59 60

Part III. Quantum Control Fifteen Years of Quantum Control: from Concept to Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anthony Peirce 1 Preamble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Molecular control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Design by intuition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 The optimal control formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Uncertainty and robust design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Experiments and closed loop design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Directions in the Theory of Quantun Control . . . . . . . . . . . . . . . . . Domenico D’Alessandro 1 Model of finite-dimensional quantum dynamics . . . . . . . . . . . . . . . . . . 2 Lie algebra structure, controllability and analysis of quantum systems 3 Methods for control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65 65 66 66 67 69 70 71 72 73 73 75 77 78 78

Part IV. Micro and Nano-scale Dynamics and Control System tools applied to micro-cantilever based devices . . . . . . . . A. Sebastian, S. Salapaka, M. V. Salapaka 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Atomic Force Microscope: operating principles and features . . . . . . . 3 Systems approach to the analysis of AFM dynamics . . . . . . . . . . . . . . 4 Broadband nanopositioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83 83 84 85 90 96 97

Contents

Micro-scale sensors and filters utilizing non-linear dynamic response of single and coupled oscillators . . . . . . . . . . . Kimberly Turner, Rajashree Baskaran, Wenhua Zhang 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 MEMS mass sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Electrostatically Coupled Oscillator filter . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

V

101 101 102 108 112

Part V. Connections with Biology Feedback Regulation of the Heat Shock Response in E. coli . . . Hana El-Samad, Mustafa Khammash, Hiroyuki Kurata, John C. Doyle 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 The Heat Shock Response: A Case Study . . . . . . . . . . . . . . . . . . . . . . . 3 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stem Cells from the Outside In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marie Csete MD, PhD 1 Introduction: Stem cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Oxygen levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimal Image Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Steven Haker, Allen Tannenbaum 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Area-Preserving Diffeomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Formulation of Optimal Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Gradient Descent for Optimal Transport . . . . . . . . . . . . . . . . . . . . . . . . 5 Optimal Image Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Optical Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Computer Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 115 117 126 127 129 129 130 131 132 133 133 134 135 136 137 138 140 140

Part VI. Control and Identification Robustness of Finite State Automata . . . . . . . . . . . . . . . . . . . . . . . . . Alexandre Megretski 1 System Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Analysis and Design of FSA/IC Models . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

147 148 153 160

VI

Contents

On the role of homogeneous forms in robustness analysis of control systems . . . . . . . . . . . . . . . . . . . . . . G. Chesi, A. Garulli, A. Tesi, A. Vicino 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Problem formulation and preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . 3 LMI-based conditions for the solution of the POFH problem . . . . . . . 4 POFH problems in control system analysis . . . . . . . . . . . . . . . . . . . . . . 5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control of Electromechanical Actuators: Valves Tapping in Rhythm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Katherine Peterson, Anna Stefanopoulou, Yan Wang 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Variable Valve Timing Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Soft Landing Methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Nonlinear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Control Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Observer Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Controller Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Extremum Seeking Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning complex systems from data: the Set Membership approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mario Milanese and Carlo Novara 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 The Nonlinear Set Membership approach . . . . . . . . . . . . . . . . . . . . . . . 3 Nonlinear systems identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Prediction of nonlinear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The mixing of state statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tryphon T. Georgiou 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 The structure of state covariances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Mixing of state statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Controllability, integrability and ergodicity . . . . . . . . . . . . . . . . . . Igor Mezi´c 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Hamiltonian systems in action-angle variables . . . . . . . . . . . . . . . . . . .

161 161 163 165 169 175 175 179 179 180 181 182 183 187 189 190 192 192 195 195 196 201 203 204 205 207 207 207 209 211 213 213 214 218

Contents

4 Controllability by flatness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Ergodicity and controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Discussion and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

VII

221 223 226 227 228

Legacy of Mohammed Dahleh Munther A. Dahleh LIDS, MIT, Cambridge MA Abstract. Mohammed led a broad research program that manifested itself in: key publications within and outside the control community, in a number of students with superb multidisciplinary training, a number of organized workshops that attracted researchers from all over the world, and friendships within the control community that became an integral part of Mohammed’s life. In this article, I will attempt to describe aspects of Mohammed’s broad and diverse legacy in our community.

1

Childhood (1961-1979)

On the 12th of February, 1961, Mohammed was born in the small town of Tulkarim in the West Bank, Palestine, together with a twin sister, who died shortly after from a vicious virus. Mohammed grew up in a well educated family; his father Abdullah was an Engineer who was educated in the US and his mother, Wisam, was an English teacher, who left her teaching career to attend to her children. Mohammed had two siblings, an older sister who is currently living in Jordan and is a high school English literature teacher, and myself. Mohammed attended a Catholic nursery school in the West Bank town of Nablus, and after 1966, he attended the Islamic Scientific college (ISC) in Amman, Jordan, where he received all of his early schooling. Mohammed developed a special reputation at the ISC. While his brilliance was observed by both the teachers and the students in the school, it was his unconventional character that was most striking about him. Mohammed did not fit the model of the ”good student” who was studious, obedient, and conformist, but rather he showed signs of intellectual and character independencies very early. Before high school, Mohammed simply ignored the rigid structure of the school. He paid moderate attention to his homework, he did not engage the teachers, and he ignored exams and rankings as well as all extra curricular activities imposed by the school. He pursued his own vast interests with vigor. His curiosity spanned many areas including Sciences, mathematics, Philosophy, languages, and especially geography. With the absence of effective youth programs at that time in Jordan that may have nurtured such broad interests, Mohammed took up reading as a venue for self-education which became a passion in his life that he shared with close friends and family. This unconventional style was accompanied with an extremely daring behavior which may surprise those who knew Mohammed only in the past 20 years. As a

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 3--14, 2003 Springer-Verlag Berlin Heidelberg

4

MuntherÿA. Dahleh

child, he repeatedly used the back of a bus as a method of transportation. One time he fell and broke his hand. He routinely climbed buildings using the collapsing external pipes-once to rescue a cat that was trapped on the second floor when the pipe broke. Surprisingly, this behavior was associated with a great sense of responsibility and loyalty to his friends and family. His generosity and kindness, with his intensity and brilliance made him a focal point among his associates and attracted many people to him with vastly different styles or interests. While the education at ISC was quite rigorous, it was quite conventional, and in indirect ways, very limiting. Mohammed recognized this early on and handcrafted his education while utilizing the school in very creative ways. He built strong relationships with specific teachers who were open minded about education, particularly the western view. While he rebelled against dogmatic interpretations of religion and culture, he took it upon himself to introduce progressive writings by both eastern and western scholars and encouraged students and teachers to critique their work. Because of his immense love for poetry, which he inherited from his mother, he organized poetry competitions in the school that became a tradition of the ISC during their various celebrations. Although Mohammed benefited tremendously from the rigorous mathematical and scientific training provided by the school. Nonetheless, he was critical of the lack of parallel experimental training which was a handicap to his education that took several years to overcome. Mohammed’s curiosity transcended subjects to people. Most of his early friends were extremely talented, but not in ways that were recognized by the society. His best friend, a neighbor, who was several years older than Mohammed was a school drop out, whose sole interest was to become a mechanic. Another school friend was an orphan raised by his grandmother who had a talent in writing comedy books. Mohammed watched both of these friends get destroyed by a very rigid system that had low tolerance to unconventional career paths. Often he spoke to his teachers about reforming such a system but to no avail. Mohammed mentioned often that most of his close friends from elementary school never made it beyond high school. I strongly believe that these experiences shaped his thinking as he thought about ways to realize his interests. The political situation in Jordan influenced Mohammed’s perspective on life. As he jokingly used to put it when confronted with a difficult situation: ”I have been through three wars, I can deal with this problem”. Black September in 1970 possibly had the most critical effect on Mohammed. Many people will have heroic stories to tell about their survival, while Mohammed’s repeated reaction was: ”I am thankful to be alive”. His recognition of the failure of war in resolving fundamental conflicts came at that early stage and shaped his thinking on addressing conflicts at all levels. He vehemently opposed war anywhere in the world and passionately supported peaceful resolutions. This was Mohammed’s approach to any conflict situation he encountered. Mohammed experienced a sudden turn around in high school. He began to focus on excelling in sciences and mathematics as he planned to pursue an academic career in research and teaching that would begin with higher education in the US. His independent study put him at least two years ahead of his class. He wrote his first review article when he was 16 years old summarizing the various methods of integration, and presenting a set of challenging problems for students to prepare for the general Baccalaureate examination after high school. This examination, known

Legacy of Mohammed Dahleh

5

as Tawjeehe, determined the student’s field of study and consequently their future career path if they chose to study in Jordan or neighbouring Arab countries. Ironically, students who did not perform well were forced to look for educational opportunities abroad, and in some cases, ended up with a better education than their fellow students who did well on the exam. When Mohammed took the Tawjeehe, he ranked as the number one student in the nation which included over sixty thousand students. His reputation spread nationwide as he emerged as one of the most talented students in the country. Despite the fact that Mohammed could choose to study anywhere in the Arab world and any topic he pleased, he still carried on with his plan and headed to the United States to get his education and pursue his dream.

2

College Years (1979-1987)

Mohammed arrived at Texas A & M (TAMU) with an intense desire to learn and explore. He chose electrical engineering as a major, although mathematics was his passion. After his freshman year, he was already known among many math professors as a star student, and by the end of his sophomore year, he was known among the electrical engineering faculty. He constructed his own education by planning all his courses and petitioning to replace required classes with ones that he deemed more appropriate or interesting. During his junior year, he was already taking graduate level courses in electromagnetic theory. He took on a project with Professor Tsang on understanding reflections of electromagnetic waves over periodic structures which became the topic of his BS thesis. This work resulted in Mohammed’s first journal publication in the Journal of Applied Physics [1]. This was Mohammed’s first experience in research and he was thrilled by it. At TAMU, Mohammed gained a great education. Apart from choosing excellent courses to take, he exploited the option of taking independent study classes from professors on topics of their research. Again, he built very close relationships with his teachers and engaged them in a process to help him build a very strong background necessary to pursue his plans. Mohammed also took the opportunity to develop a strong experimental and computational training by taking advanced courses with laboratory components. Four years later, Mohammed left TAMU and headed to Princeton to pursue his Ph.D. in applied mathematics. The decision to continue his studies in mathematics was a difficult one given that, on the one hand, Mohammed had a choice to pursue engineering at the top schools in the country, and on the other hand, because his early education was in a system that valued only two disciplines; engineering and medicine. Nevertheless, Mohammed did not allow this bias to prevent him from pursuing his dream of having an academic career in the area he passionately enjoyed.

6

MuntherÿA. Dahleh

It was at Princeton that Mohammed set forth his future career. He started his Ph.D. in the field of continuum mechanics, but after his general examination, he recognized that this was not what he really enjoyed. Mohammed struggled to find a thesis topic in the applied mathematics program that he felt strongly about. He began doubting his choice, and started considering other alternatives. He swiftly made a change and began working with Professor Hopkins in electrical engineering on adaptive control. He embraced the field with tremendous passion, and was able to answer some hard open questions about rapid switching of adaptive systems. His interest in robust control was rising, while he still maintained a broad perspective of the field and how it applied to various disciplines. In his Ph.D. thesis, Mohammed considered the open problem of adaptive stabilization for delay systems [2,4,5]. During his research, Mohammed recognized the interplay between adaptation and robustness. Consequently he introduced the idea of adaptive/robust control strategies whereby adaptation focuses on identifying and controlling systems with finite-parameter uncertainty, and the feedback control is constructed to be inherently robust to dynamic perturbations. Aspects of this formulation was published in his papers [6,11]. At Princeton, Mohammed met Anthony Peirce, a fellow student in the applied mathematics program. Mohammed and Anthony became life-time friends and collaborators. Anthony was working with Professor Rabitz, a physical chemist, on reaction diffusion problems and became aware of the complexity of the laser field design problem due to the complex molecular behavior described by Schrodinger’s equation. Anthony began discussing this problem with Mohammed, who immediately recognized it as a control problem. This interaction lead to the publication of papers [3,8] that initiated a whole research program in quantum control for Mohammed in subsequent years. Mohammed emerged from Princeton as a true scholar. Not only did he build an incredible broad background in engineering and mathematics, and utilized it in an excellent Ph.D. experience, he also mastered the ethics of research whether conducted in an independent or collaborative fashion. He also broadened his education further by building up his knowledge in social sciences and languages. Mohammed exploited the very rich environment at Princeton and became affiliated with the near-eastern program, where he participated in many of their activities and seminars. Throughout this time, he built close friendships with many people from various parts of the world, many of which he maintained for many years afterwards. At Princeton, Mohammed met his wife, Marie, who was a graduate student in the applied mathematics program. Mohammed and Marie were married in 1986, and a year later, Mohammed concluded his Ph.D. thesis and returned to his alma mater. Dividing her time between Princeton and TAMU, Marie completed her PhD thesis in 1990.

3

Faculty at TAMU 1987-1990

Mohammed returned to TAMU in 1987 as an Assistant Professor. At this new position, he engaged the faculty and students with full excitement and ambition. When he arrived, Professor Bhattacharyya and some of his students were working on problems concerning robustness with real parameter uncertainty. The work followed the seminal work of Kharitonov characterizing stability robustness with respect to

Legacy of Mohammed Dahleh

7

parametric uncertainty by checking the stability of four polynomials. Mohammed was interested in seeing the limit of such a theory in addressing realistic problems. In collaboration with Bhattacharyya and their student Chapellat, Mohammed derived a series of results that played a major role in the development of this area. Initially, they pursued extending the results of Kharitonov for multi-linear interval plants [7,10]. But their major contribution was in deriving computable conditions for analyzing classes of mixed parametric and non-parametric uncertainty [9]. Establishing this connection between H∞ robust control and parametric uncertainty was a fundamental contribution to robust analysis of uncertain systems. In 1988, Mohammed attended a workshop in Torino on Robust Identification and Control. This workshop aimed at bridging the gap between the divided camps in robust control. Mohammed met two people at this workshop whose lives became intertwined with his: Alberto Tesi and Antonio Vicino. Both Alberto and Antonio became close friends and collaborators ([19]-[21]). Mohammed’s work with them focused on showing that systems with parametric, H∞ , and sector bounded uncertainties enjoy remarkable extremal properties in robust stability and performance analysis and design of feedback. During this trip Mohammed also fell in love with Italy.

4

The Decade at the University of California at Santa Barbara, 1991-2000

Mohammed joined the University of California at Santa Barbara (UCSB) in 1991, during the formation of the Center for Control Engineering and Computation (CCEC) under the leadership of Petar Kokotovic. As the youngest CCEC faculty, Mohammed was also its most creative and innovative contributor. In 1995, along with the promotion to Full Professor, Mohammed was also named the Center’s research director. In the Center he initiated and led new research directions in atomic force microscopy and microcantilevering, in flow modeling and control, and in quantum computation and control. He also dedicated much of time and energy recruiting and mentoring younger faculty. Under the joint leadership of Petar and Mohammed, the Center for Control Engineering and Computation promoted an unprecedented culture of collaboration and inclusiveness. Many workshops were organized through the center that brought together people from all over the world, not only to interact on the technical work, but also to interact socially in order to benefit from their vastly different experiences. Some of the major workshops are: Workshop on “Differential Games and Robust Control”, May 1991; Workshop on “The Modeling of Uncertainty in Control Systems”, June 1992; Workshop on “Robust Controller Designs and Differential Games”, May 1993; Workshop on “`1 Methods in Robust Control Design”, June 1996; Workshop on “Dynamics, Control and Computation,” April 2-3, 1998; Workshop “Vistas in Control”, April 1999, to name only a few. Numerous short courses and seminars were given by leading experts in the field. Sharing their enthusiasm and responsibilities for the center, Petar and Mohammed developed a warm and enriching friendship seldom seem between men with a 27 year age difference. Their daily discussions often went beyond research topics into literature, religion and philosophy, music, history and politics. In this

8

MuntherÿA. Dahleh

way, they also stimulated broader cross-cultural interests of their colleagues and students. Mohammed widely traveled and participated in many international meetings to build relationships with researchers from all over the world in order to enhance the research program at UCSB. For example, in 1993, Mohammed attended the IMA in Minnesota where he met Sasha Megretski. Sasha’s career path was completely altered after Mohammed facilitated his interview at Iowa state University. During many workshops, Mohammed had the chance to interact closely with John Doyle. That interaction grew to a close friendship and to a very fruitful collaboration between UCSB and Caltech. Mohammed’s career at UCSB reached new levels. He was recognized as an outstanding teacher by both the Mechanical Engineering Department and the university academic senate. He promoted a truly multi-disciplinary research program built around robust control theory as its core, which spanned areas including quantum mechanics, fluids, and nanotechnology. During this time, Mohammed and Marie, had two children, Taher (1994) and Jumana (1996). Mohammed and Marie dived into parenthood with tremendous excitement and joy. It was apparent that Mohammed was experiencing a great fulfillment and flourishment in various aspects of his life. Mohammed always believed that a strong research program needs a strong core. Because of that, he devoted part of his time to the development of foundational issues in robust control. This included his work on controller design for mixed objectives [30–32,34,37] which culminated in the publication jointly with M. Salapaka of the research monograph: ”Multiple Objective Control Synthesis”. Also included is his work on non-standard robustness analysis in the presence of mixed uncertainty (e.g., plant perturbations described with different norms) [42,29,33]. Mohammed’s interest in nanotechnology began with his discussions with Arun Majumdar who was at UCSB until 1997. Arun was the head of a laboratory that housed an atomic force microscope (AFM). AFM’s have revolutionized microscopy making practical imaging and manipulation of material at the atomic scale. Mohammed recognized the immense underlying potential of this micro-cantilever based technology and the contributions dynamical and control systems theory can make to take this technology to the next frontier. This led to a research program that focused on the study of complex microcantilever interactions as documented by the articles [38–41,36]. Mohammed supervised several students on this topic, including S. Ashhab, A. Daniele, and S. Salapaka. The theoretical study of complex AFM dynamics, nanopositioning, and friction was well complemented by a laboratory that Mohammed initiated which eventually housed an Atomic Force Microscope and control and signal processing hardware to test the theories being developed This effort was also well supported by equipment grants by Digital Instruments, a leading manufacturer of AFM’s. It was Bassam Bamieh who got Mohammed interested in the Fluids area. At Mohammed’s initiative supported by Petar, Bassam came to UCSB as a visitor

Legacy of Mohammed Dahleh

9

for one year. At that time, Bassam has developed the theory behind the design of distributed controllers for spatially invariant systems, and became interested in fluids dynamics as an application domain. Studying existing approaches to modelling transition to turbulence he found the literature on what is called ”non-normal growth” and the ”pseudo-spectrum” (e.g., the work of Farrell, Butler, Henningson, Schmidt, and Trefethen et al.). Bassam presented these papers to Mohammed, and they both recognized the resemblance of this machinery to robust control theory. In fact, these researchers were gradually reinventing this machinery from scratch. For example, the pseudo-spectrum is related to the H∞ robust stability problem, and the non-normal growth results aim at quantifying transient growth of energy (which can be related to H2 norms). Mohammed and Bassam looked at a particular paper of Farrell and Ioannou (on energy amplification), and they interpreted it as computing the H2 norm of a spatio-temporal system. They realized that they can analytically prove the R3 growth of energy amplification as well as compute analytically part of the spatial frequency response of the system (the part that shows the dominance of the so-called streamwise vortices and streaks). These calculations were done numerically in the Farrell and Ioannou paper, but the analytical computations clarified the underlying amplification mechanisms, and in particular why this occurs in 3D in contrast to 2D flows. This paper was published in the journal Physics of Fluids [42], and was the starting point of this collaboration on the fluids problem. The conference version of this paper was published at the ACC and received the Hugo Shuck prize. This work drew the attention of many control theorists such as John Doyle and Petar Kokotovic who quickly started advertising it and making the connections to the appropriate experimentalists in the Fluids area. Mohammed also pursued the work on quantum control, playing the role of a reformist for the physics community. His paper in Science with Rabitz and Warren [18] created a splash among physical chemists and generated a lot of interest in robust control. Mohammed continued to analyze the robustness problems that arise in designing laser fields while being constrained to only open loop strategies due to physical constraints preventing the use of feedback in such setting. This research sprung into many other directions as Mohammed mentored Ph.D. students working in this area [26,27] Mohammed is to be credited for an unprecedented period of progress in the Mechanical Engineering Department at UCSB. Apart from his research and teaching efforts, he served as a vice-chair of the Department. During that time, he recruited three key faculty members in the dynamics and control area: Igor Mezi´c, Anna Stefanopoulou, and Bassam Bamieh. He utilized the Center for Control and Computation to integrate the activities of his department with electrical engineering. He also worked extremely hard at finding ways to support the best international students to come to UCSB. He promoted free thinking and broad education.I will take the liberty here to share with you something that Mohammed wrote in response to a criticism to academic institutions: well, I disagree with the claim that we don’t do anything useful. First of all the main mission of the university is to teach analysis, critical thinking, and creativity. To achieve this goal successfully, I believe, that there must be research that is not bound completely by the desires and requirements of the society. It should draw inspiration from life, in all its forms, and in all its complexity. That’s why in a university, people study poetry, literary

10

M. A. Dahleh

criticism, cosmology, and yes, even pornography. The product of the university is the educated, analytical student, and not a transistor radio. Maybe a transistor radio can come out, and indeed it did, from a university research environment, but that is not the intended product it was just a nice byproduct. I try to model myself, if i can, after the young English teacher in David Lodge book. I argue constantly against my engineering colleagues in support of free thinking research in all aspects of university life including engineering. I hate to think of the university as a factory. If it turns into one I will take a very early retirement. Mohammed continued his extensive interaction with the Italian control community through exchanging visits with Alberto Tesi, Antonio Vicino, and Laura Giarr´e and their research team, and through organizing joint workshops and seminars. Because of this strong technical interaction, and his interest in the Italian language and culture, Mohammed took a sabbatical leave and spent it in Siena in 1995. This collaboration continued after he went back to the U.S. as he started and promoted (informally) an exchange program with Italian Ph.D. students visiting UCSB or other faculty in the U.S. Mohammed used to always say jokingly that he will make sure he has at least one Italian student working with him at any given time. At conferences, Mohammed became the focal point of an international group of researchers known as the ”Panettone connection” (the name came from the special Italian Cake that became a tradition to eat at conferences) who interacted both technically and socially and assisted many young researchers in building their careers.

5

Mohammed’s Sickness

The joy of success was not long lived as Mohammed was diagnosed with colonliver cancer in January, 1999. Mohammed’s family and friends all came together to help him through this tragic sickness. He got admitted to UCLA and began receiving treatment by one of the best oncologists in the nation. Mohammed faced his sickness with his usual poise and courage. With an intense determination to survive, Mohammed was able to begin a reversal process, and within 3 months, he recovered much of his health. To all of us who lived through his struggles, it was clear that Mohammed’s love for life, his determination, his patience, and his faith, much more than the medication, got him through the initial sickness. I don’t think Mohammed ever felt he fully recovered, but he was clearly thankful for this opportunity and he shared this time fully with his kids, family, and friends. To all of us, this time was a great gift we dearly cherished. Mohammed then lived for the next year with an exemplary posture as a father, husband, son, brother, advisor, and friend. The joy did not last too long, and Mohammed’s attempted liver resection operation failed in February 2000. With a broken spirit, and a beaten up body, Mohammed continued to fight the disease, however, also taking the opportunity to say his goodbyes to his family and friends. Mohammed passed away on July 29, 2000.

6

Final Remarks

Mohammed’s legacy is only partially in his technical papers and the books that he authored. His full legacy is in his two gorgeous children, Taher and Jumana, and

Legacy of Mohammed Dahleh

11

in the memory all of his family, friends, students and colleagues have of him. It is in what he left to all of us; the love of life, the value of friendship, the excitement about small beautiful things. I would like to share a quotation from Mohammed: But, what is it that brings happiness and joy? To me real joy comes from the simplest things and almost unnoticed episodes...they never come from intense and excessive emotional experiences...it could be a smile, Taher’s reluctant smile as I touch his hands on the way to school in the morning, the memory of a beautiful song or poem or a face....to me the day’s monotonous experiences mean very little, and my heart and soul is constantly searching for these rare and gentle moments... when i think back on my life, these are the only feelings i maintain and my life seems utterly affected and punctuated by them... Mohammed is missed, not only for his generous support of his associates, but mostly because of his dynamic creative personality and calming presence. I will take the liberty here to quote some of his friends as they talked about him in his memorial service: Mustafa Khammash: Mohammed was a unique person who had many good qualities that one rarely finds in a single individual. He was a warm and kind person, a true gentleman. On top of all, he was a generous person, and he gave a lot of himself to others. He was at once very broad and very deep. No matter what topic was being discussed, he always brought to any discussion a new and interesting point of view that would carry the discussion to a higher level. Antony Pierce: Mohammed was not only a brilliant student, but a compassionate human being, intensely curious not only about partial differential equations, but also about history, politics, movies and literature. He was as comfortable debating the meaning of life with the more philosophically inclined, as explaining a subtle point of Functional Analysis to a fellow student, or arguing an obscure point about Arabic Grammar with an Arabic Scholar. Laura Giarr´e: Mohammed was usually saying that in a friendship (or in general in life) it is not the quantity of time you spend together that matters but the quality. In any of his meeting with friends or colleagues or students or with the last person known on a bus coming from the airport he was giving the same enthusiasm, the same intensity and the same attention, as if the person he was talking to was the most important in the world. Petar Kokotovic: Never have I worked with someone so naturally and genuinely generous, with such an ability to share with others his time, his immense energy, his deep insights, his intensity of thought, his breadth of talent, his intelligence and his love of life. As long as I live, I will remember him as a great humanist of our time. With him disappeared a vital source of inspiration and optimism. I experienced his untimely death as a personal tragedy darkening my old age. Finally, it is not possible to articulate the personal loss that each one of us have experienced in Mohammed’s death. Even though he is constantly present in our hearts, our lives will never be the same without him.

12

MuntherÿA. Dahleh

References 1. M. Dahleh, R. Nevels, L. Tsang, “Plane-Wave Diffraction by a DielectricCoated Corrugated Surface,” J. Applied Physics, Vol. 58, No. 2, pp. 646-650, 15 July, 1985. 2. M. Dahleh, W. E. Hopkins, Jr., “Adaptive Stabilization of Single-Input SingleOutput Delay Systems,” IEEE Trans. Automat. Contr., Vol. AC-31, No.6, pp. 577-579, June, 1986. 3. A. P. Peirce, M. Dahleh and H. Rabitz, “Optimal Control of Quantum Mechanical Systems: Existence, Numerical Approximations and Applications,” Physical Review A, Vol. 37, No. 12, pp. 4950-4964, 15 June, 1988. 4. M. Dahleh, “Sufficient Information for the Adaptive Stabilization of Delay Systems,” Systems and Control Letters, Vol. 11, No. 5, pp. 357-363, November, 1988. 5. M. Dahleh, “Generalization of Tychonov’s Theorem with Application to Adaptive Control of SISO Delay Systems,” Systems and Control Letters, Vol. 13, No. 5, pp. 421-427, December, 1989. 6. M. Dahleh, M. A. Dahleh, “Optimal Rejection of Persistent and Bounded Disturbances: Continuity Properties and Adaptation,” IEEE Trans. on Automat. Contr., Vol. AC-35, No. 6, pp. 687-696, June, 1990. 7. H. Chapellat, S. P. Bhattacharyya, and M. Dahleh, “Robust Stability of a Family of Disc Polynomials,” International Journal of Control, Vol. 51, No. 6, pp. 1353-1362, June, 1990. 8. M. Dahleh, A. Peirce, and H. Rabitz, “Optimal Control of Uncertain Quantum Systems,” Physical Review A, Vol. 42, No. 3, pp. 1065-1079, August, 1990. 9. H. Chapellat, M. Dahleh, and S. P. Bhattacharyya, “Robust Stability Under Structured and Unstructured Perturbations,” IEEE Trans. Automat. Contr., Vol. AC-35, No. 10, pp. 1100-1108, October, 1990. 10. H. Chapellat, M. Dahleh, and S. P. Bhattacharyya, “On Robust Nonlinear Stability of Interval Control Systems,” IEEE Trans. Automat. Contr., Vol. AC36, No. 1, pp. 59-67, January, 1991. 11. M. Dahleh, and M. A. Dahleh, “On Slowly Time Varying Systems,” Automatica, Vol. 27, No. 1, pp. 201-205, January, 1991. 12. M. Dahleh, and A. P. Peirce, “Numerical Solution of a Class of Parabolic Partial Differential Equations Arising in Optimal Control Problems with Uncertainty,” Numerical Methods for Partial Differential Equations: An International Journal, Vol. 8, No. 1, pp. 77-95, January, 1992. 13. M. Dahleh, A. P. Peirce, H. Rabitz, “Design Challenges for Control of Molecular Dynamics,” IEEE Control Systems Magazine, Vol. 12, No. 2, pp. 93-94, April, 1992. 14. M. Khammash, and M. Dahleh, “Time-Varying Control and the Robust Performance of Systems with Structured Norm-Bounded Perturbations,” Automatica, Vol. 28, No. 4, pp. 819-821, July, 1992. 15. K. L. Moore, M. Dahleh, S. P. Bhattacharyya, “Iterative Learning Control: A Survey and New Results,” Journal of Robotic Systems, Vol.9, No. 5, pp. 563-594, July, 1992. 16. H. Chapellat, M. Dahleh, “Analysis of Time-Varying Control Strategies for Optimal Disturbance Rejection and Robustness,” IEEE Trans. on Automatic Control, Vol. 37, No. 11, pp. 1734-1745, November, 1992.

Legacy of Mohammed Dahleh

13

17. H. Chapellat, M. Dahleh, and S. P. Bhattacharyya, “Robust Stability Manifolds for Multilinear Interval Systems,” IEEE Trans. on Automatic Control, Vol. 38, No. 2, pp. 314-318, February, 1993. 18. W. S. Warren, H. Rabitz, and M. Dahleh, “Coherent Control of Quantum Dynamics: The Dream is Alive,” Science, Vol. 259, No. 5101, pp. 1581-1589, March 1993. 19. M. Dahleh, A. Tesi, and A. Vicino, “Robust Stability and Performance of Interval Plants,” Systems and Control Letters, Vol. 19, No. 4. , pp. 353-363, November, 1992. 20. M. Dahleh, A. Tesi, and A. Vicino, “An Overview of Extremal Properties for Robust Control of Interval Plants,” Automatica, Vol. 29, No. 3, pp. 707-721, May, 1993. 21. M. Dahleh, A. Tesi, and A. Vicino, “On the Robust Popov Criterion for Interval Lur’e Systems,” IEEE Trans. on Automatic Control, Vol. 38, No. 9, pp. 14001405, September, 1993. 22. K. L. Moore, S. P. Bhattacharyya, M. Dahleh, “Capabilities and Limitations of Multirate Control Schemes,” Automatica, Vol. 29, No. 4, pp. 941-951, July 1993. 23. H. Chapellat, M. Dahleh, and S. P. Bhattacharyya, “Structure and Optimality of Multivariable Periodic Controllers,” IEEE Trans. on Automatic Control, Vol. 38, No. 8, pp. 1300-1303, August 1993. 24. B. Bamieh, and M. Dahleh, “On Robust Stability with Structured TimeInvariant Perturbations,” Systems and Control Letters, Vol. 21, No. 2, pp. 103108, August 1993. 25. M. Dahleh, M. Ricci, “On the SPR Condition in Output Error Estimation Schemes,” ASME Journal of Dynamic Systems, Measurement, and Control, Vol. 115, No. 4, pp. 704-708, December, 1993. 26. V. Ramakrishna, M. Salapaka, M. Dahleh, H. Rabitz, and A. Peirce, “Controllability of Molecular Systems,” Physical Review A, Vol. 51, No. 2, pp. 960-966, February, 1995. 27. M. Dahleh, A. Peirce, H. Rabitz, and V. Ramakrishna, “Control of Molecular Motion,” Proceedings of the IEEE, Vol. 84, No. 1, pp. 7-15, January 1996. 28. M. Dahleh, “On Robust Stability and Performance with Time-Varying Control,” ASME Journal of Dynamic Systems Measurement and Control, Vol. 117, No. 4, pp. 635-637, December 1995. 29. M. Dahleh, A. Megretski, and B. Bamieh, “On `∞ Robust Stability and Performance with `2 and `∞ Perturbations,” Systems and Control Letters, Vol. 28, pp. 1-6, 1996. 30. M. Salapaka, P. Voulgaris, and M. Dahleh, “ Controller Design to Optimize a Composite Performance Measure,” Journal of Optimization Theory and Applications, Vol. 91, No. 1, pp. 91-113, October 1996. 31. M. Salapaka, P. Voulgaris, and M. Dahleh, “SISO Controller Design To Minimize A Positive Combination of the `1 and H2 Norms,” Automatica, Vol. 33, No. 3, pp. 387-391, 1997. 32. M. Salapaka, M. Dahleh, and P. Voulgaris,“Mixed Objective Control Synthesis: Optimal `1 /H2 Control,” SIAM Journal on Control and Optimization, Vol. 35, No. 5, pp. 1672-1689, 1997. 33. B. Bamieh, and M. Dahleh, “On Robust Performance in H∞ ”, Systems and Control Letters, Vol. 33, pp. 301-305, 1998.

14

MuntherÿA. Dahleh

34. M. Salapaka, M. Dahleh, and P. Voulgaris, “MIMO Optimal Control Design: the Interplay between the H2 and `1 Norms,” IEEE Trans. on Automatic Control, Vol. 43, No. 10, pp. 1374-1388, October, 1998. 35. D. D’Alessandro, M. Dahleh, I. Mezi´c, “Control of Mixing in Fluid Flow: a Maximum Entropy Appoach, IEEE Trans. Automat. Control, Vol. 44, No. 10, pp. 1852-1863, October, 1999. 36. M. Napoli, B. Bamieh, M. Dahleh , “Optimal control of arrays of microcantilevers”, J DYN SYST-T ASME, 121 (4): 686-690, December, 1999. 37. M. V. Salapaka, M. Khammash, and M. Dahleh, “Solution of MIMO H2 /`1 Problem without Zero Interpolation,” SIAM Journal on Control and Optimization, Vol. 37, No. 6, pp. 1865-1873, 1999. 38. M. Ashhab, M. V. Salapaka, M. Dahleh, and I. Mezi´c, ”Melnikov-based dynamical analysis of microcantilevers in scanning probe microscopy”, Nonlinear Dynamics, 20: (3), pp. 197-220, November 1999. 39. M. Ashhab, M. V. Salapaka, M. Dahleh, and I. Mezi´c, ”Dynamical analysis and control of micro-cantilevers”, Automatica, Vol. 35, no. 10, 1663-1670 October 1999 40. M. Basso, L. Giarr´e, M. Dahleh and I. Mezi´c , “Complex dynamics in a harmonically excited Lennard-Jones oscillator: Microcantilever-sample interaction in scanning probe microscopes,” ASME Journal of Dynamics Systems, Measurement and Control, vol. 122, pp. 240-245, 2000. 41. S. Salapaka, M. Dahleh and I. Mezi´c, On the dynamics of a harmonic oscillator undergoing impacts with a vibrating platform Nonlinear Dynamics, Vol 24, Pages 333-358, 2001 42. Bamieh B, Dahleh M, “Energy amplification in channel flows with stochastic excitation”, Phisics Fluids 13 (11): 3258-3269, 2001.

A Control Theoretical Look at Internet Congestion Control Fernando Paganini1 , John Doyle2 , and Steven H. Low2 1 2

UCLA Electrical Engineering, Los Angeles, CA 90095, USA California Institute of Technology, Pasadena, CA 91125, USA

Abstract. Congestion control mechanisms in today’s Internet represent perhaps the largest artificial feedback system ever deployed, and yet one that has evolved mostly outside the scope of control theory. This can be explained by the tight constraints of decentralization and simplicity of implementation in this problem, which would appear to rule out most mathematically-based designs. Nevertheless, a recently developed framework based on fluid flow models has allowed for a belated injection of control theory into the area, with some pleasant surprises. As described in this chapter, there is enough special structure to allow us to “guess” designs with mathematically provable properties that hold in arbitrary networks, and which involve a modest complexity in implementation.

1

Introduction

At the heart of today’s Internet lies a feedback system, in charge of managing the allocation of bandwidth resources between competing traffic streams. In contrast to the telephony network where resources are allocated by the network core at call admission time, the Internet’s resources are allocated in real-time, mainly by the end systems themselves. This solution is motivated by the desire to accommodate widely heterogeneous demands, from “mice” made of a few packets, to long “elephants” greedy for whatever bandwidth is available, and to avoid the complexity of a centralized mechanism for bandwidth distribution. The fact that end-systems must control their throughput with little information about the overall network necessitates the use of feedback; such mechanisms have been incorporated since the late 1980s [5] into the transport (TCP) layer of the Internet protocol stack. For a survey of these algorithms, see [12]. While the feedback component has significant performance implications, it was historically designed by computer scientists working largely outside the orbit of feedback control theory. This can be explained in part by the cultural distance between mathematical theory and the desire for simplicity of Internet engineers. There is, however, a more fundamental reason that stems from the Internet design principle [3] of keeping the network simple, and moving complexity to the end systems. In the congestion control problem, this creates a radically decentralized, yet highly coupled feedback system, for which control theory has little to offer in terms of systematic design. Consequently, most contributions to congestion control from the control community (e.g. [1,16,14]) have focused on problems under centralized

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 17--31, 2003 Springer-Verlag Berlin Heidelberg

18

Fernando Paganini et al.

information which are relevant to other network scenarios (e.g., ATM), but have limited bearing on the Internet case. Given the apparent success of the Internet in satisfying its demands, one might wonder about the relevance of mathematical theory to this endeavor: maybe this “hacked” system has managed to solve the problem. There are, however, deficiencies of the current solutions that have serious impact in the further scalability of the network, and which have proven difficult to address without the aid of mathematical tools. A first issue concerns understanding, and potentially improving, the resource allocation equilibrium that results from current TCP, and avoiding some of its undesirable side-effects, such as induced queueing delays. There are also dynamic limitations: algorithms tuned to react quickly to changing conditions have often been found to produce dramatic oscillations. In the last few years, significant progress has been made in the theoretical understanding of both these issues, following seminal work by Kelly and coworkers [7,8] (for more references see [12]). Key to these advances is to work at the correct level of aggregation (namely, fluid flow models), and to explicitly model the congestion measure fed back to sources from congested links. In practice this measure can correspond to packet loss probability, or queueing delay, depending on the protocol variant. Interpreting such signals as prices has allowed for economic interpretations [8,10] that make explicit the equilibrium resource allocation policy specified by the control algorithms. Congestion measures allow also for dynamic models of TCP, that have been successful in matching empirical observations on oscillatory behavior [15,11]. In particular, these models predict that oscillatory instabilities will become more prevalent as network capacity scales up, if protocols are left unchanged. The availability of mathematical models now stimulate the following question: how much could control engineers improve on these systems if we were to “do it all again”? Given the decentralized information structure and other tight implementation constraints, the prospect does not look easy: nevertheless, it turns out there is enough structure in this problem to allow for mathematical “hacks” with provable properties of stability and scalability. This chapter describes one of these solutions.

2 2.1

Problem Formulation Fluid flow models

The starting point of our analysis will be a flow-level abstraction of the TCP congestion control problem. Here, each of the traffic-sources i which share the network has an associated rate xi , and these rates get aggregated in accordance to their particular routing into flows yl at each network link l, which in turn has a capacity cl . All these real-valued quantities are in packets/second. To put this abstraction into context it is worth looking at the actual network in closer detail. TCP sources send individual packets across the network to their destinations, and receive from them an acknowledgement (ACK) packet, which serves as confirmation of correct reception and is also used for timing the following transmission. Sources maintain a congestion window variable w that determines how many packets can be sent before receiving an ACK; in this way, the transmission

Internet Congestion Control

19

rate of the source is roughly x≈

w , τ

(1)

800

800

700

700

600

600

Instaneous queue (pkts)

Instaneous queue(ms)

where τ is the round-trip-time (RTT) of the communication. Clearly, the above approximation can only have meaning at longer time-scales than the RTT, and ignores all the complexity of individual packet arrival times. Contrast this with the viewpoint of queueing theory: here the packet is the essential unit, and stochastic models are used to characterize inter-arrival times, which are then used to find probability distributions of relevant quantities such as network queues. This viewpoint is in fact so ingrained that the word “randomness” would commonly be used in place of “complexity” at the end of the previous paragraph, and the fluid approximation would be presented as a first-moment analysis of the probability distributions. Note, however, that when the traffic sources remain fixed, their packet transmission times are deterministically “clocked” by the ACK process, whose complexity depends only on issues like initial ordering in queues. This is very different from the traditional abstraction of individual customers arriving at a queue following e.g. a renewal process (which could apply naturally to the arrival of new TCP sessions) so it is unclear that a stochastic model can give an accurate characterization at a finer scale than the rate abstraction.

500

400

300

200

400

300

200

100

0

500

100

0

5

10

15

20 time(s)

25

30

35

40

0

0

5

10

15

20

25

30

35

40

time(s)

Fig. 1. Simulation example: queue oscillations in TCP/RED. Left plot: controlled sources only. Right plot: 30 % “noise” sources. Fortunately, recent research has shown that fluid flow models have substantial predictive power, particularly in regard to large-scale questions such as the achieved equilibrium rates and the stability of the dynamics. For instance, Figure 1 from [11] contains results from packet-level simulations of the standard TCP protocol combined with the RED queue management scheme [4]. The left figure shows an essentially periodic oscillation of the queue of backlogged packets over time, which can in fact be explained [15,11] as a limit cycle oscillation in fluid flow models of the type we consider here. Note that in steady state there is little “randomness” observed from packet effects in these controlled sources. In the simulation on the right, randomness was deliberately added at the level of uncontrolled TCP sessions that come and go, generating 30 % of the traffic. Here there queue plot is noisy, but the dynamics are still dominated by the limit cycle instability. Still, there is one issue that is not trivially resolved when ignoring packet-level effects: what is the adequate fluid-flow model of a queue? Thinking in terms of

20

Fernando Paganini et al.

actual fluids and buffers as “tanks”, a natural choice is to write ½ yl − cl , if bl > 0 or yl > cl ; b˙ l = 0 otherwise;

(2)

where bl is the queue backlog. Namely, bl integrates the excess rate over capacity, and is saturated to be non-negative. This model is successful in predicting slow, deterministic phenomena like the oscillations of Figure 1. An alternative viewpoint is to say that nonzero queues build up due to packet randomness even before yl reaches cl , and use queueing theory formulas of the form bl = f (yl , cl ) relating expected queues to e.g. Poisson rates. From a dynamic point of view, a static function is very different from the integrator in (2), so both models could lead to very different predictions. Note, however, that these static formulas apply only to steady-state; an improvement based on approximate transient analysis of M/M/1 queues was recently done in [18], yielding an interpolation between the two types of models. It must, however, rely on the above traffic model which is hard to justify in the context of controlled TCP sources. A pragmatic solution to this modeling difficulty is to avoid giving network queues a key role in congestion feedback. This is also consistent with the objective, described later, of decoupling feedback from queueing. Below, we will base our congestion signals on a virtual queue which by construction can be made to operate fully in the integrator regime.

2.2

The congestion control loop

We return now to specifying the model in more detail. The link rates are modeled by X Rli xi (t − τlif ), (3) yl (t) = i

in which the forward transmission delays τlif between sources at links are accounted for, and the routing matrix R is defined by ½ 1 if link l belongs to source i’s route . Rli = 0 otherwise The next step is to model the feedback mechanism which communicates to sources the congestion information about the network. The key idea associate with each link l a congestion measure pl (t), which is a positive real-valued quantity. Due to its economic interpretations we will call this variable a “price” associated with using link l. The fundamental assumption we make is that sources have access to the aggregate price of all links in their route, X Rli pl (t − τlib ). (4) qi (t) = l

Here again we allow for backward delays τlib in the feedback path. As discussed in [12], such model can be used to approximate, at a fluid level, the feedback mechanism in existing protocols. The total RTT by source is given by b f + τi,l ; τi = τi,l

(5)

Internet Congestion Control

21

this quantity is available to sources in real time. Using vector notation c, y, p, x, q to collect the above variables across links or sources, we reach the following network model in the Laplace domain: y(s) = Rf (s)x(s),

(6)

T

q(s) = Rb (s) p(s).

(7)

Here T denotes transpose, and Rf and Rb are the delayed forward and backward routing matrices, obtained by replacing the “1” elements of the matrix R respecf

−τi,l s

tively by the pure delay terms e

x

b

, e−τi,l s .

Rf (s)

y

0 SOURCES

.. 0

0 ..

.

.

LINKS

0 q

Rb (s)T

p

Fig. 2. General congestion control structure. Figure 2 represents the resulting congestion control feedback loop. Tacitly assumed in the development is that both the routing and the sources participating in the feedback, remain fixed. In practice, routing usually varies at a slower time-scale, and we are focusing on the control of “elephant” TCP flows that last long enough to be controlled; the only way to model short “mice” would be as additive noise. What remains to be specified is: (i) How the links fix their prices based on link utilization; (ii) how the sources fix their rates based on their aggregate price. These operations are up to the designer, but have a main restriction: both must be decentralized, as indicated in the figure by the block-diagonal structure. For instance the source rate xi can only depend on the corresponding aggregate price qi .

2.3

Control objectives

The objective of this feedback is for source rates to converge, as quickly as possible to an equilibrium point x0 , y0 , p0 , q0 that satisfies some desired static properties. More specifically, we lay out the following design objectives: 1. Network utilization. Link equilibrium rates y0l should of course not exceed the capacity cl , but also should attempt to track it. Clearly, there may be some bottleneck links that prevent others from being at capacity, but at least one bottleneck for each source should be at almost full capacity.

22

Fernando Paganini et al.

2. Empty equilibrium queues. In this way we avoid queueing delays, which are particularly relevant for uncontrolled “mice” that share the network with our controlled sources. 3. Resource allocation. We will assume sources have a demand curve x0i = fi (q0i )

(8)

that specifies their desired equilibrium rate as a decreasing function of price. This is equivalent to assigning them a utility function Ui (xi ), in the language of [8]; in this case fi = (Ui0 )−1 . We would like the control system to reach an equilibrium that accommodates these demands. This does not in itself ensure “fairness”, but provides a tuning knob in which to address these issues; for more discussion see [8]. 4. Dynamic asymptotic stability. We aim at achieving these objectives for an arbitrary choice of network: topology, routing, and parameters such as link capacities and round trip times. Here lies the biggest challenge for design. Based on historical experience, it appears that network engineers rank the above objectives roughly in decreasing order. High utilization is a feature of protocols since TCP-Reno [5], and efforts at reducing queueing delay have come later [4]; as of today, TCP has no mechanism for influencing the resource allocation policy. As for stability, window-based protocols have built-in boundedness due to conservation of packets, but oscillatory behavior as in Figure 1 does not create the alarm it would cause in other control engineering domains. Perhaps due to these priorities, initial analytical work in [8,10] developed control laws guided mainly by equilibrium considerations, and only considered dynamic aspects after the fact. It is, however, very difficult to satisfy stability restrictions in this way, and one ends up having to make parameter choices which are very conservative in terms of dynamic response. Due to this difficulty (and not because of a change in priorities), we will address the stability question from early on in the design, attempting to negotiate the equilibrium objectives under this restriction.

3

Control design with linear scalable stability

We will design nonlinear control laws at sources and links that are meant to operate universally across networks; in each case, they will result in an equilibrium point x0 , y0 , p0 , q0 , and determine the dynamics around it. The objective of obtaining a stable equilibrium in every case means that the system must “schedule its gains” automatically; this severely narrows the family of suitable laws, a fact we will exploit in our search. Consider first the objective of link utilization: we can use the principle of integral control to impose that the equilibrium rates y0l track a target capacity c0l ; namely, writing the price dynamics ½ µl (yl − c0l ), if pl > 0 or yl > c0l ; (9) p˙ l = 0 otherwise, where µl is a constant. This law is of the type considered in [10]. Comparing to (2), we see that if c0l = cl , prices would be proportional to queue backlogs; given our

Internet Congestion Control

23

second objective of eliminating the latter in equilibrium, we will choose c0l to be slightly smaller than capacity (a “virtual” capacity, see [9]). If this system reaches equilibrium, bottlenecks with nonzero price will have y0l = c0l , and non-bottlenecks with y0l < cl will have zero price. This ensures every source will see a bottleneck, unless its own maximum demand is insufficient to fill it. To guide our search for the source control law, we will impose the requirement that the closed loop must be locally stable for arbitrary networks and delays. To begin, consider a single link, running (9), and a single source, with the linearized static control law (between incremental quantities) δx = −κδq, combined through the delay e−τ s . It is easily seen that this loop would be unstable for large τ , unless κ compensates for it. Fortunately, sources can measure their RTT , which gives a loop transfer function so we can set1 κ = α τ L(s) = αµ

e−τ s . τs

(10)

We call the above expression, with the frequency variable scaled by τ scale-invariant: this means that Nyquist plots for all values of τ would fall on a single curve Γ , depicted below for αµ = 1. In the time domain, closed loop responses for different τ ’s would be the same except for time-scale. 1

0.5

X

0

−0.5

−1

−1.5

−2 −1.5

−1

−0.5

0

0.5

jθ

Fig. 3. Nyquist plot Γ of e /jθ. Since Γ touches the negative real axis at the point −2/π, we see that our loop achieves scalable stability for all τ provided that the gain αµ < π/2. For a single link/source, the above gain condition could be imposed a priori. Suppose, however, that we have N identical sources sharing a bottleneck link. It is not difficult to see that the effective loop gain is scaled up by N ; this must be compensated for if we want stability, but in these networks neither sources nor links know what N is: how can they do the right “gain-scheduling”? 1

In fact, this compensation is implicit in any window protocol due to (1).

24

Fernando Paganini et al. The key idea in our solution is to exploit the conservation law c0l =

P i

x0i

1 implicit in the network equilibrium point, by choosing µl = c0l at each link, and a

gain x0i at each source, in addition to the 1/τi factor. In the case of a single link, but now many sources with heterogeneous delays, this gives a loop transfer function of L(jω) =

X x0i e−jτi ω , cl τi ω i

which is a convex combination of points in Γ . It follows that this convex combination will remain stable by a Nyquist argument. Will this strategy work if there are multiple bottleneck links contributing to the feedback? Intuitively, there could be an analogous increase in gain that must be compensated for. Therefore we introduce a gain M1i at each source, Mi being a bound on the number of bottleneck links in the source’s path, which we assume is available (see Section 5). This leads to a local source controller δxi = −κi δqi = −

αi x0i δqi , Mi τi

(11)

where αi < π/2 is a parameter. For this basic source controller, we will prove linear stability for an arbitrary network.

3.1

Linear stability result

Consider a small perturbation around equilibrium in the equations (6-7): x = x0+ δx, y = y0 + δy, p = p0 + δp, q = q0 + δq. Assuming the set of bottlenecks is unchanged by this perturbation, δpl is only non-zero for bottleneck links. Therefore for the local analysis to follow, we write the reduced model ¯ f (s)δx(s), δ y¯(s) = R ¯ b (s)T δ p¯(s), δq(s) = R

(12) (13)

¯ b , and the vectors δ p¯, δ y¯ are obtained by eliminating ¯f , R where the matrices R the rows corresponding to non-bottleneck links. We will assume that after this row ¯ b (0) is of full row rank, ¯ := R ¯ f (0) = R elimination, the resulting static matrix R which appears to be a generic assumption. With the source and link controllers described above, we have an open loop return ratio of the overall system given by ¯ bT (s)C I , ¯ f (s)KR L(s) = R s

(14)

where the rightmost matrix of integrators has the dimension of the number of links, and K = diag(κi ),

C = diag(

1 ). c0l

Note that there are no unstable pole/zero cancellations within L(s); the proposition below provides stability conditions for such multivariable loops with integral control. It can be established with elementary tools.

Internet Congestion Control

25

I

Proposition 1. Consider a standard unity feedback loop, with L(s) = γF (s) . s Suppose: ÿ(i) F (s) is analytic in Re(s) > 0 and bounded in Re(s) ≥ 0. ÿ(ii) F (0) has strictly positive eigenvalues. ÿ(iii) For all γ ∈ (0, 1], −1 is not an eigenvalue of L(jω), ω 6= 0. Then the closed loop is stable for all γ ∈ (0, 1]. In essence, the above conditions are a “nominal” stability requirement for small γ, that says that we have strictly negative feedback of enough rank to stabilize all the integrators, and a “robustness” argument that says we can perform a homotopy to γ = 1 without bifurcating into instability. ¯ bT (s)C; we will later ¯ f (s)KR Applying this to the L(s) in (14), we take F (s) = R add the scaling γ. Note that (i) is automatically satisfied. Since 1 ¯ R ¯ T C 12 ), eig(F (0)) = eig(C 2 RK

¯ has full row rank. Here we see the importance of condition (ii) holds provided R putting the integrators at the links (the lower dimensional portion). If, instead, we tried to integrate at the sources, the resulting feedback matrix at DC would not have enough rank to stabilize the larger number of integrators. What remains is to establish (iii). The key structure we will exploit in this problem is the equation ¯ f (−s)diag(e−τi s ), ¯ b (s) = R R ¯ bT (s) = diag(e−τi s )R ¯ f∼ (s), where which follows from (5), and allows us to write R ¯ f∼ (s) = R ¯ fT (−s) is the adjoint system. Bringing in the notation R X0 = diag(x0i ),

M = diag(

1 ), Mi

Λ(s) = diag(λi (s)),

λi (s) =

αi e−τi s , τi s

we can now rewrite L(s), for s 6= 0, as ¯ f∼ (s)C. ¯ f (s)X0 MΛ(s)R L(s) = R

(15)

We now tackle the robustness argument. Result 1. Consider an equilibrium point where rates match target capacity, i.e. ¯ f (0)x0 . Let αi < π and the delays be arbitrary. Then with L(s) as in (15), c0 = R 2 −1 6∈ eig(L(jω)), ω = 6 0. Proof: Since nonzero eigenvalues are invariant under commutation, and also many of the factors in (15) are diagonal, we observe that ³ ´ ³ ´ −1 ∈ eig L(jω) ⇐⇒ −1 ∈ eig P (jω)Λ(jω) , 1

1

1 ¯ f (jω)∗ C R ¯ f (jω)X 2 M 12 ≥ 0. P (jω) := M 2 X02 R 0

Claim: 0 ≤ P ≤ I.

(16)

26

Fernando Paganini et al.

This amounts to bounding the spectral radius ´ ³ ¯ f (jω)∗ k · kC R ¯ f (jω)X0 k. ¯ f (jω)X0 ≤ kMR ¯ f (jω)∗ C R ρ(P ) = ρ MR Any induced norm will do, but if we use the l∞ -induced (max-row-sum) norm, we find that X f 1 X −τ jω ¯ f (jω)X0 k∞−ind = max 1 |e i,l x0i | = max x0i = 1; kC R l l c0l c0l i uses l i uses l ¯ f∗ k = 1, because each row contains note we are dealing with bottlenecks. Also kMR exactly Mi elements of magnitude 1/Mi . So ρ(P ) ≤ 1 as claimed. Indeed, ρ(P ) = 1 at ω = 0, the eigenvector being the vector of all ones. Now suppose −1 ∈ eig(P (jω)Λ(jω)) for some ω. We thus have a vector u, |u| = 1 such that y = Λu, u = −P y. Now X λi |ui |2 u∗ y = u∗ Λu = i

is a convex combination of the {λi }, which are points in the curve Γ of Figure 3, scaled by αi < π2 . It is clear that such convex combinations and scaling cannot reach any point in the half-line (−∞, −1]. However, we also have 1 + u∗ y = u∗ u + u∗ y = y ∗ P (P − I)y ≤ 0, using (16). So u∗ y ∈ (−∞, −1], a contradiction. Remark 1. Some elements of the proof, in particular the use of l∞ induced norms to prove a spectral radius bound, are inspired by the work of [6] for the control laws in [8]. More recently [17] has extended the stability argument for the laws in [8] in a parallel fashion to our work. Theorem 1 establishes (iii) in Proposition 1; note that scaling down by γ is equivalent to making the αi smaller. To summarize, we have: ¯ f (s), R ¯ b (s) denote the routing matrices of sources in relation to Result 2. Let R ¯ b (0) has full row rank, and that αi < π . ¯ f (0) = R the bottleneck links. Suppose R 2 Then the system with link control (9) and linearized source control (11) is locally stable for arbitrary delays and link capacities. Our stability theorem covers the simplest possible control laws consistent with our utilization requirement, namely integrators at links and static gains at sources. Could the argument be generalized to include additional dynamics? We give the following observations: • Clearly one could include a fixed stable, inversely stable filter at all links, and its inverse at all sources, but this would have to be universally chosen. • There can be no more pure integrators. Otherwise the Nyquist plot in Figure 3 would branch towards −∞, and convex combinations of such points could reach the critical point. In particular, the strategy in [2] of adding another integrator to clear link queues would not qualify. • The source controller could include additional scalable dynamics, function of τi s; this would result in a modified Nyquist curve Γ , which is acceptable as long as its convex hull does not touch the critical point.

Internet Congestion Control

4 4.1

27

Nonlinear laws and the equilibrium structure Static source laws with scalable stability

We have provided the global law (9) with µl = c10l for price generation at the links, but so far we have only characterized sources by their linearization (11). For static source control laws, however, specifying its linearization at every equilibrium point essentially determines its nonlinear structure. Consider a static source control of the form xi = fi (qi , τi , Mi ). The linearization requirement (11) imposes that αi fi ∂fi =− , ∂qi Mi τi for some 0 < αi < π/2. Let us assume initially that αi is constant. Then the above differential equation can be solved analytically, and gives the static source control law xi = fi (qi , τi , Mi ) = xmax,i

αi qi −M i τi

e

.

(17)

Here xmax,i is a maximum rate parameter, which can vary for each source, and can also depend on Mi , τi (but not on qi ). This exponential backoff of source rates as a function of aggregate price can provide the desired control law, together with the link control in (9). We can achieve more freedom in the control law by letting the parameter αi be a function of the operating point: in general, we would allow any mapping xi = fi (qi ) that satisfies the differential inequality 0≥

π fi ∂fi ≥− . ∂qi 2 Mi τi

(18)

The essential requirement is that the slope of the source rate function (the “elasticity” in source demand) decreases with delay τi , and with the number of bottlenecks Mi . So we find that in order to obtain this very general scalable stability theorem, some restrictions apply to the sources’ demand curves (or their utility functions). This is undesirable from the point of view of our objective 3 in Section 2.3; we would prefer to leave the utility functions completely up to the sources; in particular, to have the ability to allocate equilibrium rates independently of the RTT. We remark that parallel work in [18] has derived solutions with scalable stability and arbitrary utility functions, but where the link utilization requirement is relaxed. Indeed, it appears that one must choose between the equilibrium conditions on either the source or on the link side, if one desires a scalable stability theorem. Below we show how this difficulty is overcome if we slightly relax our scalability requirement.

4.2

A lead-lag alternative for source control

The reason we are getting restrictions on source utility is that for static laws, the elasticity of the demand curve (the control gain at DC) coincides with the high frequency gain, and is thus constrained by stability. One way of decoupling the two

28

Fernando Paganini et al.

gains is to replace the source control by a dynamic, lead-lag compensation of the form δxi = −ϕi (s)δqi = −

κi (s + z) δqi . i s + zκ νi

(19)

Here the high frequency gain κi is the same as in (11), “socially acceptable” from a dynamic perspective. The DC gain νi = −fi0 (qi0 ) is the elasticity of source demand based on its own “selfish” demand curve xi0 = fi (qi0 ), that need no longer be of the form (17). The zero z is assumed fixed across sources. Can a stability theorem be obtained under these new laws? The main requirement would be that at cross-over frequency all sources respond according to their high-frequency gain, so that the previous analysis applies. The difficulty is that this implies a common agreement on the frequency scale, which means forgoing complete scalability with respect to time delay. While less elegant, this is not too serious in practice, where one can assume a known bound on the network’s RTT. We have the following result. Proposition 2. Assume that for every source i, τi ≤ τ¯. In the assumptions of Theorem 2 replace the source control by (19), with αi = α < π2 and a z = τη¯ . Then for a small enough η ∈ (0, 1) depending only on α, the closed loop is linearly stable. We defer the proof to [13], but remark that it is based again on a Nyquist argument via the eigenvalues of the loop transfer function L(jω); a perturbed version of the argument in Theorem 1 is used at frequencies above τ1¯ , and a different argument at low frequencies; the fact that the source zero z is fixed across sources is essential to this decomposition.

4.3

Nonlinear implementation of dynamic source laws

We wish to find a source control law whose equilibrium matches the desired utility function, Ui0 (x0i ) = q0i (equivalently, the demand curve xi0 = fi (qi0 )), and with linearization (19). This is not as easy as before, especially due to the requirement of fixing the zero z independently of the operating point and the RTT. The solution below is based on a slower state that follows dynamics of the type in the “primal” laws in [8,6,17]. τi ξ˙i = βi (Ui0 (xi ) − qi ),

αq

(ξi − Mi τi ) i i .

xi = xm,i e

(20) (21)

Note that (21) corresponds exactly to the rate control law in (17), with the change that the parameter xmax is now varied exponentially as xmax,i = xm,i eξi , with ξi as in (20). If βi is small, the intuition is that the sources use (17) at fast timescales, but slowly adapt their xmaxi to achieve an equilibrium rate that matches their utility function, as follows clearly from equation (20).

Internet Congestion Control

29

We now find the linearization around equilibrium; the source subscript i is omitted for brevity. For increments ξ = ξ0 + δξ, x = x0 + δx, q = q0 + δq, we obtain the linearized equations: ¢ ¡ δx ¢ ¡ − δq , τ δ ξ˙ = β U 00 (x0 )δx − δq = β − ν α δx = x0 (δξ − δq) = x0 δξ − κδq. Mτ 1 Here we have used the fact that U 00 (x0 ) = f 0 (q = − ν1 , and the expression (11) for 0) κ . Some algebra in the Laplace domain leads to the transfer function ! Ã 0 s + βx κτ δq, δx = −κ 0 s + βx ντ

that is exactly of the form in (19) if we take z=

βx0 βM = . κτ α

By choosing β, the zero of our lead-lag can be made independent of the operating point, or the delay, as desired. We recapitulate the main result as follows. Result 3. Consider the source control (20-21) where Ui (xi ) is the source utility function, and the link control (9). At equilibrium, this system will satisfy the desired demand curve xi0 = fi (qi0 ), and the bottleneck links will satisfy y0l = c0l , with empty queues. Furthermore, under the rank assumption in Theorem 2, αi < π2 , and i z = βiαM chosen as in Proposition 2, the equilibrium point will be locally stable. i We have thus satisfied all the objectives set forth in Section 2.3, except for the fact that an overall bound on the RTT had to be imposed.

5

Signaling requirements

We briefly discuss here the information needed at sources and links to implement our dynamic laws, and the resulting communication requirements. Links generate prices by integrating the excess flow yl − c0l with respect to the virtual capacity; this is easily implemented by maintaining a “virtual queue” variable, incremented upon packet arrival, and decremented at the virtual capacity rate. True bottlenecks will operate away from saturation, so the integrator model (2) for this queue is justified. The resulting price must be communicated to sources in additive way across links. For this purpose we can employ the Explicit Congestion Notification bit available in the packet header, and the technique of random exponential marking [2]: here the bit would be marked at link l with probability 1 − φ−pl , where φ > 1 is a global constant. Assuming independence, the overall probability that a packet from source i gets marked is (see [2]) 1 − φ−qi ,

30

Fernando Paganini et al.

and therefore qi can be estimated from marking statistics. Note that the estimation process will add noise, and additional delay in the feedback loop. The latter can be accounted for in the source compensation. Sources must have access to the round-trip time τi , which can be obtained by timing packets and their acknowledgments. They also need the bound Mi on the number of bottlenecks, which is not so easy to obtain, although it can be argued that in practice this number is typically not large (e.g. 2 bottlenecks per source). Alternatively, one could think of using another ECN bit to communicate this information. Once a rate is computed by the source, (1) can be used to set the congestion window. An initial implementation of such protocol has been programmed in the standard simulator ns-2 [13]; while validation work is in process, early results are encouraging.

6

Conclusion

The abstraction of fluid-flow models has allowed us to cast the congestion control problem in the familiar language of linear multivariable control. Although, due to decentralization, feedback design can only be handled in an ad-hoc way, we have found that the special structure of the problem allows us to go after a very ambitious objective: scalable local stability for arbitrary networks and delays, together with tracking of link utilization. When in addition we want to give sources the freedom of choosing their rate demand curves, we found a solution based on separation of time-scales assuming a known bound on the round-trip times. Going from flow models and theorems to actual protocols based on packet level mechanisms, requires of course a layer of “hacks” and experimentation. Whether this transition will eventually yield viable new protocols will depend on engineering aspects which are mostly outside the scope of the theory; for instance, the restriction of one ECN bit per packet, or the issue of incremental deployment of these protocols in the current network. Regardless of this outcome, it is reassuring to discover that control theory is still relevant in the world of complex networks.

References 1. E. Altman, T. Basar and R. Srikant. “Congestion control as a stochastic control problem with action delays”, Automatica, December 1999. 2. Sanjeewa Athuraliya, Victor H. Li, Steven H. Low, and Qinghe Yin, “REM: active queue management,” IEEE Network, vol. 15, no. 3, pp. 48–53, May/June 2001. 3. D.D. Clark, “The design philosophy of the DARPA Internet protocols”, Proc. ACM SIGCOMM ’88, in: ACM Computer Communication Reviews, Vol. 18, No 4., pp. 106-114, 1988. 4. S. Floyd and V. Jacobson, “Random early detection gateways for congestion avoidance”, IEEE/ACM Trans. on Networking, 1(4):397-413, Aug. 1993. 5. V. Jacobson, “Congestion avoidance and control”, Proc. ACM SIGCOMM ’88. 6. R. Johari and D. Tan, “End-to-End Congestion Control for the Internet: Delays and Stability”, Cambridge Univ. Statistical Lab. Research Report 2000-2.

Internet Congestion Control

31

7. F. P. Kelly, “Mathematical modeling of the Internet”, Fourth International Congress on Industrial and Applied Mathematics, Edinburgh, Scotland, July 1999. 8. F. P. Kelly, A. Maulloo, and D. Tan, “Rate control for communication networks: Shadow prices, proportional fairness, and stability”. Jour. Oper. Res. Soc., 49(3), pp. 237-252, 1998. 9. S. Kunniyur and R. Srikant, ”Analysis and Design of an Adaptive Virtual Queue (AVQ) Algorithm for Active Queue Management”, Sigcomm 2001, San Diego, Aug 2001. 10. S. H. Low and D. E. Lapsley, “Optimization flow control – I: basic algorithm and convergence” IEEE/ACM Trans. on Networking, Vol 7(6) Dec 1999. 11. S. H. Low, F. Paganini, J. Wang, S. A. Adlakha, and J. C. Doyle. “Dynamics of TCP/RED and a scalable control”, Proc. IEEE Infocom 2001,, New York. 12. Steven H. Low, Fernando Paganini, and John C. Doyle, “Internet congestion control” IEEE Control Systems Magazine, February 2002. 13. F. Paganini, Z. Wang, S. H. Low, and J. C. Doyle. “A new TCP/AQM for Stable Operation in Fast Networks” submitted. 14. S. Mascolo, “Congestion control in high-speed communication networks using the Smith principle”, Automatica, 1999. 15. V. Misra, W.-B Gong, and D. Towsley. “Fluid-based analysis of a network of AQM routers supporting TCP flows with an application to RED.” In Proceedings of ACM SIGCOMM, 2000. 16. H. Ozbay, S. Kalyanaraman, A. Iftar, “On rate-based congestion control in high-speed networks: design of an H∞ based flow controller for single bottleneck”, Proc. American Control Conference, 1998. 17. G. Vinnicombe, “On the stability of networks operating TCP-like congestion control”, to appear in 2002 IFAC World Congress, Barcelona, Spain. 18. G. Vinnicombe, “Robust congestion control for the Internet”, preprint, Feb 2002.

Indelible Control

?ÿ,??

Nicola Elia Iowa State University, Ames IA 50011, USA

Abstract. Indelible means robust to erasure. In this paper, we study the problem of controlling a system over communication channels that tend to erase the data they are supposed to carry. The largest probability of erasure tolerable by the closed loop is obtained by solving a robust control synthesis problem. In more general terms, we establish that the set of plants that can be stabilized over erasure channels, is fundamentally limited by the channel generated uncertainty.

Keywords: Quality of service, control over the internet, erasure channels, multiplicative noise, mean square stability robustness.

1

Introduction

The presence of communication channels between the plant and the controller brings up new questions in systems and information theory. To resolve these new issues, we need a deeper understanding of the interaction between control and information. The pragmatic approach of treating the control and the communication problems separately is naive and provides us neither with new insight nor with viable solutions to demanding applications where bandwidth is limited and the channels are shared among several users. A new effort is necessary for unifying the two theories, at least in the realm of real time feedback applications. The development of this integrated view is still in its infancy, but it has already generated a substantial body of research, ([1]-[12] this is a rather incomplete list). One important commonality between the systems that transmit information and those that control other systems, is that they have to work in spite of uncertainty. The capacity of a communication channel is a measure of how much the achievable information transmission is fundamentally limited by the channel uncertainty. Likewise, the robust stability/performance radii are also measures of how much the closed loop stability/performance are fundamentally limited by the uncertainty in the system. When a communication channel is present in a feedback loop, the controller must fight the uncertainty introduced by the channel, in order to achieve the performance or to maintain the stability of the closed loop. What ?

??

Thanks to John Doyle for suggesting this title appropriate for the content of this paper. I also feel it is appropriate for the occasion being the Memory of Mohammed Dahleh indelible indeed. This research has been supported by NSF under the Career Award grant number ECS-0093950

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 33--46, 2003 Springer-Verlag Berlin Heidelberg

34

Nicola Elia

2NCPV

%QOOWPKECVKQP %JCPPGN

0 0

%QOOWPKECVKQP %JCPPGN

%QPVTQNNGT Fig. 1. Feedback over Erasure Channels. are the fundamental limitations that the presence of a communication channel introduces in the closed loop system? In this paper, which is based on [11], we address this question by considering the problem of feedback Mean Square stabilization of a linear discrete-time system when a linear controller is connected to the plant via analog Erasure channels. An Erasure channel is modelled as a switch with Bernoulli distributed probability of been either open or closed at each sampling time. This abstraction captures the loss of data packets typical of the Internet channel. The probability of not loosing the data provides a measure of quality of the communication link. Binary Erasure channels have been considered among others in [7], with emphasis on the minimal bit/rate required for almost sure stability, and in [8] in the context of Mean Square stability of single state plants and in connection to the Anytime capacity. [13] has considered asynchronous, instead of probabilistic, switches, and exponential stability. In our set-up, finding the minimal channel quality required for the Mean Square stability of the closed loop system is equivalent to a robust control problem, where the nominal system model is deterministic and the structured model uncertainty is (linear) real parametric random time-varying. We characterize the robust Mean Square stability of the uncertain system in terms of the feasibility of Linear Matrix Inequalities and of a spectral radius condition. These results extend the one in [14,15] on the stability radius of systems with multiplicative noise. Related work in different settings has considered stability robustness of stochastic nominal plants with norm bounded uncertainties, either nonlinear memory-less, or linear dynamic and stochastic, [16,17]. In our setting instead, the nominal plant is deterministic, and the uncertainty is linear and only due to the stochastic variables, which are random time-varying gains. From the results in the paper, it emerges that the unstable poles and nonminimum phase zeros of the plant, which affect the fundamental limitation of feedback, also determine the required quality of the communication channel.

Indelible Control

2

35

Problem Set-up

Figure 1 describes the block diagram of interest. The plant is a linear discrete-time system governed by the following difference equation P lant :

x+ = Ax + Br v = Cx

(1)

where x ∈ Rn , x+ denotes the system state at the next discrete-time, A ∈ Rn×n , B ∈ Rn×l has full column rank, C ∈ Rq×n full row rank. The system is assumed unstable, but stabilizable, and detectable. In this paper, we further assume that the plant is Single Input Single Output, i.e., q = l = 1. The controller is also linear time-invariant. Its difference equations have the following expression. ˜ : K

˜K y˜ ζ + = A˜K ζ + B ˜ ˜ u ˜ = CK ζ + DK y˜

(2)

We will see that the order of the controller can be taken equal to the order of the plant.

2.1

The Communication Channels

We assume that the plant interacts with the controller through communication channels located at both input and output of the plant. As we will see, the presence of both channels significantly complicates the solution, with respect to the case most studied in the literature, which only consider the presence of one channel at either the plant input or output. The main feature that we care to model of the channels is that the data sent is lost with some probability. This is captured by the following simple definition. Definition 1 An analog (discrete-time) Erasure channel with probability of erasure e, is a mapping E : R→R E : w(k) → ξ(k)w(k) , where, ξ(k) is a discrete random variable, which can be equal to either 0 or 1 with the following probability mass function P r(ξ(k) = 1) = 1 − e P r(ξ(k) = 0) = e . ξ(0), ξ(1), . . . , ξ(k), . . . , are assumed to be independent, identically distributed random variables.

36

Nicola Elia

r

Plant

v

r

Plant

v

z2

w1

ξI

ξO ~ u

~ y

∆I z1

∆O u

~ K

a)

b)

... ∆p

z

P u

µ

~ K

w

w2

y

µ

∆1

K

y

K c)

G

Fig. 2. Transformation into the standard robust control diagram. According to this definition, the channel acts as an electric switch at each discretetime instance. When the switch is closed, the real-valued signal is transferred at the output of the channel with no error. When the switch is open, the output of the channel is zero, and the input value is lost. This model captures the feature of data packet loss typical of the internet channel. In turn, considering the data lost (or unusable) is an abstraction of the phenomenon of data being delivered with very long delay, so that using the late data brings marginal benefits with high processing cots. The approach proposed in this paper can also handle the case of random, finite delays with know distribution [12]. The simple channel model considered here allows us to analyze fundamental limitations of the linear feedback through the channels. Future research will address the issues related to digitalization and coding, which we neglect in here. For future reference, let the mean and variance of ξ(k) be denoted as follows. 4

µ = E{ξ(k)} = 1 − e 4

σ 2 = E{(ξ(k) − µ)2 } = e(1 − e) . The quality of an Erasure channel is defined as follows. Definition 2 The Quality of Service of an Erasure channel is Q = 1 − e. For simplicity , we assume in what follows that both input and output channels, E I , and E O have the same quality, in the sense that they are characterized by the same probability of erasure, e. This is not a restrictive assumption, and it can be removed. ξ I and ξ O are associated to E I , and E O respectively.

3

Channels as Model Uncertainty

The main idea of this section is to realize that the stochastic variables in the channel model are the only source of uncertainty in the closed loop system. 1 Let 1

Clearly, more general situations will also include deterministic uncertainties.

Indelible Control I

37

O

ξ I = µ(1 + ∆I ), and ξ O = µ(1 + ∆O ) where ∆I = ξ µ−µ and ∆O = ξ µ−µ . Following the transformations shown in Figure 2, we get the problem in the standard robust control framework, where the generalized plant P and controller K are purely deterministic LTI system and the uncertainty is diagonal composed by stochastic time varying independent gains with bounded variance. P , K, and G have the following state-space representation respectively:  A Ax + Bw1 + Bu  0 u :=   C Cx Cx + w2 C

P :

x+ z1 z2 y

K:

ζ + = AK ζ + BK y := u = C K ζ + DK y

= = = =

·

AK BK C K DK

B 0 0 0

0 0 0 1

 B 1 , 0 0

(3)

¸ ,

(4)

˜ K , and ˜K , CK = µC ˜K , and DK = µ2 D where AK = A˜K , BK = µB G:

χ+ = Aχ + Bw z = Cχ + Dw .

(5)

We use the notation F (P, K) = G to denote the feedback interconnection of P with K. It is useful to introduce the following system norm for G. Definition 3 The Mean Square norm of G is defined as follows:

kGkM S

v uX u p = max t kGij k22 . i=1,...,p

j=1

The square of the Mean Square norm of G the maximal among the output channels energies. Note the difference with the traditional H2 norm, which is the sum of the energies of the output channels rather than the maximum among the output channels energies. The LMI problem to compute the kGkM S are reported in the Appendix. · I ¸ © ª © ª ∆ 0 Finally, ∆ = maps z → w. Note that E ∆I = E ∆O = 0, and O 0 ∆ o n o n 2 2 2 σ ¯ 2 = E ∆I = E ∆O = σµ2 .

4

Mean Square Closed Loop Stability

Let H = F (∆, G) denote the feedback interconnection of ∆ and G, H:

χ+ = Aχ + Bw z = Cχ + Dw w = ∆z.

(6)

38

Nicola Elia

Let M = E {χχ0 }, W = E {ww0 } = E {∆zz 0 ∆}, and Z = E {zz 0 }. The Mean Square System associated with the closed loop system H in (6) describes the evolution of M , and has the following form. M + = AM A0 + BW B 0 Z = CXC 0 + DW D0 .

(7)

The stability of the Mean Square system defines the Mean Square Stability of H in (6). Definition 4 The closed loop system H in (6) is Mean Square Stable if its associated Mean Square System (7) is well-posed and stable, i.e., for any initial condition M (0) ≥ 0, lim M (k) = 0, where {M (k)} is the solution of the system (7) starting k→∞

at M = M (0).

5

Minimal Channel Quality for Means Square Stability

What is the minimal Quality of Service Q∗ = 1 − e∗ for which the closed loop system can be Mean Square Stabilized? e , which is monotonic in e. Thus, maximizing e (minimiz1−e ing Q) is equivalent to maximizing σ 2 , the variance of the random variables ∆I , and ∆O that can be tolerated by the close loop without loosing the Mean Square Stability. Thus, the minimal channel quality problem is reduced to a robust control problem. The following result states that minimal quality of the Eraser channel required for guaranteeing the Mean Square stability of the closed loop system can be computed by solving LMI optimizations and performing a line search for the minimum of a single variable function. Note that σ 2 =

Theorem 5 [11] Q∗ = 1 −

1 , 1 + µ∗M S

where µ∗M S given by the optimal cost of the following optimization problem µ∗M S = inf

inf

ϑ∈R+ K−stab,LT I

° ° −1 °θ0 (ϑ)F (P, K)θ0 (ϑ)°2

MS

;

θ0 (ϑ) = Diag(1, ϑ)

°2 ° For each value of ϑ > 0, inf K−stab,LT I °θ0−1 (ϑ)F (P, K)θ0 (ϑ)°M S can be computed by solving the LMI optimization problem in (12). The above result is a special case of a more general theory summarized in the next Section.

Indelible Control

6

39

Mean Square Stability Robustness to Structured Bounded Variance Uncertainty

In this section, we want to find the largest value of variance, σ ¯ 2 , which does not destroy the Mean Square Stability of the closed loop system. We next introduce the definition of Mean Square µ. Definition 6 The Mean Square µ of G, denoted by µM S (G, ∆), or shorthand µM S , is defined as follows µM S (G, ∆) =

1 . {sup σ ¯ 2 : the closed loop system is Mean Square Stable}

1 gives the largest Mean Square Stability radius [14]. It tells the largest variance µM S of the random variables which is tolerable by G.

6.1

Analysis

Consider the setup described in Figure 2, where the linear time invariant discretetime system G, p-input, p-output, is in feedback with a diagonal ∆ = Diag(∆i , i = 1, . . . , p). For each i = 1, . . . , p, ∆i (0), ∆i (1), . . . , ∆i (k), . . . are independent identically distributed random variables with E{∆i (k)} = 0, and E{(∆i (k))2 } = σ 2 ∀k ≥ 0. Moreover, ∆1 (k), . . . , ∆p (k) are independent for each k, although not necessarily identically distributed. ∆ acts as multiplication operator on z to provide w. i.e., w = ∆(z) ⇔ wi (k) = ∆i (k)zi (k), for i = 1, . . . , p, and for all k ≥ 0. Theorem 7 [11] Assume that G is stable with D either strictly upper or strictly lower triangular. Then the following statements are equivalent. i) The feedback interconnection of G and ∆ is Mean Square Stable. ii) There exists a positive definite matrix Q and a vector α ∈ Rp of positive elements satisfying the following Linear Matrix Inequalities: Q > AQA0 +

p X

Bj αj Bj0

j=1

¯2 αi > σ ¯ 2 Ci QCi0 + σ

p X

0 Dij αj Dij , for i = 1, . . . , p.

j=1

° −1 °2 °θ Gθ°

inf < 1. iii) σ ¯2 MS ³θ>0, Diag. ´ ˆ < 1 where ρ(·) denoted the spectral radius and iv) ρ σ ¯2G  kG11 k22 . . . kG1p k22 . ..  . ˆ= G  .. ... .  2 kGp1 k2 · · · kGpp k22 

(8)

40

Nicola Elia

The assumption on the strict triangularity of D guarantees that the Means Square system (7) is well-posed, and that W is diagonal. It also guarantees that the final result depends only on σ 2 and not on the specific distribution. Note that G = F (P, K) with P given in (3) satisfies this assumption. The following is an immediate consequence of the above theorem. ˆ = µM S (G, ∆) = ρ(G)

inf

θ>0, Diag.

° −1 °2 °θ Gθ°

MS

.

We have seen that the problem of control over Erasure communication channels of Section 2 leads to a robust control problem where a deterministic model is in feedback with a purely stochastic uncertainty made of random variables of bounded variance. This interpretation is natural whenever multiplicative noise is part of a feedback loop, and leads to the robustness problem of Section 6. Our derivation is based on the result of [14,15] on the stability radius of systems with multiplicative noise. Note that, differently from [16,17], there is no nonlinear or dynamic stochastic uncertainty in our model. Rather, we consider the randomness of the stochastic variables as the source of uncertainty. Moreover, our nominal model is purely deterministic instead of being stochastic. In the spirit of [14,18,20,19], we interpret the variance of the random variables as the size of the stochastic uncertainty in the deterministic model.

6.2

Controller Synthesis

From Theorem 5 condition iii), the synthesis problem is posed as follows. µ∗M S = =

inf

inf

K−stab,LT I θ>0, Diag.

inf

inf

θ>0, Diag. K−stab,LT I

° ° −1 °θ F (P, K)θ°2 MS °2 ° −1 °θ F (P, K)θ° , MS

(9)

where G = F (P, K) is the closed loop system from w to z with the controller in place. The available approaches to solve the synthesis problem are either based on non convex global optimization methods [24,25] or are based on heuristic suboptimal methods like the D − K iteration [22]. At any rate, for two or three uncertainty blocks, we can resort to gridding of θ > 0. For each fixed θ0 > 0, we need to solve for the controller that minimizes the Mean Square norm of the scaled system. We want to find K such that ° −1 ° °θ0 F (P, K)θ0 °2 . (10) inf MS K−stab,LT I

The approach of [23] applies to the synthesis of the linear controller that minimizes the spectral norm of G. The formulae are reported in the Appendix.

Indelible Control

7

41

Special Cases with only One Channel

In this section we specialize the result of Section 6 to the case of only one channel being present in the loop. The single channel result is of interest of its own because brings up already interesting issues without having to consider the more complex network situations. Corollary 8 Given the SISO Plant (1). Assume that only one discrete Erasure channel is present either at the plant input channel or the plant output. Then, Mean Square stability of the closed loop system is guaranteed if and only if

e<

1 = e∗ , 1 + kGk22

or

Q>

kGk22

1 + kGk22

.

The result recovers the one of Willems in [18]. Note that G is SISO thus kGkM S = e . The result of the corollary also holds for either Single kGk2 , and that σ 2 = 1−e Input or Single Output plants.

7.1

State Feedback with One Input Channel

In this section, we assume that the controller has perfect access to the system state, and the system is single input. In this case, we derive a closed form formula for the maximal tolerable probability of erasure e∗ , and the associated minimal Quality of Service as function of the unstable eigenvalues of the closed loop system. Corollary 9 Given the single input plant x+ = Ax + Br. Assume that a linear controller has perfect state measurements, and that it acts on the plant input via a discrete Erasure channel. Then, the closed loop can be Mean Square stabilized by a linear controller if and only if e< Y i

1 |λui (A)|2

= e∗ ,

or

Q>1− Y

1 |λui (A)|2

= Q∗ .

i

Y u min kF (P, K)k22 = |λi (A)|2 −1 (see [10]), and we want In this case µM S = K−stab, LT I i ! Ã Y u 2 2 |λi (A)| − 1 < 1. Note that this inequality can be re-arranged as follows σ ¯ i r Y u µ2 |λi (A)| < 1 + 2 , which provides the condition for the Uncertainty Threshold σ i Principle, [27], in the case B single input (not invertible). Thus, the Uncertainty Threshold Principle can be interpreted in term of Means Square robustness margin.

42

Nicola Elia [ Oÿ ,+ ÿ

Nÿ NEÿ

Ò

N1 NE1 O1 ,+1

Sÿ

S1 Z

Fig. 3. Schematics of the Pendubot.

8

Minimal Channel Quality for the Stabilization of a Pendubot

From Theorem 5, the quality of service requirements are dictated by µM S , which, in turn, is limited by the presence and location of the plant unstable poles and non-minimum phase zeros. We next give a practical illustration of this dependence by applying the results to a control laboratory experiment: the stabilization of a pendubot. A pendubot is a two link under-actuated planar robot (as shown in Figure 3), with torque actuation only on the first link. See [26] and reference therein for more detailed information. We are interested in the local stabilization of the equilibrium point where both links are in the vertical position. We assume that the pendubot starts close enough to this unstable equilibria. Usually this is achieved by an appropriated swing-up procedure. The continuous time linearized model is discretized with a sampling time of T = 0.005 s becoming our plant. Form the data in [26] the eigenvalues of the plant are λ1 = 1.0591, λ2 = 1.0324, λ3 = 0.9686, λ4 = 0.9442.

8.1

State Feedback

We first assume that the state of the system is perfectly known to the controller, and that the Erasure channel is at the plant input. From Corollary 9 we obtain that e∗sf = λ21λ2 = 0.8365. 1 2

8.2

Output Feedback with only Input Erasure Channel

If we assume only q1 + q2 is measured, and that there is only one channel in the loop, then from Corollary 8 we obtain e∗q1 +q2 u 0.8359. The increase in the minimal required channel quality is due to the increased difficulty in controlling the system using only part of its state. In this case it is not substantial, however, we expect that controlling the pendubot using only q2 would be more difficult than using q1 + q2 . In fact, while the Plant transfer function when we measure q1 + q2 , P lantτ →q1 +q2 =

−0.00050287(z + 1)(z − 1)2 , (z − 1.0591)(z − 1.0324)(z − 0.9686)(z − 0.9442)

Indelible Control

43

is only marginally non-minimum phase, the Plant transfer function when we measure only q2 , P lantτ →q2 =

−0.0010639(z + 1)(z − 0.9727)(z − 1.0281) , (z − 1.0591)(z − 1.0324)(z − 0.9686)(z − 0.9442)

is unstable and strictly non-minimum phase. The increased difficulty of control is reflected by the increased minimum required channel quality. We obtain e∗q2 u 1.05 · 10−2 .

8.3

Output Feedback with both Input and Output Erasure Channels

Finally, if we now consider the presence of an Erasure channel also on the output, from Theorem 5, we obtain e∗q1 +q2 u 0.4482, and e∗q2 u 3 · 10−3 , which shows how the presence of an extra Erasure channel reduces the stability margin and therefore the largest tolerable probability of erasure in both channels.

9

Conclusions

The presence of communication channels in the feedback loop, introduces stochastic uncertainty against which the closed loop must be robust. Through robust control results, we have established a direct connection between the unstable poles and nonminimum phase zeros of the plant, which also affect the fundamental limitation of feedback, and the minimal required channel quality as the maximal probability of erasure. In more general terms, we are saying that the set of plants that can be stabilized over analog erasure channels is fundamentally limited by the channel generated uncertainty. Analogously, the channel Capacity is also fundamentally limited by the channel generated uncertainty. Future research will focus on understanding the deeper connection between these results.

44

Nicola Elia

10

Appendix

10.1

LMI Optimization for Computing the Mean Square Norm

kGk2M S =

inf

P>0,S>0,γ

subject to: 

(11)

γ

 P PA PB  A0 P P 0 >0 0 P 0 I B   S C D  C 0 P 0  > 0, D0 0 I Sii < γ, i = 1, . . . , p.

10.2

Optimal Mean Square Norm synthesis

For any given θ0 > 0 the optimization problem: ° ° −1 °θ0 F (P, K)θ0 °2 inf K−stab,LT I

MS

is equivalent to the following LMI optimization γ∗ =

inf

ˆ B, ˆ C, ˆ D,γ ˆ X,Y,S,A,

γ

(12)

subject to: 

 ˆ A + B DC ˆ Bw + B DD ˆ w X I AX + B C  ˆ ˆ YBw + BD ˆ w I Y A YA + BC     0 0 0 0 ˆ ˆ X I 0 A >0  XA + C B   0 0ˆ0 0 0 0 ˆ0  A +C D B A Y+C B I Y 0 0 ˆ0 0 ˆ 0 0 Bw Y + Dw 0 0 θ0−2 + Dw B DB 0 Bw



ˆ Cz + Dz DC ˆ Dzw + Dz DD ˆ w θ0 Sθ00 C z X + Dz C ˆ 0 Dz0  XCz0 + C  X I 0  >0  Cz0 + C 0 D  ˆ 0 Dz0 I Y 0 0 0 ˆ0 0 D Dz 0 0 θ0−2 Dzw + Dw Sii < γ

i = 1, . . . , p,

where X and Y are symmetric n × n, S is symmetric p × p, A, B, C, D have dimensions n × n, n × nu , ny × n, and ny × nu respectively. Moreover, for any feasible solution of the above optimization, a full-order controller, i.e., a controller of the order of the plant, can be reconstructed from the following formulae ˆ D DK = ³ ´ ˆ − DK CX (M 0 )−1 CK = C ´ ³ ˆ − YBDK BK = N −1 B ³ ´ ˆ − N BK CX − YBCK M 0 − Y (A + BDK C) X (M 0 )−1 AK = N −1 A

Indelible Control

45

where M and N are chosen square and invertible and such that M N 0 = I − XY. One possible choice is the following: M such that M M 0 = X − Y −1 , and N such that

· P=

¸ · ¸−1 Y N X M . = 0 0 N ∗ M I

The above theorem is also saying that there is no advantage in searching for µ∗M S over controllers of order greater than the plant. Note the non-convex dependence on θ0 typical of such problems.

References 1. Wing Shing Wong, R.W. Brockett, ”Systems with finite communication bandwidth constraints. II. Stabilization with limited information feedback”, IEEE Transactions on Automatic Control, Volume: 44 Issue: 5, Page(s): 1049 -1053, May 1999. 2. Wing Shing Wong, R.W. Brockett, ”Systems with finite communication bandwidth constraints. I. State estimation problems”, IEEE Transactions on Automatic Control, Volume: 42 Issue: 9, Page(s): 1294 -1299, Sept. 1997. 3. S. Tatikonda, A. Sahai, and S.K. Mitter ”Control of LQG systems under communication constraints” Proceedings of the 1999 American Control Conference, Volume: 4 , Page(s): 2778 -2782, 1999. 4. G.N. Nair, and R.J. Evans, ”Communication-limited stabilization of linear systems”, Proceedings of the 39th IEEE Conference on Decision and Control, Volume: 1, Page(s): 1005 -1010, 2000. 5. S. Tatikonda, and S.K. Mitter, ”Control under communication constraints”, Prooceedings, of the 38th AnnualAllerton Conference on Communication, Control, and Computing, October 2000. 6. S. Tatikonda, and S.K. Mitter, ”Feedback under communication constraints: Part I”, Preprint, submitted to IEEE Transactions on Automatic Control, Feb. 2002. 7. S. Tatikonda, and S.K. Mitter, ”Feedback under communication constraints: Part II”, Preprint, submitted to IEEE Transactions on Automatic Control, Feb. 2002. 8. A. Sahai, ”Anytime Information Theory”, Ph.D. Thesis M.I.T. Feb. 2001. 9. A.S. Matveev, and A.V. Savkin, ”Optimal control of networked systems via asynchronous communication channels with irregular delays”, Proceedings of the 40th IEEE Conference on Decision and Control, Volume: 3, Page(s): 2327 -2332 vol.3, 2001. 10. N. Elia, and S. K. Mitter, ”Stabilization of linear systems with limited information”, IEEE Transactions on Automatic Control, Vol.46, No. 9, pp. 1384 -1400, Sept. 2001. 11. N. Elia ”Stabilization of Systems with Erasure Actuation and Sensory Channels” to appear in Procedings of the 40th Allerton Conference on Communication, Control, and Computing, 2002. 12. N. Elia, ”Feedback stabilization in the presence of fading channels”, submitted to American Control Conference 2003.

46

Nicola Elia

13. A. Hassibi, S.P. Boyd, and J.P. How, ”Control of asynchronous dynamical systems with rate constraints on events”, Proceedings of the 38th IEEE Conference on Decision and Control, Volume: 2, Page(s): 1345 -1351 1999. 14. S. P. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in System and Control Theory. Philadelphia: Society for Industrial and Applied Mathematics, 1994, SIAM Studies in Applied Mathematics 15. 15. L. El Ghaoui, ”State-feedback control of systems with multiplicative noise via linear matrix inequalities”, Systems and Control Letters, Issue:24, Page(s):223228, 1995. 16. A. El Bouhtouri, D. Hinrichsen and A. J. Pritchard, ”Stability radii of discretetime stochastic systems with respect to blockdiagonal perturbations”, Automatica, Volume 36, Issue 7, Page(s): 1033-1040, July 2000, 17. Jianbo Lu and R.E. Skelton, ”Mean-square small gain theorem for stochastic control: discrete-time case”, IEEE Transactions on Automatic Control, Volume: 47 Issue: 3 , Page(s): 490 -494, March 2002. 18. J.C Willems, and G.L Blankenship, ”Fequency domain stability criteria for stochastic systems”, IEEE Transactions on Automatic Control, Volume:16, Issue:4, Page(s): 292-299, Aug. 1971. 19. D.L. Kleinman, ”Optimal Stationary control of linear systems with controldependent noise”, IEEE Transactions on Automatic Control, Volume: 14, Page(s):673-677, Dec. 1969. 20. W.M. Wonham, ”Optimal stationary control of linear systems with statedependent noise,” SIAM Journal of Control, Volume: 5, Page(s): 486-500, Aug. 1967. 21. M. A. Dahleh, Control of Uncertain Systems: A Linear Programming Approach, Prentice-Hall, New Jersey, 1994. 22. G.E. Dullerud, and F. Paganini, ”A course in robost control theory: a convex approach”, Springer, New York, 2000. 23. C. Scherer, P. Gahinet, and M Chitali, ”Multiobjective output-feedback control via LMI optimization”, IEEE Transactions on Automatic Control, Volume: 42, Issue:7, Page(s):896-911, July 1997. 24. K.-C. Goh, M.G. Safonov, and G.P. Papavassilopoulos, ”A global optimization approach for the BMI problem” Proceedings of the 33rd IEEE Conference on Decision and Control, 1994., Volume: 3, Page(s): 2009 -2014, 1994. 25. M. Khammash, M.V. Salapaka, and T. Van Voorhis ” Robust synthesis in `1 : a globally optimal solution”, IEEE Transactions on Automatic Control, Volume: 46 Issue: 11 , Page(s): 1744 -1754, Nov. 2001. 26. Mingjun Zhang, and Tzyh-Jong Tarn ”Hybrid control of the Pendubot” IEEE/ASME Transactions on Mechatronics, Volume: 7 Issue: 1 , Page(s): 79 -86, March 2002 27. R.T. Ku, M Athans, ”Further results on the uncertainty threshold principle”, IEEE Transactions on Automatic Control, Volume: 22, Issue: 5, Page(s): 866868, Oct. 1977.

Optimal Control Design under Structural and Communication Constraints ? Petros G. Voulgaris University of Illinois at Urbana-Champaign, Coordinated Science Laboratory, 1308 W. Main, Urbana, 61801, USA

Abstract. We present a list of optimal disturbance rejection problems in systems in which the overall control scheme is required to have a certain structure. These structures correspond to various classes of controlled systems which include what we refer to as nested, chained, hierarchical, delayed interaction and communication, and, symmetric systems. The common thread in all of these classes is that by taking an input-output point of view we can characterize all stabilizing controllers in terms of convex constraints in the Youla-Kucera parameter. The disturbance rejection problem can therefore be casted as a convex, yet nonstandard, model matching problem. Approaches that solve this problem are presented for various optimality criteria.

1

Introduction

In large, complex and distributed systems there is often the need of considering a specific structure on their overall control scheme (e.g., [15].) There are a number of practical reasons, among which cost and reliability, that result to constraints on how an individual local control station interacts with the overall system, what part of information it has access to and what communication mechanism is in place. Hence it is important to have analysis and design techniques when interaction and communication constraints are imposed on the global controller structure. In this paper we consider the general framework of Figure 1 where G may represent a complex system consisting of subsystems interacting with each other. The overall controller for G is K. Both G and K are assumed to be linear, discrete-time systems. The controller K has to respect a specific structure that is imposed by interaction and communication constraints. A typical example of structure that has been studied extensively in the literature is when K is totally decentralized i.e., diagonal. However, the optimal performance problem when structural constraints are present still remains a challenge to the control community due to each complexity, notably the lack of a convex characterization of the problem. To mention only a few samples of related work we refer to [14,16,9] and references therein. Taking an inputoutput point of view and parametrizing all K via the Youla-Kucera [23] parameter Q one can see as a major source of difficulty the fact that structural constraints in K may lead to non-convex constraints in Q. Despite this discouraging point, a main theme in the paper is identifying specific classes of problems for which the ?

This work was supported in part by NSF grant CCR 00-85917 ITR

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 47--61, 2003 Springer-Verlag Berlin Heidelberg

48

Petros G. Voulgaris

constraints in Q are convex with the appropriate choice of the coprime factors of G. These classes can be associated with several practical applications such as integrated flight propulsion systems, platoons of vehicles, networked control, production lines, chemical processes, etc. The various classes of systems identified include what we refer to as nested, chained, hierarchical, delayed interaction and communication, and, symmetric systems. A key feature in all the structures considered is that G22 , the part of G that relates the controls u to measurements y, has the same structure as the one imposed on the controller K. It is the structure of G22 that matters for convexity; the remaining part of G can be totally unstructured. In the majority of the situations presented here there is an algebraic property between the K’s and G22 ’s under consideration. That is, they form a ring as the structure is preserved in products, additions and in (I − G22 K)−1 whenever the inverse exists, as it should, for well-posedness. In the sections to follow we first expose the various structures of interest, continue on to controller parametrization and finally describe the solution procedures to the optimal performance problems in the H2 , `1 and H∞ sense.

w-

z

G

-

y

u K

¾

Fig. 1. Standard Framework

2

Specific Structures

Throughout the paper we assume, unless stated otherwise, all systems to be finite dimensional linear, time-invariant and described in discrete-time.

2.1

Triangular Structures

In this class of systems G22 and K are triangular transfer function matrices. This class includes what we term as nested structures, chains (or strings) with leader and followers, and hierarchical type of schemes. In the sequel we elaborate more on these. Some parts of this exposition can also be found in [19,20].

Nested systems This is the case where a subsystem is inside another and there is only one-way interaction, from inside to outside, or, the reverse. A practical application that associates with this set-up is the Integrated Flight Propulsion Control

Optimal Control Design under Structural Constraints

z2¾

P2

z12

?66 f 6 ? ¾ P1 ¾

¾

z1¾ y2

y1

w2

¾ ¾

w12 w1 u1

-

49

u2

C1 ?

-

C2

Fig. 2. Nested Structure

(IFPC) e.g., [5]. To illustrate the nested problem in simple terms we consider only two nests. The generalization to n nests is straightforward. Thus we consider the case of Figure 2 where there is a system comprised of two nests (subsystems.) The internal subsystem consists of a plant P1 together with its controller C1 whereas the external consists of the plant P2 with the controller C2 . The internal and external subsystems have control inputs u1 , u2 and measured outputs y1 and y2 respectively. Due to the nested structure depicted in the figure, the control input u1 depends only on the measurement y1 whereas u2 can depend on both y1 and y2 . Moreover, y1 is affected only by u2 while y2 is affected by both u1 and u2 . The overall system is subjected to exogenous inputs (e.g., commands, disturbances, sensor noise) and there are also outputs to be regulated. In particular, we allow for inputs w1 affecting directly the internal subsystem, inputs w2 that affect the external subsystem only, and, inputs w12 that affect both subsystems. Similarly, the outputs of interest z1 , z2 and z12 are related respectively directly to the internal, directly to the external and to combination of both subsystems. A necessary assumption for the existence of a stabilizing overall controller K is that each subsystem Pi is stabilizable by each subcontroller Ci . Thus, if the exogenously unforced (with no external disturbances) state space description for P1 is

x1 (k + 1) = A1 x1 (k) + B1 u1 (k) y1 (k) = C1 x1 (k) + D1 u1 (k)

50

Petros G. Voulgaris

and for P2 x2 (k + 1) = A2 x2 (k) + B21 u1 (k) + B2 u2 (k) y2 (k) = C2 x2 (k) + D21 u1 (k) + D2 u2 (k) we have that there exist feedback and observer gains Fi and Li respectively such that Ai + Bi Fi and Ai + Li Ci are Hurwitz (i.e., eigenvalues in the unit disk) for i = 1, 2. Bringing the system of Figure 2 to the standard G − K control design framework of Figure 1 we have the following signal identifications     µ ¶ µ ¶ w1 z1 u1 y1 , z := z12  , w := w12  . , u := y := u2 y2 z2 w2 The structure of G22 is of the form µ G22 =

g1 0 g12 g2

¶

i.e., G22 has a lower (block) triangular (l.b.t.) structure. Moreover, for the controller K to correspond to the nested structure of Figure 2 it should be of the form µ K=

k1 0 k12 k2

¶

i.e., it should be a lower (block) triangular system.

Chains In the chain (or string) system of Figure 3 there are n subsystems Pi with their corresponding subcontrollers Ci . Platoons of vehicles where there is a leader and followers that are obtaining information form their leading vehicles is a good example to associate with this structure. The control action ui in the subsystem Pi affects its follower Pi+1 by a 1-step delay while the control action ui+1 in Pi+1 does not affect its leader Pi . Also, subcontroller Ci passes information to its follower Ci+1 with a 1-step delay while Ci+1 does not transmit any information to Ci . Exogenous inputs w and regulated outputs z are admitted that may couple the dynamics but are not shown in the picture for clarity. Bringing in the general G − K framework we have that in the case of the chain the structure of G22 is 

G22

g1  λg21 g2  2  =  λ g31 λg32  .  .. λn gn1 · · ·

 g3 .. .. . . · · · λgnn−1 gn

     

Optimal Control Design under Structural Constraints

51

Fig. 3. Chain Structure and similarly the structure of K is 



k1  λk21 k2  2  K =  λ k31 λk32  .  .. λn kn1 · · ·

k3 .. .. . . · · · λknn−1 kn

     

Noting that    G22 =  

1

 λ

..

.

    λn−1

 g1   g21 g2     . . . .   . . gn1 · · · · · · gn 

    

1

    

−1

λ

..

.

λ−n+1

and    K= 

1

 λ

..

.

    λn−1

 k1   k21 k2    .  . ..   .. kn1 · · · · · · kn 

    

1

    

−1

λ

..

.

λ−n+1

it follows that the chain problem is a special case of a the nested problem. An additional structure which can be imposed that requires fewer building blocks is that of Toeplitz, i.e.,

52

Petros G. Voulgaris 

G22

g1  λg2  = ..  . λn−1 gn

  g1   .. ..  . . . . . λg2 g1

with K as in G22 by replacing g’s with k’s.

Hierarchical Yet another structure in this category is that of open hierarchies considered in [19] where a decision maker receives signals from upper levels, possibly direct external commands, and, is allowed to pass information only to lower levels. 2.2

Delayed Interaction and Communication Networks

The network in this case is as in Figure 4. In this figure subsystem Pi and its subcontroller Ci interact with their respective neighbors with a 1-step delay in the transmission and reception of signals. Exogenous inputs w and regulated outputs z are admitted that may couple the dynamics but are not shown in the figure for clarity. A good relevant example to associate in this case is the control of a large networked system over a network where, as an aggregate model, the neighbor-to neighbor interaction and communication is subject to a unit delay. In the G − K framework the structure is reflected in G22 and K as 

G22

g11 λg12  g22  λg11 = .. ...  . λn−1 gn1 · · ·

 · · · λn−1 g1n n− 2  ··· λ g2n   .. ..  . . ··· gnn

and similarly for K 

k11 λk12  k22  λk11 K = .. .  . .. λn−1 kn1 · · ·

 · · · λn−1 k1n n− 2  ··· λ k2n   .. ..  . . ··· knn

For a distributed control system of a similar type in the case of spatially invariant systems we refer to [21].

Optimal Control Design under Structural Constraints

53

Fig. 4. Delayed Interaction and Communication Structure

2.3

Other Structures

Delayed Observation Sharing Two examples of communication patterns are given. Both are shown in the Figures 5, 6. In Figure 5 the measurement information from a local control station Ci is passed to the other with the delay of n time-steps. In Figure 6 information exchanges between stations Ci are performed every n timesteps through a data recording and supervising unit S. In both of these scenarios

Fig. 5. n-Step Delayed Information Exchange

54

Petros G. Voulgaris

there is no interaction between the local plants Pi . There is however a coupling through the disturbances w and the variables z to be regulated. In the G − K frame the structure of the required controller K for the first case is µ K=

k1 λn k12 n λ k21 k2

¶

For the second case, one can lift the system stacking the inputs and outputs to integer multiples of n time steps to realize that the lifted controller K should have a feedthrough D-term of the form   f1  f2 d1    DK =  .  . . .. ..   .. fn dn−1 · · · d1 where fi are full 2 × 2 matrices and di are diagonal. This is a case where n-time periodic Ci ’s are considered. Generalizations to tree-clusters of this structure can also be considered where supervisors Si and Sj communicate with each other through a higher level unit Σ every m × n time-steps.

Symmetric Structures This is the case where G22 = GT22 and K = K T . A

special case is the circulant symmetry in G22 [3].

Fig. 6. n-Step Information Sharing

Optimal Control Design under Structural Constraints

3

55

Controller Parametrization

Employing the Youla-Kucera parametrization, all stabilizing K , not necessarily with the structure required, are given by the parametrization [23] K = (Yl − Dl Q)(Xl − Nl Q)−1 = (Xr − QNr )−1 (Yr − QDr ) where Q is a stable free parameter and Yl , Dl , Xl , Nl , Xr , Nr , Yr , Dr can be obtained from a coprime factorization (e.g., [7,17]) of G22 . The structural constraints on K transform to constraints in Youla parameter Q. With a particular choice of the coprime factors the constraints in Q are convex. In fact, these constraints are the same as in the required structure for K as indicated below.

3.1

Triangular Structures

For simplicity we will treat only the case of 2-nested systems as generalizations are straightforward. In this case G22 has the state-space description µ G22 ∼ (

¶ µ ¶ µ ¶ µ ¶ A1 0 B1 0 C1 0 D1 0 , , , ). 0 A2 B21 B2 0 C2 D21 D2

By the necessary assumption on the problem formulation we have that there exist feedback and observer gains Fi and Li respectively such that Ai +Bi Fi and Ai +Li Ci are Hurwitz (i.e., eigenvalues in the unit disk) for i = 1, 2. Hence one can choose a state feedback and an observer gain for G22 respectively as µ F =

¶ µ ¶ F1 0 L1 0 , L= . 0 F2 0 L2

With this choice the standard set of doubly coprime factors in [7] have the required triangular structure and hence the structural constraints on K transform to the same constraints on Q. I.e., Q is required to be lower (block) triangular. For Toeplitz triangular structures similar type of arguments can be used to show that Toeplitz coprime factors can be chosen so that Q is constrained to be Toeplitz. This is simple enough to establish when G22 is stable since all K’s are represented as K = −Q(I − G22 Q)−1 . The unstable case requires some more work and is left to the reader to verify.

56

3.2

Petros G. Voulgaris

Delayed Interaction and Communication Structures

For this types of structures G22 has a pulse response {G22 (i)}∞ i=0 with the following band-structure: G22 (0) is diagonal, G22 (1) is 3-diagonal, G22 (2) is 5-diagonal, etc.; that is G22 (i) is a 2i + 1-diagonal matrix for i = 0, 1, . . . , n − 2. Similar is the imposed structure on K. Representing G22 as G22 = (G1 G2 . . . Gn ) and obtaining a controllable state-space description for each Gi as Gi ∼ (Ai , Bi , Ci , Di ), a state space description of G22 ∼ (A, B, C, D) can be obtained [4] with A = diag(Ai ), B = diag(Bi ), C = (C1 . . . Cn ), D = (D1 . . . Dn ). This is a controllable realization. If Fi are such that Ai + Bi Fi are Hurwitz one can use a state feedback gain F = diag(Fi ) and the standard formulas in [7] to obtain the coprime factors Nl and Dl . It is straightforward to check that F Aj B is diagonal for j = 0, 1, . . . and hence Dl and consequently Nl posses the same structure as G22 . A similar argument for K shows that a K with the band-structure can be factored as K = Yl Xl−1 with Yl , Xl coprime, possessing the band-structure. The existence of such a stabilizing K with band-structure follows from the fact that λn G22 can be stabilized by a controller (assuming G22 can). Hence as K = (Yl − Dl Q)(Xl − Nl Q) −1 it follows that K has the band-structure iff Q does.

3.3

Other Structures

The case of delayed observation sharing structures is similar to the previous subsection and thus it will not be discussed further. The result is that Q has to have the same structure as K. For the symmetric structures mentioned we consider the case where G22 is stable and symmetric, i.e., GT22 = G22 . Then all stabilizing K, possibly non-symmetric, are given as K = (I + QG22 )−1 Q. If K is to be symmetric, then

Q = K(I − G22 K)−1 = K T (I − GT22 K T )−1 = = (I − K T GT22 )−1 K T = ((I − G22 K)T )−1 K T = QT

i.e., Q is symmetric. Similar argument shows that if Q is symmetric then K = −1 (I + QG22 ) Q is symmetric. For the unstable case we refer to [22].

4

Optimal Performance

All the classes discussed in section 2 require when viewed in the G-K framework that K is stabilizing and has a specific structure (triangular, banded, etc). From the discussion in the previous section, this is equivalent to requiring that Q has the same structure as K. The structures considered correspond to subspace type of

Optimal Control Design under Structural Constraints

57

restrictions on Q. We denote by S the (closed) subspace of stable systems Q that have the required structure. The problem of interest is as follows: Problem: Find K with the appropriate structure such that, subject to internal stability, the norm kΦk is minimized. The norm kΦk may refer to any norm, e.g., H2 , H∞ or `1 . By internal stability here we mean the usual stability requirement in the G-K framework. Based on the parametrization in the previous section the problem of minimizing kΦk can be casted as µ := inf kH − U QV k Q∈S

where H, U, V are stable systems. Therefore, the resulting problem is convex, yet infinite dimensional (the pulse response coefficients of Q.)

4.1

Approaches for solving the equivalent problem

In principle, one can solve the problem by considering truncations of the Q parameter [2] and thus approximating the problem with a finite dimensional (the pulse response coefficients of the truncated Q) convex programming problem µN := inf kH − U QN V k QN

where QN is a Finite Impulse Response (FIR) of length N , system in S. It can be checked that µN → µ monotonically from above as N → ∞. The main shortcoming of this method is that it cannot indicate how close to the optimal solution is the converging upper bound µN . In the sequel we discuss approaches to completely solve the problem.

H2 -norm minimization In this case we can invoke the projection theorem

along the lines in [18] to obtain the solution directly. To this end let U = Ui Uo , V = Vo Vi be an inner-outer factorization of U and V respectively. Define the subspace M := {Z : Z = Uo QVo , Q ∈ S}. Then the following can be shown: Result 1. The optimal solution Zo for the problem µ = inf kH − Ui ZVi k Z∈M

is given by the projection onto M Zo = ΠM Ui∗ HVi∗ . Once Zo is found an optimal Q can be found as Q = Uo−r Zo Vo−l where Uo−r is a right inverse of Uo and Vo−l is a left inverse of Uo . Characterizing ΠM in a simple enough manner to allow computations is possible in many of the systems presented. For example, for triangular problems a specialized inner-outer factorization (see [6] chapt. 14) leads to triangular Uo , Vo so that M coincides with S which makes the projection ΠM Ui∗ HVi∗ trivial. An alternative however that avoids solving the nonstandard problem can be found in [8] where the original problem is transformed to

58

Petros G. Voulgaris

a standard model matching by “vectorizing” Q. The equivalent problem is of the form ° ° ¯ q° inf °h − U q∈H2

where q is a vector unconstrained with one-to-one correspondence with Q ∈ S. The solution for q is then the same as in Theorem 1 where M is simply the space H2 i.e., ¯i∗ h ¯o−r ΠH2 U qopt = U ¯o is an inner-outer factorization of U ¯ =U ¯i U ¯ and U ¯o−r is a right inverse of with U ¯o . U

`1 -norm minimization In this case one can use an extension of the scaled-Q

method in [10] to provide converging lower and lower bounds to µ. In particular, for the problem at hand let PN denote the N th truncation operator and define the two finite dimensional linear programs: νN (α) := min max{kH − Rk , α kQk} subject to PN (R) = PN (HQV ), Q ∈ S and µN (α) := min max{kH − Rk , α kQk} subject to R = U PN (Q)V, Q ∈ S where α is a scalar positive parameter. Then, using elements of duality theory the following can be shown

Result 2. There exists an a priori computable α0 such that for all α with 0 < α ≤ α0 , µN (α) → µ monotonically from above and νN (α) → µ monotonically from below as N → ∞. Hence, with the above theorem one obtains close to optimal solutions to any prespecified accuracy.

4.2

H∞ control

For the H∞ problem a Nehari-based approach [18] to get a sequence of converging lower bounds is possible. However, more efficient computations are needed. Recent work in [12,13] on multi-objective H∞ /H2 control provides a Q-based design technique that gives converging lower as well as upper bounds. This technique readily applies to the “constrained in Q” problems described in this paper when H∞ optimization is of interest. It should be noted that the main approach in this method is not conceptually different from that in [10].

Optimal Control Design under Structural Constraints

59

Remark 1 For certain problems with specific symmetries constraining the controller to be symmetric is redundant as the optimal K will posses the required structure regardless. Such is the case for example when circulant symmetry is present [3,1]. In general however this may not hold. Certainly if in the map of interest Φ = H − U QV , H or U or V are not symmetric there is no guarantee that the optimal Q optimal is symmetric. But even if H and U and V are symmetric, the following (static) example shows that the optimal Q may not be symmetric. Consider Φ = H − U Q with ¶ µ      q13 Q 1 0 γ 110  q23   H =  0 1 α, U = 1 1 0 , Q =    γα 1 00β (q31 q32 ) q33 with α 6= 0, γ 6= 0, β 6= 0, β 6= 2 then µ

 I−

 Φ= 

(γ − βq31

µ

¶ γ − q˜ α − q˜    α − βq32 ) 1 − βq33

11 11

¶

Q

with q˜ = q13 + q23 . Φ can be thought of as a model following error closed loop map with G22 = −U , G11 = H, G12 = −U , and G21 = I. For kΦkH2 to be and minimal (γ − q˜)2 + (α − q˜)2 should be minimal i.e., q13 + q23 = q˜ = α+γ 2 , but also (γ − βq31 )2 , (α − βq32 )2 should be minimal i.e., q31 = βγ , q32 = α β α+γ q31 + q32 = α+γ 6= = q13 + q32 for β 6= 2. Hence optimal Q is not β 2 symmetric. The same example above can be used to show that `1 optimization as well may lead to a non-symmetric optimal Q. An easy way to construct more examples of this form is when U (and V ) are symmetric and stably invertible; then the unconstrained optimal in any norm cost is zero and the optimizing Q is Q = U −1 HV −1 which is in general non-symmetric.

Remark 2 A number of multi-objective problems can be considered for the types of structures listed in this paper. Some of the key approaches to use are the Q-based designs as in [12,13,11] where converging upper and lower bounds for the optimal cost are obtained and, arbitrarily close to optimal, suboptimal solutions are furnished. A complete and detailed development of general structured multiobjective design however is not in the scope of this paper and is left as a subject for future research.

5

Conclusions and Discussion

In this paper we presented a list of optimal disturbance rejection problems in systems in which the overall control scheme is required to have a certain structure.

60

Petros G. Voulgaris

These structures correspond to various classes of controlled systems such as nested, chained, hierarchical, delayed interaction and communication, and, symmetric systems. These classes can be associated with several practical applications in integrated flight propulsion systems, platoons of vehicles, networked control, production lines, chemical processes, etc. The common thread in all of these classes is that by taking an input-output point of view we can characterize all stabilizing controllers in terms of convex constraints in the Youla-Kucera parameter. The disturbance rejection problem can therefore be casted as a convex, yet nonstandard, model matching problem. Approaches that solve this problem were presented for H2 , `1 and H∞ optimality.

References 1. B. Bamieh, F. Paganini and M.A. Dahleh, “Distributed control of spatiallyinvariant systems” Tech. Report CCEC 98-0520, UCSB, May 1998. 2. S.P. Boyd and C.H. Barratt, Linear Controller Design: Limits Of Performance, Prentice Hall, Englewood Cliffs, New Jersey, 1991. 3. R.W. Brockett and J.L. Willems, “Discretized PDEs: Examples of control systems defined on modules,” Automatica, vol. 10, pp. 507-515, 1974 4. C.T. Chen. Linear System Theory and Design, Holt, Rinehart and Winston Inc., 1970. 5. Z. Chen and P.G. Voulgaris, “Decentralized design for integrated flight/propulsion control of aircraft,” AIAA Guidance Navigation and Control Conference, no. AIAA-98-4504, Boston, MA, Aug. 1998. To appear in AIAA JGDC. 6. K.R. Davidson, Nest Algebras, Longman, 1988 7. B.A. Francis. A Course in H∞ Control Theory, Springer-Verlag, 1987. 8. G.C. Goodwin, M.M. Seron and M.E. Salgado, “H2 design of decentralized controllers,” Proceedings of the American Control Conference, San Diego, California, June 1999 9. A.N. Gundes and C.A. Desoer, Algebraic Theory of Linear Feedback Systems with Full and Decentralized Compensators, Springer Verlag, Heidelberg, 1990. 10. M. Khammash, “Solution of the `1 optimal control problem without zero interpolation,” Proceedings of the CDC, Kobe, Japan, December 1996. 11. M.V. Salapaka, M. Khammash and M. Dahleh, “Solution of MIMO H2 /`1 problem without zero interpolation,” Proceedings of the CDC, San Diego, CA, December 1997. 12. C.W. Scherer, “From mixed to multi-objective control,” Proceedings of the CDC, Phoenix, Arizona, December 1999. 13. C.W. Scherer, “Lower bounds in multi-objective H2 /H∞ problems,” Proceedings of the CDC, Phoenix, Arizona, December 1999. 14. D.D. Sourlas and V. Manousiouthakis, “Best achievable decentralized performance,” IEEE Trans. A-C, Vol AC-40, pp. 1858-1871, 1995. 15. D.D. Siljak, Large-Scale Dynamic Systems, North-Holland, New York, 1978. 16. K.A. Unyelioglu and U. Ozguner, “H∞ sensitivity minimization using decentralized feedback: 2-input 2-output systems,” Systems and Control Letters, 1994. 17. M. Vidyasagar. Control Systems Synthesis: A Factorization Approach, MIT press, 1985.

Optimal Control Design under Structural Constraints

61

18. P.G. Voulgaris, M.A. Dahleh and L.S. Valavani, “H∞ and H2 optimal controllers for periodic and multirate systems,” Automatica, vol. 30, no. 2, pp. 252263, 1994. 19. P.G. Voulgaris, “Control under a hierarchical decision making structure,” Proceedings of the American Control Conference, San Diego, California, June 1999. 20. P.G. Voulgaris, “Control of nested systems,” Proceedings of the American Control Conference, Chicago, Illinois, June 2000. 21. P.G. Voulgaris, G. Bianchini and B. Bamieh, “Optimal decentralized controllers for spatially distributed systems,” 38th IEEE Conference on Decision and Control,, Sidney, Australia, December 2000. 22. G.H. Yang and L. Qiu, “Optimal symmetric H2 contollers with collocated sensors and actuators,” 39th IEEE Conference on Decision and Control,, Orlando, Florida, December 2001. 23. D.C. Youla, H.A. Jabr, and J.J. Bongiorno, “Modern Wiener-Hopf design of optimal controllers–part 2: The multivariable case,” IEEE-Trans. A-C, Vol. AC21, June 1976.

Fifteen Years of Quantum Control: from Concept to Experiment Anthony Peirce Department of Mathematics The University of British Columbia 121-1984 Mathematics Road Vancouver, B.C. V6T 1Z2 Canada

Abstract. In this paper I briefly survey some of the seminal contributions of Mohammed Dahleh in the application of optimal cotrol theory to quantum molecular systems, and attempt to give an overview of recent theoretical and experimental developments that build on this control paradigm.

1

Preamble

In this paper I wish to give an account of the seminal contributions of Mohammed Dahleh to the field of quantum molecular control. Prior to the recognition of the laser field design problem as a control problem, physicists and physical chemists had attempted to design laser fields based on physical intuition. However, due to the complex dynamics of molecules which are most accurately modeled by the Schrodinger equation, it is difficult if not impossible to arrive at intuitive field designs that will achieve the desired objective. By 1985 little progress had been made on the field design process. At that time I was an Applied Mathematics graduate student in the Program in Applied and Computational Mathematics at Princeton University and was working for a physical chemist, Hersch Rabitz, on reaction diffusion problems on catalytic surfaces. From Hersch I became aware of the laser field design problem and discussed this problem with Mohammed, who was a fellow student in the Applied Mathematics Program. Mohammed immediately recognized this problem as a control problem and we agreed to collaborate on a project to formulate the quantum molecular control problem as an optimal control problem, to explore issues of existence of controllers, and to validate the field designs by means of numerical experiments. This project culminated in the publication of our first paper using optimal control in the design of laser fields for quantum molecular control [8]. There followed a plethora of papers which make use of this methodology in the design of more complex molecules than those considered in the initial paper as well as a variety of cost functionals. Being a control theorist, Mohammed was plagued by the fact that the bilinearity of the control problem, which is legislated by the way in which the laser field acts on the state in the Schrodinger equation, ruled out the possibility of exploiting the results of linear systems theory - which by that time had reached a high level of maturity and sophistication. In addition, having the instincts of a control engineer, Mohammed was alarmed that the only control fields that could be envisaged at the time were of the open loop variety - given that the desired molecular dynamics was expected to be complete in hundreds of femtoseconds while observation and

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 65--72, 2003 Springer-Verlag Berlin Heidelberg

66

Anthony Peirce

feedback on this time-scale was impossible. Having ruled out the possibility of feedback design, Mohammed and I proceeded to investigate the possibility of designing open loop controllers that are robust to uncertainties in the molecular Hamiltonian and to perturbations in initial conditions (see [3]). Mohammed also championed an investigation of the controllability of finite level quantum systems using Lie Group methods (see [9]) and proceeded to stimulate an interest in this area in a number of his students (see [2]). In section 2 I outline the objectives of molecular control and in section 3 describe some of the earlier pre-control attempts at field design. In section 4 I briefly summarize the initial control formulation in which Mohammed’s contribution was vital and demonstrate this design procedure for the case of a diatomic molecule similar to the initial numerical experiments we performed. In section 5 I discuss the extension of this methodology to the design of fields that are robust to variations in initial conditions or to parameter fluctuations. In section 6 I describe a closed-loop design scheme and refer to some of the initial laboratory experiments that have made use of optimal control. I make some concluding remarks in section 7.

2

Molecular control

Since the beginning of alchemy one of the primary goals of chemists has been to stimulate chemical reactions to form desired products. Traditionally these stimuli were applied by changing the global thermodynamic variables such as the temperature and the pressure or by adding the appropriate combination of reagents to achieve desired products. In stimulating such chemical reactions it often happens that only a certain fraction of the reagents combine to form the desired products while the remaining reagents may combine to form a number of unwanted byproducts. In addition, there are products that cannot be produced by varying such global control variables. It is, therefore, desirable to search for alternative, more selective, and perhaps more efficient ways to stimulate chemical reactions. Neighboring atoms within molecules frequently have net opposite charges on them (the water molecule is a typical example), and the dipoles formed by such pairs of atoms act as microscopic “handles” on the molecules. Using applied electric fields it is possible to try to excite the molecules in a desired way. Another possible way to effect chemical reactions is to use stimulated molecular emission to prepare large quantities of molecules in selected states which are inaccessible by simple absorption. Although these new modes of stimuli offer the possibility of more selective excitation, their success depends on being able to determine the correct field to apply in order to achieve the desired objective.

3

Design by intuition

The idea of using electric or optical fields to achieve selective chemistry was not new when Mohammed first learned of the problem in 1985. Indeed, a great deal of research in this area had been done over the previous thirty years. Unfortunately, prior to the application of control theory by Mohammed, the field designs, which were often based on intuition, were largely unsuccessful. For example, if there was a need to break a particular bond within a molecule, then simple intuition would

Fifteen Years of Quantum Control

67

indicate that excitation at the frequency associated with that bond could induce a resonance which would ultimately break the bond. However, due to the coupling between the bond in question and the remainder of the molecule, it is extremely difficult to localize the energy imparted to the molecule within the bond. It is clear that the complicated dynamics and interference structure of the molecule have to be incorporated and perhaps even exploited in the field-design process. These initial attempts were largely unsuccessful in all but the simplest of objectives. Indeed, inspection of the complex structure of the required laser field designs, constructed using control theory, clearly illustrate the limitations of the intuitive field designs akin to attempting to play a complicated piano concerto with a single finger.

4

The optimal control formulation

By 1985 the field of molecular control was ripe for the introduction of techniques from systems theory. Mohammed’s contribution was vital to the introduction of the optimal control formalism in the field of molecular control. In this section I outline the initial formulation that was used in our 1988 paper [3]. Let the spatial domain be Ω ∈ Rn and consider control on a finite time horizon [0, T ]. Let X = L2 (Ω), Xt = L2 (Ω, [0, T ]), and XHS = the Hilbert Space of HilbertSchmidt Operators. The optimal control problem is prescribed by minimizing the following cost functional:

³ ´E ˆ ˆ J[U ] = ψ(·, T ) − ψ(·), Q ψ(·, T ) − ψ(·)

ZT

D

X

+β

hU, U iHS ds 0

subject to the dynamics of the Schrodinger equation with a molecular Hamiltonian H0 : dψ i = − (H0 + U )ψ, with ψ(x, 0) = ψ0 (x) dt } over all U ∈ XHS .Here ψˆ ∈ X is a specified reference state to which we wish to push R } the final wavefunction ψ(x, T ), and H0 = −2m ∇2 +V0, and U ψ = u(x, x 0 , t)ψ(x 0 , t)dx0. Ω

Introducing the Lagrange Multiplier function p(x, t) we minimize the Lagrangian:

L[U ; ψ, p] = J[U ] + Re

 T Z Z 

0 Ω

µ

i p ψ˙ + (H0 + U )ψ }

¶∗

  dxdt 

where ()∗ denotes the complex conjugate. By taking Frechet derivatives of L[U ; ψ, p] with respect to p, ψ, and U we obtain the following necessary conditions for a minimum:

68

Anthony Peirce dψ i = − (H0 + U )ψ, with ψ(x, 0) = ψ0 (x) dt } n o dp i ˆ FV P : = − (H0 + U )p, with p(x, T ) = 2 ψ(x) − ψ(x, T ) dt } ¶ ZT Z µ i ∗ 2βU − Re(p ψ ) δU dxdt Gradient : 0 = } IV P :

(1)

0 Ω

The initial-final value problems (1) form the basis for a numerical gradient numerical search procedure to locate a minimum. A monotonically convergent algorithm due to Krotov [10] is typically used to search for a minimum. By exploiting the lower semicontinuity of the norm, the weak closure of the unit ball in L2 , and the regularity of the solution it is possible to prove the following theorem (see [8]): Result 1. There exists a solution U ∈ L2 (XHS ; [0, T ]) and a corresponding ψ ∈ Xt that solves the optimization problem. In the following example we demonstrate the control design process for a simple diatomic molecule in which the molecular potential is assumed to be given by the Morse Potential: V0 (x) = D(1 − e−γ(x−x0 ) )2

√ We assume that } = 1, m = 2, D = 10, γ = 1/ 10, T = 18 and that the initial wavepacket is a Gaussian of unit width centered at x0 = 6 and that the target wavepacket is a Gaussian having the same shape but centered at x ˆ = 8, i.e.: ˆ T ) = g(x, 8, 1) ψ(x, 0) = g(x, 6, 1) and ψ(x, 0 2 1 −1 ) 0 −4 ) where g(x, x , l) = π l 2 exp(− (x−x 2l2 In this experiment we assumed that the laser field was of the form u(x, t) = E(t)B(x) = E(t)(x − x0 ) so that the dipole moment function B(x) is assumed to be linear. In figure 1 we provide a space-time contour plot of the probability density of the wavepacket |ψ(x, t)|2 juxtaposed with the corresponding electric field E(t) : We observe the complex structure of the field E(t) as well as the corresponding dynamics of the wavepacket as it makes its way to the target state. In figure 2 we provide a 3D plot of the same probability function which is close to the target Gaussian at time T = 18. This formalism was adopted by many subsequent researchers as they endeavoured to design fields to control more complex molecules. Many researchers also used this initial optimal control framework to explore molecular control using semiclassical and even classical molecular models. In principle, this formalism should be adequate to design laser fields for molecules containing any number of atoms. Unfortunately, the number of space dimensions n required in the solution of the Schrodinger equation grows with the number of atoms in a molecule. Thus field designs are severely limited by the computing resources required to solve the initial and final value problems.

Fifteen Years of Quantum Control 2

|ψ(x,t)|

18

14

12

12

10

10 t

14

t

16

8

8

6

6

4

4

2

2

0

5

Applied field

18

16

69

0

5

x

10

15

0 −0.4

−0.2

0

E(t)

0.2

0.4

0.6

Uncertainty and robust design

Due to the fact that the molecular Hamiltonians for large complex molecules are not known precisely there are likely to be considerable uncertainties in the molecular models used in the design process. Because of these uncertainties and the restriction to open-loop controllers, Mohammed was a strong protagonist for establishing a robust design methodology. In this section I briefly outline the extension of the previous optimal control framework to achieve robust field designs. One drawback of the optimal design approach used in the work described above is that these controllers are likely to be sensitive to uncertainties in the molecular Hamiltonian and in the initial state of the system. In order to achieve more robust field designs, averaged cost functionals corresponding to those described in the previous section have been considered (see [3]). In particular the optimal control problem involves minimizing the following cost functional: J[U ] = Eα,ψ0

hD

ZT ³ ´E i ˆ ˆ ψ(·, T ) − ψ(·), Q ψ(·, T ) − ψ(·) + β hU, U iHS ds X

0

70

Anthony Peirce

subject to: dψ i = − (H0 (α) + E(t)B(α))ψ, with ψ(x, 0) = ψ0 (x) dt } Here Eα,ψ0 [·] represents the expectation operator. It is possible to define a family of cost propagator operators which make it possible to perform explicit averaging of the cost functionals. These averaged cost functionals do indeed lead to field designs that are demonstrably less sensitive to perturbations in initial conditions or to fluctuations in the parameters of the molecular Hamiltonian.

6

Experiments and closed loop design

At the time of our initial control paper in 1988, the laser fields that could be prepared in the laboratory could only vary on time-scales of tens of picoseconds whereas our calculations indicated that the pulses required to effect the laser control had to vary on time-scales involving tens of femtoseconds. Thus at the time laboratory realization of laser fields to control molecular reactions seemed a remote goal awaiting the development of new technology. However, it did not take very long for the required technology to develop to the point that optimized femtosecond pulses were used to control molecular motion (see [5]).

Fifteen Years of Quantum Control

71

In a more recent development, laser controllers have been designed by various classes of evolutionary algorithms, which exploit the fact that millions of experiments can be performed in nanoseconds. In these search algorithms, more successful candidate fields are maintained within the candidate population and allowed to share their characteristics with other “more fit” laser fields to yield offspring fields for the next generation (see [4]). In particular a laser field of the following form is assumed: 1

¯ 2 /Γ 2 −iΦ(t)

E(t) = Ae− 2 (t−t)

e

¯ + ω(t − t¯) + , where Φ(t) = Φ

1 b(t − t¯)2 + . . . 2

The unknown parameters in this family of fields Γ, t¯, ω, b are sought via a genetic or evolutionary algorithm that exploits the huge number of experiments that can be performed without having to model or even characterize the precise dynamics of the molecules. At the heart of the success of this process, which is termed “learning control”, is an elementary form of feedback loop in the design process. Making use of this methodology, numerous experiments have been performed on relatively simple molecules (see [1]) and more recently on complex organic molecules (see [6]). We see that Mohammed’s instincts as a control theorist were correct. He believed that a practical control methodology, that would be experimentally viable, would have to incorporate some form of feedback loop. It is interesting that this form of feedback loop does not appear in the standard form associated with classical system theory in which real-time observation of the state is possible.

7

Concluding remarks

Today Quantum Control is an exploding field. This year there were more than five international conferences in Quantum Control. The field has developed along the path originally charted by pioneers like Mohammed. Indeed, the optimal design methodology is still being used to design laser fields for more complex molecules. Optimal controllers have been successfully employed in laboratory experiments. But it is interesting that many of the fundamental questions that Mohammed asked when he entered the field remain open problems today: e.g. a complete characterization of controllability for quantum systems; an input-output description that will make the system more amenable to analysis; a more comprehensive robust design methodology. The possible technological benefits of this research include: molecular scale surgery to create new molecules; purification of semiconductor materials by selective removal of impurities; super fast computer memory; unprecedented resolution of molecular scale dynamical processes for the extraction of fundamental forces between atoms; high density encoding and decoding of solid state electron wavepackets for transport of information; and more recently, the use of quantum control to construct the basic building-blocks for quantum computers. Mohammed’s insight provided the spark at the inception of this new area of application of control theory. Mohammed was quick to recognize the major hurdles to progress such as the bilinearity of the control problem, robustness, and open-loop designs. In his inimitable way he stimulated many others to become interested in

72

Anthony Peirce

the topic including some of his students who proceeded to develop many of the theoretical underpinnings of the field. The field is much richer for his profound insights and multiple contributions.

References 1. C.J. Bardeen, V.V. Yakovlev, K.R. Wilson, S.D. Carpenter, P.M. Weber and W.S.Warren, “Feedback Quantum Control of Molecular Electronic Population Transfer”,Chemical Physics Letters, 280, 151-158 (1997). 2. D. D’Alessandro and M. Dahleh, “Optimal control of two-level quantum systems”, IEEE Transactions on Automatic Control, Vol. 46, No. 6, June 2001, pg. 866-876. 3. M. Dahleh, A. Peirce, and H. Rabitz, “Optimal Control of Uncertain Quantum Systems”, Physical Review A, Vol. 42, No. 3, pp. 1065-1079, August, 1990. 4. R.S. Judson and H. Rabitz,“Teaching Lasers to Control Molecules”, Phys. Rev. Lett., 68, 1500 (1992). 5. B. Kohler, V.V. Yakovlev, J. Che, J.L. Krause, M. Messina, K. Wilson, N. Schwentner, R.M. Whitnell, Y. Yan, “Quantum control of wave packet evolution with tailored femtosecond pulses”, Physical Review Letters, 74, No. 17, 3360-3363, 1995. 6. R.J. Levis and H. Rabitz, “Closing the Loop on Bond Selective Chemistry Using Tailored Strong Field Laser Pulses”, J. Phys. Chem. A, 106, 6427-6444 (2002). 7. Y. Maday and G.Turiniciy, “New formulations of monotonically convergent quantum control algorithms”, 2002 (preprint). 8. A. P. Peirce, M. Dahleh and H. Rabitz, ”Optimal Control of Quantum Mechanical Systems: Existence, Numerical Approximations and Applications”, Physical Review A, Vol. 37, No. 12, pp. 4950-4964, 15 June, 1988. 9. V. Ramakrishna, M. Salapaka, M. Dahleh, H. Rabitz, and A. Peirce, ”Controllability of Molecular Systems,” Physical Review A, Vol. 51, No. 2, pp. 960-966, 1995. 10. D. Tannor, V. Kazakov, and V. Orlov, in Time Dependent Quantum Molecular Dynamics, edited by Broeckhove J. and Lathouwers L. (Plenum, 1992), pp. 347{360).

Directions in the Theory of Quantum Control Domenico D’Alessandro1 Department of Mathematics Iowa State University, Ames IA 50011, USA e-mail: [email protected]

Abstract. We survey a number of directions in the current research on the control of finite dimensional bilinear quantum systems. We describe the model as well as the role of Lie algebra theory in determining controllability properties. We also discuss techniques for constructive controllability of these systems.

1

Model of finite-dimensional quantum dynamics

The simplest example of a finite dimensional bilinear quantum system is a spin 21 particle driven by externally applied electro-magnetic fields. The state |ψ > of a particle with spin 12 , such as an electron, is described by a vector in a two dimensional vector space over the complex field. Denoting by |+ 21 > and | − 12 > the basis vectors, we have |ψ >= α| +

1 1 > +β| − >, 2 2

(1)

with |α|2 + |β|2 = 1. In an electro-magnetic field u, of components ux , uy and uz , constant in space but possibly varying with time, the state vector |ψ > evolves, according to the Schr¨odinger equation, as i

d |ψ(t) >= H(t)|ψ(t) > . dt

(2)

In this case, H(t) is given by H(t) := γ(σx ux (t) + σy uy (t) + σz uz (t)), where the constant γ is the gyromagnetic ratio of the particle and σx,y,z are the Pauli matrices (see e.g. [33] pg. 164), µ σx :=

01 10

µ

¶ ,

σy :=

0 −i i 0

µ

¶ ,

σz :=

1 0 0 −1

¶ .

(3)

In this scheme, the functions ux,y,z play the role of controls that have to drive the state |ψ > to a desired value. In many situations of practical interest, a particle with spin 12 is not isolated but interacts with other particles with spin 21 , as for example in a molecule. For a network of n particles, the underlying vector space has dimension 2n and a basis is given by |ej >:= | ± 21 , ± 12 , ...., ± 12 >, j = 1, 2, ..., 2n , with all the possible combinations of + and −. The state |ψ > of the network is a linear combination with complex coefficients of the basis states |ej >, namely

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 73--80, 2003 Springer-Verlag Berlin Heidelberg

74

Domenico D’Alessandro

P n P n |ψ >= 2j=1 αj |ej >, with 2j=1 |αj |2 = 1. The state vector evolves as in (2) and one possible (simplified) form for H(t) (see [13], [15]) is H(t) =

X

Jjk Ijk + (

n X

γj Ijx )ux (t) + (

j=1

j
n X

γj Ijy )uy (t) + (

j=1

n X

γj Ijz )uz .

(4)

j=1

In the previous expression, the first term models the interaction between different spin 12 particles. In particular Jjk is the coupling constant between particles j and k. The matrix Ijk is defined as the Kronecker product of n matrices, all of them are equal to the 2 × 2 identity matrix except the ones in the j and k position which are equal to the Pauli matrix σz . The second, third and fourth term model the interaction of the particles with an external magnetic field whose x, y and z components are given by ux , uy and uz (usually constant), respectively. The constant γj is the gyromagnetic ratio of the j−th particle (opportunely corrected to take into account chemical shifting see e.g. [38]). The matrix Ij(x,y,z) is the Kronecker product of n matrices that are all equal to the 2 × 2 identity matrix except the matrix in the j−th position which is equal to the Pauli matrix σx,y,z . This is a model for the dynamics of a network of spin 21 particles as, for example, in a chloroform molecule ([29] pg. 342) or any other molecule or cluster where particles with spin 21 are connected via chemical bonds. The previous examples are special cases of bilinear, finite dimensional quantum control systems. In general, the state of a quantum system, with a finite number of energy levels, n, is described by a state vector |ψ >, function of time. |ψ Pn> is a linear combination of n basis states, |ej >, j = 1, ..., n, so that |ψ >:= j=1 αj |ej >, P 2 where αj , j = 1, 2, .., n, are complex coefficients, with n |α | = 1. Because of j j=1 this restriction, |ψ > can also be seen as a point varying on the complex sphere of dimension 2n − 1. |ψ > varies with time according to the Schr¨odinger equation, i

d |ψ >= H(t)|ψ > . dt

(5)

H(t) is an n × n Hermitian matrix, function of time. In many experimental situations, as in the examples above described, H(t) has the form H(t) := H0 + P m k=1 Hk uk (t), where Hk , k = 0, 1, ..., m, are n × n Hermitian matrices, and uk (t), k = 1, ..., m, represent externally applied electro-magnetic fields which play the role of controls. The solution of (5) can be written as |ψ(t) >= X(t)|ψ(0) >,

(6)

where X(t), called the evolution matrix (or propagator), is the solution of the Scr¨ odinger equation, iX˙ = H(t)X,

(7)

with initial condition equal to the n × n identity matrix. Since Hk , k = 0, 1, ..., m, are Hermitian matrices, equation (7) can be rewritten as ˙ X(t) = AX(t) +

m X k=1

Bk X(t)uk (t),

X(0) = In×n ,

(8)

Directions in Quantum Control

75

with A, B1 , ..., Bm in the Lie algebra of skew-Hermitian matrices of dimension n, u(n) (or su(n) if A and the Bk ’s have zero trace) and X varying on the corresponding Lie group of n × n unitary matrices U (n) (or special unitary matrices SU (n)). The problem of control of quantum mechanical systems can be posed as a problem of control of the state |ψ >, in the description (5), or of the evolution matrix X in the description (7). The two settings are related through (6). Although |ψ > describes completely the state of a quantum mechanical system, considering X as the object of control can be more natural and convenient in a number of situations, for the following reasons: 1) Systems of the form (8) have been among the most studied in Geometric Control Theory (see e.g. [18], [39]) and therefore many tools developed there can be directly applied. 2) In applications such as quantum computing [29], the state |ψ > represents the information and the matrix X describes the operation to be carried out according to (6). Therefore it is a natural formulation, in this context, to pose the problem in terms of the matrix X, when the desired evolution has to perform a given (logic) operation. 3) In the density matrix formulation of quantum mechanics (which is necessary when the system under consideration is an ensemble of identical systems in different states see e.g. [5], [25]), the state is described by a Hermitian matrix ρ which evolves as ρ(t) = X(t)ρ(0)X ∗ (t), so that, once again, the evolution matrix X enters in a natural way. In mathematical terms, the relation between the control problems for systems (5) and (7) is the relation between a control system (7) varying on a transformation Lie group (the Lie group being SU (n) or U (n)) and the associated system (5) varying on the corresponding homogeneous space [27] (which in the case of (5) is the complex (2n − 1)−dimensional sphere).

2

Lie algebra structure, controllability and analysis of quantum systems

The first step, in the analysis of a control problem for a quantum mechanical system, consists of determining the Lie algebra L generated by {A, B1 , ..., Bm } in (8). This Lie algebra contains the information about the states that can be obtained, in principle, by an appropriate choice of the control functions and letting the system evolve for an appropriate amount of time. In fact, the set of states that can be obtained for (8), with a trajectory starting from the identity, is given by the connected Lie subgroup of U (n) corresponding to the Lie algebra L [17], [19], [30]. We denote this subgroup here by eL . Thus, if L = su(n) (or u(n)), the state X of (8) can be driven from the identity to every matrix in SU (n) (or U (n)), by an appropriate choice of the control functions. In this case the system is said to be Controllable. If the control problem is to transfer the state of (8) from the identity to a given Xf , we must have Xf ∈ eL for the problem to be well posed. If the control problem is to transfer the state |ψ > in (5) from the value |ψ0 > to the value |ψ1 > we must have |ψ1 > in the orbit O = {|ψ >∈ C I n ||ψ >= X|ψ0 >, X ∈ eL }.

(9)

76

DomenicoÿD’Alessandro

Notions of controllability for the state, the evolution matrix and the density matrix were studied in [2] where tests to check controllability, given a basis of the Lie algebra L were described. Every control algorithm, for a class of quantum systems, must take into account the nature of the underlying Lie algebra L. Furthermore, the knowledge of the structure of the Lie algebra L may help in understanding the dynamics and designing control algorithms. For example, the Cartan decomposition of the Lie algebra su(4) was used in [6] and [32] to devise control algorithms for the system of two interacting spin 21 particles with different gyromagnetic ratios (4) and in [21] to describe the minimum time controls for this system. For many quantum mechanical systems of interest, the Lie algebra L is not known. The determination of this Lie algebra for classes of quantum systems is one of the main questions in the theory of control of finite level quantum systems. Some work in this direction was done and some examples were treated in [34]-[37]. In [1], the example of networks of spin 12 particles was considered. In this paper, the nature of the Lie algebra L was related to the properties of a graph whose nodes represent the particles and an edge connects two nodes if and only if there is interaction between the two corresponding particles. There are also studies to infer whether or not given two matrices A and B, the Lie algebra generated by them is su(n) [3] and studies of controllability that do not rely on Lie algebra theory [40] [41]. For a given system one can always determine a basis of L by repeated Lie brackets of the matrices A,B1 , ..., Bm . If the dimension is not the same as the one of su(n), then L is a proper subalgebra of su(n). In general L may be any (possibly proper) Lie subalgebra of su(n) although for generic values of A, B1 , ..., Bm it will be su(n) itself. Therefore L may, in general, have an arbitrarily complicated structure. However, it is always possible to write L as a direct sum of vector spaces L = S1 ⊕ · · · ⊕ Sr ⊕ A,

(10)

where Sj , j = 1, ..., r are simple Lie Algebras, with [Sj , Sk ] = {0} when j = 6 k, and A is a solvable Lie Algebra. This fundamental result is known as Levi’s Theorem and is discussed for example in [11] to which we also refer for the terminology of Lie Algebras theory we have used. A particularly interesting case of the decomposition (10) occurs when Sj , j = 1, ..., r, and A commute, in which case the dynamics can be decomposed as |ψ(t) >= X1 (t)X2 (t)|ψ(0) > (0) = X2 (t)X1 (t)|ψ(0) >, ⊕rj=1 Sj

(11)

with X1 varying in the Lie group corresponding to and X2 varying in the Lie group corresponding to A. Further simplification occurs when A is Abelian. A simple example of the above decomposition occurs in the study of the dynamics of two noninteracting spins which is described by the Hamiltonian (4), with n = 2 and J12 = 0. In this case, the Lie algebra L is the sum of two copies of su(2), one for each spin. These are the simple Lie algebras S1 and S2 in (10). More specifically, a basis in the Lie algebra L is given by matrices of the form R ⊗ 1 and 1 ⊗ S with R, S ∈ su(2), and 1 the 2 × 2 identity matrix and ⊗ denoting the Kronecker product. We refer to [11] for numerical algorithms for decomposition and identification of Lie algebras.

Directions in Quantum Control

3

77

Methods for control

The approach to the control of quantum mechanical systems based on Lie Group Decompositions is very simply illustrated in the case of two-level systems (such as the spin 12 particle system discussed in Section 1). Consider system (8) with one control, X˙ = AX + BXu.

(12)

Here the matrix X has dimensions 2 × 2 and belongs to SU (2). A classical result, known as Euler’s decomposition (see e.g. [33]), says that every matrix Xf ∈ SU (2), can be written as Xf = eiσy t3 eiσz t2 eiσy t1 ,

(13)

where σy and σz are the Pauli matrices (3), and t1 , t2 , t3 are nonnegative parameters. Now, assume these parameters are known and assume that there exist two real values, u1 , u2 , such that A + Bu1 = iσy and A + Bu2 = iσz . Then, the control equal to u1 , for an interval of time t1 , u2 , for an interval of time t2 , and u1 again, for an interval of time t3 , gives Xf in (13) as the solution of (12), starting from the identity, at time t1 + t2 + t3 . The main tool in the above algorithm is a factorization of every matrix in SU (2) which involves the matrices in the equation (12). In fact, factorizations of elements of SU (2) exist involving any pair of linearly independent matrices in su(2) [7], [9], [24], [31]. Using this fact, a number of algorithms have been obtained for the control of two level quantum systems. Unfortunately, decomposition results are not as satisfactory nor as constructive for higher dimensional Lie groups. Some results for decompositions of higher dimensional Lie groups to be applied to quantum control systems were obtained in [22]. To illustrate an example of results for higher dimensional systems and to show that modularity is possible in this setting, we briefly review the main idea of the algorithm derived in [7]. This algorithm drives the state of a network of two spin 12 particles with equal gyromagnetic ratios. The system (cfr. Section 1) (if we assume a σz ⊗ σz type of interaction) has the form (7), (8) with m = 2 and H(t) in (7) is given by (4) with γ1 = γ2 . Setting for simplicity all the constants equal to 1 and assuming uz = 0 (which can be done if the system is written in an appropriate rotating frame (see e.g. [32])) we have that the system (8) has the form X˙ = AX + B1 Xu1 (t) + B2 Xu2 (t).

(14)

In (14), A := iσz ⊗ σz , B1 := i(σx ⊗ 1 + 1 ⊗ σx ) and B2 := i(σy ⊗ 1 + 1 ⊗ σy ). Here 1 denotes the 2 × 2 identity matrix and σx,y,z are the Pauli matrices (3) and ‘⊗’ denotes the Kronecker product. The Lie algebra generated by A, B1 and B2 can be shown to be isomorphic to u(3). Now, if we set u2 ≡ 0, the Lie subalgebra generated by A and B1 is isomorphic to u(2) and therefore to su(2), except for multiples of the identity matrix. The same thing can be said for A and B2 , if we set u1 ≡ 0. Therefore, looking at two different control systems on SU (2), we can apply the results above recalled concerning control using Lie group decompositions of SU (2). The matrices obtained this way are sufficient, when appropriately combined in products, to generate all the matrices in the underlying Lie group U (3) and therefore a control algorithm has been obtained to drive the state of this system to any configuration.

78

DomeniocoÿD’Alessandro

Optimal Control is perhaps the most developed system theoretic method to drive the state of control systems in a desired way. This methodology has been successfully applied in the past to systems of interest in robotics [28]. These systems have mathematical similarity with the ones considered here in that the control variable multiplies the state variable. The formulation using optimal control allows the consideration of further constraints, such as the requirement for the actual trajectory to be close to a given one or for the state transfer to occur in minimum time. One possible approach is based on the application of the Pontryagin Maximum Principle (see e.g. [14]) for systems varying on Lie groups [4]. This allows us to restrict the candidate controls to two classes of functions, referred to as normal and abnormal extremals. There are two main technical problems in the explicit determination of the optimal controls for systems in the form (8) (or (5)): 1) The classification of normal and abnormal extremals has been obtained so far only for low dimensional systems [8], [10] and it becomes more difficult as the dimension of the problem increases (and the number of the available controls decreases). 2) The ‘tuning’ of the parameters in the candidate optimal control functions can be very difficult, if not impossible unless more structure is taken into account. Numerical algorithms for the design of steering controls for quantum mechanical systems have been proposed for example in [16], [42]. Other techniques developed for the control of general bilinear systems, such as averaging [23] may be applied to quantum systems. However, many of these techniques assume a driftless system which is rarely the case for quantum mechanical systems (see e.g. (4) where the interaction term contributes to the drift). Therefore extensions need to be carried out to the case of system with drift. Some work in this direction for averaging based techniques was done in [26].

4

Conclusions

We have surveyed a number of research directions in the analysis and control of finite dimensional quantum mechanical systems in a bilinear form. This include the analysis of the Lie algebra and Lie group underlying the system as well as the development of control algorithms. All the theory described here deals with closed systems namely ideal quantum systems that do not interact with the environment. The interaction with the environment causes a departure of the behavior of the system from the nominal one which is referred to as de-coherence [12]. In some special cases the system can be driven through Decoherence Free Subspaces [20] so that there is no influence of the environment on the nominal evolution. In most cases, though, the interaction with the environment is unavoidable and the methods for control described above have to be modified in order to minimize such interaction.

References 1. F. Albertini and D. D’Alessandro, The Lie algebra structure and controllability of spin systems, Linear Algebra and its Applications, Vol 350, 1-3, pp. 213-235, 2002. 2. F. Albertini and D. D’Alessandro, Notions of controllability for quantum mechanical systems, preprint http://arXiv.org, quant-ph 0106128.

Directions in Quantum Control

79

3. C. Altafini, Controllability of quantum mechanical systems by root space decomposition of su(n), preprint, arXiv: quant-ph/0110147. 4. J. Baillieul, Geometric methods for nonlinear optimal control problems, Journal of Optimization Theory and Applications, Vol. 25, No. 4, August 1978, pg. 519548. 5. K. Blum, Density Matrix Theory and Applications, Plenum Press, New York, 1981. 6. D. D’Alessandro, Algorithms for quantum control based on decompositions of Lie groups, in Proceedings 39-th conference on Decision and Control, Sidney, Australia, Dec. 2000, pg. 967-968. 7. D. D’Alessandro, Controllability of one spin and two homonuclear spins, preprint http://arXiv.org, quant-ph 0106127, to appear in Mathematics of Control, Signals and Systems. 8. D. D’Alessandro, The optimal control problem on SO(4) and its applications to quantum control, IEEE Transactions on Automatic Control, vol. 47, No. 1, January 2002, pp. 87-92. 9. D. D’Alessandro, Optimal evaluation of generalized Euler angles with application to classical and quantum control, preprint, arXiv: quant-ph/0110120. 10. D. D’Alessandro and M. Dahleh, Optimal control of two-level quantum systems, IEEE Transactions on Automatic Control, Vol. 46, No. 6, June 2001, pg. 867877. 11. W. A. de Graaf, Lie Algebras, Theory and Algorithms, North-Holland Mathematical Library, 56, Amsterdam, 2000. 12. D. P. Di Vincenzo, E. Knill, R. Laflamme and W. Zurek eds. Quantum Coherence and Decoherence, Proceedings of the ITP Conference held at the University of California, Santa Barbara, CA, 1996, Royal Society London, 1969, pp. 257-486, 1998. 13. R. R. Ernst, G. Bodenhausen and A. Wokaun, Principles of Nuclear Magnetic Resonance in One and Two Dimensions, Oxford University Press, 1987. 14. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, Applications of Mathematics, Springer Verlag, New York, 1975. 15. M. Goldman, Quantum Description of High-Resolution NMR in Liquids, Oxford University Press, 1988. 16. S. Grivopoulos and B. Bamieh, New iterative algorithms for approximate controllability of bilinear systems with applications to quantum control, Proceedings of the 41-st Conference on Decision and control, Las Vegas, NV, Dec 2002. 17. G. M. Huang, T. J. Tarn and J. W. Clark, On the controllability of quantum mechanical systems, J. Math. Phys. 24, 11, November 1983, pg. 2608-2618. 18. V. Jurdjevi´c, Geometric Control Theory, Cambridge University Press, 1997. 19. V. Jurdjevi´c and H. J. Sussmann, Control systems on Lie groups, Journal of Differential Equations, 12, 1972, 313-329. 20. J. Kempe, D. Bacon, D. A. Lidar and K. B. Whaley, Theory of decoherencefree fault-tolerant universal quantum computation, Preprint, arXiv:quantph/0004064. 21. N. Khaneja, R. Brockett, S. J. Glaser, Time Optimal Control of Spin Systems, Physical Review A, 63, 2001, 032308. 22. N. Khaneja and S.J. Glaser, Cartan Decomposition of spin systems and universal quantum computing, preprint, http://arXiv.org, quant-ph 0010100.

80

DomenicoÿD’Alessandro

23. N. E. Leonard and P. S. Krishnaprasad, Motion control of drift free, leftinvariant systems on Lie groups, IEEE Transactions on Automatic Control 40, No. 9, 1539-1554. 24. F. Lowenthal, Uniform finite generation of SU (2) and SL(2, R I ), Can. J. Math., Vol. XXIV, No. 4, 1972, pp. 713-727. 25. G. Mahler and V. A. Weberruss, Quantum Networks, Dynamics of Open Nanostructures, Springer Verlag, Berlin Heidelberg, 1998. 26. S. Martinez, J. Cortes and F.Bullo, On Analysis and Design of Oscillatory Control Systems, Preprint June 2001, submitted. 27. D. Montgomery and L. Zippin, Topological Transformation Groups, New York, Interscience Publishers, 1964. 28. R. Murray, Z. Li and S. Sastry, A Mathematical Introduction to Robotic Manipulation, Boca Raton: CRC Press, 1994. 29. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press, 2000. 30. V. Ramakrishna, M. V. Salapaka, M. Dahleh, H. Rabitz, A. Peirce, Controllability of molecular systems, Physical Review A, 51, 2, 1995, 960-966. 31. V. Ramakrishna, K. L. Flores, H. Rabitz, R. J. Ober, Quantum control by decompositions of SU (2), Physical Review A, 62,053409, 2000. 32. V. Ramakrishna, R. J. Ober, K. L. Flores and H. Rabitz, Control of a coupled two spin systems without hard pulses, preprint, http://arXiv, quant-ph 0012019. 33. J. J. Sakurai, Modern Quantum Mechanics, Addison-Wesley Pub. Co., Reading, Mass. 1994. 34. S. Schirmer, H. Fu and A. Solomon, Complete controllability of quantum systems, Physical Rev. A 63 (2001) 063410, arXiv quant-ph/0010031. 35. S. G. Schirmer, H. Fu amd A. I. Solomon, Complete controllability of finite level quantum systems, J. Phys. A, 34 (2001) 1679, arXiv quant-ph/0102017. 36. S. G. Schirmer, A. I. Solomon and J. V. Leahy, Degrees of controllability for quantum systems and application to atomic systems, preprint, arXiv:quantph/0108114 37. S. G. Schirmer, I. C. H. Pullen and A.I. Solomon, Identification of dynamical Lie algebras for finite-level quantum control systems, preprint, arXiv quantph/0203104. 38. C. P. Slichter, Principles of Magnetic Resonance, Berlin; New York: SpringerVerlag, 1990. 39. H. J. Sussmann, Lie brackets, real analyticity and geometric control, in Differential Geometric Control Theory, R. W. Brockett, R. S. Millman and H. J. Sussmann eds., pp. 1-116, Birkhauser, Boston, 1983. 40. G. Turinici, Controllable quantities in bilinear quantum systems, Proceedings of the 39-th IEEE Conference on Decision and Control, Sydney, Australia, Dec. 2000, pp. 1364-1369. 41. G. Turinici and H. Rabitz, Quantum wave function controllability, Chem. Phys., (267), 1-9, (2001) 42. W. Zhu and H. Rabitz, A rapid monotonically convergent iteration algorithm for quantum optimal control over the expectation value of a positive definite operator, Journal of Chemical Physics, vol. 109, no. 2, p.385, 1998. 43. Quantum Information and Computation, Vol. 1, No. 1, Special Issue on Quantum Entanglement, July 2001.

System tools applied to micro-cantilever based devices A. Sebastian1 , S. Salapaka2 , and M. V. Salapaka1 1 2

Iowa State University, Ames IA 50014, USA Laboratory for Information and Decision Systems, MIT, MA 02139, USA

Abstract. Micro-cantilever based devices can be used to investigate and manipulate matter at atomic scales. Taking the case study of atomic force microscope (AFM) we demonstrate the power of system tools in the analysis of micro-cantilever based devices. They capture important characteristics and predict inherent limitations in the operation of these devices. Such a systems approach is shown to complement the physical studies performed on these devices. Tractable models are developed for the AFM operating in tapping-mode. For the interrogation of samples, it is also imperative that sample positioning should be done with high precision and at high speeds. This broadband nanopositioning problem is shown to fit into the modern robust control framework. This is illustrated by the design, identification and control of such a positioning device.

1

Introduction

Desirable properties of manufactured products arise from the manner in which atoms are arranged in its material. Until recently, ways of manipulating and interrogating matter were limited to aggregate methods where the control and investigation of matter was achieved at scales much larger than atomic scales. Recent demonstrations made possible by micro-cantilever based devices provide ample evidence indicating the feasibility of rational control, manipulation and interrogation of matter at the atomic scale. Micro-cantilevers have been utilized in biological sciences in a variety of applications like sensing sequence-specific DNA [9], studying cell-cell interactions [4] and antigen/antibody interactions [7]. Another intriguing application is in the detection of single electron spin ([13,20–22]). Such research has significant ramifications for quantum computing technology and to the physics at atomic scales. For more description of the impact of micro-cantilever based devices see [24]. In spite of the vast underlying promise, considerable challenges need to be overcome to fully harness the potential of this technology. A key element of the microcantilever based technique is the manner in which the cantilever interacts with the matter it is investigating or manipulating. Although micro-cantilever based devices have been utilized ubiquitously in various applications, the dynamics of the cantilever-sample interaction is significantly complex and considerable research effort is being placed at deciphering this interaction. Most of the research effort

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 83--99, 2003 Springer-Verlag Berlin Heidelberg

84

A. Sebastian etÿal.

is towards developing sophisticated and often involved physical models of the tipsample interaction ([1], [6]). This has helped provide insights into the various physical sources of the tip-sample interaction force. However these approaches do not lend themselves towards isolating the principle characteristics and limitations of microcantilever based devices. A systems based approach significantly complements the physical studies performed on the tip-sample interaction, provides new ways of interpreting data, new ways of imaging and indicates the fundamental limitations of micro-cantilever based devices. Such an approach has been missing from the literature until the work of the authors. We highlight these studies in this article. Another pivotal requirement for harnessing the vast potential of micro-cantilever based technology is ultra-fast positioning. To achieve high throughput, fast positioning is imperative. It is also becoming increasingly evident that for many nanotechnological studies, high bandwidth is a necessity. For example, in the field of cell biology, attractive proposals on using nanotechnology have been presented where nano-probes track events in the cell. These events often have time-scales in the micro-second or nano-second regimes. Current nanopositioning technology does not meet the needed high precision and bandwidth requirements. The modern robust control paradigm offers a powerful tool to address the challenges of broadband nanopositioning. We will highlight how such an approach has yielded considerable dividends in this case. The Atomic Force Microscope (AFM) is one of the primary micro-cantilever based devices. In the next section we give a brief overview of the AFM.

2

Atomic Force Microscope: operating principles and features

The schematic of a typical AFM is shown in Figure 1. It consists of a microcantilever, a sample positioning system, a detection system and a control system. A typical cantilever is 100 − 200 µm long, 5 − 10 µm wide and has a tip of diameter ≈ 5 nm. Most of the cantilevers are micro-fabricated from silicon oxide, silicon nitride or pure silicon using photolithographic techniques. The cantilever deflects under the influence of the sample and other forces. This deflection is registered by the laser incident on the cantilever tip which reflects into a split photodiode. This setup constitutes the detection system. The minimum detectable cantilever deflection is in the order of 0.01 nm. Using the measured deflection signal, the control system moves the sample appropriately to achieve necessary objectives. Sample positioning is usually provided by a piezoelectric based positioning stage. Since its invention in 1986 a wide range of modes of operations have emerged. In contact mode or static mode operation, the cantilever deflection is primarily due to the tip-sample interaction. The deflection of the tip is used to interpret sample properties. In the tapping-mode or dynamic mode operation, the cantilever support is forced sinusoidally using a dither piezo. The changes in the oscillations introduced due to the sample are interpreted to obtain the sample properties [24]. Central to the operation described above is the micro-cantilever which largely determines the achievable sensitivity and resolution of the AFM. It is thus important to identify the cantilever parameters precisely. For many applications the micro-cantilever is well modeled as a flexible structure. P The displacement of the cantilever at time t denoted by p(t) is given by p(t) = ∞ k=1 ξk qk (t) where ξk is the

System tools applied to micro-cantilever based devices

85

Fig. 1. A typical setup of an AFM. The chief components are the micro-cantilever, a sample positioning system, an optical detection system and a feedback controller.

mode shape evaluated where the laser is incident on the cantilever. Using only a finite set of the mode shapes ξk , one can show that the state space description of the dynamics is given by, x˙ = Ax + B1 η + BF p(t) =

N X

(1)

ξk qk (t) = Cx

k=1

¢T ¡ ˙ , η is the thermal noise forcing term. F dewhere x := q1 q2 .. qN q˙1 q˙2 .. qN scribes all the external forces acting on the cantilever. A, B1 and B are functions of the mass, damping and stiffness of the cantilever. p(t) is available as a measured signal using the detection scheme shown in Figure 1. Methods are available to obtain a precise description of the multi-mode model using the thermal noise response (see [14]). In typical applications a firstmode approximation is sufficient.

3

Systems approach to the analysis of AFM dynamics

The tapping-mode operation is the most common method of imaging primarily because it is less invasive on the sample and has higher signal to noise ratio. In this mode, due to the cantilever oscillation the tip moves over a long range of highly nonlinear tip-sample potential leading to complex behavior. The complexity of the dynamics can be assessed by the experimental and theoretical studies that confirm the existence of chaotic behavior (see [2], [3], [5] and [16]). However under normal operating conditions the cantilever is found to evolve into a stable periodic orbit with a period equal to the period of forcing. It is also experimentally observed that when the tip-sample separation is sufficiently large, the periodic orbit is near sinusoidal.

86

A. Sebastian etÿal.

Much of the research on tapping-mode dynamics has revolved around solving nonlinear differential equations numerically (see [1]). As mentioned earlier even though they predict experimentally observed behavior, they fail to capture the limitations and important characteristics in the operation of tapping-mode AFM. Also the identification of model parameters is not straightforward due to their complexity. In this article we highlight a systems perspective of the tapping-mode

Fig. 2. (a) The feedback perspective of dynamic mode AFM dynamics. G corresponds to the linear cantilever model. φ is a nonlinear model for the tip-sample interaction. (b) A typical plot of tip-sample interaction, φ Vs. position, p.

dynamics. A detailed version of this development is provided in [18], [19] and a preprint. The cantilever can be imagined to be a system that takes in as inputs the dither signal, g(t) and the tip-sample interaction force, h(t). It produces the deflection signal, p(t) as the output. (see Figure 2(a)). Let G denote such a block. The state space description of G is given by (1). By ignoring the noise forcing and recognizing F = g(t) + h(t), from (1) we obtain, x˙ = Ax + B(h(t) + g(t))

(2)

p = Cx The tip-sample interaction forces now appear as a feedback block. In this perspective the instantaneous tip position is fed back to the system G through the function φ. In this way we view the tapping-mode dynamics as an inter-connection of two systems, the system G that models the cantilever and the block φ that models the sample.

3.1

Analysis of the periodic solutions

Given the complex dynamics of tapping-mode AFM, the questions on the existence and stability of periodic solutions are very relevant. As a first application of the systems viewpoint, the existence and stability of periodic solutions in tapping-mode AFM are explored. Sector bounds on the attractive part of the tip-sample interaction φ (see Figure 2(b)) can be computed for typical tip-sample interactions. Coupled with such

System tools applied to micro-cantilever based devices

87

a characterization, it is possible to describe φ in terms of integral quadratic constraints (IQC) which capture how the nonlinearity distributes the input signal’s energy over various frequencies (see [10] for a comprehensive treatment of IQCs). ¶ µ Π11 Π12 Definition 1. φ is said to satisfy the IQC defined by a multiplier Π = Π21 Π22 if µ ¶ ¶∗ Z ∞ µ yˆ(jω) yˆ(jω) Π(jω) dω ≥ 0 (3) ˆ ˆ φ(y)(jω) φ(y)(jω) −∞ for all y ∈ L2 . We denote this by φ ∈ IQC(Π). In the above definition yˆ corresponds to the Fourier transform of y. As a first step towards establishing the existence and stability of periodic solutions we prove the global exponential stability of x = 0 for the dynamics described by equation (2) with g(t) = 0. A computationally attractive way of proving the global exponential stability is using the IQCs. Assuming that Π11 ≥ 0 and Π22 ≤ 0, the global exponential stability can be concluded if φ ∈ IQC(Π) and ∃ ² > 0 such that, µ ¶ ¶∗ µ (jωI − A)−1 B (jωI − A)−1 B ˜ Π(jw) ≤ − ²I ∀ ω (4) I I ¶ µ T T ˜ = C Π11 C C Π12 . From the global exponential stability the converse where Π Π21 C Π22 Lyapunov theorem can be invoked to obtain a Lyapunov function W (x) which when evaluated along the trajectories of the forced system satisfies, • W (x) → ∞ as |x| → ∞ • There exists ξ > 0 and a continuous function a(x) > 0 for kxk ≥ ξ such that d (w(x(t)) ≤ −a(x(t)). for any solution kx(t)k ≥ ξ, dt From the existence of the above W (x), it can be shown that (2) has a solution x0 (t) bounded for −∞ < t < ∞ ([25]). Let x(t) be another solution of (2). Let x ˜(t) = x(t) − x0 (t). From (2) we obtain, ˜ y˜), p˜ = C x x ˜˙ = A˜ x + B φ(t, ˜

(5)

˜ v) := φ(v + p0 (t)) − φ(p0 (t)). The global exponential stability of (5) where φ(t, implies the stability of the solution x0 (t) which can be established by searching for a Π such that φ˜ ∈ IQC(Π) and (4) is satisfied. Using the fact that the forcing is periodic and from the uniqueness of the solution we can conclude the periodicity of x0 (t). As mentioned earlier the experimentally observed periodic orbit is nearly sinusoidal. It could be argued heuristically that the sharp bandpass characteristic of the cantilever subsystem leads to smaller higher harmonics and hence a near sinusoidal orbit. Using the IQCs, one can provide rigorous bounds on the higher harmonics. If these bounds are shown to be significantly smaller than the first harmonic, then it can be concluded that the periodic solution is almost sinusoidal. Compare this approach with Ref. [8] where the author evaluates the interaction forces only for the first harmonic which amounts to assuming sinusoidal nature of the periodic solution. The following theorem from Ref. [11] is used to obtain bounds on the higher harmonics.

88

A. Sebastian etÿal.

Result 1. In Figure 2 let g(t) = g1 ejω0 t +g−1 e−jω0 t , p(t) = P hk ejkω0 t . If

P

pk ejkω0 t and h(t) =

1. µ φ satisfies the µ by Π ¶ ¶∗ IQC defined G(jkω0 ) G(jkω0 ) 2. Π(jkω0 ) ≤ −² for all |k| 6= 1. 1 1 Then for |k0 | 6= 1, the bound |pk0 | < β|g1 | holds for all β that together with some τ > 0 satisfies the inequality     0 0 0 K1 L 1 0 ∗ 2 (6) 0 >  0 −β 0  + τ  L1 M1 0  0 0 Kk0 0 0 1 where µ ¶ µ µ ¶∗ ¶ 1 0 1 0 Kk L k Π(jkω ) = 0 L∗k Mk G−1 (jkω0 ) −1 G−1 (jkω0 ) −1 For each higher harmonic of p(t), we can solve the Linear Matrix Inequality (LMI) given by (6), and obtain β. An upper bound on the harmonic is obtained by multiplying β by the magnitude of forcing. Note that the bounds on the higher harmonics can also be used to assess the limitations on how well the tip-sample potential can be probed using the micro-cantilever. A more detailed exposition of the above approach is provided in Ref. [18] where the existence, stability and near sinusoidal nature of periodic solutions are explored under certain operating conditions. Also the existence and local orbital stability of periodic orbits are studied in Ref. [15] for a one-mode approximation of the cantilever.

3.2

Harmonic analysis based identification of tip-sample interaction

In certain cases, the existence and stability of periodic solutions can be proven in a rigorous manner ([18], [15]). In this section the assumption that tapping-mode dynamics permits the existence of a periodic solution with the same period as that of forcing is used to derive the harmonic balance equations. Let such a periodic solution be denoted by p0 (t) and let it have a period of T . Assuming the nonlinear force on the cantilever due to the sample is time invariant it followsP that h(t) is also jkωt , T periodic. Thus p0 , h and g admit expansions of the form p0 (t) = ∞ k=−∞ pk e P∞ P∞ jkωt jkωt h(t) = k=−∞ hk e and g(t) = k=−∞ gk e where ω = 2π/T . Since the cantilever model G is assumed to be linear time invariant the input and output harmonics of the system are related by, G(jkω)(gk + hk ) = pk , for all k = 0, ±1, ±2, ... The steady state periodic orbit of the cantilever depends on the forcing frequency ω, the magnitude of forcing γ, and tip-sample separation l. hk and pk are functions of γ, ω and l. By varying one of these parameters we can evaluate hk for different values of γ, ω or l. This in turn can be used to identify tip-sample interaction forces. One approach to identifying the tip-sample interaction is to assume a parametric model. Let H(θ) denote such a model where θ is a finite set of parameters. If Hk are the Fourier coefficients of H(θ)(p(t)), then the estimation of the parameters reduce to solving the minimization problem, min θ

∞ X k=0

|Hk − hk |2 .

(7)

System tools applied to micro-cantilever based devices

3.3

89

Experimental results

An AFM was operated in the tapping-mode. A silicon cantilever of 225 µm was used. The model parameters were evaluated by analyzing the response to thermal noise as mentioned earlier. The sample (silicon wafer) was sufficiently far from the cantilever initially. Once the oscillations reached the steady state the sample was moved towards the cantilever. At different values of tip-sample separation, the motion of the cantilever tip was recorded after the tip settles into a new periodic orbit. A piecewise linear model was assumed for the tip-sample interaction as depicted in Figure 3. It is motivated by the long range attractive forces and short range repulsive forces typically observed. Using the tools described earlier, it is possible to estimate the model parameters from the experimental data (see [19]). The estimated parameters were used to simulate the AFM operating in tapping-mode. The corresponding results were compared with those obtained experimentally. As predicted by the analysis earlier, the higher harmonics were insignificant compared to the first harmonic. In Figures 4 and 5 the amplitude and phase of the first harmonic of the resulting oscillations at different points of separation are compared with that obtained experimentally. The plots show good agreement between the experiments and the simulations. It is to be noted that a simple tip-sample interaction model can predict the cantilever behavior. This can be explained from the observation that even if the tip-sample interaction has finer features, their effect get filtered out due

Fig. 3. A first-mode approximation of the cantilever model and a model for the tip-sample interaction. The negative spring models the attractive forces and the positive spring models the repulsive forces. The dampers account for the energy dissipation due to sample interaction.

90

A. Sebastian etÿal. 25

Dotted line −−− Simulation Solid line −−− Experiment

24

Amplitude in nm

23

22

21

20

19

18

17 16

18

20

22

24

Separation in nm

26

28

30

Fig. 4. Using the estimated parameter values the AFM operation is simulated and the results are compared with that observed experimentally. Here the amplitude is plotted against the tip-sample separation.

−40

Dotted line −−− Simulation Solid line −−− Experiment

−50

Phase in degrees

−60

−70

−80

−90

−100

−110

−120 16

18

20

22

24

Separation in nm

26

28

30

Fig. 5. This plot depicts the good match obtained in the phase values.

to the sharp band-pass characteristic of the cantilever. This imposes a limitation on the detection of interaction forces using tapping-mode AFM. From the above analysis it is evident that the systems perspective provides unique insights which may not be obtained from a physical model. In the next section we deal with the nanopositioning aspect of micro-cantilever based devices.

4

Broadband nanopositioning

We have seen that nanopositioning systems form a very important part of microcantilever based devices. Most of these systems are piezoactuated. Piezoelectric materials achieve repeatable nanometer and sub nanometer resolutions at relatively high bandwidths. This is possible since they have no moving parts and thereby avoid undesirable effects such as backlash and stick-slip motions. Also, they can generate large forces (as high as few tens of kN ), have very fast response times (acceleration rates of 104 g can be obtained), are not affected by magnetic fields,

System tools applied to micro-cantilever based devices

91

and are operable at wide ranges of temperatures (they are functional even at near zero Kelvin temperatures). However, obtaining high positioning precision is significantly complicated or even unachievable due to nonlinear effects like hysteresis and creep. This is true especially when the piezoactuators are used in relatively long range positioning applications or for extended periods of time. Most of the commercially available devices circumvent these nonlinear effects at the cost of their performance by restricting the devices to low drive applications where the behavior is approximately linear and/or limiting their motions to specific trajectories for which nonlinear effects have been accounted for and appropriately compensated. In this section, we present the design and modeling of a nanopositioning device that we developed in collaboration with Asylum Research 1 . This device is also piezoactuated and therefore inherits all the advantages and disadvantages mentioned earlier. A modern robust control approach is taken in the design of the controller. This section is a condensed form of Ref. [17].

4.1

Device Description

The device consists of four components: a base plate which seats the nanopositioning system and the sample holder, an AFM head that seats the sample-probing system a top plate that seats the AFM head and a control system.

Fig. 6. (a) The base plate of the flexure stage. (b) The exploded view of the flexure and evaluation stages. (c) A schematic to show the serpentine design of the base plate.

The base plate consists of three main subcomponents: A flexure stage which has a small scan stage on which the sample is placed (see Figure 6). It is a 20 cm × 1

Santa Barbara, CA 93117

92

A. Sebastian etÿal.

20 cm × 5 cm steel plate with a cylindrical block in the center to hold the sample. This sample holder moves with respect to the periphery of the plate by virtue of the serpentine spring design. A piezo-stack arrangement (with a voltage amplifier) forms the actuation system. It is seated in a slot in the base plate. The amplified voltage signal applied across the stack leads to its deformation, which in turn imparts motion to the flexure stage. This actuation system is relatively cheap and provides a good travel (of about 70 µm) but has poor hysteresis and creep properties compared to commercially available J-scanners that are predominantly used. A Linear Variable Differential Transformer (LVDT) constitutes the detection system. It is seated in a slot in the base plate on the opposite side of the actuation system. An AFM head (described in the previous section) consisting of a microcantilever and associated laser/optical arrangement forms the sample probing system. The top plate is of similar dimensions as a base plate and is there to provide support to the AFM head above the sample (see Figure 6(b)). A block diagram of the nanopositioning system including the control system is shown in Figure 7. Here the transfer function G(s) refers to an identified model of the system that comprises of the actuation, flexure and detection stages of the device. The reference signal xr , the feedback law K(s) and the prefilter F (s) were implemented on a Texas Instruments TMS320C44 digital signal processor based development platform.

Fig. 7. A schematic block diagram of the closed loop system.

4.2

Identification and control design

The model G(s) was obtained from the frequency response about the null position by giving a sine sweep of small amplitudes (less than 50 mV ) up to a bandwidth 2 KHz . A fourth order non minimum phase transfer function: G(s) = 9.7×104 (s−(7.2±7.4i)×103 ) yielded a good fit to this response. (s+(1.9±4.5i)×103 )(s+(1.2±15.2i)×102 ) We obtained frequency responses at different operating points (other than the null position) and also studied how these responses changed by changing the amplitudes of the sine sweep signal. It was observed that there are considerable changes in the frequency responses with the set points and amplitudes of sine sweep signals. This emphasized the importance of designing feedback laws that are robust to model uncertainties. Another important aspect of this model is the presence of

System tools applied to micro-cantilever based devices

93

the RHP zeroes. This can be explained as the actuation and detection systems are not collocated but instead are separated by a flexure stage. The inverse response behavior which is a characteristic of flexible structures and non minimum phase systems was observed in experiments which further corroborates the model. The RHP zeros rule out high gain proportional control laws and impose a fundamental limit on the achievable bandwidth of the closed loop system. In the presence of a complex pair of RHP zeros (z1,2 = x ± iy), the “ideal” controller can achieve a sen4xs which leads to a limit on the achievable sitivity function of, S = (s+x+jy)(s+x−jy) bandwidth [12,23] of 415Hz for our model.

Traditional controller design Before proceeding it is worthwhile to indicate the control tools presently employed by the AFM industry. The most widely used microscope (Multimode, Digital Instruments) do not have feedback loops for lateral positioning. Most other variants do not employ model based control designs. The most prominent technique is the PI technique where the PI gains are left to the user to set and is often a source of confusion. As PI is the most commonly used controller, we have analyzed the PI controller’s performance. The optimized parameter values of kp = 0.01 and ki = 75 guarantees a gain margin of 1.57 and a phase margin of 89◦ while giving a bandwidth of only about 2.12Hz. H∞ controller design While designing feedback controllers for nanopositioning systems, robustness and resolution assume great importance besides high bandwidth. Feedback controllers should be robust to nonlinear effects like hysteresis and creep. The nanopositioning devices currently in use have good noise characteristics that lead to high resolution. While trying to achieve good robustness and bandwidth it is imperative that the feedback controller does not deteriorate the resolution of the device. This tradeoff is very well captured by the sensitivity S and complementary sensitivity T transfer functions. Low complementary sensitivity gains imply lesser effect of noise (since e = xr − x = T n) and therefore higher resolution; similarly, low sensitivity gains imply better tracking of reference signals (since e = Sxr ) and therefore higher bandwidth. In this way the tradeoff between the shapes of the sensitivity and complementary sensitivity functions determines the tradeoff between the resolution and the bandwidth of the device. It becomes important to design control laws that appropriately shape these closed loop transfer functions. Thus the problem of designing controllers for nanopositioning devices fits well into the modern robust control framework. Also the positioning problem in X, Y and Z directions could be addressed in a MIMO framework. In the H∞ design, an optimization problem to obtain desirable shapes for the closed loop transfer functions is solved (see [23] for a good exposition to this design methodology). The objectives were posed in an H∞ design framework to obtain a controller K such that ° ° ° W1 S ° ° ° °  W2 T  ° ° ° ° W3 KS °

<γ ∞

94

A. Sebastian etÿal.

for some prespecified γ > γopt , the optimal value. The controller was designed (using the function, hinfsyn in Matlab) for γ = 2.415 and the weighting functions Wi , i = 1, 2, 3 were chosen as follows: W1 was chosen to be a first order transfer function, designed so that its inverse (an approximate upper bound on the sensitivity function) has a gain of 0.1% at low frequencies (< 1 Hz) and a gain of ≈ 5% around 200 Hz. This weighting function puts a lower bound on the bandwidth of the closed loop system and ensures good tracking at these frequencies. Similarly, the complementary sensitivity function, T was shaped by choosing the weighting function such that its reciprocal has low gains at high frequencies and vice versa. T was made to roll off at a frequency of 400 Hz. This diminishes the effect of noise on the error signal thus providing a handle on the resolution of the device. We also shaped the KS transfer function (by choosing W3 = 0.1) so as to limit the control effort (since the control signal u is given by u = KSxr ). This was important to avoid saturating the piezoactuator (the stack-piezo saturates if it is outside the −10 V to 0 V range).

4.3

Characterization of the device

Figure 8 compares the bandwidth obtained by the PI, H∞ designs and the ideal achievable bandwidth described earlier in the section. The H∞ design improves the bandwidth by over sixty times when compared to the PI design. Also it provides a powerful paradigm for nanopositioning devices with a straightforward way of a prespecified tradeoff between bandwidth and resolution. 20 10

Magnitude (dB)

0 −10 −20 −30 −40

−3dB line ISE optimal H

−50

∞

PI 415.1Hz 138.26Hz 2.24Hz

−60 −70 0 10

1

10

frequency (Hz)

2

10

Fig. 8. Sensitivity functions obtained via the “ideal” controller, H∞ controller and PI controller.

The device was calibrated by placing a calibration sample (which had 180 nm deep grooves every 5µm) on the sample-holder and probing it by the AFM head. A triangular input of amplitude 2 V was given and the resulting LVDT output

System tools applied to micro-cantilever based devices

95

showed the presence of 7.26 grooves. This implies the device has a static sensitivity of 18.15 µm/V . It was seen that the input voltage of approximately 4 V can be given without reaching the limits of the actuator. This guarantees a travel range of 70µm. As described earlier the accuracy of positioning using piezoactuators is greatly reduced due to nonlinear effects of hysteresis and creep. However, with H∞ design, these effects are compensated and the closed loop device shows minimal hysteretic and creep effects. Figure 9 compares the hysteresis curves for open and closed loop devices. The hysteresis effects which are predominant in large scans are as much as 10% (for full range) in the open loop device while they are nearly eliminated in the closed loop configuration. This also emphasizes the robustness of the closed loop design. Even though the controllers were designed for a nominal linear model obtained about a point (null position) in the operating range, the closed loop system works remarkably well even for scans that span the whole travel range of the device. 30

position (µm)

20

(b)

max. o/p hyst = 10.14% max. i/p hyst = 9.58%

10

0

−10

−20

−30 −4

30

20

position (µm)

(a)

10

0

−10

−20

−2

0

input (V)

2

4

−30

−20

0

input (µm)

20

Fig. 9. Hysteresis in (a) the open loop configuration, (b) its elimination in the closed loop configuration.

Creep is another undesirable effect that becomes prominent in long time experiments. It is related to the effect of the applied voltage on the remnant polarization of the piezo ceramics. The piezoactuator starts drifting even at no application of any input signal to it. To measure this effect, we studied a step response of the device in open and closed loop configurations. We see that the output in the open loop case responds to the reference signal (see Figure 10(a)) but instead of reaching a steady state value it continues to decrease at a very slow rate. The response y(t) was found to approximately satisfy the creep law with a creep factor of 0.55 The same experiment conducted in the closed loop (see Figure 10(a and b)) shows that the feedback laws virtually eliminate this effect and the system tracks the reference signal nearly exactly.

96

A. Sebastian etÿal. 0.5

σ = 0.84nm

position (µm)

0

(with H∞ controller)

−0.5

open loop −1

−1.5

−2 0

p(t)=−0.43(1+0.55log(2t))

10

20

30

40

50

60

time (s)

Fig. 10. The elimination of creep in the closed loop system and its approximation by a creep law.

These effects manifest themselves in lack of repeatability of the device. This is presented in Figure 11. Plot (b) shows the forward and backward scans put back to back of a calibration sample (shown in plot(a)) by using the device in the open loop configuration. This sample has equispaced grooves at every 5 µm. However, in the scans that we obtain, we see several inconsistencies: neither the grooves are equispaced, nor are they of same widths and also the forward and backward scans are not aligned. These mismatches can lead to gross errors and can be very misleading if these images are averaged to remove the effects of noise on the sample surfaces. In contrast, in the closed loop scan (see plot (c)), there is no mismatch and the forward and backward scan images match very well with the calibration sample.

5

Conclusions

Clearer understanding of tip-sample interactions and high speed, high precision sample positioning are essential for the interrogation and manipulation of materials using micro-cantilever based devices. A systems perspective is found to be very useful in addressing these challenges. The AFM dynamics is presented as an interconnection of a linear system with a nonlinear feedback. The analysis of tapping-mode AFM dynamics reduces to the study of such a system being forced sinusoidally. This approach helps obtain tractable models for the tip-sample interaction and predicts the inherent limitations in the interrogation of sample using tapping-mode AFM. It is hoped that the systems perspective can be further exploited to develop observer based faster imaging schemes.

System tools applied to micro-cantilever based devices

97

Fig. 11. (a) The reference (calibration sample) geometry. The grooves are separated by 5 µm and has a depth of 180 nm. (b) The mismatch in the position of grooves between the forward and the backward traverses in the open loop. (c) a good match in the closed loop configuration.

Broadband nanopositioning problem is shown to fit into the framework of modern robust control. In section 4, the design, identification and control of a positioning device is described.

Acknowledgements This work is dedicated to Prof. Mohammed Dahleh. We would like to acknowledge the NSF grants: CAREER ECS-9733802 and CMS-0201560.

References 1. B. Anczykowski, D. Kruger, K. L. Babcock, and H. Fuchs. Basic properties of dynamic force spectroscopy with scanning force microscope in experiment and simulation. Ultramicroscopy, 66:251, 1996. 2. M. Ashhab, M. V. Salapaka, M. Dahleh, and I. Mezic. Dynamical analysis and control of micro-cantilevers. Automatica, 1999. 3. M. Ashhab, M. V. Salapaka, M. Dahleh, and I. Mezic. Melnikov-based dynamical analysis of microcantilevers in scanning probe microscopy. Nonlinear Dynamics, November 1999.

98

A. Sebastian etÿal.

4. M. Benoit, D. Gabriel, G. Gerisch, and H. E. Gaub. Discrete interactions in cell adhesion measured by single-molecule force spectroscopy. Nature Cell Biology, 2(6):pp. 313–317, 2000. 5. N. A. Burnham, A. J. Kulik, G. Gremaud, and G. A. D. Briggs. Nanosubharmonics: the dynamics of small nonlinear contacts. Physics Review Letters, 74:5092–5059, 1995. 6. N. A. et. al. Burnham. How does a tip tap? Nanotechnology, 8:pp. 67–75, 1997. 7. U. Dammer, M. Hegner, D. Anselmetti, P. Wagner, M. Dreier, W. Huber, and H. J. Gntherodt. Specific antigen/antibody interactions measured by force microscopy. Biophysical Journal, 70:pp. 2437–2441, 1996. 8. U. Durig. Conservative and dissipative interactions in dynamic force microscopy. Surface and Interface Analysis, 27:pp. 467–473., 1999. 9. J. Fritz, M. K. Baller, H. P. Lang, H. Rothuizen, P. Vettiger, E. Meyer, H. J. Gntherodt, Ch. Gerber, and J. K. Gimzewski. Translating biomolecular recognition into nanomechanics. Science, 288:pp. 316–318, 2001. 10. A. Megretski and A. Rantzer. System analysis via integral quadratic constraints. IEEE Transactions on Automatic Control, 47, no.6:pp. 819–830., 1997. 11. A. Megretski and A. Rantzer. Harmonic analysis of nonlinear and uncertain systems. In Proceedings of the American Control Conference, Philadelphia, June 1998. Pensylvania. 12. M. Morari and E. Zafiriou. Robust Process Control. Prentice-Hall, Englewood Cliffs, 1989. 13. D. Rugar, C. S. Yannoni, and J. A. Sidles. Mechanical detection of magnetic resonance. Nature, 360:563–566, (1992). 14. M. V. Salapaka, H. S. Bergh, J. Lai, A. Majumdar, and E. McFarland. Multimode noise analysis of cantilevers for scanning probe microscopy. Journal of Applied Physics, 81(6):2480–2487, 1997. 15. M. V. Salapaka, D. Chen, and J. P. Cleveland. Linearity of amplitude and phase in tapping-mode atomic force microscopy. Physical Review B., 61, no. 2:pp. 1106–1115, Jan 2000. 16. S. Salapaka, M. Dahleh, and I. Mezic. On the dynamics of a harmonic oscillator undergoing impacts with a vibrating platform. Nonlinear Dynamics, 24:pp. 333–358, 2001. 17. S. Salapaka, A. Sebastian, J. P. Cleveland, and M. V. Salapaka. High bandwidth nano-positioner: A robust control approach. Review of Scientific Instruments, accepted 2002. 18. A. Sebastian and M. V. Salapaka. Analysis of periodic solutions in tappingmode afm: An IQC approach. In International sysmposium on Mathematical Theory of Networks and Systems, Notre Dame, IN, August 2002. 19. A. Sebastian, M. V. Salapaka, D. J. Chen, and J. P. Cleveland. Harmonic and power balance tools for tapping-mode atomic force microscope. Journal of Applied Physics, 89, no.11:6473–6480, 2001. 20. J. A. Sidles. Noninductive detection of single proton-magnetic resonance. Appl. Phys. Lett., 58(24):2854–2856, 1991. 21. J. A. Sidles. Folded stern-gerlach experiment as a means for detecting nuclear magnetic resonance of individual nuclei. Phys. Rev. Lett., 68:1124–1127, 1992. 22. J. A. Sidles, J. L. Garbini, and G. P. Drobny. The theory of oscillator-coupled magnetic resonance with potential applications to molecular imaging. Rev. Sci. Instrum., 63:3881–3899, 1992.

System tools applied to micro-cantilever based devices

99

23. S. Skogestad and I. Postlethwaite. Multivariable Feedback Control, Analysis and Design. John Wiley and Sons, 1997. 24. R. Wiesendanger. Scanning Probe Microscopy and Spectroscopy. Cambridge University Press, 1998. 25. V. A. Yakubovic. The matrix-inequality method in the theory of the stability of nonlinear control systems. Avtomatika i Telemekhanika, vol.25, no.7:1017– 1029, 1964.

Micro-scale sensors and filters utilizing non-linear dynamic response of single and coupled oscillators Kimberly Turner1 , Rajashree Baskaran1 , Wenhua Zhang1 Department of Mechanical and Environmental Engineering University of California, Santa Barbara, CA 93106

Abstract. Microelectromechanical systems (MEMS) provides a unique platform in which to study and utilize unique nonlinear effects. In this paper, we show two examples of nonlinear behavior utilized to improve device characteristics. The first example is a MEMS-based mass sensor, capable of resolving minute changes in mass for chemical detection. The second example is a system of electrostatically coupled MEMS resonators functioning as a band-pass filter. Both examples show how nonlinearities and unique dynamics can improve device functionality. In addition, by understanding the nonlinear behavior, it can also be avoided when a linear response is desired.

1

Introduction

Microelectromechanical systems are used in many applications. In this paper, we discuss two Novel applications of MEMS which utilize nonlinear dynamics. In the first section, a mass sensor based on tracking parametric instabilities in MEMS is presented, and in the second section, signal filters are shown, which utilize coupled MEMS oscillators. Both are examples of how an understanding of nonlinear mechanics is essential to build and design microsystem applications. Novel mass sensors based on parametric resonance can achieve high sensitivity because of the sharp transition from zero to large response. Unlike harmonic resonance based mass sensors, the sensitivity of this novel class of sensors will not be as affected by damping. However, the presence of nonlinearity in the spring and electrostatic force of the oscillator significantly change the behavior of parametric resonance. In this paper, a nonlinear Mathieu equation is used to examine the effects of nonlinearity. A two-variable method is utilized to analyze this equation. The effects are validated by experiments. Using MEMS based on-chip designs to substitute for off-chip or low Quality factor components of communication devices has been a driving force in MEMS research. We present a new design of electrostatically coupled dual-oscillator MEMS filter. Such an implementation offers advantages over other filter designs in terms of tunability of the response. Filter characteristics obtained from motion measurements are presented here.

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 101--112, 2003 Springer-Verlag Berlin Heidelberg

102

2

Kimberly Turner etÿal.

MEMS mass sensor

Many MEMS based mass sensors detect mass changes by measuring shifts in resonance frequency. To implement high sensitivity, micro-scale oscillators are promising due to their small mass and high quality factor. Some cantilever mass sensors have been developed and been applied as chemical sensors, biosensors and other sensors as well [1]–[3]. We have previously introduced the framework of designing a novel parametrically driven mass sensor with higher sensitivity than normal cantilever sensor . Here, we present a detailed analysis and experimental verification of effects of cubic nonlinearity on a parametrically driven MEMS oscillator. The presence of nonlinearity in recovery spring and electrostatic force alters the dynamic behavior of parametric resonance dramatically. In the case of harmonic oscillator with time-modulated stiffness, parametric resonance exists at some specific frequency [4]. The transition from zero response (stable state) to large response (unstable state) can be very sharp [5]. In a torsional oscillator, 0.001Hz has been observed in experiment [5]. Since the transition frequency depends on system parameters, including mass, small mass change in this system can be detected by measuring the shift of transition frequency. The novel mass sensing system consists of a mechanical oscillator, with electrostatic actuator and sensor. A special comb finger configuration actuates the oscillator and tunes the stiffness of the system as well. When a square root AC voltage is applied on the actuator, the first parametric resonance can be excited at twice the natural frequency. The smallest detectable mass change depends on transition step size and paramdω eters of system and electrical signal (dm ∝ dω 3 , ω ∼ ω0 ). To improve the system sensitivity, higher stiffness and smaller mass is warranted. Signal-to-noise ratio is another important issue in sensor design. To be able to obtain good signal, larger amplitudes are desired, implying that nonlinearity in spring and electrostatic force cannot be ignored. The device we have studied is an oscillator, which was designed by Adams for the independent tuning of linear and cubic stiffness terms [6]. It is fabricated using SCREAM process [7]. A Scanning Electron Micrograph of the oscillator is shown in Figure 1. The device covers about 500 × 400µm2 . It has two sets of parallel interdigitated comb finger banks on either end of the backbone and two sets of non-interdigitated comb fingers on each side. The four folded beams provide elastic recovery force for the oscillator. The beams, backbone and the fingers are 2 microns wide and ∼ 12 microns deep. Either the interdigitated or the non-interdigitated comb fingers may be used to drive the oscillator. In this study, non-interdigitated comb fingers are used to drive the oscillator.

2.1

Theoretical Analysis

Using Newton’s laws, the 1-D motion of the device can be described by the following equation: m

dx d2 x +c + Fr (x) = Fe (t, x) dt2 dt

where x is displacement, m is mass and c is damping.

(1)

Micro-scale sensors and filters

103

Fig. 1. A Scanning Electron Micrograph of the oscillator. Note the folded beam springs (S), the two sets of interdigitated comb finger banks (C) on both end of backbone (B) and non-interdigitated comb fingers (N) on each side of backbone (B). Figure 1(a) and (b) are schematic of interdigitated comb fingers and noninterdigitated comb fingers.

Considering nonlinearity, the recovery force can be expressed as: Fr (x) = k1 x + k3 x3

(2)

where, k1 and k3 are linear stiffness and cubic stiffness respectively. 1 When a square root AC voltage signal (V = (cos(ωt)) 2 ) being applied on the non-interdigitated comb fingers, Fe is a function of displacement x and time t, [6]. Fe (x, t) = −(r1 VA2 x + r3 VA2 x3 ) cos(ωt)

(3)

Substituting Fe and Fr in equation (1) and normalizing it: 0 d2 x dx +α + (β + 2δ cos(2τ ))x + (δ3 + δ3 cos(2τ ))x3 = 0 dτ 2 dτ

4(k+r V 2 )

2r V 2

4k +4r V 2

0

4r V 3

1 A 3 3 A 3 A 1 A 2c , δ = mω , δ3 = mω ,β= where α = mω 2 , δ3 = 2 . mω 2 mω 2 This is a nonlinear Mathieu Equation. All the terms in this equation except β (which is of order 1) is of small order. Hence, we can use perturbation methods to study this system. By assuming δ = ², β = β0 + ²β1 , the equation can be rewritten as a harmonic oscillator with a perturbation (the terms on the right hand side). 0 d2 x dx + β0 x = ²(−(β1 + 2 cos(2τ ))x + (γ3 + γ3 cos(2τ ))x3 − µ ) dτ 2 dτ

(4)

where the terms in equation (4) have been scaled with respect to the small term ², 0

δ

0

γ3 = δ²3 , γ3 = ²3 and µ = α² . We know that for a linear Mathieu equation, the first order parametric resonance occurs when the driving frequency is near twice the resonant frequency (here that would correspond to β0 = 1). So, we use the method

104

Kimberly Turner etÿal.

of two variable expansion [8] to analyze the equation when driving near the first parametric resonance. The idea of the method is that the expected solution involves two time scales: the time scale of the periodic motions and a slower time scale that modulates the amplitude of the periodic motion. Here, we use the notation that ξ represents stretched time (ξ = ωt), and η represents slow time (η = ²t), [8]. Assuming x = x0 + ²x1 and β = 1 + ²β1 and using the two variable expansion method, the equation of motion can be rewritten as two equations: 4

∂ 2 x0 + x0 = 0 ∂ξ 2

(5)

4

0 ∂x0 ∂ 2 x0 ∂ 2 x1 + x1 = −4 − (β1 + 2 cos(ξ))x0 − (γ3 + γ3 cos(ξ))x30 − 2µ ∂ξ 2 ∂ξ∂η ∂ξ

(6)

It should be noted that we are working with first order expansion and O(²2 ) terms are neglected here. The solution to the harmonic oscillator in x0 is as follows, ξ ξ x0 = A(η)cos( ) + B(η)sin( ) 2 2

(7)

Unlike, the simple harmonic oscillator case, we have the “constants” A and B varying in slow time. Using this solution to evaluate the right hand side of the equation in x1 and setting the condition for removal of the resonant terms, yields the ’slow-flow’ equations in A and B (the dynamics of A and B in slow-time). 0

3Bγ3 2 γ B3 dA µ B = − B + (β1 − 1) + (A + B 2 ) − 3 dη 2 2 8 4

(8)

0

γ A3 3Aγ3 2 µ A dB = − A + (β1 + 1) − (A + B 2 ) − 3 (9) dη 2 2 8 4 The characteristics of these two equations are schematically shown in A-B plane, see Figure 2 (a). The plane can be divided into three areas. In area I, one center exists at (0, 0) which means only one stable trivial solution exists in the area. In area II, there are two centers at (±a1 , 0) and one saddle at (0, 0), corresponding to one stable nontrivial solution and one unstable trivial solution. In area III, there are two centers at (±a2 , 0), two saddles at (0, ±b) and one center at (0, 0), corresponding to one stable nontrivial solution, one unstable nontrivial solution and one stable trivial solution. For clarity of discussion, we present the influence of damping and nonlinear term on parametric resonance separately.

Effect of Damping (assuming no cubic nonlinearity) Equation (8,9)

can be simplified as

dA µ B = − + (β1 − 1) dη 2 2

(10)

dB µ A = − A + (β1 + 1) dη 2 2

(11)

Micro-scale sensors and filters

105

From Equations (10,11), the transition curve from stable to unstable areas in the first parametric resonance can be found [8]: p p (12) β = 1 ± ² 1 − µ2 = 1 ± ²2 − α2

Effect of cubic nonlinearity (assuming no damping) We look at the slow flow equations in polar coordinates here, and assume A = R cos(θ) and B = R sin(θ). Equation (8) and (9) can be modified as: dR R = − sin(2θ) dη 2

(13)

β1 3γ3ef f R2 1 dθ =− − − cos(2θ) dη 2 8 2

(14)

2γ

0

where γ3ef f = γ3 + 33 cos(2θ) is defined as effective nonlinear parameter. We study the equilibrium (fixed) points of the slow-flow variable (R∗ , θ∗ ) and analyze their stability characteristics. The equilibrium points are when the right hand side of the above equations is identically zero. For non-trivial solutions of R, equilibrium points are 0, π2 , π, 3π θ∗ = 2 4 (β1 + cos(2θ∗ )) R∗2 = − 3γ3ef f

(15)

Let us assume effective nonlinearity parameter γ3ef f < 0. In the case of θ∗ = 0 4 (β1 + 1), nontrivial solutions require β1 > −1. When and θ∗ = π, R∗2 = − 3γ3ef f

4 (β1 − 1), nontrivial solutions exist only for , R∗2 = − 3γ3ef θ∗ = π2 and θ∗ = 3π 2 f β1 > 1. The characteristics of equation (15) are schematically shown in Figure 2 (b). Since β1 = ±1 corresponds to transition curves from stable to unstable areas in β-δ plane, bifurcation occurs when we quasi-statically vary frequency of the input voltage across the transition curve. The growth of R∗ with respect to β in different areas is schematically shown in figure 2(b), where solid line represents stable solution and dashed line represents unstable solution.

2.2

Experimental Verification

A multi-dimensional MEMS motion characterization suite is used to measure the in plane movement of the device [9]. The device is placed in a vacuum chamber, where the pressure can be pumped to as low as 7mT orr. The motion is measure using a Laser Doppler Vibrometer through an optical microscope. A square root AC voltage signal is applied on the non-interdigitated comb fingers. By sweeping the driving frequency around twice the natural frequency, the frequency response of the first parametric resonance can be captured. Figure 3 is a velocity response of the device with driving voltage VA = 20V . The frequency is swept from low to high and high to low. Different step sizes are used in the sweeping. When sweeping the frequency from high to low, the device becomes unstable at about 52.7kHz. But when sweeping from low to high, the jump from unstable to stable occurs around 53.7kHz. And this jumping point depends on the step size of frequency. The smaller the step size, the larger the transition frequency (points S and R).

106

Kimberly Turner etÿal.

Fig. 2. Dynamic characteristics of Nonlinear Mathieu Equation in the β − δ plane. β = 1 ± δ are the transition curves, which divide β - δ plane p into area I, II, III. Note the damping effects on transition curves (β = 1 ± δ 1 − µ2 ) in figure (a). The number of stable points change in each area - one center at (0, 0) in area I, two centers at (±a1 , 0) and one saddle at (0, 0) in area II, three centers at (±a2 , 0) / (0, 0) and two saddles at (0, ±b) in area III. Figure (b) shows how the positions of the stable and unstable points vary as β and δ are varied quasi-statically.

2.3

Discussion

According to the analysis of Nonlinear Mathieu equation, first order parametric resonance as represented in the β-δ plane (which translates to VA − ω coordinates) may be categorized into three areas, I, II and III, see Figure 2. The corresponding A-B plane is shown in the same figure and shows stability characteristics. In area I, only one trivial solution exists. As the frequency is changed quasi-statically, a bifurcation occurs at the left boundary (β = 1 + δ). In area II, the trivial solution becomes unstable and simultaneously a stable sub-harmonic motion is born. This motion grows in amplitude as d increases. In the right boundary (β = 1 − δ), the unstable trivial solution becomes stable again and an unstable sub-harmonic motion is born. The stable sub-harmonic nontrivial solution born in area II also exists in area III. The experimentally obtained displacements exactly verify all the characteristics expected from the analysis. Figure 3 presents typical frequency response in the three zones described above. In area I, the response is very small, which is a stable trivial solution. In area II, a large response exists, corresponding to the stable nontrivial solution, while the unstable trivial solution cannot be found. In area III, one large and one small response can be found, corresponding to one stable trivial solution and one stable nontrivial solution, while the unstable nontrivial can not be observed. Analysis predicts that, depending on the initial displacement and velocity, the final displacement converges to one of the two stable states. When sweeping frequency from high to low, the initial value will be small and hence the trivial stable solution is obtained. But when sweeping the frequency from low to high, in small step sizes, the response will stabilize at the nontrivial stable solution. As we keep increasing the driving frequency, the large response will jump to zero at some frequency beyond the right transition boundary. Sweeping the driving frequency with different sep sizes, the jump point can be different. For large step

Micro-scale sensors and filters

107

Fig. 3. Experimental data of frequency-amplitude curves as the input frequency is swept quasi statically near first order parametric resonance (twice the natural frequency and β = 1) at VA = 20V . The figure can be divided into three areas, I, II and III (Refer Figure 2 for the distinction). Point P is in the right transition curve (β = 1 + δ) from area II to I and Q is in the left transition curve (β = 1 − δ) from area III to II. Note there are two experimental responses in area III, corresponding to the two stable solutions. The points R and S are in region III where the response jumps from the large amplitude stable response to the trivial solution.

size sweeps, the response jumps to the trivial stable solution at a lower frequency, see point R in Figure 3. If we sweep the driving frequency with very small steps, the jump will happen at a higher frequency, see point S in Figure 3. We theorize that the step size causes a perturbation in the initial conditions, thereby affecting the frequency at which the jump occurs. It is worthwhile to note here that, the difference in response between sweeping driving frequency in either direction (increasing or decreasing values) is due to the bifurcations occurring at the boundaries. This response appears like a system “hysteresis”, but in fact is a distinctly different dynamic behavior. The jump is very critical in the application as a mass sensor. To sweeping driving frequency in different manners, many jumps can be observed. For most of the jumps, the frequency where jump happens depends more on sweeping manners than system parameters. However, only one jump happens at the right transition boundary when we sweep the driving frequency from high to low. The jump point is very stable and depends only on the system parameters. This implies that to detect mass changes, we must sweep driving frequency from high to low in our case (γ3ef f < 0). One distinguishing feature of parametrically driven mass sensor is that the sensitivity is independent of damping. The presence of damping alters the shape of the transition curve (from ’V ’ to ’U ’), but the sharp transition still exists, even

108

Kimberly Turner etÿal.

in the air. From the analysis and experimental results, we note that the effect of the damping in the system is to shift the instability ’tongue’ from one that looks like a ’V ’ to one that looks like a ’U ’, see figure 2. This implies that there’s a minimum input voltage above which the transition will take place. The damping does not affect the sensitivity of the mass sensor, but just introduces a constraint to the input voltage signal amplitude, unlike the sensors based on detecting natural frequency shifts, such as normal cantilever sensors. This is a very critical feature of the sensor since often the quality factor cannot be controlled with precision or made very large in test situations.

2.4

Conclusion

This work presents theoretical analysis and experimental tests of nonlinear effects on parametric resonance. As shown by theory, presence of structural and electrostatic nonlinearity in a MEMS oscillator changes the behavior of parametric resonance significantly. These effects are validated by experimental results. Based on the analysis and experiments, an appropriate working mode for this type of mass sensor is found. The sensitivity of the novel mass sensor will be nearly independent of damping. To improve the sensitivity, new design and fabrication are underway.

3

Electrostatically Coupled Oscillator filter

There have been many demonstrations of single oscillator filter designs and coupled MEMS oscillators filters [10] and [11]. The significant difference in the model presented here is that the coupling between the two mechanically isolated oscillators is electrostatic (Figures 4, 5). The basic design and fabrication of the coupled oscillators has been reported elsewhere [12]. The performance of the oscillator system as a MEM filter is reported here.

Fig. 4. Schematic of the electrostatically coupled oscillator system. The features are 1micron wide and the overall area is of the order of 0.14mm2

Micro-scale sensors and filters

109

Fig. 5. Scanning Electron micrograph of the detail of the coupling region and the drive comb finger configurations. F C: Fixed comb fingers, B: Backbone and T : Torsion bar.

There are advantages in implementing the MEMS based filter using this design. A linear system approach analysis shows that the coupling mechanism’s compliance [strength] controls both the bandwidth and shape factor of the filter response [13]. Since the coupling strength is a function of the externally applied voltages, it a controllable parameter in this design. Additionally, by simultaneously implementing the frequency tuning concept [6], it allows for tunability of all the filter parameters through various input signals. Since any batch fabricated MEMS process involves some process specific variations from design values, the idea of total tunability eliminates the need for trimming using laser etching or deposition and hence can be performed after packaging as well.

3.1

Experimental Procedure

The electrostatically coupled oscillators were made using the single mask bulk micromachining process, SCREAM [7]. The actuation is electrostatic and the motion is in the out-of plane direction due to fringing field effects. The design could be implemented on-chip with a comb finger based capacitive transduction from mechanical motion to electrical signal. To demonstrate the proof of concept, the characterization was done using a high accuracy (∼ 4nm @ the filter frequency range) laser vibrometry measurement system [9] which measures displacement and velocity of the moving device. The MEMS device is mounted on a Joule Thomson refrigerator [MMR Technology] inside a vacuum chamber. The temperature was controlled using a controller interfaced with the refrigerator. The experimental set up is schematically shown in Figure 6. It should be noted that since the transduction from mechanical motion to observed signal was not capacitive, the filter characteristics cannot be directly compared to that of a mechanical filter in a circuit.

110

Kimberly Turner etÿal.

Fig. 6. Schematic of the experimental set up used to measure real time displacement and velocity data of the moving oscillator

3.2

Experimental Verification

The measured attenuation [defined here as 0dB at maximum amplitude of the oscillators and 20dB at half power points] as a function of frequency of the twocoupled oscillator system is shown in Figure 7. This data was taken at input voltage of 20V pk − pk square rooted sinusoidal signal, 10mT orr and 300K. A fractional bandwidth of 1.1% and shape factor of 3.13 was seen at the above-mentioned experimental conditions. Table 1. Experimental Characteristics of the Filter tested from Figure 7 Characteristics Experimental value Central Frequency (kHz) [f] 181.05 3db Bandwidth (khz) [BW3] 1.99 20db Bandwidth (khz) half power point [BW20] 6.23 Fractional Bandwidth [BW3/f] 1.1% Shape factor [BW20/BW3] 3.13

In addition, preliminary data for the effect of temperature on the frequency shift is presented in Figure 8. An increasing positive shift is expected due to the negative Young’s modulus temperature co-efficient. The initial negative shift could be attributed to chamber pressure being influenced by the cooling initially. The decreasing chamber pressure would reduce the damping and increase the frequency. The effect of water from the ambient air possibly condensing on the devices (there by changing the effective mass of the oscillator) at freezing point could also be the reason for the initial decreasing shift.

Micro-scale sensors and filters

111

Fig. 7. Experimental passband characteristics of the MEMS filter at 10mT orr, 300K, 20V pk − pk voltage.

) input as a function of temperature Fig. 8. Fractional frequency shift ( ∆f f

3.3

Conclusion and Future work

We have presented the characterization of a novel design of coupled oscillator system for MEMS based filter applications. This design has many advantages over mechanically coupled filter designs.The filter characteristics have been obtained from motion amplitude data. Preliminary temperature effects on frequency have been

112

Kimberly Turner etÿal.

studied and more detailed work on temperature effects are under investigation. An in-plane motion and a multi-coupled oscillator design are also under fabrication.

References 1. Lang, H.P., et al. A chemical sensor based on a micromechanical cantilever array for the identification of gases and vapors. 1998. 2. Fritz, J., et al., Translating biomolecular recognition into nanomechanics. Science, 2000. 288(5464): p. 316-18. 3. Thundat, T., et al., Detection of mercury vapor using resonating microcantilevers. Applied Physics Letters, 1995. 66(13): p. 1695-7. 4. Turner, K.L., et al., Five parametric resonances in a microelectromechanical system. Nature, 1998. 396(6707): p. 149-52. 5. Turner, K.L., et al. Parametric resonance in a microelectromechanical torsional oscillator. in ASME International Mechanical Engineering Congress and Exposition Proceedings of Microelectromechanical Systems (MEMS). 1998. Anaheim, CA, USA. 6. Adams, S.G., et al., Independent tuning of linear and nonlinear stiffness coefficients [actuators]. Journal of Microelectromechanical Systems, 1998. 7(2): p. 172-80. 7. MacDonald, N.C., SCREAM microelectromechanical systems. Microelectronic Engineering, 1996. 32(1-4): p. 49-73. 8. Rand, R.H., Lecture Notes on Nonlinear Vibrations, version 34a. 2000: Available online at http://www.tam.cornell.edu/randdocs/. 9. Turner, K.L. Multi-dimensional MEMS motion characterization using laser vibrometry. in Transducers’99 The 10th International conference on solid-state Sensors and Actuators, Digest of Technical Papers,. 1999. Sendai, Japan. 10. Nguyen, C.T.C. Frequency-selective MEMS for miniaturized communication devices. 1998. 11. Brank, J., et al., RF MEMS-based tunable filters. International Journal of RF and Microwave Computer-Aided Engineering, 2001. 11(5): p. 276-84. 12. Baskaran, R. and K.L. Turner. Electrostatic interactions in coupled micro electro mechancial systems. in SPIE’s Micro/MEMS 2001. 2001. Adelaide, Australia. 13. Johnson, R., Mechanical filters in Electronics. 1983: Wiley Series in filters.

Feedback Regulation of the Heat Shock Response in E. coli Hana El-Samad1 , Mustafa Khammash1 , Hiroyuki Kurata3 , and John C. Doyle4 1 2 3

Mechanical and Environmental Engineering, University of California at Santa Barbara, Santa Barbara CA 93106, USA Department of Biochemical Science and Engineering, Kyushu Institute of Technology, Izuka, 820-8502, Japan Control and Dynamical Systems, California Institute of Technology, Pasadena CA 91125, USA

Abstract. Systems Biology is an emerging new field defined as the study of biology as an integrated system of components that act interdependently to accomplish certain functions. This approach holds the promise of offering precious insight into aspects of biological organization that cannot be identified through a reductionist approach concerned solely with the study of individual molecules. In this work, we illustrate this viewpoint through the example of the bacterial heat shock response. The heat shock response is an important mechanism that combats harmful effects of an unmediated increase in temperature. Such an increase in temperature causes the unfolding or aggregation of the cellular proteins, which imposes a tremendous amount of stress on the cell. The heat shock response is implemented through an elaborate system of controls whose purpose is to refold denatured proteins, therefore restoring their normal function. In this paper, we present a deterministic model for the heat shock response. We use this model to gain insight into the design and performance objectives of this response. We then provide a stochastic treatment based on the Stochastic Simulation Algorithm of Gillepsie [18]. This stochastic investigation validates the use of the deterministic approach in modeling the heat shock response, and motivates the investigation of feedback structures that play a role in attenuating stochastic fluctuations.

1

Introduction

Cells and organisms are mind puzzling living machines. Often times, their complexity exceeds that of our most sophisticated technological gadgets. The complexity of biological systems is the result of layered networks of feedback and regulatory loops. This remarkable hierarchy of regulation insures the proper operation of vital functions. Moreover, it provides robustness against perturbations and uncertain environments, and endows organisms with the ability to adapt existing designs to meet new challenges. Over the last decade, a large amount of research effort has been devoted to the building of libraries and databases for the components that make up biological organization. The genome project, among many others, is an example of these efforts. Yet, in recent years, the scientific community has realized the necessity of pursuing

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 115--128, 2003 Springer-Verlag Berlin Heidelberg

116

Hana El-Samad et al.

the next logical step: integrating the available data, identifying functional relationships, and conceivably cataloguing universal design principles. The goal is not just to understand the functions of individual genes, proteins and smaller molecules like hormones, but to learn how all of these molecules interact, how the interactions scale up to the physiological level, and what kind of failures of these interactions underlay the mechanisms of disease. It is becoming apparent that in order to achieve these goals and capture the dynamic character of biological pathways, more effort should be devoted to their systematic mathematical analysis, using the same tools that have been used with remarkable success in analyzing and designing engineering control systems. Recent examples have shown that the use of these tools is not a superfluous activity, but a necessity warranted by the fact that the deduction of the complex “wiring diagrams” of even simple cellular motifs has proven to be beyond the power of casual biochemical intuition. Indeed, in a recent series of articles, the importance of an interdisciplinary and mathematical approach to biological complexity was emphasized [1,2]. It was shown through example that this approach should involve the theory of feedback control and dynamical systems, among others, as building blocks; the result being a novel “Whole-istic” or “Systems Biology”. This approach will in no way undermine wet-biology or challenge its role. Instead, it will help transform biology from a qualitative, to a more quantitative and precise science. A rigorous mathematical description of biological processes will endow us with a valuable predictive power and a set of tools that can help in the design of more productive experiments. This interplay between mathematical and wet biology hence offers a unique opportunity for the understanding of biological function from a systems perspective, which is required if we are to move beyond the study of individual components, and into the realm of modular biology [3]. The objective of this paper is to motivate this perspective through the systematic study of the heat-shock response mechanism in E. coli. In this context, we use a quantitative dynamic model to capture the feedback regulatory aspects of the heat shock mechanism, and then use theoretical, analytical, and numerical tools developed in the fields of dynamical systems and control to bring about new understanding of its biological function. The bacterial heat shock response refers to the defense mechanism by which bacteria react and adapt to a sudden increase in the ambient temperature of growth. This mechanism is commonly identified by two phases. The first phase, called the “induction phase” is the time period during which the cell senses the increase in temperature, and induces the transcription of a group of heat shock genes that help in the restoration of the normal operating conditions in the cell. The induction phase is then followed by an “adaptation phase” during which the repair signals are damped down and the transcription of the heat shock proteins declines [4]. The heat shock response is implemented through an elaborate and complex mechanism that had been greatly conserved by evolution. This response enables cells to live at a wide range of temperatures.Furthermore, the complete understanding of the heat shock response and the categorization of its functional blocks could yield valuable insight into many other cellular networks. Induction of the heat shock response is a useful indicator of biological stress in a wide range of organisms, both prokaryotic and eukaryotic. Metals, organic solvents, herbicides, fungicides, weak acids, detergents, UV irradiation, as well as other classes of toxicants, induce this stress response to various degrees, thus suggesting that induction

Heat Shock Response

117

of the heat shock response is a general indication of adverse environmental conditions. Thus the understanding of the mechanisms involved in this response is crucial for the investigation of other stress responses. Furthermore, the heat shock response is suspected to be of primary importance for its potential connections to the immune system and the development of cancer vaccines among other things. The heat shock response has been thoroughly researched in the past decade or so [4,5]. The major players in this response, and their regime of operation have been uncovered. However, the bulk of the information present is still qualitative, and some of it follows a cause-effect or a speculative type of reasoning. It is therefore desirable to develop an analytical understanding of the heat shock response. At a minimum, such an understanding would lead to the verification of the known pieces and the justification of their dynamical role. Broader goals would involve the identification of any missing components in the understanding of this response, in addition to using the heat shock example as a benchmark for the study of some general features of gene regulatory networks.

2 2.1

The Heat Shock Response: A Case Study Components and Dynamics of the Heat Shock Response

In E. coli, much of the regulation of the concentrations of proteins occurs at the level of transcription and translation [5]. The enzyme RNA polymerase (RNAP) bound to a regulatory sigma factor recognizes the promoter and then transcribes specific genes into messenger RNA (mRNA). The mRNA is translated by the ribosome into protein. At physiological temperatures (30 ◦ C to 37◦ C), RNAP is bound to the major sigma factor σ 70 . The RN AP : σ 70 complex transcribes the genes necessary for growth at normal temperatures. When E. coli are exposed to high temperatures, the special heat shock sigma factor, σ 32 encoded by the rpoH gene, is rapidly induced. σ 32 binds to RNA polymerase and directs the transcription of a small set (approximately 20) of heat shock genes [5]. The heat shock genes encode for molecular chaperones (GroEL, DnaK, DnaJ, GroES, GrpE, etc.) that are involved in refolding denatured proteins. Another class of heat shock proteins are proteases (Lon, FtsH, etc.) that function to degrade unfolded proteins. In an rpoH null mutant that does not make σ 32 , the heat shock proteins are not induced and the cells are viable only at temperatures below 20 ◦ C [6]. There are two mechanisms by which σ 32 levels are increased when the temperature is raised. First, at low temperatures, the translation start site of σ 32 is occluded by base pairing with other regions of the σ 32 mRN A. Upon temperature upshift, this base-pairing is destabilized, allowing increased translation of σ 32 mRN A, and resulting in a fast 10-fold increase in the concentration of σ 32 [7]. Second, under nonstress conditions, σ 32 is recognized and sequestered by the hsp chaperone DnaK. The concentration of the σ 32 : DnaK complex depends on the amount of DnaK that is bound to unfolded proteins. Raising the temperature produces an increase in the cellular levels of unfolded proteins which titrate DnaK away from σ 32 , resulting in more σ 32 that is capable of binding to RNA polymerase and initiating the transcription of the heat shock genes [8]. The accumulation of high levels of heat shock proteins leads to the down regulation of the response. During this phase, the abundant chaperones efficiently refold most of the denatured proteins thereby

118

Hana El-Samad et al.

decreasing the pool of unfolded protein, freeing up DnaK to sequester σ 32 from RNA polymerase. In addition, heat shock proteases such as FtsH degrade σ 32 . The result is a decrease in the concentration of σ 32 to a new steady state concentration that is dictated by the balance between the temperature-dependent translation of the rpoH mRNA and the level of σ 32 activity modulated by the hsp chaperones and proteases acting in a negative feedback fashion. The above relationships are illustrated in Fig. 1.

Fig. 1. The heat shock response in E. coli The interactions described above involve rich and complex dynamics. Those dynamics could exhibit a wide range of behavior that can only be captured through detailed modeling and thorough analysis.

2.2

The Heat Shock Model

In the model building process, we focused on developing a tractable representation of the biological dynamics presented above. Several assumptions were made. We mention the most salient of these assumptions. Although a wide range of chaperones is available in the cell, we chose DnaK as a representative of the chaperone team. We considered FtsH as the major protease that degrades σ 32 , although we assumed that its action is mediated through interaction with the σ 32 : DnaK complex [10]. All other proteases that act through chaperones to degrade σ 32 are represented by one protease. It is believed that there are some proteases (like HslVU for example) that degrade σ 32 independently of chaperones and in a temperature dependent fashion [11]. We considered HslV U as the representative of these proteases. Transcript initiation is assumed to be a pseudofirst order reaction with rate Ktr. This assumption could be justified as follows. During transcription initiation,

Heat Shock Response

119

RNA Polymerase binds reversibly to the promoter region. This process, in addition to the subsequent formation of an open complex, is assumed to achieve rapid equilibrium [12]. The rate limiting step is assumed to be the initiation of transcription from the final open complex [13]. Similarly, translation initiation is assumed to proceed with a pseudofirst-order rate KT L . For most E. coli operons, initiation and elongation rates are such that ribosome queuing does not occur. We therefore take each transcription and translation initiation reaction to be independent. Finally, we assume that mRNA and protein molecules degrade with rate αmRN A and αprot . A decay rate α corresponds a half-life of ln(2)/α. If growth in cell volume is exponential, proceeding as ekt , the resulting dilution of species concentration can be incorporated by replacing αi by αi + k for species i. Based on these assumptions, the mathematical model that we propose to describe the dynamics of the heat shock response uses first order kinetics (law of massaction) to describe both the synthesis of new proteins, and the association/dissociation activity of molecules. This modeling approach produces a set of ordinary differential equations. A sample prototype equation for the net rate of protein synthesis that can be applied to any combination of operator, promoter and σ factor is as follows d[mRN A(protein)] = Ktr [RN AP : σ : promoter] − αmRN A [mRN A(protein)] dt d[protein] = KT L [mRN A(protein)] − αprotein [protein] dt where the Ki0 s are appropriate proportionality constants, while αmRN A and αprotein are degradation constants. [RN AP : σ : promoter] is just the complex formed by the binding of the core RNA Polymerase to the σ 32 factor, and then to the corresponding gene promoter. Upon simulation, those equations exhibited numerical stiffness behavior. Usually, this behavior is due to the interaction of some fast and slow dynamics in the system. The observed numerical stiffness imposed the necessity of transforming the differential equations that describe the fast states into algebraic constraints through a singular perturbation argument. We noticed that the binding rates (association and dissociation) between proteins or between proteins and specific DNA promoters are fast compared to the rate of synthesis and degradation of mRNAs and proteins. Therefore, we assumed that they reach their steady-state very fast compared to other reactions in the system. Therefore, the algebraic equations governing the binding dynamics took the form: [protein1 : protein2 ] = Kp [protein1 ]f [protein2 ]f where [protein]f refers to the concentration of free proteins available for binding and [protein1 : protein2 ] refers to the complex formed by the binding of [protein1 ] and [protein2 ]. We also used mass-balance equations to relate the total quantity of a species in the system to its free concentration and the concentration of the different compounds where it appears. The resulting model is a set of Differential Algebraic

120

Hana El-Samad et al.

Equations (DAEs), which are of the form: ˙ X(t) = F (t; X; Y ) 0 = G(t; X; Y )

(1) (2)

X0 = X(t = t0 )

(3)

Y0 = Y (t = t0 )

(4)

where X is a 11-dimensional vector whose elements are the differential variables and Y is a 20-dimensional vector whose elements are algebraic variables. This form is known as a semi-explicit DAE, with (2) being the constraint equation. If we differentiate (2) with respect to time, we get the following: 0 = GX (t; X; Y )X˙ + GY (t; X; Y )Y˙ + Gt (t; X; Y ) ) If GY (t, X, Y ) = ∂G(t,X,Y is nonsingular, the system is an implicit ODE. There∂Y fore, the DAE system is of index one and is solvable by Backward Differentiation Formulas as implemented in specialized software packages such as DASSL [9]. The model equations will be detailed elsewhere, along with the parameters used in the model. The rate parameters used in these equations were determined using various sources. The binding and degradation constants were mostly taken from the literature of the heat shock response. The synthesis rates for different proteins, σ factors, and chaperones were tuned to produce biologically plausible numbers of these quantities in the cell. The resulting model was able to reproduce the wild type dynamics of the heat shock response, in addition to reproducing data taken from different mutants, namely the FstH null mutant. The details are given in [15]. In terms of a control diagram representation, the heat shock response takes the form of Fig. 2.

2.3

The Control and Design Method of the Heat Shock Response

In this section, we investigate the role of the components involved in the heat stress response. To this end, we use different computational tools, and concepts borrowed from systems theory. We illustrate how this process can help us gain some insight into the roles and functionalities of the different loops and feedback structures that constitute the heat shock system. We start by investigating the role of the sharp increase in the σ 32 mRN A translation upon temperature increase. The importance of this term, which resembles a feedforward term in a control design setup, is assessed by comparing the protein folding regimes in the presence and absence of this feedforward. The result is shown in Fig. 3. It could be clearly seen that in the presence of the feedforward term, the level of unfolded proteins at steady-state after heat shock is around 3% of the total protein content of the cell, while it is around 11% in the absence of this term. This impaired response is the result of insufficient chaperone production, which in addition to degrading protein folding, delays the folding process by a time of

Heat Shock Response

121

Fig. 2. Control diagram of the heat shock response in E. coli around 4 minutes. Therefore, the feedforward term, which is implemented through the partial melting of the secondary structure of the mRN A upon exposure to high temperature, seems to be playing a dual role: it is driving the high temperature steady-state of protein folding to a higher value, while maintaining a small folding time. The role of the sequestration of σ 32 by the chaperones, which implements an inner feedback loop, was investigated using small-signal sensitivity analysis. · ¸ This X type of analysis involves the computation of the derivative of the solution with Y respect to the parameter of interest θ. We put special emphasis on the sensitivity of the total chaperone level to the binding of σ 32 to RN AP and gene promoter, in addition to the transcription and translation parameters of the heat shock proteins. In this analysis, we considered the full model, as well as a modified model where binding of σ 32 to DnaK was inhibited, hence disabling the inner loop. In order to solve numerically for the sensitivity equations, we used DASPK3.0, an algorithm developed by Petzold et. al for the sensitivity analysis of large-scale differential algebraic systems [16]. Using this information, we can plot the normalized sensitivity where u is the output of interest and θ is the parameter of interest. solutions uθ ∂u ∂θ Fig.4 shows an example of this sensitivity analysis. In this plot, the sensitivity of the total DnaK level to the transcription rate is plotted against time, for the full model along with a plot for the case where the inner loop was eliminated. In the no sequestration of σ 32 by DnaK case, the sensitivity values increase tremendously. This behavior suggests that the local loop is efficiently increasing the overall system robustness to parametric uncertainty. A similar conclusion could be drawn from Fig. 5 which depicts the sensitivity of DnaK to the binding constant between σ 32 and RN AP . In addition to the analysis mentioned above, we investigated the effect of the inner loop on the total number of chaperones. The results (not shown) also sup-

122

Hana El-Samad et al.

Folded proteins

2E+06

1.6E+06

Feedforward No feedforward

1.2E+06

395

400

405

410

Time

415

420

425

Fig. 3. Level of folded proteins for the heat shock response system. Heat shock occurs at t=400 min.

Sensitivity Dnak to transcription of DnaK

1.2

1

0.8 Full model No sequestration

0.6

0.4

0.2

0 100

200

300

400 500 Time

600

700

800

Fig. 4. normalized sensitivity of DnaK to the transcription rate. Heat shock occurs at t=400min. ported the idea that this inner loop is actively involved in keeping the number of chaperones small enough as not to impose a metabolic burden on the cell. As a conclusion, it seems that the inner loop in the heat shock response is also playing a dual role: it is increasing the robustness to parametric uncertainty, while keeping a low cost for the control action. Based on this analysis, we suggest that the heat shock response has an overall design architecture that achieves efficiency, robustness, and performance. This architecture consists of three components: a feedforward term, an inner feedback loop and an outer feedback loop. These components act in concert to achieve different

Sensitivity of DnaK to binding between s32 and RNAP

Heat Shock Response

123

0.5

0.4

0.3 Full model No sequestration

0.2

0.1

0

200

400 Time

600

800

Fig. 5. Normalized sensitivity of DnaK to the binding between σ 32 and RN AP . performance objectives. The task of achieving a fast and efficient response to the heat stress is implemented through a feedforward control term for σ 32 . The biological implementation of this feedforward component is through the partial melting of the secondary structure of the mRN A upon exposure to high temperature, which enhances ribosome entry and translational initiation. This feedforward term drives the transition to the high temperature operation mode and assures reliable folding of proteins, therefore enhancing performance. The important task of maintaining a low metabolic burden on the cell and assuring robustness has necessitated the evolution of an inner local loop which acts through the regulation of the activity of σ 32 . By sequestering σ 32 , the chaperones modulate the number of the free σ 32 , therefore limiting their activity in the transcription of unneeded heat shock proteins. This sequestration of free σ 32 also buffers the effects of parameter variations and uncertainties, therefore enhancing robustness and maintaining the output signal DnaK within a narrow range, despite possibly fluctuating inputs and parameters. At the same time, an outer loop which acts through the degradation of σ 32 by proteases, assures the stability of the whole system through hindering the build-up of σ 32 resulting from continuous production. Fig. 6 is a simplified diagram that shows the basic architecture of the system as described above.

2.4

Stochastic Modelling

The approach in the previous sections follows the deterministic formalism, which regards the time behavior of a spatially homogeneous chemical system as a continuous process which is completely determined by the solution of the reaction-rate equations. This approach to chemical kinetics is very useful in many cases, where the time evolution of a chemically reacting system can be indeed treated as a continuous and deterministic process. However, this approach has serious shortcomings when the fluctuations in the molecular population levels are of importance. For example, this might be the case when the number of reactants is small so that

124

Hana El-Samad et al.

Fig. 6. Simplified control diagram of the heat shock response in E. coli “averaging” arguments cannot be applied. In our heat shock system, the number of σ 32 molecules is around 30-50 per cell. One might suspect that this relatively small number of molecules could challenge the validity of the deterministic approach, and therefore imposes the necessity for an exact stochastic investigation. A stochastic representation generally uses the Chemical Master Equation formalism where the system is considered to have the Markov property, in the sense that the time evolution of the molecular species is determined by the current state of the system and is independent of its past. This Markov property allows us to view the molecular species as random variables whose probability density function is described by a gain/loss rate equation called the Master Equation [17]. In general, the Master Equation is infinite dimensional and is not solvable, either analytically or numerically, for any but the simplest chemical systems. Therefore, instead of attempting to solve the Master Equation, another approach consists of simulating or producing realizations of the stochastic evolution of a chemically reacting system. This is implemented in the Stochastic Simulation algorithm (SSA) of Gillespie [18]. The SSA is concerned with the problem of simulating the time evolution of the concentration of molecules in a fixed volume mixture of N chemical species interacting through reaction channels Rµ (µ = 1, ...M ). The major building block of the SSA is the Next-Reaction Probability Density Function P (τ, µ) which is a joint probability density function on the space of the continuous time variable τ and the discrete reaction random variable µ. In rough terms, the function P (τ, µ)dτ gives the probability that given the state of the system at time t, the next reaction will occur in the infinitesimal time [t + τ, t + τ + dτ ] and will by a specific Rµ reaction. P (τ, µ) has an analytical description in terms of the physical properties of the system, namely in terms of the propensity functions which are calculated using elementary kinetic theory arguments [18]. With this analytical expression for P (τ, µ), it is straightforward, using a random generator, to draw a pair (τ, µ) from the set of random pairs whose probability density function is given by P (τ, µ) . This will specify the next reaction that will occur and the time at which this reaction occurs. The resulting trajectory constitutes an unbiased realization of the system, which is equivalent to solving the Master Equation. In what follows, we will apply this stochastic analysis to our model of the heat shock response. We implemented the SSA as described above, and used the algorithm to simulate the exact behavior of the heat shock response system. We focused

Heat Shock Response

125

on the time profile of total σ 32 and total DnaK. Fig. 7 and Fig. 8 show the result of these simulations. Those plots show two sample realizations based on two different seeds for the random number generator, which yields a different sequence of pairs (τ, µ). The plots also show the mean of 30 runs averaged over realizations, and the result of the deterministic simulation. We notice that the average behavior yielded by the stochastic simulation closely approaches the deterministic mean, which indicated that the deterministic approach was valid despite the small number of σ 32 involved. We suspect that this behavior is the result of a combination of factors. For example, the transcription and translation machinery for the heat shock proteins implements a high-gain, low pass filter that filters out all the high frequency components, thus smoothing out the fluctuations. The stochastic attenuation is also a result of the use of feedback. Such an effect of feedback has been mathematically and experimentally proven in the literature for the linear case and for simple genetic circuits [19]. However, the nonlinear case is more intractable and the analytical proof for this result is still under development and will be exposed elsewhere. We should point out here that in this stochastic simulation, the folding of proteins was fixed as constant disturbance which was raised to its high steady-state value at high temperature. The reason for that is the large number of proteins present in the cell, which made the step size in the algorithm inhibitably small, and therefore necessitated the elimination of the folding dynamics from the exact simulation. We don’t expect that this constant approximation of the folding dynamics would have an impact on any of our conclusions. However, the example of the heat shock response sheds some light on one of the major shortcomings of the SSA, which is the inadequacy of the algorithm in systems involving large numbers of molecules, or in systems composed of a mixture involving large number of molecules for certain molecular species, and fewer numbers for other species. On the theoretical side, this motivates the development of a modified SSA where both deterministic and stochastic dynamics can be integrated in the same framework.

180

Deterministic Stochastic mean (30 runs) Sample realization Sample realization

Sigma32 total

150 120 90 60 30 0 150

200

250 Time (minutes)

300

350

Fig. 7. The heat shock response in E. coli

126

Hana El-Samad et al. 18000 16000

DnaK total

14000 12000 10000 8000

Deterministic Stochastic mean (30 runs) Sample mean Sample mean

6000 4000 150

200

250 Time (minutes)

300

350

Fig. 8. The heat shock response in E. coli

3

Conclusions and Future Work

The modeling and analysis of the heat-shock system provides an example of the potential benefits to biology that may be offered by a systems perspective. We point out that as a direct result of the heat shock robustness analysis it has been possible to identify the parameters that have the most impact on the system’s behavior. Using this information, we are presently collaborating with biologists to devise and perform experiments aimed at measuring those key parameters. The developed model demonstrates that the current biological understanding of heat shock components and their interactions is sufficient to explain the observed phenotype in temperature upshift experiments. In particular, we have utilized the model to address an important question related to the adaptation phase of the heat shock response, where we have recently shown that the decline of the number of σ 32 during that phase is a natural outcome of the system dynamics, rather than the result of a decline in the σ 32 translation rate as was previously speculated. As another example of the benefits of the systems modeling and analysis, we are currently working with biologists to design a model-motivated sequence of biological experiments that will elucidate the controversial aspects of certain temperature downshift experiments. These downshift experiments have led to apparently contradictory results that could not be reconciled by the current understanding of the heat shock system. However, preliminary results based on our dynamical model indicate that the investigation of temperature downshift should be redirected towards the investigation of the slow decline of σ 32 after downshift. In contrast, the majority of the experimental data presented in the literature concern the synthesis rate of chaperones whose time profile is fairly unambiguous and follows readily from the system dynamics. On a more theoretical level, future research directions involve pursuing some conceptual aspects of biological modeling inspired by the study of the heat shock

Heat Shock Response

127

system. These directions include the role of stochastic fluctuations in the performance of gene networks, and the identifications of specific feedback structures that attenuate those fluctuations. This pursuit will be based on existing tools, but will inevitably result in the development of new techniques that reflect the uniqueness of the underlying biological systems. Broader research directions will also include the development of an analytical framework for biological model validation. Preliminary results on the validation of a simplified model for the heat shock response based on a sum of squares decomposition methodology has already been attempted. The results of such methods seem to be very promising.

References 1. Kitano, H. (2002) Systems biology: A Brief Overview, Science. 295, pp. 1662– 1664 2. Csete, M. E, Doyle, J. C. (2002) Reverse Engineering of Biological Complexity. Science 295, 1664–1669 3. Hartwell, L. H., Hopfield, J. et al. (1999) ¿From Molecular to Modular Cell Biology. Nature 81, C47–C52 4. Herman, C., Gross, C. A. (2000) Heat Stress. Encyclopedia of Microbiology, Academic Press, 598–605 5. Gross, C. A. (1996) Function and Regulation of the Heat Shock Proteins in Escherichia Coli and Salmonella: Cellular and Molecular Biology, (eds F.C. Neidhart), ASM press, Washington D.C., 1384–1394 6. Zhou, Y. N, Kusukawa, N. et al. (1988) Isolation and Characterization of Escherichia Coli Mutants that Lack the Heat Shock Sigma Factor σ 32 . J. Bacteriol. 170, 3640–3649 7. Strauss, D. B., Walter, W. A. et al. (1989) The Activity of σ 32 is Reduced Under Conditions of Excess Heat Shock Protein Production in Escherichia Coli. Genes Dev. 3, 2003–2010 8. Arsene, F., Tomoyasu, T. et al. (2000) The Heat Shock Response of Escherichia Coli. International Journal of Food Microbiology 55, 3–9 9. Brenan, K., Campbell,L. et al. (1989) Numerical Solution of Initial Value Problems in Differential-Algebraic Equations, Elesevier Science Publishing Co N.Y. 10. Tatsuta, T., Tomoyasu, T. et al. (1998) Heat Shock Regulation in the ftsH null Mutant of Escherichia Coli: Dissection of Stability and Activity Control Mechanims of σ 32 in vivo. Molecular Microbiology 30, 583–593 11. Kanemori, M., Nishihara, K. et al. (1997) Synergetic Roles Of HslVU and Other ATP-Dependent Proteases in Controlling In Vivo Turnover Of σ 32 and Abnormal Proteins In Escherichia Coli. J. Bacteriol., 179, 7219–7225 12. DeHaseth, P.L., Zupancic, M.L., and Record, M.T. (1998) RNA PolymerasePromoter Interactions: the Comings and Goings of RNA Polymerase. J. Bacteriol. 180, 3019–3025 13. Schmitt, B., Reiss, C. (1995) Kinetic study in vitro of Escherichia coli promoter closure during transcription initiation. Biochem. J. 306, 123–128 14. McClure, W.R. (1985) Mechanism and Control of Transcription Initiation in Prokaryotes. Ann. Rev. Biochem. 54, 171–204

128

Hana El-Samad et al.

15. Kurata, H., El-Samad, H. et al. (2001) Feedback Regulation of the Heat Shock Response in E. Coli. Proceedings of the 40th IEEE Conference on Decision and Control, 837–842 16. Li, S., Petzold,L. (2000) Software and Algorithms for Sensitivity Analysis of Large-Scale Differential Algebraic Systems. J. Comput. and Appl. Math. 125, 131–146 17. Van Kampen, N. G. (1992) Stochastic Processes in Physics and Chemistry, Elsevier-Science, Amesterdam Holland 18. Gillespie, D. T. (1977) Exact Stochastic Simulation of Coupled Chemical Reactions. J. Phys. Chem. 81, 2340–2361 19. Thattai, M., Van Oudenaarden, A. (2001) Intrinsic Noise in Gene Regulatory Networks. PNAS 98, 8614–8619

Stem Cells from the Outside In Marie Csete MD, PhD University of Michigan, Departments of Anesthesiology and Cell & Developmental Biology

Abstract. The potential of stem cells to replace damaged cells and organs is the subject of public discourse and political debate. Stem cell biology is in an explosive phase of growth, but has not yet yielded a fundamental understanding of the molecular control of stem cell fate. Surprisingly the role of the gaseous environment in the control of stem cell biology has been largely ignored. This chapter will highlight the importance of oxygen signaling in all aspects of stem cell biology-their proliferation, patterns of death and differentiation. The use of oxygen signals in cultivation of stem cells is a potent tool for identifying pharmacologic targets that can be used to enhance regeneration or to treat common degenerative diseases.

1 1.1

Introduction: Stem cells Stem cells defined

This discussion will focus on adult stem cells, that is, those stem cells present after birth. Stem cells possess several unique features when compared to other cells of the body. First, they have the ability to self-renew. Self-renewal may imply an asymmetric division of a stem cell into another stem cell plus a more differentiated daughter cell. Alternatively self-renewal can be accomplished by maintenance of some stem cells within the population of organ-specific cells. Second, stem cells are multipotential, meaning that they can generate more than one specific cell type. For example, the hematopoietic stem cell is responsible for generating red blood cells, a variety of white blood cells, and platelets throughout our lives. Stem cells are also defined by what they are not. They have the potential to generate cells with specific physiologic functions but do not themselves express the proteins to execute these specific differentiated functions. Stem cells in their undifferentiated state have high proliferative potential, and it is becoming clearer that most stem cells are highly mobile in the body.

1.2

Stem cell choices

Stem cells may remain quiescent, or sit in reserve. They can proliferate and make copies of themselves, and of course they can die. Stem cells can also generate a population of diverse daughter cells, in theory the entire cellular population of a whole organ. Here is their power, as sources of tissue for transplantation or as targets for pharmacologic agents that can enhance their ability to replace damaged organs in vivo. The growth factors that promote proliferation, the transcriptional

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 129--132, 2003 Springer-Verlag Berlin Heidelberg

130

Marie Csete

machinery that inhibits differentiation of inappropriate cells, the transcriptional machinery that promotes differentiation into needed regenerated organs are all the focus of intense study. But cues from the environment that signal stem cells to take action are not limited to secreted proteins, or to ligands displayed by adjacent cells. These cues can also be gases from the surrounding environment, though the traditional methods of cultivating stem cells in laboratory has not taken the normal gaseous environment of the body into account.

2 2.1

Oxygen levels Oxygen levels in tissue time and space

The mean oxygen concentration seen by a cell in the adult human is 3 percent. We breathe in 21% oxygen, the alveoli that form the interface of the body to the outside world reside at about 15% oxygen. With gas exchange in the lungs, the arterial circulation is 12% oxygen. After delivery of oxygen to tissues, the venous circulation returning blood to the heart is at about 5.3% oxygen. Considerable ranges of oxygen tension characterize different organs and different subcompartments of organs. But none of the cells in the body is ever subjected directly to the 20% oxygen levels used to cultivate cells in most biology laboratories. Furthermore, the mean of 3% oxygen found in adult tissues is considerably greater than that found in embryonic or fetal tissue (in which stem cells are abundant). Oxygen levels in the body are extremely tightly controlled, both globally and locally, by multiple levels of feedback signaling. Some large-scale acute physiologic responses to oxygen lack (hypoxia or low oxygen in blood) include elaboration of signals that drive respiration, dilate the blood supply to hypoxic areas, and turn on glycolytic metabolic pathways. Hypoxia also results in signals that enact responses for longer-term solutions to the lack of oxygen. Low oxygen levels cause the kidney to elaborate erythropoietin which drives marrow production of red blood cells that carry oxygen. Low levels of oxygen also drive elaboration of growth factors that promote production of new blood vessels. At the center of many of these physiologic responses is the hypoxia-inducible factor 1α or HIF − 1α [1]. When oxygen is abundant, HIF − 1α is rapidly broken down in the cells. However, a lack of oxygen causes protein modification in part of the complex that escorts HIF − 1α to the cellular site of its breakdown [2]. This exquisite feedback loop means that HIF − 1α is immediately abundant when oxygen is lacking, and HIF − 1α in turn controls production of many of the genes that are necessary for the big physiologic survival responses to oxygen deprivation. HIF − 1α is critical for survival in the face of a dynamic oxygen environment, but a multitude of cellular processes are altered by changing oxygen in the cell’s environment. Our own work shows that the entire molecular fingerprint of cells, particularly stem cells, is dramatically altered by the oxygen environment. In the laboratory, we have seen that enormous shifts in the production of messenger RNA as a consequence of changing the oxygen surrounding cultivated stem cells from room air (20%) down to more physiologic levels (2 − 6%). Hundreds of genes are turned on and hundreds are turned off by changing the oxygen around stem cells, and these are not oxygen levels that are hypoxic. Furthermore, oxygen induces changes not

Stem Cells from the Outside In

131

only at the message level, but at the protein level. For example, in stem cells derived from rat brain, we found that tyrosine hydroxylase protein is more abundant when the cells are grown in lower oxygen rather than in 20% oxygen [3]. The levels of tyrosine hydroxylase messenger RNA were not different between conditions, thus tyrosine hydroxylase abundance (important in the pathway to dopaminergic neuron differentiation) was controlled at the protein level. We have also shown the protein trafficking in the cell (whether it is in the cytoplasm or in the nucleus) is altered by the oxygen environment (unpublished). The location of a protein in the cell is a major determinant of its function, by both providing and sequestering the protein from partners necessary for physiologic function.

2.2

Oxygen levels surrounding cultivated stem cells dramatically skew stem cell fate

Not only do central nervous system (brain) stem cells cultured in low oxygen express more tyrosine hydroxylase, they also differentiate more readily into functional dopaminergic neurons [3]. Central nervous system stem cells are not the only stem cells for which differentiation patterns are substantially altered by the oxygen surrounding culture systems. Neural crest stem cells cultured in low, physiologic levels of oxygen (vs. traditional 20% oxygen) are more likely to differentiate into neurons of the sympathoadrenal lineage [4]. The primary adult satellite stem cell resident in skeletal muscle is six time more likely to differentiate into fat if cultured in 20% oxygen than if cultured in physiologic levels of oxygen [5].

2.3

Oxygen and stem cells in culture and the link to common pathologies

The high levels of oxygen (20%) used in laboratories to culture stem and other cells are never encountered in normal physiology. By comparing the stem cell outcomes of these aphysiologic conditions to stem cell fates in physiologic oxygen conditions, what can we learn? It is likely that high oxygen (20%) conditions also favor elevated levels of reactive oxygen species surrounding the cultures. In fact, we have measured reactive oxygen species in myoblast cultures and confirmed that reactive oxygen species are significantly more abundant in 20% vs. 6% oxygen conditions [6]. Reactive oxygen species are not only harmful byproducts of cellular respiration, but have specific signaling functions, and act to modify the activity of proteins including transcription factors. With the availability of better reagents to measure reactive oxygen species, it is increasingly clear that they play an important role in diseases as diverse as Parkinson’s disease, diabetes, sepsis, and alcoholic liver disease. Furthermore, the lifelong accumulation of reactive oxygen species is considered to be at the heart of cellular and organismal aging.

3

Conclusions

With something of a leap of faith, high oxygen conditions can be considered an aging environment for cultivation of stem and pluripotent cells. The propensity of skeletal muscle stem cells to undergo adipogenesis (fat cell development) in high

132

Marie Csete

oxygen conditions may mirror the phenotype of stem cells in an aged muscle environment [5]. Similarly, the decrease in dopaminergic differentiation in high oxygen conditions may capture some of the damaging local oxidative stress that destroys dopaminergic neurons in Parkinson’s disease [3]. Thus, developmental pathways under the influences of oxygen can be used to understand degenerative diseases, that may in part really be aberrant regenerative diseases. Using this framework, dissection of stem cell developmental pathways are likely to lead to the identification of pharmacologic targets for the therapy of common degenerative pathologies.

References 1. Semenza GL, “Hypoxia-inducible factor 1: oxygen homeostasis and disease pathophysiology,” Trends Mol Med, 7:345-350, 2001. 2. Ivan M, Kondo K, Yang H, Kim W, Valiando J, Ohh M, Salic A, Asara JM, Lane WS, Kaelin WG JR, “HIFalpha targeted for VHL-mediated destruction by proline hydroxylation: implications for O2 sensing,”Science 292:464-468, 2001. 3. Studer L, Csete M, Lee SH, Kabbani N, Walikonis J, Wold B, McKayR, “Enhanced proliferation, survival, and dopaminergic differentiation of CNS precursors in lowered oxygen,” . J Neurosci 20:7377-7383, 2000. 4. Morrison SJ, Csete M, Groves AK, Melega W, Wold B, Anderson DJ, “Culture in reduced levels of oxygen promotes clonogenic sympathoadrenal differentiation by isolated neural crest stem cells,”J Neurosci 20:7370-7376, 2000. 5. Csete M, Walikonis J, Slawny N, Wei Y, Korsnes S, Doyle JC, Wold B, “Oxygen-mediated regulation of skeletal muscle satellite cell proliferation and adipogenesis in culture,” J Cell Physiol 189:189-196, 2001. 6. Csete M. and Hansen, unpublished.

Optimal Image Interpolation and Optical Flow Steven Haker1 and Allen Tannenbaum2 1 2

Surgical Planning Lab, Brigham and Women’s Hospital, Boston, MA 02115 Depts. of ECE and BME, Georgia Institute of Technology, Atlanta, GA 30332-0250; and Dept. of EE, Technion, Israel Institute of Technology, Haifa, Israel

Abstract. Image interpolation is the process of generating a set of intermediate images between two given images. The technique is important for a number of key problems including optical flow, image compression, image coding, and visual tracking. Numerous techniques have been proposed. In this paper, we will consider a method based on optimal transport. This paper is dedicated to the memory of our dear friend and colleague, Professor Mohammed Dahleh.

1

Introduction

In this paper, we propose a novel approach to image interpolation. Image interpolation is a generic term given to a collection of techniques used to generate a set of intermediate images “connecting” two successive images. Our approach is based on the theory of optimal transport. The optimal transport problem was first formulated by Monge in 1781, and concerned finding the optimal way, in the sense of minimal transportation cost, of moving a pile of soil from one site to another. This problem was given a modern formulation in the work of Kantorovich [19], and so is now known as the Monge–Kantorovich problem. This type of problem has appeared in econometrics, fluid dynamics, automatic control, transportation, statistical physics, shape optimization, expert systems, and meteorology [25]. It also naturally fits into certain problems in computer vision [9]. In particular, for the general tracking problem, a robust and reliable object and shape recognition system is of major importance. A key way to carry this out is via template matching, which is the matching of some object to another within a given catalogue of objects. Typically, the match will not be exact and hence some criterion is necessary to measure the “goodness of fit.” For a description of various matching procedures, see [15] and the references therein. The matching criterion can also be considered a shape metric for measuring the similarity between two objects. In this paper, we propose to use optimal mass transport for interpolation and optical flow. Our idea is that given an interpolation method one can use MongeKantorovich (MK) optimization to define a regularity term which will give an optimal warp in the MK sense. The constraint that we will put on the transformations considered is that they obey an area-preservation property.

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 133--143, 2003 Springer-Verlag Berlin Heidelberg

134

2

StevenÿHaker and AllenÿTannenbaum

Area-Preserving Diffeomorphisms

We have been considering the use of area-preserving and conformal diffeomorphisms applied to problems in surface deformations for 3D visualization and image registration. In medical imaging, we have applied deformations based on such mappings for brain flattening in functional magnetic resonance imaging [2]. They may also be used for path planning [3]. We believe that these techniques can also be very useful for the computation of image interpolation and optical flow as we will argue below. We first show that it is straightforward to construct an area-preserving diffeomorphism from an arbitrary one. This can be taken as an initial mapping in our gradient descent approaches for the Monge-Kanotorovich problem discussed below.

2.1

Existence and Construction of Area-Preserving Flows

In this section, we outline the proof of a nice result of J. Moser [23] guaranteeing the existence of an area-preserving diffeomorphism (assuming of course the two surfaces have the same total surface area). The proof allows an explicit construction of the area-preserving map starting from an arbitrary diffeomorphism. We give a number of the details since these will be essential in our approach to optical flow to be considered below. Let f : M → N be a diffeomorphism of compact surfaces with the same total surface area. Let ω1 = f ∗ (ω) be the pullback of the area-form ω on N under f , and let ω0 be the area form of M itself. The ωi are two-forms with the same integral over M , and we want a diffeomorphism g : M → M with g ∗ (ω1 ) = ω0 . Given g, the area-preserving map of M onto N is φ = f ◦g, i.e., φ∗ (ω) = g ∗ ◦f ∗ (ω) = g ∗ (ω1 ) = ω0 . To construct g we look for g = g1 where gt is a one parameter family of diffeomorphisms starting at g0 = idM and evolving according to dgt (x) = Xt (gt (x)). dt Let ωt = (gt )∗ (ω1 ). We try to find vector fields Xt such that ωt = ω0 + t(ω1 − ω0 ). This gives dωt dt

= ω1 − ω0 =

d(gt )∗ (ω1 ) dt

= LXt (ωt ) = (diXt + iXt d)(ωt ) = (diXt )(ωt ). To get the vector fields Xt we set the first and last lines equal to get (diXt )(ωt ) = ω1 − ω0 .

Optimal Image Interpolation

135

Here the right hand side is an exact form (because the ωi have equal integrals and hence represent the same DeRahm cohomology class on M ) and hence there exists a one-form θ on M with dθ = ω1 − ω0 . Finally, one can solve the equation (iXt )(ωt ) = θ, because the ωt are area forms, and hence nondegenerate. So in order to construct the area-preserving diffeomorphims, we find θ from dθ = ω1 − ω0 , then solve the above equation for Xt , and finally integrate the vector fields Xt to compute g = g1 .

3

Formulation of Optimal Transport

We now give a modern formulation of the Monge–Kantorovich problem. Let Ω and Ω 0 be two subdomains of Rd , with smooth boundaries, each with a positive density function, µ and µ0 , respectively. We assume Z Z µ0 (1) µ= Ω

Ω0

so that the same total mass is associated with Ω and Ω 0 . We consider diffeomorphisms u from Ω to Ω 0 which map one density to the other in the sense that µ = |∇u| µ0 ◦ u,

(2)

which we will call the mass preservation (MP) property, and write u ∈ M P. Equation (2) is called the Jacobian equation. Here |∇u| denotes the determinant of the Jacobian map ∇u. In particular, Equation (2) implies, for example, that if a small region in Ω is mapped to a larger region in Ω 0 , then there must be a corresponding decrease in density in order for the mass to be preserved. A mapping u that satisfies this property may thus be thought of as defining a redistribution of a mass of material from one distribution µ to another distribution µ0 . There may be many such mappings, and we want to pick out an optimal one in some sense. Accordingly, we define the Lp Kantorovich–Wasserstein metric as follows: Z p 0 inf ku(x) − xkp µ(x) dx. (3) dp (µ, µ ) := u ∈ MP An optimal MP map, when it exists, is an MP map which minimizes this integral. This functional is seen to place a penalty on the distance the map u moves each bit of material, weighted by the material’s mass. Hence, the Kantorovich–Wasserstein metric defines the distance between two mass densities, by computing the “cheapest” way to transport the mass from one domain to the other with respect to (3). The case p = 2 has been extensively studied and will the one used in this paper for interpolation. The L2 Monge–Kantorovich problem has been studied in statistics, functional analysis, and the atmospheric sciences; see [8,5] and the references therein. A fundamental theoretical result [20,7,10], is that there is a unique optimal u ∈ M P transporting µ to µ0 , and that this u is characterized as the gradient of

136

StevenÿHaker and AllenÿTannenbaum

a convex function w, i.e., u = ∇w. Note that from Equation (2), we have that w satisfies the Monge–Amp`ere equation |Hw| µ0 ◦ (∇w) = µ,

(4)

where |Hw| denotes the determinant of the Hessian Hw of w.

4

Gradient Descent for Optimal Transport

We now abstract the situation from the previous section. We assume that we have an initial mass-preserving map u0 : Ω → Ω 0 which maps the measure µ dx to µ0 dx. This may be constructed using the techniques of Section 2. We wish to deform it, i.e. construct a family of maps ut : Ω → Ω 0 , so as to minimize the “cost functional” Z t Φ(ut (x) − x)µ(x)dx, (5) M [u ] = Ω

whilst preserving the measure preserving property. Here Φ : Rd → R is a positive C 1 function. For example, we may take Φ(x) :=

kxk2 2

in which case this is the L2 Monge-Kantorovich problem. We factor ut as u0 = ut ◦ st ,

(6)

where st : Ω → Ω is a µ-mass preserving family of diffeomorphisms. Note that when µ is constant this means that st are area or volume-preserving. These diffeomorphisms are generated by a vector field, the velocity field v t , on Ω. Thus ∂st = v t ◦ st . ∂t

(7)

Since the maps st preserve the measure µ dx, the velocity field must satisfy div (µ v t ) = 0.

(8)

By the chain rule the maps ut will satisfy ∂ut + v t · ∇ut = 0. ∂t

(9)

We note that Z Z Φ(ut (x) − x)µ(x) dx = Φ(u0 (y) − st (y)) µ(y) dy, M (t) = Ω

Ω

(10)

where x = st (y), (st )∗ (µ(x) dx) = µ(y)dy.

(11)

Optimal Image Interpolation

137

From this one computes that the cost decreases according to dM =− dt

Z Z

=− Z =−

h∇Φ(u0 − s),

∂st iµ dx ∂t

h∇Φ(ut (x) − x), µ v t i dx h∇Φ(ut (x) − x), ζ i dx

where ζ = µ v t . Next using the Helmholtz decomposition, and imposing boundary conditions for the flow to remain in Ω, we take ζ = ∇Φ(ut (x) − x) + ∇p,

(12)

div (ζ) = 0,

(13)

ζ|∂Ω is tangential to ∂Ω.

(14)

This leads to the following system of equations: ∂ut 1 = − ∇ut · [∇Φ(ut (x) − x) + ∇p] ∂t µ ∆p + div (∇Φ(ut (x) − x)) = 0 on ∂Ω ∂p + n · ∇Φ(ut (x) − x) = 0 on ∂Ω, ∂n

(15) (16) (17)

where n denotes the unit inward pointing normal to ∂Ω. Note that this latter system may also be written in the following form: 1 ∂ut = − ∇ut · (I − ∇∆−1 div )∇Φ(ut − id), ∂t µ

(18)

where id denotes the identity map. Remarks: We use standard techniques to solve Equation (18). In particular we have employed an upwinding scheme when computing ∇ut , and the FFT when inverting the Laplacian on a rectangular grid. Standard centered differences were used for the other spatial derivatives. Once we numerically solve for the right hand side of (18), we use the result to update ut . The optimal map is obtained as t → ∞. In practice, we iterate until the mean absolute curl is sufficiently small. (See [14] for more details.)

5

Optimal Image Interpolation

We now propose the use of optimal mass transport to formulate a new method for image interpolation, and as a natural corollary for the computation of optical flow.

138

StevenÿHaker and AllenÿTannenbaum

(See also [28] and the references therein.) The idea is to minimize a functional of the following form over area-preserving mappings u : Ω → Ω 0 , Z (19) M := (F ◦ u − G)2 + α2 ku(x) − xk2 dx, α ∈ R. Here the first term controls the “goodness of fit” between the (intensity) images G : Ω → R and F : Ω 0 → R, and the second Monge-Kantorovich term controls the warping of the map. More generally, one may consider the following type of problem: Z Goodness of fit + α2 ku(x) − xk2 dx. In equation (19), we took Goodness of fit =

Z

(F ◦ u − G)2 dx.

But we can choose various types of fidelity terms in this framework, e.g., one based on the concept of mutual information as in [29]. Note that our approach gives natural correspondence measures for a sequence of images (e.g., as in a video), and thus can be tailored for use in optical flow and tracking. See our discussion below about optical flow as well as a related approach for its computation.

5.1

Computation of Optimal Image Interpolation

As above, we have the functional Z Z M = (F ◦ u − G)2 dx + α2 ku(x) − xk2 dx the first term controlling the “goodness of fit” between the images G : Ω → R and F : Ω 0 → R, and the second the amount of distortion of the maps u (their distance from the identity). We minimize over area–preserving mappings u : Ω → Ω 0 . From the above computation, we derive the following descent equation for the optimum: ∂ut (20) = −∇ut · (I − ∇∆−1 div )[(F ◦ ut − G)∇G + α2 (ut − id)]. ∂t This equation may be numerically implemented using the techniques described in Section 5 above.

6

Optical Flow

As alluded to above, the theory of area-preserving maps is very relevant to the computation of optical flow. Indeed, optical flow is an essential tool for problems arising in active vision. The optical flow field is the velocity vector field of apparent motion of brightness patterns in a sequence of images [18]. One assumes that the motion of the brightness patterns is the result of relative motion, large enough to register a change in the spatial distribution of intensities on the images. Thus, relative motion between an object and a camera can give rise to optical flow. Similarly, relative motion among objects in a scene being imaged by a static camera can give rise to optical flow.

Optimal Image Interpolation

6.1

139

Optical Flow Constraint

A typical approach for optical flow is based on spatiotemporal differentiation. Even though in such a method, the optical flow typically estimates only the isobrightness contours, it has been observed that if the motion gives rise to sufficiently large intensity gradients in the images, then the optical flow field can be used as an approximation to the real velocity field and the computed optical flow can be used reliably in the solutions of a large number of problems; see [17] and the references therein. Thus, optical flow computations have been used quite successfully in problems of three-dimensional object reconstruction, and in three-dimensional scene analysis for computing information such as depth and surface orientation. In object tracking and robot navigation, optical flow has been used to track targets of interest. Discontinuities in optical flow have proved an important tool in approaching the problem of image segmentation. The problem of computing optical flow is ill-posed in the sense of Hadamard. Well-posedness has to be imposed by assuming suitable a priori knowledge. For example, a number of researchers have considered a variational formulation for imposing such a priori knowledge; see [18] and the references therein. One constraint which has often been used in the literature is the “optical flow constraint”(OFC). The OFC is a result of the simplifying assumption of constancy of the intensity, E = E(x, y, t), at any point in the image [18]. It can be expressed as the following linear equation in the unknown variables u and v Ex u + Ey v + Et = 0,

(21)

where Ex , Ey and Et are the intensity gradients in the x, y, and the temporal directions respectively, and u and v are the x and y velocity components of the apparent motion of brightness patterns in the images, respectively. It has been shown that the OFC holds provided the scene has Lambertian surfaces and is illuminated by either a uniform or an isotropic light source, the 3-D motion is translational, the optical system is calibrated and the patterns in the scene are locally rigid. It is not difficult to see from equation (21) that computation of optical flow is unique only up to computation of the flow along the intensity gradient ∇E = (Ex , Ey )T at a point in the image [18]. (The superscript T denotes “transpose.”) This is the celebrated aperture problem. One way of treating the aperture problem is through the use of regularization in computation of optical flow, and consequently the choice of an appropriate constraint. A natural choice for such a constraint is the imposition of some measure of consistency on the flow vectors situated close to one another on the image. In their pioneering work, Horn and Schunk [18] use a quadratic smoothness constraint. The immediate difficulty with this method is that at the object boundaries, where it is natural to expect discontinuities in the flow, such a smoothness constraint will have difficulty capturing the optical flow. For instance, in the case of a quadratic constraint in the form of the square of the norm of the gradient of the optical flow field [18], the Euler-Lagrange (partial) differential equations for the velocity components turn out to be linear elliptic. The corresponding parabolic equations therefore have a linear diffusive nature, and tend to blur or smooth the vector field near the edges.

140

StevenÿHaker and AllenÿTannenbaum

In the past, work has been done to try to suppress such a constraint in directions orthogonal to the occluding boundaries in an effort to capture discontinuities in image intensities that arise on the edges. In [21], an optimization problem is proposed in which the resulting Euler-Lagrange equations are nonlinear geometric heat equations which preserve edges much better.

6.2

Area-Preserving Mappings and Optical Flow

The optical flow constraint above is of course very strong. Motivated by Moser [23], we would like to propose a modification of this that also could be placed in a variational setting. Namely, the Moser construction described above allows one to do the following: Given a family of nowhere-zero 2-forms τt , we have an explicit method to determine a family of diffeomorphisms φt such that φ∗t τt = τ0 . Differentiating this expression with respect to t yields ∂ τt + h∇τt , ut i + τt div (ut ) = 0. ∂t This is very similar in form to the standard optical flow constraint with the divergence term added. One can interpret image intensity as a type of form, and applying the Moser analysis under a much less restrictive assumption than the standard optical flow constraint given in equation (21). This constraint may be then used in conjunction with an L1 total variational problem as in [21] for a natural edge-preserving optical flow approach.

7

Computer Simulations

We have applied the flow (20) to a pair of heart images. The left image of Figure 1 is the heart in the diastolic state, and the right image represents the systolic part of the cycle. Using our interpolation method, we indicate the warped deformation grid derived by optimally interpolating one image to the other in Figure 2. The simulation took about 3 seconds on a standard Sun Ultrasparc 10 Workstation. A similar method could be used to study the optical flow of such images as well.

References 1. S. Angenent, S. Haker, A. Tannenbaum, and R. Kikinis, “On area preserving maps of minimal distortion,” in System Theory: Modeling, Analysis, and Control, edited by T. Djaferis and I. Schick, Kluwer, Holland, pages 275-287, 1999. 2. S. Angenent, S. Haker, A. Tannenbaum, and R. Kikinis, “Laplace-Beltrami operator and brain surface flattening,” IEEE Trans. on Medical Imaging 18, pp. 700-711, 1999. 3. S. Angenent, S. Haker, A. Tannenbaum, and R. Kikinis, “Nondistorting flattening maps and the 3D visualization of colon CT images,” IEEE Trans. of Medical Imaging 19, pp. 665-671, 2000. 4. J. L. Barron, D. J. Fleet, and S. S. Beauchemin, “Performance of optical flow techniques,” International Journal of Computer Vision 12, pp. 43–77, 1994.

Optimal Image Interpolation

141

Fig. 1. Diastolic Heart Image (left) and Systolic Heart Image (right)

5. J.-D. Benamou and Y. Brenier, “A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem,” Numerische Mathematik 84, pp. 375-393, 2000. 6. A. D. Bimbo, P. Nesi, and J. L. C. Sanz, “Optical flow computation using extended constraints,” . IEEE Transactions on Image Processing, 5, pp.720739, 1996. 7. Y. Brenier, “Polar factorization and monotone rearrangement of vector-valued functions,” Com. Pure Appl. Math. 64, pp. 375-417, 1991. 8. M. Cullen and R. Purser, “An extended Lagrangian theory of semigeostrophic fronogenesis,” J. Atmos. Sci. 41, pp. 1477-1497. 9. D. Fry, Shape Recognition Using Metrics on the Space of Shapes, Ph.D. Thesis, Harvard University, 1993. 10. W. Gangbo and R. McGann, “The geometry of optimal transportation,” Acta Math. 177, pp. 113-161, 1996. 11. W. Gangbo and R. McGann, “Shape recognition via Wasserstein distance,” Technical Report, School of Mathematics, Georgia Institute of Technology, 1998. 12. W. Gangbo, “An elementary proof of the polar factorization of vector-valued functions,” Arch. Rational Mechanics Anal. 128, pp. 381-399, 1994. 13. M. Grayson, “The heat equation shrinks embedded plane curves to round points,” J. Differential Geometry 26 , pp. 285–314, 1987. 14. S. Haker and A. Tannenbaum, “Optimal transport and image registration,” submitted for publication in IEEE Trans. Image Processing, January 2001. 15. R. Haralick and L. Shapiro, Computer and Robot Vision, Addison-Wesley, New York, 1992. 16. E. C. Hildreth, “Computations underlying the measurement of visual motion,” Artificial Intelligence 23, pp. 309-354, 1984. 17. B. K. P. Horn, Robot Vision, MIT Press, Cambridge, Mass., 1986. 18. B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artificial Intelligence 23, pp. 185-203, 1981.

142

StevenÿHaker and AllenÿTannenbaum

Fig. 2. Deformation of the Grid

19. L. V. Kantorovich, “On a problem of Monge,” Uspekhi Mat. Nauk. 3, pp. 225226, 1948. 20. M. Knott and C. Smith, “On the optimal mapping of distributions,” J. Optim. Theory 43, pp. 39-49, 1984. 21. A. Kumar, A. Tannenbaum, and G. Balas, “Optical flow: a curve evolution approach,” IEEE Transactions on Image Processing 5, pp. 598–611, 1996. 22. R. McCann, “Polar factorization of maps on Riemannian manifolds,” preprint 2000. Available on http://www.math.toronto.edu/mccann. 23. J. Moser, “On the volume elements on a manifold,” Trans. Amer. Math. Soc. 120, pp. 286-294, 1965. 24. H.-H. Nagel and W. Enkelmann, “An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,” IEEE Trans. Pattern Analysis and Machine Intelligence PAMI-8, pp. 565–593, 1986. 25. S. Rachev and L. R¨ uschendorf, Mass Transportation Problems, Volumes I and II, Probabiltiy and Its Applications, Springer, New York, 1998. 26. A. Rantzer, “A dual to Lyapunov’s stability theorem,” Systems and Control Letters 42, pp. 161-168, 2001.

Optimal Image Interpolation

143

27. B. G. Schunck, “The motion constraints equation for optical flow,” Proceedings of the Seventh IEEE International Conference on Pattern Recognition, pp. 20– 22, 1984. 28. A. Toga, Brain Warping, Academic Press, San Diego, 1999. 29. P. Viola and S. Wells, “Alignment by maximization of mutual information,” Int. J. Computer Vision 24, pp. 137–154, 1997.

Robustness of Finite State Automata Alexandre Megretski MIT, Cambridge MA 02139, USA

Abstract. The classical robust control deals with systems which can be approximated by finite order linear time-invariant (LTI) models, uses integral constraints, such as induced gain bounds, to assess robustness with respect to the error of such approximation, and employs H-Infinity optimization to design robust linear controllers. In this paper1 , a parallel approach is developed, in which finite state stochastic automata play the role of LTI models. Analogs of the KalmanYakubovich-Popov Lemma, the S-procedure losslessness theorem, and H-Infinity design are derived.

Introduction Robustness analysis and optimization is a major source of efficient design tools for the modern control engineer. The classical robust control deals with systems which can be approximated by LTI models. The difference between such approximations and the true system dynamics is described by integral constraints, such as induced gain bounds or Integral Quadratic Constraints [1]. Constructively verifiable conditions of stability and performance, such as the small gain theorem, are used to assess stability and performance of systems defined by nominal LTI dynamics and integral constraints. Ultimately, the task of robust LTI feedback design is reduced to induced gain minimization, such as H-Infinity optimization, which employs extensively quadratic Lyapunov functions. While being the dominant tool for computer-aided design and analysis of systems modeled by near-linear differential equations, this framework fails to provide adequate treatment in the case of hybrid systems, i.e. systems which combine continuous and discrete state dynamics. A major objective of the paper is creation of an alternative robust control framework in which finite state stochastic automata serve as a basic system model. Systems under consideration are represented as interconnections of “nominal” controlled finite state automata and the “uncertain feedback” systems described by integral constraints representing modeling error. Lyapunov functions are used for analysis and design. The theorems presented in this paper are quite elementary, and can be viewed as simplified versions of the standard results of dynamic programming [2]. However, they highlight a potentially powerful framework for nonlinear feedback design. In this framework, one has to start with finding a reduced model of the original system. 1

[email protected]. This work was supported by NSF, AFOSR, and DARPA

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 147--160, 2003 Springer-Verlag Berlin Heidelberg

148

1

Alexandre Megretski

System Models

In this section, basic principles of system modeling and design using finite state stochastic automata and integral constraints are introduced.

1.1

Finite Alphabet Feedback Design

This subsection contains motivation for using the uncertain finite state automata models as defined later in the paper.

Observer-Based Feedback. Our ultimate goal is to develop tools for optimizing the controller K in the feedback loop shown on Figure 1, where P is a strictly u(t)

-

s(t) P

-

- h(t)

K

Fig. 1. General Feedback Design Setup causal discrete time system (possibly uncertain, infinite dimensional, and randomized) with control input u(t) ∈ U , sensor output s(t) ∈ S, and cost output h(t) ∈ R. We consider the case of discrete decision-making, which means that the set U (the control alphabet) is finite. The objective is to design a causal system K (the feedback controller) with input s(t) and output u(t) such that h(t) is non-negative “on average” on the trajectories of the closed loop system, which is expressed by the inequality inf

T >0

T X

Eh(t) > −∞.

(1)

t=0

The general task of designing and optimizing K is very difficult. However, an important simplifying assumption will be made throughout the paper, that K must be found in the observer-based form shown on Figure 2, where D is the one step delay block, m(t) ∈ M is the observer state, x(t) ∈ X is the observer output (the set X is finite), function E : U × S × M → X × M defines the observer dynamics, and function g : X×Ξ → U defines the control decision randomized by an independent random number generator θ(t) ∈ Θ. The challenging task of designing the observer function E is not discussed systematically in this paper. It is assumed that some preliminary effort (for example, a model reduction and quantization algorithm) has already produced E. Our objective is to develop algorithms for design and analysis of the randomized memoryless feedback part of the observerbased controller represented by function g and by the probability distribution pθ of θ.

Robustness of FSA u(t − 1)

D ¾

-

x(t)

- E -

s(t)

m(t)

149

D ¾

-

θ(t) -

- u(t)

g

m(t + 1)

Fig. 2. Observer-Based Feedback

Uncertain Finite State Automata Models. For the purpose of designing the randomized memoryless feedback defined by g and pθ , the observer based feedback system can be described by the diagram on Figure 3, where H represents the u(t) x(t)

H

- h(t)

-

θ(t)

-

g

Fig. 3. Closed Loop System combination of plant and observer. While the set of possible values of v(t) and x(t) at given time is finite, the exact dynamics of H can be extremely complicated. We propose the use of uncertain stochastic finite state automata (FSA) with integral constraints (IC) as a tool for simplified representation (abstraction) of H, as shown on Figure 4, i.e.

u(t) w(t) ξ(t)

Fig. 4. Uncertain FSA Model

- f -

x(t + 1)

- D

- σ

- x(t)

150

Alexandre Megretski x(t + 1) = f (x(t), u(t), w(t), ξ(t)),

(2)

where w(t) ∈ W is the uncertain input, representing possible mismatch between H and the FSA model, ξ(t) ∈ Ξ is the output of an independent random number generator with a given probability distribution pξ , f : X × U × W × Ξ → X is the function defining the FSA. The behavior of the uncertain input w(t) is constrained by a set of integral constraints inf

T >0

T X

Eσ(x(t), u(t), w(t)) > −∞,

(3)

t=0

which are assumed to hold for all functions σ : X ×U ×W → R from a given convex compact set σ ˜ = {σ}. In addition, a given function σ0 : X ×U ×W → R is assumed to provide a lower bound for the averages of h(t), in the sense that the performance inequality in (1) holds for every set of input/output signals (u(t), x(t), h(t)) produced by H, as long as there exist w(t), ξ(t) satisfying (2),(3), and inf

T >0

T X

Eσ0 (x(t), u(t), w(t)) > −∞.

(4)

t=0

Under these assumption, the design problem under consideration can be formulated as that of finding a function g and a random number generator distribution pθ such that the inequality (4) is satisfied for all solutions of (2) satisfying u(t) = g(x(t), θ(t)) and (3).

1.2

FSA Models

In this paper, finite state stochastic automata (FSA) are used to define nominal, i.e. precisely known, system models. Thus, they play the role of finite order LTI models of the classical robust control.

Random Variables. The set of all random variables ξ with values in a given finite set Ξ (without loss of generality, ξ can be viewed as a measurable function ξ: W→ 7 Ξ, where W = [0, 1] is the “set of elementary events”) is denoted by R(Ξ). D(Ξ) is the set of all probability distributions on Ξ, i.e. functions p : Ξ → [0, 1] such that X ¯ = 1. p(ξ) ¯ ξ∈Ξ

The function π = πΞ maps every random variable ξ ∈ R(Ξ) into its distribution ¯ = P(ξ = ξ) ¯ is the pξ = π(ξ) ∈ D(Ξ). In other words, pξ = π(ξ) means that pξ (ξ) ¯ In particular, if h : Z → R probability that the random variable ξ takes the value ξ. is a given function, and z ∈ R(Z), then the expected value Eh(z) is given by Eh(z) =

X z ¯∈Z

h(¯ z )pz (¯ z ).

Robustness of FSA

151

For two random variables x ∈ R(X) and y ∈ R(Y ), their direct product ξ = (x, y) is the random variable from R(X × Y ) defined by ξ(τ ) = (x(τ ), y(τ )) for all τ ∈ W = [0, 1]. x and y are called independent if x, y¯) = px (¯ x)py (¯ y) ∀ x ¯ ∈ X, y¯ ∈ Y, pξ (¯ where pξ , px , py are the distributions of ξ, x, y respectively. We will write x ⊥ y when x and y are independent.

Finite State Automata. Let X, V, Z be three finite sets. A function f : X × V ×Z → 7 X and a probability distribution pz ∈ D(Z) define a finite state automata A = A(f, pz ) as a relation between sequences of random variables x(t) ∈ R(X) and v(t) ∈ R(V ) expressed by x(t + 1) = f (x(t), v(t), z(t)), z(t) ⊥ (x(t), v(t)), πZ (z(t)) = pz .

(5)

Here v(t) is the input of a discrete time dynamical system with state x(t) and independent identically distributed random number generator z(t). An equivalent expression of the FSA relation (5) is given by X pt+1 x) = pt(x,v) (¯ x, v¯)pz (¯ z ), x (˜ x ˜=f (¯ x,¯ v ,¯ z) t where pt+1 x , p(x,v) are the distributions of x(t + 1) and (x(t), v(t)) respectively. In this paper, FSA are used as simplified models of systems.

Memoryless Automata. Let U, X, Θ be three finite sets. A function g : X × Θ → U and a probability distribution pθ ∈ D(Θ) define a memoryless automata M = M(g, pθ ) as a relation between sequences of random variables x(t) ∈ R(X) and u(t) ∈ R(U ) expressed by u(t) = g(x(t), θ(t)), θ(t) ⊥ x(t), πΘ (θ(t)) = pθ .

(6)

Here x(t) is the input of a memoryless system with output u(t) and an independent identically distributed random number generator θ(t). An equivalent way of defining the memoryless automata is by specifying a function pu|x : X → D(U ), which defines the conditional distribution of u(t) for every given value x(t) = x ¯. The relation between g, pθ , and pu|x is given by pu|x (¯ x, u ¯) =

X

¯ pθ (θ).

¯ g(¯ ¯ u θ: x,θ)=¯

An equivalent expression of (6) is given by X X ¯ = ptu (¯ u) = ptx (¯ x)pθ (θ) pu|x (¯ x, u ¯)ptx (¯ x), ¯ u ¯ x ¯,θ: ¯ =g(¯ x,θ)

x ¯∈X

where ptu , ptx are the distributions of u(t) and x(t) respectively. In this paper, memoryless automata are used as feedback laws.

152

Alexandre Megretski

1.3

Integral Constraints

Most systems of practical interest cannot be represented exactly as finite state automata. However, they can frequently be approximated to a reasonable degree of accuracy by FSA, just as many mildly nonlinear systems can be approximated by LTI models. Integral constraints (IC) serve as indicators of accuracy of such approximations, playing the role of L2 gain bounds or Integral Quadratic Constraints in the classical robust control. In addition, IC can express the objectives for feedback design, in the same way as L2 gain bounds serve as design objectives of the standard H-Infinity optimization.

Definition of Integral Constraints. A sequence {y(t)} of real-valued inte-

grable random variables y(t) such that inf

T >0

T X

Ey(t) > −∞

t=0

is said to satisfy the integral constraint I[y(t)] ≥ 0. Let X, V be two finite sets, and let σ : X ×V → R be a function. Two sequences of random variables x(t) ∈ R(X) and v(t) ∈ R(V ) are said to satisfy the integral constraint (IC) defined by σ if inf

T >0

T X

Eσ(x(t), v(t)) > −∞,

(7)

t=0

in which case we will write I[σ(x(t), v(t))] ≥ 0. An equivalent way to express the IC in (7) is given by inf

T >0

T X X

σ(¯ x, v¯)pt(x,v) > −∞,

t=0 x ¯,¯ v

where pt(x,v) is the distribution of (x(t), v(t)). In this paper, integral constraints are used for defining performance criteria and for constraining undermodeled behavior of uncertain models.

Integral Constraints as Performance Specifications. In this case the IC in (7) is a hypothesis to be verified by a system analysis procedure, or a design criterion to be satisfied by a design choice. For example, assume that certain elements x0 ∈ X, v0 ∈ V are designated as zero values. If the input v = v(t) of FSA (5) represents a control decision, the informal performance criterion may require that x(t), v(t) take zero values x(t) = x0 , v(t) = v0 with probability 1 as t → ∞. If σ is defined in such way x, v¯) < 0 for (¯ x, v¯) = 6 (x0 , v0 ), then (6) implies that that σ(x0 , v0 ) = 0 and σ(¯ P(x(t) = x0 , v(t) = v0 ) → 1 as t → ∞. In this case (6) plays a role similar to that of a quadratic performance criterion in the classical linear-quadratic optimization. Another example of a performance specified by an IC is as follows. Assume the input v models a disturbing noise, and one has to verify that x(t) is not very sensitive to v(t), which means that, on average, x(t) will take a “non-zero” value

Robustness of FSA

153

x(t) 6= x0 with a frequency not exceeding γ times the frequency of v(t) taking non-zero values v(t) 6= v0 , where γ > 0 is a given number quantifying the degree of sensitivity. In this case, σ can be defined by  −1, x ¯ 6= x0 , v¯ = v0 ,    γ, x ¯ = x0 , v¯ 6= v0 , σ(¯ x, v¯) = (8) γ − 1, x ¯ 6= x0 , v¯ 6= v0 ,    0, x ¯ = x0 , v¯ = v0 . and then (7) plays a role similar to that of a L2 gain bound in H-Infinity optimization.

Integral Constraints as Uncertainty Bounds. In this case the IC in (7) is a constraint limiting the behavior of uncertain signals within the system. For example, v(t) may represent the output of an undermodeled subsystem ∆ with input x(t). Assume that ∆ satisfies a “low sensitivity” condition which means that on average its output v(t) takes non-zero values v(t) = 6 v0 with a frequency smaller than 1/γ times the frequency at which its input x(t) takes non-zero values x(t) = 6 x0 . This condition can be represented by the IC I[−σ(x, v)] ≥ 0, where σ is the function defined in (8).

2

Analysis and Design of FSA/IC Models

This section presents general results on analysis and design of systems defined as finite state automata with integral constraints.

2.1

Analysis of FSA/IC Models

Analogs of the Kalman-Yakubovich-Popov Lemma and the S-procedure losslessness theorem will be formulated and proven here for FSA/IC models. First, existence of a storage function is shown to be a necessary and sufficient condition for an integral constraint to be satisfied for all trajectories of a given finite state automata model. Second, it is shown that a finite set of IC I[σk (x, v)] ≥ 0, k = 1, 2, ..., n, imposed on the set of all trajectories of a given FSA implies IC I[σ0 (x, v)] ≥ 0 if and only if there exist nonnegative coefficients ck ≥ 0 such that, for σ(¯ x, v¯) = σ0 (¯ x, v¯) −

n X

ck σk (¯ x, v¯),

k=1

the IQC I[σ(x, v)] ≥ 0 is satisfied for all trajectories of the FSA.

Integral Constraints and Storage Functions. The following statement can be used to check whether a given integral constraint is satisfied for all trajectories of a given finite state automata. Result 1. Let X, V, Z be finite sets, pz ∈ D(Z). Let f : X × V × Z → X and σ : X × V → R be two functions. Then the following two conditions are equivalent.

154

Alexandre Megretski

(a) The IC I[σ(x, v)] ≥ 0 from (7) holds for all sequences of random variables x(t), v(t), z(t) satisfying (5). (b) There exists a function H : X → R such that X H(f (¯ x, v¯, z¯))pz (¯ z) ∀ x ¯ ∈ X, v¯ ∈ V. (9) σ(¯ x, v¯) + H(¯ x) ≥ z ¯∈Z

Essentially, (9) means that Eσ(x(t), v(t)) ≥ EH(x(t + 1)) − EH(x(t))

(10)

for all solutions of the FSA equation (7), i.e., using the terminology by J.C.Willems, that the function H = H(x) can serve as a storage function for (7) with supply rate σ = σ(x, v).

Proof of Theorem 1. Taking into account (10), the implication (b)⇒(a) is straightforward: taking a sum of such inequalities with t = 0, 1, . . . , T yields T X

Eσ(x(t), v(t)) ≥ EH(x(T + 1)) − EH(x(0)).

t=0

Since X is finite, EH(x(T + 1)) is uniformly bounded, and hence inf

T ≥0

T X

Eσ(x(t), v(t)) > −∞.

t=0

To prove that (a) implies (b), consider the case when there exists no function H : X → R satisfying (9). This means that the minimum of the convex function ) ( X H(f (¯ x, v¯, z¯))pz (¯ z) (11) −H(¯ x) − σ(¯ x, v¯) + f (H) = max x ¯∈X,¯ v ∈V

z ¯∈Z

over the vector space H = {H} of all functions H : X → R is positive. In terms of the dual linear program, this means existence of a probability distribution r ∈ D(X × V ) such that X r(¯ x, v¯)σ(¯ x, v¯) = c < 0, (12) x ¯∈X,¯ v ∈V

X

X

r(˜ x, w) ¯ =

v ¯∈V

Let r¯(¯ x) =

r(¯ x, v¯)pz (¯ z) ∀ x ˜ ∈ X.

(13)

x ¯∈X,¯ v ∈V,¯ z ∈Z: f (¯ x,¯ v ,¯ z )=˜ x

X

r(¯ x, w). ¯

w∈W ¯

We will use the following (obvious) observation. x) = r¯ there exists a Lemma 1. For any random variable x ˆ ∈ R(X) such that πX (ˆ x, vˆ)) = r. random variable vˆ ∈ R(V ) such that πX×V ((ˆ

Robustness of FSA

155

Lemma 1 can be used to construct sequences of random variables x = x(t), v = v(t) satisfying (5) and such that E¯ σ (x(t), v(t)) = c < 0 ∀ t = 0, 1, 2, . . . ,

(14)

which contradicts assumption (a). Indeed, let x(0) be a random variable with values on X and probability distribution r¯. Using Lemma 1 with x ˆ = x(0), define v(0) by v(0) = vˆ. Then, by (12), the inequality in (14) holds, and, by (13), the probability distribution of x(1) is the same as that of x(0). Now the process can be repeated by applying Lemma 1 with x ˆ = x(t) and using v(t) = vˆ for t = 1, 2, . . . .

Calculation of Storage Functions Formally speaking, checking validity of a given IC on the trajectories of a given FSA using Theorem 1 amounts to solving a linear program with respect to H. However, the following observations may help in reducing complexity of the analysis. Result 2. Let M > 0 be a given number. The following conditions are equivalent: (a) inequality (9) has a solution H with x) − min H(¯ x) ≤ M ; max H(¯ x ¯∈X

x ¯∈X

(b) for all n = 1, 2, . . . the functions Hn : X → R defined by Hn+1 = α(Hn ), H0 = 0, ¯ = α(H) is given by where H ( )) ( X ¯ H(¯ x) = max 0, max −σ(¯ H(f (¯ x, v¯, z¯))pz (¯ z) , x, v¯) + v ¯

z ¯∈Z

satisfy the conditions x), max Hn (¯ x) ≤ M. 0 = min Hn (¯ x ¯∈X

x ¯∈X

Moreover, if conditions (a),(b) are satisfied then one such H is given by H(¯ x) = H− (¯ x) = lim Hn (¯ x), n→∞

and H(¯ x) ≥ H− (¯ x) for any other non-negative solution of (9). Proof. (a)⇒(b) If H : X → R is a solution of (9) then for every constant c ∈ R the x) = H(¯ x) + c is also a solution. Therefore, it is sufficient to look only function Hc (¯ for those solutions of (9) which satisfy x) = 0. min H(¯

x ¯∈X

(15)

Note that α is a monotonically non-decreasing transformation, in the sense that α+ (H1 )(¯ x) ≥ α+ (H2 )(¯ x) ∀ x ¯ ∈ X whenever H1 (¯ x) ≥ H2 (¯ x) ∀ x ¯ ∈ X. Hence H∗ ≥ 0 = H0 implies H∗ = α(H∗ ) ≥ α(H0 ) = H1 , and, further by induction, M ≥ H∗ ≥ Hn . (a)⇐(b) Since H1 ≥ 0 = H0 and α is monotonically non-decreasing, we have ¯ ∈ X the sequence Hn (¯ x) is monotonically Hn+1 ≥ Hn for all n. Hence for every x non-decreasing and bounded, and thus converges to a limit H− such that H− = x) ≤ M for all x ¯ ∈ X. Therefore H = H− satisfies (9) as well. α(H− ) and 0 ≤ H− (¯

156

Alexandre Megretski

S-Procedure Losslessness. The following statement can be used to check whether a given integral constraint is satisfied for all of those trajectories of a given finite state automata which satisfy integral constraints from a given set. Result 3. Let X, V, Z be finite sets, pz ∈ D(Z). Let f : X × V × Z → X, ˜ = {¯ σ } be a convex compact set of σ0 : X × V → R be two functions. Let σ functions σ ¯ : X × V → R such that (*) for each σ ¯ ∈ σ ˜ the IQC I[−¯ σ (x(t), v(t))] ≥ 0 does not hold for at least one random sequence (x(t), v(t)) satisfying (5). Then the following two conditions are equivalent. (a) The IC I[σ0 (x, v)] ≥ 0 holds for all sequences of random variables x(t), v(t), z(t) satisfying (5) and every IQC I[¯ σ (x(t), v(t))] ≥ 0 with σ ¯∈σ ˜. (b) There exists a function H : X → R, µ ≥ 0 and σ ¯∈σ ˜ such that X x, v¯) − µ¯ σ (¯ x, v¯) + H(¯ x) ≥ H(f (¯ x, v¯, z¯))pz (¯ z) ∀ x ¯ ∈ X, v¯ ∈ V. (16) σ0 (¯ z ¯∈Z

Proof. The proof of Theorem 3 follows the lines of the proof of Theorem 1, with some minor modifications. The implication (b)⇒(a) is obvious. To prove that (a) implies (b), assume that (b) is false. Then, by assumption (*), (9) does not have a solution H, σ, where σ = τ σ0 − (1 − τ )σ with τ ∈ [0, 1] ranges over the convex hull σ . In other words, the maximum f = f (H, σ) in (11), which is now σ ˆ of σ0 and −˜ a convex function on H × sˆ, has a strictly positive minimum. Applying standard duality yields existence of r ∈ D(X × V ) such that (13) holds, and (12) holds for all σ ∈ σ ˆ . Hence the sequence (x(t), v(t)) constructed as in the proof of Theorem 1, will satisfy the conditions Eσ(x(t), v(t)) = c < 0, Eσ(x(t), v(t)) = −c > 0, which contradicts (a).

2.2

Feedback Design for FSA Models

This subsection is devoted to the problem of designing randomized full state feedback for uncertain FSA.

Randomized Feedback in Uncertain FSA. Finite sets X, U, W, Ξ, a probability distribution pξ ∈ D(Ξ), and a function f : X × U × W × Ξ → X define a FSA A = A(f, pξ ) in which the input variable v(t) = (u(t), w(t)) ∈ R(U × W ) is partitioned into control u(t) ∈ R(U ) and disturbance w(t) ∈ R(W ). A randomized feedback for the FSA is defined by a memoryless automata u(t) = g(x(t), θ(t)) with input x(t) and random number generator θ(t) with a fixed probability distribution pθ ∈ D(Θ). Here θ(t) ⊥ (x(t)). An alternative way to define a randomized memoryless feedback is by specifying the conditional distribution pu|x : X → D(U ), where X ¯ x, u ¯) = pθ (θ). pu|x (¯ ¯ g(¯ ¯ u θ: x,θ)=¯

Robustness of FSA

157

By the meaning of the FSA model as an approximation of a complex dynamical system with input u(t), it is not reasonable to assume independence of w(t) and θ(t). Therefore, to represent the resulting feedback system F = F (f, pξ , g, pθ ) in the form (5), let us define xc (t) = (x(t), u(t)) ∈ X ×U as the state of F , z(t) = (ξ(t), θ(t+1)) ¯ θ) ¯ = pξ (ξ)p ¯ θ (t¯), w(t) as the as its random number generator with distribution pz (ξ, disturbance input, and fcl

µ· ¸ · ¸¶ · ¸ ¯ x ¯ ξ¯ f (¯ x, u ¯, w, ¯ ξ) , w, ¯ ¯ = ¯ θ) ¯ . u ¯ θ g(f (¯ x, u ¯, w, ¯ ξ),

A typical objective of feedback design is to satisfy an integral constraint I[σ(x(t), u(t), w(t))] ≥ 0, where σ is either a given function (when w plays the role of external disturbance, and optimization of nominal performance is the goal) or can be selected from a given convex set σ ˆ = {σ} of functions (in the case when some components of w model dynamical uncertainty, and hence robust performance is to be optimized).

Design Feasibility of Integral Constraints. The following result gives necessary and sufficient conditions of feasibility in a feedback optimization problem for FSA with a single integral constraint defining the design objective. Result 4. Let finite sets X, U, W, Ξ, a distribution pξ ∈ D(Ξ), and functions f : X × U × W × Ξ → X and σ : X × U × W → R be given. The following conditions are equivalent. (a) There exists a randomized feedback u(t) = g(x(t), θ(t)) such that the integral constraint I[σ(x(t), u(t), w(t))] ≥ 0 holds for all solutions of the closed loop system F = F (f, pξ , g, pθ ). (b) There exists a function H : X → R such that    X X ¯ ξ (ξ) ¯ p(¯ H(¯ x) ≥ min max u) (17) H(f (¯ x, u ¯, w, ¯ ξ))p −σ(¯ x, u ¯, w) ¯ + ¯   p∈D(U ) w∈W ¯ u ¯∈U

ξ∈Ξ

for all x ¯ ∈ X. Note that the optimal distributions p ∈ D(U ) in (17), one for each x ¯ ∈ X, define the conditional distribution p(u|x) : X → D(U ) of the desired randomized feedback. Practically, the search for the control storage function H in (17) is frequently reduced to the value iteration procedure Hn+1 = β(Hn ), H0 = 0, where the function β : H → H is defined by β(H)(¯ x) = max

  

0, min max ¯ p∈D(U ) w∈W

 X u ¯∈U



−σ(¯ x, u ¯, w) ¯ +

X ¯ ξ∈Ξ

¯ ξ (ξ) ¯ H(f (¯ x, u ¯, w, ¯ ξ))p

  

p(¯ u)

  

.

The iterative techniques suggested earlier for FSA storage function analysis extend naturally to the design case.

158

Alexandre Megretski

Proof of Theorem 4. The implication (b)⇒(a) in Theorem 4 is straightforward, since the optimal distributions p ∈ D(U ) in (17) define the conditional distribution p( u|x) : X → D(U ) of a desired randomized feedback such that Eσ(x(t), u(t), w(t)) ≥ EH(x(t + 1)) − EH(x(t)) for all random variables w(t). To prove that (a) implies (b), note first that, according to Theorem 1, the IC I[σ(x(t), u(t), w(t))] ≥ 0 is satisfied for the closed loop system if and only if there ˆ : X × U → R such that exists a function H X ¯ u ˆ (¯ ˆ x, u H(f x, u ¯, w, ¯ ξ), ˆ)pu|x (f (¯ x, u ¯, w, ¯ z¯), u ˆ)pz (¯ z) H(¯ ¯) ≥ −σ(¯ x, u ¯, w) ¯ + u ˆ ∈U,ξ∈Ξ

for all x ¯ ∈ X, u ¯ ∈ U, w ¯ ∈ W . For a fixed w ¯ ∈ W , multiplying these inequalities by x, u ¯) and summing up over all u ¯ yields (17) for pu|x (¯ H(¯ x) =

X

ˆ x, u H(¯ ¯)pu|x (¯ x, u ¯).

u ¯ ∈U

Example: Single Bit Memory Stabilization of Double Integrator.

It is known that memoryless output feedback uc (τ ) = K(yc (τ )) is not capable of stabilizing the double integrator system y¨c (τ ) = uc (τ ).

(18)

However, the stabilization can be accomplished with a single bit of memory. The problem of finite memory stabilization can be reduced to robust FSA feedback design in the following way. Consider the sampled data feedback control law uc (τ ) = −ω(u(t))2 yc (τ ) for tT ≤ τ ≤ (t + 1)T,

(19)

where T ∈ (0, π/4) is a fixed sampling rate, t = 0, 1, 2, . . . is the discrete time, w : {0, 1} → R is a given function, ³ π ´ , ω(0) = 1, ω(1) = ω1 ∈ 1, 4T and u(t) ∈ {0, 1} is the output of a FSA (to be designed) with input s(t) = sign(yc (tT )). The feedback design objective is to maximize the stabilization rate, hence h(t) can be defined as the amount by which the logarithm of the state vector length decreases over a sampling interval [tT, tT + T ]: ¶ µ |yc (tT )|2 + |y˙ c (tT )|2 − γ, h(t) = 0.5 log |yc (tT + T )|2 + |y˙ c (tT + T )|2 where γ > 0 is the parameter to be maximized. The setup is shown on Figure 5, where the A/D block represents the sign sampler.

Robustness of FSA

> ½ ½ - ½ ½K ½

uc

u

s

- 1/s2

159

yc

½

F SA ¾

A/D ¾

Fig. 5. Finite Memory Feedback for Double Integrator Let us use the observer with a single bit state m(t) ∈ M = {−1, 1} and output x(t) ∈ X = {0, 1} defined by m(t + 1) = s(t), x(t) = 0.25(m(t) − s(t))2 . Thus x(t) is the indicator that the sign of yc has changed over the last sampling interval. In terms of this paper, the transformation of the quantized control input u(t) ∈ {0, 1} into the observer output x(t) ∈ {0, 1} defines a “complex” open loop model H. A simplified model of H (its abstraction) can be defined by the equations x(t + 1) = w(t). To make this simplified model useful for design and analysis, one has to put integral constraints on the uncertain input variable w(t), and to provide a lower bound for the average value of h(t) in terms of x(t), u(t), w(t). Let · ¸ ¸ · sin(q) cos(ω1 T ) ω1−1 sin(ω1 T ) , eq = , M= cos(q) −ω1 sin(ω1 T ) cos(ω1 T ) |e0q M eq | kM 2 e0 k , ρ2 = log kM k, φ0 = arccos max , q∈[0,π] kM eq k kM e0 k where kLk denotes the largest singular value of matrix L. Here M is the matrix of the linear transformation xc (tT + T ) = M xc (tT ) of the analog state · ¸ yc (τ ) xc (τ ) = y˙ c (τ ) ρ1 = − log

of the system over a single time sampling interval τ ∈ [tT, tT + T ] when u(t) = 1; −ρ1 is the maximal possible increment of log kxc (τ )k over a single time sampling interval when u(t) = x(t) = 1; ρ2 is the maximal possible increment of log kxc (τ )k over a single time sampling interval when u(t) = 1 (note that kxc (τ )k does not change when u(t) = 0); φ0 is the minimal increment in the phase of xc (τ ) over a single time sampling interval when u(t) = 1 (the increment equals T when u(t) = 0). By construction, h(t) ≥ ρ1 − γ when x(t) = u(t) = 1. On the other hand, h(t) = 0 when u(t) = 0, and the inequality h(t) ≥ −ρ2 − γ always holds. Therefore a lower bound for h(t) can be derived in terms of x(t), u(t), w(t) according to h(t) ≥ σ0 (x(t), u(t), v(t)) = [ρ1 x(t) − ρ2 {1 − x(t)}]u(t) − γ.

160

Alexandre Megretski

On the other hand one switch of the sign of yc (τ ) occurs (on average) for every π radians increment of the phase of xc (τ ). Moreover, not more than one sign switch can take place during a single sample interval, and hence the total number of sign switches over the time interval τ ∈ [0, T t0 ] equals the sum of x(t) from t = 0 to t = t0 . Hence the integral constraint I[σ(x(t), u(t), w(t))] ≥ 0 holds for σ(x(t), u(t), w(t)) = πx(t) − φ0 u(t) − T (1 − u(t)). With these functions σ0 , σ, the FSA/IC design formulation is complete, and Theorems 1-3 can be applied to optimize a randomized feedback law. Indeed, for the design feasibility of the reduced single bit model, the performance condition I[σ0 ] ≥ 0 must be satisfied subject to I[σ] ≥ 0 and the FSA equation. According to Theorem 3, this means existence of µ ≥ 0 such that the IC I[σ0 −µσ] ≥ 0 holds subject to the FSA equation x(t + 1) = w(t). According to Theorem 4, this means existence of a storage function satisfying (17), i.e. X {−¯ σ (¯ x, u ¯) + H(w)}p(¯ ¯ u), H(¯ x) ≥ min max p

w ¯

u ¯

where x − ρ2 ]¯ u − γ − µ[π¯ x − (φ0 − T )¯ u − T ]. σ ¯ (¯ x, u ¯) = [(ρ1 + ρ2 )¯ Here the maximum with respect to w ¯ is achieved when H(w) ¯ = max H, independently of what p is, and hence the minimum with respect to the distribution p is achieved at an atomic distribution. If we assume (without loss of generality) that max H = 0, (17) further collapses to σ (¯ x, u ¯)}. 0 ≥ H(¯ x) ≥ min{−¯ u ¯

ˆ where dˆ is the minimum (with respect Finally, the largest achievable γ equals −d, to µ ≥ 0) of d(µ) = max{min{−µT, −µφ0 + ρ2 }, min{µ(π − T ), µ(π − φ0 ) − ρ1 }}. For example, for T = 0.22, ω1 = π/(4T ) ≈ 3.57 this yields the maximal γ ≈ 0.067 at µ ≈ 0.31, with the optimal control u(t) = x(t).

References 1. A. Megretski and A. Rantzer, System Analysis via Integral Quadratic Constraints, IEEE Transactions on Automatic Control, volume 47, no. 6, pp. 819– 830, June 1997. 2. Dimitri P. Bertsekas, Dynamic Programming and Optimal Control: 2nd Edition, Athena Scientific, 2000/2001.

On the role of homogeneous forms in robustness analysis of control systems G. Chesi1 , A. Garulli1 , A. Tesi2 , and A. Vicino1 1 2

Dipartimento di Ingegneria dell’Informazione, Universit`a di Siena Via Roma, 56 - 53100 Siena, Italia Dipartimento di Sistemi e Informatica, Universit`a di Firenze Via di S. Marta, 3 - 50139 Firenze, Italia

Abstract. This paper considers the problem of positivity (or nonnegativity) of one-parameter families of homogeneous polynomial forms, i.e. of forms whose coefficients depend on a scalar parameter which is allowed to vary over either a continuous or a discrete set. The main contribution of the paper is an efficiently computable sufficient condition for positivity (or nonnegativity), which requires the solution of a Linear Matrix Inequalities (LMI) optimization problem for each value of the scalar parameter. Moreover, necessity of such a condition is investigated and proven to hold for some families of homogeneous forms. The paper also shows that several important problems in the analysis of control systems can be recast as positivity (or nonnegativity) of suitable one-parameter families of homogeneous forms. In particular, two robustness problems involving linear systems and the problem of computing all the equilibria of polynomial nonlinear systems are discussed in detail.

1

Introduction

Since long time it has been known that multivariate polynomials play a key role in several areas of systems theory [1]. Indeed, many fundamental problems in automatic control, multidimensional digital filtering, multivariable network realization, require the investigation of some properties of multivariate polynomials. Among many other properties, positivity and nonnegativity of multivariate polynomials have received a lot of attention (see, e.g., [2]). Unfortunately, the procedures proposed for checking these properties are in general quite computationally demanding, thus providing a satisfactory solution only for low dimensional polynomials. Homogeneous (polynomial) forms have been usually considered as the result of the process of homogenizing a multivariate polynomial, but they have been considered also on their own. One interesting case concerns the homogeneous polynomial Lyapunov functions, whose possibility to improve robust stability results provided by quadratic Lyapunov functions is well-known (one of the first work mentioning this possibility is [3]). On the other hand, homogeneous forms have been an important research topic in mathematics since long time [4,5]. In particular, a classical result states that positive (nonnegative) homogeneous forms cannot be in general expressed as sums of squares of homogeneous forms. It is also well known that they can instead be written as the ratio of sums of squares of homogeneous forms.

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 161--177, 2003 Springer-Verlag Berlin Heidelberg

162

G. Chesi et al.

This paper deals with the problem of checking positivity and nonnegativity of families of homogeneous forms. In particular, one-parameter families of homogeneous forms, where the coefficients depend on a scalar parameter which is allowed to vary over either a continuous or a discrete set, are considered. The first contribution of the paper is a sufficient condition for positivity (nonnegativity) of all the forms in the one-parameter family. Such a condition is based on the Complete Square Matricial Representation (CSMR) of homogeneous forms recently introduced in [6], and it requires the solution of a Linear Matrix Inequalities (LMI) optimization problem for each value of the scalar parameter. Exploiting results on representation of homogeneous forms as sums of squares of homogeneous forms [4,5], it is shown that the condition turns out to be necessary for some classes of one-parameter families of homogeneous forms. Computational issues are also addressed, by showing that each LMI optimization problem amounts to minimizing the maximum eigenvalue of the symmetric matrix related to the CSMR, and by investigating the dependence of the LMI size in terms of the dimension of the argument and the degree of the homogeneous forms. The second contribution of the paper is to show that several important problems in the analysis of control systems can be cast as either Positivity of One-parameter Families of Homogeneous forms (POFH) or NonNegativity of One-parameter Families of Homogeneous forms (NNOFH) problems. In particular, two robust stability problems of uncertain classes of linear systems and the problem of computing all the equilibria of polynomial nonlinear systems are discussed in detail. Some insights are also given for other problems which can be tackled in the same way. The results in this paper are related to the convexity property of the sum of squares of homogeneous forms, which has been recently exploited in different contexts. Specifically, global bounds for polynomial functions have been established in [7]. In [8–11] it has been shown how the solution of several classes of non-convex minimum distance problems can be approximated, or even computed exactly, via convex LMI optimizations. A convex optimization framework for semialgebraic problems has been formulated in [12]. Cones of polynomials representable as the sum of squared functions have been investigated in [13,14]. Other applications related to the above convexity property can be found in [15–18]. The paper is organized as follows. In Section 2 the POFH and NNOFH problems are formulated and some preliminary material is given. Conditions in terms of LMIs for the solution of the considered problems are provided in Section 3, together with a discussion of the related computational issues. In Section 4 analysis and control problems which can be cast as either POFH or NNOFH problems are presented. Some concluding remarks are given in Section 5. Notation. Rn : real n-space; n Rn 0 : R \ {0}; ∅: empty set; x = (x1 , . . . , xn )0 : vector of Rn ; Rn×n : real n×n space; A = [aij ]: real n×n matrix; A0 : transpose of A; Ker[A]: null space of A; In : n × n identity matrix;

Homogeneous forms in robustness analysis

163

A > 0 (A ≥ 0): positive definite (semidefinite) matrix; λm [A]: minimum real eigenvalue of A; kxk: Euclidean norm. Definition 1. A function fm (x) is a (real n-variate) homogeneous form of degree m in x ∈ Rn if X ci1 ,i2 ,...,in xi11 xi22 . . . xinn , fm (x) = i1 +i2 +...+in =m

where i1 , i2 , . . . , in are nonnegative integers, and ci1 ,i2 ,...,in ∈ R are weighting coefficients. Definition 2. The (real n-variate) homogeneous form fm (x) is said positive if 6 0, and nonnegative if fm (x) ≥ 0 ∀x. fm (x) > 0 ∀x =

2

Problem formulation and preliminaries

Let us consider the family of homogeneous forms of degree 2m in x ∈ Rn ( ) X i1 i2 in n ci1 ,i2 ,...,in x1 x2 . . . xn . F2m := f2m (x) = i1 +i2 +...+in =2m n is The basic problem addressed in this paper is to check if some subset of F2m positive (nonnegative) definite, i.e, if it contains only positive (nonnegative) homogeneous forms. In particular, we consider subsets described as

{f2m (x; γ), where f2m (x; γ) =

γ ∈ Γ}, X

(1)

ci1 ,i2 ,...,in (γ) xi11 xi22 . . . xinn ,

(2)

i1 +i2 +...+in =2m

γ is a scalar parameter, and Γ may be either a discrete set, i.e., Γ = {γ1 , . . . , γN }, or a compact continuous set (e.g., Γ = [γ, γ] ⊂ R). In the latter case, the weighting coefficient functions ci1 ,i2 ,...,in (·) : Γ −→ R are assumed to depend continuously on the scalar parameter γ. Note that (1)-(2) is a one-parameter family of homogeneous forms, and it contains either a finite number or a one-dimensional set of homogeneous forms. The motivation for considering families of homogeneous forms as in (1) is that several important problems can be shown to amount to positivity (or, alternatively, nonnegativity) of one-parameter families of homogeneous forms (see Section 4). Positivity of a One-parameter Family of Homogeneous forms (POFH) problem. Let the set Γ and the coefficients functions ci1 ,i2 ,...,in (·) : Γ −→ R be given. Verify if the one-parameter family of homogeneous forms (1)-(2) is positive definite, i.e., f2m (x; γ) is positive for all γ ∈ Γ .

164

G. Chesi et al.

Analogously, one can define the problem of testing NonNegativity of a One-parameter Family of Homogeneous forms (hereafter referred to as NNOFH problem). In the remaining part of the section, we introduce the key representation of homogeneous forms that will be exploited throughout the paper. Let Ei ∈ R(n+1−i)×n be the matrix composed by the rows i, i + 1, . . . n of In . Then, the base vector x{m} ∈ Rd , which is the vector containing all monomials of degree m, is defined as  x if m = 1,            x1 x{m−1}   {m−1}    (3) x{m} =  x2 (E2 x)    ..     otherwise.   .         xn−1 (En−1 x){m−1}    xm n It is easy to check that the dimension d of x{m} is given by d=

(n + m − 1)! . (n − 1)!m!

(4)

Consider the homogeneous form f2m (x; γ) for a given γ ∈ Γ . Then, the Square Matricial Representation (SMR) of f2m (x; γ) is defined as 0

f2m (x; γ)(x) = x{m} Cf (γ)x{m} ,

(5)

where Cf (γ) = Cf (γ)0 ∈ Rd×d is a coefficient matrix. An important property of the SMR is that matrix Cf (γ) is not unique. Indeed, all the matrices Cf (γ) satisfying (5) can be parameterized as Cf (γ) + L, with L ∈ L, where L is the set of matrices o n 0 (6) L = L = L0 ∈ Rd×d : x{m} Lx{m} = 0 ∀x ∈ Rn . In [9] it has been shown that L is a linear space whose dimension is given by dL =

(n + 2m − 1)! 1 d(d + 1) − . 2 (n − 1)!(2m)!

(7)

Then, the family of matrices Cf (γ) describing f2m (x; γ) can be parameterized as Cf (γ; α) = Cf (γ) + L(α), where α ∈ RdL is a vector of free parameters and L : RdL → L is a linear parameterization of L. Hence, the Complete Square Matricial Representation (CSMR) of f2m (x; γ) is defined as 0

f2m (x; γ) = x{m} Cf (γ; α)x{m} .

(8)

In the sequel, we will refer to Cf (γ) and Cf (γ; α) as SMR and CSMR matrix of f2m (x; γ), respectively. An algorithm for the construction of the CSMR of a homogeneous form can be found in [6]. Example 1. Consider the homogeneous form f4 (x; γ) = x41 − γx31 x2 +

3 1 x21 x22 − 2γx1 x32 + (1 + γ 4 )x42 , 1 + γ2 4

Homogeneous forms in robustness analysis

165

where γ is a scalar parameter. Since n = 2 and m = 2, according to (3) and (4), the base vector is given by x{2} = (x21 , x1 x2 , x22 )0 . It is straightforward to check that   0 1 − γ2 γ 3 −γ . Cf (γ) =  − 2 1+γ 2 1 0 − γ 4 (1 + γ 4 ) is a SMR matrix of f4 (x; γ). Hence, according to (6) and (7), the CMSR matrix of f4 (x; γ) has the form 

1 γ Cf (γ; α) =  − 2 −α

 −α + 2α −γ , 1 −γ (1 + γ4) 4

−

3 1+γ 2

γ 2

where α ∈ R.

3

LMI-based conditions for the solution of the POFH problem

In this section, conditions in terms of LMIs for the solution of the POFH problem are provided. The related computational issues are also discussed.

3.1

Sufficient condition

We first introduce a result relating positivity of a homogeneous form to positivity of its CSMR matrix. Lemma 1. Consider the CSMR matrix Cf (γ; α) of the homogeneous form f2m (x; γ), and define the quantity η(γ) = max λm [Cf (γ; α)] . α∈RdL

(9)

If η(γ) > 0 then f2m (x; γ) is positive. Proof. Let α∗ ∈ RdL denote any parameter vector at which the maximum in (9) is attained. Then, the SMR matrix Cf (γ; α∗ ) is positive definite and hence the proof easily follows from (8). Note that the computation of η(γ) requires the maximization of the minimum eigenvalue of a symmetric matrix parameterized affinely. This optimization problem is convex and can be reformulated as an LMI optimization as it will be shown in Section 3.3. Lemma 1 directly leads to the following LMI-based sufficient condition for the POFH problem.

166

G. Chesi et al.

Result 1. Let the set Γ and the coefficient functions ci1 ,i2 ,...,in (·) be given, and suppose that η ∗ = inf η(γ) > 0.

(10)

γ∈Γ

Then, the one-parameter family of homogeneous forms (1)-(2) is positive definite. Clearly, a result similar to Theorem 1 holds for the NNOFH problem, the sufficient condition being η ∗ ≥ 0. Example 2. Consider again the setting of Example 1. In order to check positive definiteness of the one-parameter family f4 (x; γ) one has to compute η ∗ in (10), with   −α 1 − γ2 γ 3 −γ . η(γ) = max λm  − 2 1+γ 2 + 2α α∈RdL 1 −α −γ (1 + γ4) 4 Let us consider two cases: (i) Γ = {0, 5, 10}, (ii) Γ = [0, 5]. In case (i), one can solve (9) for each value of γ, thus obtaining η(0) = 0.2496, η(5) = 0.1574, η(10) = 0.2272. Then, η ∗ = η(5) > 0 and the family is positive definite. Conversely, in case (ii) one has to solve (9) for an entire interval of values of γ. By searching over γ (e.g., by bisection), one easily finds that η(2.5) = −0.12; hence the result in Theorem 1 cannot be applied. Figure 1 shows the values of η(γ) for 0 ≤ γ ≤ 10.

Example 1: plot of η(γ)

0.4

η(γ)

0.2 0

−0.2 −0.4

0

2

Fig. 1. Example 1: values of η(γ).

4

γ

6

8

10

Homogeneous forms in robustness analysis

3.2

167

Necessary and sufficient condition

Lemma 1 is based on the fact that if the quantity η(γ) is positive, then there exists at least one SMR matrix which is positive definite. Clearly, positivity of such a matrix is a stronger requirement than positivity of the form f2m (x; γ). However, if a homogeneous form possesses a special structure then positivity (nonnegativity) of the form is equivalent to positivity (nonnegativity) of the SMR matrix. Let us first consider the next result. Lemma 2. Suppose that f2m (x; γ) can be written as f2m (x; γ) =

h X

2 fm,i (x; γ),

(11)

i=1

where fm,i (x; γ) are suitable homogeneous forms of degree m. Then, we have η(γ) ≥ 0. Proof. If (11) holds, there exist h vectors ti ∈ Rd such that fm,i (x; γ) = t0i x{m} and f2m (x; γ) =

h ³ X i=1

t0i x{m}

´2

0

= kT x{m} k2 0

= x{m} T 0 T x{m} = x{m} C x{m} where T = [t1 . . . th ]0 ∈ Rh×d and C ∈ Rd×d is positive semidefinite. Hence, from the CSMR of f2m (x; γ) there exists α such that Cf (γ; α) = C, which in turn implies η(γ) ≥ 0. Lemma 2 implies that, when nonnegativity of the forms is addressed, the necessity of the condition η ∗ ≥ 0 is related to the property of nonnegative homogeneous forms to be represented as the sum of squares of homogeneous forms. It is known since a long time that there exist forms for which this property is not satisfied (see [4]; examples can be found in [5] and [12]). However, there exist families of homogeneous forms for which this property holds. The following result is a consequence of this fact. Lemma 3. Consider the set E = {(n, 2), n ∈ N} ∪ {(2, 2m), m ∈ N} ∪ {(3, 4)} .

(12)

If (n, 2m) ∈ E, then f2m (x; γ) is nonnegative for all γ ∈ Γ if and only if η ∗ ≥ 0. Proof. It is well known that all nonnegative homogeneous forms satisfying (n, 2m) ∈ E can be written as sums of squares, as in (11) (see e.g. [4]). Hence, the result follows from Lemma 2. If we turn our attention to positivity of the forms, it is easy to see that Lemma 2 does not hold, unless one makes the further assumption that the homogeneous forms fm,i (x; γ) allow one to construct a full-rank matrix T in the proof of the lemma. Nevertheless, this assumption turns out to be unnecessary in the cases considered by Lemma 3, as it is shown by the next result.

168

G. Chesi et al.

Result 2. Consider the set E in (12). If (n, 2m) ∈ E, then f2m (x; γ) is positive for all γ ∈ Γ if and only if η ∗ > 0. Proof. Sufficiency is guaranteed by Theorem 1. In order to prove necessity, let us assume that f2m (x; γ) is positive definite. For fixed γ ∈ Γ , let us define the normalized minimum εf (γ) = minn f2m (x; γ) x∈R

subject to

kxk = 1.

Obviously, εf (γ) > 0 since f2m (x; γ) is positive definite. Let us introduce the homogeneous form g2m (x; γ) = f2m (x; γ) − εf (γ)kxk2m . It turns out that g2m (x; γ) is nonnegative. Indeed, its normalized minimum is equal to εh (γ) = εf (γ)−εf (γ) = 0. Hence, Lemma 3 guarantees that there exists a positive semidefinite SMR matrix ¯g (γ) for g2m (x; γ). Now, let N be a diagonal SMR matrix of the form kxk2m . NoC tice that such N exists since the homogeneous form kxk2m contains only monomials with even powers of the variables xi , and moreover N ≥ Id since the coefficients of the monomials in kxk2m are greater or equal to 1. Then, from the definition of g2m , ¯f (γ) = C ¯g (γ) + εf (γ)N turns out to be a SMR matrix of g2m (x; γ). One clearly C ¯f (γ) > 0 and the result is proved by applying the above reasoning for all has that C γ ∈ Γ. Example 3. Let us consider the problem in Example 1 once again. Since n = 2 and m = 2, one has that (n, 2m) ∈ E and Theorem 2 can be applied. In particular, exploiting the results in Example 2 one can conclude that the family f2m (x; γ) with γ ∈ [0, 5] is not positive definite, due to the fact that η(2.5) = −0.12.

3.3

Computational issues

The sufficient condition provided by Theorem 1 amounts to solving a one-parameter family of optimization problems such as (9) (maximization of the minimum eigenvalue of a symmetric matrix parameterized affinely). These optimization problems are convex and can be reformulated as LMI optimizations, for which powerful solution methods have been developed. In fact, let us observe that (9) can be rewritten as the maximization of a linear function subject to a single LMI [19] η(γ) = s.t.

max

t∈R, α∈RdL

t (13)

Cf (γ; α) − tId ≥ 0.

The size of the LMI (13) is usually defined by the dimension d of the symmetric matrix Cf (γ; α) and the dL + 1 free parameters (t, α). The LMI size for different values of n and m can be derived from Table 1. From (4) and (7) it is straightforward to obtain the asymptotic expressions of d and dL when either n or m are fixed. Indeed, for fixed n we have d ≈ O(mn−1 ), dL ≈ O(m2(n−1) ),

Homogeneous forms in robustness analysis

169

while, for any fixed m, we have d ≈ O(nm ), dL ≈ O(n2m ). We observe that the numbers d and dL play a different role with respect to the required computational burden: the dimension of the matrices is fixed and cannot be changed, while the number of free parameters over which the LMI optimization is performed can be reduced in order to limit the required computational power (clearly, at the price of a more conservative sufficient condition in Theorem 1). A clever way to reduce the number dL in problems with high values of n and/or m is still an open issue. Table 1. Values of d (a) and dL (b), for some n and m. d n=2 3 4 m=1 2 3 4 2 3 6 10 3 4 10 20 4 5 15 35 5 6 21 56

5 5 15 35 70 126

dL n=2 3 4 m=1 0 0 0 2 1 6 20 3 3 27 126 4 6 75 465 5 10 165 1310

(a)

4

5 0 50 420 1990 7000

(b)

POFH problems in control system analysis

The purpose of this section is to show that several important problems in the analysis of control systems amount to solve POFH or NNOFH problems. Three specific problems are considered in detail, while insights to related problems are provided. In particular, in Section 4.1 (resp., Section 4.2) robust stability of an uncertain class of linear systems is recast as a POFH problem where the set Γ in (1)-(2) is a continuous (resp., discrete) set, while Section 4.3 is devoted to show how the results of Section 3 on NNOFH problems can be exploited to compute all the equilibria of polynomial nonlinear systems.

4.1

Robust stability of linear time-invariant systems subject to l2 parametric uncertainty

Consider the state-space uncertain control system z˙ = A(x)z

(14)

where z ∈ RM is the state vector and x = (x1 , . . . , xn )0 ∈ Rn is the vector of uncertain parameters.

170

G. Chesi et al.

Such a system is said to be robustly stable if the matrix A(x) is Hurwitz for each value of the parameter vector in a prescribed uncertainty set [20]. In particular, we focus on the case of l2 uncertainty set and, for easiness of exposition, we also assume that A(x) depends affinely on x. Therefore, consider the matrix family ( A(x) = A0 +

n X

) xi Ai , kxk ≤ ρ ,

(15)

i=1

where ρ is a given positive scalar, A0 ∈ RM ×M represents the nominal control system and is assumed to be Hurwitz, while matrices Ai ∈ RM ×M , i = 1, . . . , n, model the uncertainty structure. Our aim is to show that the robust stability problem of checking if all the matrices of the family (15) are Hurwitz, can be recast as a suitable POFH problem. To proceed, we first recall the well-known fact that the family (15) contains only Hurwitz matrices if and only if the following conditions hold (see, e.g., [21]) I)

det [A0 ] det [A(x)] > 0

II) HM −1 [A0 ] HM −1 [A(x)] > 0

∀ x : kxk ≤ ρ, ∀ x : kxk ≤ ρ,

(16)

where HM −1 [A(x)] denote the (M −1)-th Hurwitz determinant of matrix A(x). Consider the functions Fd (·) : Rn → R and Fh (·) : Rn → R defined as follows Fd (x) = det [A(x)] det [A(−x)] , Fh (x) = HM −1 [A(x)] HM −1 [A(−x)] . It is not difficult to verify that conditions I) and II) in (16) are equivalent to the following ones Fd (x) > 0

∀ x : kxk ≤ ρ,

II) Fh (x) > 0

∀ x : kxk ≤ ρ.

I)

Moreover, note that Fd (x) and Fh (x) are even polynomials, i.e., Fd (x) =

md X

fd,2i (x),

i=0 mh

Fh (x) =

X

(17) fh,2i (x),

i=0

where fd,2i (x), fh,2i (x), are homogeneous forms of degree 2i, and md , mh are such that md ≤ M, M (M − 1) mh ≤ . 2

Homogeneous forms in robustness analysis

171

Let us introduce the functions f2md (·; ·) : Rn × R → R and f2mh (·; ·) : Rn × R → R such that f2md (x; γ) =

md X kxk2(md −i) fd,2i (x) γ 2(md −i) i=0

f2mh (x; γ) =

mh X kxk2(mh −i) fh,2i (x) , γ 2(mh −i) i=0

where γ is a positive scalar, and fd,2i (x) and fh,2i (x) are the homogeneous forms in (17). It turns out that f2md (x; γ) and f2mh (x; γ) are homogeneous form in x of degree 2md and 2mh , respectively. Moreover, it is not difficult to verify that, for any γ > 0, the following equalities hold f2md (x; γ) = Fd (x)

∀ x : kxk = γ

f2mh (x; γ) = Fd (x)

∀ x : kxk = γ.

Therefore, one has that all the matrices of the family (15) are Hurwitz if and only if the one-parameter families of homogeneous forms {f2md (x; γ), γ ∈ (0, ρ]}

(18)

{f2mh (x; γ), γ ∈ (0, ρ]}

(19)

and

are positive definite. Remark. The set Γ in the families (18)-(19) are not compact, as required by the theory developed in Sections 2-3. However, one can always consider the above families with γ ∈ [ε, ρ] and study their positivity for arbitrarily small ε > 0. Nevertheless, notice that this is not a problem in practice, as the condition η(γ) > 0 is usually checked by solving (13) on a finite set of γ values, obtained by gridding the interval Γ within the desired precision. We conclude this section by recalling that the l2 parametric stability margin ρ2 of system (14), which is a popular robustness measure [20], is defined as follows ρ2 = sup {ρ ∈ R : A(x) is Hurwitz, ∀x : kxk ≤ ρ} . Hence, ρ2 can be computed by looking for the largest ρ such that the one-parameter families of homogeneous forms (18) and (19) are positive definite. On the other hand it is not difficult to verify that ρ2 can also be computed as follows √ √ ρ2 = min{ ρ2,I , ρ2,II } where ρ2,I = infn kxk2 s.t.

x∈R

Fd (x) = 0,

(20)

172

G. Chesi et al.

and ρ2,II = inf kxk2 s.t.

x∈RM

(21)

Fh (x) = 0,

Fd (x) and Fh (x) being as in (17). Note that the optimization problems (20) and (21) amount to computing the minimum distance of the origin from the polynomial constraints Fd (x) = 0 and Fh (x) = 0, respectively. Hence, they are two specific examples of the recently introduced Canonical Quadratic Distance Problem (CQDP), where the minimum Euclidean distance from a surface described by an even polynomial has to be computed. Details on CQDPs solution can be found in [6], where it is also shown that several problems arising in different fields can be formulated as CQDPs.

4.2

Robust stability of linear systems with time-varying structured uncertainties

Consider the state-space uncertain control system x(t) ˙ = A(t)x(t)

(22)

n

where x ∈ R is the state vector, ! Ã s X wi (t)Ai , A(t) = A0 +

(23)

i=1

and A0 , . . . , As ∈ Rn×n are given matrices. The uncertain parameter vector w(t) = (w1 (t), . . . , ws (t))0 ∈ Rs is assumed to satisfy for all t ≥ 0 the constraint . (24) w(t) ∈ W = co{w1 , . . . , w r }, where wk = (w1k , . . . , wsk )0 ∈ Rs , k = 1, . . . , r, are given vectors and co(·) denotes the convex hull. Lyapunov functions are a standard tool for investigating robust stability of system (22)-(24). Different classes of Lyapunov functions have been used, including quadratic, piecewise quadratic and polyhedral Lyapunov functions [22–26]. More recently, it has been shown that homogeneous polynomial Lyapunov functions lead to significant improvements of stability results [27,18]. Let X ci1 ,i2 ,...,in xi11 xi22 . . . xinn , v2m (x) = i1 +i2 +...+in =2m

be a positive definite homogeneous form of degree 2m, i.e. v2m (x) > 0, ∀x = 6 0. Our aim is to show that the problem of checking if v2m (x) is a homogeneous polynomial Lyapunov function for system (22)-(24), can be recast as a suitable POFH problem. Clearly, v2m (x) is a homogeneous polynomial Lyapunov function for system (22)(24) if its time derivative à ! !à s X X ∂ i1 i2 in wi (t)Ai x, ci ,...,in x1 x2 . . . xn A0 + v˙ 2m (x) = ∂x i +...+i =2m 1 i=1 1

n

Homogeneous forms in robustness analysis

173

is negative for all x 6= 0 and for all w(t) ∈ W. Note that v˙ 2m (x) is a homogeneous form of degree 2m whose coefficients depend affinely on the uncertain parameter vector w(t). Exploiting a well-known convexity result (see e.g. [28]), it is straightforward to verify that v2m (x) is a homogeneous polynomial Lyapunov function if and only if its time derivative is negative definite for w(t) ∈ {w1 , . . . , w r }, i.e. Ã

∂ ∂x

!Ã

X

ci1 ,...,in xi11 xi22

. . . xinn

A0 +

i1 +...+in =2m

s X

! wik Ai

x < 0,

i=1

for k = 1, . . . , r. Hence, v2m (x) is a homogeneous polynomial Lyapunov function for system (22)-(24) if and only if the following one-parameter family of homogeneous forms of degree 2m (

. f2m (x; γ) = −

Ã

∂ ∂x

γ ∈ Γ = {1, . . . , r}

X i1 +...+in =2m

!Ã ci1 ,...,in xi11 xi22

. . . xinn

A0 +

s X

! wiγ Ai

x,

i=1

}

is positive definite. We finally note that, exploiting the fact that the CSMR related to the above f2m (x; γ) is jointly affine in the weighting coefficients ci1 ,i2 ,...,in of v2m (x) and the free parameter vector α, LMI conditions for the existence of a homogeneous polynomial Lyapunov function can also be derived. Details on the derivation as well as a comparison with stability results obtained with other classes of Lyapunov functions are reported in [18,29].

4.3

Computation of equilibrium points of nonlinear polynomial systems

The determination of equilibria is a key step in the analysis of nonlinear control systems. It is well known that, while Euler-Newton iterative methods work quite satisfactorily when a local equilibrium point is of interest, no efficient method is available for computing all the equilibria. In the case of polynomial systems one can employ polynomial resultants and other algebraic geometry methods as well as homotopy methods. Algebraic geometry methods, which are based on elimination theory and Tarksi’s decision theorem [30–32], suffer the problem of spurious equilibria, thus making them effective for low dimensional systems only. On the other hand, homotopy methods are based on continuation techniques and are able to provide with probability one all the equilibria of small systems as fixed points of suitable nonlinear maps [33]. Our aim is to show that the set of equilibria of nonlinear polynomial systems can be computed by suitably exploiting the results on the NNOFH problem given in Section 3. Consider for example the nonlinear system z˙ = Az + F2 (z)

(25)

174

G. Chesi et al.

where z ∈ Rn−1 is the state vector, A ∈ Rn−1×n−1 is a given matrix, and F2 (·) : Rn−1 → Rn−1 is a vector of quadratic forms, i.e. F2 (z) = (z 0 P1 z, . . . , z 0 Pn−1 z)0 ,

(26)

n−1×n−1

, i = 1, . . . , n − 1, being given symmetric matrices. Pi ∈ R Systems (25)-(26) have received a lot of attention since long time. Indeed, they can be used for approximating general polynomial systems [34] and for modelling the closed loop behavior of bilinear systems under linear state feedback control [35,36]. The equilibria set Z of system (25)-(26) is defined as ª © Z = z ∈ Rn−1 : e0i Az + z 0 Pi z = 0, 1 ≤ i ≤ n − 1 . where ei is the i-th column of the identity matrix. Introduce an auxiliary variable zn and set µ ¶ z x= ∈ Rn . zn Consider the quadratic forms ¸ · Pi 0 qi (x) = x0 0 x, i = 1, · · · , n − 1, ei A 0

(27)

and the related set X = {x ∈ Rn , x 6= 0 : qi (x) = 0, 1 ≤ i ≤ n − 1} . It is straightforward to verify that Z can be rewritten as ª © Z = z = (x1 , . . . , xn−1 )0 ∈ Rn−1 : x ∈ X and zn = 1 . Hence, it is worth turning the attention to the characterization of X . Let us define the homogeneous form of degree 4 n−1 . X 2 qi (x), f4 (x) =

(28)

i=1

where qi (x) are as in (27). It is obvious that f4 (x) = 0 , x 6= 0 ⇐⇒ x ∈ X . Now, note that by construction f4 (x) turns out to be a sum of squares of n − 1 homogeneous forms in x of degree 2. Hence, according to Lemma 2, it follows that η = max λm [Cf (α)] ≥ 0, α∈RdL

(29)

where Cf (α) is the CSMR of the homogeneous form (28), i.e. f4 (x) = x{2} Cf (α)x{2} .

(30)

If η > 0, it turns out that f4 (x) is positive and hence X = ∅. Consider therefore the case η = 0 and let α ˆ be any minimizer in (29). From (30) we have n o X = x ∈ Rn , x 6= 0 : x{2} ∈ Ker [Cf (α)] ˆ , which in turn characterizes the set Z of equilibrium points. Details on the procedure for the actual computation of the set X can be found in [17,37], where the more general problem of determining the solution set of polynomial systems is investigated.

Homogeneous forms in robustness analysis

5

175

Conclusions

Although the role of homogeneous forms in control system analysis has been recognized since long time, only recently their great potentialities in practical problems have emerged, thanks to the possibility of exploiting efficient tools for their actual computation. In this paper it has been shown how several robustness problems can be cast as positivity tests of one-parameter families of homogeneous forms, and tackled by means of powerful optimization techniques involving LMIs. Research along these lines seems to be very promising. A key issue is how to exploit the structure of the specific problem at hand, in order to reduce the number of free variables in the involved optimization problems. It is expected that, at least in some cases, clever pruning techniques can be devised, allowing one to trade off computational complexity of the optimization procedure and suboptimality of the solutions. Another line of research concerns the development of more efficient tools for control system analysis. For example, one may enrich the class of parameter-dependent Lyapunov functions by allowing polynomial variations of the parameters, in order to obtain improved robust stability results for systems with structured uncertainties.

References 1. N. K. Bose (Ed.). Special issue on multidimensional systems. Proc. IEEE, 65(6), 1977. 2. N. K. Bose. Applied Multidimensional Systems Theory. Van Rostrand Reinhold, New York, 1982. 3. R. W. Brockett. Lie algebra and Lie groups in control theory. In D.Q. Mayne and R.W. Brockett, editors, Geometric Methods in Systems Theory, pages 43– 82. Dordrecht, Reidel, 1973. 4. G. Hardy, J.E. Littlewood, and G. P´olya. Inequalities: Second edition. Cambridge University Press, Cambridge, 1988. 5. B. Reznik. Some concrete aspects of hilbert’s 17th problem. In C. N. Delzell and J.J. Madden, editors, Real Algebraic Geometry and Ordered Structures, Cont. Math., pages 251–272, 2000. 6. G. Chesi, A. Garulli, A. Tesi, and A. Vicino. LMI-based techniques for solving quadratic distance problems. In Proc. of 40-th IEEE CDC, 2001. To appear in IEEE Trans. Aut. Control, 2003. 7. N. Z. Shor. Class of global minimum bounds of polynomial functions. Cybernetics, 23:731–734, 1987. (Russian orig. : Kibernetika, no. 6, pp. 9-11, 1987). 8. G. Chesi, R. Genesio, and A. Tesi. Optimal ellipsoidal stability domain estimates for odd polynomial systems. In Proc. of 36-th IEEE CDC, pages 3528– 3529, San Diego, December 1997. 9. G. Chesi, A. Tesi, A. Vicino, and R. Genesio. A convex approach to a class of minimum norm problems. In A. Garulli, A. Tesi, and A. Vicino, editors, Robustness in Identification and Control, pages 359–372. Springer, 1999. 10. G. Chesi, A. Garulli, A. Vicino, and R. Cipolla. Estimating the fundamental matrix via constrained least squares: a convex approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3):397–401, 2002. 11. G. Chesi, A. Tesi, A. Vicino, and R. Genesio. An LMI approach to constrained optimization with homogeneous forms. Systems and Control Letters, 42:11–19, 2001.

176

G. Chesi et al.

12. P. Parrilo. Structured Semidefinite Programs and Semialgebraic Geometry Methods in Robustness and Optimization. Ph.D. Thesis, California Institute of Technology, 2000. 13. Y. Nesterov. Squared functional systems and optimization problems. In H. Frenk, K. Roos, T. Terlaky, and S. Shang, editors, High Performance Optimization. Kluwer, 2000. 14. Y. Genin, Y. Hachez, Y. Nesterov, and P. Van Dooren. Optimization problems over positive pseudo-polynomial matrices. Technical report, http://www.auto.ucl.ac.be/∼vdooren/download/Welcome.html, 2001. 15. A. Rantzer and P. A. Parrilo. On convexity in stabilization of nonlinear systems. In Proc. of 39th IEEE Conference on Decision and Control, Sydney, December 2000. 16. B. Tibken. Estimation of the domain of attraction for polynomial systems via LMI’s. In Proc. of 39th IEEE Conference on Decision and Control, Sydney, December 2000. 17. G. Chesi, A. Garulli, A. Tesi, and A. Vicino. An LMI-based approach for characterizing the solution set of polynomial systems. In Proc. of 39-th IEEE Conf. on Decision and Control, pages 1501–1506, Sydney, December 2000. 18. G. Chesi, A. Garulli, A. Tesi, and A. Vicino. LMI-based construction of homogeneous Lyapunov functions for systems with structured uncertainties. In Proc. of 41-th IEEE CDC, Las Vegas, December 2002. 19. Y. Nesterov and A. Nemirovsky. Interior Point Polynomial Methods in Convex Programming: Theory and Applications. SIAM, Philadelphia, 1993. 20. S. P. Bhattacharyya, H. Chapellat, and L. H. Keel. Robust Control: The Parametric Approach. Prentice Hall, NJ, 1995. 21. A. Tesi and A. Vicino. Robust stability of state space models with structured uncertainties. IEEE Transactions on Automatic Control, 35:191–195, 1990. ˘ 22. D. D. Siljak. Nonlinear Systems: Parametric Analysis and Design. John Wiley & Sons, New York, 1969. 23. R. K. Brayton and C. H. Tong. Stability of dynamical systems: a constructive approach. IEEE Transactions on Circuits and Systems, 26:224–234, 1979. 24. K. Zhou and P. P. Khargonekar. Stability robustness bounds for linear statespace models with structured uncertainties. IEEE Transactions on Automatic Control, 32:621–623, 1987. 25. F. Blanchini. Nonquadratic Lyapunov functions for robust control. Automatica, 31:451–461, 1995. 26. L. Xie, S. Shishkin, and M. Fu. Piecewise Lyapunov functions for robust stability of linear time-varying systems. Systems and Control Letters, 31:165–171, 1997. 27. A. L. Zelentsovsky. Nonquadratic Lyapunov functions for robust stability analysis of linear uncertain systems. IEEE Transactions on Automatic Control, 39:135–138, 1994. 28. H. P. Horisberger and P. R. Belanger. Regulators for linear time invariant plants with uncertian parameters. IEEE Transactions on Automatic Control, 21:705–708, 1976. 29. G. Chesi, A. Garulli, A. Tesi, and A. Vicino. Homogeneous Lyapunov functions for systems with structured uncertainties. Technical report, Dipartimento di Ingegneria dell’Informazione, Universit`a di Siena, 2002. ftp://ftp.ing.unisi.it/pub/users/garulli/hplf.pdf.

Homogeneous forms in robustness analysis

177

30. A. Tarski. A Decision Method for Elementary Algebra and Geometry. Univ. California Press, Berkeley, 1951. 31. N. Jacobson. Lectures in Abstract Algebra, Vol. III. Van Nostrand, Princeton, 1964. 32. D. Cox, J. Little, and D. O’Shea. Using Algebraic Geometry. Springer, 1998. 33. A. Morgan. Solving polynomial systems using continuation for engineering and scientific problems. Prentice-Hall, New Jersey, 1987. 34. R. W. Newcomb. Nonlinear differential systems: a canonic, multivariable theory. Proc. IEEE, 65(6):930–935, 1977. 35. R. R. Mohler. Bilinear Control Processes. Academic Press, New York, 1973. 36. A. Benallou, D. A. Mellichamp, and D. E. Seborg. Characterization of equilibrium sets for bilinear systems with feedback control. Automatica, 19:183–189, 1983. 37. G. Chesi, A. Garulli, A. Tesi, and A. Vicino. Characterizing the solution set of polynomial systems in terms of homogeneous forms: an LMI approach. Technical report, Dipartimento di Ingegneria dell’Informazione, Universit`a di Siena, 2002. ftp://ftp.ing.unisi.it/pub/users/garulli/poly.pdf.

Control of Electromechanical Actuators: Valves Tapping in Rhythm Katherine Peterson1 , Anna Stefanopoulou1 , and Yan Wang1 University of Michigan, Ann Arbor MI

Abstract. Electromechanical valve (EMV) actuators can replace the camshaft allowing for electronically controlled variable valve timing (VVT) on a new generation of engines. Through the use of VVT, engine operation can be optimized to allow for improvements in fuel economy, performance, and emissions. Before EMV actuators can be used in production vehicles two critical problems need to be resolved. First, impact velocities between the valve, valve seat, and the actuator itself need to be small to avoid excessive wear on the system and ensure acceptable levels of noise. Second, the opening and closing of the valve needs to be both fast and consistent to avoid collision with the piston and to reduce variability in trapped mass. An extensive control analysis of the EMV actuator and the control difficulties are presented. Finally, a linear, a nonlinear, and a cycle-to-cycle self-tuning controllers are designed and demonstrated on a benchtop experiment.

1

Introduction

The automobiles of the 90’s are 10 times cleaner and twice as efficient as the vehicles of 1970’s. Despite these improvements the transportation sector is still responsible for a large percentage of the CO2 and other harmful emission (HC, NOx , smoke, etc) generated throughout the world. Indeed, the increasing population combined with the strong desire for personal mobility will result in 800 millions registered vehicles worldwide by the year 2020 [4]. More than 18% of these cars will be concentrated in less than 30 cities (megacities) around the world deteriorating urban air quality. The vast majority of these cars will be using Internal Combustion (IC) engines. While potential replacements for the IC engine exist such as electro-chemical propulsion based on fuel cells or batteries, the infrastructure requirements are difficult to achieve. With these projections in mind one soon realizes the pressing need for clean and efficient internal combustion engines. Indeed, engineers throughout the world are racing ahead in their efforts to improve the internal combustion engine. In this race the well-tuned mechanically connected parts in the IC engine of the past are transformed to electronically controlled mechanisms that provide many degrees of freedom for performance optimization. Electronically controlled variable valve timing (VVT) is the last frontier in IC engine automation. It allows control of the valve motion independently of the piston motion and thus together with electronic spark timing and injection can optimize the three fundamental combustion variables, namely, ignition timing, fuel and air.

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 179--193, 2003 Springer-Verlag Berlin Heidelberg

180

2

KatherineÿPeterson et al.

Variable Valve Timing Technology

Extensive effort has been and is being made in the design of variable valve control mechanisms at research and development laboratories throughout the world. Several VVT mechanisms are introduced in this section. For an overview of more systems the reader is referred to [1]. In Cam Phasing [9] a mechanism is used to adjust the phase of the camshaft to the crankshaft rotation, and thus, can shift the phase of the valve timing. In Cam Profile Switching [3] and Multi-dimensional Cams [14] two or more camshafts are used to optimize the engine performance. Depending on operating conditions a mechanism switches to the desired cam profile to achieve the best performance. While such VVT schemes can be found in the market today, the VVT variability that they achieve is small because the mechanisms they employ are extensions of the conventional cam-crank shaft designs. Alternatively electromechanical and electrohydraulic valve actuators can completely eliminate the cam-crank shaft mechanical linkage and allow a wide continuously variable valve timing. The basic working premise of the electrohydraulic valve actuator is the use of compressed or high pressure fluids to control the valve motion. By governing the fluid flow throughout the actuator, the valve timing and lift can be varied with a high degree of flexibility. In [12] an electrohydraulic actuator is presented that exploits the elastic properties of the compressed hydraulic fluid to provide continuously variable control of engine valve timing, lift, and velocity. Electromechanical valve actuators, which are the focus of this paper, use magnets as a means to govern the valve motion. The electromechanical valve (EMV) actuator studied in this paper is shown in Fig. 1 and its functionalities are discussed in [6]. The actuator governs the opening/closing of the valve through the forcing of a set of springs and electromagnets. A typical opening/closing cycle is shown in Fig. 1. Initially the armature is held in the closed position by the upper magnetic coil, causing the spring on that side to be more compressed than the opposing spring. At time trc the release command is given and the voltage across the upper magnetic coil is reduced to zero. The difference in the spring force drives the armature across the 8 mm gap, thereby causing the valve to open. A catching voltage is applied to the lower magnetic coil to ensure the armature is caught. Once the armature has been captured, a holding voltage is applied to hold the valve open. At time tro the process is reversed in order to close the valve. The experimental setup consists of the following components; Electromechanical Valve Actuator, Eddy Current Sensor, 2 PWM Drivers, 200 Volt Power Supply, and a Dspace 1103 processing board. The eddy current sensor mounted on the rear of the actuator measures the armature displacement, which is sampled by the Dspace processor at 20 kHz sampling frequency. Based on the displacement and the control algorithm, the Dspace processing board regulates the PWM frequency to each of the PWM drivers to achieve the desired performance. While effective in ensuring opening/closing of the valve, the EMV actuator suffers from large impact velocities between the armature, valve, and valve seat leading to excessive noise and wear on the system. The open loop control scheme shown in Fig. 1 results in impact velocities of approximately 1 m/s. In addition, transition times need to be both quick and consistent to avoid collision with the piston and variability in trapped mass. Before the actuator can

Control of Electromechanical Actuators: Valves Tapping in Rhythm Lower Coil Voltage Upper Coil Voltage Valve Lift

Valve starts moving

Upper Coil

Power Supply

Armature

PWM Driver

Lower Coil

Valve closed

Valve closed

tt

Voltage command

Spring PWM Driver

Catching voltage

-2

Valve open

dSPACE Controller Valve

rc

Time

2 0

Holding voltage

Spring

4 Valve Lift [mm]

Eddy current Sensor Sensor Target

181

Zero voltage -4 ro

Computer

Fig. 1. Electromechanical valve actuator (left) and typical opening/closing cycle (right). be implemented in production vehicles, impact velocities less than 0.1 m/s and transition times of less than 4 ms need to be achieved. The impact between the armature and catching coil is not a completely elastic collision, and as such the armature will bounce before finally coming to rest against the catching coil. Here, we define the impact velocity to be the largest velocity of the armature when it is in contact with the catching coil. Typically this corresponds to the first collision. The transition time is defined as the time from release to when the armature is 98% closed/open. After it has completed 98% of the travel the additional air let in or out has negligible effects on the engine performance.

3

Soft Landing Methodologies

Various methodologies have been applied in an attempt to achieve soft landing of the EMV actuator. The problem has received considerable attention with the introduction of solenoid-driven actuators [8] and proposed solutions keep on emerging as patents. For example, a search performed in Feb 2002 with keywords camless, engine, valve, control, solenoid for patents filled in Europe, United States and Japan resulted 303 relevant patents; 37 of which were filled after Sep 2001. One basic idea is to apply either a pneumatic or hydraulic damper to the system to oppose the motion of the valve. The drawback of such a system is that the extra damping will add to the transition time and power consumption. Another concept is the use of variable rate springs. The spring force would increase as the armature moved near the catching coil, helping to decelerate the armature. A simpler concept would be to use a two-stage spring system, where the second set only effects the last 1-2 mm of travel. Similarly with adding a damper, the modified springs would increase the transition time and power consumption. Alternatively, feedback could be used to regulate the voltage across the magnetic coils in order to achieve the desired performance. Feedback has the advantage that it would not increase the transition time and the power consumption should be minimized when soft landing is achieved. In [5] the authors use in iterative learning

182

KatherineÿPeterson et al.

controller (ILC) to modify the feedback from cycle to cycle to decrease the impact velocity. The feedback they start with is not well tuned, resulting in poor performance in the first few cycle before the ILC has had time to adapt. We demonstrate here that it is possible to use a well designed feedback to improve the performance within a single cycle. Using repetitive control, the authors of [13] are able to apply a repetitive learning algorithm to achieve impact velocities with a mean of 0.06 m/s. Unfortunately, due to the use of softer springs, their transition times are quite long (roughly 8 m/s). In [2] the desired impact velocity and transition time are achieved by holding the ratio of the rate of change of current to current at a constant value. Based on the impact velocity of the previous transition the value is modified to improve the response. However, the feedback is based solely on a single point measurement of current and the rate of change of current and as such may not be robust against disturbances. This paper presents both a linear and nonlinear observer based output feedback controller to achieve the desired performance. In addition, the nonlinear feedback is augmented with a self-tuning algorithm to further improve performance from cycle-to-cycle.

4

Nonlinear Model

The system consists of four states, namely, armature position (z in mm), armature velocity (v in m/s), upper coil current (iu , in A) and lower coil current (il in A). To achieve fast release a reverse polarity voltage technique, described in [15], is used to quickly drive the holding current to zero. Therefore the current in the releasing coil has little influence over the motion of the armature, and as such we can reduce the system from four states to three. Let us define the three state model as: catching coil current (i, in A), distance from the catching coil (z, in mm), and armature velocity (v in m/s). The resulting equations of motion are: Vc − ri + χ1 (i, z) v di = dt χ2 (z) dz = 1000v dt dv 1 = (−Fmag (i, z) + ks (4 − z) − bv) dt m

(1) (2) (3)

written compactly as dx = f (x, Vc ) , dt

£ ¤T x= iz v

(4)

where the catching coil voltage is denoted by Vc in V, the resistance r in Ω, the system mass m in kg, the spring stiffness ks in N/mm, the damping coefficient b in Ns/m, the magnetic force due to the catching coil Fmag in N, the back-emf χ1 v in V, and the inductance χ2 in H. The magnetic subsystem is characterized by two distinct sets of equations. The boundary between them is defined by the saturation current which is a function of the air-gap distance, ix = kc +kd z. When the coil current is less than the saturation

Control of Electromechanical Actuators: Valves Tapping in Rhythm

183

current (i < ix ) the magnetic force is a quadratic function of the current. The functions χ1 , χ2 , and Fmag are given by χ1 (i, z) =

ka i2 2ka 2ka i . , Fmag (i, z) = 2 , χ2 (z) = 1000 (k + z) b (kb + z) (kb + z)2

(5)

If the coil current is greater than the saturation current (i > ix ) then the magnetic force no longer increases quadratically with current. Electromagnetic literature refers to this region of operation as the “saturation region”. The functions χ1 , χ2 , and Fmag are given by

χsat 1 (i, z) = χ1 (i, z) exp (−ki (i − ix ))

(6)

χsat 2 (i, z) = χ2 (z) exp (−i + ix + imin )

(7)

sat Fmag

(i, z) = (Fmag (ix , z) − fmax ) exp (−ki (i − ix )) + fmax

(8)

1 (ix ,z) , and imin , ka , kb , kc , kd , ke , and kf where fmax = ke z + kf , ki = fmaxχ−F mag (ix ,z) are all constants. The dynamic behavior due to the impact (bouncing) is included in the model, by extending (3):

dv 1 = (−Fmag (i, z) + ks (4 − z) − bv + N ) dt m

(9)

where N is the normal force acting between the armature and catching coil. The force N is given by  0   if z 6= 0,   (i, 0) − 4ks  if v = 0, Fmag ½ N= if v < 0, δ if z = 0,   6 0,   if v = if v > 0 0

(10)

where δ is an impulse function calibrated to give v + = ev − , where v + is the velocity just after impact and v − is the velocity just before impact. The parameter e < 1 is chosen based on experimental data and represents the loss of kinetic energy due to the plastic collision. Comparison of the model response and experimental data is shown in Fig. 2. The data is generated by using a 120 V catching voltage applied to the lower coil 1 ms after the armature is released from the upper coil.

5

Control Analysis

In this section a detailed dynamical analysis of the system is presented using nonlinear and linear (small signal) analysis around equilibria. The results presented here will serve as the basis for the various controller designs presented in Section 7.

Velocity [m/s]

0 1 2 3 4 4

KatherineÿPeterson et al. Exp. Data Simulation

0.14 Experimental Data

0.12 Position [mm]

Position [mm]

8 6 4 2 00

Current [A]

184

0.1 0.08 0.06 Simulation

0.04 0.02

2

0

0 0

0.5

1

1.5

2

2.5 3 Time [ms]

3.5

4

4.5

5

0.02

3.8

4.0

4.2

4.4

4.6 4.8 Time [ms]

5.0

5.2

5.4

Fig. 2. Comparison of the model to experimental data (left) and the impact detail (right).

5.1

Stability

An equilibrium position is defined by the voltage that results in a constant current with magnetic force equal to the spring force. Fig. 3 shows the magnetic force for four different values of current and the spring force as a function of position. Their intersection defines four equilibrium points. Analysis of the system demonstrates that all the equilibria near z = 0 are unstable. A perturbation from equilibrium which decreases z will accelerate the armature toward the coil seat as the spring force increases linearly while the magnetic force increases quadratically with the decreasing position. This acceleration can result in high impact velocities if the current is not rapidly adjusted to a lower value. On the other hand, a perturbation which increases z will reduce the magnetic force and the spring will therefore push the armature toward the middle position where the equilibria are stable.

900 800 700 Spring Force

Force [N]

600 500 400

Increasing Current

300 200 100 0 0

0.5

1 1.5 Distance from catching coil [mm]

2

Fig. 3. Magnetic force for several values of current.

Control of Electromechanical Actuators: Valves Tapping in Rhythm

185

The addition of the normal force to (9) not only adds impact dynamics, but also explicitly shows the multiple equilibria of the system at the contact point. The system is at equilibrium for the set of states and inputs

xe = {[i, z, v] | i ≥ ie , z = 0, v = 0} i ie = {i | Fmag (i, 0) = 4ks } , Vc = . r

(11) (12)

This is a very important result for controller design considerations. Specifically: • If there exists an infinite set of equilibrium points, all of which result in the armature being held against the magnetic coil, which one should the controller drive the system to? • If linear control theory is to be used, which equilibria should be selected to linearize the system about? The answer to the first question should be based on a trade-off between robustness against bouncing and power consumption. An equilibrium point, defined by (11), that uses a small holding current runs the risk of losing the armature due to bouncing and/or disturbances acting on the valve. The smallest of the four magnetic force curves in Fig. 3just barely exceeds the spring force when the armature is in contact with the catching coil. Therefore the current corresponding to this curve should be sufficient to hold the armature against the magnetic coil. However, a small deviation in position reduces the magnetic force more than the spring force. If the armature were to bounce due to impact the magnetic force would no longer be great enough to hold the armature in place. On the other hand, a large holding current will increase the power consumption of the actuator and can potentially eliminate the projected fuel economy benefits of an engine equipped with a VVT system. Three times the minimum holding current is selected and used in the sequel. A combined design-optimization study needs to be performed to rigorously address this question in the future.

5.2

Control Difficulties

The system suffers from low control authority in controlling the armature position from the voltage input throughout the executed motion. The underlying reasons are dynamic during small gaps and static during large gaps. Specifically, during small gaps the low control authority arises from the decreased inductance combined with the increased back-emf that drives the current to zero exceedingly fast. During large gaps the magnetic force is not strong enough to balance the spring force. These phenomena need to be understood in order to design a successful controller. Let us first consider the small gaps where the current is less than the saturation current, and then substitute (5) into (1). The resulting equation is a first order approximation of the current dynamics. di = 1000 dt

µ

(Vc − ri) (kb + z) iv + 2ka (kb + z)

¶ (13)

186

KatherineÿPeterson et al.

Note that kb ¿ 1 and z is approaching zero. Let us replace the term kb + z with the small parameter ε = kb + z, resulting in ¶ µ ¶ µ (Vc − ri) ε (Vc − ri) ε2 di iv di + + iv (14) = 1000 or ε = 1000 dt 2ka ε dt 2ka Two observations should be made from (14). First, the input voltage is being multiplied by ε, therefore very near the catching coil, the voltage cannot easily affect the current dynamics. Second if we multiply through by ε we see that the current dynamics become singularly perturbed near the catching coil. For z > 1 mm these affects aren’t present as ε is greater than 1. When the gap is greater than approximately 1 mm the catching coil lacks control authority over the motion of the armature because the magnetic force is much less the spring force as shown in Fig. 3. Therefore the system response is dominated by the springs. Despite this, it is necessary to apply voltage to the catching coil while the armature is not near it. If voltage is not applied to the system until the armature is close to the catching coil, it will be extremely difficult to raise the current because of the back-emf. It is much easier to raise the current before the system becomes singularly perturbed, and then apply large inputs near the end of the transition to overcome the current decaying effects from the back-emf seen in equation (14).

5.3

Small Signal Analysis

To capture the system dynamics throughout a complete transition, we linearize the system about two different equilibria to obtain two linear models, which will henceforth be refer to as • near model, which is valid for z ∈ (0, 1) mm. • far model, which is valid elsewhere. The far model is derived by linearizing the system at an equilibrium point slightly away from the mid-position. For the first 7 mm of the transition the mechanical and electrical subsystems of the EMV actuator are essentially a decoupled mass-spring-damper and RL circuit respectively. The far model is used to capture this behavior. Near the catching coil, the mechanical and electrical subsystems are no longer decoupled as the magnetic force has influence over the armature motion and the back-emf and changing inductance become significant. From Fig. 3 we see that linearizing the system at ie will cause the near model to be highly unstable as the magnetic force drops off to a much smaller value than the spring force due to a small deviation in position. A higher value of equilibrium current will result in a near model that is (i) a better approximation of the nonlinear system behavior, and (ii) a safer equilibrium point since it can account for small normal forces during bouncing. Both the near and far model have the structure        ∆i ∆i b1 a11 0 a13 d  ∆z  =  0 0 a23   ∆z  +  0  ∆Vc dt a31 a32 a33 0 ∆v ∆v

(15)

Control of Electromechanical Actuators: Valves Tapping in Rhythm

187

d ∆x = A∆x + B∆Vc , where A and B have different values written compactly as dt for the near and far region. Note that the term a12 is zero indicating that the rate does not depend on position, although (13) clearly shows of change of current, d∆i dt strong dependency in position in non-equilibrium conditions. Indeed, linearization not being a function of position as around any equilibrium point will result in d∆i dt shown below. Taking the partial derivative of (1) with respect to position yields ¡ d ¢ ∂ χ2 v dz χ1 χ2 − (Vc − ri + χ1 v) ∂z ∂ di . (16) = 2 ∂z dt (χ2 )

To satisfy equilibrium conditions the velocity must be zero, v = 0 and the applied voltage must equal to the voltage drop in the resistance, Vc = ri. Thus, the terms in ∂ di =0 both parentheses in the numerator of (16) will be zero, and consequently, ∂z dt when evaluated at any equilibrium point. From (14) we know that this only presents a problem for the near model where the current becomes singular perturbed. For large values of z, the changing inductance and back-emf are negligible and are of no consequence. As z approaches zero, the changing inductance and back-emf are no longer negligible and must be taken into account. To account for the singularly perturbed current dynamics present in the nonequilibrium full nonlinear equations during small gaps, we assume current reaches 1 ∆Vc governed by the fast stable dynamics its steady state solution ∆i∗ = − a13 ∆v+b a11 of the near model: d∆i = 0 = a11 ∆i + a13 ∆v + b1 ∆Vc , dt

(17)

resulting in the reduced state space representation d dt

·

∆z ∆v

¸

·

0 a∗12 = a∗21 a∗22

¸·

¸ · ¸ ∆z 0 + ∗ ∆Vc . ∆v b2

(18)

Recall that the rate of change of current is approaching large negatives values and thus can be neglected from the control design for the linear near model. This is also verified by the fact that the original a11 term in (15) is Hurwitz.

6

Observer Design

High cost and implementation issues preclude the use of sensors to measure all three states. For each of the controllers presented later in Sect. 7 only a position sensor is used, and an observer is implemented to estimate velocity and current. ¤T £ where A is from Unfortunately, the observability matrix C T (CA)T (CA2 )T £ ¤ the far model and C = 0 1 0 , is ill-conditioned. Therefore one or more states are weakly observable from the position measurement. From the physics of the system it is obvious that current is the weakly observable state. For the majority of the armature travel the magnetic force, and thus current, has little influence over the armature motion (i.e. the system output). Although

188

KatherineÿPeterson et al.

this is to be expected for the far model, it is important to note that using the near model will also result in an ill-conditioned observability matrix. At small distances the magnetic force is influenced by changes in position more than changes in current. Thus the affect of current on the output is still weakly observable. Only the far model is used to design the observer. Of the two linear models, the far model is valid for almost the entire range of motion. Rather than deal with the difficulties of switching between two dynamical observers, the output injection term is used to ensure accuracy of the state estimates. Setting a31 = 0, which is small in comparison to a32 and a33 , removes d∆v dt e is given by dependence on current. The new state space matrix, A, 

 · ¸ a11 0 a13 e =  0 0 a23  = Ao A12 . A 0 Ao 0 a32 a33

(19)

The system can now be decomposed into observable and unobservable parts, Ao and Ao respectively. Using the nonlinear model presented in [15] a nonlinear exponential detector is implemented as db x = f (b x, Vc ) + L (y − yb) , dt

(20)

Current [A]

Velocity [m/s]

Position [mm]

where, the function f (b x, Vc ) is the same as (4) except that it does not include £ ¤T the saturation dynamics in (6)-(8). The matrix L is given by L = 0 l1 l2 , where l1 and £ l¤2 are tuned based on linear quadratic estimation methods on Ao and Co = 1 0 . 8 6 4 2 0 0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

2

2.5

3

3.5

4

4.5

0 2 4 0 3 2

Actual State Estimated State

1 0 0

0.5

1

1.5

Time [ms]

Fig. 4. Comparison of the estimated vs. actual states. A comparison of the actual and estimated states is presented in Fig. 4. The observer is able to estimate position and velocity with very little error. The current estimate matches the actual state closely for the initial part of the transition, with the estimation error increasing toward the end. Recall that (20) does not include the

Control of Electromechanical Actuators: Valves Tapping in Rhythm

189

saturation region in (6)-(8), thus at the end of the transition when saturation occurs the nonlinear model is not accurate. Additionally the current estimate is running open loop therefore output injection can not be used to drive the estimation error to zero.

7

Controller Design

This section presents various controllers implemented to achieve soft landing of an EMV actuator. The control difficulties outlined in Sec. 5.2 imply that for a controller to be successful it must: 1. Apply voltage during the initial motion of the armature to raise the coil current and establish a magnetic field as the armature approaches the catching coil. The strong magnetic field helps to compensate for the inevitable current drop caused by the back-emf and changing inductance near the catching coil. 2. Apply large voltage near the catching coil to avoiding bouncing and to compensate for the decreasing current and reduced influence of the voltage on the current dynamics.

7.1

Linear Controller

The linear controller uses the feedback, u = ueq − K∆b x. Where ueq = ides , ides is r the desired current value, typically 0.5-1.5 A, that the controller drives the current b is the estimate of the actual state, and xe is the equilibrium to, ∆b x=x b − xe , x point. The gain matrix, K, is determined by using the Linear Quadratic Regulator (LQR) methodology with diagonal weighting matrices Q and R. Just as two linear models are used to capture the system dynamics, the linear controller is split to compensate for the changing system dynamics. The two controller stages are named: “Flux Initialization” and “Landing”.

Stage 1 Controller, Flux Initialization: To overcome the problems inherent to the current dynamics it is desirable to bring the current near a nominal catching value before the armature approaches the catching coil. On one hand, if the current is not brought up before the armature approaches the catching coil, it will be difficult to do so later due to the reasons explained in Sect. 5.2. On the other hand, closed loop control during this time may result in actuator saturation as the magnetic force has low authority over the armature motion for z > 1 mm. To improve robustness within a cycle, our controller uses the flux initialization stage to apply closed loop control throughout the travel z ∈ (1, 8) mm. Using the far model, the weighting matrices in the LQR method are chosen to penalize deviations from the nominal catching current much more than deviations in position or velocity. Since the actuator has control authority over current, actuator saturation is avoided. Additionally, robustness is improved as the controller can compensate for variations in both position and velocity. Stage 2 Controller, Landing: As the armature approaches the catching coil, the controller switches to the second stage at z = 1 mm. The landing controller catches the armature and brings it into contact with the coil while attempting to minimize the impact velocity. The reduced order near model, given in (18), is used in the LQR method to design the landing controller. In (15), a11 is Hurwitz, and consequently the current

190

KatherineÿPeterson et al.

dynamics are stable. Therefore we need only design a stabilizing feedback for the reduced order model. A typical soft landing is presented in Fig. 5. The impact velocity represents a factor of six reduction over the open loop control. Statistical data for the impact velocity is listed in Table 1. The controller achieves a transitions time with a mean of 3.42 ms. The standard deviation, σ, is 0.2 ms, and the maximum and minimum transition times are 4.3 ms and 3.3 ms respectively.

7.2

Nonlinear Controller

Equation (14) shows the sever limitations of linear control methodologies for the EMV actuator. In a linear controller the input voltage is proportional to the error (i.e. Vc = Kz) exacerbating the decrease of the influence of the input voltage as z approaches zero. Thus large values of the gain, K, will be required to overcome the changing current dynamics. This can, and does lead to actuator saturation as shown in Fig. 5. Moreover, it leads to small voltage inputs near the catching coil, potentially creating problems with bouncing. To overcome the changing current dynamics and reduced control authority we propose the use of a nonlinear controller of the form Vc =

K1 K2 v+ . γ+z β+z

(21)

The control input is inversely proportional to the distance from the catching coil, thereby alleviating the affect of the decreasing influence of the voltage on the current dynamics. The parameters K1 , K2 , γ, and β are used to tune the controller to achieve the desired performance. Since the current is not driven near the minimum equilibrium value, the magnetic force will remain larger than the spring force for small bounces. Thereby eliminating the potential loss of the armature due to bouncing. Even thought the nonlinear controller does experience actuator saturation, the affect is much smaller than that caused by the linear controller. Whereas the linear controller causes voltage saturation before the valve is fully open/closed, the nonlinear controller does so only at the very end of the transition. Experimental results in Fig. 5 and Table 1 show that the controller achieves a mean impact velocity of 0.16 m/s. Similar to the linear controller, the nonlinear controller represents a factor of six reduction over the open loop control. The nonlinear controller achieves a mean transition time of 3.23 ms. Although the mean values achieved with the nonlinear controller are similar to the ones achieved with the linear controller, the results are more consistent as indicated by the standard deviation, σ = 0.04 ms, and the maximum and minimum transition times which are 3.3 ms and 3.2 ms respectively.

8

Extremum Seeking Control

So far the controller design has only considered closed loop compensation during a single transition. By applying a tuning algorithm from cycle to cycle it is possible to improve the performance of the actuator and adjust the controller parameters to account for changes in the system due to environmental variations. Here, an extremum seeking controller [7] is used to tune the feedback between each transition to minimize the impact velocity.

2 0

2 0 0 3

0.5

1

1.5

2

2.5

3

3.5

4

2 4 4.5

2 1 0.5

1

1.5

2

2.5

3

3.5

4

2

Velocity

4

0

2 0 0 3

2 0.5

1

1.5

2

2.5

3

3.5

4 4

0.5

1

1.5

2

2.5

3

3.5

4

0.5

1

1.5

2

2.5

3

3.5

4

2 1

0 0 200

4.5

4

Position

6

Voltage [V]

Voltage [V]

0 0 200

Nonlinear Controller Results

8

191 Velocity [m/s]

Velocity

4

4

Position

Position [mm]

6

Velocity [m/s]

Linear Controller Results

8

Current [A]

Current [A]

Position [mm]

Control of Electromechanical Actuators: Valves Tapping in Rhythm

100

100 0 0

0.5

1

1.5

2

2.5

3

3.5

4

0 0

4.5

Time [ms]

Time [ms]

Fig. 5. Experimental results achieved by using the linear (left) and nonlinear controller (right). Applying extremum seeking control to a static nonlinearity minimizes or maximizes the system output, J. While the EMV actuator with nonlinear feedback is not a static nonlinearity, it can be treated as such. If one of the four parameters K1 , K2 , γ, or β of the nonlinear feedback is taken to be the input and the impact velocity or another relevant function is used as the output, J. To account for the delay between the start of the valve transition and the armature impacting against the catching coil, the extremum seeking control is discretized with a sampling rate equal to the rate of the valve transitions. 0.4 0.35

EMV Actuator

J

a sin(wkT)

0.3

Run #1

0.25

Position y

0.2

Nonlinear Controller -

T z-1

0.15 0.1 0.05 0

β

0.05 0.1

Run #20

4

4.5

5

5.5

6

6.5

7

Time [m/s]

Fig. 6. Extremum seeking feedback as applied to the EMV actuator (left), and extremum seeking feedback results (right). The parameter β, has a very strong influence on the impact velocity, vI , and is therefore selected as the input. This method does not depend on measuring the impact velocity, which could be impractical and expensive. All that is required is an output that is proportional to the impact velocity in order to determine whether or not the previous impact velocity was larger or smaller. Here a small microphone is used to pick off the sound caused by the impact and the extremum seeking control is set to maximize the function J = − (Smin − Smeas )2 . Where Smin is the

192

KatherineÿPeterson et al.

desired sound level, and Smeas is the measured sound level. Note that Smin needs to be carefully set to a non-zero value otherwise the extremum seeking control will minimize the sound intensity by avoiding any contact which is obviously not desired. The output function can be generated by a variety of other sensors that relate to the impact velocity such as accelerometers, load washers, and knock sensors. To test the extremum seeking control the feedback is initialized at a non-optimal value of β and allowed to run, the results are shown in Table 1 and Fig. 6. By compensating for the day to day variations in the system the extremum seeking control has improved the system performance by a factor of 2. More details on this algorithm can be found in [11]. Table 1. Impact velocities achieved by the different controllers. Linear Controller

Nonlinear Controller

Extremum Seeking

Mean

0.16 m/s

0.16 m/s

0.08 m/s

σ

0.09 m/s

0.08 m/s

0.05 m/s

Max

0.35 m/s

0.32 m/s

0.20 m/s

Min

0.06 m/s

0.05 m/s

0.05 m/s

9

Conclusion

The EMV actuator presents an interesting and challenging control design problem. The controller must achieve stringent performance requirements for soft and fast landing (impact velocities below 0.1 m/s and transition time smaller than 4.0 ms). The input-to-output behavior is nonlinear with low control authority for a combination of reasons that once understood a practical control solution arises. Despite the improvement in impact velocity presented here, much work remains to be done. Before the EMV actuator can be used on a firing engine, the controller must be able to compensate for gas forces acting on the valve due to the combustion in the cylinder and any valve lash that may be present to compensate for the thermal expansion between the valve stem and the armature.

References 1. Ahmad T and Theobald MA. A Survey of Variable Valve Actuation Technology. SAE 891674. 2. Butzmann S., Melbert J., and Koch A., “Sensorless Control of Electromagnetic Actuators for Variable Valve Train,” SAE Paper No. 2000-01-1225. 3. Hatano K., Iida K., Higashi H., and Murata S., “Development of a New MultiMode Variable Valve Timing Engine”, SAE Paper No. 930878, 1993. 4. Hrovat D. and Powers W. F., “Modeling and Control of Automotive Powertrains”, Control and Dynamic Systems, Vol. 37, pp. 33-64, 1990. 5. Hoffmann W., Stefanopoulou A., “Iterative Learning Control of Electromechanical Camless Valve Actuator,” Proceedings American Control Conference, pp.2860-2866, June 2001.

Control of Electromechanical Actuators: Valves Tapping in Rhythm

193

6. Kreuter P., Heuser P., and Schebitz M., “Strategies to Improve SI-Engine Performance by Means of Variable Intake Lift, Timing and Duration”, SAE Paper No. 920449 , 1992. 7. Krstic M., Wang H.H., “Stability of extremum seeking feedback for general nonlinear dynamic systems,” Automatica, vol. 36, pp595-601, 2000. 8. R. B. Mathews, “Electromagnetic Control Device,” United States Patent 2,769,943, August 22, 1949. 9. Moriya Y., Watanabe A., Uda H., Kawamura, H., and Yoshioka M., “A Newly Developed Intelligent Variable Valve Timing System- Continuously Controlled Cam Phasing as Applied to a New 3 Liter Inline 6 Engines”, SAE Paper No. 960579, 1996. 10. Peterson, K., Stefanopoulou A., Haghgooie M., Megli, T. “Output Observer Based Feedback for Soft Landing of Electromechanical Camless Valvetrain Actuator”, Proceedings of American Control Conference, pp. 1413-1418, May 2002. 11. Peterson K., Stefanopoulou A., Wang Y., Megli T., Haghgooie M., “Nonlinear Self-Tuning Control for Soft Landing of an Electromechanical Valve Actuator,” to be presented at the 2002 IFAC on Mechatronics. 12. Schechter M. M. and Levin M. B., “Camless Engine”, SAE Paper No. 960581, 1996. 13. Tai C., Stubbs A., Tsao T.C., “Modeling and Controller Design of an Electromagnetic Engine Valve,” Proceedings of American Control Conference, pp. 2890-2895, June 2001. 14. Titolo A., “The Variable Valve Timing System - Application on a V8 Engine”, SAE Paper No. 910009, 1991. 15. Wang Y., Stefanopoulou A., Peterson K., Megli T., Haghgooie M., “Modeling and Control of Electromechanical Valve Actuator,” SAE 2002-01-1106

Learning complex systems from data: the Set Membership approach Mario Milanese and Carlo Novara Dipartimento di Automatica e Informatica, Politecnico di Torino Corso Duca degli Abruzzi 24, 10129 Torino, Italy Email: [email protected], [email protected]

Abstract. In the paper the problem of making inferences on unknown nonlinear system (e.g. identification, prediction, smoothing, filtering, control design, decision making, fault detection, etc.) based on finite and noise-corrupted measurements is considered. Inferences are usually obtained by means of models of the system, estimated from measurements within a finitely parametrized model class describing the functional form of involved nonlinearities, whose proper choice is realized by a search, from the simplest to more complex ones (linear, bilinear, polynomial, neural networks, etc.). In this paper an alternative approach, recently developed by the authors is presented. The approach, based on a Set Membership framework, does not need assumptions on the functional form of the regression function describing the system, but requires only some information on its regularity, given by bounds on the derivatives. In this way, the problem of considering approximate functional forms is circumvented. Moreover, noise is assumed to be bounded, in contrast with statistical methods, which rely on assumptions such as stationarity, ergodicity, uncorrelation, type of distribution, etc., whose validity may be difficult to be reliably tested and is lost in presence of approximate modeling. In this paper some of the main results developed by the authors are presented within a unifying framework. In particular, necessary and sufficient conditions for checking the assumptions validity are given and optimal and almost optimal algorithms are presented for the cases that the desired inferences are identification and prediction.

1

Introduction

Consider a nonlinear dynamical system of the form: yt+1 = f o (wt ) wt = [yt ... yt−ny +1 u1t ... u1t−n1 +1 .............. m um t ... ut−nm +1 ]

(1)

Pm o n where yt , u1t , ..., um t ∈ <, f : < → <, n = ny + i=1 ni . o Suppose that the function f is not known, but a set of noise corrupted measurements of yt and wt , t = 1, 2, ..., T is available, and it is of interest to make an

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 195--206, 2003 Springer-Verlag Berlin Heidelberg

196

Mario Milanese and Carlo Novara

inference on the system (e.g. identification, prediction, smoothing, filtering, control design, decision making, fault detection, etc.). . The inference is described by the operator I (f o , WT ), where WT = {w1 , w2 , o o ..., wT }. For example, I (f , WT ) = f if the desired inference is the identification of the unknown function f o and I (f o , WT ) = f o (wT ) if the inference is one-step prediction. Clearly, being f o and WT unknown, the exact inference I(f o , WT ) cannot be derived and the usual approach is to obtain from data ´ ³ some o b c cT . estimate f of f , and WT of WT and to make the approximate inference I fb, W Then, problems the inference error measured as ° ³ ° ³ the´ basic ´ arise of: i) evaluating . ° ° o b b c c E f , WT = °I f , WT − I (f , WT )°, where || · || is a suitable norm; ii) finding ³ ´ cT . a function fb minimizing E fb, W Since data are finite and noise corrupted, providing only limited information on f o , whatever approximating function fb is used, no finite bound on the inference error can be derived if no information is available on f o and on noise. The information on f o is typically given by considering that it belongs to some subset K of functions f :
2

The Nonlinear Set Membership approach

In this section, the problem of making inferences on a dynamical system is formulated in a Set Membership (SM) framework, see e.g. [9–11]. This approach has

Learning complex systems from data: the Set Membership approach

197

strong connection with Information Based Complexity (IBC) methods used in approximation theory and numerical analysis, see e.g. [12]. Consider that a set of data yt and wt generated by (1) is available, corrupted by noise: y˜t = yt + et , t = 1, .., T w et = wt + e0t , t = 1, .., T Suppose that an operator I is given, mapping the couples (f, WT ) into a normed space, called inference space. The aim is to estimate the inference I(f o , WT ) by means of a suitable operator φ mapping available measurements (˜ y , w) ˜ into the normed space of inferences. The operator φ, called inference algorithm, should ˆ = ||I(f o , WT ) − be chosen to give small (possibly minimal) inference error E(I) φ(˜ y , w)||. ˜ This error is not known, since from available data it is only known that y , w), ˜ the set of all f and WT that can have generated the data. (f o , WT ) ∈ F (˜ y , w)), ˜ Thus, the desired inference I(f o , WT ) is only known to belong to the set I(F (˜ which is unbounded, since the mapping generating data from given f is not injective. Then, whatever algorithm φ is chosen, no finite bound on the inference error can be guaranteed, unless some assumptions are made on the function f o and the noise sequences e and e0 . The typical approach in the literature is to assume a finitely parametrized functional form for f o (linear, bilinear, etc.) and statistical models on the noise sequences [1–4]. In the present SM-IBC approach, different and somewhat weaker assumptions are taken, not requiring the selection of a functional form for f o , but related to its rate of variation. Moreover, noise sequences are only supposed to be bounded. Prior assumptions on f o : . f o ∈ K = {f : W ⊆
198

Mario Milanese and Carlo Novara

Remark 1 A global bound on kgradf o (w)k2 over all W is considered. However, according to what done in other contexts (see e.g. [13,14]), a local approach can be taken in order to obtain improvements in inference accuracy. A very simple approach allowing to use local assumptions on gradf o , is based on the evaluation of a function f a approximating f o and on the application of the method described in this paper to the residue function, defined as: . ∆f (w) = f o (w) − f a (w) starting from the set of values: ˜t ) , t = 1, 2, ..., T − 1 ∆yt+1 = y˜t+1 − f a (w The global bound kgrad∆f (w)k2 = kgradf o (w) − gradf a (w)k2 ≤ γr on ∆f implies the local bound kgradf a (w)k2 − γr ≤ kgradf o (w)k2 ≤ ||gradf a (w) ||2 + γr for function f o . See [8] for more details about this local approach. Remark 2 Further improvements in inference accuracy can be obtained by assuming a bound on the weighted norm of gradf o (w): kgradf o (w)kν2 ≤ γ . pPn 2 where kxkν2 = i=1 νi xi , νi > 0. The weighted norm can be useful in order to adapt to the properties of data, by properly choosing the weights νi . This corresponds to use an unweighted norm on the scaled variable √1νi wi . Such scaling is very important when the gradient components have quite different magnitudes. See [7] for a discussion on the choice of the weights νi . A key role in this Set Membership framework is played by the Feasible Systems Set, often called “unfalsified systems set”, i.e. the set of all systems consistent with prior information and measured data. Definition 1. Feasible Systems Set The Feasible Systems Set F SST is: . F SST = {f ∈ K : y˜t+1 = f (wt + e0t ) + et+1 , (e0t , et+1 ) ∈ Bte , t = 1, 2, ..., T − 1} ª . © where: Bte = (e0t , et+1 ) : ke0t k2 ≤ δt , |et+1 | ≤ εt+1 The Feasible Systems Set F SST summarizes all the information on the mechanism generating the data that is available up to time T . If prior assumptions are “true”, then f o ∈ F SST , an important property for evaluating the accuracy of inferences that can be done on the system. Clearly, the problem of checking the validity of prior assumptions arises. Indeed, the only thing that can be actually done is to check if prior assumptions are invalidated by data, evaluating if no system exists consistent with data and assumptions, i.e. if F SST is empty. However, it is usual to introduce the concept of prior assumption validation as follows:

Learning complex systems from data: the Set Membership approach

199

Definition 2. Validation of prior assumptions Prior assumptions are considered validated if F SST 6= ∅. Note that the fact that the priors are validated by using the present data does not exclude that they may be invalidated by future data, as it is always the case in validation problems ([15]). Necessary and sufficient conditions for checking the assumptions validity, are given below in Result 1. Let us introduce the following quantities: ¡ ¢ . ht − γ kw − w et k2 f (w) = max t=1,...,T −1 ¡ ¢ (2) . f (w) = min ht + γ kw − w et k2 t=1,...,T −1

. . where: ht = yet+1 + εt+1 + γδt and ht = yet+1 − εt+1 − γδt .

Result 1. [5,8] i) f (w et ) ≥ ht , t = 1, 2, ..., T − 1, is necessary condition for prior assumptions to be validated. et ) > ht , t = 1, 2, ..., T − 1, ii) f (w is sufficient condition for prior assumptions to be validated. In the following, the F SST is assumed to be non-empty. If empty, the prior assumptions on the system and the noise are invalidated by data and have to be suitably modified to give a non-empty F SST . The previous result can be used for assessing the values of ε = [ε1 ... εT ], δ = [δ1 ... δT ] and γ. In the space (ε, δ, γ), the function: . γ ∗ (ε, δ) =

inf

F SS6=∅

γ

individuates a surface that separate falsified values of ε, δ and γ from validated ones. Clearly, ε, δ and γ must be chosen in the validated parameters region, with some “caution” (i.e. not too near to the separation surface) and possibly using information from the experimental setting. For example, if one is confident that the noise affecting the output measurements is essentially due to the measurement devices, the value of ε may be obtained by the instrumentation accuracy. On the other hand, useful information on γ values can be obtained by deriving (e.g. from a neural networks approximation or directly from data) some estimates of gradf o (w) . See [6–8] for the use of this procedure to practical applications. An inference algorithm φ is an operator mapping all available information about y , w) e until time T , summarized by F SST , into function f o , noises e and e0 , data (e an estimate Ib of inference I(f o , WT ): φ (F SST ) = Ib ' I(f o , WT ) The related inference error is: ° ³ ´ ° ° ° E Ib = °Ib − I(f o , WT )° = kφ (F SST ) − I (f o , WT )k

200

Mario Milanese and Carlo Novara

o which cannot ´be exactly computed, since it is only known that f ∈ F SST and ³ . fT = {WT : kwt − w WT ∈ B W et k2 ≤ δt , t = 1, 2, ..., T }. The tightest bound on ³ ´ E Ib is given by:

³ ´ E Ib ≤

sup

sup

f ∈F SST W ∈B (W fT ) T

kφ (F SST ) − I (f, WT )k

This motivates the following definition of the worst-case inference error. Definition 3. Worst-case inference error The worst-case inference error of Ib = φ (F SST ) is: . b = W E(I)

sup

f ∈F SST

° ° °b ° sup °I − I (f, WT )° fT ) WT ∈B (W

Looking for algorithms that minimize the worst-case inference error, leads to the following optimality concepts. Definition 4. Optimal inference estimate and algorithm An inference estimate I ∗ is called optimal if: . W E (I ∗ ) = inf W E [φ (F SST )] = rI φ

An inference algorithm φ∗ is called optimal if: W E [φ∗ (F SST )] = inf W E [φ (F SST )] , ∀F SST φ

Thus, an optimal inference algorithm gives optimal inferences for any available information up to time T . The quantity rI gives the minimal inference error that can be guaranteed by any inference estimate based on the available information up to time T and is called (local) radius of information. The following result shows that, in the case inference operator I is linear, an optimal inference algorithm can be obtained from the function: ¤ . 1£ f (w) + f (w) f c (w) = 2 . fT = {w e1 , w e2 , ..., w eT }. where f (w) and f (w) are defined in (2). Let W Result 2. [16] If the inference operator I is linear, then the algorithm: fT ) φc (F SST ) = I(f c , W is optimal for any choice of norm in the inference space.

Learning complex systems from data: the Set Membership approach

201

This result holds also for nonlinear inference operators such that fT − WT ) − I(f c , W fT ) = −I(f c + f, W fT + WT ) + I(f c , W fT ). I(f c − f, W If this result cannot be applied, finding optimal algorithms may be hard, as well known in SM-IBC theory (see e.g. [10–12] ). This motivates the interest of deriving simpler algorithms, at the expense of some degradation in the inference error with respect to an optimal algorithm. In particular, algorithms guaranteeing a degradation in the prediction error of at most 2 are widely considered in the literature (see e.g. [11,12]), and called “almost optimal”, according to the following definition. Definition 5. Almost optimal inference algorithm An inference algorithm φao is called almost optimal if: W E [φao (F SST )] ≤ 2 inf W E [φ (F SST )] , ∀F SST φ

3

Nonlinear systems identification

In the case that the desired inference is identification of f o , the inference operator is I(f, WT ) = f . Since this operator is linear, it follows from Result 2 that the algorithm φc (F SST ) = f c is an optimal identification algorithm for any norm kf k. In the case that Lp norm is used, also the radius of information rI , i.e. the minimal worst case identification error, can be easily evaluated from functions f and f . Thus, we have the following result. Result 3. [16] i) The identification algorithm φc (F SST ) = f c is optimal ii) If kf k = kf kp , with 1 ≤ p ≤ ∞, then: ° ° rI = W E (f c ) = 21 °f − f °p = inf φ W E [φ (F SST )] Some properties of optimal estimate f c are now discussed. Note that the optimal estimate f c is the Chebicheff center of F SST for any norm, i.e. supfe∈F SST ° ° ° ° ° °e ° ° °f − f c ° = inf f supfe∈F SST °fe − f °, but it does not belong to F SST , since it is not differentiable everywhere. Indeed, f c is Lipschitz-continuous and differentiable almost everywhere, as shown in the next two results. Result 4. [5,8] The function f c is Lipschitz-continuous on W. In order to formulate the differentiability result, let us introduce the notion of hyperbolic Voronoi diagrams, a generalization of the standard Voronoi diagrams (see e.g. [17]) introduced in [5]. Suppose that a set of points: . WN = {wk ∈
202

Mario Milanese and Carlo Novara

• the (n − 1)-dimensional hyperbola Hkj : . Hkj = {w ∈
The intersections between the surfaces Hkj generate other cells of dimension d, with 0 ≤ d ≤ n − 1 that we call d-faces. The cells Ck are called n-faces while the a complete partition of
∆h γ

´

and

∆ht2 t1 = ht2 − ht1 , ∆ht2 t1 = ht2 − ht1 fT −1 = {w W et , t = 1, 2, ..., T − 1}

´ ³ fT −1 , ∆h and and V 1 be the sets of the 1-faces of V W γ ³ ´ fT −1 , ∆h , respectively. Let co{V 1 ∪ V 1 } the complement in W of the set V W γ Let V

1

1

1

1

{V ∪ V 1 }, i.e. {V ∪ V 1 }∪ co{V ∪ V 1 } = W. Result 6. [8,16] n 1 o i) V ∪ V 1 is a set of zero measure on 0, ∃ f²c ∈ F SST :

kf²c − f c k < ²

Learning complex systems from data: the Set Membership approach

4

203

Prediction of nonlinear systems

For the sake of simplicity one-step ahead prediction is considered, but multistep prediction can be treated as well [6]. In this case the inference operator is I (f, WT ) = yT +1 = f (wT ). Unfortunately, since such operator is in general not linear, Result 2 cannot be applied to find an optimal prediction algorithm. However, two almost optimal prediction algorithms are presented together with upper and lower bounds on their worst case prediction errors. Conditions are also given, under which such algorithms give actually optimal prediction and their error bound is actually the minimal worst case prediction error achievable by any algorithm. In order to formulate these results,³the following are needed, based ´ definitions ³ ´ ∆h ∆h f f on the hyperbolic Voronoi diagrams V WT −1 , γ and V WT −1 , γ defined in Result 5. Since the cells of a Voronoi diagram constitute a complete³partition of ´ fT −1 , ∆h
Diagram V

V(C composed of w eT and the vertices of the Hyperbolic Voronoi Diagram ³ t ) the set ´ fT −1 , ∆h contained in Bδ (w V W eT ). Let us define the following quantities: γ

wT = w eT + δT

w eT − w et , ||w eT − w et ||2

f u (w eT ) = f (w eT ) + γδT , ( f l (w eT ) =

wT = w eT + δT

eT ) = f (w f l (w eT ) − γδT

f u (w eT ) if wT ∈ Ct maxw∈V(C ) f (w) otherwise t

( f u (w eT ) = f1c (w eT ) =

f l (w eT ) if wT ∈ Ct minw∈V(Ct ) f (w) otherwise

i 1h f u (w eT ) eT ) + f l (w 2

Result 8. [8] i) The prediction algorithm: eT ) φc1 (F SST ) = f1c (w

w eT − w et ||w eT − w et ||2

204

Mario Milanese and Carlo Novara

is almost optimal, with prediction error bounded as: h i . 1 eT ) = f l (w eT ) − f u (w 2 . = W E [φc1 (F SST )] ≤ ≤ W E [φc1 (F SST )] ≤ . ≤ W E [φc1 (F SST )] = ³ ´ . c c = max |φ1 (F SST ) − f l (w eT ) |, |φ1 (F SST ) − f u (w eT ) | eT ) is ii) If W E [φc1 (F SST )] = W E [φc1 (F SST )], then the prediction yTc +1 = f1c (w optimal with minimal worst-case prediction error given by: rI = W E [f1c (w eT )] =

¤ 1£ f (w e T ) − f (w eT ) + γδT = inf W E [φ (F SST )] φ 2

It can be noted that the condition W E [φc1 (F SST )] = W E [φc1 (F SST )] under eT ) is optimal can actually be met. In particular, which the prediction yTc +1 = f1c (w eT ) ⊆ Ct ∩ Ct . it is certainly met whenever Bδ (w c Algorithm φ requires the computation of´the vertices of the Hyperbolic Voronoi ´ ³ ³ 1 fT −1 , ∆h . A simpler almost optimal algorithm, fT −1 , ∆h and V W Diagrams V W γ γ not requiring such computation, is provided by the following result. Result 9. [6] i) The prediction algorithm: φc2 (F SST ) = f c (w eT ) is almost optimal, with prediction error bounded as: W E [φc2 (F SST )] ≤

¤ 1£ eT ) + γδT f (w e T ) − f (w 2

ii) If Bδ (w eT ) ⊆ Ct ∩Ct , then the prediction ybT +1 = f c (w eT ) is optimal with minimal worst-case prediction error given by: eT )] = rI = W E [f c (w

5

¤ 1£ f (w e T ) − f (w eT ) + γδT = inf W E [φ (F SST )] φ 2

Conclusions

In the paper, a method for making inferences on nonlinear systems has been presented, based on a Set Membership approach, requiring quite weak assumptions on noise and on involved nonlinearities. At difference with most of methods in the literature, the method does not require to know the functional form of involved nonlinearities, thus reducing the effects of modeling errors. Moreover, the noise is assumed

Learning complex systems from data: the Set Membership approach

205

only to be bounded, in contrast with standard approaches, relying on statistical assumptions such as stationarity, uncorrelation, etc., whose validity is difficult to be reliably checked and anyway is lost in presence of approximate modeling. On the base of these theoretical features, it is expected that inferences obtained by means of the proposed method may have good performances and exhibit good robustness versus imprecise knowledge of involved nonlinearities and of noise properties. These expectations appear to be confirmed by the numerical results. Indeed, several simulated and real data sets have been considered in order to test the presented method in systems identification and time series prediction. In [6,8], the method has been applied to prediction of Wolf Sunspot Numbers series, widely used in the literature as a benchmark test, displaying sensible improvements over linear and nonlinear predictors taken from the literature, in particular for multistep ahead prediction. In [8], prediction of a river flow time series has been tackled and good results have been obtained by the Nonlinear Set Membership predictors with respect to neural networks predictors and just in time predictors used in the literature for this time series. In [7], the method has been applied to model identification of a water heater, showing an identification accuracy similar to those obtained by other methods, such as neural networks, fuzzy and just in time methods, using much stronger prior assumptions. Beside these three examples, where real world data have been used, the NSM method displayed interesting performances in other examples with simulated data, such as prediction of a linear system [6,8], identification and prediction of Lorenz chaotic system [8] and H∞ identification of a linear system from frequency domain data [18]. In conclusion, the new approach to nonlinear systems analysis presented in this paper appears to give quite promising results and is being tested on larger classes of simulated and real problems. Further advances are expected by the ongoing research on the selection of the values of the bounds on function gradient and on noise and of regressors scalings.

References 1. R. Haber and H. Unbehauen, “Structure identification of nonlinear dynamic systems- a survey on input/output approaches,” Automatica, vol. 26, pp. 651– 677, 1990. 2. J. Sj¨ oberg, Q. Zhang, L. Ljung, A. Benveniste, B.Delyon, P. Glorennec, H. Hjalmarsson, and A. Juditsky, “Nonlinear black-box modeling in system identification: a unified overview,” Automatica, vol. 31, pp. 1691–1723, 1995. 3. K. S. Narenda and S. Mukhopadhyay, “Neural networks for system identification,” in Sysid ’97, vol. 2, pp. 763–770, 1997. 4. R. Isermann, S. Ernst, and O. Nelles, “Identification with dynamic neural networks -architectures, comparisons, applications-,” in Sysid ’97, vol. 3, pp. 997– 1022, 1997. 5. C. Novara and M. Milanese, “Set membership identification of nonlinear systems,” in Proc. of the 39th IEEE Conference on Decision and Control, (Sydney, AU), pp. 2831–2836, 2000. 6. C. Novara and M. Milanese, “Set membership prediction of nonlinear time series,” in Proc. of the 40th IEEE Conference on Decision and Control, (Orlando, FL), pp. 2131–2136, 2001.

206

Mario Milanese and Carlo Novara

7. M. Milanese and C. Novara, “Set Membership estimation of nonlinear regressions,” in IFAC 2002, (Barcelona, Spain), 2002. 8. C. Novara, Set Membership Identification of Nonlinear Systems. Torino, Italy: PhD Thesis, Politecnico di Torino, 2002. 9. M. Milanese and R. Tempo, “Optimal algorithms theory for robust estimation and prediction,” IEEE Transaction on Automatic Control, vol. 30, pp. 730–738, 1985. 10. M. Milanese and A. Vicino, “Optimal algorithms estimation theory for dynamic systems with set membership uncertainty: an overview,” Automatica, vol. 27, pp. 997–1009, 1991. 11. M. Milanese, J. Norton, H. P. Lahanier, and E. Walter, Bounding Approaches to System Identification. Plenum Press, 1996. 12. J. F. Traub, G. W. Wasilkowski, and H. Wozniakowski, Information-Based Complexity. Academic Press, Inc., 1988. 13. A. Stenman, F. Gustafsson, and Ljung, “Just in time models for dynamical systems,” in Proc. of the 35th IEEE Conference on Decision and Control, (Kobe, Japan), pp. 1115–1120, 1996. 14. Q. Zheng and H. Kimura, “Just in time modeling for function prediction and its applications,” Asian Journal of Control, Vol. 3, No. 1,, pp. 35–44, 2001. 15. K. R. Popper, Conjectures and Refutations: The Growth of Scientific Knowledge. London: Rontedge and Kegan Paul, 1969. 16. M. Milanese and C. Novara, “Set Membership identification of nonlinear systems,” Politecnico di Torino internal report, 2002. 17. H. Edelsbrunner, Algorithms in Combinatorial Geometry. Berlin: SpringerVerlag, 1987. 18. M. Milanese, C. Novara, and M. Taragna, “”Fast” Set Membership H∞ identification from frequency-domain data,” in European Control Conference ECC 2001, (Porto, Portugal), 2001.

The mixing of state statistics Tryphon T. Georgiou

?

??

Dedicated to the memory of Mohammed Dahleh.

Abstract. The family of candidate power spectra for the input of a linear filter which are consistent with given state statistics is in bijective correspondence with solutions of a Carath`eodory analytic interpolation problem. We point out that state covariances of distinct linear filters driven by the same stochastic input process can be conveniently integrated into data for a suitable Carath`eodory problem encompassing all the relevant constraints.

1

Introduction

A fruitful viewpoint in spectral estimation is to consider as a central object of interest the family of power spectra which are consistent with the available data. The “size” of such a family can serve as an indicator of uncertainty whereas spectral estimation algorithms as methods for selecting representative spectra. Modern spectral analysis methods typically rely on second-order statistics [5], and in this context, it is appropriate to pursue the earlier dictum by asking for the description of the family of power spectra consistent with second-order statistics. It turns out that if such statistics consist of the state covariance of a linear filter driven by the process, then the family of consistent power spectra is in bijective correspondence with solutions to an analytic interpolation problem [1–3]. This problem involves positive-real functions and is analogous to the well-known matrix Nehari problem (e.g., encountered in H∞ -control). It is the purpose of this note to address the issue of consistency of state statistics of filters driven by the same stochastic process, as well as, to indicate how to integrate such state statistics in order to determine the family of consistent power spectra for the common input.

2

The structure of state covariances

Let xk be the state process of a linear filter xk = Axk−1 + Buk , ? ??

(1)

This research was supported in part by the NSF and AFOSR. Dept. of Electrical and Computer Engineering, University of Minnesota, MN 55455.

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 207--211, 2003 Springer-Verlag Berlin Heidelberg

208

Tryphon ÿT. Georgiou

driven by a stationary, zero-mean, vector-valued stochastic process uk ∈ Cm , with k ∈ Z, A ∈ Cn×n , B ∈ Cn×m , and A having all its eigenvalues in the open unit disc. The state covariance Σ := E{xk x∗k }, E being the expectation operator, satisfies linear constraints imposed by the system dynamics. In fact, starting from the representation (cf. [4, Ch. 6]) ¶ Z π µ dµ(θ) G(ejθ )∗ , (2) G(ejθ ) Σ= 2π −π where G(λ) = (I − λA)−1 B is the transfer function of the “input-to-state” system (1) and µ the spectral distribution of {uk } (i.e., µ is a Hermitian non-decreasing matrix-valued function on [0, 2π)), it can be shown that Σ − AΣA∗ = BH + H ∗ B ∗ with H=

Z

π −π

µ

1 ∗ B + G(ejθ )ejθ A∗ 2

(3) ¶

dµ(θ) ∈ Cm×n . 2π

Conversely, the solvability of (3) in terms of H, is sufficient for the nonnegative definite matrix Σ to be the state covariance of the linear system (1) for an appropriate input. This is shown in [2,3] under the simplifying assumption that (A, B) be a reachable pair. We point out that this assumption can be readily relaxed—since we will shortly need this more general statement. To see that the reachability assumption can be relaxed, first note that if (A, B) is not reachable then any state covariance Σ is zero on the orthogonal complement of the reachability subspace of (A, B) as it follows from (2). But any non-negative matrix Σ satisfying (3) is automatically zero on the complement of the reachability subspace of (A, B). A suitable “coordinate change” (A, B, Σ) 7→ (T AT −1 , T B, T ΣT ∗ ) can now transform the data conformably into ·

Ac Ac¯c = T AT 0 Ac¯ · ¸ Bc TB = 0 · ¸ Σc 0 T ΣT ∗ = , 0 0 −1

¸

where (Ac , Bc ) is reachable and Σc ≥ 0. Then, the theory in [2,3] allows Σc to be expressed as the state covariance of the reachable subsystem ξi = Ac ξk−1 + Bc uk for a suitable input process {uk }. Reversing the steps, it follows that Σ is the

The mixing of state statistics

209

state covariance of (1) with the same process {uk } as input. We summarize these conclusions in the following proposition (which only differs from [2, Theorem 1] in that the reachability assumption is omitted). Proposition 1. Let A ∈ Cn×n , B ∈ Cn×m , with A having all eigenvalues in D := {λ : |λ| < 1}, and let Σ ∈ Cn×n be a Hermitian nonnegative definite matrix. The matrix Σ is the stationary state covariance of (1) for a suitable input process if and only if (3) is solvable for H ∈ Cm×n .

3

Mixing of state statistics

We now consider the case where the same stochastic process {uk } (of unknown spectral distribution) drives several stable linear filters in parallel, and where state statistics are available in the form of a stationary state covariance for each such filter. Thus, let (i)

(i)

xk = Ai xk−1 + Bi uk ,

(4)

and let Hermitian matrices Σi satisfy Σi ≥ 0 and Σi −

Ai Σi A∗i

= Bi Hi +

(5) Hi∗ Bi∗ ,

(6)

(i)

for i = 1, . . . , `, and xk ∈ Cni , Ai ∈ Cni ×ni , Bi ∈ Cni ×m , with ni ∈ Z+ . If Σi , for i = 1, . . . , ` represent state covariances of each filter individually, then (5-6) are satisfied, as follows from Proposition 1. However, these conditions are not sufficient. Another way of looking at the question of compatibility and consistency between the data (Ai , Bi , Σi ) is to recall from [2,3] that, for each, there is a family of possible spectral distributions (i.e., Hermitian non-negative measures on [0, 2π)) ½ ¾ Z dµ ∗ −jθ ∗ −1 jθ −1 (I − e Ai ) Bi Bi (I − e Ai ) = Σi Mi := dµ : 2π If the same process {uk } is driving all systems then Mi , i = 1, . . . , ` ought to have a non-empty intersection. In order to determine necessary and sufficient conditions, we consider the combined system xk = Axk−1 + Buk where



 (1) xk  (1)   xk   xk =   ..   .  (`) xk

210 and

Tryphon ÿT. Georgiou 

A1  0  A= .  .. 0

0 A2 .. . 0

... ... .. . ...

   B1 0  B2  0     , B =  . . .   .  0 B` A`

This may not be reachable since subsystems may share the same dynamics, but its state covariance Σ ought to satisfy (3) by Proposition 1. Now, since Σi are state covariances of subsystems, it is evident that H is made up of the solutions to (6) for i = 1, . . . , `: ¤ £ H = H1 H2 . . . H ` and nonnegativity of this Σ is the sought condition. We summarize our conclusion in the following proposition. Proposition 2. For i = 1, . . . , `, let Ai ∈ Cni ×ni , Bi ∈ Cni ×m , Σi ∈ Cni ×ni with Ai having all eigenvalues in D := {λ : |λ| < 1} and Σi being Hermitian nonnegative definite matrices. The matrices Σi are (stationary) state covariances of (4) for a suitable common stationary input process if and only if 1. the equations (6) are solvable for Hi ∈ Cm×ni , 2. and, with Σik , 1 ≤ i < k ≤ ` denoting the solutions of Σik − Ai Σik A∗k = Bi Hk + Hi∗ Bk∗ , the matrix 

Σ1 Σ12 ∗  Σ12 Σ2  Σ= . ..  .. . ∗ ∗ Σ`2 Σ`1

... ... .. . ...

(7)

 Σ1` Σ2`   , ...  Σ`

is nonnegative definite. We note that, in the above statement, if Hi (i = 1, . . . , `) exist then equations (7) always have solutions Σik (1 ≤ i < k ≤ `) because the spectra of Ai and A∗k are both contained in the open unit disc and hence the mapping Σik 7→ Σik − Ai Σik A∗k is invertible. Remark 1. It is instructive to consider the special case where Bi = 1 and Ai = zi ∈ C for i = 1, . . . , `. We replace symbols Σ, A, H by their lowercase counterparts to signify that they are scalar quantities. In this case, ¯i hi + h , σi = 1 − ai a ¯i “bar” denoting complex-conjugation, while ¯k hi + h σik = , 1 − ai a ¯k and Σ reduces to the ordinary Pick matrix · ¯ k ¸` hi + h Σ= . 1 − ai a ¯k i,k=1

The mixing of state statistics

211

Remark 2. We finally point out that the intersection of families of solutions to analytic interpolation problems can be trivially treated within the same framework. In fact, the Pick matrix for the combined problem is readily constructed from the Pick matrices of sub-problems, as we just explained. E.g., for the above setting and with ` = 2, besides the nonnegativity of Σi (i = 1, 2) and the solvability of (6) for Hi , in order for a common stochastic input to generate the observed state statistics, the matrix ¸ · Σ1 Σ12 ∗ Σ2 Σ12 ought to be nonnegative definite, where Σ12 − A1 Σ12 A∗2 = B1 H2 + H1∗ B2∗ .

(8)

Seen from a different angle, the spectra µi ∈ Mi for {uk } which are consistent with (Ai , Bi , Σi ) (i = 1, 2) are obtained via radial limits dµi (θ) ∼ lim <(Fi (rejθ )), r%1

of (matrix-valued) “positive real functions” Fi (λ) (see [2,3]) which satisfy certain interpolation condition. In fact, © (9) Fi (λ) ∈ Ci := Hi (I − λAi )−1 B + Qi (λ)Vi (λ) : Qi (λ) analytic in D, and

(10)

Fi (λ) + Fi (λ)∗ ≥ 0 for λ ∈ D}

(11)

−1

where Vi (λ) = Di + λCi (I − λAi ) B is a square inner matrix-valued function, obtained by suitable selection of Ci , Di depending on Ai , Bi (e.g., see [2]). Thus, the intersection of the families Ci (i = 1, 2) of solutions to “Nehari-like” problems can be conveniently merged into one. In fact this intersection is nonempty if and only if ¸ · Σ1 Σ12 ≥0 ∗ Σ2 Σ12 with Σ12 as in (8).

References 1. C. I. Byrnes, T.T. Georgiou, and A. Lindquist, A new approach to spectral estimation: A tunable high-resolution spectral estimator, IEEE Trans. on Signal Proc., 48(11): 3189-3206, November 2000. 2. T.T. Georgiou, The structure of state covariances and its relation to the power spectrum of the input, IEEE Trans. on Automatic Control, 47(7): 1056-1066, July 2002. 3. T.T. Georgiou, Spectral analysis based on the state covariance: the maximum entropy spectrum and linear fractional parameterization, IEEE Trans. on Automatic Control, to appear. 4. P. Masani, Recent trends in multivariate prediction theory, in Multivariate Analysis, P.R. Krishnaiah, Ed., Academic Press, pp. 351-382, 1966. 5. P. Stoica and R. Moses, Introduction to Spectral Analysis, Prentice Hall, 1997.

Controllability, integrability and ergodicity Igor Mezi´c Department of Mechanical and Environmental Engineering and Department of Mathematics, University of California, Santa Barbara, CA 93106-5070, USA

Abstract. Systems preserving a smooth measure on the phase space, such as Hamiltonian systems of classical dynamics or incompressible flows of fluid dynamics attract a lot of interest in control theory. I describe some work on the notion of controllability in systems that are measure-preserving and possess drift. Relationship between controllability, a fundamental concept in control theory, and the concepts of integrability and ergodicity, fundamental in dynamical systems theory is addressed. The basic idea is that studying reccurence (or ergodic) properties of trajectories of the drift is key to establishing necessary and sufficient conditions for controllability in such systems. The benefit of this approach is that controllability proofs contain a constructive procedure for control.

1

Introduction

Controllability (and specifically constructive controllability) of nonlinear systems is a topic of much current interest in control theory. The pillars of existing controllability theory are Lie-algebraic [9,17,16,25,30]. However, there is also a recognition that dynamical analysis of the system may provide conditions that are much weaker than these [21,34]. Ideas in these works are based on a growing body of literature that recognizes the interplay between dynamical systems theory and control as crucial for nonlinear control theory [13,2,8]. In the paper [21] we analyzed controllability theory for group translations (systems with no drift) and found that in the case when the input set is constrained (e.g. it has finite measure), sufficient conditions for controllability can be found that are related to existence of ergodic translations. An example is discrete-time translation on a circle S 1 , θn+1 = θn + un , un ∈ U ⊂ S 1 , where translation by an irrational un = ω is ergodic [29,20]. In this paper we describe a program of research, along with some characteristic results, that studies controllability of systems with drift using coupling of ergodic theory and control theory methods. This program submits that ergodic properties of the drift are central in developing controllability results. The key concept is that of ergodic partition of the drift: partition of the phase space into subsets on which drift is ergodic (roughly speaking, ergodic means that the drift will thoroughly sample the subset, see section 2.3) The resulting control design is intuitively appealing and particularly appropriate when control authority is not large: controller waits for the nominal part of the system (drift) to bring the system to the state in which the system responds maximally to control input. The control is turned on at that moment and

L. Giarré, B. Bamieh (Eds.): Multidiciplinary Research in Control, LNCIS 289, pp. 213--229, 2003 Springer-Verlag Berlin Heidelberg

214

Igor Mezi´c

then turned off again when ”control power” is lost and the system is in another ergodic subset. Drift than takes over and controller waits until the system is again positioned by the drift to the state of maximal responsiveness, and so on. It turns out that rigorous results along these lines can be proven in several interesting contexts. In section 5.2 we provide some results for a general class of measure-preserving systems for which drift is either ergodic on the whole phase space or has a finite number of ergodic components of positive measure (linking the theory to ergodic theory proper as described e.g. in [28] or [20]). Some more specific results, such as an outline of the proof of sufficient and necessary conditions for controllability of 1-degree of freedom and n-degree of freedom Hamiltonian systems with integrable drift are also provided here (with the full technical details provided in [22]), in sections 3.1 and 3.2. Control of discretetime systems whose drift is an integrable twist map - system the perturbations of which have been investigated quite thoroughly in dynamical systems theory (see e.g. [31])- has been studied from the perspective outlined here in [34]. Note that these are all in the context of research on controllability of mechanical systems that has been flourishing recently both from Lagrangian and Hamiltonian perspective [25,8,18,11,6,7,12]. In systems with integrable drift, ergodic partition is easily constructed and is related to integrals of motion, thus putting this part of the theory in the context of geometric mechanics [3,1]. Particularly suitable coordinates exist in this case, called the action-angle coordinates (see section 2.2 and [3]). Another connection with previous results that can be made is with the so-called energy method of Astrom and Furuta [5]. This is a method for control of a pendulum by changing the energy of the pendulum. The extension provided in this work includes controlling other possible integrals of motion - the so-called actions. Returning to the issue that powerful Lie-algebraic methods for controllability exist, it is interesting to ask about the connections and differences of the two approaches. Methods based on ergodic properties of the drift are capable of providing sufficient and necessary conditions for controllability (see section 3.1 and [34]) and thus include most of the Lie-algebraic results. An interesting comparison can be made in the case when drift is integrable (section 3.1). A result that combines both the ergodic properties of the drift (in form of recurrence of trajectories) and Lie algebraic notions is contained in [16].The flatness method for controllability of Hamiltonian systems are discussed in section 4 and it is seen that these crucially depend on the integrability of the control vector fields, as opposed to integrability of the drift (or more generally ergodic properties of the drift). To make the paper to some extent self-contained we start in the next section with the preliminary material on the notions of controllability, integrability and ergodicity.

2 2.1

Preliminaries Controllability

The topic of controllability is fundamental in control theory. Assume we have a continuous-time dynamical system of the form x˙ = f (x, u), x ∈ Rn , u ∈ Rm , m, n positive integers, and we are interested in the following problem:

(1)

Controllability, integrability and ergodicity

215

Problem 1. Given the initial state of the system x(0) = x0 and the final state (state at time T ) x(T ) = xT is there a curve in Rm , u(t) : [0, T ] → Rm such that the system moves from the prescribed initial state to the final state? There are numerous variants of this problem, where the system does not need to move from the exact initial to the exact final state but only from the neighborhood of one to the neighborhood of the other. The theory of controllability (and especially constructive controllability) in nonlinear systems is an immensly active area of research. We will discuss systems that preserve a smooth invariant measure such as Hamiltonian systems, defined by ∂H , ∂pi ∂H p˙ i = − , ∂qi q˙i =

(2)

where i ∈ {1, ..., n}, pi ∈ R, qi ∈ R, H(p1 , ..., pn , q1 , ..., qn , u1 , ..., um ) : R2n × Rm → R a C 1 function. Denote p = (p1 , ..., pn ), q = (q1 , ..., qn ), u = (u1 , ..., um ). Such systems preserve volume in phase space given by the form dV = dq1 ∧ dp1 ...dqn ∧ dpn [3,1]. Nothing that is said here is restricted to Hamiltonian systems with configuration manifold Rn although the developments will for simplicity of presentation be in this context. We show that the notion of controllability for such systems has close links to notions of integrability and ergodicity that are fundamental in dynamical systems theory.

2.2

Integrability

Consider a nominal system of the form (2) obtained by setting K(p, q) = H(p, q, 0), ∂K , ∂pi ∂K p˙ i = − . ∂qi q˙i =

(3)

The system (3) is called integrable if it admits n independent integrals of motion Pi : R2n → R, i = 1, ..., n in involution (see [4] for a more general definition of integrability). The condition that Pi is an integral of motion reads {Pi , K} =

n n n n X X X ∂Pi ∂K X ∂Pi ∂K ∂Pi ∂Pi − = p˙ j + q˙j = 0 ∂q ∂p ∂p ∂qj j ∂pj j ∂qj j j=1 j=1 j=1 j=1

(4)

where {·, ·} is the Poisson bracket of two functions. The independence condition means that functions Pi are such that the 2n × n Jacobian matrix [∂j Pi ] has rank

216

Igor Mezi´c

n. In addition, the involution condition requires that the Poisson brackets of the integrals vanish: {Pi , Pj } = 0. Integrable Hamiltonian systems have very simple structure: their phase (or state) space is foliated by n-dimensional tori Tn , invariant under the dynamics. These tori can be parametrized by angles θi , i = 1, ..., n, such that the motion on the tori is given simply by θ˙i = ωi (I1 , ..., In ), Where I = (I1 , ..., In ) ∈ I n , and I ⊂Rn is a closed interval. We call I’s the action variables. They are canonically conjugate to the angles θ = (θ1 , ..., θn ), and Ii = fi (P1 , ..., Pn ). Consider the specific example of the harmonic oscillator: H(p, q) =

1 2 (p + q 2 ). 2

Let I = H, θ = arctan(p/q) to obtain I˙ = 0, θ˙ = −1, and we see that in this case the function ω(I) = −1 is particularly simple. The motion is depicted in figure 1.

Fig. 1. The action-angle coordinates I, θ for a 1-degree of freedom system. A full description of the action-angle variables can be found e.g. in Arnold’s book [3].

Controllability, integrability and ergodicity

2.3

217

Ergodicity

A good account of notions from ergodic theory that will be needed here is in Petersen’s book [29]. The notion of ergodicity is to a certain extent opposite to that of integrability. Consider a system x˙ = f (x), x ∈ B ⊂ Rn ,

(5)

where B is a bounded subset of Rn , f ∈ C 1 (B). Assume that (5) preserves volume in the state space B (this is equivalent to ∇ · f = 0 on B). Then the definition of ergodicity reads as follows: Definition 1. The system (5) is called ergodic if the only functions invariant under the dynamics of (5) are those that are constant almost everywhere (a.e.) on B. In other words, (5) does not possess any L1 integrals of motion - i.e. integrable functions that are constant on trajectories - except for functions that are constant on B. Thus certainly it does not posses C 1 integrals of motion P : B → R satisfying n X ∂P x˙ j = 0, ∂x j j=1

where x = (x1 , ..., xn ) (cf. equation (4)). We immediately notice that non-trivial time-independent Hamiltonian systems cannot be ergodic on their phase space since they possess at least one integral of motion: the Hamiltonian. The definition of ergodicity given above is equivalent to the the following: Definition 2. The system (5) is ergodic iff the following holds for almost every x0 ∈ B : for any L1 function g : B → R the time average of g along trajectory of (5) starting at x0 , denoted g ∗ (x0 ), is equal to the average value of g on B, denoted by g¯ : g ∗ (x0 ) = lim

t→∞

1 t

Z

t 0

g(x(tˆ, x0 ))dtˆ =

1 V (B)

Z g(x)dV (x) = g¯.

From this statement it becomes clear that ergodic systems have the property that starting from almost any initial condition in the state space B they sample the different regions of B thoroughly. From this interpretation it follows that an intuitive connection of this property of drift with the notion of controllability of the associated control system x˙ = f (x) +

m X

ui gi (x),

i=1

should be present. Despite the fact that autonomous Hamiltonian system cannot be ergodic on their state space due to the fact that inergy is an integral of motion that foliates the phase space into subsets of codimension 1, they could be ergodic on some subsets of codimension 1 or more. For example, the restriction of the harmonic oscillator to its energy surface is ergodic (see section 3.1) - although this is not generally true for Hamiltonian systems of more than one degree of freedom.

218

3

Igor Mezi´c

Hamiltonian systems in action-angle variables

In this section we consider control systems of the following form: I˙i =

m X

uj fji (I, θ),

j=1

θ˙i = ωi (I) +

m X

uj gji (I, θ),

(6)

j=1

where i = 1, ..., n, I ∈ I n , θ ∈ Tn and the functions fji , gji are C 1 . As in the introduction, ¡ we call Ii ’s actions and ¢θi ’s angles. The control vector fields are denoted by vj = fj1 , fj2 , ..., fjn , gj1 , gj2¡, ..., gjn , j = 1, ¢ ..., m. The action part of the control vector field is¡ denoted by ¢fj = fj1 , fj2 , ..., fjn , j = 1, ..., m. The angle part is denoted by gj = gj1 , gj2 , ..., gjn , j = 1, ..., m. We also denote the matrix F = [f1 , ..., fm ] . as described in 2.2 the drift part of (6) can arise as a consequence of transformation to action-angle coordinates of an integrable Hamiltonian system of n degrees of freedom. However, we do not request the control vector fields to be Hamiltonian here.

3.1

Single degree of freedom

To set the stage consider the case m = n = 1, I˙ = uf (I, θ), θ˙ = ω(I) + ug(I, θ),

(7)

In this case the following result holds: Result 1. A system of the form (7) with bounded input¯ u ∈ [−k,¯k] is controllable if ¯ ¯ ˜ ˜ and only if for every I ∈ I there is θ(I) ∈ S 1 such that ¯f (I, θ(I)) ¯ > c 6= 0. Proof. To prove this, assume first that there is an I ∗ such that f (I ∗ , θ∗ (I ∗ )) = 0. Then I = I ∗ is an invariant manifold for the control system (6) and ¯the system¯ is ¯ ¯ ˜ ˜ not controllable. Now assume that for every I there is θ(I) such that ¯f (I, θ(I)) ¯> c= 6 0. It is enough to show that for every I there is an interval of actions J such that |I − J| ≤ ² that can be reached from I, where ² does not depend on I. Then a chain of control neighborhoods in ”action space” can be formed, as depicted in figure 2.

Fig. 2. Control mechanism in action space.

Controllability, integrability and ergodicity

219

Let θ∗ (I) be the angle at which maxθ |f (I, θ)| is reached. This maximum is not zero by assumption. The control proceeds as follows: starting from some (I0 , θ0 ), the system rotates in θ under nominal dynamics and stays on I0 until it reaches θ∗ (I). At this point a constant control u = k · sgn(f (I, θ)) is applied. It can be shown that there is an ε > 0 such that with this control, every action inside the interval [I0, I0 +ε] is reachable (for necessary estimates and slightly stronger results see [22]). Note thatRthe nominal system 7 is ergodic on each curve for which the HamiltoI nian H(I) = 0 ω(J)dJ is constant. To see this, note that the time-average of any function h : I×S 1 → R Z 1 t h∗ (I0 , θ0 ) = lim h(I(tˆ, I0 , θ0 ), θ(tˆ, I0 , θ0 ))dtˆ = t→∞ t 0 Z Z 1 t 1 T h(I0 , θ0 + ω(I0 )tˆ)dtˆ = h(I0 , θ0 + ω(I0 )tˆ)dtˆ = = lim t→∞ t 0 T 0 Z 2π 1 = h(I0 , θ)dθ. 2π 0 where the last equation was obtained by the change of variables θ = tˆ/ω(I0 ). So control is used to transfer motion between ergodic curves of constant Hamiltonian. A natural question to ask here is how does the above controllability result compare with the Lie-algebraic theory (see e.g. [16]). The system 7 has a recurrent drift (see [16], section 4.6) given by the vector field X0 = (0, ω(I)), and control vector field X1 = (f, g). Assume that X1 = 0 on I × (θ0 , θ1 ). In this case the sufficient conditions for controllability based on Lie-algebraic notions (Theorem 5. in 4.6 of [16]) fail. However, the system is still controllable. Notice that the above described result on controllability is constructive. In fact, it is conceptually closely linked with the so-called energy control design proposed for swing-up of a pendulum problem by Astrom and Furuta [5] where Lyapunov method is used. In fact, in a 1-degree of freedom system the energy E = E(I) is a function of action. The question arises naturally of generalization of such ideas to N degree of freedom systems where perhaps energy is not the only quantity that needs to be controlled. Such an extension is necessary for example in the problems of quantum control [33]. We take the perspective that energy is just one of the possible integrals of motion of the unperturbed (nominal) system. It turns out that considering control that is transverse to surfaces that are joint level sets of integrals of motion yields the appropriate generalization to N -degree of freedom systems. We present this in the next subsection.

3.2

N degrees of freedom

For more than one degree of freedom, the results are necessarily more complicated. The construction of the control input presented in the previous section used the fact that for a single degree of freedom system on every circle I = const. the motion

220

Igor Mezi´c

induced by the drift is ergodic. This is certainly not the case for N-degree of freedom case. In fact the nominal system I˙i = 0, θ˙i = ωi (I),

(8) (9)

is such that the motion evolves on tori of dimension N . In the case when there are no integers ki , i = 1, ..., N such that N X

ˆ = 0, ki ωi (I)

(10)

i=1

every trajectory on the torus Iˆ is dense, i.e. passes arbitrarily close to any point ˆ θ) (to keep an example in mind, note that in the case N = 2 this means that the (I, ratio ω1 /ω2 is irrational and trajectories wind around densely on a two-dimensional torus). We will call such a torus an irrational torus. Points on N -dimensional irrational tori that satisfy condition (10) forms a set of measure 1 in the phase space. The system dynamics restricted to one of these tori is ergodic on it [29]. The rest of the tori are filled with N − M dimensional tori on which the dynamics is dense, where M is the number of independentP sets kij , i = 1, ..., N, j = 1, ..., M of integers N j ˆ ˆ such that ωi (I) on that torus satisfy i=1 ki ωi (I) = 0. Exact controllability in this case can be achieved using additional assumptions that none of these tori of dimension lower than N is an invariant control set (for definition see [13]). For an example of such conditions in the case of discrete time, area-preserving systems in 2 dimensions, see Vaidya and Mezi´c [34]. The weaker notion of controllability almost everywhere (controllability a.e.) can be proven more easily. We first define it: Definition 3. We say that a system x˙ = f (x, u), x ∈ M, u ∈ N, where M ⊂ Rm and N ⊂ Rn , is controllable almost everywhere (controllable a.e.) if for almost every initial x0 and almost every final xf , there is a control u(t), 0 ≤ t ≤ T that takes x0 to xf . The final time T can depend on the initial and final state. Remark 1. Note that the notion of almost everywhere is taken here in the sense of ”everywhere except possibly on a set of zero volume”. The definition can be easily extended for maps on abstract measure spaces. If we can show that transition from an initial irrational torus to a final irrational torus is possible, then the system (6) is controllable since on an irrational torus the nominal dynamics will take a trajectory arbitrarily close to any desired point. The key to controllability is then transition from some initial irrational torus to a set of tori that are within a box of size ε. The following result is obtained: Result 2. Let m = n¯ and for each irrational torus I there is an angle θ˜ such that ¯ ¯ ¯ ˜ 6 0. Then the Hamiltonian system 6 is controllable a.e. ¯>c= ¯det(f1 , ..., fn )(I, θ(I)) The proof consists of observing that on any irrational torus I the nominal dynamics will bring the system arbitrarily close to θ∗ (I) - an angle at which maxθ |det(f1 , ..., fn )(I, θ)| is reached. This maximum is not zero by assumption. The control is then turned on in such a way that (F u) = (c, 0, 0, ..., 0) and the

Controllability, integrability and ergodicity

221

system lands on another irrational torus with larger I1 . This is repeated until the system reaches the desired final value of I1 . The procedure is now repeated keeping the changes of all the angles except for I2 zero and so on until all the actions are adjusted to their correct values. Finally the nominal angle dynamics is allowed to take the system arbitrarily close to the desired angle with actions being held constant. The complete proof can be found in [22]. The key observation again (just like in one-dimensional case) stems from the fact that nominal dynamics is ergodic when restricted to the tori with dense winding, which in turn fill up a set of measure 1.

4

Controllability by flatness

In the spirit integrability properties of the so-called

4.1

of the general theme of the paper that has to do with interplay of properties of the drift (examined in this section) and integrability the control vector fields, it is worthwhile to examine the case in which flatness property [25,15] of the control vector fields is exploited.

Single degree of freedom

Let us contrast the above developments with another technique used for constructive controllability in Hamiltonian systems: flatness (for a simple example, see e.g. [26]). In the case of a single degree of freedom system (7) we can write ∂Hc I˙ = u (I, θ), ∂θ ∂Hc θ˙ = ω(I) − u (I, θ), ∂I

(11)

since the transformation to action-angle coordinates is canonical. In (11) Hc is the control Hamiltonian, such that f = ∂Hc /∂θ, g = −∂Hc /∂I. Now consider the change of variables z1 = Hc (I, θ), z2 = H˙ c (I, θ) ∂Hc ∂Hc ∂Hc + ∂I ∂θ ∂θ ∂Hc = ω(I). ∂θ =u

µ ω(I) − u

∂Hc ∂I

¶

Note that Hc is an integral of motion for the control vector field (cf. with the fact that I is an integral of motion for the nominal dynamics). We assume that the transformation of coordinates is nondegenerate. The evolution in the new coordinates is given by z˙1 = z2 , d z˙2 = dt

µ

∂Hc ∂θ

¶

∂Hc ω(I) + ω(I) ˙ ∂θ Ã ! µ ¶2 dω ∂ 2 Hc ∂ 2 Hc ∂Hc ∂Hc ∂ 2 Hc ∂Hc 2 = ω (I) +u ω + −ω ∂θ2 ∂θ∂I ∂θ ∂θ dI ∂θ2 ∂I

222

Igor Mezi´c

Consider the problem of getting from (I0 , θ0 ) to (If , θf ) or equivalently from (z10, z20) = (Hc (I0 , θ0 ), ∂Hc /∂θ(I0 , θ0 )·ω(I0 )) to (z1f , z2f ) = (Hc (If , θf ), ∂Hc /∂θ(If , θf )·ω(If)). By specifying a function z1 (t), 0 ≤ t ≤ T such that z˙1 (0) = z20 , z˙1 (T ) = z2f , and setting 2

u(t) = ³

z¨1 (t) − ω 2 (I(z1 (t), z2 (t)) ∂∂θH2c (I(z1 (t), z2 (t)), θ(z1 (t), z2 (t))) ´ , ¡ c ¢2 dω 2 2 Hc ∂Hc c (I(z1 (t), z2 (t)), θ(z1 (t), z2 (t))) ω ∂∂θ∂I + ∂H − ω ∂∂θH2c ∂H ∂θ ∂θ dI ∂I

the desired controllability is achieved provided Ã

∂ 2 Hc ∂Hc ω + ∂θ∂I ∂θ

µ

∂Hc ∂θ

¶2

dω ∂ 2 Hc ∂Hc −ω dI ∂θ2 ∂I

! 6 0. (I(z1 (t), z2 (t)), θ(z1 (t), z2 (t))) = (12)

The condition 12 is typically not satisfied along a 1-dimensional manifold on the domain. It is possible to overcome this problem by bringing the system close to that manifold and then using the nominal dynamics to go across the curve [35], but the control u typically becomes large in the process, and it is not possible to achieve controllability with bounded control.

4.2

N -degrees of freedom

Let us consider the case of a general N -degrees of freedom Hamiltonian systems with N actuating vector fields: ∂H X + uj Hj , ∂pi j=1 N

q˙i =

∂H X uj Hj , − ∂qi j=1 N

p˙ i = −

(13)

We call the Hamiltonian vector field corresponding to Hj control vector field. In this case, to achieve controllability using flattness, N coordinates z2i−1 , i = 1, ..., N have to be found such that z˙2i−1 = z2i , z˙2i = fi +

N X

aj gij .

j=1

Calculating the time evolution of z2i−1 , we obtain z˙2i−1 = +

N X ∂z2i−1 ∂H ∂z2i−1 ∂H − ∂ q ∂ pj ∂ qj j ∂ pj j=1 N X k=1

uk

N X ∂z2i−1 ∂Hk ∂z2i−1 ∂Hk − . ∂ q ∂ p ∂ pj ∂ qjj j j j=1

(14)

Controllability, integrability and ergodicity

223

Examining the terms multiplying ak in the above expression, we conclude that if we find n independent functions Li , i = 1, ..., n which are integrals of motion simultaneously for the Hamiltonian vector fields generated by Hamiltonians Hk , k = 1, ..., n, then the equations of motion acquire the required form (14) with z2i−1 = Li , i = 1, ..., n and z2i =

N X ∂Li ∂H ∂Li ∂H − = {Li , H}. ∂ q ∂ pj ∂ qj j ∂ pj j=1

From z˙2i = {{Li , H}, H} +

N X

uk {{Li , H}, Hk }.

k=1

it is now clear that a sufficient condition for controllability is that the matrix H 6 0. These whose components are given by Hik = {{Li , H}, Hk } is such that det H = expressions in action-angle coordinates are complex to even write down, let alone to check.

5

Ergodicity and controllability

The developments in previous sections dealt with systems in which drift had a very simple ergodic decomposition into tori of dimension less than n - the number of degrees of freedom. In this section we first consider another example of this kind (where drift foliates the phase space into periodic orbits), but with control which is even more restricted than that considered in previous sections: impulsive control. In this case results that employ ergodicity of discrete-time systems in study of controllability [21] can be employed. After that we consider systems on the opposite side of the spectrum - those that have drift which is ergodic on the phase space or splits the phase space into a finite number of positive measure sets on which the dynamics is ergodic.

5.1

Flows on tori with impulsive control

In this section we consider translation flow on a 2−dimensional torus, given by θ˙1 = 1, θ˙2 = v,

(15) 1

where θi ∈ S , v ∈ V ⊂ R. The problem we consider is characterizing the set V such that (15) is controllable. Consider the control strategy in which v is equal to uδ(0) where δ is the Dirac delta-function (impulsive control [36,32]). We obtain θ2 (2π) = θ2 (0) + u.

(16)

Note that (16) is the Poincare map of (15) at θ1 = 0. Thus it is a map from a circle to itself. The inputs u can now be considered as elements of a set U ⊂ S 1 . Let µ be the Haar (or equivalently, Lebesgue) measure on S 1 . For such maps, the following result holds (see [21] where the results are proven for general compact Abelian groups):

224

Igor Mezi´c

Result 3. Assume that µ(U ) > 0. Then the map (16) is controllable. To see how ergodicity plays a role in proving this statement, consider figure 3.

Fig. 3. Schematic for the proof of ergodicity for the map (16). From the initial point, that without loss of generality can be taken as 0, the system can reach the set U . The goal is to reach some final point θ. Now pick ω ∈ U irrational and consider the translation of U by ω. Since translation by ω is an ergodic dynamical system [29], it is true that for any two positive measure sets A, B, lim

n→∞

n−1 1X µ((i · ω + A) ∩ B) = µ(A)µ(B), n i=0

And thus translation of U under ω will intersect θ − U after a finite number of steps. This is enough for controllability. Now, the statement for maps in turns implies controllability for (15) under assumption that `(V ) > 0, where ` is the Lebesgue measure on R. Note the key step in the proof of the above statement: for exact controllability it is enough to be able to reach a positive measure set in forward and backward time. This leads to much more general results for systems with ergodic drifts as we discuss next.

5.2

Measure-preserving systems with ergodic drift

We turn attention now to a general system of the form x˙ = f (x) + ug(x),

(17)

Controllability, integrability and ergodicity

225

where x ∈ M a compact Riemannian manifold endowed with a smooth probabilistic measure µ on the Borel σ-algebra of sets. We have the following: Proposition 1. Assume µ is an ergodic invariant measure for f. Assume also that the set reachable in forward and backward time from any x ∈ M is of positive measure. Then 17 is controllable. Proof. To prove this, let x0 and xf be the initial and the final point of the evolution. We know that there is a positive measure set A that can be reached from x0 and a positive measure set B that xf can be reached from. We also know that the drift part of the system is ergodic on M and thus, for any two sets of positive measure Z 1 t ¯ µ(Φtf (A) ∩ B)dt¯ = µ(A) · µ(B) > 0, (18) lim t→∞ t 0 ¯

where Φtf : M → M is the (measure-preserving) flow generated by f. Thus, there is a trajectory of x˙ = f that starts at xa in A and lands at xb in B (in fact condition 18 implies more: that a positive measure of initial conditions in A land in B under ¯ evolution of Φtf ). The controllability is now clear as xa can be reached from x0 and xf can be reached from xb . It is clear that the concept of controllability almost everywhere is implied by the above statement, but it is not true that the drift-only system will provide it. However, it is true that using drift only, the system can go from an arbitrary small neighborhood of x0 to an arbitrarily small neighborhood of xf . There are many examples of drifts f that are ergodic such as constant flows on n-tori given by θ˙i = ωi , i = 1, ...n where θi ∈ S 1 P and the constant frequencies ωi are such that for any set of integers 6 0 [20,29] (cf. section 3.2). Another interesting class are ki , i = 1, ..., n n i=1 ωi ki = systems that preserve a smooth measure on a C 2 Riemannian manifold, that under certain hyperbolicity assumptions [20,28] have the property that the phase space is partitioned into a finite number of positive measure sets on which the dynamics is ergodic. Let f be a drift vector field on M such that it admits a partition of the phase space into a finite number of components Ci , i = 1, ..., m on which the dynamics is ergodic. We call such f systems with finite ergodic decomposition. Let P = [pij ] be an m×m matrix of 0’s and 1’s such that pij = 1 if there is an admissible control u(t), t ∈ [0, T ] ⊂ R that connects the components Ci and Cj via trajectories that start at some Si ⊂ Ci such that µ(S 1 ) > 0, and end at Sj ⊂ Cj . Entries of the matrix Pk will be denoted pkij . We have the following result: Result 4. Consider a smooth measure-preserving system x˙ = f (x) + ug(x) where the drift f has a finite ergodic decomposition with m components. Assume that the set reachable in forward and backward time from any x ∈ M is of positive measure. 6 0 for some k ∈ {1, ..., m} for every i, j. Then the system is controllable a.e. if pkij =

226

Igor Mezi´c

Proof. Let x0 ∈ Ci and xf ∈ Cj be two arbitrary points in M. By assumption, there is a positive measure set A that is reachable from x0 and a positive measure set B that xf is reachable from. Let Ak = A∩ Ck and Bl = B∩ Cl be of positive measure. By the assumption on P there is an admissible control u(t), t ∈ [0, T ] that Φ0−T (S¢k ) ⊂ Cl . By takes a positive measure subset Sk ⊂ Ck to a positive measure u ¡ t + 1 ergodicity on component Ck , there is t1 ∈ R such that µ Φ (Ak ) ∩ Sk 6= 0. Also, ¡ ¢ µ Φ0−T (Φt1 (Ak ) ∩ Sk ) 6= 0. By ergodicity on component Cl there is also t2 ∈ R+ u ¡ t ¢ ¡ t ¢ Φ 1 (Ak ) ∩ Sk ) ∩ Bl = 6 0. This proves controllability a.e. such that µ Φ 2 ◦ Φ0−T u (but not exact controllability since the union of ergodic components is of measure 1 but does not necessarily contain the whole M ). The above result reveals the key to controllability results for systems with drifts: control needs to provide connections between different ergodic components. General volume-preserving systems can have quite a complicated structure of the ergodic partition. Consider for example the area-preserving discrete-time system called the standard map, given by xn+1 = xn + y n + ² sin(2πxn ), y n+1 = y n + ² sin(2πxn ). where x, y are defined mod 1. In figure 4 a visualization of invariant sets of this map for ² = 0.3 is done based on the methods of ergodic partition [23] in a paralel computation study [19]. In this figure the complexity of invariant sets such as the big chaotic region shown in red and periodic islands shown in light blue reveals the possible complications with determining effective methods for control of systems with drifts in the mixed (neither integrable nor ergodic) regime.

6

Discussion and conclusions

The key concept discussed here is that of the importance of the ergodic partition of the phase space of a control dynamical system under the drift field for control. In examples such as that of a single-degree of freedom Hamiltonian system in actionangle coordinates, properties of the ergodic partition can easily lead to necessary and sufficient conditions for controllability that are much weaker than the sufficient conditions based on Lie-algebraic structure. To a certain extent, ideas presented here can serve as a generalization for a number of approaches to controllability of nonlinear systems with drift such as the energy method of Astrom and Furuta [5] and methods for systems with recurrent drifts developed by Jurdjevic [16]. The connection also exists to the basic idea of the so-called control of chaos theory [27] where the ergodic properties of the attractor are used to bring the system close to the desired location. It was pointed out to the author a while ago by Petar Kokotovi´c that the key merit of the control of chaos approach is that most of the control is achieved with no control effort. The paper presented here goes towards extension of the basic control of chaos technology to systems that are not ergodic to start with but also reveals that the basic underlying idea of control of chaos is key to controllability of systems with drift. In this sense,

Controllability, integrability and ergodicity

227

Fig. 4. Invariant sets of the standard map are visualized by color contour plot. we tried to fulfill the charge of section 3 in the review article of Nijmeijer and Schumaker [24] in section 5.2 to connect the control of chaos literature with control theory. To this note, the reader will find instructive the comparison of the key idea in Astrom and Furuta [5] with the key paper in the control of chaos literature by Ott et al. [27] and then re-consider the ideas in this paper. Another review of the current issues in control theory by Brockett ([10], section 3.2) reduces the role of chaotic (ergodic) dynamics to amplification of control inputs . Hopefully the current work provides another interesting role for complicated internal dynamics of systems. One of the benefits of the approach to controllability based on ergodic properties of the drift is that the key steps fundamentally do not require any differentiable structure - despite the fact that we used it in the current paper for various technical points. Non-smooth systems and discrete time systems [21] can be treated based on the same general principle: the property key to controllability is to transfer between different components of the ergodic partition of the drift part of the system

7

Epilogue

It is only appropriate for me to present this work, that is partly an overview of ideas connecting ergodic theory and control theory, in the volume dedicated to scientific life of my friend and senior colleague at University of California, Santa Barbara, Professor Mohammed Dahleh. For it was his vision and enthusiasm for the necessity of communication between the two fields that brought me to Santa

228

Igor Mezi´c

Barbara in the first place, and it was our collaboration that taught me many of the concepts and ideas of control theory. We pursued connections of ergodic theory and control theory together [14] and spent countless hours discussing dynamical systems and control theory, among many other topics. It was always great to hear what Mohammed’s opinion is on any given subject - always presented to us with caracteristic flare and inherent kindness to ideas of others...I’ll stay eternally sad that I will not get his word on this piece. And this is the least I will miss him for. For my own sake I wish this was the closure but it is not. For sadness of this kind does not converge to zero within the open set that our lifetime is. Acknowledgements I would like to thank Jerry Marsden for directing my attention away from exact controllability. The notion of controllability almost everywhere came out of that suggestion, reflecting my measure-theoretic bent. I am thankful to Domenico D’Alessandro for useful suggestions. Umesh Vaidya and Dong-Eui Chang read the manuscript and gave me useful comments.

References 1. Abraham and J. E. Marsden, Foundations of mechanics, Addison-Wesley, New York, 1987. 2. F. Albertini and E. Sontag, Some connections between chaotic dynamical systems and control systems, SYCON-90-13 Report, Rutgers Center for Systems and Control, 1990. 3. V. I. Arnold, Mathematical methods of classical mechanics, New York, 1978. 4. V. I. Arnold, V. V. Kozlov, and A. I. neishtadt, Mathematical aspects of classical and celestial mechanics, Springer-Verlag, New York, 1997. 5. K. J. Astrom and K. Furuta, Swinging up a pendulum by energy control, Automatica 36 (2000), 287–295. 6. J. Baillieul and J. C. Willems (eds.), Mathematical control theory, SpringerVerlag, New York, 1998, Chapter 9, by J. Baillieul, ”The Geometry of Controlled Mechanical Systems”. 7. A. M. Bloch, D. E. Chang, N. E. Leonard, and J. E. Marsden, Controlled Lagrangians and the stabilization of mechanical systems II: Potential shaping, IEEE Transactions on Automatic Control 46 (2001), 1556–1572. 8. A.M. Bloch, P.S. Krishnaprasad, J.E. Marsden, and Dealvarez, Stabilization of rigid body dynamics by internal and external torque, 28 (1992), 745–756. 9. R. Brockett, System theory on group manifolds and coset spaces, SIAM Journal on Control 10 (1972), 265–284. 10. R. Brockett, New issues in the mathematics of control, In ”Mathematics Unlimited -2001 and Beyond”, Springer-Verlag (2001). 11. F. Bullo, Averaging and vibrational control of mechanical systems, SIAM Journal on Control and Optimization 41 (1999), 542–562. 12. D.E. Chang, A. M. Bloch, N. E. Leonard, J. E. Marsden, and C. A. Woolsey, The equivalence of controlled Lagrangian and controlled Hamiltonian systems, ESAIM: Control, Optimisation and Calculus of Variations 8 (2002), 393–422. 13. F. Colonius and W. Kliemann, The dynamics of control, Birkhauser, Boston, 2000. 14. D. D’Alessandro, M. Dahleh, and I. Mezi´c, Control of mixing in fluid flow: A maximum entropy approach, IEEE Transactions on Automatic Control 44 (1999), 1852–1863.

Controllability, integrability and ergodicity

229

15. M. Flies, J. Levine, P. Martin, and P. Rouchon, On differentially flat nonlinearsystems, 315 (1992), 619–624. 16. V. Jurdjevic, Geometric control theory, Cambridge University Press, Cambridge, 1997. 17. V. Jurdjevic and H. Sussmann, Control systems on Lie groups, Journal of Differential Equations (1972), 313–329. 18. N. E. Leonard and P. S. Krishnaprasad, Motion control of drift-free leftinvariant systems on Lie groups, IEEE Transactions on Automatic Control 40 (1995), 1539–1554. 19. Z. Levnaji´c, Visualization of dynamical systems using spectral properties of the koopman operator, Master’s thesis, Universita degli studi di Trieste, 2002, Research pursued at UCSB 2001-2002. 20. R. Mane, Ergodic theory and differentiable dynamics, Springer-Verlag, New York, 1983. 21. I. Mezi´c, Controllability of group translation, Proceedings of American Control Conference, 2001. 22. I. Mezi´c, Controllability of continuous-time nonlinear systems in action-angle coordinates, In preparation, 2002. 23. I. Mezi´c and S. Wiggins, A method for visualization of invariant sets of dynamical systems based on the ergodic partition, CHAOS 9 (1999), 213–218. 24. H. Nijmeijer and J. M. Schumaker, Four decades of mathematical system theory, University of Groningen Press (1999), 73–83, In ”The mathematics of systems and control: From inteligent control to behavioral systems” eds. J. W. Polderman and H. L. Trentelman. 25. H. Nijmeijer and A.J. van der Schaft, Nonlinear dynamical control systems, Springer–Verlag, New York, 1990. 26. B. R. Noack, A. Banaszuk, and I. Mezi´c, Controlling vortex motion and chaotic advection, Proceedings of the CDC, Sydney 2000, 2000. 27. E. Ott, C. Grebogi, and J. A. Yorke, Controlling chaos, Physical Review Letters 64 (1990), 1196–1199. 28. Ya. B. Pesin, Characteristic Lyapunov exponents and smooth ergodic theory, Russian math. Surveys 32 (1977), 55–114. 29. K. Petersen, Ergodic theory, Cambridge University Press, Cambridge, 1995. 30. S. Sastry, Nonlinear systems, Springer-Verlag, 1999. 31. C.L. Siegel and J.K. Moser, Lectures on celestial mechanics, Springer Verlag, Berlin, Heidelberg, 1971. 32. G. N. Silva and R. B. Vinter, Necessary conditions for optimal impulsive control problems, SIAM J. Control Optimization 35 (1997), 1829–1846. 33. U. G. Vaidya, D. D’Alessandro, and I. Mezi´c, Control of Heisenberg spin systems; Action control and lie algebraic decompositions, In preparation, 2002. 34. U. G. Vaidya and I. Mezi´c, Controllability for a class of area-preserving twist maps, Physica D (2002), Submitted. 35. D. Vainchtein and I. Mezi´c, Control of vortex elements: flat coordinates vs. adiabatic control, Submitted to Workshop on Control of Hamiltonian Systems, Seville, Spain, 2002. 36. T. Yang, Impulsive control, IEEE Transactions on Automatic Control 44 (1999), 1081–1083.