Modelling, State Observation and Diagnosis of Quantised Systems (Lecture Notes in Control and Information Sciences)

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari

282

Springer Berlin Heidelberg NewYork Hong Kong London Milan Paris Tokyo

Jochen Schr¨oder

Modelling, State Observation and Diagnosis of Quantised Systems With 125 Figures

13

Series Advisory Board A. Bensoussan · P. Fleming · M.J. Grimble · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · J.N. Tsitsiklis

Author Jochen Schr¨oder BMW Group 80788 M¨unchen Germany

Cataloging-in-Publication Data applied for Die Deutsche Bibliothek – CIP-Einheitsaufnahme Modelling, state observation and diagnosis of quantised systems / Jochen Schr¨oder. - Berlin ; Heidelberg ; New York ; Hong Kong ; London ; Milan ; Paris ; Tokyo : Springer, 2003 (Lecture notes in control and information sciences ; 282) (Engineering online library) ISBN 3-540-44075-5

ISBN 3-540-44075-5

Springer-Verlag Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution act under German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science + Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2003 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Digital data supplied by author. Data-conversion by PTP-Berlin, Stefan Sossna e.K. Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper SPIN 10889927 62/3020Rw - 5 4 3 2 1 0

Preface

Ongoing advances in science and engineering enable mankind to design and operate increasingly sophisticated systems. Both their design and operation require the understanding of the system and its interaction with the environment. This necessitates the formalisation of the knowledge about the system by models. A major issue is what kind of model is best suited for a given task. This book is about the supervision of continuous dynamical systems. Such systems are typically described by differential equations. However, this does not automatically mean that differential equations are proper models for solving supervision tasks. Instead, this book and recent approaches in literature show that supervision tasks do in general not require the use of such precise models as differential equations. This is of interest because uncertainties, typically occurring in supervision, make the use of precise models very difficult. Alternative approaches therefore use less precise models such as discrete– event descriptions to solve supervision tasks on a higher level of abstraction. Discrete–event descriptions in form of automata are one of the key elements of this book. To reach this higher level of abstraction, uncertainties by quantisation are introduced on purpose, taking into account a loss of precision. This is one of the main difference to other approaches. When using numerical models like transfer functions or differential equations, uncertainties make the analysis more difficult. Not so here, where the system is described on a qualitative level on which uncertainties are naturally incorporated. The book presents a new way to describe systems for supervision. Preparing this book I learned that the key to solve supervision problems is simplicity. The main difficulty is to find the right compromise between including and neglecting details. The experience shows that neglecting dynamics and causality by restricting models to a set of rules and purely qualitative assessments of the system is often one step too far because the dynamical properties of a system have to represented in sufficient detail. On the other hand numerical models are usually far to complicated if uncertainties are taken into account. The book suggests a compromise: the quantised systems approach. Quantisation of signals leads the way towards simplicity, from continuous– variable to discrete signals and systems. Accordingly, the theory on quantised

VI

Preface

system has to combine continuous and discrete systems theories. Thus, the book serves also as a bridge between these two theories taking a step towards closing the gap between them. Furthermore, as the quantised system is a special class of hybrid systems, the book is an interesting element also for this line of current research. Besides a profund study on quantised systems and a collection of new results in this field, the book includes many new results in automata theory, a field of discrete–event systems theory. The complete solutions to the state observation and fault diagnostic problems for stochastic automata and automata networks are firstly presented here in form of a monograph. The results are accompanied by new studies on the observability and diagnosability of such systems. However, the book is not only dedicated to a modern theory but due to my engineering background also has a focus on practical applications. The application of the methods to a chemical plant is described in detail. Further applications are outlined at the end of the book. Many examples throughout the text illustrate the theoretical concepts. A two–tank system is used as running example more than 20 times to help the reader understanding the results and to give an idea of what the theory is for. I wrote the present book in 2001 at the Technical University Hamburg– Harburg. It contains results of almost five years research on quantised systems kindly supported by the Deutsche Forschungsgemeinschaft. My deepest thanks go to my mentor Prof. Dr.–Ing. Jan Lunze. Without his guidance and support this book would definitely not exist. His exceptionally straight way of thinking inspired me in many discussions that we had at the Institute of Control Engineering. My thanks also go to Prof. Dr.–Ing. habil. Wolfgang Schwarz and Prof. Luca Console for their comments on the manuscript. Furthermore, I want to thank my colleagues and the staff of the institute. Among them, my special thanks go to Dr.–Ing. Gerwald Lichtenberg, Dr.–Ing. Bernhard Nixdorf and Dr.–Ing. Frank Schiller. I recall with great delight all the interesting, and sometimes controversial, discussions we had on diagnosis, qualitative modelling and the question whether periodical or event–based sampling is preferable. I also want to thank my students, in particular Dipl.– Ing. Lars Karweck and M. Sc. Ahmet Kayihan Kiryaman for their help. My special thanks go to M. Sc. Petek Eginlioglu for making the QuaMo–Toolbox to an efficient and reliable tool for my work. Finally I want to thank my family and in particular my wife for their unending support. Munich, June 2002

Jochen Schr¨ oder

Contents

1.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Motivation and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Problem Statement and Way of Solution . . . . . . . . . . . . . . . . . . 6 1.3 Structure of the Book and Guidelines for Reading . . . . . . . . . . 8 1.4 Running Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.

Basics of Stochastic Automata Theory . . . . . . . . . . . . . . . . . . . . 2.1 Stochastic Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Stochastic Processes and Markov Chains . . . . . . . . . . . . . . . . . . 2.3 Behaviour of Stochastic Automata . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Stochastic Automata for Fault Diagnosis . . . . . . . . . . . . . . . . . . 2.5 Properties of Stochastic Automata . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Classification of Stochastic Automata . . . . . . . . . . . . . . . 2.5.2 Classification of Automaton States* . . . . . . . . . . . . . . . . 2.5.3 Stationary Behaviour of Stochastic Automata* . . . . . . . 2.6 Stochastic Operators** . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13 13 16 19 23 24 24 26 27 33 34

3.

Modelling of Quantised Systems by Stochastic Automata . 3.1 Summary of the Modelling Approach . . . . . . . . . . . . . . . . . . . . . 3.1.1 Quantised Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Nondeterminism of the Quantised System Behaviour . . 3.1.3 Modelling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.4 Abstraction of Quantised Systems to Stochastic Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.5 Complexity of Qualitative Models . . . . . . . . . . . . . . . . . . 3.2 An Extended Introduction to Quantised Systems . . . . . . . . . . . 3.2.1 Definition of Quantised Systems . . . . . . . . . . . . . . . . . . . . 3.2.2 Interpretation of Quantised Systems as Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Representation Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 Quantised Systems for Fault Diagnosis . . . . . . . . . . . . . .

37 37 37 40 41 42 45 46 46 49 50 51

VIII

Contents

3.3 Further Aspects of Quantised Systems* . . . . . . . . . . . . . . . . . . . 3.3.1 Other Abstractions of Numerical Signals* . . . . . . . . . . . 3.3.2 Temporally Quantised Continuous–Time Systems* . . . . 3.3.3 Extension to Discrete Signal Spaces** . . . . . . . . . . . . . . . 3.3.4 Remark on Causal Relations** . . . . . . . . . . . . . . . . . . . . . 3.4 Solution to the Representation Problems . . . . . . . . . . . . . . . . . . 3.4.1 Solution to the Set Representation Problem . . . . . . . . . . 3.4.2 Solution to the Stochastic Representation Problem of Autonomous Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Qualitative Model of the Quantised System . . . . . . . . . . . . . . . . 3.5.1 Motivation and Modelling Aims . . . . . . . . . . . . . . . . . . . . 3.5.2 Definition of the Qualitative Model . . . . . . . . . . . . . . . . . 3.5.3 Properties of the Qualitative Model . . . . . . . . . . . . . . . . . 3.6 General Solution to the Stochastic Representation Problem** 3.6.1 The Frobenius–Perron Operator** . . . . . . . . . . . . . . . . . . 3.6.2 Representation of Autonomous Quantised Systems** . . 3.6.3 The Foias Operator** . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.4 Representation of Quantised Systems** . . . . . . . . . . . . . 3.6.5 Convergence of the Approximation** . . . . . . . . . . . . . . . 3.7 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.

5.

53 53 55 56 58 59 59 62 66 66 67 72 82 82 84 87 88 90 91

Further Aspects of Modelling Quantised Systems* . . . . . . . . 4.1 Motivation* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Computation of the Behavioural Relation* . . . . . . . . . . . . . . . . . 4.2.1 Computation Principle* . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Point–Mapping* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Hyperbox–Mapping* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 Parameter Uncertainties and Noise** . . . . . . . . . . . . . . . 4.2.5 Remarks** . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Relations between Quantisation and Properties of the Model* 4.3.1 State Space Partitions Adapted to the Stationary Behaviour* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 State Space Partitions Adapted to the Dynamical Behaviour* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

95 95 96 96 97 98 102 104 105

Automata Theory for Process Supervision . . . . . . . . . . . . . . . . 5.1 State Observation Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Existence of Solutions to the Observation Problems . . . 5.2 Solution to the State Observation Problems . . . . . . . . . . . . . . . . 5.2.1 General Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Recursive Form of the Solution . . . . . . . . . . . . . . . . . . . . .

123 123 123 124 125 125 128

105 114 121

Contents

6.

IX

5.3 Application of the State Observer . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Properties of the State Observer . . . . . . . . . . . . . . . . . . . 5.3.2 Observation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Stochastic Operator of the Observer** . . . . . . . . . . . . . . 5.4 Observability* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Stochastic Unobservability* . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 Stochastic Observability* . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3 Distinguishing Inputs* . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.4 Other Observability Definitions* . . . . . . . . . . . . . . . . . . . 5.5 Fault Diagnostic Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.2 Stochastic Processes Augmented by Faults . . . . . . . . . . . 5.6 Solution to the Fault Diagnostic Problem . . . . . . . . . . . . . . . . . . 5.6.1 Formal Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.2 Diagnostic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Diagnosability* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.1 Stochastic Diagnosability and Undiagnosability* . . . . . . 5.7.2 Fault Detectability and Identifiability* . . . . . . . . . . . . . . 5.7.3 Comparison to Other Diagnosability Definitions* . . . . . 5.8 Extensions** . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8.1 Diagnosis Using Fault Sequences** . . . . . . . . . . . . . . . . . 5.8.2 Isolation of I/O Signal Corruptions** . . . . . . . . . . . . . . . 5.8.3 Measurement Uncertainties** . . . . . . . . . . . . . . . . . . . . . . 5.8.4 Unknown–Input Observer** . . . . . . . . . . . . . . . . . . . . . . . 5.9 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

130 130 133 133 136 136 140 143 145 146 146 147 147 147 148 150 150 154 155 156 156 159 163 165 166

State Observation and Diagnosis of Quantised Systems . . . 6.1 State Observation of Quantised Systems . . . . . . . . . . . . . . . . . . . 6.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 Solution to the State Observation Problem . . . . . . . . . . 6.1.3 State Observation Algorithm for Quantised Systems . . 6.1.4 Remarks on the Observability of Quantised Systems* . 6.1.5 State Observation of the Running Example . . . . . . . . . . 6.2 Diagnosis of Quantised Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.2 Solution to the Quantised Fault Diagnostic Problem . . 6.2.3 Fault Diagnostic Algorithm for Quantised Systems . . . . 6.2.4 Fault Diagnosis of the Running Example . . . . . . . . . . . . 6.3 Sensor and Actuator Fault Diagnosis** . . . . . . . . . . . . . . . . . . . . 6.4 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

169 169 169 170 173 173 176 181 181 184 188 189 192 196

X

7.

Contents

Stochastic Automata Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Motivation for Using Automata Networks . . . . . . . . . . . . . . . . . 7.2 Networks of Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Introduction to Networks of Stochastic Processes . . . . . 7.2.2 Complexity Considerations . . . . . . . . . . . . . . . . . . . . . . . . 7.2.3 The Feedback Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.4 Stochastic Automata on Attributes . . . . . . . . . . . . . . . . . 7.2.5 Network Representation by Stochastic Automata on Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Behaviour of Automata Networks . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Composition of Automata Networks . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Main Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 Self–Loop–Free Composition Operation . . . . . . . . . . . . . 7.4.3 Self–Loop Elimination Operation . . . . . . . . . . . . . . . . . . . 7.4.4 Composition Algorithm and Composition Order . . . . . . 7.4.5 Behaviour Analysis Using the Composed Automaton . . 7.4.6 Automata Composition and Diagnosis in Relational Algebra** . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.7 Networks with Stochastic Direct Feedthroughs** . . . . . 7.5 State Observation and Diagnosis of Automata Networks . . . . . 7.6 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

199 199 200 200 201 202 204

8.

Component–Oriented Modelling of Quantised Systems . . . . 8.1 Networks of Quantised Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Decomposed Quantised Systems . . . . . . . . . . . . . . . . . . . . 8.1.2 Decentralised Supervision . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.3 Decomposition of Sampled Quantised Systems* . . . . . . 8.2 Qualitative Modelling of Decomposed Quantised Systems . . . . 8.2.1 Network of Causal Quantised Systems . . . . . . . . . . . . . . 8.2.2 Formal Description of Networks of Quantised Systems 8.2.3 Qualitative Modelling of Quantised Subsystems . . . . . . 8.2.4 Complexity of Networks of Qualitative Models . . . . . . . 8.3 Supervision Based on Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 State Observation of Decomposed Quantised Systems . 8.3.2 Diagnosis of Decomposed Quantised Systems . . . . . . . . 8.4 Modelling and Supervision Example . . . . . . . . . . . . . . . . . . . . . .

233 233 233 234 235 236 236 238 240 243 244 244 245 247

9.

Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 The Titration and Neutralisation Plant . . . . . . . . . . . . . . . . . . . . 9.2 Modelling of the Titration and Neutralisation Plant . . . . . . . . . 9.3 State Observation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.2 Qualitative Modelling of the Reactor . . . . . . . . . . . . . . . . 9.3.3 State Observation Results . . . . . . . . . . . . . . . . . . . . . . . . .

253 253 256 262 262 264 270

208 210 215 215 216 219 220 222 225 228 229 232

Contents

9.4 Fault Diagnosis Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.2 Component–Oriented Qualitative Modelling of TINA . 9.4.3 Diagnostic Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Further Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

XI

272 272 274 280 286

10. Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

Appendices A. Mathematical Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 A.1 A Brief Introduction to Measure Theory . . . . . . . . . . . . . . . . . . . 307 A.2 Definitions in Discrete Stochastics . . . . . . . . . . . . . . . . . . . . . . . . 310 B. The QuaMo–Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 C. Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 D. Parameters of the Titration and Neutralisation Plant . . . . . 337 E. Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

1. Introduction

1.1 Motivation and Objectives This book deals with the supervision of quantised systems shown in Figure 1.1. A quantised system consists of a continuous–variable dynamical system and quantisers that serve as interfaces between the dynamical system and the supervisor. The quantisers transform the continuous input and output signals u and y into quantised signals [u] and [y], respectively. The supervisor uses only this symbolic information to determine the supervision result.

Quantised system u

y

Continuous− variable system Quantiser

Quantiser [ y]

[u] Supervisor Supervision result

Fig. 1.1. Supervision of quantised systems.

Quantisers occur in many technical applications. For example, alarm messages in industrial plants are quantised signals because they are triggered if some system variables exceed certain bounds. Other examples are physical quantities like the concentration of substances in a reactor that cannot be precisely measured. Then, quantisation corresponds to the systematic measurement error. Further examples are actuators that can be switched only among a set of discrete values, or discrete proximity sensors. Furthermore, quantisation can occur in the system dynamics, e.g. if state variables exceed some thresholds like the maximum level in a tank or a saturation bound. J. Schröder: Modelling, State Observation and Diagnosis of Quantised Systems, LNCIS 282, pp. 1-12, 2003. Springer-Verlag Berlin Heidelberg 2003

2

1. Introduction

In most cases, however, the quantisers in Figure 1.1 will be introduced on purpose. The reason is to abstract from the numerical signal values, because supervision tasks concern a global assessment of the systems behaviour for which an abstract view appears as a natural approach. By means of the quantisers the amount of information of the continuous–variable signals is reduced to a degree just detailed enough to solve a given task. It will be shown that by this the quantisers facilitate the solution to supervision tasks. Another reason for the deliberate introduction of quantisers is that all signals are concisely dealt with as discrete–event signals. That is, for the solution to supervision tasks it is not distinguished between system–inherit quantisation or quantisation on purpose. In particular, this is useful for hybrid systems often occurring in supervision tasks. Because of the quantisers, both the continuous and discrete–event signals occurring in such systems are dealt with on the same symbolical basis. Two kinds of supervision problems are addressed by the book, namely the state observation and the fault diagnostic tasks. State observation means to reconstruct the state of a dynamical system using the measured signals and a model. Similarly, fault diagnosis means to use the measurements to decide by means of a model whether the system behaves normally or faults have occurred. With the state observation and fault diagnostic tasks only two supervision tasks are considered. However, they are key elements for solving advanced process supervision problems such as the reconfiguration of control or supervisory control. Reconfiguration concerns the task to automatically change the control law, which possibly includes structural changes of the system, to (partly) maintain the systems performance despite the presence of faults. Hence, fault diagnosis is a prerequisite of reconfiguration. Similarly, supervisory control concerns the task to keep the system within safe operating conditions. It is, therefore, relying on information about the systems state. The material presented in this book is the basis for the solution to such advanced supervision problems using quantised systems (cf. [15]). The main concern of the book is a theoretical solution to these problems. By combining continuous systems theory and the theory of discrete–event systems, the book closes the gap between these two worlds. In particular, the combination of both theories turns out to have great value for the practical application. This is because the two quantisers shown in Figure 1.1 allow to go from a precise numerical level to a qualitative description which is useful if part of the information about the system is merely qualitative. The book gives a complete introduction to the theory of quantised systems including the modelling by automata and the solution to the state observation and fault diagnostic tasks. However, the treatment of quantised systems

1.1 Motivation and Objectives

3

in this book is not limited to theory only. Many examples are used to illustrate the main ideas, mostly at the running example of a two–tank system (cf. Section 1.4). Several application examples at the end of the book show how the methods can be applied in practice. The Symbolical Versus the Classical Approach In engineering, usually numerical models like difference or differential equations are used, to design controllers, to make simulation studies and so on. This is because the solution to such tasks requires a precise analysis of the system. Supervision tasks, however, have a different nature. If compared to the aforementioned tasks they concern a global assessment of the systems behaviour. For example, diagnosis does, in general, not require that the precise magnitude of a fault is determined nor does the operation of a system within prescribed bounds necessitate the precise reconstruction of all internal variables. Instead, supervision tasks have qualitative aims like the question whether there is a leakage in a tank or whether the temperature in a reactor is high enough. Hence the question arises, whether such comparably detailed dynamical models like differential equations are required to solve these tasks, having in mind the benefits of using a less detailed model. The book will show that this is not necessary.

Proposed way of solution Problem formulation

Model class

Nature of obtained result

Symbolical formu− lation of the task

Stochastic automaton

Symbolical result

Abstraction Problem

Formulation of the supervision task Concretisation Numerical formu− lation of the task

Abstraction Quantised system Quantisation Continuous− variable system

Symbolical Level

Interpretation Supervision result

Solution

Qualitative Level

Evaluation Numerical result

Numerical Level

Classical way of solution

Fig. 1.2. The symbolical and the classical approach to process supervision.

Instead, the book proposes an alternative way of solution, based on a symbolical model – an automaton. Figure 1.2 illustrates this way of solution and

4

1. Introduction

compares it to classical approaches which are based on a numerical model. The figure contains three levels of abstraction, the numerical, the qualitative and the symbolical level. As explained above, supervision tasks are generally formulated on a qualitative level. The idea of the presented approach is formalise the tasks on the qualitative or even on the more abstract symbolical level (cf. Figure 1.2). To solve this symbolical task, a symbolical model of the physical system is required. Such a description shall be abstracted from the real system, which is in general a continuous–variable system. As shown in the figure, this step is performed by a quantisation of all continuous–variable signals and subsequent abstraction to a symbolical model. The granularity of the quantisation determines the extent to which details of the numerical description are neglected. The guideline for this signal quantisation is to refer to the system variables as roughly as possible on the qualitative level. From the introduced quantisation the quantised system emerges (cf. Figure 1.1). Because of the mixed continuous and discrete nature of the quantised system, a direct solution to the supervision task on the qualitative level turns out to be very difficult because this would mean to solve differential equations under several inequality constraints induced by the quantisers. To avoid this, a purely discrete description is abstracted from the quantised system, for which a stochastic automaton is chosen as representation form. It will be shown that stochastic automata can concisely approximate quantised systems and allow efficient solutions to supervision tasks. That is, instead differential equations and inequality constraints the system is described by a set of transition probabilities describing the stochastic automaton. The symbolical result obtained from the solution to the supervision task by means of the automaton is then transferred to the supervision result for the quantised system. It will be shown that this conclusion is possible provided that the stochastic automaton is a complete model of the quantised system. Therefore, completeness of the model is a crucial requirement that is thoroughly investigated in the book. The classical approach to solve supervision tasks is shown in the lower part of Figure 1.2. It represents a way of solution that uses standard techniques of continuous systems and control theory. There, instead of seeking for a higher level of abstraction, the supervision task is concretised. This means, for example, that it is attempted to numerically reconstruct the state, e.g. by an observer or a Kalman filter, even if only the region of the state is needed. Similarly, a concretised aim can be to estimate the magnitude of a fault even though its value is not required. From the numerical solution to these concretised problems, the supervision task is solved by evaluating the numerical result, e.g. by associating a numerical fault value “near to zero” with a faultless system.

1.1 Motivation and Objectives

5

This approach is mainly motivated by the fact that the numerical level offers a large toolbox of well known methods. Though this way of solution is applicable in classical situations it has several difficulties for process supervision. Among them are the following: • A model is used which is more detailed than required. A precise model with fixed parameters has to be used. However, supervision requires to consider models with uncertainties. Taking them into account in numerical models by using sets of parameters leads to considerably complex formulations and solutions to the task. • Numerical solutions require numerical measurements. However, alarm messages, for example, only indicate whether certain thresholds are exceeded. Similarly, cost reducing industrial measurement equipment, such like proximity sensors, provide merely quantised information. Therefore, supervision requires to use also qualitative measurements. • Many numerical methods rely on the linearity of a system. However, linearity holds, in general, only locally, whereas supervision refers to the global, typically nonlinear behaviour of a system. • Methods for nonlinear systems explicitly or implicitly rely on steadiness or a Lipschitz constraint. That is, no discontinuities can be considered which excludes systems with mixed continuous and discrete dynamics. However, in supervision tasks, such hybrid systems typically occur. The quantised system approach circumvents the difficulties that arise when using the classical approach. This and further benefits can be summarised as follows: • Reduction of information. The quantisers transform the numerical continuous–time signals u(t) and y(t) into quantised discrete–time signals [u(k)] or [y(k)], respectively. By this, the amount of information that needs to be processed by the supervisor is reduced. This is shown in Figure 1.3 at the example of a pH measurement of a plant considered in Chapter 9. The complex information contained in the measured signal pH(t) over continuous time t is reduced to a sequence of integer values [pH(k)] at discrete time instances k. The figure shows first the temporal quantisation by periodical sampling and subsequently the value quantisation through a partition of the signal space into three regions. The bars show in which region of the signal space the actual pH lies at the corresponding time instance. • Abstract view on the system. Quantising the signal values as roughly as shown in Figure 1.3 leads to an abstract view on the system. Accordingly, the model used by the supervisor must match this abstract level and can ignore details that are covered by quantisation. Hence, quantisation facilitates modelling, as the methods derived for quantised systems can cope with adequately imprecise models. • Concise treatment of hybrid systems. In many practical applications, not only continuous–variable signals but also discrete–event signals occur.

6

1. Introduction

Such discrete signals can, for example, result from system–inherent quantisers given through the measurement equipment, or because a continuous subsystem interacts with a discrete subsystem, for example with a programmable logic controller (PLC) widely used in process and manufacturing industry. Such systems, in which both discrete and continuous signals occur, are called hybrid systems. By signal quantisation, all measurement signals are transformed to discrete signals. Hence, the quantisers allow a concise treatment of hybrid systems on the supervision level.

Fig. 1.3. Reduction of information by quantisation.

In summary, the benefits of using quantised systems arise from the fact that all details irrelevant for the solution to the problem are ignored. This book and other contributions show the practical relevance and applicability of the approach (cf. e.g. [50], [101], [124]).

1.2 Problem Statement and Way of Solution In this section the steps in which supervision tasks are solved by using quantised systems are described in more detail. The way of solution is outlined as a sequence of problem statements that appears by applying a general rule of engineering: In order to get an efficient solution to supervision tasks, be as rough as possible and only as precise as necessary. This means that for a good solution a trade–off between precision and complexity must be found. The overall supervision problem can be split up as follows:

1.2 Problem Statement and Way of Solution

7

Modelling Problem: How can the detail of the on–line information about the system be adjusted to the supervision task? Solution: By signal quantisation. • All signals of the system are quantised. Adjusting the granularity of the quantisation allows to refer to each signal as precisely as necessary and as roughly as possible. The measurement precision is considerably reduced. The original system becomes a quantised system (cf. Figure 1.1). Problem: How can the relation among quantised signals be described efficiently? Solution: By models that have the form of automata. • The relations among the quantised signals are described by an abstract model in form of a stochastic automaton rather than by a numerical model of the quantised system. However, automata cannot precisely represent a quantised system. It will turn out that automata can be derived which are precise enough for the solution to a given supervision task. Supervision Problem: How can the state observation and fault diagnostic tasks be solved with quantised signals and abstract models? Solution: By using the state observation and fault diagnostic methods, which are developed for stochastic automata. • The state observation task for quantised systems is transformed into a symbolical observation task for the automaton. From the solution for automata, the solution to the original state observation task for the quantised system is derived. • Similarly, the fault diagnostic task is transformed into a fault diagnostic problem for automata. Faults in the system can be detected and identified using the automata–theoretic solution. • Both tasks refer to the quantised measurements and the automaton obtained from the quantised system. Decomposition of Complex Systems Problem: How can abstract models of complex systems be set up? Solution: By component–oriented modelling. • Each component of a system is approximated by an abstract model. This leads to a network of interacting automata rather than a single automaton. In this way complex systems are described by automata networks which reduces the complexity of the model considerably.

8

1. Introduction

Problem: How can supervision tasks be solved for complex systems? Solution: By supervision methods for automata networks. • Like for single automata, symbolical results to the state observation and fault diagnostic tasks are derived by using automata networks. All calculation steps refer to the automata network. The results are then transferred to the supervision result of the original task. Application Question: Can the proposed methods be used to solve supervision problems in practice? Answer: The applicability is shown for a neutralisation plant and further application examples. • The derived methods are applied to a chemical plant showing in detail that quantisation of signals and abstract models reduce the complexity such that these tasks can be solved under real–time constraints. • Further practical applications of the methods are briefly summarised. The above structure of the solution is reflected by the structure of the book which is explained in the following section.

1.3 Structure of the Book and Guidelines for Reading The book consists of four major parts: Modelling (Chapters 2, 3 and 4): The quantised system is investigated and its concise abstraction to stochastic automata is derived. Supervision (Chapters 5 and 6): The solution to the state observation and fault diagnostic problems is given based on a qualitative model in form of a stochastic automaton. Component–oriented modelling (Chapters 7 and 8): The quantised system is decomposed into a network of interconnected subsystems. The supervision task is solved based on a stochastic automata network. Application (Chapter 9): The practical relevance of the developed methods is shown for a neutralisation plant and further examples.

1.3 Structure of the Book and Guidelines for Reading

Automata theory

Quantised system

2.1 − 2.4

3.1 − 3.4 Modelling

4.1 − 4.3 5.1 − 5.3 5.5 − 5.6

7.1 − 7.3

9

7.4 − 7.5

6.1 − 6.2

Supervision

Component− oriented methods

8.1 − 8.5

Application 9.1 − 9.2

9.4

9.3

Application

9.5

Fig. 1.4. Structure of the book.

The three first parts each consist of an automata theoretic chapter and a chapter concerning the quantised system. Figure 1.4 shows the dependencies among the sections. The book is structured such that it is possible to study only those parts related to automata theory. They do not contain any reference to the theory of quantised systems. Furthermore, the component– oriented methods can be skipped as the remainder offers a self–contained toolbox for practical applications as shown in Sections 9.3 and 9.5. Additionally, the sections fall into three categories. Those without a mark present the main ideas and solutions. Those marked with an asterisk * contain advanced topics that are of general interest, but do not contribute to the main idea. Sections with two asterisks ** deal with special topics. Marked sections later in the book may require that foregoing marked sections of the same category have been read. The reader can skip all sections with asterisks, because unmarked sections do not refer to them, except for some remarks not essential for understanding the sections content. The proofs, however, use results of all categories without regard to this convention. Most of them have been moved to the appendix. The book is embedded in current research and has, therefore, many connections to contributions in literature. However, the main text only occasionally refers to them in order to focus on the presentation of the material. Bibliographical notes at the end of each chapter are used to discuss similarities and differences to other approaches in literature.

10

1. Introduction

Throughout all chapters a running example is used. Its purpose is to illustrate the presented concepts and methods and help to understand the close connections between the different topics of the book. The running example is introduced in the following section.

1.4 Running Example Throughout the text the two–tank system shown in Figure 1.5 is used as example. Its dynamical behaviour is intuitive but complex enough to serve as non–trivial illustration of the developed concepts.

P LC

qp

q12u

LS1

h max LS2

V12u

h1 leak

ql

h 12u

q12l

h LS4 V12l

LS3

h2 FI

LS4

q2

Fig. 1.5. Two–tank system.

The tanks are connected via two magnetic valves V 12l and V12u at heights zero and h12u , respectively. A pump is used to fill the left tank while a security switch deactivates the pump as soon as the liquid level reaches hmax . The left tank has no direct outflow unless there is a leak, which is considered as a possible fault. The right tank has an outflow and can be filled through the connections to the left tank. Though this setting only allows levels in the right tank lower than in the left, all liquid levels h2 from 0 up to hmax are admissible as initial conditions. From the tank system, two different types of measurements are available. To the tanks, four discrete level sensors LS1–LS4 are attached. Each sensor LSi detects whether the liquid level in the corresponding tank is above or below the height hLSi at which the sensor is installed. That is, only quantised level measurements are available. In contrast to this, the sensor FI provides a continuous–variable measurement qˆ2 of the outflow.

1.4 Running Example

11

The two tanks have the liquid levels h1 and h2 given in metres, which are used as state variables in the model given below. The outputs li of the discrete level sensors LSi are either zero if the liquid level is below hLSi or one if it is above. All flow rates q are in cubic metres per second, except the output qˆ2 of sensor FI in litres per minute. The normalised pump velocity is described by vp and the binary commands given to the valves by v12l and v12u . The cross–section area of the tanks is denoted by A and all other constants by c. All parameters of the model are given in Table 1.1. The two–tank system is described by the following equations. 1 h˙ 1 (t) = (qp (t) − q12l (t) − q12u (t) − ql (t)) , A 1 h˙ 2 (t) = (q12l (t) + q12u (t) − q2 (t)) , A p qˆ2 (t) = cˆ2 · h2 (t) , ( 1 for h1 (t) > hLSi li (t) = for i = 1, 2 0 otherwise , ( 1 for h2 (t) > hLSi li (t) = for i = 3, 4 0 otherwise ,

(1.1) (1.2) (1.3) (1.4)

(1.5)

with ( qp (t) = ( q12l (t) =

cp vp (t) for h1 (t) ≤ hmax , 0 < vp (t) ≤ 1 0

otherwise ,

c12l sgn(h1 (t) − h2 (t))

p

|h1 (t) − h2 (t)| for v12l = 1

0 otherwise , p  c12u h1 (t) − h12u for h1 (t) > h12u , h2 (t) ≤ h12u , v12u = 1    p     −c12u h2 (t) − h12u for h1 (t) ≤ h12u , h2 (t) > h12u , v12u = 1 q12u (t) = c12u sgn(h1 (t) − h2 (t))·  p    · |h1 (t) − h2 (t)| for h1 (t) > h12u , h2 (t) > h12u , v12u = 1    0 otherwise , ( p c2 h2 (t) for h2 (t) > 0 q2 (t) = 0 otherwise , ( p cl h1 (t) for h1 (t) > 0 and leak in the left tank ql (t) = 0 otherwise .

12

1. Introduction

The above eqns. (1.1)–(1.5) have the form of a nonlinear state space model ˙ ˆ (x(t), u(t)) , x(t) =g

(1.6)

y(t) = h(x(t), u(t)) ,

(1.7)

with the state x(t) = (h1 (t) h2 (t))0 , the input u(t) = (vp (t) v12l (t) v12u (t))0 , and either the output vector y(t) = (l1 (t) l2 (t) l3 (t) l4 (t))0 or the scalar output y(t) = qˆ2 (t). The two–tank is a hybrid system, as it has continuous and discrete variables as well as switching dynamics [2], [22], [98]. This becomes obvious from the above equations that contain continuous dynamics as well as switching conditions and several inequalities. Under a sample and hold assumption for the inputs, the continuous–time state space model (1.6) and (1.7) is transformed into a discrete–time state space model by Z (k+1)Ts ˆ (x(τ ), u(τ )) dτ, (1.8) x((k+1) Ts ) = g(x(k Ts ), u(k Ts )) = x(k Ts )+ g kTs

y(k Ts ) = h(x(k Ts ), u(k Ts )) ,

(1.9)

where Ts denotes the sampling time. In eqns. (1.8) and (1.9), the output function h remains as in eqn. (1.7). The function g is determined numerically by evaluating eqn. (1.8). For given sampling time Ts , the model given by eqns. (1.8) and (1.9) has the form of a discrete–time state space model x(k+1) = g(x(k), u(k)) y(k) = h(x(k), u(k)) which will be used in the remainder of the book. Unless otherwise stated, a sampling time of Ts = 5 s will be used.

Table 1.1. Parameters of the tank system A hmax h12u hLS1 hLS2 hLS3 hLS4 cp c12l c12u c2 cˆ2 cl Ts

1.5 · 10−2 m2 0.60 m 0.30 m 0.40 m 0.20 m 0.20 m 0.10 m 1.5 · 10−4 m3 /s 6.0 · 10−4 m5/2 /s 4.0 · 10−4 m5/2 /s 2.0 · 10−4 m5/2 /s 12.0 l/(min · m1/2 ) ≥ 0 m5/2 /s 5s

Cross–section area of each tank Maximum liquid level of each tank Height of the upper connection Height of level sensor LS1 Height of level sensor LS2 Height of level sensor LS3 Height of level sensor LS4 Feed rate of the pump Flow constant of valve V12l Flow constant of valve V12u Flow constant of the outlet Constant of outflow sensor FI Flow constant for a leak Sampling time

2. Basics of Stochastic Automata Theory

This chapter summarises results of the theory of stochastic automata. In Section 2.1, the stochastic automaton is defined and its representation by an automaton graph is explained. Mathematically, the stochastic automaton is a discrete stochastic processes. Its analysis, therefore, requires several notions and notations of stochastic processes that are introduced in Section 2.2. Based on this, the behaviour of stochastic automata is studied in Section 2.3. For the application of automata to fault diagnosis, the definition of the stochastic automaton is extended in Section 2.4. In Section 2.5 additional properties of automata like the stationary behaviour are discussed. Finally, in Section 2.6, stochastic operators are introduced and their relation to stochastic automata is shown.

2.1 Stochastic Automata Stochastic automata (SA) are state machines with a stochastic transition behaviour. The finite stochastic automaton is a 4-tuple S(Nz , Nv , Nw , L)

(2.1)

with the set of automaton states Nz = {1, 2, . . . , N }, the input alphabet Nv = {1, 2, . . . , M } and the output alphabet Nw = {1, 2, . . . , R}. The state of the automaton is denoted by z ∈ Nz , its input by v ∈ Nv and its output by w ∈ Nw . The behaviour of the automaton is described by the conditional probability that the automaton state changes from state z to state z 0 and produces the output w if its current input is v. This is described by a function L : Nz × Nw × Nz × Nv −→ [0, 1] defined by L(z 0 , w, z, v) = Prob(z 0 , w | z, v) .

(2.2)

In accordance with the terminology used for nondeterministic automata [97], the function L is called behavioural relation of the stochastic automaton. In J. Schröder: Modelling, State Observation and Diagnosis of Quantised Systems, LNCIS 282, pp. 13-35, 2003. Springer-Verlag Berlin Heidelberg 2003

2. Basics of Stochastic Automata Theory

0.76 0.1

7

0.01

9

0.08

0.33

8

0.02

0.18

0.66

0.53

0.25

0.01

03 03

0.

0.08

0.07

73

0.

0.01

0.01

0.32

0.02

0.

01

0.

0.

0.49

0.37

0.08

0.17

0.2

0.01

0.74 0.01

0.05

0.04

13

0.

3

0.19 0.56

0.24

0.18

0.01

0.57 0.08

0.01

4

3

0.

08

0.

0.0

04

2

6

0.01

0.

22

0.

0.11

5

0.01

0.16

57

26

0.

0.31 0.6

0.1

4

0.02

0.

21

0.

0.43

1

6

0.46

0.35

0.1 0.33

5

0.03

0.

33

02

0.11

01 0. 0.01 0.21

4

9

0.02

0.85 0.11

0.17

0.76

0.1

8

0.7 0.04

0.48

7

0.54 0.12

0.29

14

1

2

0.01

3

0.01

Fig. 2.1. Automaton graph with nine states, three outputs and two inputs, drawn separately for v = 1 (left) and v = 2 (right).

order to indicate that the behavioural relation represents a conditional probability distribution, the symbol L(z 0 , w | z, v) is used rather than L(z 0 , w, z, v). From eqn. (2.2) follow the properties: X

0 ≤ L(z 0 , w | z, v) ≤ 1 , X L(z 0 , w | z, v) = 1 ,

∀z 0 , z ∈ Nz , v ∈ Nv , w ∈ Nw ,

(2.3)

∀z ∈ Nz , v ∈ Nv .

(2.4)

z 0 ∈Nz w∈Nw

Note that L is defined for all v ∈ Nv and all z ∈ Nz . That is, in all states z, all inputs v may occur. Consequently, all initial states and all sequences of input symbols are admissible. Automata graphs. The SA can be interpreted as a directed graph whose vertices denote the states. The edges denote possible transitions corresponding to those values z 0 , w, z and v for which the behavioural relation L is nonzero. All edges are associated with a certain input, output and the probability (2.2). Figure 2.1 shows the automaton graph of an SA with nine states, two inputs and three outputs. For simplicity of presentation, two separate graphs have been drawn, one for each input, but in principle, both graphs must be put together. In Figure 2.1, the different outputs are distinguished by the colour and thickness of the edges, from thin black lines for w = 1 to thick light grey lines for w = 3. From the lower left corner of the automaton graph, it can be seen that the probability for the state to change from z = 2 to z 0 = 4 while producing the output w = 1 is 0.26 for the input v = 1. Consequently, the behavioural relation yields L(4, 1|2, 1) = 0.26. For the same input, the automaton can also change its state from z = 2 to z 0 = 4 giving output w = 2. The probability for this transition is L(4, 2|2, 1) = 0.21. It further

2.1 Stochastic Automata

15

can be seen that this state change cannot occur while the output is w = 3, i.e. L(4, 3|2, 1) = 0 holds. According to eqn. (2.4) the sum of the transition probabilities of all emerging edges for each state and input is one. This can be seen from Figure 2.1 for all states and inputs. Transition and output relations. The behavioural relation has the two boundary distributions1 X G(z 0 | z, v) = L(z 0 , w | z, v) (2.5) w∈Nw

and

H(w | z, v) =

X

z 0 ∈N

L(z 0 , w | z, v) .

(2.6)

z

G(z 0 | z, v) is called the state transition relation and H(w | z, v) the output relation of the SA [194]. They likewise represent conditional probability distributions G(z 0 | z, v) = Prob(z 0 | z, v)

and

and possess the properties X G(z 0 | z, v) = 1 and

X

z 0 ∈Nz

H(w | z, v) = Prob(w | z, v)

H(w | z, v) = 1 ,

(2.7)

w∈Nw

respectively. Their values can also be interpreted with the help of the automaton graph. From Figure 2.1 it can be seen that for v = 1, the probability that the state changes from z = 2 to z = 4 is G(4|2, 1) = 0.21 + 0.26 = 0.47, and the probability that for v = 1 and z = 2 the output w = 1 is produced is H(1|2, 1) = 0.24 + 0.26 + 0.11 = 0.61. The transition and output relations represent less information than the behavioural relation. This can be seen from the above example. For v = 1 the automaton state can change from z = 2 to z 0 = 1, z 0 = 4 or z 0 = 5. This information is kept in the state transition relation G. For the same input and state, the outputs w = 1 or w = 2 can occur (cf. Figure 2.1). Accordingly, the output relation H has nonzero entries for these outputs. However, the relations G and H do not yield the possibly occurring outputs for a particular transition. Consider e.g. the transition from z = 2 to z 0 = 1. According to the output relation the state z = 2 yields either the outputs w = 1 or w = 2 for the input v = 1 but from Figure 2.1 it can be seen that in fact only w = 1 can occur. This information is contained in L but not in G and H. That is, in general, the behavioural relation L cannot be obtained from G and H. Only in the case that the successor state z 0 and the output w are stochastically independent the relation 1

For a summary of basic definitions in discrete stochastics see Appendix A.2.

16

2. Basics of Stochastic Automata Theory

L(z 0 , w | z, v) = G(z 0 | z, v) · H(w | z, v)

(2.8)

holds for all states, inputs and outputs and the transition and output relations represent the same information as the behavioural relation [23]. Special cases of SA. A special case is the SA with a single input, i.e. with Nv = {1}, or, equivalently, an SA with constant input ¯v ∈ Nv . Such an SA is called autonomous SA and can be equivalently represented by the 3-tuple S(Nz , Nw , La )

(2.9)

with La : Nz × Nw × Nz −→ [0, 1] ,

La (z 0 , w | z) = L(z 0 , w | z, v¯) . (2.10)

Similarly, an SA with a singleton as output alphabet is called SA without output and can be represented by the 3-tuple S(Nz , Nv , G)

(2.11)

with G : Nz × Nz × Nv −→ [0, 1] ,

G(z 0 | z, v) = L(z 0 , 1 | z, v) .

(2.12)

From eqns. (2.4) and (2.5), it becomes clear that the relation (2.12) of the SA without output is the transition relation of the SA (2.1). Finally, as a combination of the above, an SA without input and output is equivalently represented by the 2-tuple S(Nz , Ga )

(2.13)

with Ga : Nz × Nz −→ [0, 1] ,

Ga (z 0 | z) = G(z 0 | z, v¯) .

(2.14)

The transition relation Ga is sometimes written as a stochastic transition matrix Ga with (Ga )ij = Ga (z 0 = i | z = j) ,

(2.15)

where (Ga )ij denotes the matrix element of Ga in the i−th row and the j−th column (cf. Section 2.5).

2.2 Stochastic Processes and Markov Chains The stochastic automaton introduced in the previous section represents a discrete stochastic process. The aim of this section is to show the relation of the SA to an important class of stochastic processes, the Markov chains.

2.2 Stochastic Processes and Markov Chains

17

Furthermore, additional notions and notations of stochastic processes are introduced for a more detailed description and understanding of SA. A discrete stochastic process is a system with a finite discrete state space Nz = {1, 2, . . . , N } considered over a discrete set of time instances {0, 1, 2, . . .} where the state at a certain time instance k is random. That is, the system is a stochastic process if an experiment repeated under equivalent conditions may give different results. The successive measurements of the state define the random variables zp (0), zp (1), . . . , zp (k), . . . at times 0, 1, . . . , k, . . ., respectively, characterising the discrete stochastic process that is defined as the family {zp (k), k ∈ IN} of random variables. A sequence z(0), z(1), z(2), . . . of states resulting from a single experiment is called a realisation of the discrete stochastic process. The probability that a realisation of the stochastic process yields the state z(k) ∈ Nz at time k is denoted by Prob(zp (k) = z(k)) .

(2.16)

Considering the probabilities of each state of Nz at a certain time point k yields a discrete probability distribution denoted by Prob(z(k)) =

(2.17)

{Prob(zp (k) = 1), Prob(zp (k) = 2), . . . , Prob(zp (k) = N )} . In a stochastic process, the probability distribution (2.17), in general, changes with time k. Markov chains. The random variables zp (k) are, in general, not independent of each other. Instead their dependence reflects the dynamics of the system. A special kind of dependence among the random variables defines an important class of discrete stochastic processes: the Markov chains (cf. e.g. [188], [202]). A Markov chain satisfies the relation Prob(zp (k+1) = z(k+1) | zp (0) = z(0), zp (1) = z(1), . . . , zp (k) = z(k)) = Prob(zp (k+1) = z(k+1) | zp (k) = z(k))

(2.18)

for all times k ∈ IN and for all states z(0) ∈ Nz , . . . , z(k) ∈ Nz . The interpretation of the Markov property (2.18) is that the probabilities of the future state of the stochastic process are completely determined from its current state. Note that the Markov property (2.18) does not mean that the state at time k + 1 is independent of the states at times 0, 1, . . . , k − 1, but merely that the future state is conditionally independent of these states given the state at time k. The conditional probabilities Prob(zp (k + 1) = z(k + 1) | zp (k) = z(k)) are called single–step transition probabilities of the Markov chain. If the transition probabilities do not depend on the absolute time point k, i.e. if the relation

18

2. Basics of Stochastic Automata Theory

Prob(zp (k+1) = z 0 | zp (k) = z) = Prob(zp (1) = z 0 | zp (0) = z)

(2.19)

is satisfied for all times k and for all states z, z 0 ∈ Nz , the Markov chain is called homogeneous. Relation of Markov chains to SA. To show the relation of Markov chains to SA, an input v ∈ Nv on which the transition probabilities depend is added to the stochastic process. Furthermore, an output w ∈ Nw of the process is introduced. The input and the output define random variables vp (0), vp (1), . . . and wp (0), wp (1), . . ., respectively. Such a stochastic process is called a Markov chain with input and output if the relation þ   þ z (0) = z(0), . . . , zp (k) = z(k) Prob zp (k+1) = z(k+1), wp (k) = w(k) þþ p vp (0) = v(0), . . . , vp (k) = v(k) = Prob(zp (k+1) = z(k+1), wp (k) = w(k) | zp (k) = z(k), vp (k) = v(k)) (2.20) holds for all states, inputs, outputs and for all times k. As before, the Markov chain with input and output is called homogenous if the relation Prob(zp (k+1) = z 0 , wp (k) = w | zp (k) = z, vp (k) = v) = Prob(zp (1) = z 0 , wp (0) = w | zp (0) = z, vp (0) = v)

(2.21)

is satisfied for all states, inputs, outputs and for all k. The block diagram of a Markov chain with input and output is shown in Figure 2.2.

v ( k)

Markov process z ( k)

w ( k)

Fig. 2.2. Markov chain with input and output.

From eqn. (2.21) it is clear that the SA represents a homogeneous Markov chain with input and output. Such a process is represented by an SA with the behavioural relation L(z 0 , w|z, v) = Prob(zp (k+1) = z 0 , wp (k) = w | zp (k) = z, vp (k) = v)(2.22) for any time k. Likewise, Markov chains (2.19) are represented by autonomous SA. The relation between SA and Markov chains is discussed in more detail in [23]. Remark 2.2.1. Markov chains with input are also called controlled Markov chains [20], [91] and with additional output also as controlled Markov chains with partial observations [20]. A controlled Markov chain, like the SA, does not only depend on the considered system but also on the input which can itself be described by a discrete stochastic process, i.e. as a sequence of random variables. Often, in the literature, the special case that the input is described by a single time–invariant

2.3 Behaviour of Stochastic Automata

19

random variable is considered (cf. e.g. [20]). Note that this includes the special case that the random variable yields a probability of 1 for a certain input ¯v ∈ Nv , i.e. the case that the input is not random, but a constant value for all times. Then the controlled Markov chain is identical to a Markov chain without input. Markov chains with outputs where the state cannot be observed are also known as hidden Markov chains. They are used for the recognition of speech and handwriting (cf. e.g. [45]).

2.3 Behaviour of Stochastic Automata The behaviour of SA is usually studied for a given initial state distribution. In the following, the SA (2.1) with given initial state distribution Prob(z(0)) is called initialised stochastic automaton and is given by the 5-tuple [23]: S(Nz , Nv , Nw , L, Prob(z(0))) .

(2.23)

Realisation of the stochastic process. The initialised SA can be used to generate realisations of the stochastic process represented by the SA. To generate such a realisation, an initial state z(0) is determined randomly according to the given initial state distribution Prob(z(0)). For the given input one pair of successor state and output randomly chosen according to the transition probabilities of the behavioral relation (2.2). In the automaton graph this means that one of the outgoing edges of the current state is followed, while the probability associated with the edges determines the chance of a certain edge being chosen. That is, a realisation of the discrete stochastic process is obtained by starting in a certain automaton state and following a path through the automaton graph for given inputs. Example 2.3.1. Figure 2.3 shows three realisations of the stochastic process represented by the automaton graph shown in Figure 2.1 for constant input v = 2. It shows three pairs of state and output sequences, each pair obtained from a single experiment. From Figure 2.3, the path taken in the automaton graph can be followed, starting from state z = 1. For example, in the first experiment, the edge for w = 1 (thin black line) from state z = 1 to z 0 = 1 has been followed. The probability for this transition to be chosen was 0.56. Mean behaviour of the stochastic process. Usually, the SA is used as model of the stochastic process to study its mean behaviour. This means that not a certain realisation of the SA is produced but the interest lies in all realisations simultaneously. That is, the probability distribution of all realisations is studied for a given time instance k. From the initialised SA, the conditional probability distribution of all state sequences of length k+1

2. Basics of Stochastic Automata Theory

0

3 time 6

9

0

3 time 6

1

2 1

0

3 time 6

9

0

3 time 6

9

0

3 time 6

9

3 output w

2

9 8 7 6 5 4 3 2 1

9

3 output w

3 output w

9 8 7 6 5 4 3 2 1

state z

9 8 7 6 5 4 3 2 1

state z

state z

20

2 1

0

3 time 6

9

Fig. 2.3. State (top) and output (bottom) realisations of a stochastic process represented by the SA shown in Figure 2.1 for constant input v = 2.

Z(0 . . . k) = (z(0), z(1), . . . , z(k)) ∈ Nzk+1 = Nz × . . . × Nz | {z } (k+1) times

and for given input sequence V (0 . . . k−1) = (v(0), . . . , v(k−1)) ∈ Nvk = Nv × . . . × Nv {z } |

(2.24)

k times

can be determined by Prob(Z(0 . . . k) | V (0 . . . k−1))

(2.25)

= Prob(z(0), z(1), . . . , z(k) | v(0), . . . , v(k−1)) = Prob(Z(0 . . . k) | V (0 . . . k−1), z(0)) · Prob(z(0)) =

k−1 Y

G(z(i+1) | z(i), v(i)) · Prob(z(0)) ,

i=0

where the Markov property (2.18) and the conditional independence Prob(zp (k) = z(k) | vp (k) = v(k), . . . , vp (0) = v(k)) = Prob(zp (k) = z(k) | vp (k−1) = v(k−1), . . . , vp (0) = v(k))

(2.26)

of the current state and the current input have been used. The latter holds because from the definition of a Markov process by eqn. (2.20) it follows that the input at time k only influences the state at time k+1.

2.3 Behaviour of Stochastic Automata

21

The notation of the above equation is as follows: Prob(Z(0 . . . k) | V (0 . . . k− 1)) denotes a conditional probability distribution, where the notation is like in eqn. (2.17): Prob(Z(0 . . . k) | V (0 . . . k−1)) = = {Prob(zp (0) = 1, . . . , zp (k) = 1 | vp (0) = v(0), . . . , vp (k−1) = v(k−1)), . . . , Prob(zp (0) = N, . . . , zp (k) = 1 | vp (0) = v(0), . . . , vp (k−1) = v(k−1)), . . . , Prob(zp (0) = N, . . . , zp (k) = N | vp (0) = v(0), . . . , vp (k−1) = v(k−1))

}.

Each element of this probability distribution gives the probability of a realisation of the stochastic process, i.e. of a certain state sequence. All of these realisations are considered under the same condition, i.e. for the given input sequence. Hence, the notation of eqn. (2.25) using distributions means that it holds for all possible realisations and is evaluated separately for all state sequences. Equivalently, a conditional probability distribution like in eqn. (2.25) can be interpreted as a function yielding a functional value between zero and one for each possible argument on the left–hand side of the condition, parameterised by the values on the condition side. In the following all distributions are interpreted like this. To simplify the notation, the abbreviation G(k) := G(z(k+1) | z(k), v(k)) will be used in the following. The state distribution of the initialised SA at time k > 0 is described by the boundary distribution of (2.25): Prob(z(k) | V (0 . . . k − 1)) X X X = ··· Prob(Z(0 . . . k) | V (0 . . . k−1)) z(0)∈Nz z(1)∈Nz

X =

X

z(0)∈Nz z(1)∈Nz

=

X

(2.27)

z(k−1)∈Nz

X

···

G(k−1) · G(k−2) · . . . · G(0) · Prob(z(0))

z(k−1)∈Nz

G(k−1) · G(k−2) · . . . · G(0) · Prob(z(0)) .

Z(0...k−1)∈Nzk

Equation (2.27) can be written in the recursive form k > 1 : Prob(z(k) | V (0 . . . k−1)) X = G(k−1) · Prob(z(k−1) | V (0 . . . k−2)) z(k−1)∈Nz

k = 1 : Prob(z(1) | v(0)) =

X

G(0) · Prob(z(0)) .

(2.28)

z(0)∈Nz

Equation (2.28) describes the state simulation of an SA as the state distribution at time k is determined from the state distribution and input at time k − 1. Note that eqn. (2.26) means that

22

2. Basics of Stochastic Automata Theory

Prob(z(k) | V (0 . . . k − 1)) = Prob(z(k) | V (0 . . . k))

(2.29)

holds. Similarly, for the output sequence W (0 . . . k) the relation Prob(W (0 . . . k) | V (0 . . . k)) X = Prob(W (0 . . . k) | V (0 . . . k), z(0)) · Prob(z(0)) z(0)∈Nz

=

X

X

L(k) · . . . · L(0) · Prob(z(0)) =

k Y

(2.30)

L(i) · Prob(z(0))

Z(0...k+1) i=0

Z(0...k+1)∈Nzk+2

holds, where the abbreviation L(k) := L(z(k+1), w(k) | z(k), v(k))

(2.31)

is used. Similar to the above, eqn. (2.30) can be formulated recursively as k > 0 : Prob(z(k+1), W (0 . . . k) | V (0 . . . k)) X L(k) · Prob(z(k), W (0 . . . k−1) | V (0 . . . k−1)) = z(k)∈Nz

k = 0 : Prob(z(1), w(0) | v(0)) =

X

(2.32)

L(0) · Prob(z(0)) ,

z(0)∈Nz

which yields: Prob(W (0 . . . k) | V (0 . . . k)) X = Prob(z(k+1), W (0 . . . k) | V (0 . . . k)) .

(2.33)

z(k+1)∈Nz

The current output w(k) can be obtained as boundary distribution of (2.30) or (2.33). By using eqn. (2.28), from eqns. (2.32) and (2.33) follows: Prob(w(k) | V (0 . . . k)) X = H(w(k)|z(k), v(k)) · Prob(z(k) | V (0 . . . k−1)) .

(2.34)

z(k)∈Nz

Equations (2.28) and (2.34) describe the recursive simulation of the output of an SA. Example 2.3.2. Figure 2.4 shows the mean behaviour of the stochastic process represented by the SA given in Figure 2.1 for constant input v = 2 and the initial probability distribution Prob(z(0)) = {1 0 . . . 0}. The conditional probability distributions Prob(z(k) | V (0 . . . k)) (left–hand side) and Prob(w(k) | V (0 . . . k)) (right–hand side) are shown at times k = 0, 1, . . . , 9. All probabilities are shown in grey scale. A black box means that the probability of the corresponding state or output at that time is one, white means a probability of zero. Note that the realisations of the stochastic process shown in Figure 2.3 are included in the mean behaviour.

23

1

9 8 7 6 5 4 3 2 1

probability scale

3 output w

state z

2.4 Stochastic Automata for Fault Diagnosis

2 1

0

3 time 6

9

0

3 time 6

9

0

Fig. 2.4. Mean behaviour of the state (left) and output (right) of the SA shown in Figure 2.1 initialised with Prob(zp (0) = 1) = 1 for constant input v = 2.

By means of eqns. (2.25) or (2.30), it is possible to compute the probability of each realisation. For the state sequence of the first realisation of Figure 2.3, eqn. (2.25) yields the probability (cf. Figure 2.1) Prob(Zp (0 . . . 9) = Z(0 . . . 9) | Vp (0 . . . 9) = V (0 . . . 9)) = (0.19 + 0.56) · (0.17 + 0.08) · (0.74)7 ≈ 0.0228 , with Z(0 . . . 9) = (1, 1, 4, . . . , 4) and V (0 . . . 9) = (2, 2, . . . , 2). For the output, the calculation is more difficult. Up to time k = 2, eqn. (2.30) yields Prob(Wp (0 . . . 2) = W (0 . . . 2) | Vp (0 . . . 2) = V (0 . . . 2)) = 0.56 · ( 0.19 · (0.19 + 0.17) + 0.17 · (0.16 + 0.74 + 0.1)) + +0.08 · ( 0.16 · (0.19 + 0.17) + 0.74 · (0.16 + 0.74 + 0.01) + +0.1 · (0.01 + 0.46 + 0.53)) ≈ 0.1981 , with W (0 . . . 2) = (1, 2, 2) and V (0 . . . 2) = (2, 2, 2), where the sum is over all possible state sequences starting from z = 1 resulting in the considered output sequence.

2.4 Stochastic Automata for Fault Diagnosis For the application of SA to fault diagnosis, the definition of the automaton needs to be augmented by a variable corresponding to the fault f . There are two possibilities how the fault can be introduced, both of which are used in the following. The fault can either be introduced as an additional input variable such that the SA describes the effect of a fault upon the stochastic process, or as an additional state variable such that the SA also describes the

24

2. Basics of Stochastic Automata Theory

dynamics of the fault [126]. In both cases, the SA with fault is defined as a 5-tuple S(Nz , Nv , Nw , Nf , L)

(2.35)

with the finite set of fault symbols Nf = {1, . . . , S}. Depending on whether the fault is treated as input or as state, the behavioural relation is either extended to L : Nz × Nw × Nz × Nv × Nf −→ [0, 1] L(z 0 , w | z, v, f ) = Prob(z 0 , w | z, v, f ) .

(2.36)

with the fault as additional input or to L : Nz × Nf × Nw × Nz × Nf × Nv −→ [0, 1] L(z 0 , f 0 , w | z, f, v) = Prob(z 0 , f 0 , w | z, f, v) .

(2.37)

with the fault as additional state. In both cases, the SA with fault can be interpreted as standard SA such that all results of the previous sections equivalently apply. In case that the fault is an additional input, this is done by defining an arbitrary bijective mapping ˆv , with N ˆv = {1, 2, . . . , M · S} Mvf : Nv × Nf → N ˆv is by which to each pair (v, f ) of input and fault a unique input vˆ ∈ N ˆv , Nw , L) ˆ repassigned. With this, the SA S(Nz , Nv , Nw , Nf , L) and S(Nz , N resent the same stochastic process by setting ˆ 0 , w | z, vˆ) = L(z 0 , w | z, v, f ) with vˆ = Mvf (v, f ) . L(z Analogously, if the fault is introduced as an additional state, to transform the SA with fault into a standard SA, a bijective mapping ˆz , with N ˆz = {1, 2, . . . , N · S} Mzf : Nz × Nf → N ˆ is set according to is defined, and the behavioural relation L ˆ z 0 , w | zˆ, v) = L(z 0 , f 0 , w | z, v, f ) , L(ˆ with zˆ0 = Mzf (z 0 , f 0 ) and zˆ = Mzf (z, f ).

2.5 Properties of Stochastic Automata 2.5.1 Classification of Stochastic Automata This section presents some special cases of SA and shows relations to other automata like deterministic and nondeterministic automata. Only those definitions which are relevant to this book are given. For further classifications of automata the reader is referred to the textbooks [23], [67] or [184].

2.5 Properties of Stochastic Automata

25

Deterministic automata. The SA (2.1) is called deterministic if the behavioural relation L takes only values of zero or one and hence, a function λ : Nz × Nv → Nz × Nw exists such that  1 for (z 0 , w) = λ(z, v) 0 L(z , w | z, v) = (2.38) 0 otherwise holds. In contrast to the SA, in a deterministic automaton, every state has exactly one successor state for given input, and each state transition is associated with a unique output. As deterministic automata given by eqn. (2.38) are special cases of SA, all results derived for SA also hold for deterministic automata. Semi–deterministic automata [23]. The SA (2.1) is called semi–deterministic if a function ϕ : Nz × Nv × Nw → Nz exists such that  > 0 for z 0 = ϕ(z, v, w) (2.39) L(z 0 , w | z, v) = 0 otherwise holds. According to eqn. (2.39), semi–deterministic automata have the property that if in addition to the state z and input v the output w is known, the successor state z 0 is uniquely determined. Nondeterministic automata. Here, the term nondeterministic automaton is used if the description of the automaton contains merely the possibility of a successor state/output pair to occur, but assigns no probability to its occurrence. A nondeterministic automaton is defined as 4–tuple N (Nz , Nv , Nw , Ln )

(2.40)

with Nz , Nv and Nw as defined for the SA and the nondeterministic behavioural relation Ln : Nz × Nw × Nz × Nv −→ {0, 1} .

(2.41)

In Ln , a one indicates that a transition is possible while a zero means that it is not. For given SA, the embedded nondeterministic automaton can be obtained by setting  1 if L(z 0 , w | z, v) > 0 Ln (z 0 , w, z, v) = (2.42) 0 otherwise . On the other hand, an SA cannot be obtained from a nondeterministic automaton as the SA contains more information. However, it might be reasonable for some applications to derive an SA from a given nondeterministic automaton by setting the probabilities of all pairs (z 0 , w) to the same value ý !−1  P  Ln (z 0 , w, z, v) if Ln (z 0 , w | z, v) = 1 (2.43) L(z 0 , w | z, v) = z 0 ∈Nz ,w∈Nw   0 otherwise ,

26

2. Basics of Stochastic Automata Theory

that is, the derived SA contains, like its originating nondeterministic automaton, no information concerning the frequency of occurrence of the different successor states and outputs. By using eqn. (2.43), it is possible to use several results obtained for stochastic automata directly for nondeterministic automata. Stochastic Mealy–automata [23]. SA for which the relation L(z 0 , w | z, v) = G(z 0 | z, v) · H(w | z, v)

(2.44)

0

holds for all z , z ∈ Nz , v ∈ Nv and w ∈ Nw are called stochastic Mealy– automata (cf. eqn. (2.8)). Stochastic direct feedthrough. In general, a dynamical system is said to have a direct feedthrough, if the value of the output at a certain time instance depends upon the input at that time without any delay. In analogy to this, an SA is said to have a stochastic direct feedthrough, if the probability distribution of the current output depends upon the input. Formally, the SA is said to have a stochastic direct feedthrough if ∃z ∈ Nz , ∃w ∈ Nw , ∃v1 , v2 ∈ Nv , v1 6= v2 : H(w | z, v1 ) 6= H(w | z, v2 ) (2.45) holds. Consequently, the SA has no stochastic direct feedthrough if there ¯ : Nw × Nz → [0, 1] such that exists a function H ¯ | z) H(w | z, v) = H(w

(2.46)

holds for all v ∈ Nv , w ∈ Nw and z ∈ Nz . 2.5.2 Classification of Automaton States* In the following, the states of the SA are classified according to their dynamical properties. For this purpose, the autonomous SA without output introduced in Section 2.1 is considered, which is equivalent to a homogenous Markov chain. Accordingly, terms used in Markov theory are used and results from this theory apply to SA. The results and definitions of this section are taken from [188] and are used to study the stationary and dynamical behaviour of SA and qualitative models (cf. Sections 2.5.3, 4.3.1 and 4.3.2). The k–step transition probability Prob(zp (k) = z(k) | zp (0) = z(0)) of the SA can be obtained by means of the Chapman–Kolmogorov equation [188] (2.47) Prob(zp (k) = z(k) | zp (0) = z(0)) X = Prob(zp (k) = z(k) | zp (l) = z(l)) · Prob(zp (l) = z(l) | zp (0) = z(0)) z(l)∈Nz

holding for any fixed l, 0 < l < k. Applying (2.47) recursively with l = 1 yields the result of eqn. (2.27) for constant input and given initial state. If for given z 0 at least one state z exists such that the transition probability

2.5 Properties of Stochastic Automata

27

Prob(zp (1) = z 0 | zp (0) = z) is nonzero, the state z 0 ∈ Nz is called reachable. All reachable states are further classified as periodic or non–periodic. The greatest common divisor kP of all indices k for which Prob(zp (k) = z | zp (0) = z) > 0 holds is called period of state z. A state for which kP > 1 holds is called kP –periodic, a state for which kP = 1 holds is called aperiodic. Furthermore, a state z is called recurrent if the probability ever to return to state z when starting in z is one. Otherwise, the state is called transient. ˜z ⊆ Nz of states is called irreducible if for every pair of states z 0 , z ∈ A set N ˜ Nz , there exists an index k ∈ IN such that Prob(zp (k) = z 0 | zp (0) = z) > 0 holds. In other words, in an irreducible set of states, every state can be ˜z ⊆ Nz of states is called reached from all other states. Furthermore, a set N closed, if ˜z , z ∈ N ˜z and k ≥ 1 . Prob(zp (k) = z 0 | zp (0) = z) = 0 holds for all z 0 6∈ N Every closed set of states with the corresponding part of the transition relation Ga can be interpreted as a separate autonomous SA. If a closed set of ˜z consists of only one state, this state is called absorbing. Its easy to states N show that a state z is absorbing if and only if Prob(zp (1) = z | zp (0) = z) = 1 holds (cf. e.g. [177]). The following results will be used in the next section and are taken from [188]. Lemma 2.5.1. [188] A closed set of states is irreducible if and only if it contains no further closed set of states. Lemma 2.5.2. [188] All states of an irreducible set of states are either periodic with the same period kP or aperiodic. The above definitions are also used in the following for non–autonomous SA and a certain constant input. Then, for example, a state is called recurrent under the constant input v¯, if the state of the autonomous SA corresponding to v¯ ∈ Nv is recurrent. 2.5.3 Stationary Behaviour of Stochastic Automata* Stationary behaviour of autonomous stochastic automata. The autonomous SA represents a Markov chain and results obtained for the stationary behaviour of Markov chains can be directly applied. The following summarises the main results, presented in more detail in e.g. [56], [87] or [188]. Stationary and limiting distribution. The probability distribution of the current state of the autonomous SA is according to eqn. (2.28) given by: X Prob(z(k)) = Ga (z(k) | z(k−1)) · Prob(z(k−1)) . (2.48) z(k−1)∈Nz

28

2. Basics of Stochastic Automata Theory

Using the vector notation pz (k) = ( Prob(zp (k) = 1) Prob(zp (k) = 2) . . . Prob(zp (k) = N ))

0

for the probability distributions of the state at time k, eqn. (2.48) reads as pz (k) = Ga · pz (k−1) .

(2.49)

¯ z is called stationary, if and only if A probability distribution p ¯z ¯ z = Ga · p p

(2.50)

holds. In contrast to this, the distribution obtained from ˜ z = lim (Ga )k · pz (0) p k→∞

(2.51)

is called limiting distribution of the initialised autonomous SA with initial distribution pz (0) if the limit exists. The stationary distribution is not necessarily the limiting distribution of an initialised SA as the following example shows. Example 2.5.1. Consider an autonomous SA with Nz = {1, 2} and 

Ga =

01 10



.

¯ z = (0.5 0.5)0 . The The stationary distribution of this SA is uniquely given by p limiting distribution of the SA initialised with pz (0) = (0.5 0.5)0 is given by ˜ z = (0.5 0.5)0 , but the limiting distribution of the same SA initialised with any p other probability distribution pz (0) = (p1 p2 )0 , with p1 6= p2 does not exist.

For the stationary and limiting distributions, the following result is obtained. Lemma 2.5.3. [198] For an autonomous SA without output and an irreducible set of states Nz , the following holds: ¯ z satisfying eqn. (2.50). 1. The SA has a unique stationary distribution p 2. If the states are aperiodic, the limiting distribution always exists, and is independent of the initial state distribution and is equal to the stationary ¯z . distribution p 3. If the states are periodic, the limiting distribution does not exist or is equal ¯ z , depending on the initial distribution. to the stationary distribution p Note that according to Lemma 2.5.2, for an SA with an irreducible set of states, either the condition in item two or three of Lemma 2.5.3 is satisfied. For further investigations of the stationary behaviour of the periodic case, the reader is referred to [56] or [87]. Lemma 2.5.3 is formulated for SA with an irreducible set of states Nz . In the following, it is shown that the stationary behaviour of every autonomous SA is given by the irreducible sets of states it contains and that the stationary behaviour is therefore obtained from a superposition of the results of Lemma 2.5.3 [87].

2.5 Properties of Stochastic Automata

29

Stationary distribution for a reducible set of states Nz . According to eqns. (2.3) and (2.4), the transition matrix is a square stochastic matrix, i.e. a nonnegative matrix with a column sum of one. It has at least one eigenvalue λi = 1 and for all eigenvalues, the condition |λi | ≤ 1 holds [188]. Every eigenvector corresponding to an eigenvalue of one is a stationary distribution of the corresponding SA. It can be shown that the transition matrix of an SA with an irreducible set of states possesses exactly one eigenvalue λi = 1 and has therefore a unique stationary distribution (cf. Lemma 2.5.3). A square stochastic matrix is called decomposable if it can be transformed by a symmetric permutation of its rows and columns to the form   Gu Gw ˆ Ga = (2.52) 0 Gv with square, nonzero matrices Gu and Gv and a nonzero matrix Gw . The symmetric permutation of rows and columns corresponds to a renumbering of the states of the SA. If a stochastic matrix cannot be brought to the form (2.52), it is called nondecomposable and has a unique eigenvalue of one. The nondecomposability of the transition matrix therefore corresponds to the irreducibility of the corresponding set of states of the SA. The transition matrix Ga of an SA with a reducible set of states is always decomposable and can be brought to the normal form   G11 0 0 · · · 0 G1,κ+1 G1,κ+2 · · · G1,ν  0 G22 0 · · · 0 G2,κ+1 G2,κ+2 · · · G2,ν     ..  .. .. . . .. .. .. .. ..  .  . . . . . . . .    0 0 0 0 Gκκ Gκ,κ+1 Gκ,κ+2 · · · Gκ,ν  ˜a =   G  0 0 0 · · · 0 Gκ+1,κ+1 Gκ+1,κ+2 · · · Gκ+1,ν  (2.53)    0 0 0 ··· 0 0 Gκ+2,κ+2 · · · Gκ+2,ν     .  .. .. . . . .. .. .. ..  ..  . . . . .. . . . 0

0 0 ··· 0

0

0

· · · Gν,ν

by symmetric permutation of its rows and columns, where each Gii , i = 1, 2, . . . , ν is a square, nonzero, nondecomposable matrix. Each of the submatrices Gii , i = 1, 2, . . . , κ is a stochastic matrix and has a unique eigenvalue of one. They correspond to groups of states of the Markov chain that are all recurrent [87]. In contrast to this, the submatrices Gii , i = κ+1, . . . , ν are substochastic, i.e. at least one column sum of Gii is less than one, and therefore, they do not posses eigenvalues of one [188]. They correspond to groups of states of the Markov chain that are all transient, and the submatrices Gji , i = κ+1, . . . , ν, j < i describe their transition to the recurrent groups [87]. As the set of eigenvalues of the block triangular matrix (2.53) is given by the union of the sets of eigenvalues of the individual blocks, the transition ˜ a has exactly κ eigenvalues λi = 1 and thus has exactly κ eigenvecmatrix G ¯ iz , i = 1, . . . , κ that are stationary distributions of the corresponding tors p

30

2. Basics of Stochastic Automata Theory

autonomous SA. These stationary distributions are called elementary stationary distributions as all stationary distributions of the SA can be expressed by their convex combinations [198]. That is, every stationary distribution of the SA is given by ¯ z = a1 p ¯ κz , ¯ 2z + . . . + aκ p ¯ 1z + a2 p p

with

ai ≥ 0,

κ X

ai = 1 .

(2.54)

i=1

The above results are summarised by the following lemma. Lemma 2.5.4. [188] For an autonomous SA without output and a reducible set of states Nz , the following holds: 1. By renumbering of the states, the transition matrix Ga of the SA can be brought to the form (2.53). 2. Each of the submatrices Gii , i = 1, 2, . . . , κ corresponds to an irreducible set of states. For each of these sets of states, Lemma 2.5.3 applies. 3. All stationary distributions are convex combinations of the κ eigenvectors ¯ iz corresponding to the κ eigenvalues λi = 1. p According to Lemma 2.5.4, the stationary behaviour of the SA is given as a superposition of the stationary behaviour in the irreducible set of states it contains. Remark 2.5.1. All submatrices Gii , i = 1, 2, . . . , κ have a unique eigenvalue λi = 1 but can have further eigenvalues with |λi | = 1. The number of eigenvalues with |λi | = 1 corresponds to the period KP of the irreducible set of states. Example 2.5.2. Consider the autonomous SA without output given by the automaton graph on the left–hand side of Figure 2.5. A renumbering of the states results in the SA shown on the right–hand side of Figure 2.5 which has the transition matrix:   0 0 0 0 1 0.34 0 0 0    0 0.23  0 0 0.97 0.2 0 0 0   0.3 0.14  0 0.03 0.79 0.09 0 0  0  0 0  0.2 0.07 0 0.01 0.82 0.17 0    0 0  0.28 0 0 0 0.09 0.83 0 ˜a =   . G     0 0 0 0 0 0.66 0.11 0 0     0 0 0 0 0.69 0 0  0  0    0 0 0 0 0 0 0 0.35 0.39  0 0 0 0 0.24 0 0 0 0 This transition matrix is in the decomposed normalised form (2.53) with κ = 2 and ν = 6. The two nondecomposable stochastic submatrices G11 and G22 each have an unique eigenvalue of one. The corresponding eigenvectors yield the stationary distributions 1 ¯ ˜z = ( 1 0 p 0 0 0 0 0 0 0 )0 , 2 ¯ ˜ z = ( 0 0.85 0.128 0.014 0.008 0 0 0 0 )0 . p

2.5 Properties of Stochastic Automata

0.39

0.79 0.03

0.79 0.03

0.39

0.24

3

0.97

14

14

0.97

0.28

0.01

0.69 0.66 1

0.28

0.01

0.69

0.3

0.

0.66

3 0.2

0.34

0.

0.2

0.34

0.23

9

07

0.35

6

0.3

2

8

0.24

0.11

07

5

5

0.35

1

0.17

0.

6

0.09

4 0.09

7

0.2

0.83

9

0.17

0.83

8

0.

1

0.82 0.09

0.09

4

0.11

7

1

0.82 0.2

31

2

0.23

Fig. 2.5. Autonomous SA (left) transformed to an SA (right) with decomposed normalised transition matrix.

The stationary behaviour of the SA is given by the convex combinations of these two distributions. As all states of the two irreducible sets of states are aperiodic, the limiting distribution always exists and is one stationary distribution given by the convex combination 1 2 ¯ ¯ ¯ ˜ =α·p ˜ ˜ +β·p p z

z

z

with α, β ∈ [0, 1], α + β = 1 depending on the initial distribution pz (0). If the initial state is for example z = 5, then α = 0, β = 1, whereas for z = 7, one obtains α = 0.11/(0.11 + 0.2) ≈ 0.355, β ≈ 0.645. To obtain the stationary behaviour of the original SA, the state renumbering must be reversed yielding: ¯ 1z = ( 1 0 0 0 0 0 0 0 p 0 )0 , 2 ¯ z = ( 0 0.85 0 0 0.128 0 0 0.014 0.008)0 . p

Stationary behaviour of stochastic automata with input. The SA is a controlled stochastic process, and therefore, its behaviour depends upon the behaviour of the input. Hence, it is only possible to study the stationary behaviour of the SA for stationary input. The simplest assumption on the input is to assume that for k → ∞ the input is not random but given by a single constant value v¯ = Nv . Then, the stationary behaviour of the SA can be determined by considering the autonomous SA obtained for v = v¯ and the results of autonomous SA apply. If the input is not constant but changing, the study of limiting distributions is only reasonable in combination with deterministic assumptions on the input, e.g. for periodically changing input symbols. However, stationary distributions describing the mean probability of the stochastic process being in a certain state for k → ∞ can be determined also for random input as shown in the following. It is assumed that the input is described by a stationary input process described by the probability distribution

32

2. Basics of Stochastic Automata Theory

Prob(v) = {Prob(vp = 1), Prob(vp = 2), . . . , Prob(vp = M )} .

(2.55)

If this condition is not satisfied because the input itself is the result of a stochastic process, the stationary behaviour of the input process must be determined first. The stationary input distribution then has the form of eqn. (2.55). Assuming that the inputs at different times are independent of each other, the following result is obtained for the stationary behaviour. Lemma 2.5.5. The stationary behaviour of an SA with a temporally constant input distribution (2.55) is given by the stationary behaviour of an autonomous SA with the transition relation X Ga (z 0 | z) = G(z 0 | z, v) · Prob(v) . (2.56) v∈Nv

Proof. See Appendix C.1.

t u

According to Lemma 2.5.5, the stationary behaviour of SA with input can be determined using the results for autonomous SA.

v=2

1

v=2

v=2

2

v=1

3

v=1

v=2

4

v=1

v=2 v=1

5

v=1

Fig. 2.6. SA with two inputs.

Example 2.5.3. Consider the SA with two inputs shown in Figure 2.6. For constant input v¯ = 1 or v¯ = 2, one obtains ¯z = ( 0 p

1

0

0

0 )0

or

respectively. For Prob(v) = { 0.8 , mean transition relation:  0 0 0 0  1.0 0.8 0.8 0  Ga (z 0 |z) =  0 0.2 0 0.8  0 0 0.2 0.2 0 0 0 0

¯z = ( 0 p

0

0

1

0 )0 ,

(2.57)

0.2 }, one obtains by means of eqn. (2.56) the  0 0   0  . 1.0  0

(2.58)

This matrix has a unique unit eigenvalue with the corresponding eigenvector ¯z = ( 0 p

0.76

0.19

0.05

0 )0

which is the stationary distribution of the SA with input.

(2.59)

2.6 Stochastic Operators**

33

Stationary behaviour of the output. From eqn. (2.34), it is clear that the probability of the output at time k is determined from the state distribution at that time. Consequently, the stationary output distribution is given by X Prob(w) ¯ = Ha (w|¯ ¯ z ) · Prob(¯ z) , (2.60) z¯∈Nz

with Prob(¯ z ) denoting the stationary state distribution given by eqn. (2.50), and with Ha given by X Ha (w | z) = H(w | z, v) · Prob(v) (2.61) v∈Nv

under the same assumptions as for Lemma 2.5.5. For the limiting output distribution, the same comments as given above for the limiting state distribution apply.

2.6 Stochastic Operators** Equation (2.30) describes the stochastic relation between an input and an output sequence of the same length as determined by the SA. The stochastic operator introduced in the following generalises the stochastic relation between input and output sequences. Denote the sets of all input or output sequences of arbitrary length including sequences of length zero by Nv∗ =

∞ [

Nvk , or Nw∗ =

k=0

∞ [

Nwk ,

k=0

Nvk

Nwk

respectively, where or denote the sets of all input or output sequences of length k. A stochastic operator is defined as follows [23]: Definition 2.6.1. For given Nw , Nv the triple I(Nw , Nv , τ )

(2.62)

is called a stochastic operator if the mapping τ : Nw∗ × Nv∗ → [0, 1] defines a conditional probability of all sequences W ∈ Nw∗ and V ∈ Nv∗ : τ (W |V ) = Prob(W |V ) .

(2.63)

The operator is called a finite stochastic operator if Nv and Nw are finite sets.

34

2. Basics of Stochastic Automata Theory

In order to define a conditional probability, τ (W |V ) has to fulfil the condition X (2.64) τ (W |V ) = 1, ∀ V ∈ Nv∗ . ∗ W ∈Nw

For every given initial state probability distribution Prob(z(0)), an SA represents a stochastic operator. However, the converse is not true. Not every stochastic operator can be represented by an initialised SA. If the conditional probabilities τ (W |V ) of a stochastic operator are identical to those described by eqn. (2.30) for some given initial state distribution Prob(z(0)), the stochastic operator is called SA–operator, and the SA–operator is said to be realised by the SA initialised with Prob(z(0)). If the SA is finite, then the corresponding operator is called a finite SA–operator. The following lemma gives a formal criterion to decide whether a stochastic operator is an SA– operator or not. Lemma 2.6.1. [23] A stochastic operator I(Nw , Nv , τ ) is an SA–operator if and only if for all sequences V ∈ Nv∗ , W ∈ Nw∗ and for all v˜ ∈ Nv , the two conditions 1. |W | 6= |V | ⇒ τ (W |V ) = 0 X ˜ ) | (V v˜)) = τ (W |V ) τ ((W W 2.

(2.65) (2.66)

˜ ∈N ∗ W w

˜ ) ∈ N ∗ and (V v˜) ∈ N ∗ mean that the are satisfied, where the symbols (W W w v ˜ sequence W and the input symbol v˜ are appended to the sequences W or V respectively, and |V |, |W | denote the lengths of the sequences. Note that the two conditions do not imply that the SA–operator can be realised by an SA with a finite number of states. In Section 5.3.3, this result will be used to discuss the properties of the state observation algorithm.

2.7 Bibliographical Notes Sequential machines and automata have been intensively studied in the past century and are the foundations of computer science [34]. Several textbooks concern the theory of deterministic automata, also known as finite state machines, cf. e.g. [44], [67], [86]. In the sixties the theory of stochastic automata emerged [23], [25], [180], [184], [203]. The theory was developed on the background of unreliable switching circuits but found later various applications in modelling and optimisation [18], [184], [194]. From the theory of automata emerged the discrete–event systems theory which developed as a field of its own. This term subsumes a variety of different models which have in common that they are of symbolic nature. Well–known

2.7 Bibliographical Notes

35

discrete–event models are Petri nets [81], descriptions using max–plus algebras [10], condition–event networks [46], [85], [183] or binary systems [16]. These models are widely used in representation and control, e.g. to model batch processes in chemical industry [147], [167], to design supervisory controllers [155], [158] or for fault diagnosis [119], [163]. Several results of these theories are closely related to this book. However, no results of these works are used directly, because in detail the theories differ considerably. The presentation in this chapter is mainly based on the textbook [23], but a different notation more suitable for the application to qualitative modelling is introduced. Furthermore, the textbooks [67], [184] or [194] are used, where the first contains only nondeterministic rather than stochastic automata. In its basics, automaton theory is well elaborated. However, results on stochastic automata are mostly limited to fundamental properties because research in this field almost stopped at the end of the sixties. Accordingly, several new results on stochastic automata had to be elaborated in the preparation of this book. In particular, this concerns stochastic automata networks and the state observation and diagnosis of stochastic automata (cf. Chapters 5 and 7). A focus of this chapter is the study of the stationary behaviour of stochastic automata. The studies are based on results from Markov theory as presented in the textbooks [56], [87], [188] and [198]. The presentation therefore differs from that of textbooks on automata theory that study the stationary behaviour by using the language of automata or other theoretical concepts.

3. Modelling of Quantised Systems by Stochastic Automata

The quantised systems approach is introduced in this chapter in three steps. First, in Section 3.1, the main idea is outlined and the complete path from the definition of the quantised system to its qualitative model is shown under simplifying assumptions and by using the example of the two–tank system of Section 1.4. In the second step, the quantised system definition is extended in Sections 3.2 and 3.3. In Section 3.4, its behaviour is investigated in detail. The qualitative model is derived in Section 3.5, and its most important properties are outlined. The third step is a system theoretic study of the quantised system given in Section 3.6 which yields further properties.

3.1 Summary of the Modelling Approach 3.1.1 Quantised Systems According to Figure 1.1, a model to be used for supervision must describe the relation between the quantised signals [u(k)] and [y(k)]. The causal relation between these signals is derived as shown in Figure 3.1. The upper part of the figure shows the system from the viewpoint of the supervisor, where the quantised signals appear in a non–causal manner. The lower part of the figure shows the transformation to the causal quantised system which describes the causal relation between the quantised input and output signals. In the following, causality will not be explicitly mentioned if referring to the quantised system. From the context it is always clear whether it is referred to the causal or non–causal quantised system. In the following, the quantised system shown in the lower part of Figure 3.1 is studied. It consists of three blocks where the output quantiser and the continuous–variable system are those of Figure 1.1, and the input injector is the “causal inverse” of the input quantiser. The three blocks of the quantised system are described in the following. Continuous–variable system. The core of the quantised system is the discrete–time continuous–variable system J. Schröder: Modelling, State Observation and Diagnosis of Quantised Systems, LNCIS 282, pp. 37-94, 2003. Springer-Verlag Berlin Heidelberg 2003

38

3. Modelling of Quantised Systems by Stochastic Automata

u

Quantised system Discrete-time continuousvariable system

y

Quantiser

Quantiser [ y]

[u]

Causal quantised system [u]

Injector

u

Discrete-time continuousvariable system

y

Quantiser

[ y]

Fig. 3.1. Causal relation among the quantised signals.

x(k+1) = g(x(k), u(k)) ,

x(0) = x0

(3.1)

y(k) = h(x(k), u(k))

(3.2) n

with the state vector x ∈ Rn ⊆ IR , the input vector u ∈ Rm ⊆ IR output vector y ∈ Rr ⊆ IRr .

m

and the

Quantiser. The quantiser transforms the numerical output y(k) of the continuous–variable system into the quantised output [y(k)] ∈ Ny at time k, with Ny = {1, 2, . . . , R}. It is defined by a partition of the relevant part of the output space into bounded sets Qy (i) and such that the relation [y(k)] = i ⇔ y(k) ∈ Qy (i)

(3.3)

holds for all outputs. Example 3.1.1. Consider the two–tank system of Section 1.4 with the liquid levels as output. A partition of the relevant output space into nine sets is shown in Figure 3.2. In case of the two–tank system, the partition boundaries are chosen so as to coincide with the minimal and maximal tank levels and the positions of the discrete level sensors LS1–LS4. That is, if the quantised output at time k is for example [y(k)] = 3, it means that the liquid level in the left tank is between 0.4 and 0.6 metres and in the right one between 0 and 0.1 metre. Figure 3.3 shows the effect of the quantiser on a numerical output signal of the tank system. On the left–hand side the sequences of numerical values for y1 (k) and y2 (k) are shown. The quantisation introduced by the partition is shown by the light grey boxes at each time step. The quantiser transforms the numerical sequence Y (0, 1, . . . , 8) into the quantised sequence [Y (0, 1, . . . , 8)] = (9, 8, 8, 8, 8, 3, 3, 3, 1). This quantised sequence is shown on the right–hand side of Figure 3.3.

Injector. The injector is the causal inverse of the quantiser. This means, given a quantised input [u(k)] ∈ Nu = {1, 2, . . . , M }, the injector must determine a numerical input u(k). Like the quantiser, the injector is defined via a partition of the relevant part of the input space into sets Qu (i) that are

3.1 Summary of the Modelling Approach

39

y2 0.6 Qy(7)

Qy(8)

Qy(9)

Qu(1) 0

0.2 0.1 0 0

Qy(4)

Qy(5)

Qy(6)

Qy(1)

Qy(2)

Qy(3)

0.2

0.4

Qu(2) Qu(3) 0.4

0.7

1

u

0.6 y1

Fig. 3.2. Partition of the relevant output space (left) and input space (right) of the two–tank system.

Fig. 3.3. Quantisation of a numerical output signal of the two–tank system.

for the moment assumed to be bounded. The selection of a numerical input u(k) by the injector is according to the equivalence relation [u(k)] = i ⇔ u(k) ∈ Qu (i) .

(3.4)

This means that the injector must incorporate some mechanism to choose a numerical input from the set Qu (i). Therefore, the injector is described such that it randomly selects one numerical input of the set Qu ([u]) corresponding to the given quantised input [u]. This is a reasonable definition of the injector as it shall describe the inverse operation of quantisation. A quantiser is a memoryless system assigning to all numerical values of a partition set the same quantised value. Accordingly, the injector must also be a memoryless system and all numerical values of the partition set associated to the given quantised value must be possible outcomes of the injection. This leads naturally to the definition of the injector as static random process.

40

3. Modelling of Quantised Systems by Stochastic Automata

Example 3.1.2. Consider the pump velocity vp as input to the two–tank system. A partition of the scalar input space is shown on the right–hand side of Figure 3.2. Accordingly, if the quantised input at time k is for example [u(k)] = 1 the injector randomly chooses one input u(k) between 0 and 0.4 and gives it to the pump. It is assumed in this section that the injector chooses the values according to a uniform distribution, i.e. for [u(k)] = 1, all values in the interval [0, 0.4) can occur with the same probability. 3.1.2 Nondeterminism of the Quantised System Behaviour Consider the two–tank system of Section 1.4 without any knowledge about the initial tank levels and, for simplification, without input and the output equal to the state. Furthermore, the pump is off, the Valve V12L is open and the Valve V12H is closed. The output space partition is as shown in Figure 3.2 and the sampling time is 5 seconds. Assume that at time k = 0, the quantised output [y(0)] = 3 is obtained. That is, the liquid levels of the system at k = 0 are known to be represented by an element of Qy (3). The question investigated now is: What quantised outputs [y(1)], [y(2)], . . . will be obtained in the future?

Fig. 3.4. Nondeterminism of the quantised system.

The continuous–variable model (3.1)–(3.2) can answer this question only for single numerical initial conditions. However, here the initial condition is not a single point in the state space. The initial condition is unknown and from [y(0)] = 3 it can only be deduced that 0.4 ≤ x1 ≤ 0.6 and 0 ≤ x2 ≤ 0.1 hold. That is, a set of initial conditions must be considered. Hence, eqns. (3.1) and (3.2) must be applied to all points of the set Qy (3). This results in the sets of points at times k = 1, 2, . . . that are shown in Figure 3.4 as grey areas in the output space. Alternatively, these different behaviours can be viewed over time as done in Figure 3.5 where an ensemble of output signals up to k = 5 is shown for different numerical initial conditions all having the same

3.1 Summary of the Modelling Approach

41

Fig. 3.5. Ensemble of output signals with the same quantised initial condition.

quantised initial value [y(0)] = 3. By means of Figure 3.4, the answer to the above question can be given: at time k = 1 the quantised output [y(1)] can be either 5, 6, 8 or 9, at k = 2 it can be either 4, 5 or 8 and so on. That is, due to the uncertainty concerning the initial condition, different quantised outputs can occur in the future. The behaviour of the quantised system is nondeterministic [109]. It should be pointed out that this nondeterminism is a direct consequence of the quantised measurement of the liquid levels and does not mean that there is any uncertainty in the continuous–variable system. Another reason why the behaviour of the quantised system is nondeterministic is the injector. Because a quantised input is given to the system, the corresponding set Qu (i) of numerical input values must be considered for the behaviour of the quantised system. This is discussed in detail in Section 3.4. 3.1.3 Modelling Problem In the previous section the nondeterministic behaviour was outlined using the example of the tank system. It has been shown that the description of the behaviour of the quantised system requires to consider sets of inputs, outputs and states rather than single trajectories of the continuous–variable system. Even for the simple two–tank system, the precise description of these sets is very complex. Though a precise description of the behaviour of the quantised system is possible, the reason for the introduction of quantisation should be recalled at this point. The aim is to abstract from a numerical description of the system to a qualitative one in order to simplify the solution to process supervision tasks. A precise description of the complex behaviour of the quantised system contradicts this aim. Therefore, a concise approximate description of the quantised system in form of a stochastic automaton is elaborated in the following. The approximate model should fulfil the following aims.

42

3. Modelling of Quantised Systems by Stochastic Automata

Modelling aims. I) Simplicity. The approximate description should be simple. It should allow the formulation of applicable solutions to process supervision problems. II) Completeness. The description should be complete in the sense that it contains all quantised output sequences possibly generated by the quantised system. III) Convergence. The approximation should be such that the degree of precision is adjustable by a “resolution parameter”, such that its converges to a precise description of the quantised system for increasing resolution.

3.1.4 Abstraction of Quantised Systems to Stochastic Automata Using a stochastic automaton as approximate model of the quantised system means to set the automaton inputs, outputs and states as well as its behavioural relation so as to describe the quantised system. It appears natural to set the automaton input and output alphabets according to the inputs and outputs of the quantised system. That is, each automaton input is associated with one input of the quantised system [u] = v and equivalently for the output [y] = w. The automaton states are set such that each automaton state corresponds to a set of numerical states of the continuous–variable system. That is, like the input and output space, the relevant state space of the continuous–variable system is partitioned into N sets. As before, the partition is such that each quantised state [x(k)] ∈ Nx = {1, 2, . . . , N } corresponds to one bounded set Qx (i) in the state space. In summary, this means that the following is set: Nz := Nx , Nv := Nu , Nw := Ny , and [x] = z , [u] = v , [y] = w . With these settings, the SA is called a qualitative model of the quantised system. The states, inputs and outputs of the qualitative model are called qualitative states, qualitative inputs and qualitative outputs, respectively. With these settings it is clear that the behavioural relation must be chosen so as to describe the conditional probability of a change of the quantised state given a certain quantised input and obtaining a certain quantised output: þ   g(x, u) ∈ Qx (z 0 ), þþ x ∈ Qx (z), 0 L(z , w|z, v) = Prob . (3.5) h(x, u) ∈ Qy (w) þ u ∈ Qu (v) More precisely, the behavioural relation of the SA must be chosen according to

3.1 Summary of the Modelling Approach

43

Abstraction of the quantised system L(z 0 , w|z, v) :=

λn+m (Qxu (z 0 , w, z, v)) λn+m (Qxu (z, v))

(3.6)

with Qxu (z, v) = Qx (z) × Qu (v) = {(x, u) | x ∈ Qx (z), u ∈ Qu (v)} ⊂ IRn+m Qxu (z 0 , w, z, v)) = {(x, u) | x ∈ Qx (z), u ∈ Qu (v), g(x, u) ∈ Qx (z 0 ), h(x, u) ∈ Qy (w)} ⊆ Qxu (z, v) , where λn+m (·) denotes the Lebesgue measure of the sets that can be thought of as the sets hypervolume in the combined input and state space (cf. Appendix A.1). Equation (3.6) is referred to as abstraction of the quantised system because the equation describes how a concise model of the quantised system that is more abstract than the underlying numerical model is derived. As the remainder of this book will show, the described qualitative model in form of a stochastic automaton fulfils all of the modelling aims of Section 3.1.3. Simplicity of the model is given because the SA is a causal model with finite memory and a simple structure. As shown in Section 2.3, it is possible to iteratively determine the sets of state and output sequences. Such simple iterative solutions can also be found for process supervision problems as shown in Chapters 6 and 8. These solutions fundamentally rely on the completeness of the qualitative model. It will be shown in Section 3.5 that the qualitative model obtained by abstraction is a complete model of the quantised system, for arbitrarily chosen partition of the state space. The degree of precision is determined by the granularity of the state space partition. In Section 3.6, it will be shown that for increasingly finer partition of the state space, the qualitative model converges to a precise representation of the quantised system. Clearly, increasing granularity of the partition means that the number of automaton states, and hence, the complexity of the SA increases. In summary this means that the following proposition is satisfied. Fulfilment of the modelling aims Proposition 3.1.1. The qualitative model defined by eqn. (3.6) satisfies the three modelling aims simplicity, completeness, and convergence. The proposition is proved in the remainder of this book. The following example explains how a qualitative model of the two–tank system is derived by means of abstraction.

44

3. Modelling of Quantised Systems by Stochastic Automata

Fig. 3.6. Partition of the state space and automaton states (left), explanation of the transition probabilities (right).

Example 3.1.3. Consider the two–tank system of Section 1.4 with the sampling time Ts = 5s, without input and in the configuration where the pump is off (vp = 0), V12l open and V12u closed. That is, the two–tank system is autonomous and equation (1.8) has the form x(k+1) = g a (x(k))

x(0) = x0 .

The states of the two–tank system are the liquid levels. To simplify the example, the qualitative model is considered without output such that, for this example, eqn. (3.6) becomes Ga (z 0 |z) :=

λn (Qx (z 0 , z)) λn (Qx (z))

(3.7)

with Qx (z 0 , z) = {x | x ∈ Qx (z), g a (x) ∈ Qx (z 0 )} , where Ga (z 0 |z) is the transition relation of an autonomous SA without output (cf. Section 2.1). The state space partition can be chosen arbitrarily. Here, the state space is partitioned like the output space into nine subsets. Each subset is assigned to an automaton state as shown on the left–hand side of Figure 3.6. As example, the transition probabilities Ga (z 0 |3) from state z = 3 are calculated for all z 0 . On the right–hand side of Figure 3.6, the map of the set Qx (3) is shown. It can be seen that the states x of Qx (3) map either to Qx (5), Qx (6), Qx (8) or Qx (9). Depending on the map of the state, the set Qx (3) is thus partitioned into the four subsets Qx (3, 5), Qx (3, 6), Qx (3, 8) and Qx (3, 9). The measures of these sets are: λn (Qx (3, 5)) = 0.0156, λn (Qx (3, 8)) = 0.002,

λn (Qx (3, 6)) = 0.001 λn (Qx (3, 9)) = 0.0014 .

Note that to obtain these values it is necessary to map the set Qx (3) what means to evaluate eqn. (1.8) for all points of the set. With the above values and λn (Qx (3)) = 0.02, eqn. (3.7) yields: Ga (5|3) = 0.78, Ga (6|3) = 0.05, Ga (8|3) = 0.1, Ga (9|3) = 0.07 .

3.1 Summary of the Modelling Approach

45

0.32

73 0.

0.91

0.12

78 0.

0.43

0.02

45 0.

1

6

0.07

5

0.33

0.43

0.09

0.1

0. 02

0.01

0.01

0.33

0.35

9 0.25

0.27

2

0.05

0.17

0.1

8

0.73

4

0.48

01 0.

7

0.57

0.87

3

Fig. 3.7. Automaton graph of the qualitative model of the quantised two–tank system. It can further be seen from Figure 3.6 that none of the states in Qx (3) maps to any state in Qx (1), Qx (2), Qx (3), Qx (4) or Qx (7). Hence, Ga (z 0 |3) = 0 holds for the successor states z 0 = 1, 2, 3, 4 and 7. In the same way, all other transition probabilities are determined. The resulting qualitative model of the quantised two–tank system is shown in Figure 3.7.

3.1.5 Complexity of Qualitative Models An important issue is the complexity of qualitative models meaning the number of entries of the behavioural relation (3.6) that need to be determined and stored. Of particular interest are the nonzero transition probabilities as in a practical application only these values are stored [174]. It is useful to have a rough estimate for the number of expected nonzero entries of the behavioural relation. Such an estimate is derived is the following on a heuristical basis. The number of transition probabilities depends on various parameters. Most important are the number of qualitative states N and the number of qualitative inputs M . If the qualitative model was a deterministic automaton there would be a single successor state and output for each combination of the qualitative state and input. The number of transitions would thus be N · M . Due to the nondeterminism, however, there are, in general, various qualitative successor states and outputs. The practical experience shows that the number depends mainly on the dimensions of the state and output spaces. This motivates the approach to estimate the number |TL (z, v)| of successor r states and √ outputs for each z√∈ Nz , v ∈ Nv by |TL (z, v)| ≈ ηzn0 · ηw , where n r 1 ≤ ηz0 ≤ N and 1 ≤ ηw ≤ R are system dependent parameters reflecting

46

3. Modelling of Quantised Systems by Stochastic Automata

the average number of qualitative successor states and outputs per dimension of the state and output space, respectively. An estimate of the overall number of nonzero entries |TL | of the behavioural relation L is thus given by r |TL | ≈ N · M · ηzn0 · ηw .

(3.8)

As ηz0 = 1 or ηw = 1 mean a deterministic state or output behaviour these parameters are called nondeterminism parameters with respect to the successor state or output, respectively. The motivation for introducing the nondeterminism parameters ηz0 and ηw can be explained by means of Figure 3.6. On the right–hand side of the figure the map of a region of the state space is shown. In can be observed that the mapped region extends over two intervals per dimension of the state space. This is a typical situation making ηz0 = 2 and ηw = 2 a good estimate for the practical application. For the two–tank system of the previous section the estimate (3.8) yields |TL | ≈ 9 · 1 · 22 · 20 = 36. This is a reasonable estimate if compared to the real number of nonzero transitions probabilities which is 28 (cf. Figure 3.7). Further examples throughout the book will show that the estimate (3.8) yields good results. Equation (3.8) reflects another property of qualitative models. To show this, let N¯i be the average number of qualitative states per dimension of the ¯ i the corresponding number for the qualitative input. Then state space and M eqn. (3.8) reads ¯ i )n · (M ¯ i )m · η n0 · η r , |TL | ≈ (N z w showing that the complexity of the qualitative model increases exponentially with the dimensions of the continuous–variable system. For example a fifth order system with a three dimensional input and output space and five qualitative values per dimension would yield a qualitative model with approximately |TL | ≈ 108 nonzero entries. To store and determine such an enormous number of probabilities means a high computational effort and is near to the practical limitation brought about by current computer technology. For such or even more complex systems, a component–oriented qualitative modelling approach as presented in Chapter 8 is compulsory.

3.2 An Extended Introduction to Quantised Systems 3.2.1 Definition of Quantised Systems Continuous–variable system. As before, the core of the quantised system is the discrete–time continuous–variable system

3.2 An Extended Introduction to Quantised Systems

x(k+1) = g(x(k), u(k)) ,

47

x(0) = x0

y(k) = h(x(k), u(k)) with the state vector x ∈ Rn ⊆ IRn , the input vector u ∈ Rm ⊆ IRm and the output vector y ∈ Rr ⊆ IRr . Quantiser. The definitions of both, the injector and the quantiser are extended in this section. They are defined in the following on an arbitrary space Rν ⊆ IRν of signal values ζ(k), where ζ(k) stands for the input or output signal, and is used in Section 3.2.4 also for fault signals. The quantiser invokes a surjective mapping and is defined via a partition of the signal space Rν into sets Qζ (i), i = 1, . . . , ξ by [·] : Rν → Nζ ,

[ζ] = i ⇔ ζ ∈ Qζ (i)

(3.9)

with Nζ = {1, 2, . . . , ξ}. The partition is such that the Lebesgue measure λν in Rν of all sets is nonzero: λν (Qζ (i)) > 0 ,

i ∈ Nζ .

(3.10)

This means that the sets Qζ (i) must not lie completely in a subspace of Rν . For example Qζ (i) must not be a line or a single point in IR2 . In contrast to the introduction of the quantiser in Section 3.1.1, the whole signal space is partitioned and the sets Qζ (i) can be unbounded (cf. Figure 3.8). Remark 3.2.1. The definition of the quantiser allows to divide the signal space

into arbitrarily shaped sets. Practically, the input and output signals of a system are usually quantised independently of each other. For example in case of the two– tank system of Section 1.4, the quantisation of the output space is defined by the positions of the discrete level sensors attached to each tank. Both tank levels are thus quantised independently of one another. The resulting partition is called orthogonal partition as the bounds separating the partition regions are orthogonal to each other. For the two–tank system, the output space partition resulting from the positions of the discrete–level sensors is shown in Figure 3.8.

Remark 3.2.2. Condition (3.10) means that single points in a continuous signal space cannot be assigned an own quantised value. This restriction is necessary to obtain well defined transition probabilities (cf. Section 3.5.2). However, in particular for the input and fault spaces, such singular partition sets can be useful. In this case, the restriction (3.10) can be circumvented by introducing discrete–valued dimensions as explained in Section 3.3.3. Injector. As explained above, the injector needs to realise an injective mapping Nζ → Rν . Obviously, this mapping is not well defined as the choice of a numerical value ζ(k) for given quantised value [ζ(k)] merely fulfilling the equivalence relation [ζ] = i ⇔ ζ ∈ Qζ (i) is not unique but random. Hence, for the definition of the injector, this random choice must be specified more precisely. This is done by defining the probability with which a certain value ζ(k) of Qζ ([ζ(k)]) occurs. That is, instead of the partition of the signal space, a set of non–overlapping probability density functions1 piζ : Rν → IR+ , with 1

The space of all density functions is defined in Appendix A.1.

48

3. Modelling of Quantised Systems by Stochastic Automata

y2 0.6

0.2 0.1 0

Qy(7)

Qy(8)

Qy(9)

Qy(4)

Qy(5)

Qy(6)

Qy(1)

Qy(2)

Qy(3)

0

0.2

0.4

0.6

y1

Fig. 3.8. Orthogonal output space partition of the two– tank system including unbounded sets.

piζ (ζ)

Z = 0 ⇔ ζ 6∈ Qζ (i)

and

Rν

piζ (ζ)dζ = 1

, i = 1, 2, . . . , ξ (3.11)

must be given to define the injector. In other words, the injector is a static random process defined by the set of probability density functions, randomly choosing a value ζ out of Qζ (i) for given i. Static means here that the numerical value ζ selected at time instance k does not depend on previously selected values and that the densities piζ do not depend on the time instance k. Furthermore, the random process is assumed to be independent of all other signals of the quantised system. The density functions piζ depend on the properties of the real system. For simplification, uniform distributions  1 for ζ ∈ Qζ (i) piζ (ζ) = λν (Qζ (i)) , i = 1, . . . , ξ (3.12) 0 otherwise are often assumed for all bounded sets. On unbounded sets for example declining exponential functions can be used as density functions. An example of the definition of an injector is given in the next section. The above definition of the injector by means of probability density functions might not seem intuitive. However, it is clear that if merely a region is specified, as in the quantised system, the numerical value can be anywhere in that region. The probability density function allows to describe which numerical values within that region are more or less likely to occur, which is a very general approach. In summary, the quantised system is given as shown in the box on page 49. Remark 3.2.3. Note that according to the definition of the injector, the numerical value can change its value over time even if the quantised value is constant. This is

3.2 An Extended Introduction to Quantised Systems

49

Definition of the quantised system [u (k)]

Quantised system x(k)

[ y (k)]

The quantised system is defined as tuple QS(Rn , Rm , Rr , Nu , Ny ,{p1u , ..., pM u },{Qy (1), ..., Qy (R)}, g, h) . (3.13) Its causal behaviour for arbitrary initial state x(0) = x0 is described as follows: input [u(k)] ∈ Nu yields a random input vector u(k) according to [u(k)] the probability density function pu (u) depending on the randomly selected input: successor state: x(k+1) = g(x(k), u(k)) numerical output: y(k) = h(x(k), u(k)) output [y(k)] ∈ Ny with [y(k)] satisfying y(k) ∈ Qy ([y(k)])

because the injector is a static system, i.e. a system without memory, and, therefore, its output cannot depend on previous values. However this does not imply any limitation, as such dynamical effects of the injector could be transformed to the continuous–variable system.

3.2.2 Interpretation of Quantised Systems as Discrete Stochastic Processes According to the definition of the quantised system, the output symbol [y(k)] cannot be uniquely determined from [u(k)] but depends upon the random choices of the current and the previous numerical inputs u. That is, the quantised system is a discrete stochastic process, as the quantised output [y(k)] at time k is given as a discrete random variable (cf. Section 2.2, Appendix A.2). In Section 3.1.2 this has been referred to as the nondeterminism of the quantised system. The above definition of the quantised system describes how a realisation of the stochastic process in terms of a single quantised output sequence is obtained for an unique initial condition x(0) = x0 . For studying the quantised system, however, it is not realistic to assume that the initial state is precisely known (cf. Section 3.1.2). This is firstly because the state cannot always be directly measured and, secondly, because quantised rather than numerical information is available. That is, the available information concerns merely a set Qy (i) of possible outputs from which, even if considering subsequent

50

3. Modelling of Quantised Systems by Stochastic Automata

quantised measurements, in general, no unique numerical state can be derived [36]. Hence, it must be considered that only a set of initial states is known. Therefore, concerning the initial condition of the quantised system, it must be assumed that only a subset X 0 of the state space or, more generally, an initial probability density function Z + n px0 : R → IR with px0 (x) = 0 ⇔ x 6∈ X 0 and px0 (x)dx = 1 (3.14) Rn

is known. The actual numerical initial state x0 of the embedded continuous– variable system can be any state within the set X 0 and is determined randomly according to px0 (x). That is, in addition to the random choices of the numerical inputs by the injector, the imprecisely known initial state introduces another random variable to the stochastic process represented by the quantised system. In the following, the density function px0 (x) is simply referred to as initial condition of the quantised system. For a given initial condition, the quantised system is also referred to as initialised quantised system. 3.2.3 Representation Problems The quantised system is a stochastic process and characterised by the probabilities with which the different quantised output symbols [y(k)] occur. The aim is to find these probabilities for a given sequence of quantised input symbols [U (0 . . . k)] = ([u(0)], . . . , [u(k−1)]) ∈ Nuk+1

(3.15)

and initial state probability density function px0 (x) or to determine the set of output symbols for which this probability is nonzero. These tasks are referred to as representation problems of quantised systems. Stochastic representation problem of quantised systems. Determine the conditional probability distribution Prob([y(k)] | px0 (x), [U (0 . . . k)])

(3.16)

describing the probabilities of the output symbols [y(k)] at time k, for given symbolical input sequence [U (0 . . . k)] and initial density px0 (x). Set representation problem of quantised systems. Determine the set [Y(k | X 0 , [U (0 . . . k)])] ⊆ Ny

(3.17)

of currently possible output symbols [y(k)] at time k, for given symbolical input sequence [U (0 . . . k)] and initial set of states X 0 .

3.2 An Extended Introduction to Quantised Systems

51

fault [ e (k)] Quantised system for diagnosis

Injector e (k)

[u (k)]

Injector

u (k) x (k+1) = g( x (k), u(k), e(k)) y (k) y (k) = h( x(k), u(k), e(k))

Quantiser

[ y (k)]

Fig. 3.9. Quantised system for fault diagnosis.

The solution to both representation problems is considered in Sections 3.4 and 3.6. The solution to the set representation problem presents a simplified view on the quantised system. Instead of describing the probabilities of the output symbols, it is merely considered whether or not a certain quantised output can occur. That is, sets of possible outputs are derived without assigning a probability to them. In Section 3.4.2, this simplified viewpoint is extended to a stochastic description on an intuitive level and only for autonomous systems. A complete solution to the stochastic representation problem requires several measure theoretic concepts and is, therefore, presented separately in Section 3.6. 3.2.4 Quantised Systems for Fault Diagnosis If the quantised system is to be considered for fault diagnosis, the influence of the faults upon the system must be described. This is done by introducing the fault as additional input e to the continuous–variable system x(k+1) = g(x(k), u(k), e(k)) ,

x(0) = x0

y(k) = h(x(k), u(k), e(k)) ,

(3.18) (3.19)

s

with the fault vector e ∈ Rs ⊆ IR . This means that the quantised fault [e(k)] ∈ Ne = {1, 2, . . . S} becomes an additional input to the quantised system (cf. Figure 3.9). The transformation between the numerical fault values e(k) and the quantised fault values [e(k)] is performed by a second injector. The sets resulting from the partition of the fault space are denoted by Qe (i) and the density functions by pie (e). In summary, the quantised system with fault is given by the tuple QS(Rn , Rm , Rr , Rs , Nu , Ny , Ne , {p1u , . . . , pM u },

{p1e , . . . , pSe }, {Qy (1), . . . , Qy (R)}, g, h) .

(3.20)

The introduction of the fault as additional input is a very general approach for describing faults. The fault vector e(k) appears in eqns. (3.18) and (3.19) in the nonlinear vector functions g and h. This is more general than most

52

3. Modelling of Quantised Systems by Stochastic Automata

assumptions used in the literature on fault diagnosis (cf. e.g. [11]) and will not be further restricted in the remainder of the book. In the literature on fault diagnosis, often linear systems with additive faults acting upon the inputs, states or outputs, or faults in form of parameter deviations of a linear model are considered. Models with such faults are of course special cases of eqns. (3.18) and (3.19). Note that eqns. (3.18) and (3.19) also allow to describe faults that cause structural changes of the system or changes of system order because the fault vector can be used to switch between different sets of system equations. Hence, systems given by sets of differential equations switched depending upon the fault can also be described. Example 3.2.1. Consider the two–tank system of Section 1.4 where the fault is a

leak in the left tank. The size of the leak is parameterised by the variable cl ≥ 0 which, therefore, is the fault variable e = cl /(10−4 m5/2 /s) of the scalar fault space Rs = IR+ . The fault space is partitioned into Qe (1) = [0, 0.5) ,

Qe (2) = [0.5, 2.5) ,

Qe (3) = [2.5, 8.5) ,

s

Qe (4) = R \ (Qe (1) ∪ Qe (2) ∪ Qe (3)) = [8.5, ∞) , with the semantics that [e] = 1 means no leak, [e] = 2 a small leak, [e] = 3 a big leak, and [e] = 4 a very big leak. Furthermore, the following density functions are used:   2 for e ∈ [0, 0.5) 0.5 for e ∈ [0.5, 2.5) p1e (e) = p2e (e) = 0 otherwise 0 otherwise p3e (e) =



1/6 for e ∈ [2.5, 8.5) 4 pe (e) = 0 otherwise p1e (e),

p2e (e)



exp(−e + 8.5) for e ≥ 8.5 0 otherwise

.

p3e (e)

and describe uniform distributions while The density functions R the density function p4e (e) is a declining exponential function with Rs p4e (e)de = 1 (cf. eqn. (3.11)). The partition and the density functions defining the fault injector are shown in Figure 3.10.

Remark 3.2.4. Though the density functions required for the definition of the injector are an inherent part of the technological system reflecting the nature of the fault or input, for practical cases, they often cannot be determined systematically. Therefore, usually uniform distributions like in this example are used. This is because the uniform distribution does not distinguish faults in the set Qe (i), i.e. none of the values of Qe (i) is more likely than another, and contains, from the viewpoint of information theory, no additional information compared to the partition. Extended input space. Because the fault is introduced as an additional input, it needs no extra treatment in the analysis of the quantised system. The difference between an input and a fault becomes relevant only when using the quantised system approach for fault diagnosis but is irrelevant to the analysis of its behaviour. Formally, the fault and input of the quantised system can be lumped together yielding a quantised system without fault ˜ = (u e)0 ∈ Rm+s . Accordingly, the resulting extended and with the input u input space Rm+s is partitioned into M · S sets each assigned to a quantised ˆu = {1, 2, . . . , M · S}. The partition sets in the ˜] ∈ N extended input value [u extended input space are defined as

3.3 Further Aspects of Quantised Systems*

pe 2

53

pe1(e)

1

p4e(e)

pe2(e)

0.5 0.167 0 0.5

pe3(e) 2.5

[ e]=1 [e]=2

8.5 [e ]=3

[e]=4

e e

Fig. 3.10. Definition of the fault injector of the two–tank system.

Qu˜ (Mue (i, j)) = Qu (i) × Qe (j) ⊆ Rm+s

i ∈ Nu , j ∈ N e

(3.21)

with an arbitrary bijective map ˆu Mue : Nu × Ne → N

(3.22)

assigning a unique number to each set of the partition. This is the reason why for the remainder of this chapter the quantised system is considered without fault.

3.3 Further Aspects of Quantised Systems* 3.3.1 Other Abstractions of Numerical Signals* In the literature on qualitative modelling, different approaches to abstract symbolical values from numerical signals are known (e.g. [33], [35], [89], [109], [141]). Described above and used throughout the book is the signal quantisation as suggested for qualitative modelling in [109]. This quantisation is static, i.e. the quantised value of the signal at each time point depends on the numerical signal at that time independently of its previous value. However, there are several, not necessarily static, alternatives to obtain an abstract qualitative signal from a numerical signal. Some of them are briefly outlined below. Obviously, a different definition of the quantiser also means a different definition of its causal inverse, the injector. Accordingly, an analytical description of the quantised system requires the formulation of a quantisation and injection procedure. Then, the quantised system approach can lead in a similar way as described in Sections 3.1 and 3.5 to a qualitative model. For the quantisation techniques described below, this has not yet been investigated.

54

3. Modelling of Quantised Systems by Stochastic Automata µ (ζ )

µ (ζ )

1

1

0

0 0.1 0.2 Qζ(1) Qζ(2)

0.6

ζ

0

QζF(1) QζF(2)

0 0.1 0.2

supp(QζF(1))

Qζ(3) ζ

F

Qζ (3)

0.6 F supp( Qζ (3))

F

supp( Qζ (2))

ζ ζ

Fig. 3.11. Quantisation based on crisp sets (left) and fuzzy sets (right).

Fuzzy quantisation. This well known quantisation technique is similar to the static signal quantisation used here. The main difference is that fuzzy sets are used that associate to each numerical signal value a continuous–variable degree of membership, typically between zero and one [141], [206]. Furthermore, in fuzzy quantisation, the fuzzy sets are usually not disjoint but can be overlapping. In the terminology of fuzzy sets, the sets Qζ (i) obtained from a partition of the signal space are referred to as crisp sets [141]. Figure 3.11 shows such crisp sets in their fuzzy interpretation with a membership µF (ζ) of one on the partition sets Qζ (i) and zero otherwise. On the right–hand side of the figure, the extension to fuzzy sets with a membership between zero and one is shown. It can be seen that the supports supp(QF ζ (i)) of the fuzzy sets, i.e. the values of ζ for which the corresponding fuzzy set yields a nonzero membership, are overlapping. Based on fuzzy sets, the quantiser can be defined as a map from a numerical signal value ζ to a “quantised” vector [ζ]F containing the degrees of membership to each fuzzy set. In Fuzzy theory, this vector is usually denoted as membership vector µF . The inverse quantisation then needs to assign to a “quantised” vector [ζ]F a numerical value ζ. In fuzzy control this operation is typically defined deterministically by e.g. the centre of gravity or centre of sums methods [206] rather than stochastically like the injector used here. The terms used in fuzzy control for the quantisation and injection are fuzzification and defuzzification, respectively [141]. Confluences. Confluences are qualitative differential equations that were introduced in [35] as an approach towards a qualitative physics. To obtain a confluence, all signals and parameters of a differential equation are converted to qualitative values. Each variable ζ of the differential equation can only take a qualitative value of the quantity space [ζ] ∈ {−, 0, +}. These qualitative values are introduced by defining the sets Qζ (−) = {ζ | − ∞ < ζ < 0}, Qζ (0) = {0} and Qζ (+) = {ζ | 0 < ζ < ∞}. This abstraction from a numerical signal to the three qualitative values {−, 0, +} is in principal a special case of the static signal quantisation of Section 3.2.1. By using the relation between the change of the sign of a signal and its derivative, the qualitative

3.3 Further Aspects of Quantised Systems*

55

system behaviour can be obtained by solving the confluences derived from the systems differential equations [110]. However, due to the limitation on partitions into three regions and the restriction to models in form of confluences, this approach yields, in general, numerous spurious solutions [110]. That is, a qualitative simulation yields many trajectories that cannot occur in the real system. Episodes. Based on the qualitative values used to obtain confluences, a qualitative state of a scalar numerical continuous–time signal ζ(t) is defined as triplet of the qualitative values {−, 0, +} of ζ and its temporal derivatives ζ˙ and ζ¨ at a certain time instance [28]. A time interval in which this triplet does not change is called an episode. If any of the three elements changes its value, a new episode starts. In [28] seven types of episodes also referred to as A–G type episodes are identified from the physically reasonable combinations of ˙ and [ζ]. ¨ The qualitative value of the signal itself is denoted as superscript. [ζ] A quantiser based on episodes thus transforms a numerical continuous–time signal into a sequence of episodes like (A+ B + C + C 0 C − . . .) as described in [166]. Further quantisations. The quantisation can also be realised by more complex operations. Every operation that abstracts from a signal with detailed information to a qualitative signal can be interpreted as quantisation. An example of a more abstract quantisation is presented for a practical application in [164]. The quantiser used there includes a spectral analysis of a signal and the quantised values result from evaluating the peak signal value and its peak spectral amplitude in a certain time window. Another example of a more complex definition of a quantiser are chronicles [33]. A chronicle is a combination of a short sequence of events and time constraints between them. The quantiser then is described by a chronicle detection algorithm converting a timed discrete–event signal into a sequence of chronicles [40]. 3.3.2 Temporally Quantised Continuous–Time Systems* Many systems, like the two–tank system in Section 1.4, are continuous–time systems that can be represented by a continuous–time, continuous–variable state space model ˙ ˆ (x(t), u(t)) , x(t) =g y(t) = h(x(t), u(t))

x(0) = x0

(3.23) (3.24)

with the state x, input u and output y as defined above. From such a system, a discrete–time system in the form of eqns. (3.1) and (3.2) is obtained by sampling, i.e. by considering the values of the states and outputs only at certain time points t0 , t1 , . . . , tk , . . ., combined with the assumption of a constant input between subsequent time points. Throughout the book, equidistant temporal quantisation tk = k · Ts , k ∈ IN with a fixed sampling time

56

3. Modelling of Quantised Systems by Stochastic Automata

Quantised system [u (k)]

Injector

Sampled system

u (k)

Ts

u(t) x. (t) = g ( x (t), u (t)) y(t) y (t) = h ( x (t), u (t))

y (k)

Quantiser

[ y (k)]

kTs Clock k

Fig. 3.12. Sampled quantised system with zero order hold.

Ts > 0 is considered. Then the relation between the models (3.23)–(3.24) and (3.1)–(3.2) is given by [17]: Z (k+1) Ts ˆ (x(τ ), u(τ ))dτ = g(x(k Ts ), u(k Ts )) .(3.25) g x((k+1) Ts ) = x(k Ts )+ k Ts

Figure 3.12 shows the resulting sampled quantised system. Though the existence of a function g of the discrete–time model correspondˆ of the continuous–time model is guaranteed under the ing to the function g above assumptions, it is usually difficult to derive. For linear systems ˙ x(t) = Ax(t) + Bu(t) ,

x(0) = x0

with non–singular system matrix A and input matrix B, it is well known that eqn. (3.25) yields x(k+1) = g(x(k), u(k)) = exp(ATs )x(k) + (exp(ATs )−I)A−1 Bu(k)(3.26) where I is the identity matrix [196]. For nonlinear systems, there are different approaches to approximate g e.g. by means of uniformly convergent Volterra or Taylor–Lie series (cf. e.g. [71], [77], [79], [197]). Remark 3.3.1. A different temporal quantisation method is event–based sampling [7], [138]. In event–based sampling the time points tk are set when the system trajectory crosses a certain hyperplane in the state space. Under a zero order hold assumption, the successor state can also be determined by means of eqn. (3.25) resulting in a discrete–time model (3.1)–(3.2), provided that an event determining tk+1 occurs. However, the index k of the obtained discrete–time model has no longer a temporal meaning, but describes merely the causal ordering of the events. The analogy between periodic and event–based sampling concerning the qualitative modelling approach is described in [117]. 3.3.3 Extension to Discrete Signal Spaces** The quantised system, in particular the quantiser and the injector have been introduced in Sections 3.1.1 and 3.2.1 for continuous signal spaces Rν . It is useful, however, to extend this to signals spaces Rν × N η with N η ⊆ ZZη , i.e. signal spaces possibly including discrete sets, where ν = 0 or η = 0 cover

3.3 Further Aspects of Quantised Systems*

57

the purely continuous or discrete cases, respectively. The following extends the definitions of the quantiser and injector. By this extension, it is also possible to circumvent the restriction (3.10) by reformulating the system equations (3.1) and (3.2) such that single points of a continuous set are represented by a value of a discrete set. An example of this will be given in Section 6.1.5. Extended quantiser. In analogy to Section 3.2.1, the quantiser invokes a surjective mapping and is defined via a partition of the signal space into disjoint sets Qζϕ (i) by [·] : Rν × N η → Nζϕ ,

[(ζ, ϕ)] = i ⇔ (ζ, ϕ) ∈ Qζϕ (i)

(3.27)

with Nζϕ = {1, 2, . . . , ξ}. As before, the sets Qζϕ (i) must have nonzero measure (cf. condition (3.10)), taking into account the product measure space of the measure spaces (Rν , B, λν ) of the continuous and (N η , C, γη ) of the discrete subspaces (cf. Appendix A.1). Herein, the measure of a set Q = Qν × Q η

with

Qν ∈ B(Rη ), Qη ∈ C(N η )

is given by λνη (Q) = λν (Qν ) · γη (Qη ) ,

(3.28)

where γη is the counting measure, i.e. the number of elements of Qη . Extended injector. The extended injector realises an injective mapping Nζϕ → Rν × N η . The main difference concerns the set of probability density functions piζϕ : Rν × N η → IR+

(3.29)

for which in extension to eqn. (3.11), the relations piζϕ (ζ, ϕ) = 0 ⇔ (ζ, ϕ) 6∈ Qζϕ (i) and

X Z ϕ∈N η

Rν

piζ (ζ, ϕ)dζ = 1

(3.30)

(3.31)

must hold for all i ∈ Nζϕ . Example 3.3.1. Consider the two–tank system of Section 1.4 and two faults: a leak in the left tank or the valve V12l being stuck. The fault space is thus two dimensional. The leak defined like in the example of Section 3.2.4 becomes the first fault variable: e1 = cl /(10−4 m5/2 /s), e1 ∈ IR+ . The second fault variable e2 is discrete valued, as the magnetic valve can be either faultless (e2 = 0), stuck opened (e2 = 1) or stuck closed (e2 = 2). Accordingly, the fault space is IR+ × {0, 1, 2}. Figure 3.13 shows a partition of the fault space. The partition is not orthogonal, i.e. the dimensions of the signal space are not partitioned independently. The semantics of the partition sets shown in the figure could be for example:

58

3. Modelling of Quantised Systems by Stochastic Automata

e2

Qe (5)

2 Qe (3)

Qe (4)

1 0

Qe (2)

Qe (1) 0 0.5

2.5

8.5

e1

Fig. 3.13. Non orthogonal partition of a continuous/discrete fault space. Qe (1) Qe (2) Qe (3)

: : :

no fault, leak left tank, valve V12l faulty,

Qe (4) Qe (5)

: :

uncritical multiple fault, critical multiple fault.

As example of a density function for the definition of the fault injector, consider: p4e : IR+ × {0, 1, 2} → IR+  2/11   8/11 4 pe (e1 , e2 ) =   1/11 · exp(−e1 + 8.5) 0

for e1 ∈ [0.5, 2.5), e2 = 2 for e1 ∈ [0.5, 8.5), e2 = 1 for e1 ≥ 8.5, e2 = 1 otherwise .

It can be verified that p4e satisfies condition (3.31).

3.3.4 Remark on Causal Relations** Remark 3.3.2. The quantised system introduced above describes the causal relation between the signals connected to the supervision block (cf. Figure 1.1). A non–causal way of modelling a dynamical system is described by the behavioural approach, cf. [97], [154], [201]. In contrast to a causal model, this approach focuses on the behaviour of the system, that is, the model contains all pairs of input and output signals that can occur. An example of these different views on the system are the SA and the stochastic operator introduced in Section 2.6. The SA describes a causal relation between the input and the output sequence. That is, the SA describes the set of possible output sequences for given input sequence while all input sequences are admissible. On the other hand, the stochastic operator assigns to each pair of an input and an output sequence a probability, or, in other words, it represents the set of pairs of input and output sequences the system can show. The problem of the behavioural approaches is that, in general, infinitely many pairs exist and would need to be stored, unless a generator for the pairs belonging to the behaviour can be given. This generator is then in most cases again a causal relation, like an initialised SA generates an SA–operator.

3.4 Solution to the Representation Problems

59

3.4 Solution to the Representation Problems This section concerns the solution to the representation problems of the quantised system posed in Section 3.2.3. The presentation is limited to quantised systems without fault, as in case of quantised systems with fault these can be treated by extending the input space (cf. Section 3.2.4). 3.4.1 Solution to the Set Representation Problem For a single sequence of numerical input vectors U (0 . . . k−1) = (u(0), . . . , u(k−1)) ∈ (Rm )k

(3.32)

and a unique initial state x0 ∈ X 0 , eqn. (3.1) of the continuous–variable system yields a unique state trajectory X(0 . . . k) = (x0 , x(1), . . . , x(k)) ∈ (Rn )k+1 .

(3.33)

Depending on (3.32) and (3.33), this leads to a unique output sequence Y (0 . . . k−1) = (y(0), . . . , y(k−1)) ∈ (Rr )k

(3.34)

of the continuous–variable system which is determined by eqn. (3.2). From this sequence, the output quantiser given by eqn. (3.3) allows to determine the resulting quantised output sequence (cf. Figure 3.3): [Y (0 . . . k−1)] = ([y(0)], . . . , [y(k−1)]) ∈ Nyk .

(3.35)

If, however, a quantised input sequence [U (0 . . . k − 1)] is considered, the injector randomly determines one numerical input sequence (3.32), merely ensuring that eqn. (3.4) holds at each time instance κ from 0 to k−1. Considering all possible numerical input sequences and all initial conditions x0 ∈ X 0 simultaneously results in the set of all state sequences up to time k X (0 . . . k | X 0 , [U (0 . . . k−1)]) (3.36)  þ  þ x0 ∈ X 0   þ = X(0 . . . k) þþ u(κ) ∈ Qu ([u(κ)]), κ = 0, . . . , k−1 ⊆ (Rn )k+1 .  þ eqn. (3.1) holds for κ = 0, . . . , k−1  This set, in contrast to (3.33) is not a singleton. Similarly, the set of output sequences can be written as Y(0 . . . k−1 | X 0 , [U (0 . . . k−1)]) (3.37)  þ  þ x0 ∈ X 0   þ = Y (0 . . . k−1) þþ u(κ) ∈ Qu ([u(κ)]), κ = 0, . . . , k−1  þ eqns. (3.1) and (3.2) hold for κ = 0, . . . , k−1 ⊆ (Rr )k .

60

3. Modelling of Quantised Systems by Stochastic Automata

Quantisation then yields the set of quantised output sequences for given set of initial states and given quantised input sequence, describing the causal behaviour of the quantised system: (3.38) [Y(0 . . . k−1 | X 0 , [U (0 . . . k−1)])] þ   þ ∃Y (0 . . . k−1) ∈ Y(0 . . . k−1|X 0 , [U (0 . . . k−1)]) þ = [Y (0 . . . k−1)] þ such that y(κ) ∈ Qy ([y(κ)]) holds for κ = 0, ..., k−1 ⊆ Nyk . From the above sets, the projection to the current time k can be derived. For the set of state sequences, this is (3.39) X (k | X 0 , [U (0 . . . k−1)]) = {x | ∃X(0 . . . k) ∈ X (0 . . . k|X 0 , [U (0 . . . k−1)]) such that x(k) = x} . Alternatively, this set can be determined recursively by k > 1 : X (k | X 0 , [U (0 . . . k−1)])  þ  þ x = g(˜ x, u) , for some u ∈ Qu ([u(k−1)]) = x þþ ˜ ∈ X (k−1 | X 0 , [U (0 . . . k−2)]) and x

(3.40)

k = 1 : X (1 | X 0 , [u(0)])

(3.41)

˜ ∈ X 0} . = {x | x = g(˜ x, u) , for some u ∈ Qu ([u(0)]) and x Equivalently, the set of current outputs can be obtained from projection to time k or from the set of states at time k by using eqn. (3.2) and the current quantised input: (3.42) Y(k | X 0 , [U (0 . . . k)]) = {y | ∃Y (0 . . . k) ∈ Y(0 . . . k|X 0 , [U (0 . . . k)]) such that y(k) = y} = {y | y = h(x, u), u ∈ Qu ([u(k)]), x ∈ X (k | X 0 , [U (0 . . . k−1)]) . This finally yields the solution to the set representation problem. Solution to the set representation problem of quantised systems The set of possible quantised outputs at time k for given initial set of states X 0 and sequence of quantised inputs [U (0 . . . k)] is given by [Y(k | X 0 , [U (0 . . . k)])] (3.43) þ ÿ  = [y(k)] þ Y(k | X 0 , [U (0 . . . k)]) ∩ Qy ([y(k)]) 6= ∅ ⊆ Ny , with the set Y(k | X 0 , [U (0 . . . k)]) given by eqn. (3.42) and the set of states therein determined recursively by eqns. (3.40) and (3.41). The above solution to the set representation problem is implicit. To determine the set (3.43) first, the sets (3.39) and (3.42) need to be derived. These sets,

3.4 Solution to the Representation Problems

61

Fig. 3.14. Sets of states and outputs with initial state uncertainty.

however, are not explicitly given by the above equations but they are defined “point–by–point” referring separately to each element of the set of initial states and to each input sequence compatible with the given quantised inputs. That is, eqn. (3.43) is merely a formal description of the sought set rather than a tool to practically determined this set. Example 3.4.1. Consider the two–tank system of Section 1.4 with the pump ve-

locity vp as input and the outflow qˆ2 of the right tank as output. The sets of states (3.39) are shown for k = 0, . . . , 3 on the left–hand side of Figure 3.14 with X 0 = { (x1 , x2 ) | 0.4 ≤ x1 ≤ 0.6, 0 ≤ x2 < 0.1 }

(3.44)

and u = 0, i.e. without any input uncertainty. Mapping these sets point–by–point to the output space yields the sets (3.42) shown on the right–hand side of this figure. The horizontal lines show the bounds of the output partition. From the figure the sequence of sets [Y(k | X 0 )] is obtained as [Y(k | X 0 )]

k=0 {1, 2}

k=1 {2}

k=2 {2, 3}

k=3 {2, 3}

k=4 {2}

k=5 {2} .

It can be seen that at times k = 0, 2 and 3 different quantised outputs can occur. The second source of uncertainty is the input injector. For the two–tank system this is shown on the left–hand side of Figure 3.15. Here, in contrast to Figure 3.14, the initial condition is set to a single value x0 ∈ X 0 , whereas the input is considered in the interval [0, 0.4) meaning low pump power. The sets (3.42) are depicted for k = 0, . . . , 5. As the system has no direct feedthrough, the output is a singleton at k = 0, and becomes a set with nonzero measure for k ≥ 1. It can be seen that the input uncertainty causes nondeterministic quantised outputs at times k = 2, 3 and 4. Finally, on the right–hand side of Figure 3.15, both uncertainties are combined. From this figure the sequence of sets [Y(k | X 0 , [U (0 . . . k)])] is obtained as [Y(k | X 0 , [U (0 . . . k)])]

k=0 {1, 2}

k=1 {2}

k=2 {2, 3}

k=3 {2, 3}

k=4 {2, 3}

k=5 {2, 3} .

It can be seen that the set (3.43) of possible quantised output symbols becomes larger if both uncertainties are considered.

62

3. Modelling of Quantised Systems by Stochastic Automata

Fig. 3.15. Sets of outputs with input uncertainty but without initial state uncertainty (left) and with input and initial state uncertainty (right).

3.4.2 Solution to the Stochastic Representation Problem of Autonomous Systems In the previous section, the behaviour of the quantised system has been described as evolution of sets in the state space. This is extended in the following to a probabilistic description which is required to solve the stochastic representation problem. As the complete solution to this problem refers to several measure theoretic concepts, this section is restricted to autonomous systems, making it possible to present the theory on an intuitive level. For autonomous systems, the simplified stochastic representation problem concerns the task to determine the probability distribution Prob([x(k)] | px0 (x)) .

(3.45)

A complete solution concerning the probability distribution (3.16), will however be given in Section 3.6. The presentation in this section is oriented to an introduction given in [95]. Applying the map x(k+1) = g a (x(k)) ,

x(0) = x0

(3.46)

of the autonomous system to a large number of randomly selected initial states 1 2 σ X sel 0 = {x (0), x (0), . . . , x (0)} ⊂ X 0

(3.47)

yields the σ successor states 1 1 2 2 X sel 1 = {x (1) = g a (x (0)), x (1) = g a (x (0)), . . . , xσ (1) = g a (xσ (0))} .

By means of the indicator function 1X : Rn → {0, 1}

(3.48)

3.4 Solution to the Representation Problems

 1X (x) =

1 if x ∈ X 0 if x 6∈ X ,

63

(3.49)

associated with the set X ⊆ Rn , the probability density function of the initial state can be defined as a function px0 : Rn → IR+ for which for every (not ˜ , the relation too small) region X Z σ 1X (3.50) 1 ˜ (xi (0)) px0 (x)dx ≈ σ i=1 X ˜ X holds. To indicate that px0 (x) is a density of the states at time zero, it is denoted by px (x, 0), and for arbitrary time k by px (x, k). Using this notation, the density function px (x, 1) for the successor states can be given, subsequently, by Z σ 1X px (x, 1)dx ≈ 1X (xi (1)) . (3.51) σ i=1 X The aim is now to find a relation between the density functions px (x, 0) and px (x, 1). For this, the set of all points that are mapped to a given set X ⊂ Rn is considered g −1 a (X ) := {x | g a (x) ∈ X } .

(3.52)

Obviously, xi (1) ∈ X holds if and only if xi (0) ∈ g −1 a (X ). This can be written as 1X (g a (x)) = 1g−1 (x) . a (X ) With eqn. (3.53), the density (3.51) can be written as Z σ 1X px (x, 1)dx ≈ 1g−1 (xi (0)) . a (X ) σ X i=1

(3.53)

(3.54)

˜ = g −1 (X ), the right–hand sides of eqns. (3.50) and (3.54) By choosing X a become equal, hence, Z Z px (x, 1)dx = px (x, 0)dx (3.55) X

g −1 a (X )

holds, yielding the sought relation between px (x, 0) and px (x, 1). As explained in Section 3.6, eqn. (3.55) implicitly defines the Frobenius–Perron operator P . The transformation invoked by this operator is denoted by px (x, 1) = P px (x, 0). Iterative application of the operator P results in the evolution of an initial density function in the state space caused by the system. That eqn. (3.55) defines an operator becomes clearer if it is written in an explicit form. In case that ga is a scalar diffeomorphism, a differentiation of eqn. (3.55) leads to

64

3. Modelling of Quantised Systems by Stochastic Automata

Solution to the stochastic representation problem of autonomous quantised systems The probability distribution of the quantised states at time k for the initial probability density function px0 (x) is given by Z Prob([x(k)]p = j | px0 (x)) = px (x, k)dx Qx (j)

with k > 0 : px (x, k) = P px (x, k−1) k = 0 : px (x, 0) = px0 (x) ,

(3.58)

where P denotes the Frobenius–Perron operator associated with the continuous–variable system given by g a .

d px (x, 1) = P px (x, 0) = dx

Z g −1 a (X )

px (x, 0)dx

(3.56)

for an arbitrary interval X ⊂ R (cf. Section 3.6). The right–hand side of eqn. (3.56) explicitly defines the operator P . The probability that the state is within a certain region Qx (j) in the partitioned state space can be determined from a density function px (x) by Z Prob([x]p = j) = px (x)dx , (3.57) Qx (j)

where [x]p denotes the random variable of the quantised state (cf. Appendix A.2). The solution to the stochastic representation problem for autonomous quantised systems is given in the box at the top of this page, describing the quantised system as a stochastic process. Example 3.4.2. Consider the two–tank system of Section 1.4 with the outflow qˆ2 of the right tank as output and the pump turned off (vp = 0), like in the first part of the example in Section 3.4.1. Some thousand initial points uniformly distributed over X 0 = { (x1 , x2 ) | 0.4 ≤ x1 ≤ 0.6, 0 ≤ x2 < 0.1 }

(3.59)

are considered. Mapping them by means of the system equation (1.8) of the tank system yields the successor states (3.48). By using a grid of some hundred disjoint small boxes Qx covering the state space, eqns. (3.50) and (3.51) yield an approximation of the density functions px (x, 0) and px (x, 1) shown in Figure 3.16. Successive mapping further yields the density functions px (x, 2) and px (x, 3) also shown in the figure. The relation between these densities is described by eqn. (3.58) with a fix operator P depending only on the system equation (1.8) describing the tank system. Explicit examples for the Frobenius–Perron operator are given in Section 3.6.

3.4 Solution to the Representation Problems

65

Fig. 3.16. Evolution of an initial state density function in the state space.

By using eqn. (1.3), the points are mapped to the output space. Like in the state space, counting the points over a grid of disjoint intervals approximates the density functions in the output space. These density functions are shown in Figure 3.17 for k = 0, . . . , 5. Compared to Figure 3.14, it can be seen that the densities are nonzero only on the sets of possible states or outputs, respectively. Furthermore, it can be seen that a uniform distribution px (x, 0) of the initial state does neither lead to uniform successor state distributions nor to uniform distributions in the output space.

Fig. 3.17. Evolution of densities in the output space.

66

3. Modelling of Quantised Systems by Stochastic Automata

Relation to the set representation problem. Describing the system dynamics by the temporal evolution of an initial density function in the state space is a more general approach than describing the evolution of an initial set of states. To show the formal relation between the evolution of densities and the evolution of sets, the notation of the support of a density function is used supp(px (x)) = {x | px (x) 6= 0} . By setting X 0 = supp(px (x, 0)), the relation between the evolution of sets and densities is described by: X (k | X 0 ) = supp(P k px (x, 0)) = supp(|P P {z . . . P} px (x, 0))

k ≥ 1 . (3.60)

k–times

As the set–oriented point of view allows an easier formulation and interpretation, it will be used whenever appropriate. However, in most cases the quantised system is considered in a probabilistic way because often the interpretation of the quantised system as a stochastic process is necessary. Remark 3.4.1. A sequence of densities in the state space only partly reflects the dynamics of the quantised system. Similar to the projection of the set (3.36) to the set (3.39) of states at time k, the density px (x, k) on Rn is just a projection to time k of a density on the space (Rn )k+1 describing the probability of all sequences with time horizon k. In other words, the iteratively determined density functions px (x, k) describe the random variables of the stochastic process separately at each time instance, whereas for a complete description, the relation between these random variables would also need to be described. This is because the random variables are, generally, not stochastically independent. Considering the previous example, from the sequence of densities in the output space shown in Figure 3.17, it is only possible to say that at time zero, [y] = 1 or [y] = 2 can occur and at time k = 2, the quantised outputs can be [y] = 2 or [y] = 3. The densities do not reveal whether for the given initial condition for example the sequence [Y (0 . . . 2)] = {1, 2, 3} can occur in the quantised system. However, in Section 3.6, it will be shown that the Frobenius–Perron operator can also be used to generate single numerical sequences or the probabilities of quantised sequences. Hence, the Frobenius–Perron operator can be used as representation of the quantised system.

3.5 Qualitative Model of the Quantised System 3.5.1 Motivation and Modelling Aims According to its definition, the quantised system is a stochastic process and as such its behaviour is described by a sequence of random variables (cf. Sections 2.2 and 3.2.2). This has led to the stochastic representation problem in Section 3.2.3, which asks for a description of the sequence of discrete

3.5 Qualitative Model of the Quantised System

67

probability distributions (3.16). In Section 3.4.2 it is shown that already for autonomous system the precise solution to this representation problem is very complex. Similarly, the simplified set representation problem of quantised system has led to complicated, implicitly defined sets like the set (3.43) of all quantised outputs at time k. Recalling simplicity of the solution to process supervision tasks as the first modelling aim of Section 3.1.3, it must be concluded that precise descriptions by means of eqns. (3.43) or (3.58) are not suitable. Accordingly, the claim to use a precise description of the quantised system must be abandoned for solving process supervision tasks. Instead, the aim must be to use an approximate representation of the quantised system. This leads to the additional modelling aims “completeness” and “convergence” explained in Section 3.1.3. An approximate representation of the quantised system satisfying these aims is defined now. It provides an approximate solution to the stochastic representation problem as outlined in Section 3.5.3. 3.5.2 Definition of the Qualitative Model As described in Section 3.1.4, each automaton state z is associated with the region Qx (z) of the partitioned state space. As the behavioural relation L(z 0 , w|z, v) of the SA describes the conditional probability that the automaton state changes from z to z 0 for a given input v while the output is w, setting [x] = z, [u] = v and [y] = w directly results in eqn. (3.5). Some other possibilities to assign the automaton state and thus the behavioural relation are briefly outlined in Section 4.2.5. In this section, the probability distribution on the right–hand side of eqn. (3.5) is investigated in more detail. Furthermore eqn. (3.6) will be derived. In one respect, the condition side of eqn. (3.5) is not well defined. This is because the conditions x ∈ Qx (z) and u ∈ Qu (v) do not sufficiently describe the relation between the numerical and the quantised values. For this reason, the injector has been introduced in Section 3.2.1 by probability density functions piu (u) that describe with which probability numerical inputs occur for given quantised input. This relation has not yet been considered for the numerical and the quantised state. Therefore, to further investigate the probability on the right–hand side of eqn. (3.5), density functions of the state are introduced for each Qx (i). That is, analogous to the injector, probability density functions pix : Rn → IR+ with Z i px (x) = 0 ⇔ x ∈ 6 Qx (i) and pix (x)dx = 1 , i = 1, 2, . . . , N (3.61) Rn

are defined. How these density functions are set and which consequences this definition has is discussed in Section 3.5.3. With the introduced densities pix (x), the probability distribution (3.5) is now

68

3. Modelling of Quantised Systems by Stochastic Automata

Abstraction of a qualitative model from a quantised system Definition 3.5.1. The stochastic automaton S(Nx , Nu , Ny , L) given by L(z 0 , w|z, v) :=

µn+m (Qxu (z, v) ∩ (gh)−1 (Qxy (z 0 , w))) , (3.64) µn+m (Qxu (z, v))

where µn+m denotes the measure of an arbitrary, measurable set Qxu = Qx × Qu ∈ B(Rn+m ) given by µn+m (Qxu ) = µn (Qx ) · µm (Qu ) Z Z N X i = ( px (x)) dx · Qx i=1

(3.65) (

M X

Qu j=1

pju (u)) du

is called qualitative model of the quantised system (3.13) for given set of density functions (Nx , {p1x , . . . , pN x }), provided that the transformations g and h are non–singular. The states, inputs and outputs of a qualitative model are called qualitative states, qualitative inputs and qualitative outputs, respectively.

clearly specified. Some terms from measure theory defined in Appendix A.1 are required for the following definition. Furthermore, the definition refers to the sets Qxu (z, v) = Qx (z) × Qu (v) ⊆ Rn × Rm = Rn+m ,    þ x þþ g(x, u) ∈ Qx (z 0 ), (gh)−1 (Qxy (z 0 , w)):= ⊆ Rn+m , u þ h(x, u) ∈ Qy (w)

(3.62) (3.63)

where the sets Qx (i) and Qu (j) are obtained from the density functions by setting Qx (i) = supp(pix ) and Qu (j) = supp(pju ), respectively. With this, a qualitative model of a quantised system can finally be defined as given in the box at the top of this page. Example 3.5.1. Definition 3.5.1 of the qualitative model is quite abstract. Therefore, a simple example is used to explain the definition in detail. In contrast to the example of Section 3.1.4, an input and an output space is considered. To allow a graphical presentation, a scalar system is used. The following quantised system is considered: x(k+1) = 0.8 x(k) − 0.1 u(k)

(3.66)

y(k) = 0.5 x(k) − 0.5 u(k)

(3.67)

with the output quantiser according to Qy (1) = (−∞, 0) Qy (2) = [0, ∞) ,

(3.68)

3.5 Qualitative Model of the Quantised System

69

Fig. 3.18. Partition and density functions of the input space (left) and state space (right).

the input injector according to   exp(u + 1) u < −1 0.5 u ∈ [−1, 1] p1u (u) = p2u (u) = 0 otherwise , 0 otherwise ,  exp(−u + 1) u > 1 p3u (u) = 0 otherwise , and the state densities   exp(x + 1) x < −1 0.5 x ∈ [−1, 1] p1x (x) = p2x (x) = 0 otherwise , 0 otherwise ,  exp(−x + 1) x > 1 p3x (x) = 0 otherwise .

(3.69) (3.70)

(3.71) (3.72)

Note that the density functions piu (u) and pix (x) implicitly define the input and state space partitions, respectively. The density functions (3.69)–(3.72) are shown in Figure 3.18. As example, the value L(z 0 = 1, w = 1 | z = 1, v = 1) of the behavioural relation is investigated. According to eqn. (3.64), two subsets of the combined state and input space Rn+m need to be determined for L(1, 1|1, 1), namely: Qxu (z = 1, v = 1)

and

(gh)−1 (Qxy (z 0 = 1, w = 1)) .

(3.73)

The set Qxu (z = 1, v = 1) is directly given by the partitions of the input and the state space, and corresponds to the lower left corner in the figure on the left–hand side of Figure 3.19, bounded by the black orthogonal lines. According to eqn. (3.63), the set (gh)−1 (Qxy (z 0 = 1, w = 1)) is the set of all state and input pairs (x, u) that are mapped to the region in the combined state and output space for which both [g(x, u)] = 1 and [h(x, u)] = 1 hold. The combined state and output space and its partition is shown on the right–hand side of Figure 3.19. The region of interest for L(1, 1|1, 1) is also the lower left corner in this space. The bounds of the sets (gh)−1 (Qxy (z 0 , w)) can simply be determined by mapping the partition bounds of the combined state and output space, depicted on the right– hand side of Figure 3.19 as grey lines, by means of (gh)−1 to the combined state

70

3. Modelling of Quantised Systems by Stochastic Automata

u z =2, v =3 z’=2 w =2 z =2, v =2 z’=2 w =1 z =2, v =1

z =1, v =3 1 -1

y -1 z’=3 ( gh) w =2 z =3, v =3 z’=3 w =1 z =3, v =2 0

z =1, v =2 z’=1 w =2

z =3, v =1 z =1, v =1 z’=1 w =1 (gh-1) (Q (z’=1, w =1)) Q (z =1, v=1) xy xu -1

z’=1 w =2

z’=2 w =2

z’=3 w =2

z’=1 w =1

z’=2 w =1

z’=3 w =1

x

1

-1

1

x

Fig. 3.19. Explanation of the qualitative model definition. and input space. The resulting lines are shown on the left–hand side of Figure 3.19 also in grey. They are given by x−u=0,

8x − u − 10 = 0 ,

and

8x − u + 10 = 0 .

The backwards mapped lines partition the combined state and input space into six regions, each region labelled in Figure 3.19 in grey according to the corresponding region in the combined state and output space. Now, according to eqn. (3.64), the intersection of the sets (3.73) needs to be determined. In this case the resulting set is equal to the set (gh)−1 (Qxy (z 0 = 1, w = 1)), shown as the light grey region in Figure 3.19. The transition probability L(1, 1 | 1, 1) is the quotient of the measures of the intersection and the set Qxu (z = 1, v = 1). Instead of using the Lebesgue measure Z 1 dx du λ2 (Qxu ) = Qxu

which would be the area of the sets, the measure (3.65) is used. This measure can be interpreted as area weighted by the density functions in the state and input space. Different weights exist, because the numerical inputs and states within the set Qxu (z = 1, v = 1) have different probability to occur in the quantised system. Furthermore, the measure (3.65) has the “advantage” that the unbounded sets have finite measure, what is not the case for the Lebesgue measure. The weight introduced by the input and state space densities is shown in Figure 3.20. The measure of the intersection is, according to eqn. (3.65), the volume of the light grey region shown in the lower left corner of this figure. With the intersection point x = u = −10/7 the measure is given by   µ2 Qxu (z = 1, v = 1) ∩ (gh)−1 (Qxy (z 0 = 1, w = 1)) Z =

−∞

Z =

−10/7

−10/7

−∞

Z

x 8x+10

 exp(u − 1) du exp(x − 1) dx

(exp(2x + 2) − exp(9x + 12)) dx =

7 6 exp(− ) ≈ 0.165 . 18 7

3.5 Qualitative Model of the Quantised System

71

Fig. 3.20. Explanation of the measure introduced by the densities in the state and input space.

Fig. 3.21. Qualitative model of the example quantised system. The measure of Qxu (z = 1, v = 1) is given by  Z −1 Z −1 exp(u + 1)du exp(x + 1)dx = 1 . µ2 (Qxu (z = 1, v = 1)) = −∞

−∞

0

Accordingly, eqn. (3.64) yields L(z = 1, w = 1 | z = 1, v = 1) = 0.165. Likewise, all other transition probabilities are calculated, resulting in the SA shown in Figure 3.21. The input is distinguished by the colour of the edges, from black for v = 1 to light grey for v = 3. Thin edges mean w = 1 and thick edges w = 2.

Remark 3.5.1. Definition 3.5.1 of the qualitative model can be extended to discrete signal spaces as introduced in Section 3.3.3. The definition of the extended injector can be applied equivalently to the densities introduced in the state space. The measure µn+m is then given by eqns. (3.28) and (3.31).

72

3. Modelling of Quantised Systems by Stochastic Automata

3.5.3 Properties of the Qualitative Model A–priori densities. A prerequisite of Definition 3.5.1 is the knowledge of the densities pix (x) describing the conditional a–priori probability distribution of the numerical state variables for given qualitative state i. These densities could theoretically be obtained after observing the system over a long time with all possible combinations of input sequences applied to the system, and evaluating the frequencies of occurrence of the numerical states within each region. Practically, this way of modelling is only applicable for plants running for a long time without changes and with an automated measurement system. Otherwise, heuristic settings like uniform distributions must be used on all bounded sets. Definition 3.5.1 is therefore usually written in a simplified form, implicitly assuming bounded signal spaces Rn ⊂ IRn , Rm ⊂ IRm , Rr ⊂ IRr and uniform distributions  1 for x ∈ Qx (i) pix (x) = λn (Qx (i)) , i = 1, . . . , N (3.74) 0 otherwise for the state, and input as in eqn. (3.12) (cf. [173]). With this, eqn. (3.64) can be simplified to eqn. (3.6) where the Lebesgue measure λn+m instead of the measure (3.65) is used. That is, the transition probabilities can be determined from the quotient of the hypervolumes of Qxu (z, v) and the subset of Qxu (z, v) mapped to the region with quantised values z 0 and w. Due to eqns. (3.11) and (3.61), the denominator of eqn. (3.64) is always one: Z N M X X µn+m (Qxu (z, v)) = ( pix (x)) · ( pju (u)) dx du Qxu (z,v) i=1

Z =

Qxu (z,v)

Z =

Qu (v)

j=1

pzx (x) · pvu (u) dx du

ýZ Qx (z)

! pzx (x) dx

pvu (u) du

=1. This means that eqn. (3.64) can be simplified to: L(z 0 , w | z, v) = µn+m (Qxu (z, v) ∩ (gh)−1 (Qxy (z 0 , w)) .

(3.75)

In fact, eqn. (3.64) has only been written as fraction to show its relation to eqn. (3.6) and to Ulam’s piecewise constant approximation of the Frobenius– Perron operator (cf. Section 3.6, [195]). Interpretation of the model state as density function. The evolution of an initial qualitative state described by the qualitative model by using eqn. (2.28) yields a probability distribution Prob(z(k)) at time k. This distribution can equivalently be represented as a vector 0

pz (k) = ( Prob(zp (k) = 1) Prob(zp (k) = 2) . . . Prob(zp (k) = N )) (3.76)

3.5 Qualitative Model of the Quantised System

73

with pz (k) ∈ W N , where W N is the space of all N-dimensional discrete probability vectors: 0 N W N = {pz = (p1z . . . pN z ) ∈ [0, 1] |

N X

piz = 1} .

(3.77)

i=1

Each discrete probability vector and, thereby, each state of the SA, can be interpreted as probability density function by means of the densities pix (x) defined by the qualitative model of the quantised system. This is done by the operator IN called density injector IN : W N → D

IN pz =

N X

piz · pix (x) ,

(3.78)

i=1

where D denotes the space of all probability density functions (cf. Appendix A.1). An example how the density injector works is given in the next section. The inverse operation of the density injector is performed by the density quantiser QN defined by Z QN : D → W N QN px (x) = pz , piz = px (x)dx . (3.79) Qx (i)

The initial condition pz (0) of the SA is thus obtained from the initial condition px0 (x) of the quantised system by: pz (0) = QN px0 (x). Note that the relation pz = QN (IN pz ) holds, whereas px (x) = IN (QN px (x)), in general, does not. The latter means that the information kept by the qualitative model is not equivalent to the information contained in the current state density px (x, k) characterising the future behaviour of the quantised system. That is, the current state distribution Prob(z(k)) of the qualitative model does not represent the state of the quantised system in a system theoretic sense which has already been noted in [97]. Consequently, the dynamic behaviour of the quantised system and the qualitative model will, in general, differ. With respect to this, in [111] is shown that the state of the qualitative model does, in general, not allow to precisely predict the future of the quantised system and can, therefore, not be referred to as the state of the quantised system. The question, of which information about the behaviour of the quantised system is kept in the qualitative model is discussed in detail in the following and in Section 4.3. The Markov property of the qualitative model. As the qualitative model used here is a stochastic automaton, it possesses the Markov property. That is, the relation þ   zp (k+1) = z(k+1), þþ Zp (0...k) = Z(0..k), Prob þ Vp (0...k) = V (0...k) wp (k) = w(k)

74

3. Modelling of Quantised Systems by Stochastic Automata

 = Prob

þ  zp (k+1) = z(k+1), þþ zp (k) = z(k), þ vp (k) = v(k) wp (k) = w(k)

holds for all input, output and state sequences and for all k. In contrast to this, the corresponding relation þ   [x(k+1)]p = z(k+1), þþ [X(0...k)]p = Z(0..k), Prob þ [U (0...k)]p = V (0...k) [y(k)]p = w(k) þ   [x(k+1)]p = z(k+1), þþ [x(k)]p = z(k), = Prob (3.80) þ [u(k)]p = v(k) [y(k)]p = w(k) for the quantised system does not, in general, hold for k > 0. That is, the quantised system is, in general, not a Markov process (cf. Section 2.2, [108], [116]). Only under rather restrictive conditions on the system (3.1) and (3.2) and on the state space partition the quantised system can become a Markov process (cf. [111], [116]). This is investigated in detail in Section 4.3.2. The reason why the quantised system does not, in general, possess the Markov property is illustrated by an example. Furthermore, the following example illustrates the definitions of eqns. (3.78) and (3.79). Example 3.5.2. The quantised system (3.66)–(3.72) is considered with the initial

condition

px (x, 0) = px0 (x) =



0.5 x ∈ [−1, 1] . 0 otherwise.

(3.81)

The SA is initialised with pz (0) = QN px (x, 0) = (0 1 0)0 . The probability density function p˜x (x, 0) = IN pz (0) obtained from the automaton state by means of the density injector (3.78) is equal to px (x, 0), i.e. in this particular case the relation px (x, 0) = IN (QN px (x, 0)) holds. The initial condition (3.81) of the quantised system is thus represented by the discrete probability distribution pz (0) in this case. As explained in Section 3.4.2 for autonomous systems, the initial density px (x, 0) can be mapped to the density px (x, 1) at time k = 1 by the Frobenius–Perron operator P . The example system is not autonomous. However, there exist similar operators for each quantised input, called Foias operators describing the map of densities for non–autonomous systems (cf. Section 3.6). The Foias operator is denoted by z[u] . The densities px (x, 1) and px (x, 2) resulting from successive application of the Foias operator for [u] = 3 starting with px (x, 0) are shown in the upper part of Figure 3.22. Applying the density quantiser (3.79) to these densities yields QN px (x, 1) = (0.023 0.977 0)0 and QN px (x, 2) = (0.013 0.987 0)0 . (3.82) On the other hand, by applying eqn. (2.28) to pz (0) with v = 3, the SA yields pz (1) = (0.023 0.977 0)0

and

pz (2) = (0.044 0.956 0)0

(3.83)

(cf. automaton graph in Figure 3.21). The result QN px (x, 2) 6= pz (2) shows that the quantised system yields different results than its Markovian approximation. The reason is shown in the lower part of Figure 3.22. In the first step, the probabilities of QN px (x, 1) and pz (1) are equal due to the definition of the transition probabilities by eqn. (3.64). However, the automaton inherently contains the interpretation of pz (1) as density functions reflected by the density injector (3.78). That is, each step performed by the automaton corresponds to the successive application

3.5 Qualitative Model of the Quantised System

75

Fig. 3.22. Non–Markovian behaviour of the quantised system and approximation due to the SA. of a density injection IN , a map by the Foias operator z[u] and a density quantisation QN , i.e. pz (k+1) = QN (z[u] (IN pz (k))) holds. This is shown in the lower part of Figure 3.22. From the two densities shown in the middle of the lower part of this figure, it can be seen that the densities px (x, 1) and p˜x (x, 1) = IN (QN px (x, 1)) are obviously different, though QN px (x, 1) = QN p˜x (x, 1) holds. It is therefore, clear that the automaton that obtains its state distribution pz (2) from QN (z[u] (IN pz (1))) = QN (z[u] p˜x (x, 1)) yields a different result than the quantised system which yields the distribution QN (z[u] px (x, 1)). The density z[u] p˜x (x, 1) is shown in the lower right corner of the figure. It can be verified that QN (z[u] p˜x (x, 1)) = pz (2) holds.

The completeness property of the qualitative model. The second modelling aim of Section 3.1.3 is the completeness of the qualitative model. The following chapters will show, that this is a fundamental requirement for the applicability of qualitative models for the solution to supervision tasks. This importance has been recognised in the literature on qualitative modelling by several authors (cf. e.g. [4], [108], [153], [187]). As completeness does not refer to the probabilities of the state or output sequences, this section is restricted to the set–oriented viewpoint, corresponding to the set representation problem investigated in Section 3.4.1. Before giving a formal definition of completeness, the sets of qualitative sequences occurring due to the quantised system and due to the SA are investigated first.

76

3. Modelling of Quantised Systems by Stochastic Automata

Sequences generated by the SA. From eqn. (2.30) follows that Prob(W (0 . . . k) | V (0 . . . k), z(0)) k X Y = L(z(i+1), w(i) | z(i), v(i)) Z(1...k+1)∈Nzk+1 i=0

holds. With this, the set of all output sequences generated by an SA for given input sequence V (0 . . . k) can be written as W(0 . . . k | V (0 . . . k)) = (3.84) N [ = {W (0 . . . k) | Prob(W (0 . . . k) | V (0 . . . k), z(0)) > 0} . z(0)=1

Similarly, the set of all state sequences for given initial state distribution Prob(z(0)) is obtained from eqn. (2.25): Z(0 . . . k+1 | V (0 . . . k), Prob(z(0))) = (3.85) k Y = {Z(0 . . . k+1) | G(z(i+1) | z(i), v(i)) · Prob(z(0)) > 0} . i=0

Sequences occurring in the quantised system. In analogy to eqn. (3.38) the set of all quantised state sequences for given set of initial states and given quantised input sequence is given by [X (0 . . . k | X 0 , [U (0 . . . k−1)])] (3.86) þ   þ ∃ X(0 . . . k) ∈ X (0 . . . k|X 0 , [U (0 . . . k−1)]) = [X(0 . . . k)] þþ such that x(κ) ∈ Qx ([x(κ)]) holds for κ = 0, . . . , k ⊆ Nxk+1 , with X (0 . . . k|X 0 , [U (0 . . . k − 1)]) according to eqn. (3.36). Note that considering the set (3.86) implicitly includes the assumption that all numerical state sequences of the set (3.36) and all numerical input sequences can really occur in the quantised system. This holds because the densities pix (x) and piu (u) are nonzero on Qx (i) and Qu (i), respectively (cf. eqns. (3.11), (3.61) and Section 3.4.1). For the measures µn and µm , this means that λn (Qx ) > 0 ⇒ µn (Qx ) > 0 and λm (Qu ) > 0 ⇒ µm (Qu ) > 0 hold for arbitrary sets Qx and Qu , respectively. For reasons given below, some sequences must be excluded from the sets (3.38) and (3.86) to exclude singular cases. That is, for the definition of completeness the sets [X (0 . . . k | X 0 , [U (0 . . . k−1)])]∗ (3.87)  þ  þ [X(0 . . . k)] ∈ [X (0 . . . k | X 0 , [U (0 . . . k−1)])],   þ = [X(0 . . . k)] þþ λn (Qx ([x(κ)]) ∩ X (κ | X 0 , [U (0 . . . κ−1)])) > 0   þ holds for all κ = 0, . . . , k

3.5 Qualitative Model of the Quantised System

77

and (3.88) [Y(0 . . . k−1 | X 0 , [U (0 . . . k−1)])]∗  þ  þ [Y (0...k−1)] ∈ [Y(0...k−1 | X 0 , [U (0...k−1)])],   þ = [Y (0 . . . k−1)] þþ λr (Qy ([y(κ)]) ∩ Y(κ | Y 0 , [U (0 . . . κ−1)])) > 0   þ holds for all κ = 0, . . . , k−1 are considered. Definitions of completeness. With the above sets, the input/output completeness (I/O–completeness) of an SA with respect to a quantised system is defined as follows: Definition 3.5.2. A stochastic automaton S(Nz , Nv , Nw , L) is called I/O– complete with respect to a given quantised system (3.13), if [Y (0 . . . k)] ∈ [Y(0 . . . k | Rn , [U (0 . . . k)])]∗ ⇒ [Y (0 . . . k)] ∈ W(0 . . . k | [U (0 . . . k)])

(3.89)

holds for all quantised output sequences [Y (0 . . . k)] and for all quantised input sequences [U (0 . . . k)]. That is, an SA is called I/O–complete if all quantised output sequences generated by the quantised system for all possible initial conditions are also generated by the SA for all initial states. The input/state completeness (I/S–completeness) of an SA is defined in a similar way. Definition 3.5.3. A stochastic automaton S(Nz , Nv , Nw , L) is called I/S– complete with respect to a given quantised system (3.13) and a given state space partition if [X(0 . . . k+1)] ∈ [X (0 . . . k+1 | X 0 , [U (0 . . . k)])]∗ ⇒ [X(0 . . . k+1)] ∈ Z(0 . . . k+1 | Prob(z(0)), [U (0 . . . k)])

(3.90)

holds with Prob(zp (0) = z) > 0 ⇔ µn (X 0 ∩ Qx (z)) > 0 ,

∀z ∈ Nz

(3.91)

for all quantised state sequences [X(0 . . . k)], for all quantised input sequences [U (0 . . . k)] and for all sets of initial states X 0 . Completeness of the qualitative model Theorem 3.5.1. The qualitative model defined by Definition 3.5.1 is an I/O–complete and I/S–complete model of the quantised system. Proof. See Appendix C.2.

t u

78

3. Modelling of Quantised Systems by Stochastic Automata

Remark 3.5.2. By definition, the sets (3.38) and (3.86) include all quantised sequences possibly occurring in the quantised system. This includes those corresponding only to a single numerical sequence, i.e. immeasurably few if compared to all other sequences. That is, from a probabilistic viewpoint, these sequences occur, but with zero probability in the quantised system. As a probabilistic model is used, these sequences must be generated with probability zero. Therefore, such sequences are excluded from the sets (3.38) and (3.86) by defining the sets (3.87) and (3.88). That is, the definition excludes all those quantised sequences for which the underlying bundle of trajectories intersects with the regions of the partition only on a subset with a measure of zero. Consider for example the set of output symbols shown on the right–hand side of Figure 3.14. With the parameters used in this example, the set of output symbols at time k = 4 is given by the singleton {2}. Assume, however, that the parameters are slightly modified such that only the outermost numerical output at k = 4 becomes y(4) = 6.0 and does, hence, no longer lie in the interval [3, 6). Then the set of possible quantised outputs would be given by {2, 3}. However, the probability that the quantised output [y(4)] = 3 is measured is zero. Therefore, in the set (3.87), such an output symbol would be excluded, whereas the set (3.38) would include this symbol. Completeness only holds with respect to those quantised sequences that occur with nonzero probability.

Remark 3.5.3. Alternatively, relations (3.89) and (3.90) can be written as: [Y(0 . . . k | [U (0 . . . k)])]∗ ⊇ W(0 . . . k | [U (0 . . . k)])

(3.92)

and [X (0 . . . k+1 | X 0 , [U (0 . . . k)])]∗ ⊇ Z(0 . . . k+1 | Prob(z(0)), [U (0 . . . k)]) . (3.93)

Remark 3.5.4. As explained later in Chapter 6, I/O–completeness of the qualitative model is necessary for its applicability to fault diagnosis, while I/S and I/O–completeness are required for state observation. I/O–completeness has, therefore, been formulated independently of the states of the SA to allow its use for diagnosis even if the automaton state has no direct relation to the state of the quantised system. However, whenever it is not necessary to distinguish between I/O– and I/S–completeness, the term completeness of the model refers to both I/O and I/S–completeness. A sufficient completeness condition. Denote the set of all possible successor state and output pairs (z 0 , w) for given state z and input v according to the behavioural relation L by TL (z, v) = {(z 0 , w) | L(z 0 , w | z, v) > 0} ⊆ Nz × Nw .

(3.94)

With this, the following result can be formulated. Lemma 3.5.1. Every stochastic automaton S(Nz , Nv , Nw , L⊕ ) whose behavioural relation satisfies TL⊕ (z, v) ⊇ TL (z, v)

(3.95)

for all z ∈ Nz and v ∈ Nv is an I/O– and I/S–complete model of the quantised system.

3.5 Qualitative Model of the Quantised System

Proof. See Appendix C.3.

79

t u

In summary, it can be said that the qualitative model does not yield the same sequences and probabilities as the quantised system. However, according to Theorem 3.5.1, it is proved that all sequences occurring in the quantised system are generated by the qualitative model with nonzero probability. According to Lemma 3.5.1, this still holds if the behavioural relation contains additional transitions with nonzero probability or differs in the probabilities contained in L. A prerequisite for fulfilling relation (3.90) or (3.93) is that the initial distribution of the qualitative model is chosen such that a qualitative state associated with a region Qx (i) that is overlapping with X 0 is not assigned a probability of zero (cf. relation (3.91)). Remark 3.5.5. Note from the above, it is only considered whether a probability is zero or not. That is, completeness can also be obtained by a model that assigns arbitrary nonzero probabilities to the sequences of the quantised system. Completeness alone is a very weak condition, satisfied already by an SA generating all existing output and state sequences. Therefore, the modelling aims of Section 3.1.3 include the convergence of the probabilities yielded by the qualitative model to those of the quantised system. This convergence is discussed in Section 3.6.5. Remark 3.5.6. The set of output sequences (3.84) generated by the qualitative model depends on the state space partition. The number of elements of the set can be changed by modifying the shape or the number of partition sets Qx (i). However, relation (3.92) holds independent of the state space partitions. It is therefore reasonable to refer to the size of the set (3.84) with respect to the set (3.88) as the accuracy of a qualitative model [156], [187]. With respect to their accuracy, it is then possible to define an order of the models. That is, a model yielding the set W1 is called more accurate than a model yielding W2 if the relation W1 (0 . . . k | [U (0 . . . k)]) ⊂ W2 (0 . . . k | [U (0 . . . k)])

(3.96)

holds while both models are I/O–complete.

Approximate solution to the stochastic representation problem. In this section, an approximate solution to the stochastic representation problem of Section 3.2.3 is given using the qualitative model. As the qualitative model is only an approximate representation of the quantised system, the probabilities determined by means of the model differ from the probabilities with which the quantised states or outputs actually occur in the quantised system. Therefore, the following equations are denoted in terms of the vari\ or [y(k)] \ for the state and output variables of the qualitative ables [x(k)] model, respectively. The probability distributions (3.16) are approximated by \ | [U (0 . . . k)]) , Prob([y(k)] | px0 (x), [U (0 . . . k)]) ≈ Prob([y(k)] \ | [U (0 . . . k)]) are determined recursively where the distributions Prob( [y(k)] using eqns. (2.28) and (2.34) by

80

3. Modelling of Quantised Systems by Stochastic Automata

\ | [U (0 . . . k)]) Prob([y(k)] (3.97) X \ | [x(k)], \ [u(k)]) · Prob([x(k)] \ | [U (0 . . . k−1)]) = H([y(k)] \ [x(k)]∈N x

and \ | [U (0 . . . k−1)]) = k > 1 : Prob([x(k)] (3.98) X \ | [x(k)], \ [u(k)]) · Prob([x(k−1)] \ | [U (0 . . . k−2)]) G([x(k+1)] \ [x(k−1)]∈N x

\ | [u(0)] = k = 1 : Prob([x(1)] X \ | [x(0)], \ [u(0)]) · Prob([x(0)]) \ , G([x(1)] \ [x(0)]∈N x

\ = QN px (x), where G and H are the state transition relawith Prob([x(0)]) 0 tion and output relation of the qualitative model, respectively (cf. eqns. (2.5) and(2.6)). Equations (3.97) and (3.98) give approximate probabilities of the quantised outputs and quantised states for a given sequence of quantised inputs and for a given initial state distribution. Therefore, these equations are referred to as qualitative simulation of the quantised output or state of the quantised system, respectively. Note that due to the completeness of the qualitative model the following relations hold \ | [U (0 . . . k)]) > 0 Prob([y(k)] | px0 (x), [U (0 . . . k)]) > 0 ⇒ Prob([y(k)] \ | [U (0 . . . k−1)]) > 0 , Prob([x(k)] | px (x), [U (0 . . . k−1)]) > 0 ⇒ Prob([x(k)] 0

provided that the qualitative model is initialised with the probability distri\ = QN px (x). That is, the completeness of the qualitative bution Prob([x(0)]) 0 model guarantees that the model yields a nonzero probability for every quantised output and quantised state possibly occurring in the quantised system. Qualitative simulation of the running example. Example 3.5.3. Consider the tank system of Section 1.4 with the valve V 12l opened and V12u closed. The tank system is considered without input and the outflow qˆ2 of the right tank as output (cf. Section 3.4.1). The state space is partitioned as shown in Figure 3.6 and the output space as shown on the right–hand side of Figure 3.14 by the horizontal lines. With this, the qualitative model shown on the left–hand side of Figure 2.1 is obtained, where the outputs are distinguished by the colour and thickness of the edges, from thin black lines for [y] = 1 to thick light grey lines for [y] = 3. The results of the qualitative simulation by means of eqns. (2.28) and (2.34) starting in the qualitative state [x] = 3 are shown on the left–hand sides of Figures 3.23 and 3.24. Shown are the probabilities of the qualitative states and outputs, respectively, in grey scale for times k = 0, . . . , 5, where the boxes correspond the regions of the corresponding partitioned signal space. For comparison, the probability distributions resulting from a quantisation of the densities depicted

3.5 Qualitative Model of the Quantised System

81

Fig. 3.23. Simulation of the qualitative model (left) and the quantised system (right). The distributions (quantised densities) are shown in the state space.

Fig. 3.24. Simulation of the qualitative model (left) and the quantised system (right). The distributions (quantised densities) are shown in the output space.

in Figures 3.16 and 3.17 are shown on the right–hand side of Figures 3.23 and 3.24, starting with a uniform distribution over the set Qx (3) at k = 0. Additionally, the supports of the densities are shown, plotted onto the state space in Figure 3.23 or shown as interval bars right of the distributions in Figure 3.24. It can be seen that the qualitative model yields a nonzero probability for all qualitative states and outputs for which the quantised system yields a nonzero probability. This shows the completeness of the model. It can further be seen that the qualitative model also yields nonzero probabilities for states and outputs that the quantised system cannot generate. These states and outputs are so–called spurious solutions. Only the state distribution at time k = 1 and the output distribution at k = 0 are identical to those of the quantised system. This is because the initial conditions of the quantised system and the qualitative model coincide in this example and because the qualitative model contains by construction the single–step transition probabilities of the quantised system. Although the probabilities of the qualitative model differ from those of the quantised system for k > 1, its probabilities are approximated by the SA.

Quantised systems augmented by faults. Using the concept of the extended input space introduced in Section 3.2.4, Definition 3.5.1 can naturally be extended to the quantised systems augmented by faults. The automaton input v˜ = [˜ u] then contains the information of both the quantised input and

82

3. Modelling of Quantised Systems by Stochastic Automata

˜ 0 , w | z, v, f ) instead of the quantised fault. This is resolved by writing L(z 0 L(z , w | z, v˜), where f = [e], f ∈ Nf = Ne is used for the quantised fault. Then, v corresponds again to the input of the quantised system. Using the ˜ 0 , w | z, v, f ) are obtained from: bijective map (3.22), the values of L(z ˜ 0 , w | z, v, f ) := L(z 0 , w | z, v˜) = L(z 0 , w | z, Mue (v, f )) (3.99) L(z for all

z 0 , z ∈ Nz , v ∈ Nv , w ∈ Nw , f ∈ Nf .

That is, the qualitative model of the quantised system augmented by faults ˜ with L ˜ given by eqn. (3.99). is given by the SA S(Nx , Nu , Ny , Ne , L)

3.6 General Solution to the Stochastic Representation Problem** In Section 3.4.2, the Frobenius–Perron operator (FPO) is introduced on an intuitive level. In this section the FPO is described in detail. The given introduction is mainly based on the textbook [95] and partly on [21]. The results on the FPO and the definition of the Foias operator in Section 3.6.3 given below are reported. The main result of this section is the use of these operators to formulate the solution to the stochastic representation problems of quantised systems posed in Section 3.2.3. Further, this representation allows the use of a result obtained for the FPO to show that the third modelling aim of Section 3.1.3 is satisfied. The study of this section requires that the basic concepts from measure theory, summarised in Appendix A.1, are known. 3.6.1 The Frobenius–Perron Operator** FPO on measures. In fact, the FPO derived in Section 3.4.2 which transforms one density into another is just a special case of a more general definition of the FPO given in the following. Consider the Borel σ–algebra B(Rn ) on Rn ⊆ IRn . All measures µ on (Rn , B(Rn )) that satisfy µ(A) < ∞ for all bounded A ∈ B(Rn ) are called locally finite measures on Rn . The space of all locally finite measures is denoted by M. The subspace of M which contains only finite measures is denoted by Mfin . To derive the FPO, the autonomous dynamical system Rn → Rn ,

x(k+1) = g a (x(k))

(3.100) k

is considered. The system defines a relation between two measures µ , µk+1 ∈ Mfin that is given by the measure conservation principle [179] n µk+1 (A) = µk (g −1 a (A)) for all A ∈ B(R )

(3.101)

3.6 General Solution to the Stochastic Representation Problem**

83

which holds for all Borel measurable transformations g a . Equation (3.101) uniquely defines the operator Pˆ : Mfin → Mfin ,

µk+1 (A) = Pˆ µk (A)

which is called Frobenius–Perron operator on measures [95]. By means of eqn. (3.101), the FPO on measures describes the evolution of an initial measure µ0 over discrete time k. The FPO on measures is very abstract but it covers two cases that are of interest for the quantised system. FPO on the L1 space. First, from the FPO on measures, the FPO on L1 functions is derived, whose special case is the FPO on densities derived in Section 3.4.2. Consider measures ν absolute continuous to the Borel measure µ with a nonnegative L1 function p. Then Z p(x)µ(dx) , for all A ∈ B(Rn ) ν(A) = A

holds. If g a is non–singular, then the left–hand side of eqn. (3.101) is a continuous measure and the right–hand side of eqn. (3.101) can be written as Z ν(g −1 (A)) = p(x)µ(dx) , for all A ∈ B(Rn ) . a g −1 a (A)

Hence eqn. (3.101) yields the operator Z Z P p(x)µ(dx) = p(x)µ(dx) . A

g −1 a (A)

(3.102)

In [95] it is shown that eqn. (3.102) can be extended to all L1 functions p. The theorem of Radon–Nikodym guarantees the existence and uniqueness of the operator P given by eqn. (3.102) provided that g a is non–singular. The operator P : L1 → L1 is called Frobenius–Perron operator. It can be shown that the FPO satisfies 1. P (c1 p1 +c2 p2 ) = c1 P p1 +c2 P p2 , for all p1 , p2 ∈ L1 , c1 , c2 ∈ IR (linearity), 2. P p ≥ 0 if p ≥ 0, p ∈ L1 and 3. ||P p||L1 = ||p||L1 if p ≥ 0, otherwise ||P p||L1 ≤ ||p||L1 , p ∈ L1 , where p ≥ 0 means that p is almost everywhere nonnegative. From the second and the third property, it is clear that the FPO given by eqn. (3.102) equivalently defines a density operator D → D, i.e. every p ∈ D yields P p ∈ D. Note that in contrast to eqn. (3.102) the FPO on measures is defined for any Borel measurable transformation also if g a is singular. That is, the FPO on measures can also transform continuous measures into measures without density [95].

84

3. Modelling of Quantised Systems by Stochastic Automata

FPO on Dirac measures. Consider the dynamical system (3.100) with a point as initial condition: x(0) = x0 ∈ Rn . This initial condition is represented by the Dirac measure δx0 (A) as  0 if x0 6∈ A δx0 (A) = 1 if x0 ∈ A holds. From this follows by eqn. (3.101) that  0 if x0 6∈ g −1 −1 a (A) ˆ P δx0 (A) = δx0 (g a (A)) = 1 if x0 ∈ g −1 a (A) which can be written as  0 if g a (x0 ) 6∈ A Pˆ δx0 (A) = = δga (x0 ) (A) . 1 if g a (x0 ) ∈ A Thus, Pˆ δx0 = δga (x0 ) holds. That is, from the FPO on measures, the trajectory of the dynamical system (3.100) is obtained by starting from a Dirac measure δxk+1 = Pˆ δxk = δga (xk ) , given δx0 .

(3.103)

3.6.2 Representation of Autonomous Quantised Systems** From the previous section, it is clear that the FPO on measures describes the temporal evolution of densities as well as the evolution of a single trajectory. It therefore provides a useful tool for the description of the quantised systems allowing unique as well as uncertain initial conditions. This makes it possible to give a compact notation of the conditional probability distribution Prob([x(k)] | µ0 ) of the quantised states for given initial measure µ0 . For notational convenience, the vector notation p[x(k)] = ( Prob([x(k)]p = 1 | µ0 ) . . . Prob([x(k)]p = N | µ0 ) )0 is used instead of Prob([x(k)] | µ0 ), denoting the i–th component of p[x(k)] by pi[x(k)] . Denoting the measure µk (A) of a set A ∈ B(Rn ) at time k by µ(A, k), the autonomous quantised system is described as follows: Representation of autonomous quantised systems µ(A, k+1) = Pˆ µ(A, k) , µ(A, 0) = µ0 ˆ N µ(A, k) . p[x(k)] = Q

(3.104)

ˆ N the In eqn. (3.104) Pˆ is the FPO on measures corresponding to g a , and Q measure quantiser given by

3.6 General Solution to the Stochastic Representation Problem**

Z N i ˆ ˆ QN : Mfin → W , QN µ(A, k) = p[x(k)] , p[x(k)] =

85

µ(dx, k) . (3.105)

Qx (i)

ˆN ) That is, the autonomous quantised system is represented by the pair ( Pˆ , Q of the FPO on measures and the measure quantiser. The representation by eqn. (3.104) is general but abstract. To further evaluate this equation, the following considerations are restricted to densities px (x) ∈ D and non–singular transformations g a . With this, eqn. (3.104) becomes px (x, k+1) = P px (x, k) ,

px (x, 0) = px,0 (x)

p[x(k)] = QN px (x, k) ,

(3.106) (3.107)

where px (x, k) ∈ D denotes the density at time k, P the FPO on densities given by eqn. (3.102) and QN the density quantiser given by eqn. (3.79) [173]. Equations (3.106) and (3.107) provide the solution to the stochastic representation problem derived on an intuitive level in Section 3.4.2. If g a is a diffeomorphism, i.e. if g a is bijective and both, g a and g −1 are a differentiable, the FPO can be written explicitly as [95] þ  −1 þ þ ∂g a þþ −1 þ , (3.108) P px (x, k) = px (g a (x), k) · þdet ∂x þ where | det(∂g −1 a /∂x)| denotes the absolute value of the determinant of the Jacobian of g −1 a . Example 3.6.1. First order system. As first example, consider IR+ → IR+ ,

x(k+1) = ga (x) = (x(k))2 .

√ The FPO of is obtained from eqn. (3.108) with ga−1 = x as √ this transformation √ P p(x) = p( x)/(2 x). For the initial density px0 (x) = 5 · 1[0.6,0.8] the FPO yields: px (x, k) = P k px (x, 0) =

5 √ · 1 2k 2k . (2 x)k [0.6 ,0.8 ]

The densities are shown on the left–hand side of Figure 3.25 for k = 0, 1, 2. Applying the density quantisation (3.107) with Qx (1) = [0, 0.5), Qx (2) = [0.5, 1) and Qx (3) = [1, ∞) to px (x, 1) yields Z 1 Z 0.5 5 · 1[0.36,0.64] 5 · 1[0.36,0.64] √ √ dx = 0.5355 , p2[x(k)] = dx = 0.4645 , p1[x(k)] = 2 x 2 x 0.5 0 and p3[x(k)] = 0. This is also shown in Figure 3.25. Piecewise affine systems. As second example, consider the affine system IRn → IRn ,

x(k+1) = g a (x) = A · x(k) + b

with a non–singular system matrix A. The inverse of g a is given by g −1 a (x) = A−1 (x − b). Hence, eqn. (3.108) yields the FPO P px (x) = px (A−1 (x − b)) · | det A−1 | .

86

3. Modelling of Quantised Systems by Stochastic Automata

Fig. 3.25. Sequence of density functions for x(k+1) = x(k)2 (left) and for the tank system (right). This can be extended to piecewise affine systems  1   g a (x) if x ∈ Q1 .. .. x(k+1) = g a (x(k)) = , ξ≥2 IRn → IRn , . .   ξ g a (x) if x ∈ Qξ with any partition Qi of Rn into sets with nonzero measure and g ia (x(k)) = Ai · x(k) + bi . For such a system, eqn. (3.108) yields the FPO P px (x) =

ξ X i=1

px ((Ai )−1 (x − bi )) · | det((Ai )−1 )| · 1gia (Qi ) (x) .

Two–tank system. As last with both connecting valves the left tank √ is constant and x˙ 2 = −c2 x2 for x2 ≥ 0. By

example, consider the two–tank system of Section 1.4 closed and the pump turned off. Hence, the level in the level of the right tank is decreasing with the rate means of eqn. (3.25) the discrete–time description

x1 (k+1) = ga,1 (x) = x1 (k) 1/2 x2 (k+1) = ga,2 (x) = x2 (k) − 2(d3 − x2 (k)3/2 )/(3c2 /4 − 3x2 )

(3.109)

on R2 = [0, hmax ]2 with d = (c22 /4 − c2 Ts x2 (k)1/2 + x2 (k))1/2 and with the sampling −1 −1 is given by ga,1 = ga,1 and ga,2 which is time Ts is obtained. The inverse g −1 a obtained from ga,2 by replacing c2 by −c2 , because√backwards in time, the level in the right tank is increasing with the rate x˙ 2 = +c2 x2 . With this, the FPO of the tank system is given by −1 P px (x, k) = px ( (x1 ga,2 (x2 ))0 , k) · |D|

with D =1−

3(d˜3 − x2 ) 3/2

1/2

1/2

x2 (3c2 /4 + 3x2 )2

+

(3.110)

˜ + c2 /2 · Ts x−1/2 ) − x1/2 d(1 2 2 1/2

c2 /4 + x2

and d˜ = (c22 /4 + c2 Ts x2 (k)1/2 + x2 (k))1/2 . As initial density px (x, 0), a Gaussian distribution around x ¯2 = 0.55 with a standard deviation of 0.02, restricted and normalised to the interval [0.5, 0.6] is used. The sequence obtained from (3.110) with a sampling time of Ts = 5 is shown for k = 0, . . . , 6 on the right–hand side of Figure 3.25.

3.6 General Solution to the Stochastic Representation Problem**

87

3.6.3 The Foias Operator** The FPO of Section 3.6.1 is now extended to dynamical systems with random perturbations of the form Rn × R m → Rn ,

x(k+1) = g(x(k), u(k)) ,

(3.111) n

where the u(k) are independent random vectors, Rn ⊂ IR is a closed set and Rm ⊂ IRm is Borel measurable. It is assumed that eqn. (3.111) describes a regular stochastic dynamical system, i.e. that the following is satisfied [95]: 1. For every fixed u, the function g(x, u) is continuous in x and for every fixed x, it is measurable in u. 2. The random vectors u(k) have values in IRm and are described by a probability measure ν(B), B ∈ B(Rm ) constant over time. 3. The initial random vector x0 has values in Rn and the random vectors x0 , u(0), u(1), . . . are independent. Under these assumptions, the following equation defines an operator describing the evolution of a measure under the dynamical system (3.111) [95]: ˆ : Mfin → Mfin z ˆ k (A) = µk+1 (A) = zµ for all A ∈ B(Rn ) .

Z Rn

Z Rm

 1A (g(x, u))ν(du) µk (dx)

(3.112)

ˆ is called Foias operator on measures. It is straightforward to The operator z show that for g(x, u) = g a (x) + 0u, the Foias operator on measures becomes the FPO on measures:  Z Z Z k k ˆ zµ (A) = 1A (g a (x))ν(du) µ (dx) = 1A (g a (x))µk (dx) Rn Rm Rn Z = 1ga (A) (x)µk (dx) = µ(g −1 a (A)) . Rn

It is easier to interpret eqn. (3.112) if it is written in a slightly different way. ¯ , eqn. (3.101) yields with Similarly to the above, for constant input u(k) = u ¯ ¯ ) = gu ¯ denoted by Pˆ u¯ g(x, u (x) the FPO on measures corresponding to u a where Z Z ¯ −1 k ) (A)) = µ (dx) = 1(gua¯ )−1 (A) µk (dx) Pˆ u¯ µk (A) = µk ((g u a ¯ −1 (A) (g u a)

Z

=

Rn

Rn

¯ k 1A (g u a (x))µ (dx) .

Hence, eqn. (3.112) can be written as Z  Z k k ˆ zµ (A) = 1A (g(x, u))µ (dx) ν(du) n m ZR  R  = Pˆ u µk (A) ν(du) , for all A ∈ B(Rn ) . Rm

(3.113)

88

3. Modelling of Quantised Systems by Stochastic Automata

That is, the Foias operator can be interpreted as “mean” FPO for all inputs u. The integral in eqn. (3.113) “sums” the FPO’s corresponding to each input weighted with the measure ν. From eqn. (3.113), the Foias operator on densities or on the L1 space can be derived in analogy to Section 3.6.1 with the restriction to non–singular transformations and absolute continuous measures. Denoting the density corresponding to the continuous probability measure ν by pu (u), the Foias operator is given by Z (P u px (x, k)) · pu (u) µ(du) , (3.114) z : L1 → L1 , zpx (x, k) = Rm

where P u denotes the FPO corresponding to input u and µ the Borel measure on B(Rm ). An example will be given in the next section. 3.6.4 Representation of Quantised Systems** The non–autonomous quantised system is now described by means of the Foias operator. According to Section 3.2, the input injector yields random [u] vectors u and is described by densities pu (u) for each qualitative input [u]. Hence, for each qualitative input [u], there exists a Foias operator denoted ˆ [u] : Mfin → Mfin describing the dynamics of the quantised system in by z terms of the measure transformation Z   k ˆ z[u] µ (A) = Pˆ u µk (A) ν[u] (du) , for all A ∈ B(Rn ) (3.115) Rm

with

Z ν[u] (B) =

B

p[u] u µ(du) ,

for all B ∈ B(Rm ) ,

where µ denotes the Borel measure. Denoting, as before, the measure of a set A at time k in the state space by µ(A, k), and writing the conditional probability distribution Prob([y(k)] | [U (0 . . . k)], µ0 ) of the quantised outputs at time k as vector p[y(k)] = ( Prob([y(k)]p = 1 | [U (0 . . . k)], µ0 ) . . . Prob([y(k)]p = R | [U (0 . . . k)], µ0 ) )0 , the quantised system is described as follows: Representation of quantised systems ˆ [u(k)] µ(A, k) , µ(A, k+1) = z ˆ [u(k)] µ(A, k) . p[y(k)] = Q R

µ(A, 0) = µ0

(3.116)

3.6 General Solution to the Stochastic Representation Problem**

89

ˆ [u] is the Foias operator on measures corresponding to g In eqn. (3.116) z ˆ [u(k)] µ(A, k) the output measure for the current quantised input [u(k)], and Q R quantiser depending on h and the current quantised input [u(k)] given by ˆ [u(k)] : Mfin → W R Q R ˆ [u(k)] µ(A, k) = p[y(k)] , Q R

pi[y(k)] =

(3.117)

Z h−1 (Qy (i))

µ(dx, k)ν[u] (du) .

ˆ [u] That is, the quantised system is represented by a set of Foias operators z [u] ˆ and output quantisers Q R for each qualitative input [u] = 1, . . . , M . As before, the operator representation (3.116) on measures can be transformed into an operator representation on densities (cf. Section 3.6.2). Example 3.6.2. Consider the quantised system defined in Section 3.5.2 by means

of eqns. (3.66)–(3.72). Due to eqn. (3.113) the FPO in dependence on the input u is needed to determine the Foias operator for each quantised input. From eqn. (3.66), the inverse (gau )−1 (x) = (x+0.1u)/0.8 is obtained, yielding according to eqn. (3.108) the FPO 1 P u px (x) = px ((x + 0.1u)/0.8) · . 0.8 Hence, according to eqn. (3.113), the Foias operators depending on the quantised input [u] in density notation are given by: Z 1 z[u(k)] px (x) = px ((x + 0.1u)/0.8) · (3.118) · pu[u(k)] (u) du , 0.8 IR [u(k)]

with pu according to eqns. (3.69) and (3.70). The solution to the above integral depends on the state density function px and can therefore not be given explicitly. However, it provides an analytical description of the transformed densities. As example, consider the density transformation shown in Figure 3.22. The figure corresponds to the quantised system given by eqns. (3.66)–(3.72). Consider the initial density px (x, 0) = 0.5 · 1[−1,1] (x) and the quantised input [u] = 3. With this, eqn. (3.118) yields: Z 0.5 px (x, 1) = 1[−1,1] ((x + 0.1u)/0.8) · 1[1,∞) exp(−u + 1) du 0.8  IR 0 if x > 0.7 if −0.9 ≤ x ≤ 0.7 = 5/8(1 − exp(10x − 7))  5/8(exp(10x + 9) − exp(10x − 7)) if x < −0.9 . This density is shown for k = 1 in Figure 3.22. For the output quantiser, note that from eqns. (3.67) and (3.68) follows that h−1 (Qy (1)) = {(x, u) | 0.5x − 0.5u < 0} = {(x, u) | x < u} and, equivalently, h−1 (Qy (2)) = {(x, u) | x > u} hold. The output quantiser in eqn. (3.116) yields p[y(k)] = (p1[y(k)] p2[y(k)] )0 . For the example system in density notation this is   Z ∞ Z u Z ∞ Z ∞ 2 p1[y(k)] = px (x, k)dx p[u(k)] du , p = p (x, k)dx pu[u(k)] du . x u [y(k)] −∞

−∞

−∞

u

For the first two time instances the integrals yield p[y(0)] = (1 0)0 and p[y(1)] = (1 0)0 .

90

3. Modelling of Quantised Systems by Stochastic Automata

3.6.5 Convergence of the Approximation** In this section, it is shown that for finer state space partition the qualitative model converges to a precise representation of the quantised system. To formally derive this result, a finite approximation of the FPO for autonomous systems is presented. This approximation became a standard tool in the analysis of nonlinear dynamical system (cf. e.g. [61], [68]). Consider the Borel σ–algebra B(Rn ) on Rn and the autonomous system (3.100). Furthermore, consider a partition of Rn into sets Qx (i), i = 1, . . . , N with nonzero Borel measure µ(Qx (i)) > 0 and the subset ∆N of all L1 functions that are represented by simple functions on the partitioned space ∆N = {s(x) | s(x) =

N X

αi · bi (x) , αi ∈ IR} ⊂ L1 ,

bi (x) =

i=1

1Qx (i) (x) . µ(Qx (i))

The operator PN : ∆N → ∆N defined by PN bj (x) =

N X

Prob(i | j) · bi (x)

(3.119)

i=1

with

Prob(i | j) =

µ(g −1 a (Qx (i)) ∩ Qx (j)) µ(Qx (j))

is called discrete or quantised FPO (cf. e.g. [48], [68], [173]). The quantised FPO has been introduced in [195] and is also called Ulam’s piecewise constant approximation of the FPO [38]. Note that the conditional probabilities of eqn. (3.119) are those of the transition relation Ga (z 0 |z) of the qualitative model in the autonomous case, if uniform distributions are used for all pix (cf. Section 3.5.2, eqn. (3.64)). That is, the quantised FPO is a qualitative model of the autonomous quantised system. Because of the above, results obtained in the literature on the quantised FPO directly apply to qualitative modelling. Two such results are given below. For the formulation of the first result, the density quantiser of Section 3.5.3 is modified according to Z N X ˜ N : L 1 → ∆N , Q ˜ N px (x) = Q px (x)µ(dx) . pi[x] bi (x), pi[x] = i=1

Qx (i)

Note that the weights pi[x] are identical to the weights of the density quantiser given by eqn. (3.79). Lemma 3.6.1. [38],[96] For all px (x) ∈ ∆N the relation ˜ N P px (x) PN px (x) = Q holds.

3.7 Bibliographical Notes

91

According to this lemma, the quantised map of any function px (x) ∈ ∆N by the FPO and by the qualitative model are equal. This result has already been obtained in Section 3.5.3 and is illustrated by Figure 3.22. In fact, the lemma proves that the weights of the two densities shown in the middle of the lower part of Figure 3.22 are identical under the influence of the quantiser. That is, for any quantisation the qualitative model precisely contains the single–step transition probabilities of the quantised system. The second result concerns the convergence of the approximation to the FPO. Convergence of the qualitative model to a representation of the autonomous quantised system Lemma 3.6.2. [38],[96] For all px (x) ∈ ∆N the following relation holds PN px (x) N−→ P px (x) →∞ provided that all partition sets Qx (i) shrink as N grows. This important result means that for increasingly finer partition the quantised FPO converges to the FPO for any transformation g a . Hence, it is proved for the autonomous case that the qualitative model converges to the precise representation of quantised system with increasing number of qualitative states. This shows that the qualitative model satisfies the third modelling aim (cf. Section 3.1.3). It is intuitively clear that the theorem can be extended to the non–autonomous case, i.e. that the qualitative model converges to a set of Foias operators for increasingly finer partition. In fact, the proof given in [96] uses standard techniques of approximation theory and should, therefore, be extendible to the non–autonomous case without any problems. However, this has not been studied in the mathematical literature so far, and a formal proof lies beyond the scope of this book.

3.7 Bibliographical Notes The quantised system is a hybrid system as it concerns discrete as well as continuous dynamics. Hence, its study needs to combine results from discrete and continuous systems theories. The following gives an overview of results in both fields which are relevant for the chapter. Historically, continuous and discrete–event systems theories developed independently of each other leading to well elaborated methods for linear and nonlinear systems on the one hand and discrete–event systems on the other.

92

3. Modelling of Quantised Systems by Stochastic Automata

Recently, there are strong research activities seeking to combine both theories within a unified framework for the representation of hybrid systems, cf. e.g. [2], [3], [13], [22], [60]. These studies are focused on an accurate representation of hybrid phenomena (cf. [3],[13]), the design of supervisory controllers (cf. e.g. [2], [84]) and reachability analysis e.g. in [93], [137], [149]. In contrast to this, the focus here is on the description of the continuous dynamics of the quantised system under the influence of quantisation, and on a concise, purely discrete abstraction of the quantised system. Accordingly, the book is concerned separately with both foundations of hybrid systems theory. That is, for the representation of quantised systems mainly methods from continuous systems theory are used, whereas discrete–event systems play a major role for the abstraction of the quantised system. Continuous systems theory is used here to analyse the dynamics of quantised systems. It has its foundations in linear systems theory, cf. e.g. [41], [49], [112] and can be extended to nonlinear systems that behave locally linear [192], [197]. These studies rely on the assumption that similar initial conditions result in a similar system behaviour, giving rise to the trajectory– oriented viewpoint commonly used in control. However, already very simple nonlinear systems exist that do not have this property. Such systems are no longer adequately described by trajectories but require to consider ensembles of trajectories [95]. For discrete–time systems this ensemble–oriented viewpoint leads to descriptions based on the Frobenius–Perron operator or, in the non–autonomous case, the Foias operator. As shown in this chapter, the quantisation similarly necessitates to consider trajectories bundles. The corresponding studies in this chapter are based on [21], [38], [95], [96] and due to the measure theoretic background of the operators also refer to [39], [63]. The non–autonomous case is only rarely discussed in the mathematical literature. For additive perturbations, an approximation of the Foias operator is discussed in [48]. Other approaches including some convergence results are given in [37] and [57]. Different approaches that have been developed in discrete systems theory are related to this chapter. Of particular interest are approaches which deal with discrete–event abstractions of continuous systems. Works in this field are, like this book, situated between continuous systems theory on the one hand and discrete–event systems theory on the other. First approaches in this area aimed at finding deterministic discrete–event abstractions of continuous systems [83], [199]. However, hard restrictions on the class of continuous systems were necessary for the existence of such abstractions [83]. Without these restrictions, nondeterministic behaviours are obtained requiring the use of nondeterministic discrete–event models. These approaches can be distinguished with respect to the way how time appears in the discrete model and, closely related to this issue, which kind of discrete–event model is used for the abstraction of the continuous system.

3.7 Bibliographical Notes

93

Firstly, there are approaches preserving the continuous–time of a system and, hence, lead to continuous–time discrete–event abstractions in form of bond–graphs [136] or continuous–time hidden Markov processes [205]. The main problem of these approaches is that the discrete models produce a huge number of spurious solutions, i.e. the behaviour of the abstracted discrete– event model differs considerably from that of the original system. Therefore, in most approaches time is discretised, either event–based, i.e. it is sampled whenever a system variable exceeds a certain threshold, or by means of periodical sampling. Nondeterministic automata are used as discrete–event abstractions with event–based sampling in [51], [88], [146], [148] and [154], whereas in [139] and [190] stochastic or hybrid automata are used. Finally, discrete abstractions of discrete–time systems are considered in [109] and [153] which use stochastic or nondeterministic automata as discrete–event model, respectively. A common property of the above approaches is the completeness of the discrete–event abstraction with respect to the original system. That is, the discrete–event model yields all behaviours that the continuous–variable system can show. In general, discrete models yield additional behaviours which are referred to as spurious solutions. The reason for this is given by the fact that quantised systems do not possess the Markov property whereas stochastic automata do [111]. In the book the term “qualitative model” is used for the discrete–event abstraction of a continuous system. This term has its background in artificial intelligence, particularly in the fields called qualitative physics, qualitative reasoning or qualitative simulation, cf. e.g. [32], [35], [89], [90], [151], [200]. The aim of these approaches is to describe the global relation among the most important system variables which is in contrast to the numerical models commonly used in engineering. These methods have been extended to uncertain knowledge representation forms like Bayesian networks, cf. [75], [76], [119], [145], [160]. Though the aim to qualitatively describe a system is the same as in this book, the approaches in the field of artificial intelligence differ significantly from the approach presented here. The main difference is that most approaches in artificial intelligence consider static rather than dynamical system. As far as dynamical systems are concerned, their dynamics are approximated very roughly e.g. by just considering the sign of a variable and of its change [35]. One focus of the methods elaborated in artificial intelligence and computer science is the way how the qualitative knowledge is represented and processed. With respect to this, relational database theory and graph theory provide widely used tools [8], [171]. In [65] these techniques are applied to component–oriented modelling with application to diagnosis. The results ob-

94

3. Modelling of Quantised Systems by Stochastic Automata

tained there are used and extended in this book to the description of dynamical systems in form of stochastic automata networks. The idea to use stochastic automata as discrete–event abstractions of discrete– time systems is in continuation of the ideas presented in [109] that are elaborated in more detail in [97]. The chapter contains several new results if compared to these publications. One of the main differences concerns the way how the discrete–event abstraction of a discrete–time quantised systems by means of a stochastic automaton is defined (cf. Definition 3.5.1). The definition given here is the first which is completely embedded in probability theory and which is precise in detail. The quantised system is formally introduced as stochastic process and is described by the temporal evolution of an initial density function in the state space extending former descriptions based on sets of state sequences. Theorem 3.5.1 proves the completeness of the discrete–event abstraction extending an earlier proof concerning nondeterministic automata [97]. Furthermore, the stochastic process represented by the quantised system is constructively described by means of the Frobenius–Perron and Foias operators in Section 3.6. This shows a novel perspective on the representation of quantised systems and on the application of the Frobenius–Perron and Foias operators. Finally, the relation between Definition 3.5.1 and the piecewise constant approximation of the Frobenius–Perron operator is shown. A first outcome of this new relation is the application of a convergence result obtained for the approximation of the Frobenius–Perron operator. For the abstraction of quantised systems this leads to the novel result that the qualitative model converges to a precise representation of the quantised system for increasingly finer state space partitions (cf. Lemma 3.6.2).

4. Further Aspects of Modelling Quantised Systems*

This chapter concerns further aspects of modelling quantised systems. Section 4.2 presents methods to compute the qualitative model for a given quantised system. Further, it is outlined how complete models can be obtained in case of parameter uncertainties or noise. In Section 4.3, the dependence of the stationary and dynamical behaviour of the qualitative model upon the state space partition is discussed.

4.1 Motivation* For the application of qualitative models, it is necessary to compute the behavioural relation for a large system class. In principle, this is done by evaluating eqn. (3.64) for all z 0 , w, z and v in a step usually referred to as abstraction of a continuous–variable system to a qualitative model. This is because the qualitative model resulting from this computation describes the system on a higher level of abstraction (cf. [88], [109], [148], [153]). Practically, the computation of the behavioural relation is a major issue. This has mainly two reasons. The first is that the limits of computational resources can be reached easily already for relatively small systems. The second is that for the application of the computed qualitative model its completeness is a necessary condition. This means that the computation must not yield a probability of zero for a transition that can occur. Section 4.2 concerns both problems. Two methods are presented for computing the behavioural relation, while only the second can guarantee the completeness of the computation result. Section 4.2.4 presents an extension important for the practical application. This concerns a method to consider parameter uncertainties and noise in the abstraction of qualitative models. These considerations are closely related to the computational aspects studied before. Section 4.3 concerns a completely different topic, namely the accuracy with which the qualitative model describes the quantised system. Obviously, the J. Schröder: Modelling, State Observation and Diagnosis of Quantised Systems, LNCIS 282, pp. 95-122, 2003. Springer-Verlag Berlin Heidelberg 2003

96

4. Further Aspects of Modelling Quantised Systems*

accuracy mainly depends on the quantisation of the signals. Whereas the partitions of the inputs and outputs are typically determined by the technological system, the partition of the state space can always be adapted to the required accuracy of the qualitative model. This concerns both the number of partition sets and their shape. In Section 4.3, conditions on the state space partition are derived under which the qualitative model has, in a sense explained there, ideal stationary and dynamical behaviour. The aim is to derive guidelines for the choice of the state space partition in practical applications. Remark 4.1.1. The discussion in Section 4.3 concerns merely the partitions without considering the density functions pix (x) or piu (u) on each set. For simplification, it is therefore assumed that the densities represent uniform distributions on all bounded sets and are described by declining exponential functions on the unbounded sets. However, the results hold for a larger class of densities which is not investigated in detail.

4.2 Computation of the Behavioural Relation* 4.2.1 Computation Principle* To explain how the behavioural relation is computed, eqn. (3.64) is first written as µn+m (Qxu (z 0 , w, z, v)) L(z 0 , w|z, v) = (4.1) µn+m (Qxu (z, v)) with Qxu (z 0 , w, z, v) =

  x u

þ  þ g(x, u) ∈ Qx (z 0 ), h(x, u) ∈ Qy (w) þ þ x ∈ Qx (z), u ∈ Qu (v)

⊆ Qxu (z, v) . This shows two important aspects: 1. The probabilities of all quantised successor state/output pairs (z 0 , w) for a given pair (z, v) are obtained from a partition of the set Qxu (z, v) into subsets Qxu (z 0 , w, z, v). This partition depends on the system equations (3.1) and (3.2) and on the partitions of the state and output space. 2. The behavioural relation can be computed separately for each pair (z, v). Example 4.2.1. Figure 4.1 shows this for a scalar input and scalar state space. It shows the region Qxu (2, 2) of a partitioned input and state space. The map of this region with some functions g and h shows that for all points [g(x, u)] = 2 holds, whereas some points (x, u) ∈ Qxu (2, 2) yield [h(x, u)] = 2 while for others [h(x, u)] = 3 holds. Accordingly, the set Qxu (2, 2) is partitioned into two sets Qxu (2, 2, 2, 2) and Qxu (2, 3, 2, 2). Consequently, the behavioural relation has two nonzero entries for (z, v) = (2, 2), namely, L(2, 2 | 2, 2) > 0 and L(2, 3 | 2, 2) > 0. All other entries for (z, v) = (2, 2) are zero. The nonzero values are obtained according to eqn. (4.1) by computing the measures of the sets Qxu (2, 2, 2, 2), Qxu (2, 3, 2, 2) and Qxu (2, 2).

4.2 Computation of the Behavioural Relation* u

Qxu( 2,3,2,2)

y

IRn+m g h

[u]=2

Q xu(z=2,v=2 )

[x ]=1

[x ] =2

IRn+r map of Q xu(2,2)

[ y]=2 [ y] =1

Qxu( 2,2,2,2)

[u]=1

[ y] =3

97

[x ]=3

x

[x ]=1

[x ] =2

[x ]=3

x

Fig. 4.1. Partition of Qxu (z, v) for the computation of the behavioural relation.

Practically, the measures in eqn. (4.1) can rarely be analytically described even in case of simple systems. Furthermore, even if the sets Qxu (z 0 , w, z, v) can be described analytically by various inequalities, computing their measures requires the computation of the integral (3.65) in a multi–dimensional space which is still difficult. Therefore, the aim of the methods described below is to provide computationally feasible ways to approximate the measures in eqn. (4.1). The crucial point is to guarantee that the computation does not yield a zero for sets with nonzero measure to ensure the completeness of the resulting model (cf. Lemma 3.5.1). 4.2.2 Point–Mapping* Point–mapping is the standard method to compute transition probabilities from one region of a partitioned space to another. In the following, the method is briefly summarised and applied to compute an estimate of the behavioural relation. The main idea is to take a selection of κ points of each region Qxu (z, v) denoted as ˜ xu (z, v) = {(x1 , u1 ), (x2 , u2 ), ..., (xκ , uκ )}, Q with

(xi , ui ) ∈ Qxu (z, v), i = 1, ..., κ .

The points must be selected such that they are distributed according to the density z v pzv xu (x, u) = px (x) · pu (u)

(4.2)

˜ xu (z, v), eqns. (3.1) on Qxu (z, v) (cf. Section 3.5.2). For each point (xi , ui ) of Q and (3.2) are used to compute the points (g(xi , ui ), h(xi , ui )). The state and output space partitions are used to determine the pair (z 0 , w) of the quantised successor state and output. Doing this for all points, the sets ˜ xu (z 0 , w, z, v) = {(x, u)|(x, u) ∈ Q ˜ xu (z, v), g(xi , ui ) ∈ Qx (z 0 ) Q and h(xi , ui ) ∈ Qy (w)}

98

4. Further Aspects of Modelling Quantised Systems*

can be constructed for all z 0 ∈ Nz , w ∈ Nw . With this, the behavioural relation L of the qualitative model is approximated by ˜ xu (z 0 , w, z, v)) Λ(Q Lþ (z 0 , w|z, v) = , (4.3) ˜ xu (z, v)) Λ(Q where Λ(·) denotes the number of points contained in the set. According to the law of large numbers, the estimates (4.3) of the transition probabilities converge to those defined by eqn. (3.64) or (4.1) for κ → ∞, [97]. The described method can directly be implemented on a computer as the sets ˜ xu (z, v) are finite, and thus a finite number of mappings with eqns. (3.1) Q ˜ xu (z 0 , w, z, v). Furthermore, the implementation is simple and (3.2) leads to Q as only quantisation and mapping functions are required. However, the problem of point–mapping is that the completeness of the obtained model is not guaranteed. More precisely, for the obtained behavioural relation Lþ , the relation TLÿ (z, v) ⊆ TL (z, v)

∀ (z, v) ∈ Nz × Nv

(4.4)

rather than condition (3.95) holds. This means that the SA is complete only in the ideal case that TLÿ (z, v) = TL (z, v) holds for all pairs (z, v). In Chapter 9 it is shown that even for simple systems, the number of points that must be mapped until TLÿ (z, v) = TL (z, v) is obtained, is so high that the computation would last several years. Hence, practically only incomplete models not fulfilling the modelling aim can be obtained by means of point–mapping. Such a model cannot be used for process supervision. As TL (z, v) is unknown, there is no way to check whether a model using Lþ is complete or not. Therefore, point-mapping is not suited for the abstraction task. Remark 4.2.1. Instead of distributing the points according to eqn. (4.2), it is also possible to distribute the points uniformly over Qxu (z, v) and use a measure weighted according to (4.2) instead of the counting measure Λ. 4.2.3 Hyperbox–Mapping* In this section, a method to compute the behavioural relation is presented that guarantees completeness. For this it must be assumed that the functions g and h satisfy a Lipschitz condition, i.e. that a number ψ ∈ IR+ exists such that þþ þþ þþ þþ þþ þþ þþ g(x1 , u1 ) − g(x2 , u2 ) þþ þþ ≤ ψ · þþ x1 − x2 þþ þþ (4.5) þþ u1 − u2 þþ þþ h(x1 , u1 ) − h(x2 , u2 ) þþ ∞

∞

holds. For practical reasons the infinity norm || · ||∞ is used in eqn. (4.5). For simplicity, it is assumed that all partitions are orthogonal, i.e. that all dimensions of the signal spaces are divided independently into intervals resulting in regions in form of hyperboxes (cf. Figure 4.2). An extension of this method to non–orthogonal partitions is given in [43].

4.2 Computation of the Behavioural Relation* u

Qxu( 2,3,2,2)

g h

r0

[u]=2

y

IRn+m

(x 0 ,u 0 )

[ y] =3

map of Qxu( 2,2)

r 0.ψ [ y] =1

Qxu(2,2,2,2)

[x ]=1

[x ] =2

IRn+r

[ y]=2 Q xu(z=2,v=2 )

[u]=1

99

[x ]=3

x

( x 0+ ,y0 )

[x ]=1

[x ]=2

[x ]=3

x

Fig. 4.2. Hyperbox–mapping.

The idea is described based on the same example as in Section 4.2.1 and is illustrated by Figure 4.2. On the left–hand side of the figure, the state– input space is shown. The dashed lines symbolise the partition bounds. The set Qxu (2, 2) is considered. For simplicity, it is assumed that Qxu (2, 2) is quadratic with a sidelength of 2r0 . Hence, using the centre point (x0 , u0 ), the set Qxu (2, 2) can be described by Qxu (z, v) = {(x, u) | ||(x, u) − (x0 , u0 )||∞ ≤ r0 } .

(4.6)

That is, the set Qxu (z = 2, v = 2) can be represented by a box {(x0 , u0 ), r0 } in the state–input space given by its centre (x0 , u0 ) and r0 which is referred to as radius of the box due to its interpretation as radius of a circle when denoting the set (4.6) in the Euclidean norm. Using the functions g and h, the point (x0 , u0 ) is mapped to the state–output space resulting in the point (x0+ , y0 ) depicted in black on the right–hand side of Figure 4.2. Due to the Lipschitz constraint (4.5), the map of the set Qxu (z, v) is known to be contained in the set = {(x, y) | ||(x, y) − (x0+ , y0 )||∞ ≤ r0 · ψ} , Qapprox xy i.e. the map of the box {(x0 , u0 ), r0 } in the state–input space is conservatively approximated by the box {(x0+ , y0 ), r0 ψ} in the state–output space. Figure 4.2 shows that the approximation box completely covers the map of Qxu (2, 2). Quantising the set Qapprox in the state-output space reveals that it overlaps xy with regions corresponding to several successor state/output combinations (z 0 , w). According to Figure 4.2, possible combinations (z 0 , w) due to Qapprox xy are TL⊕ (z = 2, v = 2) = {(1, 1), (2, 1), (1, 2), (2, 2), (1, 3), (2, 3)} , whereas quantising the map of Qxu (z, v) yields: TL (z = 2, v = 2) = {(2, 2), (2, 3)} .

(4.7)

100

4. Further Aspects of Modelling Quantised Systems*

yields TL⊕ ⊇ TL Hence setting the transitions of L⊕ according to Qapprox xy and, thus, according to Lemma 3.5.1, a complete model of the quantised system. The above method obviously yields a complete model of the quantised system. It is now extended such that further calculations yield better approximations of the behavioural relation L. Whenever the set Qapprox yields more than one xy quantised successor state/output combination like in (4.7), the set Qxu (z, v) in the state-input space is partitioned into subsets as follows: By introducing two additional points per dimensions, the box {(x0 , u0 ), r0 } is divided into smaller boxes. In case of Figure 4.2, this results in nine boxes with radius r0 /3. The additional points, depicted in medium grey, are mapped, resulting in boxes with radius ψr0 /3 in the state–output space. Only those boxes that do not yield a unique quantised successor state/output pair (z 0 , w) need further investigation and can be subdivided as long as needed to obtain the desired precision of the transition probabilities. In Figure 4.2, only the map of the box in the upper left corner yields two possible pairs (z 0 , w). It is further subdivided into nine boxes with radius r0 /9 with the centre point shown in light grey. Of these boxes, three yield again no unique quantised successor state/output pair and could be further subdivided. Continuing the subdivision, there will, in general, always remain boxes that yield more than one quantised successor state/output pair. Thus, the computation needs to be stopped after some subdivisions. To guarantee completeness, the measures of the boxes still yielding more than one successor state/output pair (z 0 , w) after the last subdivision are equally distributed among the measures µn+m (Qxu (z 0 , w, z, v)) of the corresponding sets. As an example, the measures obtained from the left–hand side of Figure 4.2 after the two subdivision steps are computed. It is assumed that p2x (x) and p2u (u) are uniform distributions such that eqn. (4.1) can be used with the Lebesgue measure λn+m . From the figure, it can be seen that eight of the boxes with radius r0 /3 were not further subdivided as the approximation of their map uniquely yields (z 0 , w) = (2, 2). Their area is (2r0 /3)2 each, which completely counts for λn+m (Qxu (2, 2, 2, 2)). The map of the ninth box in the upper left corner of Qxu (2, 2) yielded two different quantised state/output pairs (z 0 , w) and was therefore further subdivided into nine boxes. As the figure shows, six approximated maps of these boxes uniquely yield (z 0 , w) = (2, 2), whereas the other three again yield both possible quantised state/output pairs. Stopping the subdivision after two steps, one half of the area of the smallest boxes which is (2/9r0 )2 is added to λn+m (Qxu (2, 2, 2, 2)) and the other half to λn+m (Qxu (2, 3, 2, 2)). Accordingly, the following approximation for the measures is obtained: λn+m (Qxu (2, 2, 2, 2)) ≈ 8 · (2/3r0 )2 + 6 · (2/9r0 )2 + 3 · (2/9r0 )2 /2 λn+m (Qxu (2, 3, 2, 2)) ≈ 3 · (2/9r0 )2 /2 .

4.2 Computation of the Behavioural Relation*

With λn+m (Qxu (2, 2)) = (2r0 )2 , behavioural relation:   53/54 L⊕ (z 0 , w | 2, 2) = 1/54  0

101

eqn. (4.1) yields the following entries of the for z 0 = 2, w = 2 for z 0 = 2, w = 3 otherwise .

Extension to non–quadratic boxes. Practically, the partition regions are in general not quadratic and the system variables have different value domains. Therefore, rectangular hyperboxes are used instead of quadratic ones which were used above only to explain the main idea. The boxes are thus described by a centre point (x0 , u0 ) and a radius vector r 0 ∈ IRn+m . With this, the description of the set (4.6) becomes Qxu (z, v) = {(x, u)|x|i − x0,i | < r0,i , i = 1, ..., n and |ui − u0,i | < r0,i , i = n+1, ..., n+m} . Instead of a Lipschitz number ψ, a nonnegative matrix Ψ ∈ IR(n+r)×(n+m) is used in eqn. (4.5) mapping the radius vector r 0 from the state-input space to the radius vector r 00 = Ψ · r 0

(4.8)

in the state-output space. The elements (Ψ )ij of the Lipschitz matrix describe how much the mapped point changes in its i–th dimension, if changing the original point in its j–th coordinate. In general, all entries of P si are nonzero. The described method is explained in detail in [43]. A similar method for autonomous systems is described in [61]. The properties of the hyperbox– mapping method are summarised in the following lemma. Abstraction by hyperbox–mapping Lemma 4.2.1. By means of the hyperbox–mapping method a complete model is obtained. Furthermore, the estimated behavioural relation converges to that of Definition 3.5.1 for increasing number of subdivisions. Proof. See Appendix C.4, [61].

t u

Figure 4.3 illustrates the above results by comparing the number of nonzero entries of the approximate behavioural relations obtained by point–mapping or by hyperbox–mapping, respectively. The number of obtained nonzero transitions is shown over the number of mapped points or the number of mapped boxes. For an infinite number of points or boxes, both methods yield the correct number |TL | of entries as given by the behavioural relation L of the qualitative model. According to Lemma 4.2.1, each result obtained by means of hyperbox–mapping with a finite number of boxes yields a complete model,

102

4. Further Aspects of Modelling Quantised Systems*

Fig. 4.3. Comparison of the approximate behavioural relations obtained by point–mapping or hyperbox–mapping.

i.e. L⊕ has at least as many nonzero entries as L. On the other hand, point– mapping yields, in general, an incomplete model, i.e. a behavioural relation Lþ with fewer entries than L. Note that with the number of points or boxes to be mapped also the computation time increases. 4.2.4 Parameter Uncertainties and Noise** One motivation for using qualitative rather than numerical models is that often no precise description of the system behaviour can be found. Consequently, numerical models used for abstraction are typically models based on “first principles” neglecting higher order effects and are, hence, no precise descriptions. Accordingly, the assumption that a precise numerical model (3.1) and (3.2) is given in a practical application has to be relaxed. Instead of the model (3.1) and (3.2) considered so far as core of the quantised system, the description x(k+1) = g(x(k), u(k), ρ(k)) , y(k) = h(x(k), u(k), ρ(k)) ,

x(0) = x0

(4.9) (4.10)

is used, where uncertainties are explicitly represented by a parameter vector ρ ∈ Rq ⊆ IRq . The uncertainties are modelled as static random process described by the probability density Z pρ : Rq → IR+ , with pρ (ρ)dρ = 1 . (4.11) Rq

That is, according to eqns. (4.9) and (4.10), the successor states and outputs are random, depending on the vector ρ determined at each time instance k independent of the state, input or previous parameter values.

4.2 Computation of the Behavioural Relation*

103

The qualitative model of Definition 3.5.1 can easily be extended to describe the quantised system with the continuous–variable model (4.9) and (4.10). This is because the uncertain parameters are of the same nature as the uncertainties brought about by the input injector (cf. Section 3.2.1). Also in case of uncertainties, the qualitative model shall describe the conditional probability of a transition in the quantised system but, in addition to Definition 3.5.1, considering all possible parameter values ρ. This is achieved by extending the definition of the behavioural relation to L(z 0 , w|z, v) :=

µn+m+q (Qxuρ (z, v) ∩ (gh)−1 (Qxy (z 0 , w))) µn+m+q (Qxuρ (z, v))

(4.12)

with Qxuρ (z, v) = Qx (z) × Qu (v) × supp(pρ (ρ)) ⊆ Rn+m+q ,     x þþ 0  g(x, u, ρ) ∈ Qx (z ), (gh)−1 (Qxy (z 0 , w)):=  u  þþ ⊆ Rn+m , h(x, u, ρ) ∈ Qy (w)   ρ and the measure µn+m+q (Qxuρ ) =

Z (

N X

Qxuρ i=1

pix (x)) · (

M X

(4.13) (4.14)

pju (u)) · pρ (ρ) dx du dρ (4.15)

j=1

of an arbitrary measurable set Qxuρ ⊆ Rn+m+q . For the computation of the behavioural relation (4.12) the methods described in the previous section directly apply. Bounded additive noise. A special case of eqns. (4.9) and (4.10) is additive noise x(k+1) = g(x(k), u(k)) + ρx (k) , y(k) = h(x(k), u(k)) + ρy (k) ,

x(0) = x0

(4.16) (4.17)

with the random variables ρx ∈ Rn and ρy ∈ Rr described by density functions pρx (ρx ) and pρy (ρy ), respectively. It is further assumed that the noise is bounded and uniformly distributed over a set of values according to  1/λn (Qρx ) ρx ∈ Qρx pρx (ρx ) = , Qρx = {ρx ||ρx − ρx,0 ||∞ ≤ r x,0 } (4.18) 0 otherwise and pρy (ρy ) =



1/λr (Qρy ) ρy ∈ Qρy , Qρy = {ρy ||ρy − ρy,0 ||∞ ≤ r y,0 } . (4.19) 0 otherwise

This makes it possible to abstract the qualitative model with almost no additional effort compared to the undisturbed system. The abstraction of a quantised system with (4.16)–(4.19) can be performed by applying the hyperbox– mapping method to the undisturbed system using eqns. (4.16)–(4.17) with

104

4. Further Aspects of Modelling Quantised Systems*

ρx (k) = ρx,0 and ρy (k) = ρy,0 and by determining the radii of the boxes mapped to the state–output space by r 00 = Ψ · r 0 + (r x,0

r y,0 )0

(4.20)

(cf. Section 4.2.3). This is obviously the same as applying the hyperbox– mapping to the abstraction task defined by (4.12) and the quantised system with (4.16)–(4.19) without performing any subdivision of the hyperboxes in the dimensions of the parameter space. The described method is interesting for the practical application of qualitative modelling. All kinds of uncertainties and noise can be conservatively approximated by using the two vectors r x,0 and r y,0 . The qualitative model is computed by the same method as before but using (4.20) instead of (4.8). Note that the uncertainties described by r x,0 and r y,0 introduce a kind of “natural bound” for a reasonable number of subdivision used in the hyperbox–mapping as they introduce by eqn. (4.20) a lower bound for the size of the mapped box.

4.2.5 Remarks** Other means to obtain a qualitative model. Definition 3.5.1 describes a systematic way to obtain a qualitative model from a numerical model of the quantised system. However, there are several alternatives to obtain a qualitative model. One is stochastic qualitative identification described in [97] and applied to an industrial application in [101]. This method is based on data archives containing measurements of the system variables over a long period of time covering all possible operating conditions. The qualitative model is obtained by evaluating the frequency of measured qualitative state changes. As the qualitative model describes the system on an abstract level, it is also possible to obtain the model by using expert knowledge in form of rules or logic formulae. Similarly, other discrete representations like Petri nets can be transformed into SA [115]. A combination of qualitative identification and expert knowledge for qualitative modelling has been used in [101]. Other concise models of the quantised system. Section 3.5.2 defines the qualitative model such that regions in the state space are the states of the qualitative model. An alternative way is described in [100], [153] where the current quantised output as well as the l previous quantised inputs and outputs are used as states of a nondeterministic automaton. In [153] the I/O– completeness of such an automaton with respect to the quantised system is shown. Furthermore, it is shown that the length l of the “history” stored in

4.3 Relations between Quantisation and Properties of the Model*

105

the automaton state leads to a totally ordered set of qualitative models with respect to their accuracy (cf. relation (3.96)).

4.3 Relations between Quantisation and Properties of the Model* 4.3.1 State Space Partitions Adapted to the Stationary Behaviour* The stationary behaviour of the qualitative model is given by the stationary behaviour of the SA derived in Section 2.5.3. It is investigated under which conditions the stationary distributions of the qualitative model coincides with the stationary distributions of the quantised system. Stationary behaviour of autonomous quantised systems. First, only autonomous systems are considered. As explained in Section 3.4.2 the dynamics of the autonomous quantised system is described by px (x, k+1) = P px (x, k) ,

px (x, 0) = px0 (x) ,

where P is the Frobenius–Perron operator of the continuous–variable system. A stationary density p¯x (x) of the Frobenius–Perron operator must satisfy p¯x (x) = P p¯x (x) .

(4.21)

Such a density function is also called invariant density. With this, the aim that all stationary distributions of the qualitative model are equal to those of the quantised system can be formulated as !

¯ z = QN p¯x (x) , p

(4.22)

where QN denotes the density quantiser defined by eqn. (3.79). In the mathematical literature, the stationary densities of the Frobenius–Perron operator have been studied in detail (cf. e.g. [62], [95]). However, the focus there is on cases in which the system (3.46) behaves chaotically or its attractors have complex structures [87], [192]. Such behaviours are of minor interest in the context of qualitative modelling, as systems treated on a supervision level are in general controlled systems with simpler dynamics. Therefore, the following investigations are restricted to systems for which the limit value ¯ = limk→∞ x(k) exists for all initial conditions x0 ∈ Rn . The points x ¯ in x the state space to which the state of the system converges are called fixed ¯ = g a (¯ points of the system. They are obtained from the condition x x). In the following, it is assumed that the fixed points do not lie on a boundary between two partition sets.

106

4. Further Aspects of Modelling Quantised Systems*

Asymptotically stable systems. A system is called asymptotically stable ¯ [87]. if every trajectory ends in a unique asymptotically stable fixed point x Hence, for any given partition, the quantised system has the unique quantised stationary distribution [¯ x]

¯ qs p z = δN ,

(4.23)

where δ iN ∈ W N denotes the unit vector (0 . . . 1 . . . 0) where the i-th element is equal to one. The following theorem formulates a necessary and sufficient condition on the state space partition such that the qualitative model satisfies ¯ qs ¯z = p p z . Lemma 4.3.1. The qualitative model of an autonomous quantised system ¯ with embedded continuous–variable system with a single stable fixed point x has the stationary distribution (4.23) if and only if the state space partition is such that the automaton state z¯ = [¯ x] is absorbing, i .e. Ga (¯ z | z¯) = 1 holds. Proof. See Appendix C.5.

t u

0.36

02

0.53 0.57

0.36

0. 0.15

0.51

0.01 0.07

0.51

3

0.81

06

0.

0.81

0.34

11

0.

0.19

0.93

0.09

3

0.11

0.34

2

1

5

06

0.

0.17

07

0.

0.27

0.19

6

03

0.13

0.13

1

9

0.5

0.11

0.49

6

0.82

5

0.06

0.

4

02

0.12

0.2

8 0.05

7

0.36

0.94

9

0.06

0.94

8

0.

0.53

0.73 0.18

0.06

03

0.

4

0.09

7

0.54

0.76 0.35

1

0.19

2

0.09

0.85

Fig. 4.4. Automaton graphs of the two–tank system with orthogonal (left) and non–orthogonal (right) state space partition.

The condition Ga (¯ z | z¯) = 1 means that all points of the set Qx (¯ z ) are mapped onto this set, i.e. g a (x) ∈ Qx (¯ z ) must hold for all x ∈ Qx (¯ z ). From the ¯ asymptotic stability of the system, it is clear that a neighbourhood of x and, hence, a partition which fulfils this condition always exists. For sampled continuous–time systems, these are for example all sets Qx (¯ z ) bounded by a contour line v(x) = const. of a Ljapunov function v(x) of the system. Concerning the limiting distribution of the qualitative model, the following holds.

4.3 Relations between Quantisation and Properties of the Model*

107

Lemma 4.3.2. Under the conditions of Lemma 4.3.1 the stationary distribution is the limiting distribution of the qualitative model for all initial conditions. Proof. See Appendix C.6.

t u

Example 4.3.1. Consider the tank system of Section 1.4 with the valve V 12l closed, V12u opened, constant pump velocity vp (k) = 0.5 and no leak. With these settings, the two–tank system is autonomous, and asymptotically stable in Rn = [0, 0.6] × ¯ = (0.335 0.141)0 . With the state space partition shown on the [0, 0.6] with x left–hand side of Figure 3.6 and a sampling time of Ts = 5s, the qualitative model shown on the left–hand side of Figure 4.4 is obtained. The SA is reducible. From Lemma 2.5.4 it can be calculated that the SA has one stationary distribution, namely ¯z = ( 0 p

0.572

0

0

0.221

¯ qs p z

0

0

0.046

0.161 ) ,

(4.24)

δ 59 .

¯ yields whereas quantising x = That is, for the correct stationary qualitative state z¯ = 5 the qualitative model merely yields a probability of 0.221 for k → ∞. In Figure 4.5, an alternative state space partition is shown with several trajectories of the two–tank system that have each the temporal length of the sampling period. It can be seen that the elliptic set is chosen such that all trajectories point into that region. The qualitative model resulting from this partition is shown on the right–hand side of Figure 4.4, where the automaton states correspond to the nine regions like in Figure 3.6. It can be seen that Ga (5 | 5) = 1 holds, and that both ¯ qs ¯ z = δ 59 . the qualitative model and the quantised system yield p z = p

Fig. 4.5. Non–orthogonal partition of the state space for correct stationary behaviour of the qualitative model.

Systems with multiple fixed points. The above result is now extended ¯ i, i = to the case that the system has κ asymptotically stable fixed points x

108

4. Further Aspects of Modelling Quantised Systems*

1, . . . , κ . The attraction area of each asymptotically stable fixed point is denoted by X i , assuming that the attraction areas cover almost the whole ˆ = Rn \ Sκ X i of the state space state space. That is, the “remainder” X i=1 consisting of points not converging to one of the asymptotically stable fixed ˆ ) = 0. That is, for almost all ¯ i is assumed to have zero measure: λn (X points x ˆ , the trajectory ends in one fixed point x ¯ i. initial conditions, i.e. for x0 6∈ X The stationary distributions of the quantised system can be formulated like above for any partition. Only for simplicity of presentation it is assumed that each fixed points lies in a different partition set. With this, there are κ quantised stationary distributions of the quantised system: [¯ x ]

i ¯ qs p z,i = δ N ,

i = 1, . . . , κ .

(4.25)

Recall that an autonomous SA must have κ irreducible sets of states in order ¯ iz (cf. Section 2.5.3). Furto posses κ elementary stationary distributions p ther, all stationary distributions of the SA are convex combinations of the ¯ iz as described by eqn. (2.54). As the elementary stationary distributions p stationary distributions of the quantised system are given by eqn. (4.25), the elementary distributions of the SA must have the form (4.25). Otherwise the stationary behaviour of the SA cannot be equal to the stationary behaviour of the quantised system. Denoting the set of all quantised states containing ¯x = {[¯ a fixed point by N x1 ], [¯ x2 ], . . . , [¯ xκ ]} ⊆ Nx , the following result is obtained. Identical stationary distributions of an autonomous quantised system and its qualitative model Theorem 4.3.1. The qualitative model of an autonomous quantised system with embedded continuous–variable system with κ asymptoti¯ i , i = 1, . . . , κ has κ stationary distributions cally stable fixed points x of the form (4.25) if and only if the state space partition is such that all ¯x are absorbing, i .e. Ga (¯ qualitative states z¯i = [¯ xi ], [¯ xi ] ∈ N zi | z¯i ) = 1 holds for all i = 1, . . . , κ. Proof. See Appendix C.7.

t u

Note that Theorem 4.3.1 does not mean that for a given numerical initial ¯ i the limiting distribution of the qualitative model state x0 converging to x [x ] [x¯ ] initialized with pz (0) = δN 0 is δ N i . Theorem 4.3.1 only guarantees that the the stationary distributions of the quantised system are also stationary distributions of the qualitative model. However, the theorem does not say for which initial conditions the limiting distribution of the SA, which is always a convex combination of its elementary stationary distributions, coincides with the limiting distribution of the quantised system. Hence, choosing the initial

4.3 Relations between Quantisation and Properties of the Model*

109

[x ]

distribution pz (0) = δN 0 for the SA does not mean that limk→∞ pz (k) = [x¯ ] δ N i holds. However, the partition can be chosen such that [x¯ ]

[x ]

¯ i = lim x(k), x(0) = x0 ⇒ δ N i = lim pz (k), pz (0) = δ N 0 x k→∞

k→∞

(4.26)

ˆ . The holds for almost all x0 ∈ Rn , i.e. all initial conditions satisfying x0 6∈ X following results gives a necessary and sufficient condition for the state space partition such that relation (4.26) holds. Lemma 4.3.3. Relation (4.26) holds for almost all x0 ∈ Rn if and only if, in addition to the conditions of Theorem 4.3.1, the state space partition is ˜x,i , i = 1, . . . , κ such that there exists a partition of the set Nx into sets N ˜ with [¯ xi ] ∈ Nx,i , i = 1, . . . , κ that are closed sets of states in the qualitative model. Proof. See Appendix C.8.

t u

Systems with stable invariant manifolds. Before illustrating the above results by an example, they are first extended to systems having a continuum of asymptotically stable fixed points, so called asymptotically stable invariant manifolds [9], [26]. For simplicity of presentation, it is assumed that the system has one compact invariant manifold. In contrast to the above, the invariant manifold is given by a set ¯ ⊂ Rn = {¯ ¯ = g (¯ x|x x)} , (4.27) X a

lying in an 0 < n ¯ ≤ n dimensional subspace of Rn . Asymptotic stability of ¯ holds for all the manifold means that it is assumed that lim k→∞ x(k) ∈ X x(0) ∈ Rn . ¯ does not lie on the boundary between two partition It is assumed that X sets. More precisely, this means that if δQx (i) denotes the hull of a partition ¯ ∩δQx (i)) = 0 holds for all i = 1, . . . , N . However, set, it is assumed that λn¯ (X the invariant manifold can extend over several partition sets. The set of all quantised states in which the manifold lies is denoted by ¯ ) > 0} = {[x] , [x] , . . . , [x] } ⊆ Nx . (4.28) ¯x = {[x] | λn¯ (Qx ([x]) ∩ X N 1

2

κ

With this, Theorem 4.3.1 and Lemma 4.3.3 equivalently hold, i.e. if and ¯x are absorbing and each qualitative only if all qualitative states [x]i of N ˜x,i , state belongs to a different closed set of states Nx,i , where the sets N i = 1, . . . , κ partition Nx , then the limiting distributions of the qualitative model and the quantised system coincide for almost all initial conditions x0 or [x0 ] respectively. That is, the same conditions on the state space partition are obtained if a continuum of asymptotically stable fixed points lies in a set Qx (i). Example 4.3.2. Consider the tank system of Section 1.4 with the valve V 12l closed, V12u opened, pump off and no leak in the left tank. With this, the autonomous system has an asymptotically stable compact invariant manifold given by:

110

4. Further Aspects of Modelling Quantised Systems* ¯ = {x = (x1 , x2 )0 | x1 ∈ [0, 0.3], x2 = 0} . X

(4.29)

On the left–hand side of Figure 4.6, the invariant manifold and trajectories of the system each with a temporal length corresponding to the sampling time of 5 seconds are shown. Furthermore, the figure shows the state space partition. The partition is such that the conditions of Theorem 4.3.1 as well as of Lemma 4.3.3 are satisfied. This can be verified at the resulting qualitative model shown on the right–hand side of the figure. The invariant manifold lies in the two partition sets Qx (1) and Qx (2). Hence, according to Theorem 4.3.1, Ga (1|1) = 1 and Ga (2|2) = 1 must hold, which is satisfied by the SA. Lemma 4.3.3 is satisfied because the set of automaton states ˜x,1 = {1, 4, 7} and N ˜x,2 = {2, 3, 5, 6, 8, 9} can be partitioned into two closed sets N ¯ 1z = δ 19 containing [x] = 1 or [x] = 2 respectively. It can easily be calculated that p ¯ 2z = δ 29 are the two stationary distributions of the qualitative model. and p

Fig. 4.6. Trajectories of the two–tank system and state space partition (left) and resulting qualitative model (right).

The SA shown in Figure 4.6 can be compared to the SA shown on the left–hand side of Figure 2.5 which is the qualitative model of the two–tank system for the same configuration but with the orthogonal state space partition shown on the left– hand side of Figure 3.6. Due to the rectangular shape of Qx (2) the SA is such that ¯ 2z 6= δ 29 is obtained (cf. Example in Section 2.5.3). Ga (2|2) = 0.97 holds. Hence, p The set Qx (1) of the orthogonal partition and of the partition in Figure 4.6 are ¯ 1z = δ 19 . However, starting in the same. Therefore, the SA of Figure 2.5 also yields p z = 7 yields the limiting distribution limk→∞ pz (k) = ( 0.355 0 0.548 0 0 0.083 0 0.009 0.005 )0 . ˜x,1 = {1, 4, 7} of qualitative states of the SA in Figure 2.5 This is because the set N is not closed. Comparing the orthogonal partition with the partition in Figure 4.6 ˜x,1 = {1, 4, 7} is made a closed set by forming the region Qx (7) shows that N according to the flow of the vector field.

4.3 Relations between Quantisation and Properties of the Model*

111

Remark 4.3.1. The formulation of Theorem 4.3.1 refers to asymptotically stable ¯ i = ∞. Systems with fixed points. This does not exclude infinity, i.e. fixed points x such fixed points are usually referred to as, at least for some initial conditions, unstable systems. However, using a single unbounded partition set ensuring that none of its states map to a bounded set yields an absorbing qualitative state including infinity and hence a correct stationary behaviour of the qualitative model. Extension to non–autonomous systems. As the previous section has shown, the partition must be set in accordance with the dynamics of the continuous–variable system to obtain a qualitative model with the corresponding stationary properties as the quantised system. Hence, if the system dynamics change with an input, the partition of the state space needs to change too. Since only one state space partition can be chosen for all inputs, ideal stationary behaviour can, in general, not be obtained for all inputs. However, results from the autonomous case extended to non–autonomous systems can yield “better” stationary behaviour of the qualitative model than arbitrary partitions as the following considerations show. Discrete–variable inputs. First, a quantised system is considered where the injector assigns to each quantised input [u] = i a fixed input vector ui , i = 1, . . . , M . Then, for each constant input [u], the quantised system is equal to an autonomous quantised system considered in the previous section. Hence, under the same conditions as in the previous section, a partition can be set up in accordance with Theorem 4.3.1 and Lemma 4.3.3 separately for each quantised input. However, merging the obtained partitions does not necessarily yield a qualitative model with the same stationary distributions as the quantised system. The results for merged partitions are not derived formally but illustrated using an example.

Fig. 4.7. Merged non–conflicting (left) and conflicting (right) partition of the state space for non–autonomous systems.

Example 4.3.3. Consider Examples 4.3.1 and 4.3.2 of the two–tank system. Using the two discrete inputs u = 0.5 and u = 0 for the pump, the first or second example

112

4. Further Aspects of Modelling Quantised Systems*

is obtained, respectively. On the left–hand side of Figure 4.7, a partition resulting from Theorem 4.3.1 and Lemma 4.3.3 is shown, where the partitions, obtained separately for each input, are merged. As the bounds derived for both inputs do not cross each other, the depicted partition yields a qualitative model with correct stationary behaviour for both inputs. Obviously, merging the partitions can lead to conflicts in the sense that partition sets designed for separate inputs overlap such that the resulting shapes violate the conditions of Theorem 4.3.1 or Lemma 4.3.3. Such a situation is shown on the right–hand side of Figure 4.7 for the two–tank system. Shown is the case that the elliptic partition set necessary for correct stationary behaviour for u = 0.5 is divided into two sets from the partition obtained for u = 0. It is intuitively clear that quantised systems exist for which such a conflict cannot be avoided by any shape of the partition. However, conflicting merged partitions yield in principle the same information concerning the stationary behaviour as the separate partition before merging. This is shown at the conflicting partition shown on the right–hand side of Figure 4.7. The qualitative model resulting from this partition yields for u = 0.5 the stationary distribution: ¯z = ( 0 p

0

0.2135

0.7865

0

0 )0 .

(4.30)

4 This distribution is different from the stationary distribution pqs z = δ 6 of the quan¯ z is such that the probability for the elliptic set as a whole tised system. However, p is still one. This means that the automaton states z = 3 and z = 4 form an irreducible set of states. In other words, if the partitions are designed for each input ¯x,i are absorbing, then the merged partition can either still such that the states of N have these absorbing states (left–hand side of Figure 4.7), or some of the absorbing states become an irreducible set of two or more states, like qualitative states 3 and 4 on the right–hand side of Figure 4.7.

Continuous–variable inputs. Finally, continuous–variable inputs are considered. For each qualitative input, conditions on the partition will be derived such that the stationary behaviours of the quantised system and the qualitative model coincide. The partitions derived separately for each input are then merged with the same restrictions explained in the previous section. For simplicity, it is assumed that the system has a single asymptotically stable fixed point for all inputs. The set Qu (¯ v ) of numerical inputs u corresponding to the quantised input [u] = v¯ is considered. In general, the fixed point depends upon the input. That is, the location of the fixed point is described by a function ¯ =g ¯ (u). Assuming that the function g ¯ (u) is continuous, the Q u → Rn : x region of fixed points under any input u ∈ Qu (¯ v ) is a compact set given by: ¯ = {x | x = g ¯ (u), u ∈ Qu (¯ v )} ⊆ Rn . X From this, a result similar to Lemma 4.3.1 is obtained. Lemma 4.3.4. The qualitative model of a quantised system with embedded continuous–variable system with a single asymptotically stable fixed point for v ) has, for constant qualitative input v¯, the same stationary all inputs u ∈ Qu (¯ distribution as the quantised system if the state space partition is such that

4.3 Relations between Quantisation and Properties of the Model*

113

¯ is absorbing under the automaton state z¯ corresponding to a set Qx (¯ z) ⊃ X the constant input v¯, i .e. if G(¯ z | z¯, v¯) = 1 holds. Proof. See Appendix C.9.

t u

In contrast to Lemma 4.3.1, it is difficult is to find a region satisfying the condition of Lemma 4.3.4 and its existence cannot be guaranteed. This is because a constant quantised input does not mean that the numerical input u is also constant. In fact, the numerical input can change its value at each time instance taking an arbitrary value from the set Qu (¯ v ). Hence, it is in general not possible to obtain such a region by determining sets for each numerical input u ∈ Qu (¯ v ) satisfying Lemma 4.3.1 and joining them. Instead, z ) must satisfy the the condition of Lemma 4.3.4 means that the set Qx (¯ condition ˆ = g(x, u), x ∈ Qx (¯ g(Qx (¯ z ), Qu (¯ v )) = {ˆ x|x z ), u ∈ Qu (¯ v )} !

⊆ Qx (¯ z) .

(4.31)

Equation (4.31) can be interpreted in two ways. If the condition is not satisfied either the shape of the set Qx (¯ z ) or the partition of the input space must be changed. The following example shows how sets Qx (¯ z ) and Qu (¯ v ) can be found such that condition (4.31) is satisfied.

Fig. 4.8. Construction of an input and state space partition according to the stationary behaviour.

Example 4.3.4. Consider the tank system of Section 1.4 with the valve V 12l closed, V12u opened, no leak and the pump velocity vp (k) as input. First, the input vp (k) = 0.5 is chosen. Considering this input as constant, the system becomes autonomous and Lemma 4.3.1 can be applied. For this input, Figure 4.5 shows a partition satisfying the condition of Lemma 4.3.1. That is, setting Qx (¯ z ) to the ellipsoid shown in Figure 4.5, the relation g(Qx (¯ z ), 0.5) ⊆ Qx (¯ z ) is satisfied. Now,

114

4. Further Aspects of Modelling Quantised Systems*

a neighbourhood of vp (k) = 0.5 is taken as set Qu (¯ v ). This is shown on the left– hand side of Figure 4.8. For the interval Qu (¯ v ) = [0.48, 0.55] the set Qx (¯ z ) maps onto itself for all u ∈ Qu (¯ v ) satisfying condition (4.31). The condition is violated if the interval Qu (¯ v ) is chosen too large. This is shown on the right–hand side of Figure 4.8 for Qu (¯ v ) = [0.2, 0.8]. There are two possible ways to proceed. Either, Qu (¯ v ) = [0.48, 0.55] is chosen as one quantised input or the shape of the set Qx (¯ z ) is modified in order to be able to further increase the size of Qu (¯ v ). However, using Qu (¯ v ) = [0.48, 0.55] as quantised input and the state space partition of Figure 4.5 yields an SA where the state z¯ is absorbing under the input v¯. Hence, according to Lemma 4.3.4, the stationary distributions of the qualitative model and the quantised system coincide. Similarly, the rest of the input space can be partitioned. The state space partition is obtained by merging the partitions for all quantised inputs each satisfying the conditions of Lemma 4.3.4, as described above.

Remark 4.3.2. According the above result, it is possible to find a state and input space partition such that the stationary distributions of the quantised system become stationary distributions of the qualitative model for a large class of quantised systems. However, this cannot, in general, be extended to limiting distributions. That is, to formulate a result similar to Lemma 4.3.3 requires that the manifolds separating attraction areas do not depend on the input. This is generally not the case. 4.3.2 State Space Partitions Adapted to the Dynamical Behaviour* The discussion in the previous section has shown that for autonomous systems, state space partitions can be derived for which the quantised system and the qualitative model have the same stationary and limiting distributions. It has further been shown that ideal stationary behaviour for non– autonomous system cannot always be achieved. However, as explained in the previous section, partitions of non–autonomous systems that result from merging partitions designed separately for each input are still related to the ideal stationary behaviour and are therefore preferable. In this section, it is shown how partitions yielding qualitative models with the same stationary and dynamical behaviour as the quantised system can be constructed. It will be shown that partitions which yield ideal dynamical behaviour are partitions for ideal stationary behaviour plus additional constraints on the partition sets. This section is restricted to autonomous systems as in the non–autonomous case, no precise results can be derived. As aforementioned, merged partitions yield, in general, a satisfactory behaviour of the qualitative model of non– autonomous quantised system, as the partition is then adapted to the system dynamics for the different quantised inputs.

4.3 Relations between Quantisation and Properties of the Model*

115

In the following, systems satisfying the conditions of Theorem 4.3.1 or systems with invariant manifolds described in Section 4.3.1 are considered. The starting point of the following considerations are partition sets satisfying Theorem 4.3.1, and do thus already yield ideal stationary behaviour of the qualitative model. Conditions on the shape and number of the remaining regions will be derived such that the qualitative model has the same dynamical behaviour as the autonomous quantised system. With such partitions, the quantised system has the Markov property, i.e. it satisfies eqn. (3.80), because the quantised system in this case can be represented by a Markov chain (cf. Section 2.2). Therefore, state space partitions of quantised systems which posses the Markov property are called Markov partitions. A special case of Markov partitions are deterministic partitions, i.e. partitions that result in a deterministic behaviour of the quantised system. Such partitions are considered first. Conditions for such partition have first been investigated in [111], where for a given partition, rather restrictive conditions are derived for an embedded linear continuous–variable system such that the quantised systems yields deterministic behaviour. Similarly, for embedded linear systems, a first approach to deterministic partitions is given in [97] which has been extended to a larger class of deterministic partitions in [116]. Based on the results in [116] this is extended to nonlinear systems in the following. ¯z yielding Deterministic partitions. Consider any set Qx (¯ zi ) with z¯i ∈ N a qualitative model with Ga (¯ zi | z¯i ) = 1 (cf. Theorem 4.3.1). The set g −1 zi )) = {x | g a (x) ∈ Qx (¯ zi )} ⊆ Rn a (Qx (¯

(4.32)

zi ). has the property that all numerical states x it contains are mapped to Qx (¯ Hence a partition of the state space including the sets Qx (¯ zi +0) = Qx (¯ zi )

and

Qx (¯ zi +1) = g −1 zi )) \ Qx (¯ zi ) a (Qx (¯

(4.33)

yields a qualitative model with Ga (¯ zi | z¯i ) = 1 and Ga (¯ zi | z¯i + 1) = 1. Further sets are constructed by iteratively mapping Qx (¯ zi +1) with g −1 a yielding: Qx (¯ zi +2) = g −1 zi +1)), . . . , a (Qx (¯ Qx (¯ zi +νi +1) = g −1 zi +νi )), . . . . a (Qx (¯ The obtained sets, Qx (¯ zi+νi ), partition the state space and yield a qualitative model for which Ga (¯ zi + νi | z¯i + νi + 1) = 1 holds for all νi . Doing this for all quantised states containing fixed points leads to the following algorithm which can be used for all autonomous quantised systems with non–singular transformation g a with asymptotically stable fixed points.

116

4. Further Aspects of Modelling Quantised Systems*

Natural state space partition Algorithm 4.1 Given: System (3.100) with asymptotically stable fixed points Disjoint sets Qx,i with g a (Qx,i ) ⊆ Qx,i , i = 1, . . . , κ Maximum number of mappings per set νimax , i = 1, . . . , κ Do:

For i = 1, . . . , κ P −1 set z¯i := 1 + ij=1 Nj set Qx (¯ zi ) := Qx,i determine Qx (¯ zi +1) := (g −1 zi )) \ Qx (¯ zi )) ∩ Rn a (Qx (¯ set νi = 1 do until λn (Qx (¯ zi +νi )) = 0 or νi = νimax determine Qx (¯ zi +νi +1) := (g −1 zi +νi ))) ∩ Rn a (Qx (¯ set νi := νi +1 loop set Ni := νi +1 Loop P Result: Disjoint sets Qx (j), j = 1, . . . , N with N = κi=1 Ni

If Algorithm 4.1 is stopped because the maximum number νimax of sets for the attraction area of fixed points in Qx (¯ zi ) is reached, the sets Qx (j) obtained from Algorithm 4.1 only cover a subset of Rn . In general the remainder of zi + νimax ), resulting in a the state space maps to itself as well as to Qx (¯ nondeterministic behaviour [116]. Hence, the quantised system can only be deterministic in the set R˜n =

N [

Qx (j) .

j=1

Only if Ni ≤ νimax holds for all i = 1, . . . , κ then the sets Qx (j) do cover the whole state space R˜n = Rn . As the sets Qx (j) determined by Algorithm 4.1 are disjoint, they are a partition of R˜n . Note that from the construction of R˜n follows that g a can be defined on this subset, i.e. g a : R˜n → R˜n holds. Definition 4.3.1. A state space partition obtained by Algorithm 4.1 is called natural partition of R˜n for a given autonomous quantised system. Note that the only freedom in designing a natural partition concerns the shape of the partition sets containing fixed points. All other partition sets are thereby determined. Example 4.3.5. Consider the tank system of Section 1.4 with the valve V 12l closed, V12u opened, pump off and no leak. A partition for correct stationary behaviour is shown in Figure 4.6. Giving the two sets containing fixed points to Algorithm 4.1 yields the natural partition shown on the left–hand side of Figure 4.9. The corresponding qualitative model is the deterministic automaton shown on the right–hand side of the figure.

4.3 Relations between Quantisation and Properties of the Model*

117

Fig. 4.9. Natural state space partition for the two–tank system (left) and resulting qualitative model (right).

Fig. 4.10. Qualitative simulation with a natural partition (left) and an orthogonal partition (right). To show the advantage of natural partitions, the behaviour of a qualitative model based on the natural partition of Figure 4.9 is compared to a qualitative model with an orthogonal partition. The orthogonal partition is such that the interval [0, 0.6] of possible liquid levels is partitioned into 5 equally sized intervals for each state resulting in 25 qualitative states. Compared are the results of a simulation of the qualitative output, which is the outflow of the right tank partitioned like in previous examples into three intervals. As initial distributions for the simulation, the density px (x, 0) = 1X 0 /λ2 (X 0 ) with X 0 = [0, 0.1] × [0.35, 0.55] is quantised by the density quantisers QN corresponding to each state space partition. The results of the simulation are given in Figure 4.10. Shown are the probabilities of the qualitative outputs [y(k)] = 1, . . . , 3 obtained by using the qualitative model with the natural partition (left–hand side) and the orthogonal partition (right–hand side) for k = 0, . . . , 5. The probabilities are shown in grey scale, where the boxes correspond to the partition of the output space. To compare the results with those of the quantised system, the interval of outflows resulting from a simulation of the quantised system with the initial condition px (x, 0) are shown for each time step

118

4. Further Aspects of Modelling Quantised Systems*

Deterministic state space partition Algorithm 4.2 ˜ n into subsets Qx (l), l = 1, . . . , N Given: Deterministic partition of R Partition of any set Qx (˜ l), ˜ l 6= z¯i into sets Qx (˜ l, γ), γ = 1, . . . , Γ Do:

Set ν := 0 Set Nxrem := {1, . . . , N } \ {˜ l} For γ = 1, . . . , Γ ˜ x := (g −1 (Qx (˜ ˜n determine Q l, γ))) ∩ R a ˜ do until λn (Qx ) = 0 set ν := ν +1 ˜x set Qx (˜ l, Γ +ν) := Q ˜ x ∩ Qx (j)) > 0 holds determine Qx (j) for which λ(Q rem rem set Nx \ {j} := Nx ˜ x := (g −1 (Q ˜ x )) ∩ R ˜n determine Q a loop Loop + + + ˜ n into subsets Q+ Set the new partition of R x (l ) with l ∈ Nx = {1, . . . , N + }, N + = |Nxrem |+Γ +ν, by uniquely assigning to + ˜ each set Q+ x (l ) one of the sets Qx (l, γ), γ = 1, . . . , Γ + ν or Qx (l), l ∈ Nxrem ˜ n into sets Qx (l+ ), l+ = 1, ..., N + Result: New deterministic partition of R

as interval bars right of the probability bars. It can be seen that the qualitative model based on the natural partition yields considerably fewer spurious solutions while having less automaton states.

Further deterministic partitions. Every natural partition is a deterministic partition. However, there also exist other deterministic partitions. These partitions are based on natural partitions and are obtained by introducing further regions within any set of the natural partition. That is, a deterministic partition remains deterministic if any set Qx (¯ zi+j), 1 ≤ j < Ni is partitioned into subsets Qx (¯ zi+j, γ) and the sets obtained from Algorithm 4.1 by mapping Qx (¯ zi+j) with g −1 zi+j+νi ), a are also further partitioned. That is, all sets Qx (¯ νi = 1, . . . , Ni −j−1 must be partitioned according to the sets obtained from i zi +j, γ)). By this, the overall number of partition sets and thus the g −ν a (Qx (¯ granularity of the partition is increased. After mapping, all partition sets are renumbered by assigning a unique number to each set. The described method is summarised by Algorithm 4.2. Example 4.3.6. Consider the natural partition for the two–tank system shown in

Figure 4.9. To this partition, Algorithm 4.2 is applied, dividing Qx (3) into two sets Qx (3, 1) and Qx (3, 2). The result of Algorithm 4.2 is shown on the left–hand side of Figure 4.11, where the black bounds shown in the left part of this figure result from dividing Qx (3). The shown partition contains further bounds, depicted in

4.3 Relations between Quantisation and Properties of the Model*

119

grey, resulting from three further applications of Algorithm 4.2. First, the former set Qx (15) of Figure 4.9 is partitioned into two sets resulting in the dark grey bounds. In the resulting partition, the medium grey bound separating the sets Qx (23) and Qx (24) in Figure 4.11 was introduced, and thereafter, the light grey bound between the sets Qx (26) and Qx (27). The qualitative model resulting from the derived deterministic partition is shown on the right–hand side of the figure.

Fig. 4.11. Deterministic state space partition for the two–tank system derived from a natural partition (left) and resulting qualitative model (right).

The above example makes it clear that by means of Algorithm 4.2 arbitrarily fine partitions can in principle be obtained. However, the shape of most of the partition sets is determined by the system dynamics and cannot be chosen. Remark 4.3.3. In [97], deterministic partitions for linear systems x(k+1) = Ax(k) transformed to a canonical form where all eigenvalues of A are in its principal diagonal are considered. For such systems, it can be shown that orthogonal partitions, where the ratios between neighbouring partition bounds are the eigenvalues of A, are deterministic. In [118] it is shown that these partitions represent a subclass of the partitions obtained by Algorithm 4.2 for such linear systems. Dependence upon the sampling time. For sampled quantised systems introduced in Section 3.3.2, there is a straightforward dependence between the number of sets of a natural partition and the sampling time. Therefore, the bounds of the sets containing fixed points are considered. Following the solution curves φ(x, t) for all states x on these bounds backwards in time, the bounds of the natural partition are given by the solution curves at times φ(x, t − k · Ts ), k = 1, 2, . . .. Hence, doubling the sampling time to 2Ts means that the corresponding natural partition contains only every second bound. Similarly, for 0.5Ts between each pair of bounds at kTs and (k+1)Ts , a further

120

4. Further Aspects of Modelling Quantised Systems*

bound at (k + 0.5)Ts is introduced. This is shown in Figure 4.12. On the left–hand side, a natural partition with a sampling time of Ts = 10 seconds is shown, on the right–hand side with Ts = 2.5 seconds. Comparing these partitions to the partition in Figure 4.9 shows the above. In summary, this means that the smaller the sampling time Ts of a continuous–variable system is chosen, the higher is the number of partition sets of a natural partition.

Fig. 4.12. Natural state space partition for the two–tank system with a sampling time Ts = 10s (left) and Ts = 2.5s (right).

Markov partitions. The idea for the definition of Markov partitions is to allow possibly nondeterministic behaviour of the quantised system but such that eqn. (3.80) holds for all times k [117], [179]. For autonomous systems (3.46), this means that the condition Prob([x(k+1)] | [X(0 . . . k)]) = Prob([x(k+1)] | [x(k)])

(4.34)

shall hold for all times k and for all quantised state sequences. However, the probabilities of eqn. (4.34) do not only depend on the partition but also on the densities pix . Using the term Markov partition therefore always includes an assumption concerning the densities in order to ensure that eqn. (4.34) depends merely on the shape of the partition sets. In [179], the considerations are restricted to piecewise linear systems, which preserve piecewise constant density functions according to Section 3.6.2. In [116], Markov partitions are defined by restricting all densities to uniform distributions. This is generalised by the definition given below. Note that the probability Prob([x(k+1)] | [X(0 . . . k)]) of the qualitative states at times k+1 only depends on the initial distribution px0 (x). This leads to the following definition. Definition 4.3.2. A state space partition for which eqn. (4.34) holds for all times k and for all initial conditions px0 (x) satisfying px0 (x) = IN pz for any pz ∈ W N is called a Markov partition of the autonomous quantised system.

4.4 Bibliographical Notes

121

Obviously, Markov partitions depend on the system equation (3.46). It is intuitively clear but has yet to be proved that nondeterministic Markov partitions do not exist for arbitrary systems. In particular, neither the existence nor the precise definition of Markov partitions have been investigated for non–autonomous quantised systems. However, for autonomous quantised systems the following result holds, with the referred sets (3.85) and (3.87) simplified for the autonomous case. Autonomous quantised systems represented by stochastic automata Theorem 4.3.2. If and only if the state space partition is a Markov partition are the (quantised) dynamical behaviours (3.85) and (3.87) of a qualitative model and its originating autonomous quantised system identical for all times k. Then the autonomous quantised system is represented by its qualitative model. Proof. See Appendix C.10.

t u

It is clear that if the condition of Theorem 4.3.2 is satisfied the qualitative model generates no spurious solutions and (3.93) holds with equality sign. Lemma 4.3.5. Every deterministic partition of an autonomous quantised system is a Markov partition independently of the densities pix . Proof. See Appendix C.11.

t u

Remark 4.3.4. There also exist nondeterministic Markov partitions. Consider for example autonomous linear systems with Rn ⊂ IRn , λn (Rn ) < ∞ and uniform distributions pix . Then a nondeterministic Markov partition can be obtained by ˜ x ) > 0 holds. means of Algorithm 4.2 by exiting from the inner loop while λn (Q By exiting the loop, a qualitative state zi of the underlying natural partition is not further divided and thus has nondeterministic transitions to the sets that resulted from the last–performed inner loop of the algorithm. However, the linearity of the system guarantees that the mapped density IN δ zNi corresponding to state zi is represented by the uniform distributions in its qualitative successor states.

4.4 Bibliographical Notes The classical method to compute transition probabilities from one region of a partitioned space to another is point–mapping [69], [87]. It was originally introduced for the analysis of nonlinear autonomous systems and is also referred to as Monte–Carlo method [61]. In [97] it has first been applied to qualitative modelling and has been extended to non–autonomous system with output.

122

4. Further Aspects of Modelling Quantised Systems*

For systems satisfying a Lipschitz constraint, the hyperbox–mapping describes a new method to compute the qualitative model for given quantised system guaranteeing completeness of the obtained result. Independently of this method for qualitative modelling, a similar method for analysing the stationary behaviour of autonomous nonlinear systems has been developed [61]. These results are used in Lemma 4.2.1 to show the convergence of the computed transition probabilities to those of Definition 3.5.1. The second part of this chapter concerns the stationary behaviour of quantised systems in dependence upon the state space partition. This has not been investigated in literature so far. The derived methods use results on the stationary behaviour of continuous–variable systems (cf. [9], [26], [87] and [192]) and of Markov chains (cf. [56], [188] or [198]). A dynamical continuation of the conditions derived for the stationary behaviour results in partitions yielding deterministic behaviour of autonomous quantised systems. Algorithms 4.1 and 4.2 describe their construction for a class of nonlinear systems. The presented methods extend earlier results in this field that concern either merely linear systems [97], [111] or are restricted to orthogonal partitions [83], [111].

5. Automata Theory for Process Supervision

In this chapter the state observation and fault diagnostic problems are posed and solved for stochastic automata. The observation problem is introduced in Section 5.1, solved in Section 5.2 and discussed in Section 5.3. In Section 5.4 results in the observability of stochastic automata are given. Next, the fault diagnostic problem is posed and solved based on the solution to the state observation problem in Sections 5.5 and 5.6. Diagnosability is then discussed in Section 5.7 and several extensions relevant to the practical application of the presented methods are given in Section 5.8.

5.1 State Observation Problems 5.1.1 Problem Statement The state observation of a stochastic process concerns the situation in which the state is to be determined from the input and output measurements obtained from a realisation of the process. The task can either be to observe the initial state or to reconstruct the state at time instance k up to which measurements are available (cf. Figure 5.1). In this section stochastic processes represented by stochastic automata (SA) are considered. Whenever the term stochastic process (SP) is used in the following sections, it always refers to processes represented by SA, i.e. Markov chains with input and output. The following two observation problems are posed: Initial state observation problem. Given: SP represented by the SA S(Nz , Nv , Nw , L). Sequences V (0 . . . k), W (0 . . . k) of input and output symbols. A–priori initial state distribution Prob(z(0)). Find: A–posteriori initial state distribution Prob(z(0) | V (0 . . . k), W (0 . . . k)).

J. Schröder: Modelling, State Observation and Diagnosis of Quantised Systems, LNCIS 282, pp. 123-167, 2003. Springer-Verlag Berlin Heidelberg 2003

124

5. Automata Theory for Process Supervision

v ( k)

Markov process z ( k)

w ( k)

Stochastic automaton State observer Prob( z ( k) V (0... k), W(0... k) )

Fig. 5.1. Current state observation problem.

Current state observation problem. Given: SP represented by the SA S(Nz , Nv , Nw , L). Sequences V (0 . . . k), W (0 . . . k) of input and output symbols. A–priori initial state distribution Prob(z(0)). Find: Current state distribution Prob(z(k) | V (0 . . . k), W (0 . . . k)). It will be shown in the following that both observation problems yield, in general, no unique solution but sets of more than one possible state. These sets are denoted by Zk (0 | V, W ) = {z(0) | Prob(zp (0) = z(0) | V (0 . . . k), W (0 . . . k)) > 0}

(5.1)

and Zk (k | V, W ) = {z(k) | Prob(zp (k) = z(k) | V (0 . . . k), W (0 . . . k)) > 0} (5.2) which include all states z(0) or z(k) from or to which the automaton may move while getting the given input sequence and generating the given output sequence. As will be discussed in Section 5.4 in detail, observability definitions given in literature concern only the situation in which the sets Zk (0 | V, W ) or Zk (k | V, W ) become singletons for a finite time horizon k and, hence, the state can be unambiguously determined. Here, the observation problem is solved under the more general condition that both sets may have more than one element for all times k. 5.1.2 Existence of Solutions to the Observation Problems This section investigates the existence of a solution to both observation problems. A pair (V, W ) of an input sequence V ∈ Nvk+1 and an output sequence W ∈ Nwk+1 is called I/O pair. Roughly speaking, solutions exist if the given I/O pair can “occur” for the given SP. That is, V must be an admissible input sequence and an initial state z(0) must exist such that the SP generates the given output sequence W with non–vanishing probability. According to

5.2 Solution to the State Observation Problems

125

eqn. (2.24) and the definition of the behavioural relation, the set of admissible input sequences V (0 . . . k) considered in this book is given by Nvk+1 . Definition 5.1.1. An I/O pair is called consistent with the SA, if V is an admissible input sequence and there exists an initial state probability distribution Prob(z(0)) such that eqn. (2.30) yields a non–vanishing probability Prob(W |V ). This and the following definitions or results analogously hold for the SP represented by an SA. That is, an I/O pair is called consistent with an SP if it is consistent with the SA representing the SP. Lemma 5.1.1. An I/O pair (V, W ) is consistent with the SA (SP), if and only if the two conditions 1. the lengths |V | and |W | of the input and output sequences are equal (5.3) X 2. L(k) · L(k−1) · ... · L(0) > 0 holds for V (0 . . . k) and W (0 . . . k) (5.4) Z(0...k+1)

are satisfied, with the abbreviation L(k) given in eqn. (2.31). Proof. See Appendix C.12.

t u

Note that the consistency of an I/O pair (V, W ) with the SA implies that P (W, V ) = P (W |V ) · P (V ) > 0

(5.5)

holds because for each admissible input sequence, P (V ) > 0 is valid. The connection between the consistency of the pair (V, W ) and the existence of solutions to the observation problems is obvious. The consistency guarantees that an initial state distribution exists such that Prob(W |V ) > 0 holds (cf. Proof of Lemma 5.1.1). Hence, there exists at least one state sequence Z(0 . . . k) for which Prob(Z|V, W ) > 0 holds. The first element z(0) of this state sequence belongs to Zk (0 | V, W ) and the last element z(k) to Zk (k | V, W ). If no such initial state distribution exists, then there is no state sequence such that the SA generates the output sequence W . This result is summarised in the following corollary. Corollary 5.1.1. Solutions to both observation problems exist if and only if the I/O pair (V, W ) is consistent with the SP.

5.2 Solution to the State Observation Problems 5.2.1 General Solution In this section, the solution to the state observation problem is presented and illustrated with a simple example.

126

5. Automata Theory for Process Supervision

The solution to the observation problem requires one additional assumption. This concerns the conditional independence of the current input of the states given the previous inputs and outputs: Prob(v(k) | Z(0 . . . k), V (0 . . . k−1), W (0 . . . k−1))

(5.6)

= Prob(v(k) | V (0 . . . k−1), W (0 . . . k−1)) . This assumption is referred to as independence of the current input of the state. Condition (5.6) is satisfied as long as there is no feedback from the output w to the input of the SP. However, certain kinds of feedback are allowed as the input does not necessarily need to be independent of previous inputs and outputs. From the structure of the observation problem shown in Figure 5.1, it is clear that the conditional independence of v(k) from the states Z(0 . . . k −1) given the measurements up to time k −1 always holds because the measurements represent all available information about the state up to time k−1. However, condition (5.6) additionally requires the independence of v(k) of the current state z(k). As the output w(k) in general depends on z(k), this means that direct feedback from the output of the SP to its input must be excluded. Hence, if the feedback is via a dynamical system, it must not have a stochastic direct feedthrough (cf. Section 2.5.1). This will be discussed in more detail in Section 7.2.3. Throughout this chapter it is assumed that eqn. (5.6) holds without further reference to it. Observation of state sequences Theorem 5.2.1. Consider an SP with the a–priori initial state probability distribution Prob(z(0)). If the I/O pair (V, W ) is consistent with the SP and the current input is independent of the current state, the following relation holds: Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)) P z(k+1) L(k) · . . . · L(1) · L(0) · Prob(z(0)) P . = Z(0...k+1) L(k) · . . . · L(1) · L(0) · Prob(z(0)) Proof. See Appendix C.13.

(5.7) t u

Equation (5.7) describes for all Z(0 . . . k) ∈ Nzk+1 the probability that the SA with input sequence V (0 . . . k) has generated the state sequence Z(0 . . . k) provided that the output sequence W (0 . . . k) has occurred. From Theorem 5.2.1, the solutions to both observation problems can be obtained by determining the boundary distributions Prob(z(0) | V (0 . . . k), W (0 . . . k)) X = Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)) Z(1...k)∈Nzk

5.2 Solution to the State Observation Problems

127

or Prob(z(k) | V (0 . . . k), W (0 . . . k)) X = Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)) Z(0...k−1)∈Nzk

of the distribution (5.7), respectively. The results are summarised in the following theorem, which is a direct consequence of Theorem 5.2.1. Theorem 5.2.2. Consider an SP with the a–priori initial state probability distribution Prob(z(0)). If the I/O pair (V, W ) is consistent with the SP and the current input is independent of the current state, the a–posteriori initial and current state distributions are given by Prob(z(0) | V (0 . . . k), W (0 . . . k)) P Z(1...k+1) L(k) · L(k−1) · . . . · L(1) · L(0) · Prob(z(0)) =P Z(0...k+1) L(k) · L(k−1) · . . . · L(1) · L(0) · Prob(z(0))

(5.8)

and Prob(z(k) | V (0 . . . k), W (0 . . . k)) P Z(0...k−1),z(k+1) L(k) · L(k−1) · . . . · L(1) · L(0) · Prob(z(0)) P = , (5.9) Z(0...k+1) L(k) · L(k−1) · . . . · L(1) · L(0) · Prob(z(0)) respectively, and the sets of initial or current states by eqns. (5.1) and (5.2).

w=1, 0.5

6

w=1, 0.5

7

w=1, 1

8

4

w=2, 1

1

w=1, 0.5

w=3, 1

5

w=1, 0.3

Note that the given initial state probability distribution Prob(z(0)) represents the a–priori knowledge about the initial state of the stochastic process whereas the probability distribution Prob(z(0) | V (0 . . . k), W (0 . . . k)) determined in eqn. (5.8) describes the a–posteriori probability distribution, which is obtained by exploiting the information about the initial state included in the I/O pair (V, W ).

w=2, 0.5

w=1, 0.5

2

w=1, 1

3

w=1, 1

w=1, 0.2

Fig. 5.2. Automaton graph.

Example 5.2.1. Consider the SP represented by the SA whose automaton graph is shown in Figure 5.2 and whose initial state distribution is given by

128

5. Automata Theory for Process Supervision

Table 5.1. Probability distributions Prob(Z | V, W ) of the example process. k=0 k=1 V (0 . . . 0) = (1) V (0 . . . 1) = (1, 1) W (0 . . . 0) = (1) W (0 . . . 1) = (1, 1) Z(0...0) Prob(Z|V, W ) Z(0...1) Prob(Z|V, W ) (1) 0.1818 (1, 1) 0.0923 (2) 0.1818 (1, 2) 0.1538 (3) 0.1818 (1, 3) 0.0615 (5) 0.1818 (2, 3) 0.3077 (6) 0.0909 (5, 5) 0.1538 (7) 0.1818 (5, 6) 0.0769 (6, 7) 0.1538 k=3 V (0 . . . 3) = (1, 1, 1, 1) W (0 . . . 3) = (1, 1, 1, 3) Z(0...3) Prob(Z|V, W ) (5, 6, 7, 8) 1

k=2 V (0 . . . 2) = (1, 1, 1) W (0 . . . 2) = (1, 1, 1) Z(0...2) Prob(Z|V, W ) (1, 1, 1) 0.0632 (1, 1, 2) 0.1053 (1, 1, 3) 0.0421 (1, 2, 3) 0.3509 (5, 5, 5) 0.1754 (5, 5, 6) 0.0877 (5, 6, 7) 0.1754

k=4 V (0 . . . 4) = (1, 1, 1, 1, 1) W (0 . . . 4) = (1, 1, 1, 3, 3) Z(0...4) Prob(Z|V, W ) (5, 6, 7, 8, 8) 1

Prob(z(0)) = { 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125 } . The SA is autonomous. Accordingly, the SA has a unique input symbol v = 1. The three different outputs are distinguished in Figure 5.2 by the colour and thickness of the edges, from thin black lines for w = 1 to thick light grey lines for w = 3. All state sequences which occur with non–vanishing probability for the input sequence V (0 . . . 4) = (1, 1, 1, 1, 1) and yield the output sequence W (0 . . . 4) = (1, 1, 1, 3, 3) are shown in Table 5.1 as well as their probabilities Prob(z(0) | v(0), w(0)), . . . , Prob(Z(0 . . . 4) | V (0 . . . 4), W (0 . . . 4)) obtained by means of eqn. (5.7). Note that Prob(z(0) | v(0), w(0)) which is shown in the second column of the table differs from the a–priori probability distribution Prob(z(0)) because the a–posteriori probability includes the information contained in the I/O pair (v(0), w(0)) = (1, 1). This is the reason why the states z = 4 and z = 8, which have the a–priori probability 0.125, do not appear in the first column. The a–posteriori initial and current state distributions obtained from Table 5.1 by means of eqns. (5.8) and (5.9), respectively, are shown in Figure 5.3. The sets Zk (0 | V, W ) and Zk (k | V, W ) include all states with nonzero probability corresponding to the grey and black boxes.

5.2.2 Recursive Form of the Solution For the practical application, the elements of the sequences V (0 . . . k) and W (0 . . . k) appear one after the other for k = 0, 1, 2, . . . and should be processed in this way. Therefore, the following recursive relation for determining the probability distribution and the solution set of the current state observation problem is important.

8 7 6 5 4 3 2 1

state z

state z

5.2 Solution to the State Observation Problems

0

1

time

3

4

129

8 7 6 5 4 3 2 1 0

1

time

3

4

Fig. 5.3. Observation results: A–posteriori initial state distribution (left) and a–posteriori current state distributions (right).

Recursive solution to the current state observation problem Theorem 5.2.3. Consider an SP with the a–priori initial state probability distribution Prob(z(0)). If the I/O pair (V, W ) is consistent with the SP and the current input is independent of the current state, the a–posteriori current state probability distribution is given by k>0: Prob(z(k) | V (0 . . . k), W (0 . . . k)) (5.10) P z(k+1) L(k) · Prob(z(k) | V (0 . . . k−1), W (0 . . . k−1)) =P z(k),z(k+1) L(k) · Prob(z(k) | V (0 . . . k−1), W (0 . . . k−1)) k=0:

P

Prob(z(0) | v(0), w(0)) = P

z(1)

L(0) · Prob(z(0))

z(0),z(1)

(5.11)

L(0) · Prob(z(0))

with the recursive relation k>1: Prob(z(k) | V (0 . . . k−1), W (0 . . . k−1)) (5.12) P z(k−1) L(k−1)·Prob(z(k−1) | V (0 . . . k−2), W (0 . . . k−2)) =P z(k),z(k−1) L(k−1)·Prob(z(k−1) | V (0 . . . k−2), W (0 . . . k−2)) k=1:

P

Prob(z(1) | v(0), w(0)) = P

z(0)

L(0) · Prob(z(0))

z(1),z(0)

L(0) · Prob(z(0))

.

(5.13)

130

5. Automata Theory for Process Supervision

Proof. See Appendix C.14.

t u

Equation (5.12) describes a “prediction” of the state z(k) at time k−1 by using the information about the movement of the SA until time k−1. It is a recursive relation initialised at k = 1 by eqn. (5.13), determining Prob(z(k) | V (0 . . . k− 1), W (0 . . . k−1)) from Prob(z(k−1) | V (0 . . . k−2), W (0 . . . k−2)) and the new measurements v(k−1) and w(k−1). Hence, only the probability distribution Prob(z(k −1) | V (0 . . . k −2), W (0 . . . k −2)) consisting of N values has to be stored in an implementation of the observer (cf. Section 5.3.2). Equation (5.10) describes how the prediction from the previous time instance has to be corrected after the new measurements v(k) and w(k) became available. This step can be interpreted as a “projection” of the prediction onto the set of those states z(k) compatible with the latest measurements [126]. The result of the recursion is the a–posteriori probability distribution Prob(z(k) | V (0 . . . k), W (0 . . . k)) of the current state z(k) for the given measurements until time k. Remark 5.2.1. The recursive relations (5.10), (5.13) can be derived directly from an explicit representation of the probability distribution Prob(z(k+1), z(k) | V (0 . . . k), W (0 . . . k)) as shown in [131] and [169].

Remark 5.2.2. Theorems 5.2.1–5.2.3 also hold for processes represented by deterministic automata. Furthermore, they can be applied to descriptions based on nondeterministic automata either by using the transformation (2.43) or by replacing the probability distribution Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)) by some indicator function which is positive whenever Z(0 . . . k) is a possible state sequence of the nondeterministic automaton [127].

5.3 Application of the State Observer 5.3.1 Properties of the State Observer The following remarks concern the application of both observation methods. The first two comments deal with problems that may appear if the results presented in Theorems 5.2.2 and 5.2.3 are used to implement an observation algorithm, which processes the measured sequences V and W . Section 5.3.2 presents such an algorithm. The following remarks concern extensions of the algorithm, necessary to handle inconsistent I/O pairs or wrong a–priori knowledge about the initial state of the SP. Inconsistent I/O pairs. If the I/O pair processed by the observation algorithm has been generated by the SP with an initial state z(0) for which

5.3 Application of the State Observer

131

Prob(zp (0) = z(0)) > 0 holds, the resulting I/O pair is consistent. However, in a practical application, the sequences may be subjected to some measurement noise, which may cause the I/O pair to become inconsistent. The question arises as to what happens with the observation result. Since the lengths of V and W are equal, the inconsistency of the given I/O pair leads to a violation of condition (5.4), which makes the denominators in eqns. (5.7)–(5.13) vanish. This means that no solutions to the observation problems are found. The observation algorithm should therefore be interrupted if the mentioned denominators become zero, as this indicates the inconsistency of the I/O pair. Such a test is included in the algorithm of Section 5.3.2. A-priori knowledge about the initial state. In the solutions to the state observation problems, the a–priori initial state distribution Prob(z(0)) has to be known. Since this a–priori knowledge is often not available, for the practical application, Prob(z(0)) is measured or “guessed”, and the question arises as to what happens if the a–priori knowledge about z(0) has a conflict with the initial state of the actual realisation of the SP. To answer this question, assume that Prob(ˆ z (0)) denotes the approximate initial state probability distribution and consider the sets Z0 = {z(0) | Prob(zp (0) = z(0)) > 0} ⊆ Nz , z (0) | Prob(ˆ zp (0) = zˆ(0)) > 0} ⊆ Nz . Zˆ0 = {ˆ The SP starts from an initial state z(0) ∈ Z0 , whereas in the observation algorithm it is assumed that the SP starts from some state z(0) ∈ Zˆ0 . The a–priori knowledge about the initial state has no conflict with the real system if the relation Zˆ0 ⊇ Z0 (5.14) holds. Then the solutions to the observation problems ensure that the relations z(0) ∈ Zk (0 | V, W ) and z(k) ∈ Zk (k | V, W ) are valid, i.e. the sets of initial or current automaton states determined by the observation algorithm include the true state of the SP. If the probability distribution Prob(ˆz (0)) used in the observation algorithm is different from the real distribution Prob(z(0)), the computed probability distributions might be wrong, but they will be accepted in practice as solutions to the observation problems due to the lack of a better a–priori knowledge. If, however, condition (5.14) is violated, it is possible that Prob(W | V, z(0)) is zero for all initial states z(0) ∈ Zˆ0 . Consequently, Zˆ0 ∩ Zk (0 | V, W ) = ∅ holds, which means that the solution to the initial state observation problem at time k is restricted to a set of states which has no intersection with the set Zˆ0 given by the a–priori knowledge. In this case, the denominators in eqns. (5.7)–(5.13) vanish and no solutions to the observation problems exist.

132

5. Automata Theory for Process Supervision

Like in the case of an inconsistent I/O pair, the violation of condition (5.14) makes the denominators vanish, which can be used as an indicator to stop the observation algorithm. This and the preceding remark show that the observation algorithm cannot distinguish between an inconsistent I/O pair and a wrong a–priori initial state distribution. As a consequence, in a practical application the set Zˆ0 has to be chosen “large enough”. A secure way is to assume Prob(ˆ z (0)) such that Zˆ0 = Nz holds, for example, by using the uniform initial state distribution 1 (5.15) for all zˆ ∈ Nz . N Current state observation starting from a known initial state. State observation problems are usually solved if the initial state z(0) is not known. However, for nondeterministic systems, the problem of reconstructing the current state makes sense also if z(0) is completely known as shown in the following example. Prob(ˆ zp (0) = zˆ) =

8 7 6 5 4 3 2 1

state z

state z

Example 5.3.1. Assume that in Example 5.2.1 the initial state is known to be z(0) = 1. If the automaton was deterministic, the current states z(k) could be determined unambiguously for given input sequence V . However, since the SA is nondeterministic, the current state observation problem remains to be solved. Simulation by means of eqn. (2.27) with the input sequence V (0 . . . 4) = (1, 1, 1, 1, 1) yields the result shown on the left–hand side of Figure 5.4. On the right–hand side of the figure, the observation result for the above input sequence and the output sequence W (0 . . . 4) = (1, 1, 1, 1, 1) is shown. For k = 1 the results obtained from simulation and from observation are equal because for all state transitions, the same output occurs (cf. Figure 5.2). However, since in state z = 4 the only possible output w = 2 contradicts the measurements, the simulation and the observation results differ for k > 1. In particular, the state observation yields the result that the automaton cannot be in state z = 4 whereas the simulation yields a high probability for this state.

0

1

time

3

4

8 7 6 5 4 3 2 1 0

1

time

3

4

Fig. 5.4. Comparison of simulation (left) and observation (right) for known initial state distribution.

5.3 Application of the State Observer

133

5.3.2 Observation Algorithm To show in which way the observation method developed in Section 5.2.2 can be applied, in the following an observation algorithm is described. It is based on the recursive solution given in Theorem 5.2.3 and the considerations given in the previous section. For its formulation, the following functions are used, where the variable z corresponds to the state z(k) at time instance k: function

corresponding to

pint : Nz → [0, 1] pres : Nz → [0, 1] haux : Nz → [0, 1]

Prob(z(k) | V (0 . . . k−1), W (0 . . . k−1)) Prob(z(k) | V (0 . . . k), W (0 . . . k)) P z(k+1) L(k) · Prob(z(k) | V (0 . . . k−1), W (0 . . . k−1))

The observation algorithm is given by Algorithm 5.1. Its formulation is such that it runs forever provided that the I/O pair is consistent and that the a–priori distribution satisfies condition (5.14). At the beginning of each loop, the algorithm waits for a new pair of measurements (v, w). This pair, the behavioural relation L and the N stored values of the function pint (z) are used to compute the observation result pres (z) and an update of pint (z). The computation of pres and pint is according to eqns. (5.10) and (5.12) respectively. A foregoing check guarantees that a division by zero cannot occur. As discussed in Section 5.3.1, a vanishing denominator indicates an inconsistent I/O pair or a wrong a–priori distribution. In this case the algorithm is aborted. Note that in each cycle of the algorithm, very few calculations have to be carried out. This makes the algorithm applicable under relatively strong real– time constraints. Since the algorithm is based on the recursive solution to the current state observation problem, neither its computational nor its storage complexity increases with the length of the measurement sequences V and W. 5.3.3 Stochastic Operator of the Observer** From an abstract point of view, a state observer is a dynamical system that obtains the I/O pairs as inputs and generates the observation result as output. Since the solution to the observation problem is not unique but described by the probability distribution Prob(z(k) | V, W ), the state observer is not a deterministic dynamical system but a stochastic process with input (v, w) ∈ Nv × Nw and output z ∈ Nz , where the output is generated with the probability Prob(z(k) | V, W ). In this section it will be shown that the state observer can be represented as stochastic operator, but it is in general no SA-operator, and hence cannot be represented by any SA. At first, this is an astonishing result, because the state observer uses the behavioural relation L of the given SA and “simply inverts” the automaton in order to determine the unknown state from the

134

5. Automata Theory for Process Supervision

Observation algorithm for stochastic processes represented by stochastic automata Algorithm 5.1 Given: Stochastic automaton S(Nz , Nv , Nw , L) A–priori initial state probability distribution Prob(ˆ z (0)) Do:

Set pint (z) = Prob(ˆ zp (0) = z) for all z ∈ Nz Do wait for new measurements v and w P z , w | z, v) · pint (z) for all z ∈ Nz determine haux (z) = z¯ L(¯ P if z haux (z) = 0 holds, stop the algorithm (inconsistent I/O pair or wrong initial state distribution) P P z )/ z haux (z) for all z ∈ Nz determine pint (z) = z¯ L(z, w | z¯, v)· pint (¯ P for all z ∈ Nz determine pres (z) = haux (z)/ z haux (z) Loop

Result: Current state probabilities pres (z)

known input and output. Hence, it is reasonable to presume that the observer itself can be represented by some SA. In order to use the notations introduced in Section 2.6, the state observer is treated as a process with a single input vobs that replaces the pair (v, w). The input vobs is defined via the bijective mapping Mobs : Mobs : Nv × Nw → Nvobs ,

vobs = Mobs (v, w) ,

−1 (v, w) = Mobs (vobs ) .

Vobs denotes a sequence of observer input symbols which is obtained by applying the mapping Mobs successively to all pairs (v, w) of the sequences V and W . The state observer does not define a mapping Nv∗obs × Nz∗ → [0, 1] because it does not yield any result Z if its input (V, W ) is an inconsistent I/O pair. In order to construct a stochastic operator representing the observer, the state ˜z = Nz ∪ {} on which set is extended by introducing the extended state set N the operator will be defined. To simplify the notation, the denominator of the state observer given by eqn. (5.9) is abbreviated by Π(Vobs (0 . . . k)) =

X

k Y

L(i) · Prob(z(0)).

Z(0...k+1) i=0

˜v∗ × Nz∗ → [0, 1]. The aim now is to construct a stochastic operator τ : N obs There are different possibilities how the operator can be defined for inconsistent I/O pairs. The following formulation of the operator is such that if

5.3 Application of the State Observer

135

the I/O pair becomes inconsistent at time k1 , a non–vanishing probability is associated with all state sequences that are continued by repeating the state , i.e. for inconsistent I/O pairs the operator shall yield   Prob Z(0 . . . k) = (z(0), z(1), ..., z(k1 −1), , , ...,  )| V (0 . . . k), W (0 . . . k) | {z } k−k1 +1 times

= Prob(Z(0 . . . k1 −1) | V (0 . . . k1 −1), W (0 . . . k1 −1)) . The operator τ (Z|Vobs ) is defined as follows: τ (Z(0 . . . k) | Vobs (0 . . . k)) P Qk i=0 L(i)·Prob(z(0))   z(k+1)Π(V  obs (0...k))     0      = τ (Z(0 . . . k−1) | V (0 . . . k−1)) obs          1   

Π(Vobs (0 . . . k)) > 0 Π(Vobs (0 . . . k)) = 0 and z(k) ∈ Nz Π(Vobs (0 . . . k)) = 0, z(k) =  and k > 0 Π(Vobs (0 . . . k)) = 0, z(k) =  and k = 0 .

As long as the I/O pair is consistent, the equality τ (Z(0 . . . k) | Vobs (0 . . . k)) = Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)) holds. The remaining three cases concern inconsistent I/O pairs. In the second case, the operator yields for inconsistent I/O pairs a probability of zero for sequences that end with some state z ∈ Nz . If the last state is , the probability is the same as that of the state sequence of length k−1, cf. case three. Finally, to the state sequence Z = () a probability of one is assigned if Π(vobs (0)) = 0 holds. ˜z , τ ) is a stochastic operator. Lemma 5.3.1. (Nvobs , N Proof. See Appendix C.15.

t u

Lemma 5.3.2. The state observer is, in general, no SA–operator and therefore, cannot be realised by any SA. Proof. See Appendix C.16.

t u

Remark 5.3.1. The reason why the observer is not an SA-operator is due to the fact that the state observer associates a non–vanishing probability only with those state sequences that the SA can follow for the given I/O pair with an increasing time k. If it becomes obvious from the I/O pair at time k+1 that a certain state z(k) which has been associated with a positive probability Prob(z(k) | V (0...k), W (0...k)) cannot have been assumed, the observer does not produce a continuation of the corresponding state sequence. As, in general, this “cutting off ” of probabilities does not end, no SA can represent the observer.

136

5. Automata Theory for Process Supervision

Remark 5.3.2. At the first glance, the result of Lemma 5.3.2 seems to contradict to the result of [140], where it is proved that the state observer of a nondeterministic automaton is itself a nondeterministic automaton. However, the latter result has to be considered under the assumption that the nondeterministic automaton is observable in the sense that it is possible to uniquely determine the automaton state. Hence, the observer has to reduce the set of possible automaton states until only one state remains. This selection process can be represented by an automaton. As no such assumption is made here, Lemma 5.3.2 is in accordance with this result. In fact, it appears that the observability condition of [140] identifies the class of SA for which the observer is an SA–operator. Remark 5.3.3. As a consequence of Lemma 5.3.2, state observers cannot be analysed by methods elaborated for stochastic automata. In particular, the convergence property, which is known to be a fundamental property of observers for continuous– variable systems, cannot be tested by using techniques concerning the stationary behaviour of SA described in Section 2.5.3. Remark 5.3.4. Finally it should be remarked that the observer can be interpreted as a continuous–variable system (3.1) with the input vector u = (v w)0 and the state vector x according to pint (z) (cf. Section 5.3.2). Hence, from the viewpoint of continuous system theory, the observer is an N –th order dynamical system.

5.4 Observability* The observability of an SP is an important issue for the application of the state observer. However, known results in automata theory always refer to the case that the automaton state can be determined unambiguously after a finite number of steps. Furthermore, most results for example in [140] or [104] refer to nondeterministic rather than stochastic automata. Therefore, in this section, a broader notion of observability of an SP is introduced. Other existing observability definitions are summarised and compared to the definitions given here in Section 5.4.4. 5.4.1 Stochastic Unobservability* Global stochastic unobservability. The following definition of observability is based on a comparison of the results that are obtained by using the I/O pair (V, W ) on one hand, and the input sequence V alone on the other. Roughly speaking, the SP is observable if it is possible to determine the state more precisely by exploiting the output information than using the input alone. Before giving the definition of observability, stochastic unobservability is first defined.

5.4 Observability*

137

Definition 5.4.1. An SP is called stochastically unobservable if the relation Prob(zp (1) = z 0 , wp (0) = w | zp (0) = z, vp (0) = v)

(5.16)

0

= Prob(zp (1) = z | zp (0) = z, vp (0) = v) · Prob(wp (0) = w | vp (0) = v) holds for all z 0 , z ∈ Nz , v ∈ Nv and w ∈ Nw . In Definition 5.4.1, the probability distribution Prob(wp (0) = w | vp (0) = v) means the probability of an output for given input without knowing the state. It can be thought of as mean probability of the output for all states of Nz : X Prob(wp (0) = w | vp (0) = v) = Prob(wp (0) = w | zp (0) = z, vp (0) = v)/N . z∈Nz

Definition 5.4.1 has an obvious interpretation. Equation (5.16) says that the output w does not depend on z and, hence, does not provide any information about the current automaton state. Furthermore, eqn. (5.16) implies that w does not depend on z 0 and, hence, the output does not provide any information about the successor state. Corollary 5.4.1. An SP is stochastically unobservable if and only if for the SA representing the SP, the following two conditions hold: 1. The value of the output relation H(w | z, v) does not depend on z and ˜ | v). That is, can, therefore, be written as H(w ˜ | v) = H(w | zi , v) H(w holds for all zi ∈ Nz , v ∈ Nv and w ∈ Nw . 2. The SA is a stochastic Mealy–automaton. That is, the SA satisfies relation L(z 0 , w | z, v) = G(z 0 | z, v) · H(w | z, v) for all z 0 , z ∈ Nz , v ∈ Nv and w ∈ Nw (cf . eqn. (2.44)). The consequences of stochastic unobservability are shown by comparing state observation with simulation. For both simulation and observation, the initial state distribution Prob(z(0)) and the input sequence V have to be known. In simulation, the current state probability distribution is determined by eqn. (2.27) which propagates the initial state probability distribution according to the state transition relation G of the SA and yields Prob(z(k) | V (0 . . . k−1)). Using eqn. (2.29), the set of states that the SP can assume at time k is given by Zk,sim (k | V ) = {z(k) | Prob(z(k) | V (0 . . . k)) > 0} . The state observer described by Theorem 5.2.2 has access to the additional information included in the output sequence W . It determines the probability distribution Prob(z(k)|V, W ) according to eqn. (5.9) and the set of

138

5. Automata Theory for Process Supervision

states Zk (k | V, W ) resulting from eqn. (5.2). The following result shows the consequence of unobservability: Lemma 5.4.1. If the SP is stochastically unobservable, then for all consistent I/O pairs (V, W ), the following relation holds: Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)) = Prob(Z(0 . . . k) | V (0 . . . k)) . (5.17) Proof. See Appendix C.17. According to Lemma 5.4.1, the observation yields the same result as a simulation if the SP is unobservable. That is, the output sequence W (0 . . . k) does not include additional information compared to Prob(z(0)) and V (0 . . . k). The following corollary is a direct consequence of Lemma 5.4.1 and the fact that the solutions to the current and initial state observation problem are obtained from calculating the boundary distribution of the current and initial state of the distribution Prob(Z(0 . . . k) | V (0 . . . k), W (0 . . . k)). Corollary 5.4.2. If the SP is stochastically unobservable, then for all consistent I/O pairs (V, W ), the results of the current state observation and simulation are identical, i .e. the relations Prob(z(k) | V (0 . . . k), W (0 . . . k)) = Prob(z(k) | V (0 . . . k)) , Zk (k | V, W ) = Zk,sim (k | V )

(5.18) (5.19)

hold for all k. Furthermore, no information about the initial state is obtained from the I/O pair: Prob(z(0) | V (0 . . . k), W (0 . . . k)) = Prob(z(0))

∀k .

(5.20)

Local stochastic unobservability. The SP may not satisfy condition (5.16) for all z ∈ Nz , but only for a subset Nzu ⊆ Nz . Then the following definition applies: Definition 5.4.2. An SP is called locally stochastically unobservable within a set Nzu ⊆ Nz of at least two states (|Nzu | ≥ 2) if condition (5.16) holds for all z ∈ Nzu , z 0 ∈ Nz , v ∈ Nv and w ∈ Nw , with Prob(wp (0) = w | vp (0) = v) P = Prob(wp (0) = w | zp (0) = z, vp (0) = v)/|Nzu | .

(5.21)

z∈Nzu

The subsequent corollary follows immediately from Definition 5.4.2 and relation (2.22). Corollary 5.4.3. An SP is stochastically unobservable within a set of states Nzu ⊆ Nz , |Nzu | ≥ 2 if and only if for the SA representing the SP satisfies the following two conditions:

5.4 Observability*

139

Fig. 5.5. SA with a locally (left) or strongly (right) stochastically unobservable set of states.

1. The output relation H(w | z, v) does not depend on z within the set of ˜ | v). That is, states Nzu and can therefore be written as H(w ˜ | v) = H(w | zi , v) H(w holds for all zi ∈ Nzu , v ∈ Nv and w ∈ Nw . 2. The relation ˜ | v) L(z 0 , w | z, v) = G(z 0 | z, v) · H(w holds for all z ∈ Nzu , z 0 ∈ Nz , w ∈ Nw and v ∈ Nv , where G denotes the transition relation (2.5) of the SA. In Section 5.4.3, a result similar to Lemma 5.4.1 and Corollary 5.4.2 will be derived for the observation within an unobservable set of states Nzu . It is clear that if the SP is locally stochastically unobservable within a set of states Nzu , this does not mean that the states within Nzu can never be distinguished. It is possible, for example, that each state within a set Nzu has a different and unique successor state not in Nzu that occurs independently of the output and can unambiguously be measured in the next time step. Then the measurement at the next time step unambiguously reveals the state of the SP one time step before, though this state lies within a set of stochastically unobservable states. This motivates the following stronger definition of stochastic unobservability within a set of states. Definition 5.4.3. An SP is called strongly stochastically unobservable within a set Nzu ⊆ Nz of at least two states (|Nzu | ≥ 2) if the SP is stochastically unobservable within Nzu and Nzu is a closed set of states under all inputs v ∈ Nv . To illustrate Definitions 5.4.2 and 5.4.3, Figure 5.5 shows two autonomous SA. The SA shown on the left–hand side has a locally unobservable set of states. However, this does not mean that the states within the unobservable

140

5. Automata Theory for Process Supervision

set cannot be distinguished. This is because the SP can leave the locally unobservable set of states and outside of this set the output contains information about the previous states. In contrast to this, the SP corresponding to the SA shown on the right–hand side of Figure 5.5 cannot leave the closed set of states Nzu = {3, 4, 5} if the initial states lies within it. Hence, these states can never be distinguished from the measurements. It is clear that if for the initial set of states Z0 ⊆ Nzu holds, and Nzu is a strongly stochastically unobservable set of states, then Lemma 5.4.1 and Corollary 5.4.2 equivalently hold for all times k. This is summarised in the following corollary: Corollary 5.4.4. If the SP is strongly stochastically unobservable within Nzu and for the initial set of states Z0 ⊆ Nzu holds, then for all consistent I/O pairs (V, W ), the results of state observation and simulation are identical, i .e. the relations (5.17)–(5.19) and (5.20) hold for all k. 5.4.2 Stochastic Observability* Stochastic observability can now be defined as the property of an SP not to possess sets of states in which it is unobservable. Definition 5.4.4. An SP is called stochastically observable if it does not contain any set of states in which it is locally stochastically unobservable. According to this definition, the SP is called observable if there exists at least one input for which the state observation yields more information about the state than simulation, independent of current state of the process. This fact is represented by the following corollary, which follows directly from Lemma 5.4.1 and Definition 5.4.4: Corollary 5.4.5. If the SA is stochastically observable, then Zk (k | V, W ) ⊆ Zk,sim (k | V )

(5.22)

holds for all times k. In contrast to this, for a stochastically unobservable SA the relation Zk (k | V, W ) = Zk,sim (k | V ) holds for all initial states, for all input sequences and for all times k. Note that while unobservability always means that stochastic independencies hold for all inputs, observability is given if there exists at least one input for which stochastic dependencies exist. Accordingly, it is reasonable to distin-

5.4 Observability*

141

SA containing locally, but no strongly unob− servable sets of states Set of all observable SA

SA containing strongly unobservable sets of states

Set of all weakly observable SA

Set of all globally unobservable SA

Fig. 5.6. Relation among the different kinds of stochastic observability and unobservability.

guish between those inputs that yield stochastic dependencies and those that do not. This will be considered in Section 5.4.3. The notion of strong unobservability leads to a weaker definition of observability. Definition 5.4.5. An SP is called weakly stochastically observable if there does not exist any set of states in which it is strongly stochastically unobservable. Weak stochastic observability of an SP means that the number of steps for which state observation definitely yields the same result as simulation is finite. This is because there exists an input sequence that possibly brings the SP out of the locally unobservable set of states Nzu and for which the output contains information about the state as soon as the SP can possibly leave the set Nzu . Then observation differs from simulation. This shall be explained at the autonomous SA shown on the left–hand side of Figure 5.5. The SA is not observable as it contains a locally unobservable set of states Nzu = {3, 4, 5}. However, as the set of states is not closed the SA is weakly observable. In this example, state observation equals simulation for no more than two steps. This is because after the first time instance at which it has been possible to leave the set of states Nzu , the output can reveal whether the set of states Nzu has been left. Note that because the probabilities for leaving the set Nzu via states z = 3 or z = 4 are different, even the non–occurrence of an output different from w = 1 contains information about the states within Nzu . Only if the initial state distribution contains a probability of 1 for the state z(0) = 5, state observation and simulation will be equal until the measurements at time k = 2 are obtained. The relation of the various