Automatic Control of Bioprocesses (Control Systems, Robotics and Manufacturing)

Bioprocess Control

Bioprocess Control

Edited by Denis Dochain

First published in France in 2001 by Hermes Science entitled “Automatique des bioprocédés” First published in Great Britain and the United States in 2008 by ISTE Ltd and John Wiley & Sons, Inc. Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address: ISTE Ltd 6 Fitzroy Square London W1T 5DX UK

John Wiley & Sons, Inc. 111 River Street Hoboken, NJ 07030 USA

www.iste.co.uk

www.wiley.com

© ISTE Ltd, 2008 © Hermes Science, 2001 The rights of Denis Dochain to be identified as the author of this work have been asserted by him in accordance with the Copyright, Designs and Patents Act 1988. Library of Congress Cataloging-in-Publication Data Automatique des bioprocédés. English. Bioprocess control / edited by Denis Dochain. p. ; cm. Translation from French. Includes bibliographical references and index. ISBN: 978-1-84821-025-7 1. Biotechnological process control. 2. Biotechnological process monitoring. I. Dochain, D. (Denis), 1956- II. Title. [DNLM: 1. Biomedical Engineering. 2. Bioreactors. 3. Biotechnology. 4. Models, Biological. QT 36 A939 2008a] TP248.25.M65A9813 2008 660.6--dc22 2007046923 British Library Cataloguing-in-Publication Data A CIP record for this book is available from the British Library ISBN: 978-1-84821-025-7 Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire.

Contents

Chapter 1. What are the Challenges for the Control of Bioprocesses? . . . Denis D OCHAIN 1.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2. Specific problems of bioprocess control . . . . . . . . . . . . 1.3. A schematic view of monitoring and control of a bioprocess 1.4. Modeling and identification of bioprocesses: some key ideas 1.5. Software sensors: tools for bioprocess monitoring . . . . . . 1.6. Bioprocess control: basic concepts and advanced control . . 1.7. Bioprocess monitoring: the central issue . . . . . . . . . . . . 1.8. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.9. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

11 12 12 13 14 15 15 16 16

Chapter 2. Dynamic Models of Biochemical Processes: Properties of Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Olivier B ERNARD and Isabelle Q UEINNEC

17

2.1. Introduction . . . . . . . . . . . . . . . 2.2. Description of biochemical processes 2.2.1. Micro-organisms and their use . 2.2.2. Types of bioreactors . . . . . . . 2.2.3. Three operating modes . . . . . . 2.3. Mass balance modeling . . . . . . . . 2.3.1. Introduction . . . . . . . . . . . . 2.3.2. Reaction scheme . . . . . . . . . 2.3.3. Choice of reactions and variables 2.3.4. Example 1 . . . . . . . . . . . . . 2.4. Mass balance models . . . . . . . . . . 2.4.1. Introduction . . . . . . . . . . . . 2.4.2. Example 2 . . . . . . . . . . . . . 2.4.3. Example 3 . . . . . . . . . . . . .

5

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . .

11

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

17 18 18 19 19 21 21 21 23 23 24 24 24 25

6

Bioprocess Control

2.4.4. Matrix representation . . . . . . . . . . . . . . . . . . . . 2.4.4.1. Example 2 (continuation) . . . . . . . . . . . . . . . 2.4.4.2. Example 1 (continuation) . . . . . . . . . . . . . . . 2.4.5. Gaseous flow . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.6. Electroneutrality and affinity constants . . . . . . . . . . 2.4.7. Example 1 (continuation) . . . . . . . . . . . . . . . . . . 2.4.8. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5. Kinetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2. Mathematical constraints . . . . . . . . . . . . . . . . . . 2.5.2.1. Positivity of variables . . . . . . . . . . . . . . . . . 2.5.2.2. Variables necessary for the reaction . . . . . . . . . 2.5.2.3. Example 1 (continuation) . . . . . . . . . . . . . . . 2.5.2.4. Phenomenological knowledge . . . . . . . . . . . . 2.5.3. Specific growth rate . . . . . . . . . . . . . . . . . . . . . 2.5.4. Representation of kinetics by means of a neural network 2.6. Validation of the model . . . . . . . . . . . . . . . . . . . . . . 2.6.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.2. Validation of the reaction scheme . . . . . . . . . . . . . . 2.6.2.1. Mathematical principle . . . . . . . . . . . . . . . . 2.6.2.2. Example 4 . . . . . . . . . . . . . . . . . . . . . . . 2.6.3. Qualitative validation of model . . . . . . . . . . . . . . . 2.6.4. Global validation of the model . . . . . . . . . . . . . . . 2.7. Properties of the models . . . . . . . . . . . . . . . . . . . . . . 2.7.1. Boundedness and positivity of variables . . . . . . . . . . 2.7.2. Equilibrium points and local behavior . . . . . . . . . . . 2.7.2.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . 2.8. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25 26 26 27 27 28 29 30 30 30 30 31 31 31 32 34 35 35 35 35 36 37 39 39 39 40 40 42 43

Chapter 3. Identification of Bioprocess Models . . . . . . . . . . . . . . . . . Denis D OCHAIN and Peter VANROLLEGHEM

47

3.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2. Structural identifiability . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1. Development in Taylor series . . . . . . . . . . . . . . . . . . . . . 3.2.2. Generating series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3. Examples for the application of the methods of development in series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4. Some observations on the methods for testing structural identifiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3. Practical identifiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1. Theoretical framework . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2. Confidence interval of the estimated parameters . . . . . . . . . .

47 48 49 50 50 51 52 52 54

Contents

7

3.3.3. Sensitivity functions . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4. Optimum experiment design for parameter estimation (OED/PE) . . . . 3.4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2. Theoretical basis for the OED/PE . . . . . . . . . . . . . . . . . . 3.4.3. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5. Estimation algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1. Choice of two datasets . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2. Elements of parameter estimation: least squares estimation in the linear case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.3. Overview of the parameter estimation algorithms . . . . . . . . . . 3.6. A case study: identification of parameters for a process modeled for anaerobic digestion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1. The model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2. Experiment design . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.3. Choice of data for calibration and validation . . . . . . . . . . . . 3.6.4. Parameter identification . . . . . . . . . . . . . . . . . . . . . . . . 3.6.5. Analysis of the results . . . . . . . . . . . . . . . . . . . . . . . . . 3.7. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55 57 57 59 61 63 63

Chapter 4. State Estimation for Bioprocesses . . . . . . . . . . . . . . . . . . Olivier B ERNARD and Jean-Luc G OUZÉ

79

4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Notions on system observability . . . . . . . . . . . . . . . . . . . . . . 4.2.1. System observability: definitions . . . . . . . . . . . . . . . . . . . 4.2.2. General definition of an observer . . . . . . . . . . . . . . . . . . . 4.2.3. How to manage the uncertainties in the model or in the output . . 4.3. Observers for linear systems . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1. Luenberger observer . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2. The linear case up to an output injection . . . . . . . . . . . . . . . 4.3.3. Local observation of a nonlinear system around an equilibrium point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.4. PI observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.5. Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.6. The extended Kalman filter . . . . . . . . . . . . . . . . . . . . . . 4.4. High gain observers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1. Definitions, hypotheses . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2. Change of variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.3. Fixed gain observer . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4. Variable gain observers (Kalman-like observer) . . . . . . . . . . . 4.4.5. Example: growth of micro-algae . . . . . . . . . . . . . . . . . . . 4.5. Observers for mass balance-based systems . . . . . . . . . . . . . . . . 4.5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2. Definitions, hypotheses . . . . . . . . . . . . . . . . . . . . . . . .

79 80 80 81 83 84 85 86

64 65 68 69 70 70 71 75 75

86 87 87 89 89 89 90 91 91 92 94 94 96

8

Bioprocess Control

4.5.3. The asymptotic observer . . . . . . . . . . . . . . . 4.5.4. Example . . . . . . . . . . . . . . . . . . . . . . . . 4.5.5. Improvements . . . . . . . . . . . . . . . . . . . . . 4.6. Interval observers . . . . . . . . . . . . . . . . . . . . . . 4.6.1. Principle . . . . . . . . . . . . . . . . . . . . . . . . 4.6.2. The linear case up to an output injection . . . . . . 4.6.3. Interval estimator for an activated sludge process . 4.6.4. Bundle of observers . . . . . . . . . . . . . . . . . 4.7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 4.8. Appendix: a comparison theorem . . . . . . . . . . . . . 4.9. Bibliography . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

96 98 99 101 102 103 105 107 110 111 112

Chapter 5. Recursive Parameter Estimation . . . . . . . . . . . . . . . . . . 115 Denis D OCHAIN 5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. Parameter estimation based on the structure of the observer . . . . . . . 5.2.1. Example: culture of animal cells . . . . . . . . . . . . . . . . . . . 5.2.2. Estimator based on the structure of the observer . . . . . . . . . . 5.2.3. Example: culture of animal cells (continued) . . . . . . . . . . . . 5.2.4. Calibration of the estimator based on the structure of the observer: theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.5. Calibration of the estimator based on the structure of the observer: application to the culture of animal cells . . . . . . . . . . . . . . . 5.2.6. Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3. Recursive least squares estimator . . . . . . . . . . . . . . . . . . . . . . 5.4. Adaptive state observer . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1. Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 116 116 117 119 119 124 127 129 133 138 140 141

Chapter 6. Basic Concepts of Bioprocess Control . . . . . . . . . . . . . . . 143 Denis D OCHAIN and Jérôme H ARMAND 6.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2. Bioprocess control: basic concepts . . . . . . . . . . . . . . . . . . . . 6.2.1. Biological system dynamics . . . . . . . . . . . . . . . . . . . . . 6.2.2. Sources of uncertainties and disturbances of biological systems 6.3. Stability of biological processes . . . . . . . . . . . . . . . . . . . . . 6.3.1. Basic concept of the stability of a dynamic system . . . . . . . . 6.3.2. Equilibrium point . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3. Stability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4. Basic concepts of biological process control . . . . . . . . . . . . . . . 6.4.1. Regulation and tracking control . . . . . . . . . . . . . . . . . . . 6.4.2. Strategy selection: direct and indirect control . . . . . . . . . . .

. . . . . . . . . . .

143 144 144 146 147 147 148 149 150 150 151

Contents

6.4.3. Selection of synthesis method . . . . . . . . . . . . . . . . . . . . . 6.5. Synthesis of biological process control laws . . . . . . . . . . . . . . . . 6.5.1. Representation of systems . . . . . . . . . . . . . . . . . . . . . . . 6.5.2. Structure of control laws . . . . . . . . . . . . . . . . . . . . . . . . 6.6. Advanced control laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.1. A nonlinear PI controller . . . . . . . . . . . . . . . . . . . . . . . 6.6.2. Robust control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7. Specific approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.1. Pulse control: a dialog with bacteria . . . . . . . . . . . . . . . . . 6.7.2. Overall process optimization: towards integrating the control objectives in the initial stage of bioprocess design . . . . . . . . . 6.8. Conclusions and perspectives . . . . . . . . . . . . . . . . . . . . . . . . 6.9. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

152 153 153 154 160 160 162 165 165 167 170 170

Chapter 7. Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Denis D OCHAIN, Martin G UAY, Michel P ERRIER and Mariana T ITICA 7.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2. Adaptive linearizing control of bioprocesses . . . . . . . . . . 7.2.1. Design of the adaptive linearizing controller . . . . . . . 7.2.2. Example 1: anaerobic digestion . . . . . . . . . . . . . . . 7.2.2.1. Model order reduction . . . . . . . . . . . . . . . . . 7.2.2.2. Adaptive linearizing control design . . . . . . . . . 7.2.3. Example 2: activated sludge process . . . . . . . . . . . . 7.3. Adaptive extremum-seeking control of bioprocesses . . . . . . 7.3.1. Fed-batch reactor model . . . . . . . . . . . . . . . . . . . 7.3.2. Estimation and controller design . . . . . . . . . . . . . . 7.3.2.1. Estimation equation for the gaseous outflow rate y . 7.3.2.2. Design of the adaptive extremum-seeking controller 7.3.2.3. Stability and convergence analysis . . . . . . . . . . 7.3.2.4. A note on dither signal design . . . . . . . . . . . . 7.3.3. Simulation results . . . . . . . . . . . . . . . . . . . . . . 7.4. Appendix: analysis of the parameter convergence . . . . . . . 7.5. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

173 174 174 176 177 179 183 188 189 191 191 192 195 196 197 202 207

Chapter 8. Tools for Fault Detection and Diagnosis . . . . . . . . . . . . . . 211 Jean-Philippe S TEYER, Antoine G ÉNOVÉSI and Jérôme H ARMAND 8.1. Introduction . . . . . . . . . . . . . . . . 8.2. General definitions . . . . . . . . . . . . 8.2.1. Terminology . . . . . . . . . . . . 8.2.2. Fault types . . . . . . . . . . . . . 8.3. Fault detection and diagnosis . . . . . . 8.3.1. Methods based directly on signals

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

211 212 212 213 214 215

10

Bioprocess Control

8.3.1.1. Hardware redundancy . . . . . . . . . . 8.3.1.2. Specific sensors . . . . . . . . . . . . . 8.3.1.3. Comparison of thresholds . . . . . . . . 8.3.1.4. Spectral analysis . . . . . . . . . . . . . 8.3.1.5. Statistical approaches . . . . . . . . . . 8.3.2. Model-based methods . . . . . . . . . . . . . 8.3.2.1. Parity space . . . . . . . . . . . . . . . . 8.3.2.2. Observers . . . . . . . . . . . . . . . . . 8.3.2.3. Parametric estimation . . . . . . . . . . 8.3.3. Methods based on expertise . . . . . . . . . . 8.3.3.1. AI models . . . . . . . . . . . . . . . . . 8.3.3.2. Artificial neural networks . . . . . . . . 8.3.3.3. Fuzzy inference systems . . . . . . . . . 8.3.4. Choice and combined use of diverse methods 8.4. Application to biological processes . . . . . . . . . 8.4.1. “Simple” biological processes . . . . . . . . 8.4.2. Wastewater treatment processes . . . . . . . 8.5. Conclusion . . . . . . . . . . . . . . . . . . . . . . 8.6. Bibliography . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

215 216 217 217 218 218 219 220 221 222 223 224 225 227 227 228 229 231 232

List of Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Chapter 1

What are the Challenges for the Control of Bioprocesses?

1.1. Introduction In simple terms, we can define a fermentation process as the growth of microorganisms (bacteria, yeasts, mushrooms, etc.) resulting from the consumption of substrates or nutrients (sources of carbon, oxygen, nitrogen, phosphorus, etc.). This growth is possible only when favorable “environmental” conditions are present. Environmental conditions refer to physicochemical conditions (pH, temperature, agitation, ventilation, etc.) necessary for good microbial activity. Techniques in the field of biotechnology can be roughly grouped into three major categories: 1. microbiology and genetic engineering; 2. bioprocess engineering; 3. bioprocess control. Microbiology and genetic engineering aim to develop microorganisms, which allow for the production of new products, or aim to choose the best microbial strains so as to obtain certain desired products or product quality. Process engineering chooses the best operating modes or develops processes and/or reactors, which change and improve the output and/or the productivity of bioprocesses. Automatic control aims to increase the output and/or productivity by developing methods of monitoring

Chapter written by Denis D OCHAIN.

11

12

Bioprocess Control

and control, enabling real-time optimization of the bioprocess operation. These approaches are obviously complementary to one another. This book discusses the matter within the context of the final approach. 1.2. Specific problems of bioprocess control Over the past several decades, biotechnological processes have been increasingly used industrially, which is attributed to several reasons (improvement of profitability and quality in production industries, new legislative standards in processing industries, etc.). The problems arising from this industrialization are generally the same as those encountered in any processing industry and we face, in the field of bioprocessing, almost all of the problems that are being tackled in automatic control. Thus, system requirements for supervision, control and monitoring of the processes in order to optimize operation or detect malfunctions are on the increase. However, in reality, very few installations are provided with such systems. Two principal reasons explain this situation: – first of all, biological processes are complex processes involving living organisms whose characteristics are, by nature, very difficult to apprehend. In fact, the modeling of these systems faces two major difficulties. On the one hand, lack of reproducibility of experiments and inaccuracy of measurements result not only in one or several difficulties related to selection of model structure but also in difficulties related to the concepts of structural and practical identifiability at the time of identification of a set of given parameters. On the other hand, difficulties also occur at the time of the validation phase of these models whose sets of parameters could have precisely evolved over course of time. These variations can be the consequence of metabolic changes of biomass or even genetic modifications that could not be foreseen and observed from a macroscopic point of view; – the second major difficulty is the almost systematic absence of sensors providing access to measurements necessary to know the internal functioning of biological processes. The majority of the key variables associated with these systems (concentration of biomass, substrates and products) can be measured only using analyzers on a laboratory scale – where they exist – which are generally very expensive and often require heavy and expensive maintenance. Thus, the majority of the control strategies used in industries are very often limited to indirect control of fermentation processes by control loops of the environmental variables such as dissolved oxygen concentration, temperature, pH, etc. 1.3. A schematic view of monitoring and control of a bioprocess Use of a computer to monitor and control a biological process is represented schematically in Figure 1.1. In the situation outlined, the actuator is the feed rate of the reactor. Its value is the output of the control algorithm, which uses the information of the available process. This information regroups, on the one hand, the state

What are the Challenges for the Control of Bioprocesses?

13

Screen monitor

Set point Computer Keyboard Monitoring algorithm

Software sensors

Control action Output gas

Effluent Concentrated substrate Supply of water

Liquid Sample Bioreactor

Sensors: analyzers and measuring systems

Figure 1.1. Schematic representation of bioprocess control system

of the process to date (i.e. measurements) and, on the other hand, available a priori knowledge (for example, in the form of a “material balance” model type) relative to a dynamic biological process and mutual interactions of different process variables. In certain cases – and in particular, when control objectives directly use variables that could not be measured (certain concentrations of biomass, substrates and/or products) or key parameters of the biological process (growth rate or more generally production rate, yield coefficients, transfer parameters) – information resulting from “in-line” measurements and a priori knowledge will be combined to synthesize “software sensors” or “observers,” whose principles and methods will be presented in Chapters 4 and 5. Thus, according to the available process knowledge and control objectives specified by the user, we will be able to develop and implement more or less complex control algorithms. 1.4. Modeling and identification of bioprocesses: some key ideas The dynamic model concept plays a central role in automatic control. It is in fact on the basis of the time required for the development of the knowledge process that

14

Bioprocess Control

the total design, analysis and implementation of monitoring and control methods are carried out. Within the framework of bioprocesses, the most natural way to determine the models that will enable the characterization of the process dynamics is to consider the material balance (and possibly energy) of major components of the process. It is this approach that we will consider in this work (although certain elements of hybrid modeling, which combines balance equations and neural networks, will be addressed in the chapter on modeling). One of the important aspects of the balance models is that they consist of two types of terms representing, respectively, conversion (i.e. kinetics of various biochemical reactions of the process and conversion yields of various substrates in terms of biomass and products) and the dynamics of transport (which regroups transit of matter within the process in solid, liquid or gaseous form and the transfer phenomena between phases). These models have various properties, which can prove to be interesting for the design of monitoring and control algorithms for bioprocesses, and which will, thus, be reviewed in Chapter 2. Moreover, we will introduce in Chapter 4 on state observers a state transformation that makes it possible to write part of the bioprocess equations in a form independent of the process kinetics. This transformation is largely related to the concept of reaction invariants, which are well known in the literature in chemistry and chemical engineering. An important stage of modeling consists not only of choosing a model suitable and appropriate for describing the bioprocess dynamics studied but also of calibrating the parameters of this model. This stage is far from being understood and therefore no solution has been obtained, given the complexity of models as well as the (frequent) lack of sufficiently numerous and reliable experimental data. Chapter 3 will attempt to introduce the problem of identification of the parameters of the models of the bioprocess (in dealing with questions of structural and practical identifiability as well as experiment design for its identification) and suitable methods to carry out this identification. 1.5. Software sensors: tools for bioprocess monitoring As noted above, sometimes many important variables of the process are not accessible to be measured online. Similarly, many parameters remain unclear and/or are likely to vary with time. There is, thus, a fundamental need to develop a model, which makes it possible to carry out a real-time follow-up of variables and key parameters of the bioprocess. Thus, Chapters 4 and 5 will attempt, respectively, to develop software tools to rebuild the evolution of these parameters and variables in the course of time. Insofar as their design gives reliable values to these parameters and variables, they play the role of sensors and will thus be called “software sensors”. The material is divided between the two chapters on the basis of distinction between state variables (i.e. primarily, component concentrations) whose evolution in time is described by differential equations and parameters (kinetic, conversion and transfer parameters), which are either the functions of process variables (as is typically the case for kinetic parameters

What are the Challenges for the Control of Bioprocesses?

15

such as specific growth rates) or constants (output parameters, transfer parameters)1. For state variables, we will proceed with the design of “software sensors” called state observers (Chapter 4), whereas for estimating the unknown or unclear parameters online, parameter estimators will be used (Chapter 5). Due to space considerations, Chapter 5 will deal exclusively with the estimation of kinetic parameters, which proves to be a more crucial problem to be solved. However, the methods which are developed are also applicable to other parameters. 1.6. Bioprocess control: basic concepts and advanced control An important aspect of bioprocess control is to lay down a stable real time operation, less susceptible to various disturbances, close to a certain state or desired profile compatible with an optimal operating condition. Chapter 6 will attempt to develop the basic concepts of automatic control applied to bioprocesses, particularly the concepts of control and setpoint tracking, feedback, feedforward control and proportional and integral actions. We can also initiate certain control methods specific to bioprocesses. The following chapter will concentrate on the development of more sophisticated control methods with the objective of guaranteeing the best possible bioprocess operation while accounting, in particular, for disturbances and modeling uncertainties. Emphasis will be placed, particularly, on optimal control and adaptive control methods based on the balance model as developed in the chapter on modeling. The objective is clearly to obtain control laws, which seek the best compromise between what is well known in bioprocess dynamics (for example, the reaction scheme and the material balance) and what is less understood (for example, the kinetics). 1.7. Bioprocess monitoring: the central issue With the exception of real-time monitoring of state variables and parameters, there has been little consideration of bioprocess monitoring. In particular, how to manage bioprocesses with respect to various operation problems, which are about malfunctioning or broken down sensors, actuators (valves, pumps, agitators, etc.), or even more basically malfunction of the bioprocess itself, if it starts to deviate from the nominal state (let us not forget that the process implements living organisms, which can possibly undergo certain, at least partial, transformations or changes, which are likely to bring the process to a different state from that expected). This issue is obviously important and cannot be ignored if we wish to guarantee a good real time process operation. This problem calls for all the process information (which is obtained from modeling, physical and software sensors or control). This will be covered in the final chapter.

1. The models used in practice are often so simplified with respect to reality that these parameters can “apparently” undergo certain variations with time. However, it is important to note that these variations are nothing but a reflection of the inaccuracy or inadequacy of the selected model.

16

Bioprocess Control

1.8. Conclusions A certain number of works exist in the literature, which deal with the application of automatic control in bioprocesses. This book is largely based on the following books: [BAS 90, VAN 98]. However, we should also mention other reference works worthy of interest: [MOS 88, PAV 94, PON 92, SCH 00]. Due to lack of space, we have not considered certain topics, which could, however, legitimately have had a place in this book. Initially, the informed reader would have noted that there is no chapter on instrumentation, which is, however, an essential link in monitoring and control. Fortunately, the reader will be able to complement the reading of this work with that (in French) of Boudrant, Corrieu, and Coulet [BOU 94] or that (in English) of Pons [PON 92]. In addition, we did not have the space for approaches such as metabolic engineering, a type of approach, which is already playing a growing role in bioprocess control. We suggest the reader consult the following book on metabolic engineering: [STE 98]. 1.9. Bibliography [BAS 90] G. BASTIN and D. D OCHAIN, On-line Estimation and Adaptive Control of Bioreactors, Elsevier, Amsterdam, 1990. [BOU 94] J. B OUDRANT, G. C ORRIEU and P. C OULET, Capteurs et Mesures en Biotechnologie, Lavoisier, Paris, 1994. [MOS 88] A. M OSER, Bioprocess Technology. Kinetics and Reactors, Springer Verlag, New York, 1988. [PAV 94] A. PAVÉ, Modélisation en Biologie et en Ecologie, Aléas, Lyon, 1994. [PON 92] M.N. P ONS, Bioprocess Monitoring and Control, Hanser, Munich, 1992. [SCH 00] K. S CHÜGERL and K.H. B ELLGARDT, Bioreaction Engineering. Modeling and Control, Springer, Berlin, 2000. [STE 98] G. S TEPHANOPOULOS, J. N IELSEN and A. A RISTIDOU, Metabolic Engineering, Academic Press, Boston, 1998. [VAN 98] J. VAN I MPE, P. VANROLLEGHEM and D. I SERENTANT, Advanced Instrumentation, Data Interpretation and Control of Biotechnological Processes, Kluwer, Amsterdam, 1998.

Chapter 2

Dynamic Models of Biochemical Processes: Properties of Models

2.1. Introduction Modeling biochemical processes is a delicate exercise. Contrary to physics, where there are laws that have been known to man for centuries (Ohm’s law, ideal gas law, Newton’s second law, the principles of thermodynamics, etc.) the majority of the models in biology depend on empirical laws. As it is not possible to base them only on available (and validated) knowledge, it is very important to be able to characterize the reliability of the laws used in the construction of the model. This implies hierarchism in the construction of biochemical models. In this chapter, we will see how to organize knowledge in the model in order to distinguish a reliable part established on the basis of a mass balance, and a more fragile part which will describe the bacterial kinetics. The quality of the model and, above all, its structure must correspond to the objective for which the model was built. In fact, a model can be developed for very different purposes, which will have to be clearly identified from the beginning. Thus, the model could be used to: – reproduce an observed behavior; – explain an observed behavior; – predict the evolution of a system; – help in understanding the mechanisms of the studied system; – estimate variables which are not measured; Chapter written by Olivier B ERNARD and Isabelle Q UEINNEC.

17

18

Bioprocess Control

– estimate parameters of the process; – act on the system to steer and control its variables; – detect an anomaly in the functioning of the process; – etc. The modeling objectives will generally lead to a formalism for designing the model. If we want to explain spatial heterogenity in a fermentor, it will be necessary to resort to a spatialized model (generally described by partial derivative equations). If the objective is to improve the production of a metabolite during the transitional stages, it will be necessary to represent the dynamics of the system. In the same way, the tools that we want to use to achieve the goal will guide us in choosing the model type (continuous/discrete, deterministic/stochastic, etc.). Furthermore, within the limit of these objectives, the model will also have to be adapted to the data available. In fact, a complex model implying a great number of parameters will require a large quantity of data to be identified and validated. Finally, considering the lack of validated laws in biology, the key stage of modeling is the validation of the model. This will be the subject of a special section. It is in fact fundamental to be able to show, on the basis of experimental data, that the model correctly achieves the assigned goals; we invite the reader to refer to [PAV 94] for a thorough reflection on modeling. 2.2. Description of biochemical processes 2.2.1. Micro-organisms and their use Microbial fermentation is a process in which a population of micro-organisms (bacteria, yeasts, moulds, etc.) are grown using certain nutritive elements (nutrients) under favorable surrounding conditions (temperature, pH, agitation, aeration, etc.). It schematically corresponds to the transformation of substances (generally carbonaceous substrates) into products, resulting from the metabolic activity of cells. The main components of the reaction are as follows: – substrates, denoted as Si , which are necessary for the growth of micro-organisms, or even which are precursors of a compound to be produced. These substrates generally contain a source of carbon (glucose, ethanol, etc.) and sometimes nitrogen (NO3 , NH4 , etc.) and phosphorus (PO4 , etc.); – microbial biomasses, denoted as Xi ; – end products, denoted as Pi , for agri-foods (oils, cheese, beer, wines, etc.), chemistry (solvents, enzymes, amino acids, etc.), the pharmaceutical industry (antibiotics, hormones, vitamins, etc.) or for the production of energy (ethanol, biogas, etc.) etc.

Dynamic Models of Biochemical Processes: Properties of Models

19

Mineral salt and vitamins are added to these main components, which, although they seldom appear in the models, are essential for growth. Each type of micro-organism contains some characteristics related to its genetic inheritance and to its regulating systems and fermentation can have various uses: – microbial growth: the primary objective is the growth of the micro-organism itself. This is the case of fermentations aiming to produce baking yeast; – metabolic production: the objective is to synthesize a preferred metabolite (ethanol, penicillin, etc.) using the cell; – substrate consumption: in this case, it is the degradation of the substrate which is desired. In this category, we will primarily find depollution processes (biological treatment of wastewater, breakdown of specific pollutants, etc.); – phenomenologic study: here, the purpose of fermentation is the study of the micro-organism. This can, for example, be to better understand how the micro-organism develops in the natural environment. The majority of biotechnological processes developed at an industrial level use microbial cultures made up of a single species of micro-organism for the possible synthesis of a well defined product (pure culture). However, in certain cases, several species can be made to grow simultaneously, but this is possible only if they are not too competitive. 2.2.2. Types of bioreactors From the view point of mathematical modeling, biological reactors can be divided into two major classes [BAI 86]: – stirred tank reactors (STR) for which the reacting medium is homogenous and the reaction is described by ordinary differential equations; – reactors with a spatial concentration gradient, such as fixed beds, fluidized beds, air lifts, etc., for which the reaction is described by partial derivative equations. In this chapter, we are interested only in the first class of stirred tank reactors, and any reader interested in the second class should see [JAC 96, DOC 94], etc. 2.2.3. Three operating modes Operating modes of bioreactors are generally characterized by liquid exchanges, i.e. by the type of substrate supply of the reactor. We can distinguish three main modes (Figure 2.1). Discontinuous (or batch) mode All the nutritive elements necessary for biological growth are introduced at the beginning of the reaction. Neither supply nor removal (except for some measurements)

20

Bioprocess Control

Batch

Semi-continuous

Continuous

Figure 2.1. Various operating modes of biological processes

is carried out thereafter, and the reaction takes place at constant volume. The only possible actions of the operator relate specifically to the environmental variables (pH, temperature, stirring velocity, aeration, etc.). Thus, few means are necessary for its implementation, which in fact is its attraction, from the industrial point of view. The second advantage is to guarantee the purity of cultures because the risks of culture contamination are less. A disadvantage is the minimal means which make it possible to operate the fermentor to optimize the use of micro-organisms. It also suffers from a major disadvantage: the initial supply of a high quantity of substrate generally inhibits the growth of micro-organisms which consume it, which results in lengthened durations of the process, and limit the acceptable initial load. Semi-continuous (or fed-batch) mode This operating mode is distinguished from the preceding mode by a supply of various nutritive elements as and when there is a need by the micro-organisms. It primarily makes it possible to eliminate the inhibition problems associated with the preceding mode, and to function at specific growth rates close to their maximum value [QUE 99]. From a previously inoculated initial volume the reactor is supplied by a flow controlled in a closed loop. In addition, it is the latter point which strongly limits the use of fedbatch at an industrial scale. Lastly, this operating mode, just like the previous one, is more particularly recommended when the recovery of the products is carried out at intervals (intracellular accumulation for example) or when it is dangerous to release residual toxic matters (which may happen in the continuous mode). Continuous (or chemostat) mode This is the most widely used mode in the field of the biological treatment of water. Characterized by a constant reaction volume, it reaches a state where the extraction of reaction medium equals the nutritive flow rate. The continuous processes work at steady state for fixed supply conditions, by maintaining the system in a stationary state, while avoiding any inhibiting phenomenon owing to the dilution effect due to the supply. Although it generally works in open loop, it is perhaps the richest operating mode from a dynamic point of view, because it makes it possible to study transitory

Dynamic Models of Biochemical Processes: Properties of Models

21

phenomena [GUI 96], the characteristics of a micro-organism over long periods of growth, optimization problems, etc. Moreover, it enables significant productions in small-sized reactors. Sequencing batch reactors (SBR) This is in fact a combination at one time of various operating modes. The idea is to recover the biomass by sedimentation or the supernatant product between two production (or treatment) sequences. The succession of various stages is represented in Figure 2.2.

1–Filling up

2–Reaction

3–Sedimentation

4–Withdrawal of a fraction of the sludge

5–Drain and return to step 1

Figure 2.2. SBR (sequencing batch reactors): time sequence of various stages

In the same way, the SFBR (sequencing fed-batch reactor) differs from the SBR only in the way in which the first filling stage is carried out, as and when there is a need and not in a single action. 2.3. Mass balance modeling 2.3.1. Introduction Modeling biological systems is a delicate task because there are no laws characterizing the evolution of micro-organisms. Nevertheless, these systems, like all physical systems, must comply with rules such as the conservation of mass, electroneutrality of solutions, etc. In this chapter, we will see how to frame the model around these physical laws, so as to guarantee some robustness. 2.3.2. Reaction scheme At the macroscopic level, the reaction scheme of a biochemical process describes the set of the main biological and chemical reactions. For this we adopt [BAS 90] formalism similar to that of chemistry, by simply defining the transformation of two reactants A and B into a product C in the following form: A + B −→ C

22

Bioprocess Control

By convention, contrary to chemistry, we do not consider stoichiometric coefficients in these reactions. In general the reaction rate corresponds to the growing rate of the implied biomass. At the macroscopic level, the reaction scheme is in fact a synthetic way of summarizing all reactions which are supposed to determine the dynamics of the process. Thus, a reaction scheme is generally based on assumptions related to the phenomenological details available. Unlike in chemistry, all the compounds intervening in the reaction are not strictly represented (which is fortunate, because it would be difficult to make a complete balance of compounds (Fe, Pb, F, etc.) necessary for microbial growth). The principal transformation reactions of the mass in a bioprocess are as follows: – growth of micro-organisms and biosynthesis (related to secondary metabolism) S1 + S2 + · · · + Sp −→ X + P1 + · · · + Pk when the growth is aerobic, we find in the substrates of the reaction a source of carbon, nitrogen, phosphorus and mineral salts, as well as oxygen. In the products, there is CO2 ; – synthesis of a product via primary metabolism S1 + S2 + · · · + Sq −→ P1 + · · · + Pl In this case, the manufacture of products is not dependent on the bacterial growth, but generally depends on enzymes produced by micro-organisms; – mortality X −→ Xd where, Xd corresponds to a dead biomass (whereas X is the living biomass). However, this equation can be completed only by adding the reaction rate and stoichiometric coefficients to it. This is why we will prefer to express the reaction scheme in a more complete form: ϕ

kA A + kB B −−→ kC C + X where ϕ is the reaction rate, here corresponding to the rate of biomass formation. The consumption yield of A is kA , that of B, is kB , and kC is the production yield of C. The production rate of C is therefore KC ϕ and the consumption rates of A and B are respectively kA ϕ and kB ϕ. Thereafter, we will assume that the reaction scheme is described by a set of k biological or chemical reactions. We will consider n variables (substrates, products, biomasses, etc.).

Dynamic Models of Biochemical Processes: Properties of Models

23

2.3.3. Choice of reactions and variables The choice of the number of reactions to be considered and components which intervene in these reactions is very important for modeling. It will be carried out based on the knowledge that we have on the process and measurements which could have been carried out. The reaction scheme, as we will see thereafter, will condition the structure of the model. It will thus have to be chosen with parsimony, bearing in mind the objectives of the model and the precision which is expected. The required number of reactions and the reaction scheme can be determined directly from a set of available experimental data [BER 05a, BER 05b]. In addition, in section 2.6 we will present a means of validating the reaction scheme. When we write the mass balance, we make the following assumptions: – the reaction scheme summarizes the distribution of mass and flows between various reactions intervening in the process; – yield coefficients are constant. 2.3.4. Example 1 Here, we will consider the example of anaerobic digestion. This wastewater treatment process uses anaerobic bacteria to degrade organic matter (S1 ). It is in fact a very complex process, in which a great number of bacterial populations intervene [MOS 83, DEL 01]. If the objective is to control this ecosystem, we will need a relatively simple model. We will also limit ourselves to considering two bacterial populations. Thus, we assume that the dynamics of the system can be summarized in two main stages: – acidogenesis stage (at rate r1 (·)), during which substrate S1 is degraded by acidogenic bacteria (X1 ) and is transformed into volatile fatty acids (S2 ) and into CO2 : r1 (·)

k1 S1 −−−−→ X1 + k2 S2 + k4 CO2

(2.1)

– methanogenesis stage (at rate r2 (·)), where the volatile fatty acids (VFA) are degraded into CH4 and CO2 by methanogenic bacteria (X2 ). r2 (·)

k3 S2 −−−−→ X2 + k5 CO2 + k6 CH4

(2.2)

The constants k1 , k2 and k4 respectively represent stoichiometric coefficients associated with consumption of substrate S1 , production of VFA and CO2 in the acidogenesis process. k3 , k5 and k6 respectively represent stoichiometric coefficients in the consumption of VFA and in the production of CO2 and CH4 during the methanogenesis process. It should be noted that this reaction scheme has no biological reality insofar as biomasses X1 and X2 represent a flora of different species. This is the same for substrates S1 and S2 which gather a group of heterogenous compounds. There are

24

Bioprocess Control

many models of this process [HIL 77, MOS 83, MOL 86]. Some authors, according to the modeling objective, chose much finer descriptions of implied processes [COS 91, BAT 97, BAT 02]. 2.4. Mass balance models 2.4.1. Introduction In stirred tank reactors, whatever the operating mode (batch, fed-batch, continuous), the dynamic behavior of the components of the biological reaction arises directly from the expression of mass fluxes, which translates that the variation of the quantity of a compound is equal to the sum of what is produced or supplied, reduced from what is consumed or decanted. In this case, using the reaction scheme, and knowing the hydrodynamics of the fermentor (mass inflow and outflow), we will see that we can directly obtain the mass balance model. 2.4.2. Example 2 Let us consider the growth of a UG5 saccharomyces cerevisiae yeast population on glucose S, and producing ethanol P according to the following reaction scheme: r(·)

k1 S −−−→ X + k2 P Thus, it is represented by the following ordinary differential equations: d(V X) = r(·)V − Qout X (2.3) dt d(V S) = −k1 r(·)V + Qin Sin − Qout S (2.4) dt d(V P ) = k2 r(·)V − Qout P (2.5) dt dV = Qin − Qout (2.6) dt in which X represents the concentration of micro-organisms (g/l), S the concentration of substrate (g/l), P the concentration of formed product (g/l), V the reaction volume (l), Qin the supply flow rate (l/h), Qout the withdrawal rate (l/h) and Sin the substrate influent concentration (g/l). r(·) represents growth rate of micro-organisms (1/h), and k1 and k2 are the coefficients respectively representing the yield of consumption and production. To this basic model, we could add terms of maintenance, mortality or additional equations concerning co-substrates of growth, intermediate products of the reaction, etc.

Dynamic Models of Biochemical Processes: Properties of Models

25

Equations (2.3)–(2.5) are generally re-expressed in terms of concentrations rather than quantities, which gives: dX dt dS dt dP dt dV dt

Qin X V Qin = −k1 r(·) + (Sin − S) V Qin = k2 r(·) − P V

= r(·) −

= Qin − Qout

(2.7) (2.8) (2.9) (2.10)

2.4.3. Example 3 However, growth and production are not strictly linked. We can particularly quote the case of intracellular accumulation of the product during the growing phase, which is then released when growth is completed (because of a substrate limitation for example). This is the case for penicillin production by the growth of the Penicillium fungus on glucose [BAJ 81, LIM 86]. In such a case, the reaction scheme will be divided into two parts: growth of the fungus and production of penicillin: rx (·)

k1 S −−−−→ X rp (·)

k2 S −−−−→ P Thus, the corresponding model is as follows: dX dt dS dt dP dt dV dt

= rx (·) −

Qin X V

= −k1 rx (·) − k2 rp (·) + = rp (·) −

(2.11) Qin (Sin − S) V

Qin P V

= Qin − Qout

(2.12) (2.13) (2.14)

2.4.4. Matrix representation The reaction scheme leads to equations which describe the distribution of the mass in the bioreactor [BAS 90] in an equivalent way: dξ = Kr(·) + D(ξin − ξ) − Q(ξ) + F dt

(2.15)

26

Bioprocess Control

in which ξ represents the process state vector, ξin the influent concentration vector and r(·) the reaction rate vector. K is the matrix containing stoichiometric coefficients (yields). Q(ξ), to which we will return in the following section, represents the terms of gaseous exchange between the gas phase and liquid phase. F is the supply of mass in gaseous form (like for example the mass flow of gaseous oxygen transferred to the reaction medium in aerobic fermentations). Finally, D is the dilution rate, equal to the ratio of supply flow rate Qin on the volume of reactor V . NOTE 2.1. This relatively standard matrix representation is more particularly adapted to the case of continuous fermentations for which volume is constant. On the other hand in the case of fed-batch reactions, the dilution rate varies with volume, which must also appear in the state vector. 2.4.4.1. Example 2 (continuation) If we consider model (2.7)–(2.10) in the case of a continuous reaction (Qout = Qin , D = QVin ), the model can be rewritten in matrix form (2.15) with: ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ X 1 0 ξ = ⎣S ⎦ , K = ⎣−k1 ⎦ , ξin = ⎣Sin ⎦ P k2 0 2.4.4.2. Example 1 (continuation) Let us reconsider the example of anaerobic digestion whose reaction scheme is given by (2.1) and (2.2). We will assume that methane is much less soluble, and thus that it passes very quickly in the gas phase. Dissolved CO2 is stored in the liquid phase where it enters the inorganic carbon compartment (C). dX1 dt dX2 dt dS1 dt dS2 dt dC dt

= r1 (·) − DX1

(2.16)

= r2 (·) − DX2

(2.17)

= D(S1in − S1 ) − k1 r1 (·)

(2.18)

= D(S2in − S2 ) + k2 r1 (·) − k3 r2 (·)

(2.19)

= D(Cin − C) − qC (ξ) + k4 r1 (·) + k5 r2 (·)

(2.20)

S1in , S2in and Cin are respectively concentrations of the inflow of substrate, VFA and dissolved inorganic carbon. The term qC (ξ) represents the flow of inorganic carbon (in the form of CO2 ) from liquid phase to gas phase.

Dynamic Models of Biochemical Processes: Properties of Models

27

2.4.5. Gaseous flow At the time of writing the gas balance, we must take into account the species which have a gas phase. These species can thus pass in the gas phase and escape from the fermentor (they can also pass from the gas phase to the liquid phase). It will thus be necessary to add a transfer term with the exterior in the balancing term. For this, Henry’s law is used which expresses the molar flow of compound C from liquid phase to its gas phase: qc = KL a(C − C  )

(2.21)

NOTE 2.2. If qc < 0, it means that the gas flow will take place from the gas phase to liquid phase. The transfer coefficient KL a (1/h) strongly depends on the operating conditions and in particular on agitation, pressure and transferring surface between the liquid and gas phases (size of the bubbles) [MER 77, BAI 86]. Modeling of the evolution of these parameters according to the operating conditions can prove very delicate. Quantity C  is the concentration of saturation in dissolved C. This quantity is related to partial pressure in gaseous C in gas phase (PC ) using Henry’s constant: C  = KH P C

(2.22)

Henry’s constant can also vary, but in a lesser manner, according to the culture medium and temperature. In addition, when several gas species are simultaneously found in the liquid phase, they must follow the ideal gas law, which will result in a constant relationship between molar flow and partial pressure. Thus, for m gas species C1 · · · Cm : Pc1 Pc2 Pcm = = ··· = qc1 qc2 qcm

(2.23)

2.4.6. Electroneutrality and affinity constants The electroneutrality of solutions is the second rule in which biological systems do not derogate: the concentration of anions balanced by the number of charges must equalize the concentration of cations balanced in the same manner. Pure chemical reactions are often well known, and an affinity constant is generally associated with them. This constant is often related to the concentration in protons H + , thus to the pH.

28

Bioprocess Control

2.4.7. Example 1 (continuation) Gas flow Methane flow is directly related to the methanogenesis rate: qM = k6 r2 (·)

(2.24)

qC (ξ) = KL a(CO2 − KH PC )

(2.25)

CO2 flow follows Henry’s law:

where PC is the partial pressure of CO2 . Affinity constants In the example of anaerobic digestion, we will make use of electroneutrality of the solution and chemical balances. We will consider that in the usual range of pH for this type of process (6 ≤ pH ≤ 8) VFA are in their ionized form. Dissolved CO2 is in equilibrium with bicarbonate: CO2 + H2 O ←→ HCO3− + H +

(2.26)

The affinity constant of the reaction is thus Kb =

HCO3− H + CO2

(2.27)

Electroneutrality of the solution Cations (Z) are mainly ions which are not affected by biochemical reactions (Na+ , . . .). Thus, the dynamics of the cations will simply follow, without modification, the concentration of cations Zin in the input, so that: dZ = D(Zin − Z) dt

(2.28)

As for anions they are mainly represented by VFA and bicarbonate, therefore electroneutrality ensures that Z = S2 + HCO3−

(2.29)

Dynamic Models of Biochemical Processes: Properties of Models

29

Matrix representation By adding equation (2.28), the model is finally expressed in matrix form (2.15) with: ⎤ ⎡ ⎤ ⎡ X1 1 0 ⎢X2 ⎥ ⎢ 0 1 ⎥ ⎥ ⎢ ⎥ ⎢  ⎢Z⎥ ⎢ r1 (·) 0 0 ⎥ ⎥ ⎢ ⎥ ⎢ (2.30) ξ = ⎢ ⎥ , r(·) = , K=⎢ 0 ⎥ r2 (·) ⎥ ⎢ S1 ⎥ ⎢−k1 ⎣ S2 ⎦ ⎣ k2 −k3 ⎦ C k4 k5 ⎤ 0 ⎢ 0 ⎥ ⎥ ⎢ ⎢ Zin ⎥ ⎥, ⎢ ξin = ⎢ ⎥ ⎢S1in ⎥ ⎣S2in ⎦ Cin ⎡

⎡

⎤ 0 ⎢ 0 ⎥ ⎢ ⎥ ⎢ 0 ⎥ ⎢ ⎥, Q=⎢ ⎥ ⎢ 0 ⎥ ⎣ 0 ⎦ qC (ξ)

(2.31)

An elimination of variables HCO− 3 , CO2 and PC by using equations (2.25), (2.23), (2.27) and (2.29) leads to the following expression for PC (ξ) (see [BER 02]):

φ − φ2 − 4KH PT (C + S2 − Z) PC (ξ) = (2.32) 2KH by showing: φ = C + S2 − Z + KH PT +

k6 kL a r2 (·).

Finally, this gives us: qC (ξ) = kL a(C + S2 − Z − KH PC (ξ))

(2.33)

2.4.8. Conclusion At this stage, we thus arrive at a model based on the following physical and chemical principles: – mass balance; – ionic balance; – affinity constant; – ideal gas law; – Henry’s law. The most important assumption (in terms of reliability) is that which proposes the mass balance. This will have to be verified afterwards.

30

Bioprocess Control

We will see in the following chapters that this model could be used as such in order to monitor the process (observers) or to control it (controllers). Nevertheless, if the objective defined at the beginning includes a simulation phase, it will be necessary to represent the reaction rates ri (·) according to the state of the system and the inputs (forcing environmental variables). This stage is much more delicate, and we will have to make many assumptions that are difficult to verify. 2.5. Kinetics 2.5.1. Introduction In some cases (optimal control, simulation, prediction, etc.) it is necessary to have an expression relating reaction rates to some variables of the system. However, it should be remembered that they are generally approximate relationships based on empirical considerations, and that we thus move away from the physical modeling established in the preceding section. In this chapter, we will see how to treat assumptions made on the kinetics level on a hierarchical basis and thus arrive at a description based on two reliability levels of the kinetics. 2.5.2. Mathematical constraints 2.5.2.1. Positivity of variables A priori, we know a certain number of physical constraints that the model must respect: the variables must remain positive and will be bounded if the mass entering the bioreactor is bounded. These physical constraints will impose constraints on the structure of ri (·). Some quantities (percentages, ratios, etc.) have to remain between two known bounds. To guarantee that the model preserves this property, a sufficient condition is given in the following property: i PROPERTY 2.1 (H1). For each state variable ξi , the field dξ dt on the axis ξi = 0 must point towards the acceptable part of the space. In other words, if the variable ξ ∈ [Lmin , Lmax ], the following property must be verified:

dξi ≥0 dt dξi ≤0 =⇒ dt

ξ = Lmin =⇒ ξ = Lmax

Specific case: in order for a variable ξi to remain positive, it should be ensured that i ξi = 0 ⇒ dξ dt ≥ 0.

Dynamic Models of Biochemical Processes: Properties of Models

31

2.5.2.2. Variables necessary for the reaction The second important constraint that will have to respect the biochemical kinetics is related to the reaction scheme. The reaction cannot take place if one of the reagents necessary for the considered reaction is missing. This explains the following property: PROPERTY 2.2. If ξj is a reactant of reaction i, then ξj can be factorized in ri : ri (ξ, u) = ξj νij (ξ, u) Thus, we easily verify that ξj = 0 ⇒ ri (ξ, u) = 0. In the same way, for the reactions associated with a biomass X, we will also have the same property. Thus, a growth reaction will be written: ri (ξ, u) = μi (ξ, u)X The term μi is then called the specific growth rate. 2.5.2.3. Example 1 (continuation) Let us consider the model given by equations (2.16) to (2.19). Let us apply the positivity principle of variables: X1 = 0 =⇒ r1 (·) ≥ 0

(2.34)

X2 = 0 =⇒ r2 (·) ≥ 0

(2.35)

S1 = 0 =⇒ D(S1in − S1 ) − k1 r1 (·) ≥ 0

(2.36)

S2 = 0 =⇒ D(S2in − S2 ) + k2 r1 (·) − k3 r2 (·) ≥ 0

(2.37)

Equations (2.34) and (2.35) give little information. In order that (2.36) and (2.37) are followed irrespective of experimental conditions, it is necessary that: r1 (·) = S1 φ1 (·)

and

r2 (·) = S2 φ2 (·)

In addition, biomasses X1 and X2 are necessary for reactions 1 and 2 respectively, and from this: r1 (·) = μ1 (·)X1

and

r2 (·) = μ2 (·)X2

Thus, finally we must have: r1 (·) = S1 X1 ν1 (·)

(2.38)

r2 (·) = S2 X1 ν2 (·)

(2.39)

2.5.2.4. Phenomenological knowledge In order to propose an expression for reaction kinetics, we will make use of the phenomenological knowledge available, although this is often highly speculative.

32

Bioprocess Control

First of all, laboratory experiments often make it possible to determine the variables which act on the reaction rates. We saw that among these variables we must find the reactants and possibly the biomass. Then, it is necessary to determine if the reaction rate is activated or inhibited by these variables. It frequently occurs that a variable is activated and that it becomes inhibiting at a very high concentration (toxicity). It now remains to propose an analytical expression which will enable us to consider mathematical constraints and phenomenological details on the process. For this, on the one hand we stress the importance of experimental observations (when they exist), and on the other hand of existing models in the literature. In all cases, we prefer the principle of parsimony, so that the models obtained can be calibrated and validated thereafter. The following sections present the models most often selected to describe the growth of micro-organisms. 2.5.3. Specific growth rate Even if the specific growth rate strongly depends on the operating conditions (temperature, pH, etc.), the reactive medium (concentrations in carbonaceous, nitrogenized, phosphorated compounds, in mineral salt, in oxygen, etc.), the expression most commonly used is Monod’s empirical model [MON 42], which, to describe the bacterial growth, used the law introduced at the beginning of the 20th century by MichaëlisMenten for enzymatic kinetics: μ = μmax

S Ks + S

(2.40)

This expression, in which μmax is the maximum specific growth rate (1/h) and Ks the half-saturation constant (g/l), allows us to describe the phenomenon of limited growth for lack of substrate, and complete breakdown when the substrate is no longer available. Let us note that the analogies with enzymatic kinetics have often been used to determine a growing model [SEG 84, EDE 88]. In addition, the inhibition phenomena due to excess of substrate are generally modeled by Haldane’s expression, introduced in the case of enzymatic reactions, and taken again by Andrews [AND 68] in the case of biological reactions: μ = μmax with Ki inhibition constant (g/l).

S Ks + S +

S2 Ki

(2.41)

Dynamic Models of Biochemical Processes: Properties of Models

33

, Haldane s model 0.4 0.35 μ max = 0.41; K s = 0.002 ; K i = 0.35

Growth rates (1/h)

0.3 0.25 0.2 0.15 0.1 0.05 0

0

0.05

0.1

0.15

0.2 0.25 0.3 Substrate (g/l)

0.35

0.4

0.45

0.5

Figure 2.3. Haldane’s model – μ(S)

It should be noted that many other algebraic relations were established to describe these limitation and/or inhibition phenomena, but their use remains marginal. In the same manner, certain models take into account the influence of concentration of microorganisms, co-metabolite, temperature, pH (see [DOC 86, BAI 86, STE 98] for a list of about 50 models). The case of oxygen is a little different. In fact, in the case of processes functioning in an aerobic environment, oxygen corresponds to a co-substrate of the reaction, and can thus be treated as such, i.e. to intervene in the form of a Monod type term in the expression of a specific growth rate, thus leading to the following expression:  μ = μmax

S Ks + S



O2 KO2 + O2

(2.42)

with concentration O2 in dissolved oxygen (g/l) and KO2 the half-saturation constant for oxygen (g/l). However, this expression is often omitted under the assumption that the reactor is sufficiently aerated and that KO2 is very small compared to the concentration of dissolved oxygen present in the reactor under normal conditions.

34

Bioprocess Control

In certain cases, a co-substrate can also act in a competitive way. Let us consider a bacterium such as Pseudomonas putida, which can, under certain conditions, simultaneously use phenol (S1 ) and glucose (S2 ) as sources of energy. Therefore, the specific growth rate can be expressed as the sum of two specific growth rates on each of the substrates: μ = μ1 (S1 , S2 ) + μ2 (S1 , S2 ) The competitive activation/inhibition phenomenon in the bisubstrate mixture can be described by an expression of the form [WAN 96]: μi (Si , Sj ) =

μmax,i Si Ks,i + Si +

Si2 Ki,i

,

+ K3,i Sj + K4,i Si Sj

i, j = 1, 2, i = j (2.43)

In the absence of substrate Sj , μi (Si , 0), is brought to the previously described Andrews expression. By a judicious choice of K3,i , K4,i , i = 1, 2, we can then easily describe the inhibition phenomena whether competitive or not, cross or partial [WAN 96]. 2.5.4. Representation of kinetics by means of a neural network In an alternative way, we can model the kinetics using a neural network (hybrid models). We do not make an assumption a priori on kinetics (or simply the constraints which guarantee that the system trajectories have an acceptable biological meaning) and we directly identify kinetics while learning the network. Nevertheless, it is necessary to decide on variables which influence kinetics, and which will be the inputs of the neural network. A diagrammatic view of the network is represented in Figure 2.4 for a single hidden layer. The expression of the network output according to the inputs is expressed in the following way:  m  nh

ωk φ υki Si (2.44) μ(S1 , . . . , Sm ) = k=1

i=1

where nh represents the number of neurons in the hidden layer. ωk and υki are respectively the weight of input and output layers. Function φ represents the activation function of the neuron (sigmoid, hyperbolic tangent, Gaussian, etc.). The choice of network type and the number of neurons is a relatively standard question; see [HER 91] for more precise details. Once the structure of the network is chosen, it is then necessary to identify the weight of the network during the learning phase. For more precise details, see [CHE 00].

Dynamic Models of Biochemical Processes: Properties of Models

ν1,1 S1

35

ω1 ω2

S2

ω3 ωk

μ

νm,1 Sm

Input layer

ω nh ν m,n h

Hidden layer

Output layer

Figure 2.4. Diagram of a neural network containing a hidden layer

2.6. Validation of the model 2.6.1. Introduction The last phase of modeling is undoubtedly the most important phase, but also that which is generally neglected. It is all the more important as we saw that it was necessary to make a great number of often speculative assumptions. Before using a model, it is crucial to validate it correctly. This validation stage must be kept on through the identification phase which will be discussed in Chapter 6. The general objective of validation is to verify that the model responds well to the objectives laid down. More precisely, we will see how to individually test the assumptions which were made during the construction of the model. For this we proceed in 3 stages, during which we test: – the reaction scheme; – the qualitative predictions of the model; – the model in its totality (reaction scheme + kinetics + parameters). The validation must be made on the basis of a data file not having been used to construct or identify the model, and for different experimental conditions (if not, we would instead be testing experimental reproducibility). If these conditions are not observed, the model cannot claim to be validated. 2.6.2. Validation of the reaction scheme 2.6.2.1. Mathematical principle The procedure that we propose here relies on an important characteristic, which is a consequence of the mass conservation within the bioreactor. In fact it consists

36

Bioprocess Control

of verifying the consistency of data compared to mass flows induced by the reaction scheme [STE 98, BER 05a]. PROPERTY 2.3. Matrix K is of dimension n × k. We suppose that there are more variables than reactions: n > k. In these conditions, we have a minimum n − k independent vectors vi ∈ Rn such that: viT K = 01×k By convention, we normalize the first component of the vectors vi such that vi1 = 1. Consequence: let us consider the real variable wi = viT ξ, this variable satisfies the following equation: dwi = D(wiin − wi ) − viT Q(ξ) + viT F dt

(2.45)

with wiin = viT ξin . Let us integrate expression (2.45) between two moments t1 and t2 , and develop components vij of vector vi . n

vij φξj (t1 , t2 ) = φξ1 (t1 , t2 )

(2.46)

j=2

by noting  φξj (t1 , t2 ) = ξj (t2 ) − ξj (t1 ) −

t2

(D(τ )(ξjin (τ ) − ξj (τ )) − Qj (τ ) + F (τ ))dτ

t1

φξj (t1 , t2 ) can be numerically estimated on the basis of experimental measurements of ξj over the course of time. Expression (2.46) is a linear relation connecting vij to terms φξj (t1 , t2 ). As φξj (t1 , t2 ) can be calculated between different moments t1 and t2 , (2.46) is a linear regression whose validity could be tested in experiments. 2.6.2.2. Example 4 Here, we will consider the simple example of growth of the filamentous fungus Pycnoporus cinnabarinus (X) on two substrates, a glucose source of carbon (C) and an ammonium source of nitrogen (N ). It is thus supposed that the reaction scheme is made up of a reaction: N + C −→ X

Dynamic Models of Biochemical Processes: Properties of Models



37

Therefore, stoichiometric matrix K associated with this reaction is the following  T  : ξ= X N C  K = 1 −k1

−k2

T

 ξin = 0

and

Nin

Cin

T

(2.47)

We will consider the following two vectors in the space orthogonal to K:   T T 1 1 0 and v2 = 1 0 v1 = 1 k1 k2 We can now define the following quantities:  t2 D(τ )X(τ ) φX (t1 , t2 ) = X(t2 ) − X(t1 ) + t1



t2

φN (t1 , t2 ) = N (t2 ) − N (t1 ) −  φC (t1 , t2 ) = C(t2 ) − C(t1 ) −

D(τ )(Nin (τ ) − N (τ ))dτ

t1 t2

D(τ )(Cin (τ ) − C(τ ))dτ

t1

which will enable us to write the following two regressions: 1 φN (t1 , t2 ) k1 1 φX (t1 , t2 ) = φC (t1 , t2 ) k2 φX (t1 , t2 ) =

(2.48) (2.49)

It is now easy to verify if relations (2.48) and (2.49) are valid by seeing, on the basis of experimental measurements, if there is a linear relationship between the different φξi (t1 , t2 ). Figure 2.5 shows an example of validation on the basis of a series of experiments. The regression obtained is highly significant, which means that relations (2.48) and (2.49) are valid. Consequently matrix K, which is in the orthogonal of v1 T  and v2 is strictly of the type K = 1 −α1 −α2 : the reaction scheme is validated and so is the mass balance. 2.6.3. Qualitative validation of model At this stage, we assume that the reaction scheme, and thus the mass balance model were validated. We will thus consider the simulation model including reaction kinetics. The first thing to do is to verify if the qualitative properties of the model correspond to experimental observations.

38

Bioprocess Control

X estimated from N (g/l)

2.5 A 2 1.5 1 0.5 0.5

1 1.5 2 X measured (g/l)

2.5

1 1.5 2 X measured (g/l)

2.5

X estimated from S (g/l)

2.5 B 2 1.5 1 0.5 0.5

Figure 2.5. Validation of linear relation relating φX and φN (A) and φX and φC (B)

When the inputs of the model are constant, does it predict a steady state, or a more complex behavior (limit cycle, chaos, etc.) in agreement with observations? These qualitative properties are often dependent on the dilution rate of the bioreactor, as well as influent concentrations. Is a qualitative change observed (e.g. a stable equilibrium becomes unstable) when we vary these inputs? In addition, more precise qualitative properties on the nature of transients (oscillation type, extrema, etc.) can sometimes be deduced from the structure of a dynamic system ([JEF 86, SAC 90, GOU 98, BER 95, BER 02]). For example, Hansen and Hubell (1980) [HAN 80] study the competition between two species of bacteria in a chemostat. They show that the result of the competition predicted by the model depends on the dilution rate. For a dilution rate lower than a certain value it is one species which wins, whereas for higher values, it is the other species. These qualitative properties are verified in experiments.

Dynamic Models of Biochemical Processes: Properties of Models

39

2.6.4. Global validation of the model This is the most standard way to validate a model, by quantitatively comparing the results obtained by the model with experimental data. In this approach, the model is considered as a whole. If the adequacy between model and experiment is not good, it will be difficult to determine if the problem is mainly structural (bad assumptions on the reaction scheme), related to mathematical expressions used for kinetics, or if the parameters of the model are not correctly estimated. If the first two stages of the validation are successfully crossed, it can be assumed that the problem is mainly due to incorrect parameter values. In theory, validation of a model consists of verifying statistical assumptions: differences between predictions and measurements must follow a normal law of zero average [WAL 97]. Practically, and considering the difficulty of validating in the strictest sense the biotechnological models, we are satisfied with a good visual adequacy. We can possibly reinforce this subjective criterion by an analysis of correlation between predictions and measurements. 2.7. Properties of the models 2.7.1. Boundedness and positivity of variables We saw in section 2.5.2 that the models must be constructed, in such a manner as to respect constraints as positivity of state variables. We will see here that the models based on mass balance are models of the BIBS type (input bounded state). For this, we use the following property which arises from the principle of mass conservation: HYPOTHESIS 2.1 (H2). There is a vector v + whose components are strictly positive, such as: T

v + K = 01×k T

Consequences: if we set w+ = v + ξ, w satisfies the equation of type (2.45): T T dw+ + = D(win − w+ ) − v + Q(ξ) + v + F dt

(2.50)

We will make an assumption on Q(ξ), which is verified in most cases: HYPOTHESIS 2.2 (H3). Q(ξ) can be compared to a linear expression as follows: T

T

v + Q(ξ) ≥ av + ξ + b where a is a positive real b ∈ Rn .

40

Bioprocess Control

HYPOTHESIS 2.3 (H4). Mass supply in gaseous form F is bounded: F ≤ Fmax , in accordance with physical reality. T

This hypothesis is verified if v + Q(ξ) = 0, or if Q(ξ) is described by Henry’s law. PROPERTY 2.4. If hypothesis (H1), (H2), (H3) and (H4) are verified, then the system is of BIBS type. Proof. The dynamics of w+ can thus be raised by  T + Dwin + v + Fmax − b dw+ + ≤ (D + a) −w (2.51) dt D+a   T + Dwin +v + Fmax −b . and by applying property 2.1, we deduce w+ ≤ min w+ (0), D+a In other words,



wi+ ξi is bounded. As wi+ > 0, state variables ξi are bounded.

2.7.2. Equilibrium points and local behavior 2.7.2.1. Introduction In this section we recall the principles of the study of model properties; see [KHA 02] for more details. In general, bioreactor models are nonlinear, and often of large dimensions. They include many parameters which also intervene in a nonlinear way. However, passing dimension 3, it becomes mathematically very difficult to characterize the behavior of a dynamic system. However, we show that the models with mass balance have structural properties which can help in understanding the system. In this section, we describe the dynamic system using an equation: dξ = f (ξ, u) dt

(2.52)

However, we remember that f (ξ, u) = Kr(ξ) + D(ξin − ξ) − Q(ξ) + F . We consider the case where u = (D, ξin , F ) is constant. Equilibrium points and local stability dξ Equilibrium points are points such as = 0 when all the inputs are maintained dt as constant.

Dynamic Models of Biochemical Processes: Properties of Models

41

The nonlinear systems differ generically from linear systems by the fact that they can have several equilibrium points. The first stage in model analysis consists of testing if these equilibrium points are locally stable. For this, we consider the Jacobian matrix: J(ξ) =

Df (ξ) Dξ

Equilibrium ξ0 is locally stable if and only if all the eigenvalues of J(ξ0 ) have strictly negative real parts. If there is an eigenvalue with a positive real part, the equilibrium is unstable. We cannot conclude on the stability of the system if the eigenvalues have a non-positive real part and two of them are pure imaginary conjugates. Global behavior The dynamics of a nonlinear system can be very complicated, and behaviors such as limit cycles and chaos can appear in addition to equilibria. It is thus important to find when there is a single locally stable equilibrium, if it is globally stable, i.e. if, irrespective of initial conditions, trajectories will converge towards this equilibrium. The standard method to prove that a steady state is globally stable is based on the Lyapunov functions [KHA 02]. To find a Lyapunov function for a biological system is a difficult task, though; see [LI 98] for constructive methods to find Lyapunov functions in a broad range of growth models. Asymptotic behavior In section 2.6.2 we saw that generally there were n − k vectors vi which make it possible to calculate the wi quantities whose dynamics satisfy equation (2.45). Generally, there exist q independent vectors among the vi , denoted vi0 which verify vi0t Q(ξ) = 0

(2.53)

Moreover, without loss of generality, let us consider the case without mass supply in gas form (F = 0). The associated dynamics of wi0 are thus simple: dwi0 0 = D(wiin − wi0 ) dt

(2.54)

Under the conditions which we consider (i.e. constants D and ξin ), the solutions 0 . This means that the solutions of sysof (2.54) asymptotically converge towards wiin 0 . tem (2.52) will converge towards the hyperplane given by vi0t ξ = wiin The state of the system will thus asymptotically tend towards the vectorial subspace of dimension n − q orthogonal to the q vectors vi0 . This makes it possible to bring back the study of (2.52) of dimension n to dimension n − q.

42

Bioprocess Control

Example 4 (continuation) Let us consider the model given by equation (2.47). Moreover assume that the kinetics follow Monod’s law with respect to the two substrates C and N : r(ξ) = μmax

N C X KC + C KN + N

(2.55)

Let us again take the two vectors v1 and v2 identified in section 2.6.2. These vectors simply verify equation (2.53). Therefore, X +

N k1

−→

Nin k1

and X +

C k2

−→

Cin k2 .

The system study in dimension 3 is thus brought back to that of the following system, of dimension 1: dX Nin − k1 X Cin − k2 X = μmax X − DX dt KC + Cin − k2 X KN + Nin − k1 X

(2.56)

It can be verified that this system has three real equilibrium points (one of them being the trivial equilibrium X = 0). These ordered equilibria are respectively locally stable, unstable and locally stable. Non-trivial equilibria, depending on values of the parameters, are positive (and thus acceptable) or not. For the parametric domains where there is only one strictly positive equilibrium, this equilibrium is then globally stable. 2.8. Conclusion In this chapter, we presented a constructive and systematic method to develop bioprocess models in 4 stages. Let us recall here that bioprocess modeling should be carried out only with the vision of an objective clearly identified at the beginning, and in relation to the quality and quantity of information available to establish and validate the model. The first stage of modeling consists of collecting physical and chemical principles, and making an assumption about a reaction scheme to arrive at a model of mass balance. In the second stage, it is necessary to benefit from the constraints that the model must respect, and make use of empirical relations to determine an expression for kinetics. The third stage of modeling consists of identifying the parameters; these aspects are detailed in Chapter 6. Lastly, we should not neglect the final modeling stage: validation of the model. During this phase, quality of the model must be tested in the most objective way possible, and the operating domains for which it reaches its limits must be identified. In this chapter, we were interested only in the models for processes assumed to be infinitely mixed, and only these spatially homogenous systems will be considered in this book.

Dynamic Models of Biochemical Processes: Properties of Models

43

Finally, let us note that here we restricted ourselves to the knowledge models, resulting from physical considerations. In fact, the black box models of input-output behavior are more concerned with the identification of parameters. 2.9. Bibliography [AND 68] J.F. A NDREWS, “A mathematical model for the continuous culture of microorganisms utilizing inhibitory substrate”, Biotechnol. & Bioeng., vol. 10, pp. 707–723, 1968. [BAI 86] J.E. BAILEY and D.F. O LLIS, Biochemical Engineering Fundamentals, McGrawHill, New York, 1986. [BAJ 81] R.K. BAJPAI and M. R EUSS, “Evaluation of feeding strategies in carbon-regulated secondary metabolite production through mathematical modelling”, Biotechnol. & Bioeng., vol. 23, pp. 717–738, 1981. [BAS 90] G. BASTIN and D. D OCHAIN, On-line Estimation and Adaptive Control of Bioreactors, Elsevier, Amsterdam, 1990. [BAT 97] D. BATSTONE, J. K ELLERAND, B. N EWELL and M. N EWLAND, “Model development and full scale validation for anaerobic treatment of protein and fat based wastewater”, Water Science and Technology, vol. 36, pp. 423–431, 1997. [BAT 02] D.J. BATSTONE, J. K ELLER, I. A NGELIDAKI, S.V. K ALYUZHNYI, S.G. PAVLOSTATHIS, A. ROZZI, W.T.M. S ANDERS, H. S IEGRIST and V.A. VAVILIN , Anaerobic Digestion Model No.1 (ADM1). IWA Task Group for Mathematical Modelling of Anaerobic Digestion Processes, IWA Publishing, London, 2002. [BER 95] O. B ERNARD and J.-L. G OUZÉ, “Transient behavior of biological loop models, with application to the Droop model”, Mathematical Biosciences, vol. 127, no. 1, pp. 19– 43, 1995. [BER 01] O. B ERNARD, Z. H ADJ -S ADOK, D. D OCHAIN, A. G ENOVESI, and J.-P. S TEYER, “Dynamical model development and parameter identification for an anaerobic wastewater treatment process”, Biotech. Bioeng., vol. 75, pp. 424–438, 2001. [BER 02] O. B ERNARD and J.-L. G OUZÉ, “Global qualitative behavior of a class of nonlinear biological systems: application to the qualitative validation of phytoplankton growth models”, Artif. Intel., vol. 136, no. 1, pp. 29–59, 2002. [BER 05a] O. B ERNARD and G. BASTIN, “Identification of reaction networks for bioprocesses: determination of a partially unknown pseudo-stoichiometric matrix”, Bioprocess Biosyst. Eng., vol. 27, no. 5, pp. 293–301, 2005. [BER 05b] O. B ERNARD and G. BASTIN, “On the estimation of the pseudo-stoichiometric matrix for macroscopic mass balance modelling of biotechnological processes”, Math. Biosciences, vol. 193, no. 1, pp. 51–77, 2005. [CHE 00] L. C HEN, O. B ERNARD, G. BASTIN and P. A NGELOV, “Hybrid modelling of biotechnological processes using neural networks”, Control Eng. Practice, vol. 8, pp. 821– 827, 2000.

44

Bioprocess Control

[COS 91] D.J. C OSTELLO, P.F. G REENFIELD and P.L. L EE, “Dynamic modelling of a singlestage high-rate anaerobic reactor - I. Model derivation”, Water Research, vol. 25, pp. 847– 858, 1991. [DEL 01] C. D ELBES, R. M OLETTA and J.-J. G ODON, “Bacterial and archaeal 16s rdna and 16s rrna dynamics during an acetate crisis in an anaerobic digestor ecosystem”, FEMS Microbiology Ecology, vol. 35, pp. 19–26, 2001. [DOC 86] D. D OCHAIN, On line Parameter Estimation, Adaptive State Estimation and Control of Fermentation Processes, PhD thesis, Université Catholique de Louvain, Belgium, 1986. [DOC 94] D. D OCHAIN, Contribution to the Analysis and Control of Distributed Parameter Systems with Application to (Bio) chemical Processes and Robotics, Thèse d’agrégation de l’enseignement supérieur, Université Catholique de Louvain, Belgium, 1994. [EDE 88] L. E DELSTEIN, Mathematical Models in Biology, Random House, New York, 1988. [GOU 98] J.-L. G OUZÉ, “Positive and negative circuits in dynamical systems”, Journal of Biological Systems, vol. 6, no. 1, pp. 11–15, 1998. [GUI 96] V. G UILLOU, I. Q UEINNEC, J.L. U RIBELARREA and A. PAREILLEUX, “Online sensitive lightness measurement of cell-mass in Saccharomyces Cerevisiae culture”, Biotechnology Techniques, vol. 10, no. 1, pp. 19–24, 1996. [HAN 80] S.R. H ANSEN and S.P. H UBBELL, “Single-Nutrient Microbial Competition”, Science, vol. 207, no. 28, pp. 1491–1493, 1980. [HER 91] J. H ERTZ, A. K ROGH and R.G. PALMER, Introduction to the Theory of Neural Computation, Addison-Wesley, 1991. [HIL 77] D.T. H ILL and C.L. BARTH, “A dynamic model for simulation of animal waste digestion”, Journal of the Water Pollution Control Association, vol. 10, pp. 2129–2143, 1977. [JAC 96] J. JACOB, H. P INGAUD, J.M. L E L ANN, S. B OURREL, J.P. BABARY and B. C APDEVILLE, “Dynamic simulation of biofilters”, Simulation – Practice and Theory, vol. 4, pp. 335–348, 1996. [JEF 86] C. J EFFRIES, “Qualitative Stability of Certain Nonlinear systems”, Linear Algebra and its Applications, vol. 75, pp. 133–144, 1986. [KHA 02] H.K. K HALIL, Nonlinear Systems, Prentice-Hall, Englewood Cliffs, 2002. [LI 98] W. L I, “Global asymptotic behavior of the chemostat: general response functions and different removal rates”, SIAM J. Appl. Math., vol. 59, pp. 411–422, 1998. [LIM 86] H.C. L IM, Y.J. TAYEB, J.M. M ODAK and P. B ONTE, “Computational algorithms for optimal feed rates for a class of fed-batch fermentation: numerical results for penicillin and cell mass production”, Biotechnol. & Bioeng., vol. 28, pp. 1408–1420, 1986. [MER 77] J.C. M ERCHUK, “Further considerations on the enhancement factor for oxygen absorption into fermentation broth”, Biotechnol. & Bioeng., vol. 19, pp. 1885–1889, 1977.

Dynamic Models of Biochemical Processes: Properties of Models

45

[MOL 86] R. M OLETTA, D. V ERRIER and G. A LBAGNAC, “Dynamic modelling of anaerobic digestion”, Water Research, vol. 20, pp. 427–434, 1986. [MON 42] J. M ONOD, Recherches sur la Croissance des Cultures Bactériennes, Hermann, Paris, 1942. [MOS 83] F.E. M OSEY, “Mathematical modelling of the anaerobic digestion process: regulatory mechanisms for the formation of short-chain volatile acids from glucose”, Water Science and Technology, vol. 15, pp. 209–232, 1983. [PAV 94] A. PAVÉ, Modélisation en Biologie et en Ecologie, Aléas, Lyon, 1994. [QUE 99] I. Q UEINNEC, D. L ÉONARD and C. B EN YOUSSEF, “State observation for fedbatch control of phenol degradation by R alstonia eutropha”, in Proc. of the 4th European Control Conference, Karlsruhe, Germany, 1999. [SAC 90] E. S ACKS, “A dynamic systems perspective on qualitative simulation”, Artif. Intell., vol. 42, pp. 349–362, 1990. [SEG 84] L.A. S EGEL, Modeling Dynamic Phenomena in Molecular and Cellular Biology, Cambridge University Press, Cambridge, 1984. [STE 98] G.N. S TEPHANOPOULOS, A.A. A RISTIDOU and J. N IELSEN, Metabolic Engineering. Principles and Methodologies, Academic Press, San Diego, 1998. [WAL 97] E. WALTER and L. P RONZATO, Identification of Parametric Models from Experimental Data, Springer-Verlag, Heidelberg, 1997. [WAN 96] K.W. WANG, B.C. BALTZIS and G.A. L EWANDOWSKI, “Kinetics of phenol biodegradation in the presence of glucose”, Biotechnol. & Bioeng., vol. 51, pp. 87–94, 1996.

46

Chapter 3

Identification of Bioprocess Models

3.1. Introduction The complexity of dynamic bioprocess models, which are usually employed, is actually a reflection of the complexity of the processes themselves as well as of the detailed knowledge about them acquired over the course of the past decades. In the real sense, they comprise a large number of parameters, to which it will be necessary to assign a numerical value either on the basis of prior knowledge about their value or on the basis of experimental data. The quality of the estimation of the parameters will inherently depend on the quantity and quality of the data made available for the calibration (or identification) procedure of the parameters of the model. Apart from these limitations with regard to the available information, another problem emerges with regard to identifying the parameters of the model: the dynamic models of the bioprocesses are intrinsically nonlinear and the parameters of the model can be strongly correlated among themselves. The study into the identifiability of the parameters of the model before their estimation turns out to be an essential task. The central question for analyzing the identifiability can be summarized as follows: let us assume that a certain number of variables of the state of the model are available to the required extent; on the basis of the structure of the model (structural identifiability) or on the basis of the type and quality of the data available (practical identifiability), can we expect to be in a position to attribute a unique value to each parameter of the model by means of parameter estimation? The answer to the analysis of the identifiability cannot usually be expressed in the form of a “yes” or a “no”, but when it gives the results (this is not necessarily evident in the case of nonlinear

Chapter written by Denis D OCHAIN and Peter VANROLLEGHEM.

47

48

Bioprocess Control

models), these can very well indicate whether it is possible to identify such subsets or combinations of parameters. The chapter is organized as follows. First, we will deal with the questions of structural identifiability and practical identifiability, before seeing how to design and implement the experiments (experiment design), which will enable us to obtain the best data for the estimation of parameters. We will then review different techniques for parameter identification. We will conclude with a case study: the parameter identification of a model of anaerobic digestion. 3.2. Structural identifiability In the case of structural identifiability, we will examine if, in the (academic) case of ideal measurements (continuous measurements without noise, which are perfectly adjusted with respect to the model), all the parameters of the studied model are identifiable. It may lead to the conclusion that only a combination of parameters is identifiable. If the number of combinations of parameters is less than the number of parameters of the model, or there is no one-to-one relationship between these combinations and parameters, then only prior knowledge of certain parameters enables us to solve the problem of identifiability. Let us consider a simple example: in the model y = ax1 + bx2 + c(x1 + x2 ), only the parameters a + c and b + c are structurally identifiable (and not the three parameters, a, b, c); two parameters (for example, a and b) will be identifiable if the value of the third (here c) is known beforehand. For linear systems, the structural identifiability is very well understood and there are many tests available for this (see, for example [GOD 85]). On the other hand, the problem turns out to be much more complex for nonlinear models and at present only a few methods enable us to test their identifiability, often with the possibility of highlighting only the necessary and/or sufficient conditions of local identifiability (i.e., valid only for certain domains of the model) [WAL 82]. The list below recapitulates the available methods, including the references to applications based on models including Monod kinetics: – transformation of the nonlinear model into a linear model [GOD 85]. Applications: [BOU 98, DOC 95, SPE 00]; – development in series: - Taylor series [GOD 85, POH 78]. Applications: [BOU 98, DOC 95, HOL 82a, JEP 96, SPE 00, PET 00b]; - Generating series [WAL 95]. Applications: [PET 00a]; – similarity transformation or local state isomorphism [WAL 95]. Applications: [JUL 97, JUL 98]; – study of the observability of the nonlinear system [CAS 85]. Applications: [BOU 98].

Identification of Bioprocess Models

49

In general, it will be difficult to determine simultaneously for both linear and nonlinear models, the approach that will require the minimum of calculations for a specific example. In most of the cases, we choose the methods of development in series because they are relatively simple to implement and in particular, because their development follows a continuous approach (we can always calculate the higher order terms of the series). As for the Taylor series method, they are generated in the time domain, whereas the generating series are developed in the input domain. These can be viewed as an extension of the Taylor series method in the case where we consider a class of inputs [RAK 85]. In the case of a model without an input variable, the generating series method becomes equivalent to the Taylor series method [VAJ 89]. 3.2.1. Development in Taylor series The approach using the development in Taylor series was initially developed by Pohjanpalo [POH 78]. Let us consider the general model below: ⎧ ⎪ ⎨ dx(t) = f (x(t), u(t), t, θ), x(0) = x0 (θ) dt (3.1) ⎪ ⎩y(t, θ) = g(x(t), θ)

Here, x, u, y and θ represent the state vector, input vector, output vector (measured variables) and parameter vector, respectively. The development in Taylor series of y(t) around t = 0 is equal to: y(t, θ) = y(0) + t

dy t2 dy 2 t k dk y (0) + (0) + · · · + (0), dt 2! dt k! dtk

k = 0, 1, . . . , ∞ (3.2)

This approach assumes that the variable y(t) and its successive derivatives are known (through the value y). Let us write the expressions of the successive derivatives of y(t) (3.2) obtained through calculation by model (3.1) as follows: dk y (0) = ak (θ) dtk

(3.3)

where ak (θ) are (typically nonlinear) functions of the parameters θ. Equation (3.3) is a system of nonlinear equations, where the unknowns are the parameters θ and where k the terms ddtky (0) are known from hypothesis. The method consists of solving this system of equations for a limited number of derivatives of y(t). It is evident that this number should at least be equal to the number of unknown parameters. A sufficient condition of structural identifiability is that there is a unique solution for this equation system.

50

Bioprocess Control

3.2.2. Generating series In the case of the generating series, we will consider the general model under the following form [WAL 95]: ⎧ m

⎪ ⎨ dx(t) = f 0 (x(t), θ) + ui (t)f i (x(t), θ), x(0) = x0 (θ) dt (3.4) i=1 ⎪ ⎩ y(t, θ) = g(x(t), θ) where the inputs ui (t) appear in the linear form. The generating series method is based on the output function g(x(t), θ) and its successive Lie derivatives, evaluated at t = 0: Lf i g(x(t), θ)t=0 =

n

f i,k (x, θ)

k=1

∂ g(x(t), θ)t=0 = si (θ) ∂xk

(3.5)

f i,k is the k th component of f i . We also see here a system of equations, where the unknowns are parameters θ (in the functions si (θ)). Therefore, a sufficient condition for structural identifiability will be that there is a unique solution for θ in the above system of equations, similar to the one in the Taylor series development (3.3). 3.2.3. Examples for the application of the methods of development in series The implementation of the two series development methods can follow the following steps: 1) calculation of the successive derivatives of g(x, θ); 2) selection of the set of θ parameters to be identified; 3) evaluation of the successive derivatives by inserting the known quantities and the lower order derivatives; 4) expression of the derivatives as functions of parameter combinations chosen in 2; 5) solution of the equations of step 4; 6) if the solution is unique, the parameters θ are structurally identifiable; otherwise, return to step 2. Let us apply the procedure to a model of the degradation of a substrate S in a batch reactor with Monod kinetics: dS μmax X S =− S(t = 0) = S0 (3.6) dt Y KS + S where the rate of respiration OUR(t) is measured. This is connected to the substrate S using the following equation: OUR = −(1 − Y )

S μmax X Y KS + S

(3.7)

Identification of Bioprocess Models

51

Let us calculate the derivatives of OUR(t) at t = 0: OUR(0) =

S0 μmax X(1 − Y ) Y KS + S 0

dOU R μ2 X 2 (1 − Y ) KS S0 (0) = − max 2 dt Y (KS + S0 )3 d2 OUR μ3max X 3 (1 − Y ) KS S0 (KS − 2S0 ) (0) = dt2 Y3 (KS + S0 )5

(3.8) (3.9) (3.10)

There are five parameters to be identified: Y , μmax , X, KS and S0 . If we write i OUR zi = d dt (0), i = 0, 1, 2, . . . , and consider the combinations of the following i parameters: θ1 =

μmax X(1 − Y ) , Y

θ2 = (1 − Y )S0 ,

θ3 = (1 − Y )KS

(3.11)

Equations (3.8), (3.9) and (3.10) can be rewritten in the following (equivalent) form: θ1 θ2 (3.12) z0 = θ2 + θ3 z1 = − z2 =

θ12 θ2 θ3 (θ2 + θ3 )3

θ13 θ2 θ3 (θ3 − 2θ2 ) (θ2 + θ3 )5

(3.13) (3.14)

Consequently, “parameters” θ1 , θ2 and θ3 can be (formally) determined from the values of zi (or otherwise the successive derivatives of OUR(t)) by reversing the expressions above: z0 (z0 z2 − 3z12 ) z0 z2 − z12

(3.15)

θ2 = −

2z02 z1 z0 z2 − 3z12

(3.16)

θ3 = −

(4z02 z13 ) (z0 z2 − 3z12 )(z0 z2 − z12 )

(3.17)

θ1 =

3.2.4. Some observations on the methods for testing structural identifiability From the above example, we can draw some interesting conclusions on the implementation of the methods for testing structural identifiability.

52

Bioprocess Control

The advantage of the development methods in series is that it enables us to follow a clearly identified procedure, which is not necessarily the case for the other methods. Having said this, it should also be noted that the maximum number of equations necessary for obtaining a result is unknown. In any case, generally, it is not certain that additional information will be brought in by adding derivatives of a higher order. This absence of an upper limit for the number of terms to be considered means that the result is only a sufficient (but not necessary) condition of identifiability. Furthermore, it will always be difficult to know the combinations of identifiable parameters beforehand. In any case, there is no systematic rule. Having said this, a good basis for an initial choice is frequently either good sense or the structure of terms ) ) in the successive developments. For example, the choice of θ1 (= − μmax X(1−Y Y appears to be obvious from equations (3.8), (3.9) and (3.10). Recently, Petersen [PET 00a] proposed a method, which enables us to suggest potentially identifiable combinations for a class of bioprocess models described by Monod kinetics. 3.3. Practical identifiability Practical identifiability is complementary to structural identifiability, in that we are now interested in the impact of the available experimental data (we are no longer considering ideal measurements!) concerned with the identifiability of the studied model’s parameters. A typical example of the distinction between the two concepts of identifiability is Monod’s model used in a process of simple microbial growth (a substrate, a biomass). We can show that this model is structurally identifiable [ABO 78], but in actual practice, this often turns out to be non-identifiable, given the low quality of available experimental data and their limited number (e.g. [HOL 82a]). This is manifested, for example, in the following form: any variation of a parameter (for example, μmax in the case of Monod’s model) can be almost perfectly compensated by a proportional variation of another parameter (KS in the case of Monod’s model) and yet can still reproduce a goodness of fit between the experimental data and the numerical predictions of the dynamic model. In addition, it may be possible that the algorithms employed for estimating the value of the parameters of a nonlinear model have bad convergence properties or they might turn out not to be well-conditioned numerically. In that case, the estimated values might be very sensitive to the conditions of estimation. We can then have parameter values that are susceptible to variation over large ranges, as a result of which the physical interpretation of their values becomes difficult. The main question is thus as follows: given certain experimental data, what precision can we obtain for the estimated values of parameters, or, in other words, what effect can a small variation in the values of parameters have on the good fit between model and data? 3.3.1. Theoretical framework Let us see how to formulate this question mathematically [MUN 91]. The parameter estimation can, in reality, be formulated as the minimization of the following

Identification of Bioprocess Models

53

quadratic criterion using an optimum choice of values for the parameters θ: J(θ) =

N

ˆ − yi )T Qi (yi (θ) ˆ − yi ) (yi (θ)

(3.18)

i=1

ˆ are the vectors of the N values measured and the predictions of the where yi and yi (θ) model at the instants ti (i = 1 to N ) and Qi is a weighted (square) matrix, respectively. The value of J obtained for a set of parameters slightly differing from the optimum values is given by the following expression [MUN 89]:  N  T   N

∂y

∂y T (t (t ) Q ) δθ + tr(Ci Ci−1 ) (3.19) E[J(θ + δθ)] ∼ δθ = i i i ∂θ ∂θ i=1 i=1 where Ci represents the matrix for the covariance of the measurement errors (Qi is often chosen to be equal to Ci−1 ). An important consequence of (3.19) is that for optimizing the practical identifiability (i.e., for maximizing the difference between J(θ + δθ) and Jopt (θ)), it will be necessary to maximize the term enclosed in the brackets [·]. This term is in reality the Fisher information matrix and expresses the information contained in the experimental data [LJU 87]: F =

T  ∂y (ti ) Qi (ti ) ∂θ ∂θ

N 

∂y i=1

(3.20)

This matrix is the inverse of the covariance matrix of estimation errors of the best unbiased linear estimator [GOD 85]:  N  T  −1

∂y ∂y −1 (ti ) Qi (ti ) (3.21) V =F = ∂θ ∂θ i=1 The terms ∂y ∂θ are the sensitivity functions of the output variables. They quantify the dependence of the model’s predictions with respect to the parameters. Approximation (3.19) of the objective function enables the tracing of the curves of the function for the values given in the space of the parameters. In the case of a model with two parameters (μmax and Ks in the case of Monod’s model, for example), these curves are ellipses. As underlined by Munack [MUN 89], the axes of the ellipses correspond to the characteristic vectors of the Fisher matrix and their length is proportional to the square root of the inverse of the corresponding eigenvalue. Thus, the ratio of the greatest eigenvalue to the smallest one (as absolute value) is a measure for the form of the function close to the optimum estimated values of the parameters. Figure 3.1 (left) shows a very much flattened ellipse corresponding to a very high ratio of eigenvalues, whereas in the figure on the right, the eigenvalues are closely packed.

54

Bioprocess Control

0.55 0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05

9 8 7 6 5 4 1

2

3

4

5

6

7

8

9 10 11

3 38

38.5

39

39.5

40

40.5

Figure 3.1. Contour trace of an objective function that is ill-conditioned (left) and well-conditioned (right) in the case of estimation of two parameters [PET 00a]

3.3.2. Confidence interval of the estimated parameters An important result of the practical identifiability is the possibility of calculating the parameter variance, which enables us to calculate a confidence interval for them. If covariance matrix V (3.21) is known and considering Qi to be the inverse of the measurement error covariance matrix we can calculate in an approximate manner the typical deviations for the parameters as follows:

(3.22) σ(θi ) = Vii The confidence intervals are thus obtained in the following manner: θ ± tα;N −p σ(θi )

(3.23)

for a confidence level of 100(1 − α)% and the values of t obtained from Student distributions. On applying equation (3.22), an error occurs because Vii contains the measurement error. In the case where a single measured variable is used for identification, a more realistic estimation of the confidence interval can be obtained by evaluating: s2 =

Jopt (θ) N −p

(3.24)

Here, p is the number of parameters of the model and Jopt (θ) is defined by equation (3.18) with Qi as the identity matrix. In fact, s2 calculated using (3.24) also contains the model error. The standard deviations can thus be calculated in an approximate manner as follows:

(3.25) σ(θi ) = s Vii

Identification of Bioprocess Models

55

One method of obtaining the exact confidence intervals for the nonlinear systems is to explore all the values of the objective function systematically for a very large number of parameter combinations. This is done at the cost of several calculations using a computer because the number of evaluations grows exponentially with the number of parameters [LOB 91]. In general, the confidence intervals are described by the hypervolumes limited by hypersurfaces in the space of parameters. In the particular case of a two-parameter model, this corresponds to a 2-dimensional domain in 3-dimensional space generated by (θ1 , θ2 , J(θ)). This enables us to see certain problems of parameter estimation (Figure 3.1). In this space, the objective function is a surface and the confidence interval is delimited by a curve. The confidence interval (1 − α) corresponds to the set of parameter values, for which the objective function is smaller than the limit value [BEA 60]:  p Fα;p,N −p Jopt ∗ 1 + (3.26) N −p where N and p are the number of measurements and parameters, respectively, and F is the value of the F -distribution with p and N − p degrees of freedom and a level of confidence α. In Figure 3.1, the contour curves are calculated according to this approach for increasing values of α. 3.3.3. Sensitivity functions As suggested above, the sensitivity functions play a key role in the evaluation of practical identifiability of parameters. An important property is that when the sensitivity functions are proportional, covariance matrix (3.21) is singular. The model is therefore not identifiable [ROB 85]. It turns out that in numerous cases of models used for biological processes, the sensitivity functions are often quasi-linearly dependent. This often gives rise to estimated values of strongly correlated parameters. This can be seen in Figure 3.1 (left) where the functional of the error is widely spread (as a “valley”), thereby indicating that more than one combination of parameters can describe the same data in an equivalent manner. Therefore, one means of studying the practical identifiability of a model involves tracing the sensitivity functions. There are numerous examples of such an approach available in the scientific literature, especially in the case of Monod’s model with biomass and substrate measurements [HOL 82a, HOL 82b, MAR 89, POS 90, ROB 85, VIA 85, PET 00c]. ∂y

Different approaches are possible for determining the sensitivity functions ∂θji . The most precise method is analytical derivation. However, for complex models it

56

Bioprocess Control

is necessary to make use of symbolic manipulation software. We can also resort to numerical calculations on the basis of the numerical values of parameters, which deviate slightly from the nominal values. In practice, if we consider a deviation Δθi for parameter θi , the sensitivity of the output variable yj to θi is calculated as follows: ∂yj yj (θi ) − yj (θi + Δθi ) = ∂θi Δθi

(3.27)

To illustrate, let us perform the calculation in the case of Monod’s model (3.6) using the measurements of OUR. The sensitivity of OUR with respect to μmax is equal to:   ∂ OUR ∂S ∂ dS d = − (1 − Y ) (3.28) = −(1 − Y ) ∂μmax ∂μmax dt dt ∂μmax is obtained through integration (with zero as In this equation, the sensitivity ∂μ∂S max initial condition) of the following differential equation:   d ∂S S ∂ μmax X = − dt ∂μmax ∂μmax Y KS + S (3.29)  μmax KS S ∂S X + =− Y KS + S (KS + S)2 ∂μmax where the substrate concentration S is calculated by integrating the dynamic equation of the substrate: μmax X S dS =− dt Y KS + S

(3.30)

The simultaneous solution of differential equations (3.29) and (3.30) enables us to obtain sensitivity functions (3.28). We can proceed in a similar manner for the sensitivity of OUR with respect to KS . We thus obtain the following relationships:  ∂ OUR ∂S d = −(1 − Y ) (3.31) ∂KS dt ∂KS   ∂S KS ∂S S d μmax X − =− (3.32) dt ∂KS Y (KS + S)2 ∂KS (KS + S)2 We can observe by analyzing the above relationships that the sensitivity functions depend on the value of the model parameters. This is a characteristic of the nonlinear models [DRA 81]. The Fisher matrix will therefore also be a function of the model parameter value: this will have consequences on the optimum design of the experiments (see below).

OUR

Substrate

0

5

10

15

20

25

30

0.250

5.0E−03

Km

0.125

2.5E−03

0.000

0.0E+00

−0.125

–2.5E+03

−0.250

–5.0E+03

μmax

−0.375

–7.5E+03

−0.500 0

Time (min)

5

10

15

20

57

dOUR/dμmax

35 30 25 20 15 10 5 0

dOUR/dKm

0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00

Substrate (mg/l)

OUR (mg O2/l.min)

Identification of Bioprocess Models

25

–1.0E+04 30

Time (min)

Figure 3.2. Sensitivity functions (right) for a profile of OUR with Monod’s model (left)

An example of a profile of OUR with the evolutions of the sensitivity functions is shown in Figure 3.2. It can be observed that the sensitivity functions for KS and μmax show an almost linear dependence. Intuitively, the sensitivity functions express the dependence of a variable on a change in the value of a parameter. In the example shown in Figure 3.2, these conditions of quasi-linearity are predominant while the substrate concentration has fallen to a value close to the affinity constant KS . It can be easily understood that a possible first approach for improving the information contained in the data is to choose the sampling instances corresponding to the zones where the parameters exhibit influence, i.e. when the sensitivity is high. 3.4. Optimum experiment design for parameter estimation (OED/PE) 3.4.1. Introduction One method for improving the quality of data for identification, with a view to obtaining the values of reliable parameters, is to plan the experiments, which serve for the collection of data in order to increase the information contained in these data for identification. This task is made particularly difficult in the context of bioprocesses, given the limited experimental resources both from the point of view of the time required for the experiment (this can turn out to be quite long and can extend to many weeks if not months) and the costs involved (the measurements can turn out to be quite cumbersome). The goals pursued by the experiment design in the context of identification may be as follows [VAN 98]: – reliable selection of structures for sufficient mathematical models for describing the process dynamics; – accurate estimation of the model parameters; – combination of the above two goals. Each of these objectives is associated with one or another quantitative function, which will determine the aspects on which the experiments are to be focused. In this

58

Bioprocess Control

chapter, we will focus on the experiment design in order to estimate the parameters (second objective). If the reader is interested, they can refer to the chapter in [VAN 98] on aspects of experiment design for the other two goals. Experiment design typically comprises three steps: 1) definition of an objective function of the OED, which is the mathematical translation of the goal pursued by the experiment design; 2) inventory of the degrees of freedom that are available and the experimental constraints; 3) determination of the optimum for the objective function by playing in a suitable manner with the degrees of freedom while at the same time ensuring that the constraints are respected. Basically, the algorithm which carries out the experiment design does so through numerical simulation of experiments so as to quantify the potential effects of the proposed experimental conditions on the objective function. On the basis of the results given by an algorithm of optimization that is capable of taking into account the experimental constraints, we can propose an optimum experiment, which will be subsequently implemented. As the experiment design is largely dependent on the process model, it is evident that a prerequisite for good planning is the existence of a sufficiently reliable process model. The following questions can serve as a guide before the study of the experiment design is implemented. What shall we measure? This question highlights the importance of a good definition of the system (choice of input, output and state variables, etc.). Where shall we measure? This is the problem of sensor location. When shall we measure? Here we are concerned with the choice of sampling time(s). What types of manipulations are involved? This is often the most important question. Once the degrees of freedom are determined, we still have to choose the type of signal to be implemented. This often requires imagination and creativity for designing the signals, which will act on the procedure in such a manner that we can obtain high-quality information. As examples of possible signals, we can have periodic (sinusoidal) signals, pulses, pseudo-random binary signals (PRBS) [LUB 91], etc. What about the treatment of data? Even though most of the time not considered to be an integral part of experiment design, the pretreatment of data is often applied

Identification of Bioprocess Models

59

before identifying the model. Typically, this includes the elimination of measurement noise, the detection and elimination of anomalous data and the elimination of dynamics of no interest. The quality of a set of estimated parameter values can be evaluated in diverse ways, for example on the basis of the quality of predictions given by the model. Nevertheless, in most cases, we characterize the quality of parameter estimation on the basis of the estimation errors (confidence intervals as given above (3.23)) or, more generally, on the basis of covariance matrix (3.21). This appears from now on to be clearly linked to the practical identifiability. If the objective of experiment design is to improve the estimation of parameters, it is evident that covariance matrix (3.21) or its elements, or its inverse (Fisher matrix) will constitute a main element thereof. It should be noted that other methods can also turn out to be equally appropriate for treating the experiment design. For example, it is possible that the transformation of the model into a numerically more robust equivalent form would result in a more reliable estimation [RAT 86] (see also the case study at the end of the chapter). We can also think of using prior information (such as the knowledge of the maximum growth rate) in order to impose bounds on the values of parameters and improve in this manner the identifiability of other parameters [MUN 89]. 3.4.2. Theoretical basis for the OED/PE First, let us recall that the variable that describes the reliability of the estimated value of a parameter, i.e. the standard deviation of the estimation error of the parameter, is given by the following relationship (3.22) in the case of a single measured output variable:

σ(θi ) = s Vii (3.33) We can thus see that two terms can be manipulated in order to increase the precision of the estimation of the parameter. The first term (square of the average residue s2 ) is calculated from the sum of errors raised to the power of two between N predictions of the model and N experimental data, Jopt (3.24): s2 =

Jopt (θ) N −p

(3.34)

It is possible to reduce this value by increasing the amount of experimental data N , for example by repeating the experiments. This is particularly useful when p is not negligible with respect to N . When N is very large, the increase in the value of the denominator will be proportional to the increase in the value of the objective function Jopt which is typically a sum N of errors, raised to the power of two.

60

Bioprocess Control

Alternatively, we can reduce the covariance of the estimation error V (3.21). We will now concentrate on the methods, which enable the reduction of V . Different approaches have been developed for planning experiments on the basis of covariance matrix V , or equivalently, on the basis of Fisher information matrix F , via the optimization of certain scalar measurements of these matrices [MUN 91]: – A – optimal design criterion: min[tr(F −1 )] – Modified A – optimal design criterion: max[tr(F )] – D – optimal design criterion: max[det(F )] – E – optimal design criterion: max[λmin (F )]  (F )  – Modified E – optimal design criterion: min λλmax min (F ) where λmin (F ) and λmax (F ) are the lowest and the highest eigenvalues of the Fisher information matrix, respectively. These different criteria can be interpreted as follows [MUN 91]. Design criteria A and D minimize the arithmetic and geometric means of the identification errors, respectively, whereas criterion E enables the minimization of the largest error. For the two E criteria, the optimization is carried out over the eigenvalues of the Fisher matrix: this ensures the maximization of the distance with respect to the singular (non-informative) case. The modified E criterion should be interpreted on the basis of the objective function’s form. The ratio of the highest eigenvalue to the lowest eigenvalue is an indication of this form. As the aim is to have eigenvalues that are as closely packed to one another as possible, it amounts to having a circle (even so it must be noted that the criterion is not directed at the objective function, i.e. the volume of the confidence interval; in other words, optimizing the modified E criterion can very well lead to a rounder but wider confidence interval). When λmin (F ) is equal to zero, this ratio equals infinity; i.e., there are infinite combinations of parameters, which enable us to describe the experimental data and from then on the experiment becomes non-informative. The Fisher matrix is thus singular. In that case, criteria D and E are equal to zero, while it is impossible to determine the A criterion as the inversion of F is impossible. This case enables us to also highlight problems, which can arise with the modified A criterion: even in the case of a non-informative and thus a non-identifiable experiment, the modified A criterion can always be maximized because one of the other eigenvalues can be very high [GOO 87]. Furthermore, it should be remembered that the elements of the Fisher information matrix depend on the units of the parameters. As a result, the eigenvalues of the Fischer matrix depend on the units; this can therefore form the subject matter of resetting the scale. Further, it is possible that an experiment that is optimum for a certain set of units is not optimum for another. This is true for each of the above criteria, except for the D criterion ([PET 00a]), even though the absolute value of this criterion varies for different units of parameters, the optimum experiment remains the same. Only the optimum experiment generated by the D criterion is independent of the units selected. On the other hand, this dependence of the Fisher information matrix with respect to

Identification of Bioprocess Models

61

the units can be used favorably in the case where numerical problems crop up during its inversion. In that case, we can modify the condition number (which is equivalent to the modified E criterion) by simply changing the units and thus retain the reliability of the matrix inversion. This can turn out to be of special interest for a certain number of numerical optimization algorithms used for parameter estimation (see below). Finally, let us mention that other criteria can be proposed, for example the parameter estimation error can be reduced by using the variance of this parameter as the criterion. 3.4.3. Examples Let us now examine some examples of experiment design for parameter estimation. Vialas et al. [VIA 85] proposed a more frequent sampling for certain defined experimental periods. Holmberg [HOL 82a] showed that the identifiability of the parameters for Monod’s model depends very much, for the batch experiments, on the initial value of the substrate concentration; according to Holmberg the initial optimum substrate concentration depends on the noise level and the sampling instant. It is also evident on the basis of her results that the experiment design depends on the value of the parameters, which might induce a modification in the experiment design if we have to take into account the changing character of the process in the course of time. Munack [MUN 89] proposed different modifications of batch experiments and showed that considerable improvements could be achieved in the confidence intervals by means of the experiment design techniques. Vanrolleghem et al. (1995) highlighted the improvement in the accuracy of estimation for the two parameters of a biodegradation model with Monod kinetics. The addition of a substrate pulse (Figure 3.3) enabled the reduction of the confidence interval by 25%. This diminution in the variance was distributed equivalently between the two parameters μmax and KS , which is a typical result of an OED based on the D criterion. A similar study was carried out by [PET 00a] for a real application, where a model for the treatment of wastewater had to be identified. The model contained two submodels, one for the oxidation of nitrogen (nitrification) and the other for the oxidation of carbon. In this case also, the accuracy was improved by adding to the wastewater sample a determined quantity of ammonia. The confidence interval was reduced by 20% for the carbon oxidation submodel and by 50% for the nitrification submodel. Baetens et al. (2000) identified six parameters in a biological phosphate removal process. The experiment design was evaluated by considering more than one degree of freedom. The D criterion could be improved by a factor of at least four. This corresponds to an improvement in the confidence intervals in the estimation of parameters by a factor of two. The evolution of the modified E criterion for changing experimental conditions showed that the longest axis of the confidence ellipsoid could be reduced by a factor of two, as the increase in the modified E criterion by a factor of four could be obtained by increasing the acetate dosage.

Bioprocess Control 0.70

0.70

0.60

0.60

OUR (mg O2/l.min)

OUR (mg O2/l.min)

62

0.50 0.40 0.30 0.20 0.10 0.00

0.50 0.40 0.30 0.20 0.10 0.00

0

5

10

15 Time (min)

20

25

30

0

5

10

15

20

25

30

Time (min)

Figure 3.3. Experimental respirograms obtained for a batch experiment and for a fed-batch experiment with an additional pulse after 14.6 minutes. The parameter variances are 50% less for the fed-batch experiment

Another approach was developed by Petersen et al. [PET 00c] related to the degrees of freedom for the experiment design. The differences in the accuracy of parameter estimation were evaluated when different sets of measured variables were used for identifying the parameters of a nitrification model. The following possibilities were evaluated: – a single measurement of dissolved oxygen SO in an aerated batch reactor; – two measurements of dissolved oxygen SO at the inlet and at the outlet of the respiration chamber; – the respiration rate OURex calculated from these two measurements of oxygen; – the rate of proton production Hp obtained from the pH regulator; – two measurements of dissolved oxygen SO and the rate of proton production Hp . Despite the fact that in this case the mass-transfer parameters KL a and SO,sat should be estimated (in addition to the three nitrification parameters μmax , KN H , and SN H (0)), the estimations of nitrification parameters can be more accurate with the measurements of dissolved oxygen. The confidence intervals are then 10 times smaller (the variances are 100 times smaller). The high(er) level of noise in the respiration rate measurements undoubtedly provides a partial explanation for this phenomenon. This result highlights the importance of an appropriate choice of the measured variables used for the estimation of parameters. Munack [MUN 91] has also shown that the effect of different measurement configurations could lead to datasets having information contents that are significantly different. In his analysis, not only the type of data but also the amount and their position in the core of the reactor play a part. Versyck et al. [VER 97] highlighted a remarkable result with regard to experiment design, in liaison with optimum control: the profile of optimum control for a

Identification of Bioprocess Models

63

growth model with Haldane kinetics constitutes an excellent starting point for finding an experiment for which the modified E criterion is optimum (equal to 1). In addition, it is shown in [VER 00] that, in the case of modeling of the temperature’s effect on the kinetics, the uniqueness of the modified E criterion can be attained using an appropriate temperature profile. The authors also show that more than one experiment design enables us to attain this optimum and that it is therefore possible to take into account other selection of criteria, such as practical feasibility or validity of the model (see also [BAL 94]). 3.5. Estimation algorithms Until now we have been concentrating on analyzing the identifiability of parameters and the design of experiments for parameter estimation. Now we move on to the next question. We have constructed and/or selected a model and we have a set of appropriate data for the estimation (obtained ideally after experiment design): how are we going to proceed in order to estimate the values of the parameters of this model? As this chapter gives only an overview of parameter identification, we will not deal with the algorithms used for parameter estimation in a detailed manner. In any case, it could hardly happen that the readers would be asked to develop the estimation algorithm of these parameters by themselves; they would more than certainly have to resort to software available on the market (of which Matlab and its tool boxes are typical examples). We would like to suggest to readers with an interest in the subject to refer to specialist papers, such as [WAL 94]. 3.5.1. Choice of two datasets Once the experimental data are available and before passing on to the estimation of the model parameters, the first essential and fundamental stage is the separation of the data into two distinct datasets: 1) a first set of data used for calibrating the parameters; 2) the remaining data for validating the estimation. The first dataset will be employed for calculating the parameter estimations using an appropriate estimation algorithm. The second dataset will be used for verifying that these parameters reproduce the process dynamics well. The independence between these two datasets is fundamental for validating the identification procedure (while it can be expected that the model obtained on the basis of certain data would reproduce these data, and it will be very important to know whether this model reproduces other data, which have not been used for calibrating its parameters). The separation of the data should be performed in such a manner that the first dataset (calibration) be sufficiently informative and thus covers a sufficient spectrum of experimental conditions (in accordance with an experiment schedule, for example). The second dataset should

64

Bioprocess Control

contain sufficient data in order to render the validation as credible as possible. We will avoid having datasets that are very unbalanced in terms of the number of data, even though generally the first set of data (calibration) will have more data than the second set (validation). 3.5.2. Elements of parameter estimation: least squares estimation in the linear case In section 3.3, we stated that the estimation of parameters consisted of basically optimizing a certain error criterion, typically of prediction, such as the objective function (3.18). The concrete form taken by the estimation algorithm will depend largely on the model, especially depending on whether or not the model is linear in the parameters. For example, in the linear case with one equation, the model will be generically written in the following form: yi = φTi θ

(3.35)

where θ is the parameter vector, φi is called the “regressor”, yi contains the independent terms of parameters, and index i corresponds to the temporal index. The choice of a representation of the model in discrete time is natural insofar as the estimation is based on the experimental data themselves available in the form sampled on schedule. To illustrate this, let us consider the balance equation of a substrate S involved in two reactions: growth reaction (where additionally it is assumed that the dependence of the kinetics with respect to the substrate concentration is of the first order) and maintenance reaction. This equation is written as follows: dS = −k1 αSX − km X + DSin − DS dt

(3.36)

Let us assume that neither α (kinetic growth constant) nor km (maintenance coefficient) are known and that we have the data for the concentration of substrate S and biomass X as well as for dilution rate D and of inlet substrate concentration in the feed Sin at time intervals T (i.e., following a sampling period T ). Let us also assume that the yield coefficient k1 is known. Vector θ is thus equal to:  α θ= (3.37) km Si+1 −Si By approximating the time derivative dS (where dt with a finite difference T index i indicates the value at instant t = iT ), we can rewrite balance equation (3.36) as follows:

Si+1 − Si − T Di Sin,i + T Di Si = −k1 Si Xi T α + Xi T km

(3.38)

Identification of Bioprocess Models

65

In other words, in the formalism of equation (3.35), yi and φi are equal to:  −k1 Si Xi T yi = Si+1 − Si − T Di Sin,i + T Di Si , φi = (3.39) Xi T In the simple case of the linear equation in parameters θ, the linear regression techniques, for example an estimator using least squares, can be used. In that case, we consider objective function (3.18) in the following form: J(θ) =

N

ˆ − yi )2 βi (yi (θ)

(3.40)

i=1

The least squares estimator, which minimizes this criterion is written in simple terms as: −1 N  N

θˆ = βi φi φTi βi φi yi (3.41) i=1

i=1

When βi in the objective function is not equal to 1, we mean a weighted least squares estimator. A possible choice for βi is to take the inverse of the variance of the measurement yi . 3.5.3. Overview of the parameter estimation algorithms In general, the parameter estimation follows the logic described above for the simple case of the monovariable linear model. It is based on the minimization of a criterion, whose typical model is shown above (3.18)1 J(θ) =

N

ˆ − yi )T Qi (yi (θ) ˆ − yi ) (yi (θ)

(3.42)

i=1

The weights Qi can be different at each sampling instance, which can turn out to be of interest in the case of a measurement quality, which varies with time. A typical choice for matrix Qi consists of taking the inverse of the covariance matrix of measurement errors (as suggested above in the monovariable case).

1. However, use is sometimes made of other objective functions, based for example on the minimization of the absolute value of the prediction error, on the minimization of the maximum absolute error, or on the maximization of the number of changes of sign in the residue sequence.

66

Bioprocess Control

Before reviewing the minimization methods, which enable the determination of the values of the selected objective function’s parameters, we can say that in general the problems encountered during parameter estimation using the least-squares type methods are due to three kinds of phenomena. First, it is possible that the residues may not be distributed normally, contrary to the assumptions on the basis of conception of this type of algorithm. Then, for giving correct weighting to the residues, it will be necessary to know the value of the variance of the error at each measurement point. This can be performed precisely only when there is a sufficient number of repeated measurements at a point, which is rarely the case. Finally, the outliers can lead to an inaccurate estimation of the parameters as well as to biased confidence intervals. The search for an optimum for a multivariable function is a common problem in numerous fields of research, and there is substantial expertise on this subject. However, attention should be drawn to the fact that the minimum (the set of parameters minimizing the criterion) can either be local or global (see Figure 3.4): in the first case, it corresponds to a minimum in a region limited in the space, whereas in the second case, it corresponds to the minimum in all of the parameter space. Despite considerable efforts, as of today there is no minimization algorithm, which could ensure the determination of the global minimum for any problem of nonlinear optimization. We see in particular that the convergence toward a (local or global) minimum depends ˆ very much on the initial values selected for the parameters θ(0). We can easily understand this phenomenon on the basis of curve 3.4; by taking the initial values to the left of the local minimum, it is probable that the minimization algorithm will be fixed at the local minimum and will not attain the global minimum, which is attained by starting from the right side of the same. In order to avoid convergence only to a local minimum, it might turn out to be important to select a sufficient number of iterations of the minimization algorithm by taking the sets of initial parameter values that are sufficiently different in order to cover as wide a spectrum of values as possible.

Local minimum Global minimum

Parameter Figure 3.4. Local and global minima for an objective function

Identification of Bioprocess Models

67

Schuetze [SCH 98] tried to classify the diverse minimization methods into two groups: the local minimization methods and those of global minimization, even if there is something artificial about this insofar as all the so-called global methods bring into play a local procedure at a certain level. Local minimization There are a large number of procedures for local minimization. Among them, many are based on the derivatives of the objective function to be minimized, information that is thus either considered directly available or numerically calculated. The basic idea of these so-called gradient methods is that a search direction for the minimum is sought in the parameter space. Once this direction is selected, the algorithm chooses the amplitude with which the values of parameters are modified. If we can show the convergence of the method, the convergence can turn out to be very slow [DRA 81]. An alternative involves the use of Newton’s method. Even though it enables a possibly quicker convergence, the disadvantage of the method is that it can diverge or oscillate around the solution. The modification that is probably the best known among the basic methods is the Levenberg-Marquardt algorithm [MAR 63]. It is a compromise between the gradient method and Newton’s method. A suitable choice of the algorithm parameters enables us to combine the advantages of the gradient method (convergence properties) with those of Newton’s method (rapid convergence). In order to improve the convergence, we can make use of the fact that the majority of functions are reasonably well approximated by a quadratic function close to the optimum. The BFGS methods (Broyden, Fletcher, Goldfarb and Shanno) [FLE 87] is probably the best known of this class of methods. [VAN 96] has, however, noted that this type of method, if they converge very quickly, can be sensitive to the local minima. One local minimization method that is not based on the calculation or approximation of the derivatives is the Powell method, with the improvements proposed by Brent [PRE 86, GEG 92]. It is based on a repeated combination of a one-dimensional search along a set of many directions. This probably involves one of the best methods in terms of compromise between convergence speed and insensitivity to local minima. Another well-known local minimization method, which does not require information about the derivatives, was proposed by Nelder and Mead; it is better known as the Simplex method (not to be confused with the simplex method in linear programming!). A simplex is a set of (p + 1) parameter vectors in the parameter space. The method consists of a succession of comparisons of the objective function with these edges, followed by a replacement of the edges having the highest value by another point in the space of the parameters of dimension p. The method is well appreciated

68

Bioprocess Control

for its robustness with respect to the local minima, its ease of implementation and its reasonable convergence speed [PRE 86, SCH 98, VAN 96]. Global minimization The global minimization methods can be roughly classified into two groups [SCH 98]. The first group comprises purely deterministic methods, such as the gridding method. It consists of evaluating the objective function for a large number of points predefined over a grid covering the parameter space. If there is a sufficient number of function evaluations, there are chances of attaining the minima. This method is not very effective, unless it is improved by refining the grid after a series of evaluations. The second group of global methods can be called random probing methods, as random decisions are included in the procedure for attaining the optimum. Among them, the adaptive methods take into account the information obtained during the preceding evaluations. For one method (simulated annealing [PRE 86]), the idea is that the search will not always be toward a possible solution (which could just be a local minimum), but may be, from time to time, along another direction. This method can be viewed as preceding the popular methods such as genetic algorithms (GA) [GOL 89]. These algorithms commence with an initial population of prospective solutions (somewhat similar to the edges in the Simplex method) sampled in a random manner in the parameter space. In genetic algorithms, new prospective solutions are obtained by imitating the process of biological evolution of cross breeding, mutation and selection among the parameter “populations”. The definition of the parameters of the algorithm itself is crucial for correct implementation. 3.6. A case study: identification of parameters for a process modeled for anaerobic digestion The anaerobic digestion model (2.16)–(2.29) formed the subject matter for a systematic study to identify parameters on the basis of a fixed bed reactor of LBE - INRA at Narbonne (see [BER 01, BER 00] for a more detailed study). This case study is remarkable in view of many aspects. Firstly, the identification of parameters is riddled with traps that are typical for biological systems: the process is very complex (numerous bacterial populations participate in the process; they can have different behaviors depending on the operating conditions). There are no direct measurements for each of the acidogenic and methanogenic bacterial populations and in general, a limited number of process variables are accessible. The process is slow and can be destabilized easily by the accumulation of fatty acids. These characteristics have significant consequences for the selected model. The model cannot be very complex lest it should turn out to be non-identifiable, whereas the simplified modeling hypotheses can have an impact on its capacity to predict the dynamics of the process. The choice of the structure is therefore a critical stage in view of the fact that the model should necessarily contain elements that are essential for the process dynamics.

Identification of Bioprocess Models

69

Moreover, the identification itself contains original elements with respect to the rest of the chapter. Constraints on the process have led us to conceive an experimental plan, not on the basis of the above techniques but motivated by concern for covering as wide a range of operating conditions as possible, while at the same time limiting the duration (here necessarily long) of the experiments as much as possible. In addition, the structure of the reaction system models enables us to separate the parameters into three classes (yield coefficients, kinetic parameters and transfer parameters) and to carry out the identification of each class of parameters in a distinct manner. In this case study, emphasis is laid on parameter estimation using linear regression. Having said this, it will be prudent to draw attention to the fact that this approach cannot turn out to be optimum from a statistical point of view insofar as the statistical conditions on the variables used in the linear regression could not be completely fulfilled in order to enable an estimation that might be statistically accurate and sufficiently reliable. In general, care must be taken during the interpretation of the estimated values of parameters and their standard deviation. Furthermore, these parameter values given by linear regression were suggested to be used as initial values of an estimation on the basis of the nonlinear model. However, such an estimation has not turned out to be useful in the present case. 3.6.1. The model Let us once again proceed from the equations developed in Chapter 2. A variant of these was considered as here we are dealing with a fixed bed reactor, where the bacteria are fixed on supports. Thus, formally there is no dilution term in the balance equations. However, on the other hand, it was noted that a portion of the biomass was detached: it was hypothesized that the rate of detachment was proportional to the rate of dilution, represented by coefficient α. Under these conditions, the model is rewritten as follows: dX1 dt dX2 dt dS1 dt dS2 dt dZ dt dC dt

= μ1 X1 − αDX1

(3.43)

= μ2 X2 − αDX2

(3.44)

= D(S1in − S1 ) − k1 μ1 X1

(3.45)

= D(S2in − S2 ) + k2 μ1 X1 − k3 μ2 X2

(3.46)

= D(Zin − Z)

(3.47)

= D(Cin − C) − qC + k4 μ1 X1 + k5 μ2 X2

(3.48)

70

Bioprocess Control

qC = kL a(C − S2 − Z − KH PC )

φ − φ2 − 4KH PT (C + S2 − Z) PC = 2KH φ = C + S2 − Z + KH P T +

k6 μ2 X2 kL a

qM = k6 μ2 X2  C − Z + S2 pH = −log10 Kb Z − S2

(3.49) (3.50) (3.51) (3.52) (3.53)

In addition, growth models were chosen, one Monod model for acidogenesis and one Haldane model for methanization: μmax 1 S1 μ0 S2 , μ2 = (3.54) μ1 = KS1 + S1 KS2 + S2 + S22 /KI2 In the absence of systematic rule, this choice was dictated by a desire to have kinetic models that are sufficiently simple, coherent with those normally used for anaerobic digestion, but also capable of highlighting the potential instability of the process in the presence of the accumulation of volatile fatty acids (explaining the choice of the Haldane model for μ2 ). Knowing that Kb and KH are known chemical and physical constants (Kb = 6.5 10−7 mol/l, KH = 16 mmol/l/atm), we note that the model contains 13 parameters to be identified. In addition, the variables available for the measurement are the dilution rate D, the inlet concentrations, S1in , S2in , Zin , Cin , the gas flow rates qC , qM , the concentrations S1 , S2 , Z, C, and the pH. 3.6.2. Experiment design Given the complexity of the model and the large number of parameters as well as the experimental constraints (long time constants and potential instability of the process), the strategy followed here for experiment design consisted of covering a number of operating points sufficiently representative of how the process works. The experiment design is given in Table 3.1. 3.6.3. Choice of data for calibration and validation In this case, one of the priorities was to obtain a model, which would be capable of correctly reproducing as a priority the equilibrium conditions. This is why the data collected were separated into two categories: the data corresponding to the steadystate conditions for calibrating the parameters and the dynamic data for the validation. The periods corresponding to the values used for the calibration are represented in Figures 3.5 and 3.6 by a bold line on the abscissa.

Identification of Bioprocess Models

D(day−1 )

S1in (g/l)

S2in (mmole/l)

pH

0.34

9.5

93.6

5.12

73.68

4.46

38.06

4.49

0.35

10

0.35

4.8

0.36

15.6

0.26

10.6

72.98

4.42

0.51

10.7

71.6

4.47

0.53

9.1

68.78

5.30

112.7

71

4.42

Table 3.1. Characteristics of average feed conditions

250

150

Qch4 (l/h)

Qco2 (l/h)

200 100

150 100

50 50 0

0 10

20

30

40

50

60

70

10

20

30

40

50

60

70

10

20

30 40 50 Time (days)

60

70

7.5

350 300

7 6.5

200

pH

Qtot (l/h)

250

150

6

100 5.5

50 0

10

20

30 40 50 Time (days)

60

70

5

Figure 3.5. Simulated values (fine line) and data values (thick line) of gas flow and pH. The periods used for the calibration are represented in bold on the abscissa

3.6.4. Parameter identification With such a complex model, we can expect to encounter problems of structural and practical identifiability. We expect to be in a position to alleviate, at least partially,

72

Bioprocess Control 10

40

9

35

8

30 VFA (mmol/l)

COD g/l

7 6 5 4 3

20 15 10

2

5

1

0

0 10

20

30

40

50

60

70

80

70

70

60

60

50 C (mmol/l)

Z (mmol/l)

25

50 40 30

10

20

30

40

50

60

70

10

20

30

40

50

60

70

40 30 20

20

10

10

0

0 10

20

30

40

Time (days)

50

60

70

Time (days)

Figure 3.6. Simulated values (continuous line) and data values (o) of S1 , S2 , Z and C. The periods used for the calibration are represented in bold on the abscissa

the problems of practical identifiability, thanks to experiment design. However, what about the problems of structural identifiability? Without prejudging the results of an analysis, which could turn out to be complex, it seems to be interesting to take advantage of the structure of the model and its properties for formulating the problem of identification of parameters in a different form. We will see in the next chapter (state observers) that there is a state transformation (4.42)2 which enables us to rewrite a portion of the model in a form independent of the kinetics. This transformation is doubly interesting in our case. On the one hand, it enables us to proceed with the identification of kinetic and other model parameters in a distinct manner. On the other hand, this separation is of particular interest insofar as the kinetic modeling is its weakest link, given that we lack the physical base for the choice of an appropriate model and in addition, it is the chief source of nonlinearities in the model.

2. This transformation is presented in detail and in a more general form in [BAS 90].

Identification of Bioprocess Models

73

It is basically this approach that was adopted here. As the identification is carried out on the basis of the steady-state data, static balance equations of biomasses X1 and X2 and the expressions for the specific growth rates, we derive the following expressions: α 1 α 1 = + KS1 ¯ D μ1 max μ1 max S1

(3.55)

1 α 1 α 1 α ¯ S2 = + KS2 + ¯ D μ0 μ0 S2 KI2 μ0

(3.56)

These equations are linear in the parameters μ1 αmax , KS1 μ1 αmax , μα0 , KS2 μα0 and 1 α KI2 μ0 . They can be determined through linear regression. The only problem is that it is not possible to distinguish between μ1 max and α; this is why we have considered a value from the literature ([GHO 74]) for μ1 max . This choice turns out to be even more acceptable, and a sensitivity analysis shows a low sensitivity of μ1 max . We can thus use the equation for the CO2 flow rate for determining kL a, by recalling that qC = kL a(CO2 − KH PC ) and that the reaction for the dissociation of bicarbonate enables the linking of CO2 to C and to pH according to the relationship: CO2 = C

1 1 + Kb 10pH

(3.57)

This enables us to determine the kinetic parameters and the transfer parameters independently of the performance coefficients. The results are summarized in Table 3.2. Parameter

Units

Value

Standard deviation

μ1 max

textDay −1

1.2

(1)

KS1

g/l

7.1

5.0

0.74

0.9

9.28

13.7

μ0

textDay

−1

KS2

mmol

KI2

mmol

α

/

0.5

0.4

kL a

Day−1

19.8

3.5

16

17.9

Table 3.2. Estimated values of kinetic and transfer parameters. (1) taken from [GHO 74]

74

Bioprocess Control

The values of the yield coefficients now have to be calculated. The main difficulty results from the absence of measurements of populations X1 and X2 . Without these measurements, it turns out that the yield coefficients are not structurally identifiable. This can be shown by considering, for example, a rescaling for X1 and X2 : X1 = λ1 X1 , X2 = λ2 X2 . This rescaling can be compensated by putting the yield coefficients in scale as follows k1 =

k1  k2  k4  k3  k5  k6 , k2 = , k4 = , k3 = , k5 = , k6 = λ1 λ1 λ1 λ2 λ2 λ2

(3.58)

In order to overcome the difficulty, the identification is carried out in two stages: first, the estimation of the structurally identifiable ratios of the yield coefficients (k2 /k1 , k6 /k3 , k5 /k3 , k4 /k1 ); second, the use of the data of volatile suspended solids (VSS) in order to determine the values of each performance coefficient. The estimation of the ratios of performance coefficients is performed on the basis of balance equations in steady state, which can be rewritten as follows after eliminating X1 and X2 :  k4 k2 k5 k5 qC = D(Cin − C) + + D(S1in − S1 ) + D(S2in − S2 ) k1 k1 k3 k3 qM =

k2 k6 k6 D(S1in − S1 ) + D(S2in − S2 ) k1 k3 k3

The ratios kk41 + kk21 kk53 , kk53 , kk21 kk63 , kk63 can thus be determined using linear regression. The result of the calibration is presented in Table 3.3. Ratio

Units

Value

Standard deviation

k2 /k1

mmol/g

2.72

2.16

k6 /k3

/

1.62

0.12

k5 /k3

/

1.28

0.13

k4 /k1

mmol/g

1.18

3.02

Table 3.3. Estimated values of the ratios of performance coefficients

The determination of the coefficients is thus made on the basis of VSS measurements. In fact, the VSS represents an approximate indicator of X1 + X2 . The determination of the VSS distribution between the two bacterial populations has been performed by considering a ratio (= 0.2) between the acidogenic bacteria X1 and the total biomass X1 + X2 taken from [SAN 94]. The results are presented in Table 3.4.

Identification of Bioprocess Models

Parameter

Units

Value

Standard deviation

k1

g S1 /g X1

k2

mmol S2 /g X1

116.5

k3

mmol S2 /g X2

268

k4

mmol CO2 /g X1

50.6

143.6

k5

mmol CO2 /g X2

343.6

75.8

k6

mmol CH4 /g X2

453.0

90.9

42.14

75

18.94 113.6 52.31

Table 3.4. Estimated values of performance coefficients

3.6.5. Analysis of the results The first essential stage after the calibration of parameters is their validation for the set of data not used for the calibration. This can be seen in Figures 3.5 and 3.6. We can generally see a good fit between the model and the transient experimental data. This demonstrates the interest in a systematic approach for the identification of parameters, all the more in a case as difficult and complex as this one. However, the reliability of the estimated values of the parameters remains to be evaluated. This corresponds to the standard deviation of each parameter shown in Tables 3.2, 3.3 and 3.4. The increase in this value should be noted while converting the ratios of the performance coefficient to their individual values. This reflects the large uncertainty associated with the determination of biomass concentrations on the basis of VSS data and the assumed ratio of their distribution. We will also note the rather high uncertainty of kinetic parameters, which is not at all surprising, given the fact that the chosen structures are heuristic. Finally, in the ratios of performance coefficients, a worse reliability is found for the parameters involving k1 , which is probably an effect of the variable nature of the concentration of feed S1in . 3.7. Bibliography [ABO 78] S. A BORHEY and D. W ILLIAMSON, “State and parameter estimation of microbial growth processes”, Automatica, vol. 14, pp. 493–498, 1978. [BAL 94] M. BALTES, R. S CHNEIDER, C. S TURM and M. R EUSS, “Optimal experimental design for parameter estimation in unstructured growth models”, Biotechnol. Prog., vol. 10, pp. 480–488, 1994. [BAS 90] G. BASTIN and D. D OCHAIN, On-line Estimation and Adaptive Control of Bioreactors, Elsevier, Amsterdam, 1990.

76

Bioprocess Control

[BEA 60] E.M.L. B EALE, “Confidence regions in non-linear estimation”, J. Roy. Statist. Soc. Ser. B, vol. 22, pp. 41–88, 1960. [BER 00] O. B ERNARD, Z. H ADJ -S ADOK and D. D OCHAIN, “Advanced monitoring and control of anaerobic treatment plants: II - Dynamical model development and identification”, in Proc. Watermatex 2000, pp. 3.57–3.64, 2000. [BER 01] O. B ERNARD, Z. H ADJ -S ADOK, D. D OCHAIN, A. G ENOVESI and J.P H . S TEYER, “Dynamical model development and parameter identification for an anaerobic wastewater treatment process”, Biotech. Bioeng., vol. 75, pp. 424–438, 2001. [BOU 98] S.V. B OURREL, J.P. BABARY, S. J ULIEN, M.T. N IHTILÄ and D. D OCHAIN, “Modelling and identification of a fixed-bed denitrification bioreactor”, System Analysis Modelling Simulation (SAMS), vol. 30, pp. 289–309, 1998. [CAS 85] J.L. C ASTI, Nonlinear System Theory, Mathematics in Science and Engineering, vol. 175, Academic Press, Orlando, 1985. [DOC 95] D. D OCHAIN, P.A. VANROLLEGHEM and M. VAN DAELE, “Structural identifiability of biokinetic models of activated sludge respiration”, Wat. Res., vol. 29, pp. 2571–2579, 1995. [DRA 81] N.R. D RAPER and H. S MITH, Applied Regression Analysis, Wiley, New York, 1981. [FLE 87] R. F LETCHER, Practical Methods of Optimization, Wiley, New York, 1987. [GEG 92] K.R. G EGENFURTNER, “PRAXIS: Brent’s algorithm for function minimization”, Behaviour Research Methods, Instruments and Computers, vol. 24, pp. 560–564, 1992. [GHO 74] S. G HOSH and F. P OHLAND, “Kinetics of substrate assimilation and product formation in anaerobic digestion”, J. Wat. Pollut. Control Fed., vol. 46, pp. 748–759, 1974. [GOD 85] K.R. G ODFREY and J.J. D I S TEFANO III, “Identifiability of model parameters”, in Identification and System Parameter Estimation, pp. 89–114, Pergamon Press, Oxford, 1985. [GOL 89] D.E. G OLDBERG, Genetic algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Menlo Park, California, 1989. [GOO 87] G.C. G OODWIN, “Identification: Experiment Design”, in Systems and Control Encyclopedia, M. S INGH (ed.), vol. 4, pp. 2257–2264, Pergamon Press, Oxford, 1987. [HOL 82a] A. H OLMBERG, “On the practical identifiability of microbial growth models incorporating Michaelis-Menten type nonlinearities”, Math. Biosci., vol. 62, pp. 23–43, 1982. [HOL 82b] A. H OLMBERG and J. R ANTA, “Procedures for parameter and state estimation of microbial growth process models”, Automatica, vol. 18, pp. 181–193, 1982. [JEP 96] U. J EPPSSON, Modelling aspects of wastewater treatment processes, Doctoral thesis, Dep. Ind. Elec. Eng. Aut., Lund Inst. Technol., Sweden, 1996. [JUL 97] S. J ULIEN, Modélisation et estimation pour le contrôle d’un procédé boues activées éliminant l’azote des eaux résiduaires urbaines, Doctoral thesis, LASS-CNRS, Toulouse, Rapport LAAS No. 97499, 1997.

Identification of Bioprocess Models

77

[JUL 98] S. J ULIEN, J.P. BABARY and P. L ESSARD, “Theoretical and practical identifiability of a reduced order model in an activated sludge process doing nitrification and denitrification”, Wat. Sci. Tech., vol. 37, no. 12, pp. 309–316, 1998. [LJU 87] L. L JUNG, System Identification – Theory for the User, Prentice-Hall, Englewood Cliffs, New Jersey, 1987. [LOB 91] J.R. L OBRY and J.P. F LANDROIS, “Comparison of estimates of Monod’s growth model parameters from the same data set”, Binary, vol. 3, pp. 20–23, 1991. [LUB 91] A. L ÜBBERT, “Characterization of bioreactors”, in Biotechnology, A Multi-volume Comprehensive Treatise, K. S CHÜGERL (ed.), Measuring, Modelling and Control, vol. 4, pp. 107–148, VCH, Weinheim, 1991. [MAR 63] D.W. M ARQUARDT, “An algorithm for least squares estimation of nonlinear parameters”, J. Soc. Ind. Appl. Math., vol. 11, pp. 431–441, 1963. [MAR 89] S. M ARSILI -L IBELLI, “Modelling, identification and control of the activated sludge process”, Adv. Biochem. Eng. Biotechnol., vol. 38, pp. 90–148, 1989. [MUN 89] A. M UNACK, “Optimal feeding strategy for identification of Monod-type models by fed-batch experiments”, in Computer Applications in Fermentation Technology: Modelling and Control of Biotechnological Processes, N. F ISH, R. F OX and N. T HORNHILL (eds.), pp. 195–204, Elsevier, Amsterdam, 1989. [MUN 91] A. M UNACK, “Optimization of sampling”, in Biotechnology, a Multi-volume Comprehensive Treatise, K. S CHÜGERL (ed.), Measuring, Modelling and Control, vol. 4, pp. 251–264, VCH, Weinheim, 1991. [PET 00a] B. P ETERSEN, Calibration, identifiability and optimal experimental design of activated sludge models, PhD thesis, Fac Agr. Appl. Biol. Sci. Ghent University, Belgium, 337 pages, 2000. [PET 00b] B. P ETERSEN, K. G ERNAEY and P.A. VANROLLEGHEM, “Improved theoretical identifiability of model parameters by combined respirometric-titrimetric measurements. A generalisation of results”, in Proc. IMACS 3rd MATHMOD Conference, vol. 2, pp. 639–642, Vienna, Austria, 2000. [PET 00c] B. P ETERSEN, K. G ERNAEY and P.A. VANROLLEGHEM, “Practical identifiability of model parameters by combined respirometric-titrimetric measurements”, in Proc. Watermatex 2000, 2000. [POH 78] H. P OHJANPALO, “System identifiability based on the power series expansion of the solution”, Math. Biosci., vol. 41, pp. 21–33, 1978. [POS 90] C. P OSTEN and A. M UNACK, “On-line application of parameter estimation accuracy to biotechnical processes”, Proc. ACC, pp. 2181–2186, 1990. [PRE 86] W.H. P RESS, B.P. F LANNERY, S.A. T EUKOLSKY and W.T. V ETTERLING, Numerical Recipes: The Art of Scientific Computing, Cambridge University Press, Cambridge, UK, 1986. [RAK 85] A. R AKSANYI , Y. L ECOURTIER , E. WALTER AND A. V ENOT, “Identifiability and distinguishability testing via computer algebra”, Math. Biosci., vol. 77, pp. 245–266, 1985.

78

Bioprocess Control

[RAT 86] D.A. R ATKOWSKY, “A suitable parameterization of the Michaelis-Menten enzyme reaction”, Biochem. J., vol. 240, pp. 357–360, 1986. [ROB 85] J.A. ROBINSON, “Determining microbial parameters using nonlinear regression analysis: Advantages and limitations in microbial ecology”, Adv. Microb. Ecol., vol. 8, pp. 61–114, 1985. [SAN 94] J. S ANCHEZ, S. A RIJO, M. M UNOZ, M. M ORINIGO and J. B ORREGO, “Microbial colonisation of different support materials used to enhance the methanogenic process”, Appl. Microbiol. Biotechnol., vol. 41, pp. 480–486, 1994. [SCH 98] M. S CHUETZE, Integrated simulation and optimum control of the urban wastewater system, PhD thesis, Imperial College, London, UK, 1998. [SPE 00] M. S PÉRANDIO and E. PAUL, “Estimation of wastewater biodegradable COD fractions by combining respirometric experiments in various SO/XO ratios”, Wat. Res., vol. 34, pp. 1233–1246, 2000. [VAJ 89] S. VAJDA, K.R. G ODFREY and H. R ABITZ, “Similarity transformation approach to identifiability analysis of nonlinear compartmental models”, Math. Biosci., vol. 93, pp. 217– 248, 1989. [VAN 96] P.A. VANROLLEGHEM and K.J. K EESMAN, “Identification of biodegradation models under model and data uncertainty”, Wat. Sci. Tech., vol. 33, no. 2, pp. 91–105, 1996. [VAN 98] P.A. VANROLLEGHEM and D. D OCHAIN, “Bioprocess model identification”, in Advanced Instrumentation, Data Interpretation and Control of Biotechnological Processes, J. VAN I MPE, P. VANROLLEGHEM and D. I SERENTANT (eds.), pp. 251–318, Kluwer, Amsterdam, 1998. [VER 97] K.J. V ERSYCK, J.E. C LAES and J.F. VAN I MPE, “Practical identification of unstructured growth kinetics by application of optimal experimental design”, Biotechnol. Prog., vol. 13, pp. 524–531, 1997. [VER 00] K.J. V ERSYCK and J.F. VAN I MPE, “Optimal design of system identification experiments for bioprocesses”, Journal A, vol. 41, no. 2, pp. 25–34, 2000. [VIA 85] C. V IALAS, A. C HERUY and S. G ENTIL, “An experimental approach to improve the Monod model identification”, in Modelling and Control of Biotechnological Processes, A. J OHNSON (ed.), pp. 155–160, Pergamon, Oxford, 1985. [WAL 82] E. WALTER and Y. L ECOURTIER, “Global approaches to identifiability testing for linear and nonlinear state space models”, Math. Computers Sim., vol. 24, pp. 472–482, 1982. [WAL 94] E. WALTER and L. P RONZATO, Identification de Modèles Paramétriques à partir de Données Expérimentales, Masson, Paris, 1994. [WAL 95] E. WALTER and L. P RONZATO, “On the identifiability and distinguishability of nonlinear parametric models”, in Proc. Symp. on Appl. of Mod. and Cont. in Agric. and Bioind., IMACS, V.A.3.1–8, Brussels, Belgium, 1995.

Chapter 4

State Estimation for Bioprocesses

4.1. Introduction One of the main limitations to the improvement of monitoring and optimization of bioreactors is probably due to the difficulty in measuring chemical and biological variables. In fact there are very few sensors which are at the same time cheap and reliable and that can be used for online measurement. The measurement of some biological variables (biomass, cellular quota, etc.) is sometimes very difficult and can necessitate complicated and sophisticated operations. The challenge is to estimate the internal state of a bioreactor when only a few measurements are available. In this chapter we propose methods to build observers which will use the available measurements to estimate non-measured state variables (or at least some of them). The principle of this so-called “software sensor” is to use the process model to asymptotically reconstruct the state on the basis of the available outputs. As will be detailed in this chapter, the system must be observable, or at least detectable, in order to estimate the internal state. There are numerous methods to design an observer. They rely on ideas that can be very different. Thus the best observer must be chosen with respect to the type of problem. The choice will then be strongly connected to the quality and the uncertainties of the model and the data. If the biological kinetics are not precisely known, the mass balance will be the core of the asymptotic observers. If there are bounded uncertainties on the inputs and/or on the parameters, then we will estimate intervals in which the state of the system should lie. If the model has been correctly validated, then we can

Chapter written by Olivier B ERNARD and Jean-Luc G OUZÉ.

79

80

Bioprocess Control

fully exploit it and – if the outputs are not corrupted with a high level of noise – we can develop a high gain observer. The type of observer to be developed must not be based only on the model quality; it must also take into account the objectives to be achieved. In fact, an observer can have a purpose other than monitoring a bioreactor: it can be developed to apply a control action which needs an estimate of the internal state. It can also be used to determine that there was no failure in the process. 4.2. Notions on system observability We will only recall the main useful notions, we will give references for the more technical parts (see [KAI 80, LUE 79, GAU 01]). The observability notion is fundamental in automatic control. Intuitively, we try to estimate the state variables from the available measurements. If this is possible from a theoretical point of view, the system is said to be observable. Then another question is how to design an observer which is another dynamic system providing a state estimate. Let us mention that the question of observability and of observer design are very different: the observability property does not give any clue on how to build an observer. The theory is extensively developed in the linear case (see next section) and, in the nonlinear case, has been strongly developed during recent years, but for particular classes of models. 4.2.1. System observability: definitions We will consider the general continuous time system: ⎧ ⎨ dx (t) = f x(t), u(t); xt0  = x0 (S) dt   ⎩ y(t) = h x(t)

(4.1)

where x ∈ Rn is the state vector, u ∈ Rm is the input vector, y ∈ Rp is the output vector, x0 is the initial condition for initial time t0 , f : Rn × Rm → Rn and h : Rn → Rp . The functions are assumed to be sufficiently smooth in order to avoid problems of existence and solution uniqueness. EXAMPLE 4.1. For the bioreactors described by a mass balance model, we have:         f x(t), u(t) = Kr x(t) + D xin (t) − x(t) − Q x(t) where D and xin stand for the input vector.

State Estimation for Bioprocesses

81

We assume therefore that, for system (S), – the input u(t) is known; – the output y(t) is known; – functions f and h, are known, i.e. the model is known (r(·) is known in the mass balance-based modeling). We want to estimate x(t); the observability is a theoretical notion that states if it is possible. DEFINITION 4.1. Two states x0 and x0 are said to be indiscernible if for any input time function u(t) and for any t ≥ 0, the outputs h(x(t, x0 )) and h(x(t, x0 )) that result are equal. DEFINITION 4.2. The system is said to be observable if it does not have any distinct couples of initial state x0 , x0 that are indiscernible. This means that for any input the initial condition can be uniquely estimated from the output. There exist slightly different definitions, but we will not detail these nuances here. It can be noted that generally for nonlinear systems the observability depends on the input; a system can be observable for some inputs and not observable for others. DEFINITION 4.3. An input is said to be universal if it can distinguish any couple of initial conditions. DEFINITION 4.4. A non-universal input is said to be singular. Even in the case where all the inputs are universal (the system is said to be uniformly observable and can be rewritten under a specific form; see section 4.4), this can be insufficient in practice. We impose then that the universal property persists with time, and we obtain (at least for some systems) the notion of a regularly persisting input (see Hypothesis 4.5, section 4.5.3). For the linear systems things are much simpler (see below). 4.2.2. General definition of an observer Once the system has been proved to be observable, the next step is building the observer in order to estimate state variable x from the inputs, the outputs and the model. The observer principle is presented in Figure 4.1. This is a second dynamic system that will be coupled to the first one thanks to the measured output.

82

Bioprocess Control

u

Process x

Observer z

y

x

Figure 4.1. Observer principle

DEFINITION 4.5. An observer is an auxiliary system coupled with the original system: ⎧     ⎪ ⎨ dz (t) = fˆ z(t), u(t), y(t) ; z t0 = z0 (O) dt (4.2)   ⎪ ⎩x ˆ ˆ(t) = h z(t), u(t), y(t) ˆ q × Rm × Rp → Rn such that with z ∈ Rq , fˆRq × Rm × Rp → Rq and hR ˆ(t) = 0 lim x(t) − x

t→∞

(4.3)

It is the classical definition which may be insufficient in some cases. It is stated that the estimation error tends asymptotically toward zero. Moreover we try to regulate the error decreasing rate (convergence rate). We can explain this with a simple n linear example: let us consider the linear system dx dt = Ax + Bu where x ∈ R and let us assume that matrix A is stable. A trivial observer can be obtained with a copy of x x + Bu. In fact, the error e = x − x ˆ follows the same dynamics the system: dˆ dt = Aˆ de dt = Ae and therefore converges towards zero. Let us note that this observer does not necessitate any output. This example shows that the stable internal dynamics are sufficient to estimate the final state. This example highlights a property which will be called detectability for linear systems and which will be the basis of the asymptotic observer (section 4.4) in a different framework. As a consequence, an additional requested property is to be able to regulate the convergence rate of the observer in order to be able to reconstruct the state variables more rapidly than the dynamics of the system. Let us note that the observer variable (z in O) may be larger than the state variable to be estimated x. Another property that we desire is that if the observer is properly initiated, i.e. with the true value x(0), then its estimation remains equal to x(t) for all t. This suggests a peculiar structure for the observer.

State Estimation for Bioprocesses

83

DEFINITION 4.6. Often, the following observer is taken: ⎧ dˆ     x ⎪ ˆ(t), u(t) + k z(t), h(ˆ x(t)) − y(t) ⎨ (t) = f x dt (O) ⎪ ⎩ dz (t) = fˆz(t), u(t), y(t) with k z(t), 0 = 0 dt This is a copy of the system with a correcting term depending on the discrepancy between the true measured outputs and the value of the output calculated using the observer. The correction amplitude is regulated thanks to function k that can be seen as a gain (it is an internal regulation of the observer). In the ideal case, gain k can be regulated in order to obtain a converging rate as large as requested. DEFINITION 4.7. System (O) is said to be an exponential observer if, for any positive λ, gain k can be tuned such that       ˆ(t) − x(t) ≤ e−λt x ˆ(0) − x(0). ∀ x0 , x ˆ(0), z0 ∀t > 0, x 4.2.3. How to manage the uncertainties in the model or in the output In real life – and especially in the biological field – we often consider that there are noises either in the output (measurement noise) or in the state equation (model noise). In general, the model noise is assumed to be additive (see section 4.3.5), which is a strong hypothesis (it could be, for example, multiplicative). Another important case which often appears in the bioprocesses is when the model integrates some unknown parts. For example, the biological kinetics in the mass balance models for bioreactors are generally not precisely known [BAS 90]. How do we manage these two problems which have some related aspects?: – Linear filtering, and more specifically Kalman filtering. This is the most popular method. It assumes that the noises are additive and white; it minimizes the error variance (see below). – The L2 , H 2 or H ∞ approach. This consists of assuming that the noises or perturbations w(t) belong to a given class of functions (L2 ) and minimizing their impact on the output using the transfer function. In the H 2 approach, we try to minimize the norm of this transfer function. In the H ∞ approach, we try to minimize the input effect in the worst case (see [BAS 91]). For example, for a γ > 0 and R positive definite matrix, we want observer x ˆ to verify:  ∞ 2 2   x ˆ(t) − x(t)R − γ 2 w(t) dt ≤ 0. sup w(·)

0

84

Bioprocess Control

– Disturbance rejection. We try to build observers independently from the unknown perturbation. The disturbance is canceled thanks to linear combinations of variables [DAR 94, KUD 80, HOU 91]. Asymptotic observers belong to this class of systems (see section 4.5). – Bounds on the perturbations and on the uncertainties. We assume that uncertainties are bounded, and try to design interval observers which provide the best possible bounds for the variables to be estimated. For some cases, we try to minimize this bounding (section 4.6). An approach using interval analysis may also be efficient for state or parameter estimation [JAU 01, KIE 04]. – We can also use these bounds to design sliding mode observers which have a correcting term of the type sign(x − x ˆ). Note that the way these observers take the uncertainties into account generates discontinuous dynamics on the sliding manifolds [EDW 94]. – Robustness using LMI. There exist efficient algorithms to solve Linear Matrix Inequality (LMI) problems. It is possible to check if a matrix depending on parameters is positive definite; then it supports a Lyapunov function used to design an observer that is robust with respect to some uncertainties [ARC 01]. NOTE. It is possible to construct examples where a system is observable when the model is known and becomes unobservable when a part of the model is unknown. For such cases the requirement for a classical observer may be relaxed. In particular, we will no longer assume that   (4.4) ˆ(t) = 0 lim x(t) − x t→∞

but that the discrepancy tends toward a reasonable value for practical applications. 4.3. Observers for linear systems For single output linear stationary systems we have: ⎧ dx   ⎨ (t) = Ax(t) + Bu(t) SL : dt ⎩ y(t) = Cx(t) with A ∈ Mn×n (R) (n ≥ 2), C ∈ M1×n (R). The well known observability criterion is formulated as follows: ⎛ ⎞ C ⎜ CA ⎟   ⎜ ⎟ SL observable ⇐⇒ rank ⎜ .. ⎟ = n. ⎝ . ⎠ CAn−1

(4.5)

State Estimation for Bioprocesses

85

which relies on the fact that the observability space is generated by the vectors (C, CA, . . . , CAn−1 ). The canonical observability forms, that can be obtained after a linear change of coordinates, highlight the observation structure. They will reappear in the nonlinear case for the high gain observer (section 4.6). THEOREM 4.1. If the pair (A, C) is observable, then there exists an invertible matrix P such that: A0 = P −1 AP with

⎛

−an ⎜−an−1 ⎜ . ⎜ A0 = ⎜ .. ⎜ ⎝ −a2 −a1

1 0 0 1 .. .. . . 0 0 0 0

··· ··· .. . ··· ···

C0 = CP ⎞ 0 0⎟ ⎟ ⎟ ⎟ ⎟ 1⎠ 0

 C0 = 1 0

···

0



What happens if the system is not observable? We can rewrite it in two parts, as it is shown in the following theorem. Here A1 and A3 are two square matrices with dimensions corresponding to x1 and x2 . The canonical form clearly shows that x1 cannot be estimated from x2 . THEOREM 4.2. General canonical form: dx1 = A1 x1 + A2 x2 + B1 u dt dx2 = A3 x2 + B2 u dt y = C2 x2 Matrix A1 forces the dynamics of the unobservable part; if they are stable, then the dynamics of the total error will be stable, but the unobservable part will tend toward zero with its own dynamics (given by A1 ); the system is said to be detectable. 4.3.1. Luenberger observer If system (4.5) is observable, a Luenberger observer [LUE 66] can be derived:   dˆ x(t) = Aˆ x(t) + Bu(t) + K C x ˆ(t) − y(t) dt where K is a dimension n gain vector, which allows us to regulate the convergence rate of the observer.

86

Bioprocess Control

In fact, the dynamics of the observation error e = x − x ˆ is: de = (A + KC)e dt Let us note that these dynamics do not depend on the input. The pole placement theorem sates that the error dynamics can be arbitrarily chosen. THEOREM 4.3. If (A, C) is observable, vector K can be chosen in order to obtain arbitrary linear dynamics of the observation error. In particular, the gain vector K can be chosen in order that the error converges rapidly toward zero. However, the observer will then be very sensitive to perturbations (measurement noise for example). A good compromise must be chosen between stability and precision. The Kalman filter is a way to manage this compromise. 4.3.2. The linear case up to an output injection There is a very simple case for which a linear observer can be designed for a nonlinear system; this is the case where the nonlinearity depends only on output y. ⎧ ⎨ dx (t) = Ax(t) + φt, y(t) + Bu(t) (4.6) (S) : dt ⎩ y(t) = Cx(t) φ is a nonlinear (known) function which takes its values in Rn . The following “Luenberger like” observer generates a linear observation error equation:     dˆ x(t) = Aˆ x(t) + φ t, y(t) + Bu(t) + K C x ˆ(t) − y(t) dt The dynamics can be arbitrarily chosen if the pair (A, C) is observable. 4.3.3. Local observation of a nonlinear system around an equilibrium point Let us consider general system (4.1), and let us assume that it admits a single equilibrium point (working point) at (xe , ue ). The system can then be linearized around this point. THEOREM 4.4. The linearized system of (4.1) around (xe , ue ) is ⎧ ⎨ dX (t) = AX + BU (S) dt ⎩ Y (t) = CX with A=

∂f (x, u) ∂x

B=

∂f (x, u) ∂u

C=

∂h(x) ∂x

(4.7)

State Estimation for Bioprocesses

87

Matrices A, B, C are estimated at xe , ue . Variables X, U, Y are deviations toward equilibrium: U = u − ee ,

X = x − xe ,

Y = y − Cxe

If the pair (A, C) is observable, the nonlinear system is locally observable around the equilibrium. 4.3.4. PI observer The Luenberger observer is based on a correction of the estimations with a term related to the difference between the measured outputs and the predicted outputs. The idea behind the proportional integral observer is to use the integral of this error term. We consider the auxiliary variable w: ˆ  t   Cx ˆ(τ ) − y(τ ) dτ. w ˆ= 0

The PI observer for system (4.8) will then be rewritten: ⎧   dˆ x ⎪ ⎪ ˆ(t) − y + KP w ˆ ⎨ (t) = Ax(t) + Bu(t) + KI C x dt ⎪ dw ⎪ ⎩ ˆ (t)(t) = C x ˆ−y dt The error equation (ex = x ˆ − x and ew = w) ˆ is then: ⎛ ⎞ dex    ⎜ dt (t) ⎟ ex (t) ⎜ ⎟ = F + KI C KP ⎝ dew ⎠ C 0 ew (t) (t) dt

(4.8)

(4.9)

The gains KI and Kp can be chosen so as to ensure stable error dynamics [BEA 89]. The integrator addition provides more robustness to the observer to deal with measurement noise or modeling uncertainties. 4.3.5. Kalman filter The Kalman filter (see [AND 90]) is very famous in the framework of linear systems; it can be seen as a Luenberger observer with a time varying gain; this allows us to minimize the error estimate variance.

88

Bioprocess Control

A stochastic representation can be given by the observable system: ⎧   ⎪ ⎨ dx (t) = Ax(t) + Bu(t) + w(t); x t0 = x0 dt ⎪ ⎩y(t) = Cx(t) + v(t)

(4.10)

where w(t) and v(t) are independent centered white noises (Gaussian perturbations), with respective covariances Q(t) and R(t). Let us also assume that the initial distribution is Gaussian, such that: &  T '   = P0 ˆ0 ; E x0 − x ˆ0 x0 − x ˆ0 (4.11) E x0 = x where E represents the expected value and P0 is the initial covariance matrix of the error. The filter is written in several steps: 1. Initialization:   ˆ0 ; E x0 = x

&  T ' = P0 E x0 − x ˆ0 x0 − x ˆ0

(4.12)

2. Estimation of the state vector:   dˆ x (t) = Aˆ x(t) + Bu(t) + K(t) y(t) − C x ˆ(t) ; dt

  ˆ0 x ˆ t0 = x

(4.13)

3. Error covariance propagation (Riccati equation): dP (t) = AP (t) + P (t)AT − P (t)C T R(t)−1 CP (t) + Q(t) dt

(4.14)

4. Gain calculation: K(t) = P (t)C T R(t)−1

(4.15)

Some points can be emphasized: – this filter can still be applied when matrices A and C depend on time (the observability must nevertheless be proved); – the estimation of the positive definite matrices R, Q, P0 is often very delicate, especially when the noise properties are not known; – a deterministic interpretation of this observer can be given: it consists of minimizing the integral from 0 to t of the square of the error; – this observer can be extended by adding a term −θP (t) in the Riccati equation. This exponential forgetting factor allows us to consider the cases where Q = 0.

State Estimation for Bioprocesses

89

4.3.6. The extended Kalman filter This idea consists of linearizing a nonlinear system around its estimated trajectory. Then the problem is equivalent to building a Kalman filter for non-stationary system. Let us consider the system ⎧ ⎨ dx (t) = f x(t) + w(t); xt0  = x0 dt (4.16)   ⎩ y(t) = h x(t) + v(t) and the observer is designed as above, with a change in the second step. 2. Estimation of the state vector:        dˆ x ˆ0 (t) = f x ˆ(t) + K(t) y(t) − h x ˆ(t) ; x ˆ t0 = x dt and using the matrices of the tangent linearized:     ∂f x(t)  ∂h x(t)  A(t) = C(t) = ∂x(t) x(t)=ˆx(t) ∂x(t) x(t)=ˆx(t)

(4.17)

(4.18)

This extended filter is often used, even if only few theoretical results guarantee its convergence (see section 4.4.4). 4.4. High gain observers 4.4.1. Definitions, hypotheses In this chapter, we will assume that a simulation model of the process is available (i.e. with modeling of the biological kinetics). We also assume that the model has been carefully validated: the high gain observers are dedicated to the nonlinear systems and require high quality models. We will now consider the affine systems with respect to the input, that are described as follows: dξ = f (ξ) + ug(ξ) (4.19) dt We consider here the case where u ∈ R. For bioreactors, the input often corresponds to the dilution rate u = D. In this case f (ξ) = Kr(ξ) − Q(ξ) and g(ξ) = ξin − ξ. Moreover, we assume that the output is a function of the state: y = h(ξ) ∈ R. HYPOTHESIS 4.1. We will state the two following hypotheses: (i) system (4.19) is observable for any input; (ii) there exists a positively invariant compact K, such that for any time t, ξ(t) ∈ K.

90

Bioprocess Control

We will denote Lf h(ξ) =

Dh Dξ f (ξ),

which is the Lie derivative of h along the

h(ξ). vector field f . By convention, we will write Lpf h(ξ) = Lf Lp−1 f 4.4.2. Change of variable Let us consider the following change of coordinates, defined on the compact set K: & φ : ξ −→ ζ = h(ξ),

Lf h(ξ),

(n−1)

· · · , Lf

'T h(ξ)

(4.20)

This change of variable consists of considering (in the autonomous case) output y and its n − 1 first derivatives as new coordinates. HYPOTHESIS 4.2. Mapping φ is a global diffeomorphism. We can verify [GAU 81] that under Hypothesis 4.2 φ transforms (4.19) into:

with

⎛

0 ⎜0 ⎜ A=⎜ ⎜· ⎝0 0 ⎛ ⎜ ⎜ ˜ ψ(ζ) =⎜ ⎝

dζ ˜ + ψ(ζ)u ¯ = Aζ + ψ(ζ) dt

(4.21)

y = Cζ

(4.22)

1 0 · 0 0

0 1 · ··· ···

0 .. .

0   Lnf h φ−1 (ζ)

··· ··· · 0 0 ⎞ ⎟ ⎟ ⎟, ⎠

⎞ 0 0⎟ ⎟ ·⎟ ⎟, 1⎠ 0

C = [1, 0, . . . , 0]

⎞   ψ¯1 ζ1   ⎟ ⎜ ψ¯2 ζ1 , ζ2 ⎟ ⎜ ¯ ⎟ ⎜ ψ(ζ) = ⎜ .. ⎟ . ⎠ ⎝   ¯ ψ2 ζ1 , ζ2 , . . . , ζn ⎛

with    (i−1)  −1 ψ¯i (z) = ψ¯i ζ1 , . . . , ζi = Lg Lf h φ (ζ)

(4.23)

In this canonical form, all the system nonlinearities have been concentrated in ˜ ¯ terms ψ(ζ) and ψ(ζ). We will present the various observers using this canonical form (let us note that this canonical form is very close to the one in section 4.3 for the observer pole assignment).

State Estimation for Bioprocesses

91

Let us note that an observer in the new basis will provide an estimate ζˆ which will estimate ζ, i.e. the successive output derivatives. The idea consists of writing the observer in this canonical basis i.e. a numerical differentiator of the output. Then, going back to the initial coordinates (applying φ−1 (ζ)), the observer will be expressed in the original basis. To design a high gain observer, we need an additional technical hypothesis. HYPOTHESIS 4.3. Mappings ψ˜ and ψ¯ defined in (4.21) are globally Lipschitz on K. Intuitively, this hypothesis will allow us to dominate the nonlinear part, imposing that the dynamics of the observer can be faster than the system dynamics (this explains the idea of “high gain”). 4.4.3. Fixed gain observer PROPOSITION 4.1 ([GAU 92, BUS 02]). For a sufficiently high gain θ, and under Hypotheses 4.1, 4.2 and 4.3 the following differential system is an exponential observer of (4.19):  −1   ∂φ dˆ x = f (ˆ x) + ug(ˆ x) − x) − y (4.24) Sθ−1 C t h(ˆ dt ∂x x=ˆx where Sθ , is the solution of the equation θSθ + At Sθ + Sθ A = C t C. Sθ can be calculated as follows: Sθ (i, j) =

(−1)i+j (i + j − 2)! θi+j−1 (i − 1)!(j − 1)!

(4.25)

For the convergence proof and other details see [GAU 92]. 4.4.4. Variable gain observers (Kalman-like observer) The extended Kalman filter is often used in a framework where its convergence is not guaranteed (see section 4.3.5). We show here how to build a high gain observer very close to the Kalman filter (after a change of variable), whose convergence is guaranteed. PROPOSITION 4.2 ([DEZ 92]). For a gain θ that is sufficiently high, and under Hypotheses 4.1, 4.2 and 4.3 the following differential system is an exponential observer of (4.19): ⎧  −1   ⎪ x 1 ∂φ ⎪ dˆ ⎨ = f (ˆ x) + ug(ˆ x) − x) − y S −1 C t h(ˆ dt r ∂x x=ˆx (4.26) ⎪ dS 1 t ⎪ t  ⎩ = −SQθ S − A (ˆ x, u)S − SA (ˆ x, u) + C C dt r

92

Bioprocess Control

with r > 0, Qθ is calculated from the two positive definite symmetric matrices Δθ and Q:   Δθ = diag θ, θ2 , . . . , θn (4.27) Qθ = Δθ QΔθ Matrix A can be calculated from diffeomorphism φ:   ∂ψ ˆ u) = A + ∂φ A (ξ, +u ∂ζ ζ=φ(ξ) ∂ζ ζ=φ(ξ) ˆ ˆ

(4.28)

(4.29)

See [DEZ 92] for the proof of the convergence of this observer and for more details, especially for the choice of r and matrix Q. It is worth noting that, even if the filtering and noise attenuation performances of this extended Kalman filer are a priori better, this observer is above all a high gain observer; it will therefore present the same generic high sensitivity with respect to the measurement noises and modeling errors. The advantages of Kalman-like high gain observers have a price: this observer is differential equations more difficult to implement. In fact, we have to integrate n(n+3) 2 instead of n equations for the simple high gain observer. 4.4.5. Example: growth of micro-algae We will consider the growth of micro-algae in a continuous photobioreactor. The algal development is limited by a nitrogen source (NO3 ) denoted S, and uses principally inorganic dissolved carbon (C), mainly in the form of CO2 . The algal biomass (X) will then correspond to an amount of particulate nitrogen (N ). In order to simultaneously describe the cellular carbon and nitrogen uptake, we will consider the following reaction scheme r1 (·)

S −−−−→ N r2 (·)

C −−−−→ X Setting ξ = (X, N, S, C)t , the mass balance-based model (4.33) can be written with: ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ 0 0 0 1 ⎜ ⎟ ⎜ ⎟ ⎜1 0 ⎟ ⎟ , ξin = ⎜ 0 ⎟ Q(ξ) = ⎜ 0 ⎟ K=⎜ ⎠ ⎠ ⎝ ⎝ ⎝−1 0 ⎠ Sin 0 Cin Qc (ξ) 0 −k1

State Estimation for Bioprocesses

93

The units for carbon and nitrogen are the same for biomass and substrate, and moreover the nitrogen uptake yield is assumed to be unitary. The nutrient uptake rate is assumed to follow a Michaelis-Menten law [DUG 67]: S r1 (ξ) = ρmax X S + kS The algal growth from carbon is r2 (ξ) = μ(ξ)X, where the growth rate μ(ξ) is described by the Droop law [DRO 68]:  kq μ(ξ) = μ(q) = μ ¯ 1− (4.30) q Variable q represents the internal nitrogen quota defined by the amount of nitrogen N . per biomass unit: q = X We assume that biomass is measured (it is estimated by its total biovolume), and will be used to design a high gain observer to determine S and q. In this case, the nitrate concentration in the renewal medium (Sin ) can be controlled. More precisely, Sin can vary as follows: Sin = sin (1 + u) where u is the control, and sin the nominal concentration, corresponding to u = 0. In the following sections, we will consider only the first three equations of this system, and we will consider the following change of variables: N ; x3 = sSin – x1 = ρsminX ; x2 = Xk q – a1 =

ks sin ;

a2 = μ ¯; a3 =

ρm kq

that leads to the following system: ⎧ ⎨ dx = f (x) + ug(x) dt ⎩y = hx  1 with:

 ⎞ 1 a − Dx 1 − x 1 1 ⎛ ⎞ ⎟ ⎜ 2 x2 x1 ⎜  ⎟ x3 ⎟ ⎜ ⎝ ⎠ x = x2 , f (x) = ⎜a3 − a2 x2 − 1 ⎟ , ⎟ ⎜ a1 + x3 x3 ⎝   x1 x3 ⎠ D 1 − x3 − a1 + x3 ⎛ ⎞ 0   g(x) = ⎝ 0 ⎠ , h x1 = x1 D

(4.31)

⎛

(4.32)

94

Bioprocess Control

The high gain observer for model (4.31) is then given by: ⎛ ⎞ 3θ     ⎜ ⎟ x ˆ x ˆ2 ⎟ D ⎜ ⎜ 3θ 2 1 − 1 − x ˆ2 + 3θ2 2 ⎟ ⎜ x ˆ1 a2 a2 x ˆ1 ⎟ G(ˆ x) = ⎜ ⎟ ⎜  2  ⎟ ⎜ ⎟ 2 ˆ2 a1 + x ˆ3 ⎝ ⎠ ˆ32 + θ3 x ˆ31 + 3θ2 B 3θB a1 a2 a3 x ˆ1 with: ˆ31 = B

ˆ32 B

  ˆ3 1 D2 a3 x + 2a2 + x ˆ22 2a2 − 3D − a1 a3 x ˆ1 a1 + x ˆ3 a2

   ˆ3 a3 x D −x ˆ2 2 1− + 4a2 − 4D a1 + x ˆ3 a2  ˆ3 )2 ˆ3 x ˆ2 (a1 + x a3 x = x ˆ2 (2D − 3a2 ) + 4a2 + 2 a1 a2 a3 x ˆ1 a1 + x ˆ3

An experiment where u fluctuates sinusoidally was used to validate the observer. Figure 4.2 proves the observer efficiency when the model is well known. The observer predictions are in agreement with the experimental measurements. For more details on this example, see [BER 98, BER 01]. 4.5. Observers for mass balance-based systems 4.5.1. Introduction In the previous sections we have considered the case where the uncertainties were due to noise on the outputs and, in some cases, were due to modeling noise. We have seen in Chapter 2 that the bioprocess models are often not known very well. In particular, when the model is written on the basis of a mass balance analysis, a term representing the reaction rates appears. This term which represents the biological kinetics with respect to the model state variable is sometimes speculative. Often the modeling of the reaction rate is not reliable enough to base an observer on it. In this section we will use the results for the observers with unknown inputs [KUD 80, HOU 91, DAR 94], whose principle relies on a cancelation of the unknown part after a change of variable in order to build the observer. We will show how to build an observer for a system represented by a mass balance and for which the kinetics would not have been specified. We will see that the main condition to designing such an observer is that enough variables are measured. In particular we will not assume any observability property. This is not particularly

State Estimation for Bioprocesses

95

12

A

10

9

Biovolume (10 μm3/l)

14

8 6 4 2

Quota (10− 9 μmol/mm3 )

0

0

1

2

3

4

5

10

B

8 6 4 2 0

0

1

2

3

4

5

Nitrate (μmol/l)

0.5

C

0.4 0.3 0.2 0.1 0

0

1

2

3

4

5

Time (days) Figure 4.2. Comparison between direct measurements (•) and observer predictions (—) for model (4.31): (A) biomass estimated from total algal biovolume; (B) internal quota; (C) nitrate concentration

surprising since the observability property relies on its full description (including the kinetics) which is not used to build the mass balance observer. In fact, it is not really an observer in the strict sense, but more precisely a detector, relying on hypothesis that the non-observable part is stable.

96

Bioprocess Control

4.5.2. Definitions, hypotheses In this chapter we will consider the biotechnological processes that are modeled with a mass balance model: dξ = Kr(ξ) − D(t)ξ + D(t)ξin (t) − Q(ξ) (4.33) dt with ξ ∈ Rn

r ∈ Rp

(4.34)

We assume that the set of available measurements y can be decomposed into three vectors: T  (4.35) y = y1 y2 y3 where: – y1 is a set of q measured state variables. To simplify the notations, we will order the components of the state so that y1 corresponds to the q first components of ξ; – y2 represents the measured gaseous flow rates: y2 = Q(ξ); – y3 represents the other available measurements (pH, conductivity, etc.) that are related to the state using the following relationship: y3 = h(ξ). Let us rewrite system (4.33) after splitting the measured part (ξ1 = y1 ) from the other part of the state (ξ2 ). dξ1 = K1 r(ξ) − Dξ1 + Dξin1 − Q1 (ξ) dt

(4.36)

dξ2 = K2 r(ξ) − Dξ2 + Dξin2 − Q2 (ξ) dt

(4.37)

Matrices K1 and K2 and vectors ξin1 , ξin2 , Q1 and Q2 are such that    ξin1 K1 Q1 K= , ξin = , Q= K2 ξin2 Q2 4.5.3. The asymptotic observer In order to build the asymptotic observers we need the two following technical hypotheses. HYPOTHESIS 4.4. (i) There are more measured quantities than reactions: q ≥ p; (ii) matrix K1 is of full rank. Hypothesis 4.4(ii) means that a non-zero r cannot cancel K1 r (a reaction cannot compensate the other ones with respect to the measured variables).

State Estimation for Bioprocesses

97

Consequences: under Hypothesis 4.4, the q × p matrix K1 admits a left inverse; there exists a p × q matrix G such that: GK1 = Ip×p

(4.38)

Let us set: A = −K2 G, and let us consider the following linear change of coordinates: ζ1 = ξ1

(4.39)

ζ2 = Aξ1 + ξ2

(4.40)

This change of variable transforms (4.36) and (4.37) into: dζ1 = K1 r(T ζ) − Dζ1 + Dζin1 − Q1 (T ζ) dt     dζ2 = −D ζ2 − ζin2 − AQ1 (T ζ) + Q2 (T ζ) dt with

 T =

Ip 0p,n−p , −A In−p

  M = A In−p

(4.41) (4.42)

(4.43)

and ζin2 = M ξin

(4.44)

The equation of ζ2 can be rewritten using output y2 :   dζ2 = −D ζ2 − ζin2 − M y2 dt

(4.45)

NOTE. System (4.45) is a linear system up to an output injection (see section 4.3.2). We can now design an observer for this system by simply copying equations (4.45). However, we must first state an hypothesis to guarantee the observer convergence. HYPOTHESIS 4.5. The positive scalar variable D is a regularly persisting input i.e. there exist positive constants c1 and c2 such that, for all time instant t:  t+c2 0 < c1 ≤ D(τ )dτ t

In practice, c2 must be low with respect to the time constant of the system. Moreover, cc12 must be high because it determines the minimal converging rate of the observer.

98

Bioprocess Control

LEMMA 4.1 (see [BAS 90]). Under Hypothesis 4.5, solution ξˆ2 of the following asymptotic observer:   dζˆ2 = −D ζˆ2 − ζin2 − M y2 dt ξˆ2 = ζˆ2 − Ay1

(4.46)

converges asymptotically toward solution ξ2 of reduced system (4.37). Proof. It can easily be verified that the estimation error e2 = ξˆ2 − ξ2 = ζˆ2 − ζ2 satisfies: de2 = −De2 . (4.47) dt and converges asymptotically toward ξ2 if Hypothesis 4.5 is fulfilled [BAS 90]. 4.5.4. Example We will consider as an example the growth of the filamentous fungus Pycnoporus cinnabarinus (X) [BER 99]. The fungus uses two substrates to grow: glucose as a carbon source (C) and ammonium as a nitrogen source (N ). The reaction scheme is assumed to be composed by one reaction: N + C −→ X The model is then of the same type as (4.33), with: T    T ξ = N C X , K = −k1 −k2 1 , ξin = Nin , Cin ,  The following measurements are available: y1 = N

C

T

T 0

.

The state partition will then be the following:  T ξ1 = N C , ξ2 = X associated with:

 K1 = −k1

−k2

T

,

K2 = 1

of left inverses. We will consider two of them: Matrix  K1 has an infinite number  G1 = − k11 , 0 and G2 = 0, − k12 . These two matrices will naturally lead to two observers. The first one based on the nitrogen measurements:  dζˆ21 Nin = −D ζˆ21 − dt k1 (4.48) N ˆ 1 = ζˆ21 − X k1

State Estimation for Bioprocesses 3

3

A

B 2.5

2.5

2 X

X

2 1.5

1.5

1

1

0.5

0.5

0 0

99

1

2 Time (days)

3

0 0

4

1

2 Time (days)

3

4

Figure 4.3. Comparison between direct biomass measurements of Pycnoporus cinnabarinus (o) and observer predictions based on the nitrogen measurement (A) or on the carbon measurement (B)

and the other one based on carbon: dζˆ2



Cin = −D ζˆ22 − dt k2



2

(4.49)

ˆ 2 = ζˆ2 − C X 2 k2 The results of these observers obtained with experimental data are presented in Figure 4.3. In this case, the observer based on the nitrogen measurements is more reliable. 4.5.5. Improvements The asymptotic observers work in open loop. In fact, their estimate relies on the mass balances and are not corrected by a discrepancy between measured and estimated quantities. It assumes that the mass balance model is ideal. Nevertheless, the yield parameters are difficult to estimate properly, and in some cases (wastewater treatment) the mass inputs in the system are not precisely known. In this case it can be dangerous to base the observer only on the mass balance model without taking into account some measurements on the system that reflect its actual state. It can be possible to estimate these unknown parameters online, but we will see here another method aiming at improving the observer robustness with respect to some uncertainties. In this section we will see how to use the available measurements y3 to improve the asymptotic observer performances. We assume here that y3 ∈ R. We define the mapping h:       h : ξ1 , ξ2 ∈ Rp × Rn−p −→ y3 = h ξ1 , ξ2 ∈ R We suppose that h satisfies the following hypothesis. HYPOTHESIS 4.6. The mapping h is monotonous with respect to ξ2 , i.e.: fixed sign on the considered domain Ω.

Dh Dξ2

has a

100

Bioprocess Control

EXAMPLE 4.2. In the example detailed hereafter, h(ξ1 , ξ2 ) = αS + βP , and thus:  Dh = α Dξ2

β



(4.50)

which has a fixed sign. Of course, h can be nonlinear. PROPOSITION 4.3. Let λ ∈ Rp be a unitary constant vector ( λ = 1), whose signs Dh ), θ is a positive scalar (which can depend are chosen such that sign(λ) = sign( Dξ 2 on time) and zin = M ξin . The following system:       dˆ z = −D zˆ − zin − M y2 − θλ h y1 , zˆ − Ay1 − y3 dt

(4.51)

is an asymptotic observer of reduced system (4.45). For the proof of this property, and for more details, see [BER 00]. EXAMPLE 4.3. We will consider a bacterial biomass (X) growing in a bioreactor. The micro-organisms uptake substrate S and metabolize a product P : S −→ X + P The associated model is then: ⎧ dX ⎪ ⎪ = r(ξ) − DX ⎪ ⎪ dt ⎪ ⎪ ⎨   dS = −c1 r(ξ) + D Sin − S ⎪ dt ⎪ ⎪ ⎪ ⎪ dP ⎪ ⎩ = c2 r(ξ) − DP dt

(4.52)

where Sin is the influent substrate, c1 and c2 are the yield coefficients. We assume that the bacterial biomass and the conductivity of the solution can be measured. The conductivity is related to a positive linear combination of the ions in the liquid i.e. S and P . We thus obtain: y1 = X

(4.53)

y2 = (0, 0, 0)t

(4.54)

y3 = αS + βP

(4.55)

We suppose that the substrate concentration in the influent Sin is not precisely known, and we will use an estimate denoted Sˆin .

State Estimation for Bioprocesses

101

Thanks to Proposition  4.3 we can design the following observer (for the sake of clarity we choose λ = 1 0 which satisfies the correct hypotheses) ⎧ dˆ     z1 ⎪ = D zˆin1 − zˆ1 − θ αSˆ + β Pˆ − y3 ⎪ ⎪ ⎪ dt ⎪ ⎪ ⎪ ⎪ ⎪ dˆ z ⎪ ⎪ = −Dˆ z2 ⎨ dt 2 (4.56) ⎪ y1 ⎪ ⎪ ˆ S = zˆ1 − ⎪ ⎪ ⎪ c1 ⎪ ⎪ ⎪ ⎪ ⎪ ⎩Pˆ = zˆ2 + y1 c2 with zˆin1 = Sˆin . Let us now show the robustness properties when the estimate Sin is false. If S  and P represent equilibrium values of S and P , we denote by Sˆ and Pˆ  the equilibrium values for the closed loop observer. If the observer is in open loop (θ = 0), using Sˆin , a direct calculation provides: 

Sˆ = S  + Sˆ1in − S1in

(4.57)

The prediction error is thus exactly the error on S1in . With the closed loop observer, the steady state is: Sˆ = S  +

 D ˆ S1in − S1in θα + D

(4.58)

If gain θ is high, it is easy to see that Sˆ  S  ; the bias is reduced by the closed loop observer. 4.6. Interval observers The usual observers implicitly assume that the model is a good approximation of the real system. Nevertheless, we have seen that a model of a bioprocess is often poorly known. In this case, the observation principle must be revisited: generally it will no longer be possible to build an exact observer (which would guarantee: e(t) = ˆ x(t) − x(t) → 0 when t → ∞) whose convergence rate could be regulated (such as for example an exponential observer). Therefore, in this case the result must be weaker. We present here a possible method (among others) consisting of bounding the uncertainty on the model. The bound on the variable to be estimated is deduced. To simplify, we will first present the linear (or close to the linear) case (see [GOU 00, RAP 03]).

102

Bioprocess Control

4.6.1. Principle The idea is to use the known dynamic bounds of the uncertainties: the dynamic bounds on the model uncertainties allow us to derive (in the appropriate cases) the dynamic bounds on the state variable to be estimated. Figure 4.4 summarizes the philosophy of the interval estimation. U+

^

X+ X

U

^

X− U−

Figure 4.4. Principle of interval estimation for bounded uncertainties: a priori bounds on the uncertainties U provide bounds on the non-measured state X

Let us consider the general system: ⎧     dx   ⎨ (t) = f x(t), u(t), w(t) ; x t0 = x0 dt S0   ⎩ y(t) = h x(t), v(t)

(4.59)

where x ∈ Rn is the state vector y ∈ Rp is the output vector, u ∈ Rm the input vector, x0 the initial condition at t0 , f : Rn × Rm × Rr → Rn and h : Rn × Rs → Rp . The unknown quantities w ∈ Rr and v ∈ Rs are characterized by their upper and lower bounds: w− (t) ≤ w(t) ≤ w+ (t) ∀t ≥ t0

(4.60)

v − (t) ≤ v(t) ≤ v + (t) ∀t ≥ t0

(4.61)

NOTE. The operator ≤ applies to vectors; it corresponds to inequalities between each component. Based on the fixed model structure (S0 ) and the set of known variables, a dynamic auxiliary system can be designed as follows: ⎧ −       dz ⎪ ⎪ ⎪ = f − z − , z + , u, y, w− , w+ , v − , v + ; z − t0 = g − x− , x+ 0 0 ⎪ ⎪ dt ⎪ ⎪ ⎪ +         ⎨ dz + = f + z − , z + , u, y, w− , w+ , v − , v + ; z + t0 = g + x− O0 0 , x0 dt ⎪ ⎪   ⎪ − ⎪ x = h− z − , z + , u, y, w− , w+ , v − , v + ⎪ ⎪ ⎪ ⎪ ⎩x+ = h+ z − , z + , u, y, w− , w+ , v − , v +  (4.62) with z − , z + ∈ Rq , the other functions being defined in the appropriate domains.

State Estimation for Bioprocesses

103

DEFINITION 4.8 (interval estimator). System (O0 ) is an interval estimator of system + − + (S0 ) if for any pair of initial conditions x− 0 ≤ x0 , there exist bounds z (t0 ), z (t0 ) such that the coupled system (S0 , O0 ) verifies: x− (t) ≤ x(t) ≤ x+ (t);

∀t ≥ t0

(4.63)

The interval estimator is derived from the coupling between two estimators providing each an under-estimate x− (t) and an over-estimate x+ (t) of x(t). The estimator provides a dynamic interval [x− (t), x+ (t)] containing the unknown value x(t) (Figure 4.4). Of course, this interval can be very large and therefore useless. The next step consists of trying to reduce as far as possible this interval and increase the convergence rate toward this interval, for example with an exponential convergence rate. Then, we move back to classical observation problems, with the important difference that we do not require the observation error (the interval amplitude) to tend asymptotically exactly toward zero. 4.6.2. The linear case up to an output injection First, let us take a very simple case. We again consider the following system: ⎧     ⎨ dx (t) = Ax(t) + φ t, y(t) S : dt (4.64) ⎩y(t) = Cx(t) with A ∈ Mn×n (R) (n ≥ 2), C ∈ M1×n (R). If the mapping φ : R+ × R → Rn is known, a Luenberger observer can be designed (section 4.3.1). What happens now if function φ is not well known? We assume that it can be bounded and that the bounds are known. Thus, functions φ− , φ+ : R+ × R → Rn , are known and sufficiently smooth, such that: φ− (t, y) ≤ φ(t, y) ≤ φ+ (t, y),

∀(t, y) ∈ R+ × R

(4.65)

We will then use these bounds to design an upper and a lower estimator:     dx + (t) = Ax+ (t) + φ+ t, y(t) + K Cx+ (t) − y(t) dt     dx − (t) = Ax− (t) + φ− t, y(t) + K Cx− (t) − y(t) . dt If we now consider the “upper” error e+ (t) = x+ (t) − x(t), we obtain: de + = (A + KC)e+ + b+ (t) dt

(4.66) (4.67)

104

Bioprocess Control

with     b+ (t) = φ+ t, y(t) − φ t, y(t) . It follows that b+ is positive, and the following lemma can easily be proved: LEMMA 4.2. If the elements of matrix (A + KC) are positive outside the diagonal (the matrix is said to be cooperative), then e+ (0) ≥ 0 implies e+ (t) ≥ 0 for any positive t. Of course, we have the same Lemma for the lower error: e− (t) = x(t) − x− (t) and the total error e(t) = e− (t) + e+ (t). The following theorems can be deduced: THEOREM 4.5. If the gains of vector K can be chosen such that matrix (A + KC) is cooperative, and if we have an initial estimate such that x− (0) ≤ x(0) ≤ x+ (0) then equations (4.66), (4.67) provide an interval estimator for system (4.64). THEOREM 4.6. If hypotheses of the proceeding theorem are verified, if matrix (A + KC) is stable, and if moreover the error on φ can be bounded, i.e. if we have: b(t) = φ+ (t, y) − φ− (t, y) ≤ B where B is a positive constant, then error e(t) converges asymptotically toward an interval smaller (for each component) than the positive vector: emax = −(A + KC)−1 B In particular, if the components of emax are zero, then the corresponding components for e(t) converge towards zero. The proofs are straightforward; the proof of the first theorem follows directly from the above lemma. The proof of the second theorem is due to the differential inequality (A + KC)e + b(t) ≤ (A + KC)e + B which implies (with equal initial conditions) e(t) ≤ em (t), where em (t) is the solution of

de dt m

∀t ≥ 0

= (A + KC)em + B.

State Estimation for Bioprocesses

105

Some observations: – we use in the observer design the fundamental hypothesis that it is possible to derive inequalities between the variables from inequalities on the left hand side of the differential equations. This hypothesis is connected with the comparison of the solutions of differential equations (see section 4.8). There exist other techniques to estimate the intervals; they are more precise but less explicit [JAU 01]; – we also need the assumption that the initial estimate is valid x− (0) ≤ x(0) ≤ x+ (0); a large estimate can be chosen in practice; – the problem of regulating the convergence rate has not been considered here; is it possible to choose a gain K that will ensure cooperativity, stability and an arbitrary convergence rate? This is a complicated problem; see [RAP 03] for more details. We illustrate this approach with an example of such an estimator for a biochemical process (see also [GOU 00, ALC 02]). 4.6.3. Interval estimator for an activated sludge process We consider a very simplified model of an activated sludge process, used for biological wastewater treatment. The objective is to process wastewater, with an influent flow rate Qin and a pollutant (substrate) concentration sin . We are aiming for a concentration of the effluent that is lower than sout . The process is composed of an aerator (bioreactor) followed by a settler separating the liquid and solid (biomass) phases. Then we recycle a part of the biomass toward the aerator. Let us denote by x, s and xr the three state variables of this simple model, representing respectively the biomass and substrate concentrations in the aerator, and the recycled biomass in the settler. Qin , Qout , Qr and Qw are the flow rates, Va and Vs the volumes (see Figure 4.5). We suppose that the biological reactions only take place in the aerator. 1 μ(·)x s −−−−→ x Y Q in , s in

aerator

x ,s

settler

Va

Qout , sout

Vs Qr , x r

Qw , x r

Figure 4.5. Diagram of an activated sludge process

106

Bioprocess Control

Y is a yield coefficient and μ(·) the bacterial growth rate. If we take into account the biomass recycling, we obtain: ⎧ dx ⎪ ⎪ = μ(·)x − (1 + r)D(t)x + rD(t)xr ⎪ ⎪ dt ⎪ ⎪ ⎨ ds (4.68) = − μ(·)x − (1 + r)D(t)s + D(t)sin (t) Y ⎪ dt ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ dxr = v(1 + r)D(t)x − v(w + r)D(t)xr dt with D(t) =

Qin ; Va

r=

Qr ; Qin

w=

Qw ; Qin

v=

Va Vs

We state the following hypotheses: – we measure only s and we want to estimate x and xr ; – we assume that μ(·) is not known, and we want to design an asymptotic observer (see section 4.5.3); – we know a bounding (even if very loose) of the initial conditions for x and xr ; – the substrate input for sin fluctuates but is not known. However, we know dynamic bounds for sin (t): + s− in (t) ≤ sin (t) ≤ sin (t)

∀t ≥ 0.

These hypotheses correspond to what happens in a urban wastewater treatment plant. The influent varies but is not measured. However, it can be bounded by two periodic functions corresponding to human activities. These bounds will probably evolve with respect to seasons. We will design an asymptotic interval estimator, which will provide bounds for the variables to be estimated. First we carry out a change of variable to eliminate μ(·) (see section 4.5.3):    x Y z (4.69) Z=X+ s; Z = 1 ; X = z2 xr 0 and we obtain the 2-dimensional system:    dZ x0 + Y s0 = D(t) AZ + B(s, t) ; Z0 = xr 0 dt   −(1 + r) r Y sin (t) A= ; B(s, t) = −Y v(1 + r)s v(1 + r) −v(w + r)

State Estimation for Bioprocesses

107

We can now build two estimators (upper and lower) which use unknown bounds on the influent concentration sin :   ⎧   ⎪ Y dZˆ + + ⎪ + + + ˆ ˆ ⎪ = D(t) AZ + B (s, t) ; Z (0) = X0 + s0 ⎪ ⎪ 0 ⎪ ⎪ dt ⎪ ⎪   ⎪ ⎪ ˆ− ⎪   Y d Z ⎪ ⎪ ⎪ = D(t) AZˆ − + B − (s, t) ; Zˆ − (0) = X0− + s0 ⎪ ⎨ dt 0 (4.70)   ⎪ ⎪ Y ⎪ + + ˆ = Zˆ − ⎪ X s ⎪ ⎪ ⎪ 0 ⎪ ⎪ ⎪   ⎪ ⎪ ⎪ Y ⎪ ⎪ − − ˆ ˆ ⎪ s ⎩X = Z − 0 with B + (s, t) =

&

s+ in (t) −v(1+r)s

'

Y ; B − (s, t) =

&

s− in (t)

'

−v(1+r)s

Y

In this simple case, the convergence rate is fixed by the system. In Figure 4.6, we have represented the influent concentration and its bounds, the measurement s and the two estimates with the bounds. This very simple observer illustrates how to take into account the knowledge on the dynamic bounds. Under certain hypotheses, it can be shown that this observer can be regulated [GOU 00]. 4.6.4. Bundle of observers Now we will develop a set of regulatable interval observers for a considered system. We benefit from the possibility of regulating the error dynamics in order to run in parallel a broad set of interval observers associated with various gains that all guarantee a bounding of the real state. It is then possible to select the inner envelope of the so-called observer bundle in order to obtain a new observer with a much better convergence rate and much smaller interval predictions [BER 04]. This approach will be illustrated on a very simple example which describes the behavior of the concentrations of a biomass x and a substrate s in a perfectly mixed bioreactor: x˙ = μ(s)x − ux   s˙ = u sin − s − kμ(s)x y=s

(4.71)

108

Bioprocess Control s (mg/l)

Sin (mg/l) 200

20

(a)

190

(b)

18

180

16

170

14

160

12

150

10

140

8

130

6 4

120 110

t (h)

2

t (h)

0

100 0

50 sin sin+ sin-

100

150

200

250

300

350

400

450

0

500

50

x (mg/l) 2500

150

200

250

300

350

400

450

500

350

400

450

500

xr (mg/l) 4500

(c)

2250

100

(d)

4000

2000

3500

1750 3000 1500 2500 1250 2000 1000 1500 750 1000

500

500

250

t (h)

t (h) 0

0 0

50 x x+ x-

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

xr xr+ xr-

Figure 4.6. Interval observer: (a) influent bounds, (b) measurements of S, (c) interval estimations of X, (d) interval estimations of Xr

sin corresponds to the concentration of influent substrate, u is the dilution input and k is the conversion yield coefficient. The biological activity of the system are featured by the non-negative growth rate function μ(s) such that μ(0) = 0, known to be highly uncertain. For the sake of simplicity, μ(s) is assumed to be a C 1 function. Our objective is to develop a robust observer based on the interval approach with improved convergence properties in order to estimate the biomass concentration x(t) in a reactor, when monitoring substrate s(t). In the following we assume that the following hypothesis is fulfilled. HYPOTHESIS 4.7. The growth rate is bounded by two known functions μ(s) and μ(s) and a positive constant a > 0: 1. 0 ≤ μ(s) ≤ μ(s) ≤ μ(s) ≤ a, ∀s ≥ 0 2. μ(0) = 0 Let us introduce the variable: z = kx + θ(t)s

(4.72)

where θ(t) ∈ Θ is a gain function. Considering equation (4.71), the dynamics of z can be written as follows:   ˙ z˙ = (1 − θ)μ(s)(z − θs) + u θsin − z + θs (4.73)

State Estimation for Bioprocesses

109

The idea used in [LEM 05] was to implement a hybrid observer combining two types of observers: a bounded error observer with high convergence rate but poor accuracy and an asymptotic observer with fixed convergence rate but high accuracy. HYPOTHESIS 4.8. The influent substrate sin is an unknown input of system (4.71), with known bounds: + s− in (t) ≤ sin (t) ≤ sin (t)

(4.74)

PROPOSITION 4.4. Given x0 , x0 such that x0 ∈ [x0 , x0 ], then for a gain θ, the following systems define interval observers for system (4.71): – for θ(t) < 0     ˙ z˙ θ = (1 − θ) μ(s)z θ − θμ(s)s + u θs− in − z θ + sθ (4.75)     ˙ z˙ θ = (1 − θ) μ(s)z θ − θμ(s)s + u θs+ in − z θ + sθ – for 0 ≤ θ(t) < 1     ˙ z˙ θ = (1 − θ) μ(s)z θ − θμ(s)s + u θs+ in − z θ + sθ     ˙ z˙θ = (1 − θ) μ(s)z θ − θμ(s)s + u θs− in − z θ + sθ

(4.76)

– for θ(t) ≥ 1     ˙ z˙ θ = (1 − θ) μ(s)z θ − θμ(s)s + u θs+ in − z θ + sθ     ˙ z˙ θ = (1 − θ) μ(s)z θ − θμ(s)s + u θs− in − z θ + sθ

(4.77)

with   xθ = z θ − θs /k

and

  xθ = z θ − θs /k

(4.78)

Proof. See [MOI 07]. One of the key properties related to an interval observer is that we can compare the solutions generated by two or more observers. We thus run several observers in parallel (for different values of gain θ), which all provide guaranteed interval estimates for the state. This is what we then call an observer bundle. Each bundle has an envelope that provides the best bounds i.e., we take the inner envelope from all the sets of estimates generated by different gain values. It is worth noting that we combine the transient behavior of some unstable observers (this may improve transient estimations), with the asymptotic stability of others (this guarantees the boundedness of the envelope).

110

Bioprocess Control

40

biomass (g/l)

30

20

10

0

0

10

20

30

40

time (days) Figure 4.7. Bundle of observer to estimate a biomass from substrate measurements and range of influent substrate

A regular reinitialization of the bundle can be performed to restart all the observers with the best available interval predicted by the bundle. We consider the time interval [tk , tk + Δt ] (we denote by Δt the reinitialization time interval) where the observers run. Then at time instant tk we take the best interval estimates performed by the previous estimation period to reinitialize the whole bundle. The objective behind the regular reinitialization of the interval estimates is to improve the observer efficiency by feeding it with the best available estimate, and thus take the benefit of the transients of some of them. The simulation example in Figure 4.7 performed with model (4.71) when s is measured demonstrates the efficiency of the approach; see [BER 04, MOI 05, MOI 07] for more details on the design of a bundle of observers. 4.7. Conclusion We have seen a set of methods to design an observer for a bioreactor. Other techniques exist and we do not pretend to be exhaustive. Let us mention for example the methods based on neural networks, where the system and its observer are estimated at the same time. The convergence rate of the obtained neural network cannot be regulated.

State Estimation for Bioprocesses

111

The presented observers assumed that the required model parameters were known and fixed. In some cases these parameters can evolve. Algorithms to estimate the parameters must then be used; they lead to adaptive observers. Of course in that case, the convergence of the full observer-parameter estimator system must be demonstrated. The choice of the type of observer to be developed must be made above all by considering the reliability of the model and the available measurements. A triple tradeoff must then be managed between robustness with respect to modeling uncertainties, robustness with respect to disturbances and convergence rate. Finally, to implement an observer in a computer, a time discretization phase is required. This step will be based on Euler-type algorithms. This step is not difficult, but it requires care. In particular, if the sampling rate is too high with respect to the discretization rate, continuous/discrete observers must be used [PEN 02]. To conclude, we insist that the observers must first be validated before they can be used. For this, their predictions must extensively be compared to direct measurements that were not used during the calibration process. 4.8. Appendix: a comparison theorem We recall here a general theorem in the nonlinear case. It can be useful to apply the interval estimation techniques; see [SMI 95] for details and for a general presentation. DEFINITION 4.9. A nonlinear system in dimension n is said to be cooperative if its Jacobian matrix is positive outside its diagonal on a convex domain. We now give the comparison theorem between x(t) and y(t) defined by the two systems dx = f (x, t); dt

x(0) = x0

dy = g(y, t); dt

y(0) = y0

where f, g : U × R+ → Rn are sufficiently regular on a convex domain U ⊂ Rn . PROPOSITION 4.5. If – ∀z ∈ U , ∀t ≥ 0, f (z, t) ≤ g(z, t) – g is cooperative – x0 ≤ y0 then x(t) ≤ y(t) for t > 0.

112

Bioprocess Control

The inequalities must be considered component-wise. For a cooperative system, the main property is thus that the order between two solutions is conserved for any time. This property is fundamental for the design of interval estimators.

4.9. Bibliography [ALC 02] V. A LCARAZ -G ONZALEZ, J. H ARMAND, A. R APAPORT, J.-P. S TEYER, A. G ON ZALEZ and C. P ELAYO -O RTIZ , “Software sensors for highly uncertain WWTPs: a new approach based on interval observers”, Wat. Res., vol. 36, pp. 2515–2524, 2002. [AND 90] B.D.O. A NDERSON and J.B. M OORE, Optimal Control. Linear Quadratic Methods., Prentice Hall, Englewood Cliffs, NJ, 1990. [ARC 01] M. A RCAK and P. KOKOTOVI C´ , “Observer-based control of systems with sloperestricted nonlinearities”, IEEE Transactions on Automatic Control, vol. 46, no. 7, pp. 1146–1151, 2001. [BAS 90] G. BASTIN and D. D OCHAIN, On-line Estimation and Adaptive Control of Bioreactors, Elsevier, 1990. [BAS 91] T. BASAR and P. B ERNHARD, H ∞ -Optimal Control and Related Minimal Design Problems: a Dynamic Game Approach, Birkhaüser, Boston, 1991. [BEA 89] S. B EALE and B. S HAFAI, “Robust control system design with a proportional integral observer”, Int. J. Contr., vol. 50, no. 1, pp. 97–111, 1989. [BER 98] O. B ERNARD, G. S ALLET and A. S CIANDRA, “Nonlinear observers for a class of biological systems. Application to validation of a phytoplanktonic growth model”, IEEE Trans. Autom. Contr., vol. 43, pp. 1056–1065, 1998. [BER 99] O. B ERNARD, G. BASTIN, C. S TENTELAIRE, L. L ESAGE -M EESSEN and M. A STHER, “Mass balance modelling of vanillin production from vanillic acid by cultures of the fungus Pycnoporus cinnabarinus in bioreactors”, Biotech. Bioeng, pp. 558–571, 1999. [BER 00] O. B ERNARD, J.-L. G OUZÉ and Z. H ADJ -S ADOK, “Observers for the biotechnological processes with unknown kinetics. Application to wastewater treatment”, in Proceedings of CDC 2000, Sydney, Australia, 11–15 December, 2000. [BER 01] O. B ERNARD, A. S CIANDRA and G. S ALLET, “A non-linear software sensor to monitor the internal nitrogen quota of phytoplanktonic cells”, Oceanologica Acta, vol. 24, pp. 435–442, 2001. [BER 04] O. B ERNARD and J.-L. G OUZÉ, “Closed loop observers bundle for uncertain biotechnological models”, J. Process. Contr., vol. 14:7, pp. 765–774, 2004. [BUS 02] E. B USVELLE and J. G AUTHIER, “High-gain and non-high-gain observers for nonlinear systems”, in Contemporary Trends in Nonlinear Geometric Control Theory and its Applications. International Conference, Mexico City, Mexico, A. A NZALDO -M ENESES, B. B ONNARD, J.P. G AUTHIER and F. M ONROY-P ÉREZ (eds.), World Scientific, 2002.

State Estimation for Bioprocesses

113

[DAR 94] M. DAROUACH, M. Z ASADZINSKI and S.J. X U, “Full-order observers for linear systems with unknown inputs”, IEEE Trans. Automat. Contr., vol. AC-39, pp. 606–609, 1994. [DEZ 92] F. D EZA, E. B USVELLE, J. G AUTHIER and D. R AKOTOPARA, “High gain estimation for nonlinear systems”, System and Control Letters, vol. 18, pp. 292–299, 1992. [DRO 68] M. D ROOP, “Vitamin B12 and marine ecology. IV. The kinetics of uptake growth and inhibition in Monochrysis lutheri”, J. Mar. Biol. Assoc., vol. 48, no. 3, pp. 689–733, 1968. [DUG 67] R. D UGDALE, “Nutrient limitation in the sea: dynamics, identification and significance”, Limnol. Oceanogr., vol. 12, pp. 685–695, 1967. [EDW 94] C. E DWARDS and S.K. S PURGEON, “On the development of discontinuous observers”, Int. J. Control, vol. 59, no. 5, pp. 1211–1229, 1994. [GAU 81] J. G AUTHIER and G. B ORNARD, “Observability for any u(t) of a class of nonlinear systems”, IEEE Trans. Autom. Contr., vol. 26, no. 4, pp. 922–926, 1981. [GAU 92] J. G AUTHIER, H. H AMMOURI and S. OTHMAN, “A simple observer for nonlinear systems applications to bioreactors”, IEEE Trans. Autom. Contr., vol. 37, pp. 875–880, 1992. [GAU 01] J. G AUTHIER and I.A.K. K UPKA, Deterministic Observation Theory and Applications, Cambridge University Press, Cambridge, 2001. [GOU 00] J.-L. G OUZÉ, A. R APAPORT and Z. H ADJ -S ADOK, “Interval observers for uncertain biological systems”, Ecological Modelling, vol. 133, pp. 45–56, 2000. [HOU 91] M. H OU and P. M ÜLLER, “Design of observers for linear systems with unknown inputs”, IEEE Trans. Autom. Contr., vol. AC-37, no. 6, pp. 871–875, 1991. [JAU 01] L. JAULIN, M. K IEFFER,O. D IDRIT and E. WALTER, Applied Interval Analysis, with Examples in Parameter and State Estimation, Robust Control and Robotics, SpringerVerlag, London, 2001. [KAI 80] T. K AILATH, Linear Systems, Prentice-Hall, Inc., Englewood Cliffs, N.J., London, 1980. [KIE 04] M. K IEFFER and E. WALTER, “Guaranteed nonlinear state estimator for cooperative systems”, Numerical Algorithms, vol. 37, pp. 187–198, 2004. [KUD 80] P. K UDVA, N. V ISWANADHAM and A. R AMAKRISHNA, “Observers for linear systems with unknown inputs”, IEEE Trans. Autom. Contr., vol. AC-25, no. 1, pp. 113–115, 1980. [LEM 05] V. L EMESLE and J.-L. G OUZÉ, “Hybrid bounded error observers for uncertain bioreactor models”, Bioprocess and Biosystems Engineering, vol. 27, no. 5, pp. 311–318, Springer-Verlag, 2005. [LUE 66] D.G. L UENBERGER, “Observers of multivariables systems”, IEEE Transaction on Automatic Control, vol. 11, pp. 190–197, 1966.

114

Bioprocess Control

[LUE 79] D.G. L UENBERGER, Introduction to Dynamic Systems: Theory, Models and Applications, Wiley, 1979. [MOI 05] M. M OISAN and O. B ERNARD, “Interval Observers for non monotone systems. Application to bioprocess models”, in Proceedings of the 16th IFAC World Conference, Prague, Czech Republic, 4th–8th July, 2005. [MOI 07] M. M OISAN, O. B ERNARD and J.-L. G OUZÉ, “Near optimal interval observers bundle for uncertain bioreactors”, in Proceedings of the 9th European Control Conference, Kos, Greece, 2007. [PEN 02] M. P ENGOV, E. R ICHARD and J.-C. V IVALDA, “Continuous-discrete observers for global stabilization of nonlinear systems with applications to bioreactors”, Europ. J. Contr., vol. 8, pp. 465–476, 2002. [RAP 03] A. R APAPORT and J.-L. G OUZÉ, “Parallelotopic and practical observers for nonlinear uncertain systems”, Int. Journal. Control, vol. 76, no. 3, pp. 237–251, 2003. [SMI 95] H.L. S MITH, Monotone Dynamical Dystems: an Introduction to the Theory of Competitive and Cooperative Systems, American Mathematical Society, Providence, Rhode Island, 1995.

Chapter 5

Recursive Parameter Estimation

5.1. Introduction In Chapter 4, we dealt with the reconstruction of the evolution of state variables (essentially the concentrations of substrates, products and biomass) over the course of time. In Chapter 3, we calculated the values of the parameters of the model chosen by us for our bioprocess model on the basis of the available experimental data. In this chapter, we deal with a problem that is complementary to the problems described in Chapters 3 and 4: how do we calculate online the value of parameters for our bioprocess model? The motivation for addressing this question is as follows. We have seen earlier that it could be difficult to determine certain parameters with precision. In addition, certain essential parameters such as specific rates often cannot be modeled in a reliable and precise manner. For these reasons, it may turn out to be important, especially in the context of monitoring and control, to be in a position to evaluate over the course of time the value of the most “uncertain” parameters (not only from the point of view of determining the values but also from that of analytical characterization with respect to the process variables) and their variation. The approaches developed here will be substantially different from those considered in Chapter 3 insofar as we want to estimate a parameter at every instant. Therefore, the estimation algorithms in this chapter will be of the recursive type. Here we will be essentially concentrating on two types of approach involving the recursive estimation of parameters: 1) estimator based on the structure of the observer; 2) estimator using recursive least squares. Chapter written by Denis D OCHAIN.

115

116

Bioprocess Control

We will then see how to combine parameter and state estimations in the core of the same estimation algorithm, which we will call an adaptive observer (section 5.4). 5.2. Parameter estimation based on the structure of the observer Let us recall the equations for the general dynamic model for bioreactors: dξ = Kφ(ξ) + F − Q − Dξ dt

(5.1)

where F and Q are the inflow rate of matter and the gaseous outflow rate, respectively [BAS 90]. Let us consider the sufficiently general case where the most uncertain parameters are concentrated in the process kinetics. Let us also assume that the not well-known parameters are linear in the dynamic model. We can, for the requirements of the cause, rewrite the equations for the model as follows: dξ = KG(ξ)θ + F − Q − Dξ dt

(5.2)

Here, θ is the vector of unknown parameters, and G(ξ) a matrix, which is dependent on the state of the process. These equations will serve as a base for designing the estimator. First of all, let us introduce an example that will serve to illustrate the method throughout this section. 5.2.1. Example: culture of animal cells The process with respect to an animal cell, which we are going to consider here, is a culture of human fetus kidney cells (HEK-293) [SIE 99]. It has been shown in [SIE 99] that the process is characterized by the following reaction scheme: k1 S + k2 C −→ X + k3 G

(5.3)

k4 S −→ X + k5 L

(5.4)

Here S, C, X, G and L represent glucose, dissolved oxygen, yeasts, carbon dioxide and lactate, respectively. Reactions (5.3) and (5.4) are, respectively, the oxidation reaction (respiration) on glucose and that of glycolysis (fermentation) on glucose. The dynamics of the process in a stirred tank reactor are given in the matrix format mentioned above (5.2) by the following vectors and matrices: ⎡ ⎤ ⎡ ⎤ −k1 −k4 S ⎢−k2 ⎢C ⎥  0 ⎥ ⎢ ⎥ ⎢ ⎥ μR X ⎢ ⎥ ⎢ ⎥ 1 ⎥ , G(ξ)θ = ξ = ⎢X ⎥ , K = ⎢ 1 (5.5) μF X ⎣ k3 ⎣G⎦ 0 ⎦ L 0 k5

Recursive Parameter Estimation

⎡ ⎤ DSin ⎢ Qin ⎥ ⎢ ⎥ ⎥ F =⎢ ⎢ 0 ⎥, ⎣ 0 ⎦ 0

117

⎡

⎤ 0 ⎢0⎥ ⎢ ⎥ ⎥ Q=⎢ ⎢0⎥ ⎣Q1 ⎦ 0

(5.6)

kj (j = 1 to 5) are the yield coefficients, Sin is the feed concentration of glucose (g/l), Qin is the feed rate of oxygen (g/l/h), Q1 is the outflow rate of CO2 (g/l/h) and μi (i = R, F ) are the specific growth rates (1/h) associated with each growth reaction. Let us now briefly discuss the various possibilities for the distribution of terms in G(x) and θ. The above model suggests the following distribution (this will actually be the one which will be considered below):   X 0 μ G(ξ) = , θ= R (5.7) 0 X μF However, there are other possible options, depending on the level of knowledge and on the uncertainty regarding the process kinetics. For example, we can consider: 1) that the whole vector for the reaction rate is unknown:  μR X G(ξ) = I, θ = (5.8) μF X 2) or that, depending on the kinetic laws, the reaction rates are explicitly linked to the limiting substrates (the simplest way to express this is to write μR = ρ1 SC and μF = ρ2 S), and to assume that the parameters ρi are unknown; thus we have:   SCX 0 ρ G(ξ) = , θ= 1 (5.9) ρ2 0 SX 3) or that we have certain knowledge about the structure of the kinetic models (e.g. Monod’s model) and that the maximum specific growth rate μmax,i (i = 1, 2) is unknown while the affinity constants KS1 , KS2 , KC1 are known; this will lead to the following definitions for G(x) and θ: ⎡ ⎤ SCX  0 μmax,1 ⎢ (KS1 + S)(KC1 + C) ⎥ G(ξ) = ⎣ (5.10) SX ⎦ , θ = μmax,2 0 KS2 + S 5.2.2. Estimator based on the structure of the observer Let us assume the following: – Assumption 1. The p parameters of θ are unknown and possibly time-varying (with bounded time variations: dθ dt < M ). – Assumption 2. Let p state variables be measured online.

118

Bioprocess Control

Starting from Assumption 2, let us define the partition of state as follows:  ξ ξ= 1 (5.11) ξ2 Here, ξ1 are the measured variables and ξ2 the unmeasured variables. The dynamic equations can now be rewritten as follows: dξ1 = K1 G(ξ)θ + F1 − Q1 − Dξ1 dt dξ2 = K2 G(ξ)θ + F2 − Q2 − Dξ2 dt

(5.12) (5.13)

Let us make the following assumptions as well: – Assumption 3. G is a diagonal matrix (G = diag{gi }, i = 1 to p) and is full rank for all the admissible values of ξ (in our case, this means essentially that only the positive values of the state variables (i.e. concentrations) are considered). – Assumption 4. K1 is full rank, and its coefficients are known. – Assumption 5. Feed rates F1 (associated with ξ1 ), gas outflows Q1 (associated with ξ1 ) and the dilution rate D are known (via online measurements or choice of the user). Under these assumptions, it is possible to construct an asymptotic observer for variables ξ2 independent of the knowledge about parameters θ (see Chapter 4). In continuation of the section, we will consider that the states included in ξ2 are either accessible to the online measurement or are available via an asymptotic observer or that the dynamics of ξ1 are independent of ξ2 . The concept of the estimator on the basis of the structure of the observer is based on equation (5.12) and actually follows a development similar to that of the Luenberger observer. This gives the following estimation equations: dξˆ1 = K1 G(ξ)θˆ + F1 − Q1 − Dξ1 − Ω(ξ1 − ξˆ1 ) dt dθˆ = [K1 G(ξ)]T Γ(ξ1 − ξˆ1 ) dt

(5.14) (5.15)

The essential motivation for determining the structure of the estimator is as follows. As for any traditional observer, the estimator equations are a combination of the process model equations (K1 G(ξ)θˆ + F1 − Q1 − Dξ1 ) and the terms of correction (−Ω(ξ1 − ξˆ1 ) and [K1 G(ξ)]T Γ(ξ1 − ξˆ1 )) on the measured variables, terms that help in arriving at the estimation. In the estimator based on the structure of the observer,

Recursive Parameter Estimation

119

parameters θ have the status of state variables without dynamics. The weighting factor [K1 G(ξ)]T in equation (5.15) is actually the term with which the unknown parameter in the equation of the model is multiplied: its introduction into the equations of the estimator is based on the concept of the traditional estimators (e.g. [GOO 77]) and is generally called a regressor. The estimator based on the structure of the observer was originally developed for estimating specific growth rates [BAS 86] and was subsequently extended to the online estimation of kinetic parameters [BAS 90]. This has found important roles over a large range of bioprocesses occurring in food, pharmaceutical and environmental applications (e.g. [ATR 88, BAS 90, BOU 00, CLA 98, OLI 96, FLA 89, POM 92, POM 95, SUL 97]). NOTE. It is obvious that we can further generalize the approach by considering the case in which certain kinetics are perfectly known and others are not known. The general dynamic model will thus take the following form: dξ = Ki Gi (ξ)θ + Kc φc (ξ) + F − Q − Dξ dt

(5.16)

Here, φc (ξ) is the vector for the known reaction rates and Ki and Kc are the yield coefficient sub-matrices associated with the unknown and known kinetics, respectively. The design of the estimator follows the same approach as that described above. 5.2.3. Example: culture of animal cells (continued) Let us assume that the glucose concentration S and lactate concentration L are measured online and that the aim is to estimate in real time the specific growth rates. Now the vectors and matrices are written in the following specific form:   −k1 X −k4 X S , K1 G(ξ) = ξ1 = (5.17) L 0 k5 X   DSin μ (5.18) F1 − Q1 = θ= R , μF 0 5.2.4. Calibration of the estimator based on the structure of the observer: theory The theoretical analysis concerning the stability of the estimator based on the observer’s structure is available in the paper [BAS 90]: the essential requirements for this stability are the defined negativity of ΩT Γ + ΓΩ and the persistent excitation of the signal K1 G(ξ), i.e.: 1) negative definiteness: ΩT Γ + ΓΩ < 0;

120

Bioprocess Control

2) persistent excitation: there are positive constants α and β such that αI ≤ ( t+β [K1 G(ξ)]T K1 G(ξ)dτ , for all t ≥ 0. t However, the calibration of the estimator is in practice a difficult problem that has to be solved, which is attributed to the strong interaction among the unknown parameters in the estimator equations and to the dependence of the dynamics on the process variables. This influence of latter constraint can be reduced if the process is made to work around a permanent regime; however, it will be crucial when the system involves a large spectrum of operating conditions (in particular, the case of the batch and fed-batch, reactors, as well as in the case of start-up, shut-down or setpoint change (change in the type of production, for example) of the process) with possibly large variations in the process variables and thus matrix G(ξ). Good performances in terms of tracking the variations of the estimated parameters is particularly important under these circumstances. However, in the above form (5.14)-(5.15) of the estimator, the adjustment of the concept parameters of estimator Γ and Ω may be very conservative and may result in bad performances with respect to tracking the variation in certain operating regions. The objectives of calibration for the estimator based on the structure of the observer mentioned above can be satisfied if we consider the following two stages in the reformulation of the algorithm: 1) a state transformation; 2) the rearrangement of the state vector entries of the estimator. State transformation Let us define the following state transformation: z = K1−1 ξ

(5.19)

Now the dynamic equations of the process can be rewritten as follows: dz = G(z)θ + K1−1 (F1 − Q1 − Dz) dt

(5.20)

With the above transformation, one and only one variable is associated with each unknown parameter θ. In our case, the invertibility of matrix K1 is a result of the independence of the p reactions and p measured variables (see also [BAS 90]). The above transformation has already been proposed in [POM 90, OLI 96]. We can now proceed with a new concept of the estimator based on the structure of the observer on the basis of equation (5.20): dˆ z = G(z)θˆ + K1−1 (F1 − Q1 − Dz) − Ω(z − zˆ) dt

(5.21)

Recursive Parameter Estimation

dθˆ = Γ(z − zˆ) dt

121

(5.22)

With transformation (5.19), the estimator based on the structure of the observer has its equations reformulated in a decoupled format for the unknown parameters θi (i = 1 to p). Due to the decoupled formulation of the estimation, an immediate choice for matrices Ω and Γ are diagonal matrices: Ω = diag{−ωi },

Γ = diag{γi },

ωi > 0, γi > 0, i = 1 to p

(5.23)

In this formulation for the estimation algorithm, we eliminated regressor G(z) from the estimation equations of θ (5.22): as one of its main roles consists of transferring explicitly the coupling between the unknown parameters and the measured variables, it is no longer essential here. Rearrangement of the state vector elements of the estimator The final stage before the formulation of the rule for the calibration of the estimator consists of rearranging the equations of the estimator. Let us initially regroup each variable zi with parameter θi to which it is linked and rearrange the elements of the  T vector z, θ in the following order in vector ζ: ⎡ ⎤ z1 ⎢ θ1 ⎥ ⎢ ⎥ ⎢ z2 ⎥ ⎢ ⎥ θ ⎥ (5.24) ζ=⎢ ⎢ .2 ⎥ ⎢ .. ⎥ ⎢ ⎥ ⎣z ⎦ p θp Rule for base calibration Let us define the estimation error as follows: e = ζ − ζˆ

(5.25)

The dynamics of the estimation error are obtained immediately from equations (5.20), (5.21) and (5.22): de = Ae + b dt Here, A is a block-diagonal matrix with the dimension blocks 2 × 2:  −ωi gi (z) A = diag{Ai }, Ai = , i = 1 to p 0 −γi

(5.26)

(5.27)

122

Bioprocess Control

Here b is equal to: ⎡

⎤ 0 ⎢ dθ1 ⎥ ⎢ ⎥ ⎢ dt ⎥ ⎢ ⎥ ⎢ 0 ⎥ ⎢ ⎥ ⎢ dθ2 ⎥ ⎥ b=⎢ ⎢ dt ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ . ⎥ ⎢ ⎥ ⎢ 0 ⎥ ⎣ ⎦ dθp dt

(5.28)

The characteristic equation of matrix A, det(λI − A), is equal to: det(λI − A) =

p )

(λ2 + ωi λ + γi gi (z))

(5.29)

i=1

The key idea of the calibration rule consists of choosing each γi in an inversely proportional manner to the corresponding term gi (z): γi =

γ¯i , gi (z)

γ¯i > 0, i = 1 to p

(5.30)

With this choice, characteristic equation (5.29) is rewritten as follows: det(λI − A) =

p )

(λ2 + ωi λ + γ¯i )

(5.31)

i=1

and the dynamics of the estimator based on the structure of the observer is now independent of the state variables of the process. Such a choice actually corresponds to a Lyapunov transformation (see [PER 93]). This is obviously valid for the values of gi (z) = 0: this condition is usually easily encountered in bioprocess applications, as will be illustrated in the next section. The values of the calibration parameters can be selected in a way so as to fix the dynamics of the estimator for each unknown parameter θi . As the estimator is reduced, via the transformations, to a set of second order linear systems, one being independent of another, the traditional rules for assigning the dynamics of a set of second order linear systems are immediately applicable here. We therefore suggest to the reader to have a look at standard automatic control textbooks for more detailed information on the subject. Nevertheless, we suggest below the following basic guidelines.

Recursive Parameter Estimation

123

An important basic choice consists of considering real poles for the dynamics of the estimator: γi ≥ 0 ωi2 − 4¯

(5.32)

Now the objective is to avoid in the estimation of the parameters inducing oscillations that do not correspond to any physical phenomenon associated with the estimated reaction rates. In this approach, Pomerleau and Perrier [POM 90] suggest the choice of double poles, i.e.: γ¯i =

ωi2 4

(5.33)

In this case, the adjustment of the estimator is reduced to the choice of a single calibration parameter ωi by the estimated parameter. This enables us to have a procedure for adjustment that is both simple (a single parameter for adjustment) and flexible (each estimator of the parameter can be calibrated in different ways if necessary, for example when the variations of the estimated parameters are different). Up to now, we have suggested that it is possible to assign freely the dynamics of the estimator. However, in the presence of noisy data, it appears that a compromise should be made between a rapid convergence of the estimator and a good attenuation of the measurement noise. A detailed analysis is performed in the paper by Bastin and Dochain [BAS 90] (pp. 162–172), which studies the performances of the estimator based on the structure of the observer both in theory and in numerical simulation in the presence of bounded noisy data in the particular case of the estimation of the specific growth rate in a simple microbial growth process. This is characterized by the dynamic material balance equations as follows: dX = μX − DX dt dS = −k1 μX + DSin − DS dt

(5.34) (5.35)

The theoretical analysis of optimization is based on the evaluation of asymptotic properties of the estimator and gives as a result the following optimum value for ω1 : * k1 M1 ω1,opt = 2 , 0<α<1 (5.36) 2 α(k1 M2 + Smax M2 ) Here, M1 and M2 are the upper limits of the time derivative μ and the measurement noise, respectively, and Smax is the maximum value of the substrate concentration in the feed Sin . This result is probably more on the conservative side insofar as

124

Bioprocess Control

it is based on the upper limits of the measurement noise, on the temporal variation of μ and on the substrate concentration in the feed. However, it has been qualitatively confirmed by numerical simulation studies, which also give a value for calibration parameters that minimizes the estimation error. As this value is conservative, the theoretical optimum should be interpreted carefully; however, because it is qualitatively correct, our suggestion (also based on our hands-on experience), in the presence of measurement noise, is to perform the numerical simulations using a plausible kinetic model with simulated noise so as to obtain an initial first set for the calibration parameters; these parameters will be adjusted once the estimator has been applied to the process. 5.2.5. Calibration of the estimator based on the structure of the observer: application to the culture of animal cells In our example involving the animal cell culture, transformation (5.19) is written as: ⎡ ⎤ k4 S  − L − ⎢ k1 z k1 k5 ⎥ ⎥ z= 1 =⎢ ⎣ ⎦ z2 L k5

(5.37)

This leads to the following formulation for the estimator based on the structure of the observer: ⎧ 1 dˆ z1 ⎪ ⎪ =μ ˆR X − Dz1 − DSin + ω1 (z1 − zˆ1 ) ⎨ dt k1 (5.38) ⎪ dˆ μ R ⎪ ⎩ = γ1 (z1 − zˆ1 ) dt ⎧ z2 ⎪ ⎪ dˆ =μ ˆF X − Dz2 + ω2 (z2 − zˆ2 ) ⎨ dt (5.39) ⎪ μF ⎪ ⎩ dˆ = γ2 (z2 − zˆ2 ) dt In order to illustrate the performances of the estimator based on the structure of the observer and on the calibration rule proposed, we will show some numerical simulations. These have been calculated on the basis of the dynamic model for the animal cell culture process presented above. The process is run in the fed-batch, mode, i.e., with a variable volume V for which the dynamics are given by the balance equation: dV = Fin dt

(5.40)

Recursive Parameter Estimation

125

Here, Fin is the feed rate, which is related to the dilution rate as follows: D = Fin /V . The simulation model necessitates the use of kinetic models for estimating the specific growth rate; these have been determined experimentally as having qualitatively the following structures [SIE 99]: μR = μmax,1

KL S KR + S KL + L

(5.41)

μF = μmax,2

S KF + S

(5.42)

i.e., Monod structures with an inhibition of the respiration by the lactate. The model parameters are equal to those in [SIE 99]: μmax,1 = 0.055 h−1 ,

KR = 10 mM,

KL = 50 mM

μmax,2 = 0.045 h−1 ,

KF = 10 mM,

k1 = 1.7,

k4 = 8.5,

k5 = 17

The variables were initialized for the numerical simulation with the following values: X(0) = 0.18 106 cells/ml, V (0) = 19 L,

S(0) = 21 mM,

Vf = 19.05 L,

L(0) = 0.13 mM

F = 0.5 mL/h,

Sin = 3.3 M

(5.43) (5.44)

The duration of the simulated operation of the reactor is 110 hours, i.e. 100 hours with a constant flow and 10 hours in batch. The value of the biomass concentration X is given via an asymptotic observer having the following form: dZ 1 = −DZ + DSin dt k1 ˆ = Z − S − k4 − k1 L X k1 k1 k5

(5.45) (5.46)

The initial estimation values for the specific growth rates are fixed as follows: μ ˆR (0) = 0 h−1 ,

μ ˆF (0) = 0 h−1

(5.47)

In the numerical simulations, Z has been initialized by considering 20% error on the initial value of the biomass, i.e. Z(0) = 1.2 X(0)+S(0)/k1 +(k4 −k1 )L(0)/k1 /k5 . The presentation is identical for Figures 5.1, 5.3 and 5.4: a and b show the glucose and lactate concentrations, respectively; c and d show the results of estimation for the specific growth rates μR and μF , respectively; e shows the biomass concentration (the simulation values in Figures 5.1e and 5.4e, the offline data in Figure 6e, compared with the estimation given by the asymptotic observer; and Figure 5.3e also shows the validation of the estimation by the estimator based on the structure of the observer, which is described above), 5.3f gives the volume.

126

Bioprocess Control

15

L (mM)

S (mM)

20

10

40 20

5 0

0 0

20

40

60

80

0

100

20

40

60

80

100

0.03

: ω1 = 10 : ω1 = 1

0.03 0.02

μF (h−1)

μR (h−1)

0.04

0.01

0.01 0

0 0

20

40

60

80

: ω2 = 10 : ω2 = 1

0.02

100

0

20

40

0

20

40

60

80

100

60

80

100

4

: estimation 2

V (l)

X (106 cells/ml)

19.06 6 19.04 19.02 19

0 0

20

40

60

80

100

t (h)

t (h)

Figure 5.1. Numerical simulation of the estimator based on the structure of the observer with different gains

0.04

μF (h−1)

μR (h−1)

0.03 0.03 0.02 0.01 0

0

20

40

60

t (h)

80

100

0.02 0.01 0

0

20

40

60

80

100

t (h)

Figure 5.2. Comparison of the uncoupled design (- -) with the traditional design (. -) for ωi = 10

Figure 5.1 shows the convergence for different values of ω1 and ω2 (1 h−1 , 10 h−1 ). It should be mentioned that, as expected, the growth rate increases with ωi . Figure 5.2 illustrates the advantage of the design developed above (5.30) in comparison with the “traditional” design (5.15). In this numerical simulation, the same values of calibration parameters are considered: ωi = 10, γi = γ¯i = 25 (i = 1, 2). Note that if the performances of each design are similar in the second part of the numerical simulations, those of the “traditional” design are worse than those with pole placement. Figure 5.3 illustrates the performances in the presence of a

Recursive Parameter Estimation 50

a

20

b

40 15

L (mM)

S (mM)

127

10 5

30 20 10

0

0 0

20

40

60

80

0

100

20

40

60

80

100

0.1

0.05

: estimation

c

μF (h−1)

μR (h−1)

0.08 0.06 0.04

: estimation

0.02 0

0 20

40

60

80

100

0

20

40

60

80

100

19.06

6

e

4

V (l)

X (106 cells/ml)

0

d

: estimation 2 0 0

20

40

60

80

100

t (h)

f

19.04 19.02 19

0

20

40

60

80

100

t (h)

Figure 5.3. Numerical simulation of the estimator based on the structure of the observer with a multi-slot variation of the specific rates of maximum growth

square wave signal for the specific rates of maximum growth μmax,1 (0.055 → 0.155 at the instants t = 20, 40, 60 and 80) and μmax,2 (0.045 → 0.145 at the instants t = 30, 50, 70 and 100), with ω1 and ω2 being equal to 10 h−1 : it should be mentioned that even though X increases by more than one order of magnitude (0.18 → 6) over the entire duration of the process operation, the convergence speed remains the same and is not affected by the value of X. 5.2.6. Experimental results The estimator based on the structure of the observer (5.38), (5.39), (5.45) and (5.46) was applied to a 22 liter pilot bioreactor [SIE 99]. Figure 5.4 presents a set of experimental results. Glucose (Figure 5.4a) and lactate (Figure 5.4b) are measured via an FIA type biosensor (flow injection analysis). The temperature and pH are maintained at constant values of 37°C and 7.2, respectively. The initial conditions are similar to those considered for the numerical simulations. During the experiment, glucose was controlled at a set-point value equal to 1 mM by a linearizing adaptive controller (for more details on the method, see Chapter 7). The estimator based on the structure of the observer was initialized as follows: ˆF (0) = 0.01 h−1 , μ ˆR (0) = μ

ω1 = ω2 = 0.325 h−1

(5.48)

128

Bioprocess Control 20 15

L (mM)

S (mM)

1.5

1

0.5 0

50

100

10 5 0

150

0

50

100

150

0

50

100

150

0

50

100

150

0.05 0.04

μF (h−1)

μR (h−1)

0 20.05 20.1 20.15 20.2

0 0

50

100

150

8

19.1 : asymptotic observer o : offline data : validation

6 4

V (l)

X (g/l)

0.02

19.05

2 0

0

50

100

150

19

t (h)

t (h)

Figure 5.4. Estimator based on the structure of the observer: experimental results for a culture of animal cells

It should be mentioned that in this particular case, the same initial conditions and the same calibration parameters have been used and that they have given satisfactory results for both the estimators. Due to the lack of reliability of the models for estimating specific growth rates, the validation for the estimation of parameters was performed on the basis of the biomass data. Figure 5.4 compares the offline biomass data (o) with the estimation Xv (dashed line) of the biomass based on the material-balance equation of model (5.5)-(5.6) and reconstructed from the values estimated online for ˆF : the specific growth rates μ ˆR and μ dXv =μ ˆ R Xv + μ ˆF Xv − DXv dt

(5.49)

Note that the estimator based on the structure of the observer detects the decrease in lactate in the vicinity of instant t = 120 h and thus gives a negative value to μ ˆF . It should also be mentioned that μR is pushed towards negative values at the beginning of the culture, which is largely due to the measurement noise. The choice of values for the calibration parameters typically follow the above procedure. Numerical simulations have been used in order to give a first initial set of values based on the plausible measurement noise and on the specific growth rate models.

Recursive Parameter Estimation

129

0.05 0.04

μF (h−1)

μR (h−1)

0 20.05 20.1 20.15 20.2

0.02 0

0

50

100

150

0

50

100

150

X (g/l)

8 : asymptotic observer o : offline data : validation

6 4 2 0

0

50

100

150

t (h)

Figure 5.5. Experimental results for the non-optimum calibration parameters

They have been subsequently adjusted so as to obtain the best possible validation using the experimental biomass data. It should also be stated that the sensitivity of the calibration procedure can vary widely with applications. Experience shows that the hydrodynamics play a key role in this sensitivity. Typically, the calibration is easy and the choice of the calibration parameters is a lot less sensitive for the processes in which the hydrodynamic term is quantitatively important with respect to the kinetic term, i.e. for the fed-batch processes with a high feed rate – and thus large variations in volume – (typical examples are the applications in the yeast growing processes as presented in [CLA 98, POM 90, POM 92]) and continuous processes. The calibration will be longer, especially on account of the high sensitivity of the estimation results with respect to the values of the calibration parameters, for the batch processes and fed-batch processes with a low feed rate. Our example of animal cell culture is of particular interest here insofar as it belongs to the latter class of processes. Figure 5.5 shows the results of estimation in the case where ωi (i = 1, 2) is equal to 0.25 (i.e. 23% of variation with respect to the best results): it should be noted that the estimated specific growth rates have less variation and they do not generate an acceptable validation over the biomass data using this set of calibration parameters. 5.3. Recursive least squares estimator Let us again proceed from the general dynamic model (5.2) as presented in the previous section. As already seen, this model is linear with respect to the unknown parameters θ. From then on, an alternative solution for the parametric estimation consists of using a linear regression method, for example the least squares method, in its recursive form.

130

Bioprocess Control

In order to present the algorithms as simply as possible, we will start from the discrete time form of the general dynamic model (an approach based on the general dynamic model in its differential equation form is given in [BAS 90]). Let us now consider the linear equation for the parameters θ, assumed to be unknown, as follows: yt = φTt θt

(5.50)

Here, t is the time index. The algorithm for the recursive least squares (RLS) with the forgetting factor based on equation (5.50) is written in the following form (see for example [KWA 72] for a detailed approach for the derivation of the recursive form from the non-recursive form): θˆt+1 = θˆt + gt φt (yt − φTt θˆt ) '  −1 T gt−1 & g0 > 0, 0 < γ ≤ 1 I − φt γI + φTt gt−1 φt φt gt−1 gt = γ

(5.51) (5.52)

Here, I is the identity matrix and gt is the gain of the estimator. The forgetting factor has the role of enabling the tracking of the possible variations of unknown parameter θ by giving more weight to the recent data in preference to the old data. γ = 1 means that an equal weight is given to all the data and that there is no forgetting effect. The value of the forgetting factor is usually (but not necessarily) chosen between 0.9 and 0.99 in practical applications. Let us now apply the RLS algorithm to the estimation of the kinetic parameters on the basis of our general dynamic model (5.2). If we consider a Euler approximation for the time derivative: ξt+1 − ξt dξ = dt Δt

(5.53)

with Δt being the sampling period, model (5.2) is written in a discrete form as follows: ξt+1 = ξt + Δt(KG(ξt )θt (ξt ) − Dt ξt − Qt + Ft )

(5.54)

The RLS algorithm is applied immediately by considering that: yt = ξt+1 − ξt + Δt(Ft − Qt − Dt ξt ),

φTt = ΔtKG(ξt )

(5.55)

The selection of initial gain g0 is typically an important choice for starting the RLS estimator correctly. It is obvious that the gain should be positive. Usually, the literature suggests starting with a diagonal matrix with large values. However, in practice, very high values often induce very large transients when the algorithm is at the initial stage of application. Our recommendation for the RLS algorithm with forgetting factor is to

Recursive Parameter Estimation

131

select initial values for the gain matrix elements in such a manner that they are of the same order of magnitude as reached in steady state. This is easy to implement in the case of estimation of a single parameter. Actually, gain matrix gt (in the same manner as for regressor φt ) is modified such that it becomes scalar and is written in the following form: gt =

gt−1 γ + gt−1 φ2t

(5.56)

In steady state, gt = gt−1 , and from now on, we can write the following expression for gt : gt =

1−γ φ2t

We now select the initial value g0 in the vicinity of

(5.57)

1−γ . φ20

Application to an anaerobic digestion process In this example, we will consider the simplified two-stage reaction scheme for anaerobic digestion: S1 −→ X1 + S2 + P1

(acidogenesis)

(5.58)

S2 −→ X2 + P1 + P2

(methanization)

(5.59)

Here, S1 , S2 , X1 , X2 , P1 and P2 represent the organic matter, volatile fatty acids, acidogenic bacteria, methanogenic bacteria, carbon dioxide and methane, respectively. The associated general dynamic model is given by the following vectors and matrices: ⎡ ⎡ ⎡ ⎤ ⎤ ⎡ ⎤ ⎤ −k1 DSin 0 S1 0 ⎢ k3 −k2 ⎥ ⎢ 0 ⎥ ⎢0⎥ ⎢ S2 ⎥ ⎢ ⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎥ ⎢ 1 ⎢ ⎢ ⎥ ⎢X1 ⎥ ⎥ 0 ⎥ ⎢ ⎥ ⎥, F = ⎢ 0 ⎥, Q = ⎢ 0 ⎥ (5.60) ξ=⎢ ⎢ ⎢0⎥ ⎥ ⎢X2 ⎥ , K = ⎢ 0 ⎥ 1 ⎥ ⎢ ⎢ 0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ k4 ⎣ 0 ⎦ ⎣Q1 ⎦ ⎣ P1 ⎦ k5 ⎦ P2 0 k6 Q2 0   ρ μ X ρ= 1 = 1 1 (5.61) ρ2 μ2 X2 Let us consider the case in which methane P2 and volatile fatty acids S2 are accessible for online measurement. We will make use of these measurements for estimating online the specific growth rates μ1 and μ2 of acidogenesis and methanization.

132

Bioprocess Control

In discrete time and on considering the hypothesis of the low solubility of methane, the material balance with regard to methane and volatile fatty acids can be written as follows: Q2t = k6 μ2t X2t

(5.62)

S2,t+1 = S2t − ΔtDt S2t + k2 μ1t ΔtX1t − k3 μ2t ΔtX2t

(5.63)

As μ2 appears in both equations, whereas μ1 appears only in the second, we made a cascade design of the estimators. At each stage, we proceeded as follows: 1) estimation of μ2 on the basis of equation (5.62); 2) estimation of μ1 on the basis of equation (5.63) and using the estimated value of μ2 as given in stage 1. The RLS estimator was implemented on a 60-liter pilot reactor at the Biological Engineering Unit of the Université Catholique de Louvain (Louvain-la-Neuve, Belgium) ([BAS 90]). The yield coefficients are equal to: k1 = 3.2,

k2 = 28.6,

k3 = 5.7,

k6 = 27.3

(5.64)

Sin (gDCO/l)

QCH4 (1/h)

S2 (gDCO/l)

Figure 5.6 shows the data used for the online estimation (volatile fatty acids S2 , flow rate of methane Q2 (marked as QCH4 in the figure), concentration of organic

4 2 0 0

20

40

60

80

100

120

0

20

40

60

80

100

120

0

20

40

60

80

100

120

0

20

40

60

80

100

120

2 1 0

40 20 0

D (h−1 )

0.1 0.05 0

t (days)

Figure 5.6. Experimental data of anaerobic digestion (unit GEBI, UCL, Belgium)

Recursive Parameter Estimation

133

μ1 (h−1)

0.1

0.05

0

0

20

40

60

80

100

120

0

20

40

60

80

100

120

60

80

100

120

μ2 (h−1)

0.1

0.05

0

X (gDCO/l)

10 o : Offline analysis 5

0

0

20

40

t (days)

Figure 5.7. Recursive estimation of the specific growth rates in an anaerobic digestion process

matter in the feed Sin and dilution rate D). The estimation results are shown in Figure 5.7. The biomass concentration values X1 and X2 are given by an asymptotic observer. The RLS estimators were initialized as follows: estimation of μ2 :

yt = Q2t , φt = k6 X2t , γ = 0.9, g0 = 0.01

(5.65)

estimation of μ1 :

yt = S2,t+1 − S2t + ΔtDt S2t + k3 μ ˆ2t ΔtX2t

(5.66)

φt = k2 ΔtX1t , γ = 0.9, g0 = 0.01

(5.67)

The estimated values μ ˆ1 and μ ˆ2 were validated on considering the offline data for total and soluble COD (chemical oxygen demand). The difference between these two measurements turns out to be a reasonably acceptable measurement of the total biomass concentration X1 + X2 in the reactor in this specific case. These data were compared with the values calculated on the basis of the estimated values for μ1 and μ2 using the balance equation for the total biomass X1 + X2 : d(X 1 + X2 ) = −D(X ˆ 1 X1 + μ ˆ 2 X2 1 + X2 ) + μ dt

(5.68)

5.4. Adaptive state observer Up to now, we have seen how to estimate online the concentrations of components (via state observers, developed in Chapter 4) and process parameters (via two types of

134

Bioprocess Control

estimators developed above). However, it might be of interest to combine the two types of estimation within the same algorithm. This is what we will (try to) do now. The term “try to” in the above sentence is not purely trivial; it actually reveals the state of the art as of now. The question of the joint estimation of the states and parameters is far from being resolved at present. The adaptive observer, for example, has been the subject matter of numerous developments (see, for example, [NAR 89, CHE 90, BAS 90]). Although the theoretical results appear to be of interest, their implementation has not yielded the desired results. The objective of this section consists of developing a joint adaptive observer for both states and parameters in the case for which the structure of the kinetic models is assumed to be known, and along with the estimation of state, a parameter of kinetic models is estimated online. In this sense, this observer is placed somewhere between the “traditional” observers of state, of the Kalman and Luenberger type (for which it is necessary to know the structure of the kinetic model and the value of its parameters), and the asymptotic observer (which does not require any knowledge of the kinetics). One of the key points with regard to stability analysis of the present observer is the use of “nominal” kinetic models (i.e., with nominal values (by “default”) of parameters around which the values of not well-known parameters are assumed to fluctuate). For simplification, we will assume here that the not well-known parameters are, similar to the process models, linear. We can now write the dynamic equations of the model in the following form: dx = f (x, u, θ) + g(x, u) dt

(5.69)

¯ + b(x, u)Δθ f (x, u, θ) = f¯(x, u, θ)

(5.70)

with f (x, u, θ) equal to:

Proceeding from the above equations and on considering the not well-known parameters (here Δθ) as states with zero dynamics ( dΔθ = 0), we can use the dt basic observer equations (see Chapter 4) for writing the equations for our adaptive observer: dˆ x ¯ + b(ˆ + + g(x, u) + K(y − yˆ) = f¯(ˆ x, u, θ) x, u)Δθ dt

(5.71)

+ dΔθ = Γ(y − yˆ) dt

(5.72)

It is necessary to make, at this stage, certain important observations. Even though the adaptive observers design was the subject matter of active research papers in the 1980s, subsequently the theoretical analysis of these algorithms has quickly proved to be very complex and their application to (bio)processes is not yielding the desired results. This is why we limit ourselves here to the following points:

Recursive Parameter Estimation

135

1) basing the design on a model with nominal values of not well-known parameters; 2) considering only linear models for the not well-known parameters; 3) considering only one not well-known parameter per measured variable; 4) considering the so-called averaging (see for example [MAR 96]) with respect to the adaptive control as a key calibration rule for the adaptive observer. The first point is important for increasing flexibility in the choice of the observer dynamics (see analysis below). The second point is an a priori choice, which simplifies the approach and the theoretical analysis (but we will nevertheless illustrate the observer in the case of not well-known parameters, which are nonlinear in the model). The third point is probably the most important from the point of view of the guarantee of success of its implementation; it can be easily understood from a practical point of view: how can we expect to be in a position to treat simultaneously in a correct manner the uncertainty over many kinetic parameters and make the distinction between each of them when there is not adequate information about them, in particular one piece of independent information per unknown parameter. Finally, the fourth point can be decisive in the matter of implementation of the adaptive observer: one important underlying idea is that the dynamics of the parametric adaptation should be slower than the dynamics of the rest of the observer. For illustrating the results, we will consider the example of a simple microbial growth with a substrate S and a biomass X. The dynamic balance equations in a stirred tank reactor are written down as follows: dS = −k1 μX + DSin − DS dt dX = μX − DX dt

(5.73) (5.74)

Here, k1 , μ, D, and Sin represent the performance coefficient, the specific growth rate (h−1 ), the dilution rate (h−1 ) and the substrate feed concentration (g/l), respectively. Let us consider Monod’s kinetics for μ: μ = μmax

S KS + S

(5.75)

with μmax being the rate of maximum specific growth (h−1 ) and KS the saturation constant (g/l). Let us assume that μmax is the not well-known parameter. We will then write: ¯max + Δμmax μmax = μ

(5.76)

136

Bioprocess Control

Here, μ ¯max is the nominal value of μmax . Equation (5.70) is written down as follows: SX 1 SX dS = −k1 μ + DSin − DS − Δμmax ¯max dt KS + S Y KS + S

(5.77)

dX SX SX =μ ¯max − DX + Δμmax dt KS + S KS + S

(5.78)

At this stage, we will consider an asymptotic observer for estimating the value of X, which appears in the regressor of Δμmax . The introduction of the asymptotic observer is motivated by the theoretical analysis below. As we have seen in the preceding chapter, it is based on the transformation of state: Z = S + k1 X. Its dynamic equation is equal to: dZ = −DZ + DSin dt

(5.79)

The state estimate Xe of X is readily derived from these two equations and is written in the following form: Xe =

Z −S k1

(5.80)

Equations (5.79) and (5.80) are thus the asymptotic observer in our example. The adaptive observer is thus written down as: ˆ SX SXe  dSˆ ˆ = −k1 μ + DSin − DS − k1 Δμmax + K1 (S − S) ¯max dt KS + S KS + S (5.81) ˆ ˆ dX SX ˆ + SXe Δμ ˆ  =μ ¯max − DX max − K2 (S − S) dt KS + S KS + S  dΔμ max ˆ = −γ(S − S) dt

(5.82)

(5.83)

The analysis of theoretical properties of stability and convergence of the adaptive observer is based on the dynamic error equations. If we define: ˆ eX = X − X, ˆ eμ = Δμmax − Δμ  eS = S − S, max

(5.84)

these are written down as follows: de = Ae + BeXe dt

(5.85)

Recursive Parameter Estimation

with:

⎡ μ ¯max S SXe ⎤ −K1 −k1 −k1 ⎢ KS + S KS + S ⎥ ⎥ ⎢ ⎥, ⎢ SXe μ ¯max S A =⎢ ⎥ K − D ⎦ ⎣ 2 KS + S KS + S γ 0 0

⎡

⎤

eS e = ⎣eX ⎦ , eμ

137

⎡ Δμmax S ⎤ −k1 ⎢ KS + S ⎥ ⎥ ⎢ ⎢ B = ⎢ Δμmax S ⎥ ⎥ ⎣ KS + S ⎦ 0

eXe = X − Xe From the theory of asymptotic observers, we know that eXe will asymptotically tend towards zero, limt→∞ eXe = 0. We also know that (see for example [BAS 90]) matrix B is bounded. In a similar way, if K1 , K2 and γ are bounded, then A is bounded. From then on, the error dynamics are a time-varying system with an input, eXe , which tends asymptotically towards zero. If matrix A is asymptotically stable, we can use a traditional stability result (e.g. [WIL 70, p. 55]) in order to state that the estimation errors will asymptotically tend towards zero. Let us now verify that A is a stable matrix. Its characteristic polynomial det(λI − A) is equal to:  μ ¯max S 3 2 λ + λ K1 + D − KS + S   μ ¯max S SXe μ ¯max S + γk1 + λ K1 D − + K2 k1 KS + S KS + S KS + S + γk1 D

SXe KS + S

It is now easy to verify that A is asymptotically stable using a suitable choice of gains K1 , K2 and γ, assigning, for example, the dynamics of the observer with three eigenvalues λ1 , λ2 and λ3 . Figures 5.8 and 5.9 illustrate the performances of the observer under the following conditions: Y = 0.5, μmax = 0.33 h−1 , KS = 5 g/l, Sin = 5 g/l, D = 0.05 h−1 X(0) = 1 g/l, S(0) = 0.5 g/l

(Figure 5.8)

X(0) = 2.054 g/l, S(0) = 0.893 g/l

(Figure 5.9)

ˆ Figure 5.8 shows the case of an initial transient, where S(0) = S(0) and ˆ ˆ X(0) = 0. In Figure 5.9, the estimation variables have been initialized at: S(0) = ˆ S(0) and X(0) = 1 g/l and a square wave signal was added for Sin (between 5 and 6 g/l, variations at instants t = 40, 80, 120 and 160 h) and an error of 10% over μmax . The gains were at first chosen in such a way as to assign the dynamics with

138

Bioprocess Control 2.5

X (g/l)

2 1.5

: simulated value : adaptive observer : asymptotic observer

1 0.5 0 0

10

20

30

40

50

60

70

80

90

100

0

10

20

30

40

50

60

70

80

90

100

80

90

100

S (g/l)

1.5

1

0.5

∆ μmax (h−1)

100

∆ μmax = 0.03 h−1

50

0 0

10

20

30

40

50

60

70

t (h) Figure 5.8. Adaptive observer (initial transient)

λ1 = λ2 = λ3 = 5 h−1 , and then γ was reduced by a factor of 20 in order to follow the averaging recommendations (the factor 20 is rather arbitrary (it could very well have been 10); it just gives an approximate idea of a typical value for γ). Note that the adaptive observer (which assumes the knowledge of the kinetic model) is capable of converging more rapidly than the asymptotic observer (which ignores the kinetic model). It should also be mentioned that the adaptive observer (5.81), (5.82) and (5.83) is capable of managing the nonlinear uncertainties such as those over KS , as has been illustrated in Figure 5.10, which shows the results of a numerical simulation produced under the same conditions as in Figure 5.8, but with 10% error over KS . The adaptive observer compensates for the error over KS using the estimated value of μmax . 5.4.1. Generalization The generalization of the adaptive observer is based on the general dynamic model: dξ = −Dξ + Kφ + F − Q dt

(5.86)

Recursive Parameter Estimation

139

X (g/l)

2 : simulated value : adaptive observer : asymptotic observer

1 0 0

20

40

60

80

100

120

140

160

180

200

0

20

40

60

80

100

120

140

160

180

200

100

120

140

160

180

200

1.1

S (g/l)

1 0.9 0.8

∆ μmax (h−1)

0.7

0.6

∆ μmax = 0.03 h−1

0.4 0.2 0 0

20

40

60

80

t (h) Figure 5.9. Adaptive observer (pulse signal for Sin )

If we consider that the measured output variables y are components of the process, then the part of the general dynamic model associated with y is written down as follows: dy = −Dy + Ky φ + Fy − Qy dt

(5.87)

Here, Ky , Fy and Qy form the group of lines of K, F and Q dependent on y. The key hypotheses at this level consist of considering that the number of components measured is (at least) equal to the number of not well-known kinetics and that the measured components are independent (which basically requires that Ky is a fullrank matrix). The generalization thus consists of using transformation (5.19), which has already been used in section 5.2 and enables us to associate a single variable with every unknown kinetic parameter. Now we immediately extend the design of the adaptive observer to every decoupled sub-system as was done in section 5.2.

140

Bioprocess Control

X (g/l)

2 : simulated value : adaptive observer : asymptotic observer

1 0 0

20

40

60

80

100

120

140

160

180

200

0

20

40

60

80

100

120

140

160

180

200

100

120

140

160

180

200

1.1

S (g/l)

1 0.9 0.8

∆ μmax (h−1)

0.7

0.4

∆ Ks = 0.5 g/1

0.2 0 0

20

40

60

80

t (h)

Figure 5.10. Adaptive observer (10% error over KS )

5.5. Conclusions In this chapter, we have introduced methods enabling us to estimate online the kinetic parameters, either singly (estimator based on the structure of the observer and estimator through recursive least squares), or in combination with the estimation of state variables of the process (adaptive observer). For the first as well as the last case we have proceeded towards a theoretical analysis of the properties of stability and convergence and have insisted on the laws of adjustment, enabling us to facilitate a good practical implementation of the estimation algorithms. Here we have concentrated solely on the estimation of kinetic parameters. This proceeded partly due to a deliberate choice, i.e., insisting on the need for developing the methods of suitable parametric estimation first and foremost in order to estimate the parameters of the bioprocess models, which usually have the largest concentration of uncertainties. This also enabled us to simplify the approach by concentrating on only one type of parameter. Having said this, it is obvious that whatever has been written can be extended, mutatis mutandis, to any other parameter of the bioprocess models, be they yield parameters, transfer parameters and so on. The reader can find examples for the development and application of methods for the estimation of other parameters in [BAS 90].

Recursive Parameter Estimation

141

5.6. Bibliography [ATR 88] D. ATROUNE, A. C HERUY, J.P. F LANDROIS and G. C ARRET, “Choice of the specific growth rate formulation in biotechnological processes”, in Proc. IMACS 12th World Congress on Scientific Computation, 1988. [BAS 86] G. BASTIN and D. D OCHAIN, “On-line estimation of microbial growth rates”, Automatica, vol. 22, no. 6, pp. 705–711, 1986. [BAS 90] G. BASTIN and D. D OCHAIN, On-line Estimation and Adaptive Control of Bioreactors, Elsevier, Amsterdam, 1990. [BOU 00] S. B OURREL, D. D OCHAIN, J.P. BABARY and I. Q UEINNEC, “Modelling, identification and control of a denitrifying biofilter”, Journal of Process Control, vol. 10, pp. 73–91, 2000. [CHE 90] Y.H. C HEN, “Adaptive robust observers for nonlinear uncertain systems”, Int. J. Systems Sci., vol. 21, no. 5, pp. 803–814, 1990. [CLA 98] J.E. C LAES and J. VAN I MPE, “On-line monitoring and optimal adaptive control of the fed-batch baker’s yeast fermentation”, in Proc. CAB7 7th Int. Conf. Computer Appl. in Biotechnol., pp. 405–410, 1998. [FLA 89] J.M. F LAUS, M.N. P ONS, A. C HERUY and J.M. E NGASSER, “Adaptive algorithm for estimation and control of fed-batch bioprocesses”, in Proc. Computer Appl. in Ferment. Technol.: Modelling and Control of Biotechnol. Processes, M.N. F ISH, R.I. F OX and N.F. T HORNHILL (eds.), pp. 349–353, 1989. [GOO 77] G.C. G OODWIN and R.L. PAYNE, Dynamic System Identification: Experiment Design and Data Analysis, Academic Press, New York, 1977. [KWA 72] H. K WAKERNAAK and R. S IVAN, Linear Optimal Control Systems, John Wiley, New York, 1972. [MAR 96] I. M AREELS and J.W. P OLDERMAN, Adaptive Systems. An Introduction, Birkhauser, Boston, 1996. [NAR 89] K.S. NARENDRA and A.M. A NNASWAMY, Stable Adaptive Systems, PrenticeHall, Englewood Cliffs, NJ, 1989. [OLI 96] R. O LIVEIRA, E.C. F ERREIRA, F. O LIVEIRA and S. F EYO DE A ZEVEDO, “A study on the convergence of observer-based kinetics estimators in stirred tank bioreactors”, J. Proc. Control, vol. 6, no. 6, pp. 367–371, 1996. [PER 93] M. P ERRIER and D. D OCHAIN, “Evaluation of control strategies for anaerobic digestion processes”, Int. J. Adaptive Cont. Signal Processing, vol. 7, no. 4, pp. 309–321, 1993. [POM 90] Y. P OMERLEAU and M. P ERRIER, “Estimation of multiple specific growth rates in bioprocesses”, AIChE J., vol. 27, pp. 231–236, 1990. [POM 92] Y. P OMERLEAU and G. V IEL, “Industrial application of adaptive nonlinear control for baker’s yeast production”, in Proc. 5th Int. Conf. Computer Appl. in Biotechnol., N.M. K ARIM and G. S TEPHANOPOULOS (eds.), pp. 315–318, Pergamon Press, Oxford, 1992.

142

Bioprocess Control

[POM 95] Y. P OMERLEAU, M. P ERRIER and D. B OURQUE, “Dynamics and control of the fed-batch production of Poly-β-hydroxybutyrate by Methylobacterium extorquens”, in Proc. 6th Int. Conf. Computer Appl. in Biotechnol., pp. 107–112, 1995. [SIE 99] P. S IEGWART, K. M ALE, J.C ÔTÉ, J.H.T. L UONG, M. P ERRIER and A. K AMEN, “Adaptive control at low-level glucose concentration to study HEK-293 cell metabolism in serum pure cultures”, Biotechnol. Progress, forthcoming, 1999. [SUL 97] D. S ULMON, Design and Implementation of Monitoring Tools for a Biotechnological Process, Engineering thesis, Erasmus project, Université Catholique de Louvain, Belgium and University of Stuttgart, Germany, 1997. [WIL 70] J.L. W ILLEMS, Stability Theory of Dynamical Systems, Nelson, London, 1970.

Chapter 6

Basic Concepts of Bioprocess Control

6.1. Introduction This chapter aims at giving a somewhat new introduction to the basic automatic control concepts and using them in the context of bioprocess control using simple examples. The next chapter will more precisely deal with the idea of certain so-called advanced control laws (optimum and adaptive control) that are specifically adapted to the particular aspects and characteristic challenges of bioprocess control. Here we will only cover briefly the notions of nonlinear and robust control as well as the case of a specific approach to control and to global optimization. As already mentioned in Chapter 1, the problems posed by the industrialization of biological processes are similar to those seen in any industrial process and we encounter, in the field of bioprocesses, almost all the problems that are tackled in automatic control. Thus, the requirements of monitoring and control systems of processes with a view to optimizing their operation or detecting malfunctions are on the increase. However, in the real sense, the use of installations equipped with such systems is even now far from being generalized. As has already been underlined by us, two types of difficulties explain to a great extent this lack of generalized application of sophisticated monitoring and control systems to bioprocesses: some of them are connected with the modeling of the bioprocess dynamics and others are connected with instrumentation problems and with the difficulty of reliably measuring online all the key process variables.

Chapter written by Denis D OCHAIN and Jérôme H ARMAND.

143

144

Bioprocess Control

The use of a control system for monitoring and controlling a biological process is shown schematically in Figure 1.1. This scheme introduced by us in the first chapter itself will serve here as a basis for introducing the fundamental automatic control concepts. The central element of this scheme is the process (schematically represented here by a reactor). On this process, we carry out a number of measurements, either in the liquid medium (typically, pH, temperature or concentration measurements) or in the gas medium (typically in the form of measurements of gas flow rates). These measurement data are fed into a computer, which can carry out, in particular, process monitoring and control. In the preceding chapters, we dealt with the means to convert these available measurements into exploitable values (parameters, variables) in real time for process monitoring with the help of “software sensors.” These physical and software measurements can in turn be used for studying the regulation of the process: we now deal with the control of the process. The controller (schematically shown here as being integrated within the computer) enables us to formulate a control action to be applied to the bioprocess on the basis of (physical and software) measurements and knowledge about the process dynamics and the chosen control objectives. In Figure 1.1, this control action is represented by the modulation of a valve opening, enabling us to adjust the feed rate of the substrate in the reactor. In automatic control jargon, we call the controlling valve the “actuator.” The schematic diagram shows a basic automatic control concept, which we will deal with later, in a more detailed manner: the closed loop concept. This loop is made up of the sensorscomputer-actuators process assembly. On the basis of the knowledge available about the process and the control objectives as specified by the user, we can develop and implement more or less complex control algorithms. This chapter is organized as follows. First, we will introduce the basic concepts of biological process control while specifying more particularly the notions of stability, performance and disturbance. Secondly, we will deal with the basic concepts that are necessary for the synthesis of a controller (dynamic model, effects of proportional and integral actions and feedforward action). Finally, we will deal with the PID type (Proportional-Integral-Derivative) linear approach using a nonlinear approach before dealing with certain aspects linked with their robustness with respect to non-measured parameters and variables of biological models. Here we will only deal with the “robust control” approach, and the “adaptive control” approach will be dealt with in the next chapter. 6.2. Bioprocess control: basic concepts 6.2.1. Biological system dynamics We can define briefly the process control as applicable to bioprocesses as follows. The main objective of the process control is to maintain it under stable and, if possible,

Basic Concepts of Bioprocess Control

145

optimal operating conditions in spite of internal/external disturbances liable to affect its proper operation. Certain essential automatic control terms, i.e. the notions of stability, performance, optimality and disturbances have been described above. Now, we will define each of these terms within the specific context of biological processes. For providing the definition, let us consider the simple example given below: the growth of a biomass on a substrate in a stirred tank reactor. This reaction is schematically described in the following form: S −→ X

(6.1)

where S represents the substrate and X the biomass. The dynamic material balance as applied to this reaction in a stirred tank reactor can be written as follows: Fin 1 dS = (Sin − S) − μX dt V YX/S Fin dX = μX − X dt V dV = Fin − Fout dt

(6.2) (6.3) (6.4)

where Fin (l/h) is the reactor’s feed rate, Fout (t) (l/h) the withdrawal rate, V (t) (l) the reactor’s active volume, Sin the substrate concentration at the reactor inlet, YX/S the yield coefficient of the biomass on the substrate and μ the biomass specific growth rate. As seen in Chapter 2, we can make out three distinct operating modes depending on the values of Fin and Fout : 1) continuous: Fin = Fout = 0; 2) fed-batch (or “semi-continuous”): Fin = 0 and Fout = 0; 3) batch (or “discontinuous”): Fin = Fout = 0. The following observations with regard to the model are arranged sequentially (6.2)–(6.4): – this is a dynamic model: it describes the time evolution (dynamics) of system variables. It is an essential component in the synthesis of a control system; – this model type is particularly important, as it describes the development of diverse variables, regardless of the operating mode defined above. Thus, it provides a certain degree of universality for representing stirred tank reactors; – we will use the same notations for describing a reaction component (X for biomass and S for substrate) and for designating its concentration.

146

Bioprocess Control

In subsequent sections in the chapter, we will use the (usual) notation of the dilution rate D in the place of feed rate. Dilution rate D (1/h) is defined as the ratio of feed rate Fin to reactor volume V : D=

Fin V

(6.5)

The above balance equations (6.2)–(6.4) can now be rewritten as follows: dS 1 = D(Sin − S) − μX dt YX/S

(6.6)

dX = μX − DX dt

(6.7)

dV = DV − Fout dt

(6.8)

Let us now clarify the expression for the specific growth rate μ. Consider the Haldane expression: μ=

μ∗ S KS + S + S 2 /KI

(6.9)

where KS is Monod’s constant, KI is the inhibition constant and μ∗ is related to the maximum specific growth rate μmax using the relationship:   , KS ∗ (6.10) μ = μmax 1 + 2 KI A complete description of model (6.6)–(6.8) has been given thus far. Subsequently, when we consider a certain number of this system’s control problems, we will use either the feed rate Fin or inlet substrate concentration Sin as the control variable. 6.2.2. Sources of uncertainties and disturbances of biological systems The concept of disturbance associated with biotechnological processes can be illustrated as follows. Consider the following case: – Sin is not measured and varies with time; – system model is perfect; – control action is the feed rate Fin (or equivalently, the dilution rate D). Examination of model (6.6)–(6.9) shows that a variation of Sin will bring about variations in the system conditions X and S. Thus, variable Sin indicates a system

Basic Concepts of Bioprocess Control

147

disturbance. The algorithm enabling the compensation for these variations by acting on the process by means of Fin is precisely a control law, whose characteristics will be derived from the analysis of the model and its properties (6.6)–(6.9). Consider now the case of the uncertain system model. In other words, certain values of model parameters (KS , KI , or even μmax ) – from which control algorithms will be built – will not correspond to their real values, as these have not been well identified or have been identified from data full of errors or from data that varied with time. These uncertainties are thus additional sources of disturbance. Another source of uncertainty is related to the approximations made during the selection of the structure of the model. Actually, for simplification, a certain number of assumptions were introduced while writing down model (6.2)–(6.9). For example, a perfectly homogenous mixing was assumed. Furthermore, the mortality, or at the very least the non-active biomass, was neglected. Let us also note that the effect of certain environmental variables was not taken into account: temperature and pH were not included in the equations or in the expression for the specific biomass growth rate. These assumptions, though valid in certain cases might no longer be confirmed in other situations or if the experimental conditions are changed (for example, when there is a malfunction of actuators and/or sensors). It is then probable that because the model does not necessarily take into account the possibility of the occurrence of this malfunction, its validity requires reassessment. In other words, even when a model takes on a certain universality with respect to a given criterion (in our example, universality with respect to the operating mode), it should be noted that it is valid only in a particular dynamic context, in which the procedure is supposed to evolve as long as the control law is active, and in the absence of which the theoretical guarantees of stability and/or of performance are meaningless. Thus, it is absolutely essential to analyze with the greatest care the performance specifications demanded by a user before designing and implementing a control law on a real biological process. 6.3. Stability of biological processes 6.3.1. Basic concept of the stability of a dynamic system The stability of dynamic systems is at the core of a control engineer’s concerns while analyzing any dynamic system with a view to controlling it. It should be kept in mind that one of the basic contributions of automatic control as a science was the formulation and solution of a certain number of stabilization problems of dynamic systems by using the feedback concept. For this, numerous definitions were proposed and many stability tests developed depending on the available models for the diverse systems under consideration [WIL 70]. In order to clearly specify the scope of the stability concept as applicable to bioprocesses, we will now focus on the Lyapunov stability. Let us consider an equilibrium

148

Bioprocess Control

0.3 μo = 6.3 h−1 KS = 10 g/1

Specific growth rate μ (h−1)

0.25

KI = 0.1 g/1

0.2

0.15

0.1

0.05

0

0

1

2 3 4 Substrate concentration S (g/l)

5

6

Figure 6.1. Specific growth rate μ according to the Haldane expression

point of a dynamic system. By definition, this equilibrium point is said to be stable, if, after a small disturbance (small variation in one of the inputs), the system converges to the initial equilibrium point. It will be labeled as unstable, if, as a result of this small disturbance the system deviates more and more from this point over the course of time; we will then have “divergence”. Let us specify this concept for system (6.2)– (6.9). Figure 6.1 shows the profile of μ according to substrate concentration S. We will show below that the system is stable (unstable) when the substrate concentration is to the left (right) of value S for which the growth rate is maximum. For this, let us at first specify the concept of the equilibrium point. 6.3.2. Equilibrium point ¯ X]. ¯ The equilibrium point of a dynamic system is, by definition, a constant state [S, This point comprises the values of X and S, which, for a given value of the input ¯ S¯in ], cancel out the model’s dynamic equations. In our example, this means vector [D, ¯ ¯ is given as: that [S, X] dS ¯ S¯in − S) ¯ − 1 μ ¯ =0 = 0 ⇐⇒ D( ¯X dt YX/S

(6.11)

dX ¯ X ¯ =0 = 0 ⇐⇒ (¯ μ − D) dt

(6.12)

Basic Concepts of Bioprocess Control

149

We can see from equation (6.12) that there can be basically (at least) two equilibrium points. One of them corresponds to X = 0. This refers to the case, when there is wash-out of the bioreactor. It defines the situation in which the quantity of biomass removed from the reactor over the course of a given period exceeds the quantity of biomass ¯ > μmax , which is an developed in the course of this period. This occurs when D undesirable equilibrium point. The other solution of (6.12) gives: ¯ μ ¯=D

(6.13)

By replacing μ ¯ in equation (6.11), we obtain: ¯ ¯ = YX/S (S¯in − S) X

(6.14)

In this last case, in order to know more accurately the values of S and X at equilibrium, it is necessary to know the kinetic model of μ. If we consider that the growth rate is described by the Haldane model, then we can easily deduce that there are three possible equilibrium points, whose analytical expressions are given by the following equations: ¯ 1 = 0, 1) equilibrium point corresponding to the wash-out of the reactor: X ¯ ¯ S1 = Sin ; √ ¯ 2 = YX/S (S¯in − S¯2 ) with S¯2 ≤ KS KI ; 2) operating equilibrium point 1: X √ ¯ 3 = YX/S (S¯in − S¯3 ) with S¯3 ≥ KS KI 3) operating equilibrium point 2: X Operating equilibrium points 2 and 3 are the solutions for the second-order algebraic equation obtained by introducing the Haldane expression in equation (6.12): ¯ D ¯ − μ∗ )S + DK ¯ S =0 S 2 + (D KI

(6.15)

6.3.3. Stability analysis In order to make the stability analysis of the earlier defined points 2 and 3, we will proceed with a linear approximation of model (6.2)–(6.9) around these points (we then obtain the system’s tangent linearized model). Regardless of the admissible input values D and Sin this valid tangent model around the equilibrium points can be written as: dx = Ax + Bu dt

(6.16)

150

Bioprocess Control

where x is the state vector describing the variations of S and X around equilibrium ¯ and S: ¯ points X  S − S¯ x= (6.17) ¯ X −X ¯ and S¯in : and u is the input vector describing the variations of D and Sin around D  ¯ D−D u= (6.18) Sin − S¯in Matrix A will be given by: ⎡

0

Ω

⎢ A=⎣ 1 ¯ D − YX/S with

−

1 YX/S

 ¯ Ω=μ X

KS + S

⎥ ¯ − Ω⎦ D

¯2  S KI ¯2 2 S +K I

KS −

∗

⎤ (6.19)

(6.20)

The stability of the equilibrium points is thus determined by the eigenvalues of A: the equilibrium point under consideration will be stable, if the real part of all the eigenvalues of A are negative; it will be unstable if the real part of (at least) one eigenvalue is positive. In our example, the two eigenvalues of A are equal to (−D) and to (−1/YX/S Ω), respectively. The first of these values is always negative as D ¯2] is positive. However, the second eigenvalue is negative for equilibrium point [S¯2 , X ¯ 3 ]. We will therefore deduce from this that but positive for equilibrium point [S¯3 , X ¯ 2 ] is stable and equilibrium point [S¯3 , X ¯ 3 ] is unstable. equilibrium point [S¯2 , X 6.4. Basic concepts of biological process control 6.4.1. Regulation and tracking control When we encounter control problems, two typical situations might arise, depending on whether the set point (i.e. the desired value of the variable to be controlled, which we shall try to keep as near as possible to this) is constant – in which case we deal with regulation – or varies over time – in which case we deal with tracking control. Problems with regard to regulation are commonly seen in continuous process control, for which the concentrations of certain components in the medium or even certain physical–chemical variables (temperature, pH, level, etc.) should be kept constant. However, when we are dealing with the control of reactors operating in fed-batch or batch mode, the regulation of certain variables may no longer be suitable. For example, when it is a question of maximizing the biomass production in a fed-batch reactor,

Basic Concepts of Bioprocess Control

151

it is possible to show that the optimum solution consists of controlling the process in such a way that its states follow the trajectories as pre-defined by the user and depend on the feed conditions of the process. In this example, the profile to be imposed on the biomass concentration is of the “exponential” type. However, there are also other typical cases of operating conditions, where a “feedback” type of control is necessary: start-up, shut-down, grade change, etc. 6.4.2. Strategy selection: direct and indirect control We introduce here the concepts of direct and indirect control of biological processes depending on the user-specified control objectives. Actually, as already underlined by us in the introduction, biological processes suffer from a quasi-systematic lack of sensors which enable online measurements of key biological process variables. Accordingly, users are frequently constrained to limiting their control to the regulation or tracking control of a certain number of easy-to-measure environmental variables such as pH, temperature and flow rate. The biological variables are in that case not directly regulated and the users merely ensure optimum conditions for the growth of micro-organisms independently of non-observable biological parameters (growth rate, concentrations of different elements). This approach is justified insofar as the environmental variables (pH, temperature, mixing properties, etc.) precisely influence the growth rates of micro-organisms according to the profiles that are very similar to that obtained while modeling the effect of substrate concentration on the growth rate with the help of a Haldane expression. Thus, there is a range of values of these environmental variables for which the growth of micro-organisms is optimum (see Figure 6.2). We now deal with the indirect control of biological processes to the extent that the biological parameters are not directly measured. The advantage of this type of approach lies not only in the simplicity of implementation (this involves the implementation of mono-variable control loops for which the tuning rules are simple and well known) but also in the simplicity of its maintenance: the sensors are cheap; the defects are relatively easy to detect and localize, etc. However, if we refer to the concepts of disturbances and uncertainty described above, it is quite obvious that when there is a need for more specific performances (yield optimization, online monitoring of environmental parameters, etc.), the indirect approaches are not suitable. In this case, there are two solutions depending on the degree of sophistication. On the one hand, complexity of the indirect approach increases with the introduction of software sensors. They allow for the control of the estimation of the variables or biological parameters within the framework of the process model and that of available online measurements. On the other hand, this approach increases the complexity of the controller with regard to its development, implementation as well as maintenance. An alternative involves the introduction of direct controllers, which necessitate the use of advanced instrumentation (automated

152

Bioprocess Control

1

Specific growth rate μ (h−1)

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 270

280

290 300 310 Temperature T (K)

320

330

Figure 6.2. Effect of temperature on μ

concentration analyzers, automated measurements of biological parameters, etc.). In a favorable case, the rule for adequate control can be reduced to a simple additional control loop. However, it is probable that the monitoring and maintenance of this sensor will be much more difficult than in the case of an indirect approach. These observations highlight the existence of a compromise between the complexity of the control law and the degree of sophistication for the instrumentation to be implemented in order to achieve the aims of control. In fact, it should be noted that the additional degree of sophistication – software in the first case and equipment in the second – influences the development and the type of system for the maintenance and monitoring of the installation necessary for ensuring its good operation. In other words, the following question should be considered: will it be better to use a complex indirect control law and to limit the installation’s monitoring or will it be more worthwhile to go in for an approach involving direct control and to implement a more elaborate monitoring system? This fundamental question, some of the points with regard to this question are considered and studied in more detail in the chapter on fault detection and diagnosis (Chapter 8), and an accurate answer to this question, which is extremely difficult to obtain, are in fact the basis of numerous current research papers. 6.4.3. Selection of synthesis method Selecting a method for synthesis depends on numerous factors: 1) In the first place, as underlined earlier, this selection depends on the control objectives themselves and on the strategy employed. The more precise the control

Basic Concepts of Bioprocess Control

153

specifications, the more justified the analytical approach will be. We can of course subsequently envisage various combinations of the “precision of specifications/complexity of control strategy.” We can thus imagine very strict but easy-to-verify control specifications insofar as the variable to be controlled is directly measured by very complex means. Conversely, a problem in regulating an easily measurable environmental variable can be resolved by a very simple controller based on a very succinct linear model. 2) The choice of the type of control law will thus depend on the available a priori knowledge about the process. From this point of view, we recapitulate here some basic rules that can guide the users in their choice of synthesis method: - when an analytical model – of any nature – is available, then it can be readily used; - when an analytical model is not available but considerable expertise with regard to working of the process is available, then we can use heuristic approaches (fuzzy logic, knowledge-based systems, etc.); - when more time is required to obtain an analytical model and there is not much expertise available but on the other hand a large quantity of data is present, then it might be of interest to employ a statistical or neural approach enabling us to obtain in particular the nonlinear behavior models (see for example [MON 94]). 6.5. Synthesis of biological process control laws As underlined in the preceding section, we may have to synthesize more or less complex control laws defined according to the user-defined control objectives. By their nature, the direct control strategies will probably be less complex than the indirect control strategies. They therefore form part of the so-called conventional control laws, which we will present at first and then distinguish from the so-called advanced control laws, which will be dealt with subsequently. As any control law is based on a representation – however simple it might be – of the system to be controlled, we will start with a recapitulation of the various types of models that can be used to represent a system. 6.5.1. Representation of systems In this section, we will focus on the basic components necessary for designing a control law. The process model is one of the essential components for the design of control algorithms. In fact, the latter will enable us to understand the system dynamics and the interactions of its diverse variables better. The model properties and uncertainties will affect in particular: – the manner in which the states evolve in transient regimes; – the way in which inputs (especially, non-measured inputs or disturbances) affect the states and thus the system outputs;

154

Bioprocess Control

– the selection of control strategies and strategies for the rejection of disturbances to be implemented in order to meet the control objectives. Depending on the means implemented to connect these diverse components (inputs/states/outputs), we can distinguish various model types. The most complete representation of the biological processes is a “material-balance” modeling. However, other representations can also be envisaged. When the system structure is difficult to apprehend, an input-output representation (or black box) – abstraction of the explicit representation of states – can be obtained by system identification (see for example [LJU 87]). This modeling itself might be linear or nonlinear (as in the case of the neural networks cited above) and could bring in a representation of uncertainties and/or disturbances of determinist or stochastic form. Another possibility is the use of so-called hybrid models [FEY 97], wherein we sometimes use material-balance models in which the biomass growth rates are modeled by neural networks. We will now see the basic components that form the control laws and in particular the diverse corrective actions that might be envisaged. 6.5.2. Structure of control laws In fact, irrespective of the methods employed for writing the algorithms of a control law, we make out two specific control structures: feedback and feedforward structures. On referring to Figure 6.3, we see that control law by feedback necessarily uses the system’s output measurements for controlling it. On the contrary, feedforward employs the measurements available at the disturbance inputs for correcting their undesirable effects on the variables of the output to be controlled. The main disadvantage of the latter, when used alone, is that there can be no correction of the effects caused by internal variations of the system to be controlled. Disturbances Not measured Measured

Correction by anticipation Set points

Controller

Inputs

Outputs

Disturbances

Process Controls

Feedback

Figure 6.3. Structure of a controlled system

Measurements

Basic Concepts of Bioprocess Control

155

Due to the above reason, it is clear that a control law should preferably use the measurements available at the output and if possible, employ a “mixed” structure associating the two types of correction. Thus, the possible internal system variations will be compensated by feedback while the variations in the disturbance inputs will be anticipated and the bad influences on outputs minimized by the action in anticipation. Proportional action The simplest expression for a control law (if we exclude the On-Off controllers) is the feedback of measured information enabling a correction proportional to the error existing between this measured variable and the set point. In a mono-variable context, the controller gain is the only synthesis parameter. In our example, the direction of action is given by the sign of the error, which means that the controller gain will be positive: – if S ∗ − S < 0, the measurement is above the set point and thus the feed rate has to be reduced so as to lower the substrate concentration; – if S ∗ − S > 0, the measurement is below the set point and thus the feed rate has to be increased so as to increase the substrate concentration; – if S ∗ = S, then the feed rate is to be kept constant. The proportional controller which enables the regulation of the substrate concentration around a set point S ∗ is written as: D = D∗ + Kp (S ∗ − S)

(6.21)

where D∗ is the value of D to be applied to the system in order to obtain the equilibrium S¯ = S ∗ in the absence of disturbance (i.e. the model is presumed to be perfectly known). In our example for simple microbial growth (6.6)–(6.9), it is noted that D∗ corresponds to the value of a specific growth rate corresponding to S ∗ : D∗ = μ(S ∗ )

(6.22)

It should be noted that √ in the case of our example for the microbial growth with Haldane kinetics, if S ∗ > KS KI , we are somewhere around the unstable equilibrium point and the feedback loop is absolutely necessary for maintaining S around S∗. Now consider the presence of a disturbance, which for example might take on the form of a constant bias ΔD on the actuator. In other words, instead of applying D as a dilution rate, we actually apply unintentionally: Dreal = D + ΔD

(6.23)

156

Bioprocess Control

The real feed rate being biased, this correction will be stabilized around a constant value, which cancels the dynamic equations of the biological system (6.6), and the substrate value will never attain its set-point value. Thus, in a permanent regime, there will be a permanent error between the controlled variable and the set point. To check this, we have to consider equations (6.6) and (6.7) in closed loop (by replacing D with Dreal ) and confirm that the equilibrium value of S, which cancels these equations, depends on ΔD. Let us examine in particular and in more detail the dynamic equation of the control error S ∗ − S. On the basis of equations (6.6), (6.21) and (6.23), it is written in the following form: d(S ∗ − S) = −Kp (Sin − S)(S ∗ − S) dt − (D∗ + ΔD)(Sin − S) +

1 YX/S

(6.24) μX

The RHS of the above equation contains two terms: one term proportional to the control error S ∗ − S, and a second term, which is the difference between a term containing μX and a term proportional to D∗ + ΔD. Under certain reasonable assumptions concerning the above equation (as well as with regard to equation (6.7))1, we note that when ΔD = 0, the two terms of equation (6.24) cancel each other out for S = S ∗ . On the contrary, when ΔD = 0, and on recalling equilibrium equations (6.14) and (6.13), it is seen that the derivative of S ∗ − S is canceled out for a value of S equal to: 1 ¯ S¯ = S ∗ + (ΔD + μ(S ∗ ) − μ(S)) Kp

(6.25)

This will differ from S ∗ . From the above equation, it can be seen that the presence of the term ΔD induces a static error (S¯ = S ∗ ) in the feedback loop. However, this is not very useful for its explicit calculation. In fact, we require a specific growth model in order to carry out this calculation. For example, in the case of a Monod model: μ=

μmax S KS + S

(6.26)

the steady-state value of S is the solution of the algebraic equation: Kp S¯2 + (α − KP S ∗ + ΔD)S − KS (αS ∗ − ΔD)

(6.27)

1. We will not develop these hypotheses here lest the reader should lose the essential thread of our subject.

Basic Concepts of Bioprocess Control

157

with: α = K p KS +

μmax KS KS + S ∗

(6.28)

Only one solution of the above equation is positive. It is given by: S=

−α + KP S ∗ − ΔD 2Kp

(KP S ∗ − α + ΔD)2 + 4KS Kp (αS ∗ − ΔD) + 2Kp

(6.29)

In general, the substrate concentration will differ from the desired value S ∗ . It can be seen that for ΔD = 0, this difference vanishes. It is important to note that this difference is inversely proportional to the gain Kp . In other words, we can reduce the static control error by an appropriate selection of the controller gain but this is liable to influence the performance of the control system via, for example, an amplification of noise over the measurements of S. In order to compensate for this control error, it is necessary to introduce an integral term. Integral action As shown above, integral action is required in order to meet a basic requirement of any feedback loop: eliminating the control error in steady state. The integral action is always associated with a proportional term and is written as follows in the case of a PI controller:   1 t ∗ (S − S(τ ))dτ (6.30) D = D∗ + Kp (S ∗ − S) + τi 0 where τi is the integrator’s time constant. In order to produce the effect of canceling out the control error by integral action, let us go back to the dynamic equation of the error with the new control equation (6.30) as well as equation (6.23). This is written (with the “disturbance” term ΔD) as follows:  Kp (Sin − S) t ∗ d(S ∗ − S) = −Kp (Sin − S)(S ∗ − S) − (S − S)dτ dt τi 0 (6.31) 1 ∗ − (D + ΔD)(Sin − S) + μX YX/S The presence of an integral term makes the calculations more complicated. In order to overcome this difficulty, we will derive once more the regulation error S ∗ − S or

158

Bioprocess Control

more precisely the substrate concentration S. We now obtain the following equation:  Kp (Sin − S) X ∂μ dS d2 S + + D + Kp (Sin − S) + S dt2 Y ∂S dt τi (6.32) Kp (Sin − S) ∗ μ S − (μ − D)X = τi Y We know that under normal regime (i.e. in the absence of biomass wash-out), μ = D. Under this simplifying assumption, we will end up with a second order differential K (S −S) ∗ S proportional to S ∗ . equation with an “input” term p τini The homogenous differential equation (i.e., without the “input” term) is asymptotically stable, if the real parts of the roots of the associated characteristic polynomial Kp (Sin −S) ∂μ are all negative, i.e., if here given by p2 + (D + Kp (Sin − S) + X Y ∂S )p + τi (2nd order system) all the polynomial coefficients are of the same sign. This will be ∂μ is positive (for example, the Monod model) or by considering Kp to the case if ∂S ∂μ would become negative (as can occur in the be sufficiently large in the case where ∂S case of a Haldane model (part to the right of the maximum, Figure 6.1)). At equilibrium, the derivative terms are eliminated. We will thus obtain: Kp (Sin − S) ∗ Kp (Sin − S) S= S τi τi

(6.33)

or, in other words, S = S ∗ , and the regulation’s static error is eliminated by the integral action. Derivative action A derivative term of the control error can be added in the control law. This will possibly accelerate in particular the dynamics of the feedback loop. We thus obtain the PID controller. The PID controller is frequently used in the industry (though often in the PI form, which is easy to tune, in the process industry). The controller action is fundamentally calculated on the basis of the following equation [SEB 89]:   1 t ∗ d(S ∗ − S) ∗ ∗ (6.34) (S − S(τ ))dτ + τd D = D + Kp (S − S) + τi 0 dt Deadtime compensation When the effects of varying an input do not immediately affect an output, we say that there is a deadtime (delay) in the system. The value of this deadtime depends on the system under consideration. If we want to control a system with deadtime, the controller can integrate a system for compensating for this deadtime. The simplest arrangement consists of integrating into the control law a predictive device enabling

Basic Concepts of Bioprocess Control

159

a prediction of the variable’s future value (index of present instant + deadtime) and in controlling this new predicted variable. This is the “Smith predictor” concept. Let us note here that in the case of biological processes, deadtimes are often negligible with respect to the system’s time constants. However, in certain cases of large systems, it may be necessary to integrate them into the models. Nonetheless, deadtime compensation, except in mono-variable cases, is not generally an easy task. Zero-setting mechanism of integrators Up to now, we have not mentioned the presence of physical limits to the values of a control action. It is not possible that a control action could take on an infinite value; this is always bounded by physical constraints. For example, a flow rate should be positive and has an upper bound determined by the maximum flow rate of the pump or the valve through which the fluid passes. In other words, the real flow rate applicable to the bioreactor, for example in the case of the PI controller described above, is given by: if 0 ≤ Dcalc ≤ Dmax

(6.35)

=0

if D < 0

(6.36)

= Dmax

if Dcalc > Dmax

(6.37)

Dreal = Dcalc

with:

  1 t ∗ Dcalc = D∗ + Kp (S ∗ − S) + (S − S(τ ))dτ τi 0

(6.38)

An unintended side effect of the integral action might occur when the control is fixed at the bounds. This effect, called “wind-up,” is due to the fact that the calculation continues to integrate the control error, which maintains the calculated value far from the control bound and thus prevents it from coming back sufficiently rapidly to a value within the control bounds. This is often manifested in the form of an oscillation of the control variable and thus that of the controlled variable. The mechanism used for preventing the occurrence of this phenomenon is called “anti-reset windup”; it consists for example of “emptying” the integrator when the calculated control variable attains the control bounds. On rewriting the PI controller equation with integral action in differential form: Dcalc = D∗ + Kp (S ∗ − S) + xint

(6.39)

Kp ∗ dxint = (S − S) dt τi

(6.40)

Integral action (6.40) with “anti-reset windup” action becomes: dxint Kp ∗ 1 = (S − S) + (Dreal − Dcalc ) dt τi τi

(6.41)

160

Bioprocess Control

6.6. Advanced control laws We presented above the basic components of bioprocess control by integrating them in the traditional structure of a PID. In the next chapter we will see how to develop control algorithms that are more sophisticated, more specifically linked with optimal and adaptive controls. However, before that, we would like to briefly deal here with the advantages and disadvantages of the approach with PI-type linear control from the viewpoint of an approach, again of the PI type, but which takes into account intrinsic nonlinearities of bioprocess dynamics. 6.6.1. A nonlinear PI controller Let us now consider the application of a PI control law to the nonlinear biological system as described in equations (6.6)–(6.9). As in the previous case, the control aims at regulating the substrate concentration S around a constant set point S ∗ . The control law is written as: (t C1 (S ∗ − S) + C2 0 (S ∗ − S(τ ))dτ + k1 μX (6.42) D(t) = Sin − S with k1 = 1/YX/S , C1 > 0 and C2 > 0. On introducing this control law in equation (6.2), it is easy to show that the dynamics of S will thus be described by:  t d(S ∗ − S) ∗ = −C1 (S − S) − C2 (S ∗ − S(τ ))dτ (6.43) dt 0 This equation means that if the measurement of S deviates from its set-point value, the nonlinear PI controller, whose expression is given by (6.42), will tend to bring S towards the set point at a speed determined by the choice of parameters C1 and C2 . Let us now list the advantages and disadvantages of the nonlinear and linear PItype control approaches presented up until now. First, it is important to note that the second algorithm is undoubtedly more complex than the first one: – it requires online measurement of Sin and X; – it requires knowledge of the parameters of the model, in particular μ; – its implementation necessitates the use of a computer, whereas the first algorithm can be implemented on industrial housings that are easy to parameterize. Linear approach This control, if well tuned, generally enables us to obtain satisfactory results. It should also be underlined that this controller is very popular because of its ease of

Basic Concepts of Bioprocess Control

161

1.2 S* = 1 g/l

1

S (g/l)

0.8 0.6 0.4 0.2 0

0

5

10

15

20

t (h) Figure 6.4. Simulation of a linear PI controller for a fed-batch process

implementation. However, its performances degrade (especially in the transient state) in the presence of sizeable disturbances, which will remove the controlled variable from its set point to a considerable extent. Such a situation is routinely seen in the start-up phases of the reactor or in certain problems involving the control of batch and fed-batch reactors. This situation is illustrated in Figure 6.4, which shows the regulation of a substrate S with the following conditions: S ∗ = 1 g/l,

Kp = 2,

τi = 0.5,

S(0) = 0 g/l

(6.44)

The PI parameter values were selected in such a way that the system dynamics in closed loop are approximately the same as the “average” dynamics observed in open loop. In fact, it is seen that the controller gain increases considerably at the beginning of simulation (this causes large variations in S around the set point) and then decreases considerably (the controller is thus incapable of maintaining S around the set point at the end of the exponential growth phase of the biomass). Nonlinear approach The main advantage of nonlinear regulation is directly connected to the disadvantages of the linear approach as explained above: if it is correctly regulated, a nonlinear controller can cover a wider operating range than is the case with a linear controller. In our example, the source of maximum nonlinearity is the term k1 μX. It is especially important in the control of fed-batch processes. As already underlined elsewhere [AXE 89], the integration of this term characterizing the exponential growth of biomass is a key factor in the control law’s efficacy. The other term, Sin − S, also plays an important role. In fact, in the case where the inlet substrate concentration

162

Bioprocess Control

represents the disturbance, this term plays the role of a feedforward action, enabling us to compensate for its undesirable effects on S. However, it has to be mentioned at the outset that for implementing this controller, we must have recourse to a numerical control acquisition system (which should not be a real problem nowadays). Furthermore, it is important to underline that this controller’s performances will be to a great extent influenced by the quality of its underlying model. This means that the assumptions made in the system modeling should not be violated when the controller is active. Let it be noted that this second algorithm necessitates online measurements of S, Sin , X and μ. If it is possible to measure S, the measurement of Sin will probably be relatively easy. In fact, our capacity to measure Sin is intimately linked to the context of the use of the bioprocess. Depending on the industrial sector, Sin might actually be a control variable (especially in the pharmaceutical and food industries). In that case, it is fixed by the user and thus known. However, in other sectors such as wastewater treatment, it will be considered as the main source of disturbance and its measurement will be very difficult, even impossible to obtain online. As for the measurement of X, the problem is more delicate, regardless of the industrial sector under consideration and if this measurement is possible, it will not generally be online. Finally, one of the most probable sources of uncertainty is due to the necessity of knowing μ. On the basis of these observations, it is seen that it is most important to make these control laws robust, i.e. to render them as insensitive as possible to diverse disturbances and uncertainties, so as to mitigate some of the abovementioned disadvantages. We will see in the next chapter how the linearizing adaptive control helps in attaining this type of objective. Another way to confer robustness to the control is to use the robust control concepts, described briefly below. 6.6.2. Robust control Whether it is a question of robustness in performance or in stability (a controller’s capacity to ensure these properties in the presence of disturbances and uncertainties), the main interest of robust approaches consists precisely of guaranteeing these properties a priori. In other words, the characteristics of disturbances and uncertainties to which the control law is subjected will be predefined by the user. Thus, this approach has two important consequences: – at the outset, this means that the user has an a priori knowledge of the disturbances and uncertainties affecting the system under consideration. He should therefore

Basic Concepts of Bioprocess Control

163

define them very well beforehand to enable the selection of the most appropriate representation to the extent possible; – because we have to fix the class and amplitudes of the disturbances affecting the process, robust approaches are often conservative. In other words, given the possibility of the worst-case scenario, the control performances are frequently sub-optimal. This is the case when the disturbances affecting the process are not of the maximum amplitude anticipated a priori. For the specific case of bioprocess control, the works carried out by research groups that are active in wastewater treatment have been decisive. Actually, in this sector of activity, it is very difficult to obtain the models and if they are available, they are usually marred by large parametric errors. However, there is one additional difficulty that is specific to these processes. It is very hard to measure their inputs. In fact, the feed to be treated is often heavily loaded with “suspended solids” (SS). However, with the process playing a filtering role, the effluent is often less loaded with this SS. As a result, if certain substrate concentrations are measurable at the process outlet – which necessitates the installation of these rather evolved filtration systems anyway – it does not hold good for their input concentrations. Thus, such a control problem is encountered in this type of process, the inputs are considered unmeasured variables that vary over time by large amplitudes. However, the nonlinear control law presented above necessitates the knowledge of these inputs. According to the expertise available on the process, certain approaches propose the inclusion of a disturbance model in the system’s representation. Under certain conditions, it will thus be possible to estimate them online and to compensate for their effects on the variables to be controlled. This approach has been applied recently to biological processes for pollution abatement and has served as the basis for these strategies for reconfiguring the control laws of an industrial equalization system [DEV 00] and those of a pilot anaerobic digester [HAR 00a]. However, the range of these strategies is relatively limited when applied to an industrial context, because if the disturbances and uncertainties can at times be modeled by simple functions (constant, ramp, sinusoidal, etc.), it is not rare that they are combined with discontinuities of considerable amplitudes or they take the aspect of noises with such diverse characteristics, which makes them generally impossible to identify. As a result, there is considerable risk that a control law that is robust at a given disturbance class, loses its robustness temporarily when applied to a real biological system. In order to mitigate this problem, one solution consists of minimizing the number of assumptions necessary for structuring the uncertainties and disturbances susceptible to affect the system. An approach consists, for a given disturbance, of specifying only the upper and lower limits within which the real disturbance is assumed to evolve. The idea of this approach is thus to replace the knowledge of exact values of unknown inputs by the knowledge of the values within which they evolve. Thus, Sin is replaced − + and upper limit Sin in the expressions of D or in the equations by its lower limit Sin employed for estimating certain variables such as X or μ.

164

Bioprocess Control

To illustrate this approach, let us consider once again biological model (6.6)–(6.7). The following assumptions were made for the synthesis: − + and Sin between which it evolves; – Sin is unknown, but we know limits Sin – only the variable S to be controlled is measured. The control law aims at maintaining S around a set-point value S ∗ . We will use a nonlinear approach in order to synthesize a control law that is valid over a large working range. If we suitably select Z − (0) and Z + (0) such that Z − (0) ≤ Z(0) ≤ Z + (0), we can show that the asymptotic observers (see Chapter 4) ⎧ −   ⎪ ⎨ dZ = D S − − Z − in dt (6.45) ⎪ ⎩Z − = S + k X − 1 and

⎧ +   ⎪ ⎨ dZ = D S + − Z + in dt ⎪ ⎩Z + = S + k X +

(6.46)

1

will give estimations X − and X + which verify X − (t) ≤ X(t) ≤ X + (t), whatever − + ≤ Sin ≤ Sin . the values of Sin such that Sin It is now possible to synthesize a control law, which is written as: D=

C1 (S ∗ − S) − k1 μX −/+ −/+

Sin

−S

(6.47)

by assuming a robust “worst case” philosophy: – let us assume that the least disturbing case (or the least serious) is the one in which the measurement is below the set point. In the case of a bioprocess control employed in pollution abatement, this means that the measurement is less than the rejection norms. We are therefore in a favorable situation. The value of D to be applied − in (6.47); to the process will therefore be calculated by using X + and Sin – if the measurement is equal to the set point, the control does not vary and we apply the previously calculated control; – if the measurement is above the set point, we are thus in a unfavorable case with respect to our aim for control. Following the worst case principle, we then use the + for the minimum biomass concentration X − and a maximum feed concentration Sin calculation of D. In order to judge the relevance of this control law, it is important to formulate the following observations:

Basic Concepts of Bioprocess Control

165

– in the first place, we can show that this control law enables us to stabilize S around S ∗ with some precision [RAP 02]; – we can generalize nonlinear robust control (6.47) at all the uncertainties and disturbances that can be reduced to unknown inputs. In fact, this control law generalizes the nonlinear control laws proposed up until now: depending on whether we possess no knowledge (in which case we assume that 0 ≤ μ ≤ max(μ)) or knowledge about μ (where we assume a perfect knowledge of μ), the control laws proposed until now range from an all-or-none to adaptive control. In fact, it is easy to show that control law (6.47) generalizes these two control laws, namely, AON and adaptive, into a single −/+ expression given precisely by (6.47) with Sin = Sin when Sin is known. 6.7. Specific approaches Apart from developing control algorithms specifically for biological processes (based on material balance equations), much work was done in order to develop simple approaches to bioprocess control (in order to make them applicable in an industrial context with minimum instrumentation). Their motivations lie in the recent results obtained in basic microbiology, which attempt to show that the bacterial floral activity, while being highly diversified, is optimum when it is disturbed. From a microscopic viewpoint, these systems would evolve constantly without ever attaining equilibrium even if the macroscopic observations are stationary. Thus, the micro-flora evolves without disruption. Certain species that are in the majority at one instant disappear, while others that were in the minority form the majority. In other words, the optimization of these systems would rather require an approach involving this state of constant disequilibrium than automatic control where more traditional stabilization problems exist. It should be noted here that if this is the behavior of biological systems, the traditionally used material balance models do not enable us to understand these phenomena, the available models being necessarily developed by experts in microbial ecology. It still remains true that these papers are opening new lines of research and that the justification for the control algorithm to be presented by us remains essentially intuitive and experimental insofar as it was validated over many pilot processes. We will illustrate the specific approaches with a case study. 6.7.1. Pulse control: a dialog with bacteria As we have just underlined, the control method and the process optimization of anaerobic digestion presented below approach the problem in a manner that is diametrally opposed to the conventional methods. In fact, though the first objective is similar to that in the conventional approach (i.e., optimizing the wastewater treatment by maximizing the applied charge), the basic idea consists of adding disturbances voluntarily to the effluent feed rate (for details regarding the control principle, see [STE 99]). By subsequently analyzing the responses of certain key parameters of the system to these

166

Bioprocess Control

Loading rate (kg COD m−3 j−1)

100 80 60 40 20 Date (1996) 0 01/05

10/06

20/07

29/08

08/10

17/11

Figure 6.5. Starting a semi-industrial installation using the “pulse method”

disturbances, it is possible to characterize the “physiological state” of the installation and to know if the process is organically overloaded or otherwise. The selection of measured parameters was very important and the sensors were chosen depending on their industrial advantages (i.e., low cost, easy maintenance, numerous examples of use, etc.). Thus, by employing only two “rustic” sensors (i.e., gas flow rate and pH), it was possible to start a semi-industrial installation and the applied charge was automatically adapted to the feed concentration to be treated (Figure 6.5). It should be noted that pH is used only for monitoring purposes and is not used directly in the control law. In this application, the feed rate is used as the actuator. This control law principle is based on a concept that can be termed “dialog with bacteria”. In order to understand this, let it be recalled at the outset that anaerobic digestion is a biological process wherein organic material is transformed into gas (methane and CO2 ). Therefore, the response in terms of system gas after an input pulse tells us about the physiological state of the biological reactor. In fact, if we know the average gas yield of this type of reactor, we can easily calculate the average additional quantity of gas that has to be released when a certain level of pollution is introduced into the reactor. The analysis of the gas response of the reactor will therefore carry information on which the control law is based. If, in the wake of a disturbance (flow pulse), the gas response is slightly more than that predicted, it can be deduced that the feed concentration has slightly dropped or the reactor’s yield has improved. This means that the major portion of the organic matter introduced was transformed into gas. The process is therefore working in a sub-optimal state and is suitable for receiving a higher load than the current one. We will now deal with a situation wherein the feed rate is increased.

Basic Concepts of Bioprocess Control

167

If, as a result of disturbance, the gas response conforms to the prediction, we will deduce that the process is working under nominal state. Depending on the desired working mode, it will then be possible either to increase the feed rate (we are thus acting according to the logic of optimizing the treated quantity) or to simply maintain it as constant (logic of least risk). Finally, if the gas response is significantly less than that predicted, then the organic matter has accumulated in the reactor and it has overloaded. It will thus be necessary, in order to avoid an incident, to reduce the feed rate. Compared with the conventional methods of reactor start-up, this method enabled us to attain a nominal state of working very rapidly while always maintaining performance yields of more than 70% (6.6) and very short hydraulic residence times (i.e., a few minutes, see Figure 6.7). Finally, this strategy enables us to adapt automatically the load to be treated in spite of variations in the effluent, both in nature and in concentration (6.8). 6.7.2. Overall process optimization: towards integrating the control objectives in the initial stage of bioprocess design Generally, any process is designed from static considerations of performance and/or stability. The control engineer comes into the picture in order to control this process in such a manner that the conditions as defined by the user – theoretically in a static regime and based on his design – are verified in a dynamic context (in the presence of disturbances affecting the process). In the large majority of cases, this approach is satisfactory. However, under certain conditions, it is at times possible that it could lead to a significant improvement of a process by integrating at the initial design stage the problems and constraints connected with its control. 100

Yield in COT (%)

90 80 70 60 Date (1996) 50 01/05

10/06

20/07

29/08

08/10

17/11

Figure 6.6. Purification yields obtained with increasing load

168

Bioprocess Control

100

Yield in COT (%)

90 80 70 60 Hydraulic dwell time (h) 50 0

1

2

3

4

5

6

Figure 6.7. Hydraulic dwell times depending on the installation’s purification yield

COD input

Dilution

COD input

20 g/l

10 g COD/l

20 g/l

120 100

300 DAE

80

250 60 200 40 150

20

DGP

Product gas flow rate (l/h)

Effluent feed rate (l/h)

350

Time (h) 100

0 140

180

220

260

300

Figure 6.8. Automatic adaptation of the feed rate by the “pulse method”; case of diluting the effluent to be treated

This approach is justified by the following observations: 1) The parameters for the design of a process (type, dimension, etc.) are themselves frequently the results of different processes and optimization. Therefore, the integration of the control objectives at this design stage will bring about an additional degree of freedom for resolving the initial problem. It should, however, be noted that this set of issues brings in a modification of this initial optimization problem as well as certain constraints linked with the practical implementation of the conceived control loops. In other words, this type of synthesis results in making the optimization problem from which the process design parameters arise more complex.

Basic Concepts of Bioprocess Control

169

2) The design of a process in static regime excludes (when it is not related to stabilization) the recourse to feedback loops. This design, therefore, excludes the portion of the space that would otherwise be occupied by the implementation of one or more control loops. As a result, not only is the optimization problem generally going to be more complex when we integrate the control objectives in the design of a process, but also, thanks to the nonlinearities in this process, the space covered by the synthesis parameters is going to be more considerable. In other words, it will generally be more difficult to attain an overall minimum while resolving the new optimization problem. This explains the fact that the major portion of the available results is related to the determination of conditions that are uniquely necessary but not adequate for optimization. The above-cited limitations explain the very small number of results obtained to date in the field of bioprocesses [HAR 97, HAR 00b]. Of course, we see a relatively abundant literature in the field of chemical engineering, but the approaches proposed essentially make use of numerical resolution algorithms and extremely heavy calculations (see, for example, the papers proposed in [STE 00] or even [BAN 00]). In view of cost-effectiveness, one of the major problems posed by bioprocess control is the selection of the number, quality, and optimum localization of the sensors to be used to control these processes. One of the essential contributions of an integrated approach is precisely the optimization of the configuration and the localization of diverse sensors and actuators to be employed, given a performance objective to be attained in closed loop [HAR 97, LU 99]. From among the examples of possible applications of an integrated approach in the field of bioprocesses, we can mention: – optimum distribution of sensors along the bioreactors presenting a spatial gradient of concentrations. This is particularly the case for fixed bed and fluidized bed reactors; – selection of recirculation points in large-sized aerobic systems for wastewater treatment; – minimizing the cost of bioprocess construction using an optimization of their dimensioning (height/radius ratio of tubular reactors) produced simultaneously with the calculation of their controllers; – research into new configurations of systems using passive integration (the result of optimization shows that by accurately using the dimensioning parameters, the expected performances are attained without the addition of a control law). An example of passive integration is the use of a reactor configuration in which the gas produced is used as an energy source for ensuring the agitation of the medium. Such a result is, for example, attained when we use an inverse fluidized bed in anaerobic digestion. The biogas that is formed is re-circulated, which ensures, on the one hand, the homogenity

170

Bioprocess Control

of the medium and on the other hand, the stability of biological reactions, thanks to chemical phenomena. 6.8. Conclusions and perspectives This chapter dealt with the basic principles of bioprocess control. After defining the properties characterizing the control laws, we illustrated with a simple example the advantages and disadvantages of linear and nonlinear approaches. With a view to minimizing the disadvantages of these control laws, we went on to underline the interest in developing advanced control methods. Some of these will be dealt with in more detail in the next chapter. We briefly presented here the “robust control” approach. With a view to optimizing the bioprocess in its dimensioning (controller parameters, structure of controllers, structure of actuators/sensors necessary for obtaining a degree of pre-specified performances), we finally presented an approach with its principle being based on the integration of control objectives at the initial stage of process design. We proposed a certain number of possible applications of this approach before underlining its main advantages and disadvantages. It is obvious that this last point forms one of the most promising research prospects. 6.9. Bibliography [AXE 89] J.P. A XELSSON, Modelling and Control of Fermentation Processes, PhD thesis, Lund, Sweden, 1989. [BAN 00] V. BANSAL, R. ROSS, J.D. P ERKINS and E.N. P ISTIKOPOULOS, “The interaction of design and control: double-effect distillation”, J. Proc. Control, vol. 10, pp. 219–227, 2000. [DEV 00] M. D EVISSCHER, J. H ARMAND, J.P. S TEYER and P.A. VANROLLEGHEM, “Control design of an industrial equalization system – Handling system constraints, actuator faults and varying operating conditions”, in SAFEPROCESS2000, Budapest, Hungary, 14– 16 June, 2000. [FEY 97] F. F EYO D E A ZEVEDO, B. DAHM and F.R. O LIVIERA, “Hydrid modelling of biochemical processes: a comparison with the conventional approach”, Comp. Chem. Eng., vol. 21, pp. S751–S756, 1997. [HAR 97] J. H ARMAND, Identification et commande des procédés biologiques de dépollution, PhD thesis, University of Perpignan, France, 192 pages, 1997. [HAR 00a] J. H ARMAND, A.G. M ANH and J.P. S TEYER, “Identification and disturbance accommodating control of a fluidized bed anaerobic digester”, Biopr. Eng., vol. 23, no. 2, pp. 177–185, 2000. [HAR 00b] J. H ARMAND and J.P. S TEYER, “Economical Design of Biological Systems: New Tools for Advanced Integrated Process and Control Design”, in Proc. Int. Symp. Math. Th. of Net. and Syst., Perpignan, France, June, 19–23, 2000. [LJU 87] L. L JUNG, System Identification: Theory for the User, Prentice Hall, Englewood Cliffs, NJ, 1987.

Basic Concepts of Bioprocess Control

171

[LU 99] J. L U and R.E. S KELTON, “Integrating Instrumentation and control”, Int. J. Control, vol. 72, no. 9, pp. 799–814, 1999. [MON 94] G. M ONTAGUE and J. M ORRIS, “Neural network contribution in biotechnology”, TIBTECH, vol. 12, pp. 312–323, 1994. [RAP 02] A. R APAPORT and J. H ARMAND, “Robust regulation of a class of partially observed nonlinear continuous bioreactors”, J. Proc. Control, vol. 12, no. 2, pp. 291–302, 2002. [SEB 89] D. S EBORG, T.F. E DGAR and D.A. M ELLICHAMP, Process Dynamics and Control, John Wiley, New York, 1989. [STE 99] J.P. S TEYER, P. B UFFIÈRE, D. ROLAND and R. M OLETTA, “Advanced control of anaerobic of anaerobic digestion processes through disturbances monitoring”, Wat. Res., vol. 33, no. 9, pp. 2059–2068, 1999. [STE 00] G. S TEPHANOPOULOS and C. N G, “Perspectives on the systhesis of plant-wide control structures”, J. Proc. Control, vol. 10, pp. 97–111, 2000. [WIL 70] J.L. W ILLEMS, Stability Theory of Dynamical Systems, Nelson, London, 1970.

172

Chapter 7

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

7.1. Introduction Industrial-scale biotechnological processes have progressed vigorously over recent decades. As already mentioned, the problems arising from the implementation of these processes are similar to those of more classical industrial processes and the need for automatic control in order to optimize production efficiency, improve product quality or detect disturbances in process operation is obvious. Nevertheless, automatic control of industrial biotechnological processes is facing two major difficulties: a) it remains difficult to develop models taking into account the numerous factors that can influence microorganism growth and other biochemical reactions; b) another essential difficulty lies in the absence, in most cases, of cheap and reliable instrumentation suited to real-time monitoring. The classical monitoring and control methods do not prove very efficient to tide over these difficulties. The efficiency of any control system highly depends on the design of the control and monitoring techniques and the care taken in their design. In fact, monitoring or control algorithms will prove to be efficient if they are able to incorporate the important well-known information on the process while being able to deal with the missing information (lack of online measurements, uncertainty on the dynamics, etc.) in a “robust” way, i.e. such that the missing information will not significantly

Chapter written by Denis D OCHAIN, Martin G UAY, Michel P ERRIER and Mariana T ITICA.

173

174

Bioprocess Control

deteriorate the control performance of the process. The versatility of computing platforms greatly facilitates the design and implementation of sophisticated controllers (beyond the classical PID as presented in Chapter 6). These controllers may arise from quite complex theory (nonlinear control, adaptive control, extremum-seeking control) but, as will be shown, their structure and implementation will remain rather simple while including the key features of simple PIDs. In this chapter, we shall show how to incorporate the well-known features of the dynamics of biochemical processes (basically, the reaction network and the material balances) in control algorithms which are capable of dealing with process uncertainty (in particular on the reaction kinetics) by introducing, an adaptation scheme for example in the control algorithms. It is also important to draw to the attention of the reader that these control systems are not just the object of academic research but are already used in several applications (see e.g. [BAS 90, CHE 91]). Adaptive as well as non-adaptive linearizing control of bioreactors has been a quite active research area over recent decades. In addition to the works by the authors of this chapter and their colleagues, let us also mention e.g. [ALV 88, CHI 91, DAH 91, FLA 91, GOL 86, HEN 92, HOO 86]. The chapter is organized as follows. We shall first concentrate on the design of adaptive linearizing controllers for bioprocesses based on a reduced order model of the process. The methodology will be illustrated by anaerobic digestion and activated sludge. We shall then consider online optimization approaches, adaptive extremumseeking control, whose specificity is to combine feedback control with a search algorithm for the optimal process operating conditions. In that section, we shall consider the illustrative example of fed-batch reactors. 7.2. Adaptive linearizing control of bioprocesses 7.2.1. Design of the adaptive linearizing controller Let us first concentrate on the design of model-based controllers for bioreactors. The key idea of the control design is here again to take advantage of what is well known about the dynamics of bioprocesses (reaction network and mass balances) which are summarized in the general dynamic model already presented in the preceding chapters: dξ = Kr(ξ) + F − Q − Dξ dt

(7.1)

while taking into account the model uncertainty (mainly the kinetics). Since the model is generally nonlinear, the model-based control design will result in a linearizing control structure, in which the online estimation of the unknown variables (component

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

175

concentrations) and parameters (reaction rates and yield coefficients) are incorporated. The resulting controller will be an adaptive linearizing controller. After having introduced the general control design formulation, we shall present two typical adaptive linearizing control designs: 1) control of anaerobic digestion processes; 2) control of activated sludge processes. One interesting aspect of the control design in the first example will be to show how to eliminate the (unknown) kinetics terms by incorporating the gas measurements into the controller. The first example is concerned with the SISO (Single Input Single Output) control case, while the second one will concentrate on the design of a MIMO (Multi-Input Multi-Output) adaptive linearizing controller. The design of the control algorithm is based on the general dynamic model (7.1) or on a reduced-order form of (7.1). In each case, the control design follows the same line of reasoning described below. Let us first define the control problem. The objective is to control the concentration of some reactant components (one in Example 1, two in Example 2) by acting on flow rates (the air flow rate and the recycle flow rate in Example 2, the dilution rate in Example 1) under the following practical constraints: 1) the components to be controlled are assumed to be measured online; 2) the concentrations of the other components (particularly of the biomass) are not available for online measurement; 3) the reaction rate vector r is unknown; 4) most of the yield coefficients are unknown (only the ratio k1 /k2 will be assumed to be known, in Example 2); 5) the gas-liquid mass transfer coefficients are known; 6) the feedrates F , the dilution rate D and the gaseous outflow rates Q are known (either by user’s choice or by measurement). By defining y the controlled component(s), the dynamics of y are simply the equation(s) of y in model (7.1) and can be rewritten as follows: dy = f (Fy , Qy , Ky r) + g(Fy , y)u dt

(7.2)

where index y holds for the rows of K, F and Q corresponding to the controlled output y. Assume now that we wish to have a linear stable closed-loop (process + controller) dynamic behavior, i.e.:   dy = C1 y ∗ − y , dt

C1 > 0

(7.3)

176

Bioprocess Control

with y ∗ the desired value of y. By combining equations (7.2), (7.3) and if g is invertible (i.e. if g −1 exists), we readily obtain the control law:  −1   ∗    u = g Fy , y C1 y − y − f Fy , Qy , Ky r (7.4) Since the kinetics and most of the yield coefficients are assumed to be unknown, they are replaced by online estimates of selected parameters:  −1   ∗    ˆ y rˆ C1 y − y − f Fy , Qy , K u = g Fy , y (7.5) The above controller (7.5) is also known as the model reference adaptive linearizing control law (see e.g. [BAS 90]). As we shall see in the examples, the unknown parameters appear linearly in the equations and will therefore be estimated by using linear regression techniques (Example 2) or via a Lyapunov design estimation approach (Example 1). Moreover, the unknown components that may appear explicitly in output equation (7.2) via the reaction rate r will be replaced either by an auxiliary variable that is easy to calculate (Example 2) or by gaseous outflow rates (Example 1); in the latter instance, the effect will also make the kinetics terms disappear. 7.2.2. Example 1: anaerobic digestion As already mentioned in the preceding chapters, anaerobic digestion is a biological wastewater treatment process that produces methane. Four major metabolic pathways [MOS 83] can be identified in this process: two for acidogenesis and two for methanization. In the first acidogenic path (Path 1), glucose is decomposed into fatty volatile acids (acetate, propionate), hydrogen and inorganic carbon by acidogenic bacteria. In the second acidogenic path (Path 2), OHPA (Obligate Hydrogen Producing Acetogens) decompose propionate into acetate, hydrogen and inorganic carbon. In a first methanization path (Path 3), acetate is transformed into methane and inorganic carbon by acetoclastic methanogenic bacteria, while in the second methanization path (Path 4), hydrogen combines with inorganic carbon to produce methane under the action of hydrogenophilic methanogenic bacteria. The process can then be described by the following reaction network: S1 −→ X1 + S2 + S3 + S4 + S5

(7.6)

S2 −→ X2 + S3 + S4 + S5

(7.7)

S3 −→ X3 + S5 + P1

(7.8)

S4 + S5 −→ X4 + P1

(7.9)

where S1 , S2 , S3 , S4 , S5 , X1 , X2 , X3 , X4 and P1 are respectively glucose, propionate, acetate, hydrogen, inorganic carbon, acidogenic bacteria, OHPA (obligate hydrogen-producing acetogens), acetoclastic methanogenic bacteria, hydrogenophilic

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

177

methanogenic bacteria and methane. The dynamic model of the anaerobic digestion process (N = 10, M = 4) in a stirred tank reactor can be described within the above formalism (7.1) by using the following definitions: ⎡ ⎤ ⎡ ⎤ 1 0 0 0 X1 ⎢−k ⎢S ⎥ 0 0 0 ⎥ ⎢ 21 ⎥ ⎢ 1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 ⎢X2 ⎥ 1 0 0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ k41 −k42 0 ⎢ S2 ⎥ 0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 ⎢X ⎥ 0 1 0 ⎥ ⎢ ⎥ ⎢ 3⎥ (7.10) ξ = ⎢ ⎥, K = ⎢ ⎥ ⎢ k61 ⎢ S3 ⎥ k62 0 −k63 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 ⎢X4 ⎥ 0 0 1 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢k ⎢S ⎥ k82 0 −k84 ⎥ ⎢ 81 ⎥ ⎢ 4⎥ ⎢ ⎥ ⎢ ⎥ ⎣ k91 ⎣ S5 ⎦ k92 k93 −k94 ⎦ P1 0 0 k03 k04 ⎡ ⎤ ⎡ ⎤ 0 0 ⎢0⎥ ⎢DS ⎥ ⎢ ⎥ ⎢ in ⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢ 0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎡ ⎤ ⎡ ⎤ ⎢0⎥ ⎢ 0 ⎥ r1 μ1 X1 ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢ 0 ⎥ ⎢r ⎥ ⎢μ X ⎥ ⎢ ⎥ ⎢ ⎢ 2⎥ ⎢ 2 2⎥ ⎥ F =⎢ (7.11) ⎥, Q = ⎢ ⎥, r = ⎢ ⎥ = ⎢ ⎥ ⎢0⎥ ⎢ 0 ⎥ ⎣r3 ⎦ ⎣μ3 X3 ⎦ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢ 0 ⎥ r4 μ4 X4 ⎢ ⎥ ⎢ ⎥ ⎢Q ⎥ ⎢ 0 ⎥ ⎢ 1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣Q2 ⎦ ⎣ 0 ⎦ 0 Q3 where μ1 , μ2 , μ3 , μ4 are the specific growth rates (h−1 ) of reactions (7.6), (7.7), (7.8), (7.9), respectively, and Sin , Q1 , Q2 and Q3 represent respectively the influent glucose concentration (g/l) and the gaseous outflow rates (g/l/h) of H2 , CO2 and CH4 . 7.2.2.1. Model order reduction The examples of bioprocesses presented in the preceding chapters and here above have shown that a bioreactor dynamic model may, in some instances, be fairly complex and involve a large number of differential equations. However, there are many practical applications where a simplified reduced order model is sufficient from an engineering viewpoint. Model simplification can be achieved by using the singular perturbation technique, which is a technique for transforming a set of n + m differential equations into a set of n differential equations and a set of m algebraic equations. This technique is suitable when neglecting the dynamics of substrates and of products with low solubility in the liquid phase. As explained in [BAS 90], the rule for

178

Bioprocess Control

model simplification is actually simple. Consider that, for some i, the dynamics of component ξi are to be neglected. The dynamics of ξi are described by equation (7.1): dξi = −Dξi + Ki r + Fi − Qi dt

(7.12)

where Ki is the line of K corresponding to the component ξi . The simplification is then achieved by setting ξi and dξi /dt to zero i.e. by replacing differential equation (7.12) by the following algebraic equation: Ki r = −Fi + Qi

(7.13)

It has been shown that the above model order reduction rule is valid for low solubility products and for bioprocesses with fast and slow reactions. In the latter instance, the above order reduction rule (7.13) applies to substrates of fast reactions (as long as they intervene only in fast reactions) (see [VAN 91] for further details). Note the close connection between the singular perturbation reduction and the quasi-steadystate (QSS) approximation, which is widely used in (bio)chemical engineering. This suggests the following comment: singular perturbation can be viewed as an efficient mathematical tool to rigourously justify QSS approximations on a systematic basis via an appropriate analysis (including the choice of an appropriate small perturbation parameter). Let us see how to apply the above model order reduction rule (7.13) to the anaerobic digestion. First of all, it is well-known that methane is a product of low solubility. Furthermore, assume that the second methanization path (hydrogen consumption) is limiting, i.e. that the first three reactions (7.6), (7.7), (7.8) are fast and the fourth one (7.9) is slow. We can then apply the model order reduction rule (7.13) to the glucose concentration S1 , the propionate concentration S2 , the acetate concentration S3 and the dissolved methane concentration P1 . By setting their values and their time derivatives to zero: S1 = S2 = S3 = P1 = 0, dS2 dS3 dP1 dS1 = = = =0 dt dt dt dt

(7.14)

we reduce their differential equations to the following set of algebraic equations: ⎡ −k21 ⎢k ⎢ 41 ⎢ ⎣ k61 0

0 −k42 k62 0

0 0 −k63 k03

⎤⎡ ⎤ ⎡ ⎤ r1 −DSin 0 ⎢ ⎥ ⎢ ⎥ 0 ⎥ ⎥ ⎢r2 ⎥ ⎢ 0 ⎥ ⎥⎢ ⎥ = ⎢ ⎥ 0 ⎦ ⎣r3 ⎦ ⎣ 0 ⎦ Q3 k04 r4

(7.15)

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

179

By inverting the submatrix of the yield coefficients of the left hand side of (7.15), we can express the reaction rates r1 , r2 , r3 and r4 as functions of the feedrate DSin and of the gaseous methane outflow rate Q3 : r1 =

1 DSin k21

(7.16)

r2 =

k41 DSin k21 k42

(7.17)

r3 =

k41 k62 + k42 k61 DSin k21 k42 k63

(7.18)

r4 =

1 k03 k41 k62 + k42 k61 Q3 − DSin k04 k04 k21 k42 k63

(7.19)

Let us replace the reaction rates r1 , r2 and r4 by their above expressions (7.16), (7.17), (7.19) in the dynamic equation of the hydrogen concentration S4 , which is then rewritten as follows: dS4 = −DS4 − Q1 − k1 Q3 + k2 DSin dt

(7.20)

where k1 and k2 are defined as follows: k1 =

k84 , k04

k2 =

k81 k41 k82 k03 k84 k41 k62 + k42 k61 + + k21 k21 k42 k04 k21 k42 k63

(7.21)

Equation (7.20) will be used for the design of an adaptive linearizing controller of the hydrogen concentration. Note that coefficients k1 and k2 are nonlinear combinations of the yield coefficients kij . 7.2.2.2. Adaptive linearizing control design The importance of implementing efficient control systems for anaerobic digestion processes clearly appears from the following two points: 1) anaerobic digestion is intrinsically a very unstable process: variations of the input variables (hydraulic flow rate, influent organic load) may easily lead the process to a wash-out, i.e. a state where the bacterial life has disappeared. This phenomenon takes place under the form of acid accumulation in the reactor (see [BIN 83, FRI 84]). It is therefore essential to implement controllers which are capable of stabilizing the process via a carefully designed control strategy; 2) if the process is used for wastewater treatment purposes, the control objective consists of maintaining the output pollution at a prescribed level despite the fluctuations of the input pollution (organic load). From the above two comments, it is clear that control strategies should concentrate on the control of the substrate concentration (which characterizes the pollution level

180

Bioprocess Control

and the presence of acids). However, intricate difficulties inherent to the process make the control problem very hard to solve. Anaerobic digestion is a complex process in which many different bacterial populations intervene. Its kinetics are basically nonlinear and non-stationary, and they are far from being fully understood. Moreover, the concentration of the different bacterial populations are not available from any direct measurement, even from offline analyses. Finally, there remains the problem of choosing an appropriate substrate to be controlled. COD (chemical oxygen demand) is a priori a very good candidate: an adaptive linearizing controller has been designed, theoretically analyzed and experimentally validated on a pilot anaerobic digester (for further details, see [DOC 84, DOC 86, REN 88]). However, the industrial applicability of this control solution may be limited by the need for online COD (or equivalent substrate concentration) measurements. Therefore, there is a clear incentive to look for alternative substrate candidates. In this context, the use of hydrogen as a controlled variable appears to be potentially interesting since hydrogen plays an important role in the kinetics and stability of anaerobic digestion, particularly when the organic substrate is mainly composed of glucose (e.g. waste from the sugar industry), and since hydrogen is easy to measure online [PAU 90]. Let us now deal with the controller design. One specific aspect of the anaerobic digestion example is that the design of the controller is based on a “reduced-order” form (equation (7.20)) of the general dynamic model (7.10)-(7.11). Let us further motivate the use of a reduced order model. The design of a controller based on a reduced order model will result in a simpler controller structure but will still contain the important process features if the model reduction has been correctly performed (here the controller will incorporate the influent organic matter concentration Sin which can be viewed as a feedforward term and the methane gas production rate Q3 which gives important information about the state of the process). Recall that the order reduction is based on the notion of fast and slow reactions. It is difficult to know a priori which reactions in the anaerobic digestion reaction network will be, in a broad range of operating conditions, fast or slow. On the other hand, we may wish to avoid undesirable effects such as the accumulation of propionate or hydrogen; then, in that situation, the corresponding reaction (reaction (7.7) or (7.9)) can be considered as the limiting, i.e. “slow”, reaction, and the others as “fast” reactions. For instance, assume that the main problem is to avoid the accumulation of hydrogen, then reaction (7.9) is the slow reaction and reactions (7.6)–(7.8) are the fast reactions. Let us go back to our control problem. If we consider that the control input is the dilution rate D, the expressions of f and g are here equal to: f = −Q1 − k1 Q3 ,

g = k2 Sin − S4

(7.22)

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

181

Figure 7.1. Control of the hydrogen concentration in an anaerobic digestion reactor

Controller (7.5) is written here as follows:   C1 S4∗ − S4 + Q1 + kˆ1 Q3 D= kˆ2 Sin − S4

(7.23)

The unknown parameters are now k1 and k2 . They can be estimated online by using an appropriate updating, for instance here a “Lyapunov design” estimation equation: dkˆi = C2i (S4∗ − S4 ), C2i > 0, i = 1, 2 (7.24) dt A typical simulation is shown in Figure 7.1 (see also [DOC 91, DOC 92, PER 93]). In this simulation, the hydrogen gaseous outflow rate Q1 has been assumed to be negligible, and parameter k2 has been assumed to be known. Therefore, only k1 has been estimated online. The anaerobic process has been simulated by integration of the mass balance equations with the following yield coefficients, specific growth rate expressions and initial conditions: Yield coefficients k1 = 3.2, k2 = 1.5, k3 = 0.7, k4 = 12, k5 = 0.27, k6 = 0.53 k7 = 1.15, k8 = 0.6, k9 = 0.1, k14 = 0.3, k15 = 0.08

182

Bioprocess Control

Specific growth rate expressions μ1 =

μmax 1 S1  , 1 + S2 + S3 /KI1

μ3 =

μmax 3 S3 , KS3 + S3 + S32 /KI3



μ2 =

S μ  max 2 2  KS2 1 + S3 /KI2

μ4 =

μmax 4 S4 KS4 + S4 + S4 /KI4

with: μmax1 = 0.2, KI1 = 10, μmax2 = 0.5, KS2 = 0.4, KI2 = 0.5 μmax3 = 0.4, KS3 = 0.5, KI3 = 4, μmax4 = 0.5, KS2 = 4, KI4 = 3 Initial steady-state conditions S1 (0) = 0.79 g/l, S2 (0) = 0.28 g/l, S3 (0) = 0.31 g/l, S4 (0) = 3.0 μM X1 (0) = 7.6 g/l, X2 (0) = 3.3 g/l, X3 (0) = 0.29 g/l, X4 (0) = 1.6 g/l The performance of the controller was evaluated for an influent substrate concentration (Sin1 ) change from 25 to 35 g/l applied at t = 2 days. The value of the design parameter C1 was chosen to be equal to 1 d−1 . The parameter estimation dynamics is driven by C2 . As for the online estimation of the specific growth rates (section 7.5), in order to have a closed-loop dynamic independent of the (time-varying) value of the regressor Q3 , the design parameter C2 was chosen to be inversely proportional to the regressor, i.e.: C2 =

C1 4Q23

(7.25)

The concentration of each substrate and bacterial population is shown in Figures 7.1a and b, respectively, for dissolved hydrogen control. The controller proves to be able to maintain the hydrogen concentration at the desired set-point. The concentration of bacterial populations slowly reaches new levels while the other substrates go back to their initial values. NOTE. A similar approach has been used to design an adaptive linearizing controller for propionate as well as for COD (as already mentioned). It is worth noting that the three control designs result in similar controller structures [DOC 92, PER 93]. The controllers of propionate and of COD have been successfully applied to a pilot-scale anaerobic digestion reactor [REN 88, REN 91]. Discussion One of the key features of adaptive linearizing can be illustrated on the basis of the anaerobic digestion example: it allows us to incorporate the well-known characteristics of the process dynamics, while keeping the usual features of classical controllers:

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

183

– proportional action: via the term C1 (S4∗ − S4 ); – integral action: via the adaptation mechanism (7.24); – feedforward action: via the presence of Sin . In fact the controller is capable of anticipating the effect of a variation of the influent substrate concentration: for example, an increase of Sin will result in a decrease of the control input D, inversely proportional to this increase. In addition, the controller contains a state “estimate” via the term kˆ1 Q3 . If the process is in good operating conditions, in particular if the biomass is in an adequate state to produce a lot of methane, then it is possible to treat high amounts of organic matter, since the control action D is proportional to Q3 . Finally, note that in addition to the integral action, the estimation of “physical” parameters (here k1 , which is in fact a conversion coefficient of hydrogen into methane) has the further advantage of giving useful information, which can be used for monitoring the process, and possibly also for analyzing the internal working of the process. 7.2.3. Example 2: activated sludge process The activated sludge process is another traditional biological (but aerobic) wastewater treatment process. It is usually operated in two sequential tanks: an aerator (in which the degradation of the pollutants S takes place) and a settler (which is used to recycle part of the biomass X to the aerator). The reaction in the aerator is usually described by a simple microbial growth (see e.g. [HAM 75, MAR 84]): S + C −→ X

(7.26)

while the dynamics of the settler are described by the following mass balance equation: Fin + FR FR + FW dXR = X− XR dt VS VS

(7.27)

where XR is the concentration of the recycled biomass (g/l), Fin , FR and FW are the influent, recycle and waste flow rates (g/l/h), respectively, and VS the settler volume (7.1). By considering the volume V (7.1) of the aerator and defining: Fin , V

D2 =

FR , V

D1 = Din + D2 ,

D3 =

Fin + FR VS

Din =

D4 =

FR + FW VS

(7.28)

(7.29)

184

Bioprocess Control

The dynamic equations of the process (N = 4, M = 1) can be rewritten in the formalism of the general dynamic model (7.1) with the following definitions: ⎡ ⎡ ⎤ ⎤ ⎡ ⎤ Din Sin −k1 S ⎢ ΔQ ⎥ ⎢−k ⎥ ⎢C ⎥ O2 ⎥ ⎢ ⎢ 2⎥ ⎢ ⎥ (7.30) ξ = ⎢ ⎥, K = ⎢ ⎥, F = ⎢ ⎥ , r = μX ⎣ 0 ⎦ ⎣ 1 ⎦ ⎣X⎦ XR 0 0 ⎡ ⎤ D1 0 0 0 ⎢0 D 0 0 ⎥ 2 ⎢ ⎥ D=⎢ (7.31) ⎥, Q = 0 ⎣0 0 D1 −D2 ⎦ 0 0 −D3 D4 Note that D is now a matrix. In activated sludge processes, the oxygen feed rate term ΔQO2 in the dynamic equation of the dissolved oxygen is usually set equal to the liquid-gas oxygen transfer rate:   ΔQO2 = kL a CS − C (7.32) where kL a is the oxygen mass transfer coefficient and CS the saturation constant. In line with [HOL 82], we shall also consider in the following that kL a is a linear function of the air flow rate W : kL a = a0 W,

a0 > 0

(7.33)

This example is treated in detail in [DOC 92]. The control problem we shall consider now can be formulated as follows. As with any aerobic fermentation processes, proper aeration is crucial to process efficiency, and adequate control of the dissolved oxygen concentration in the aerator is very important. However, it is also important to limit load variations and substrate concentration variations by acting on them. As suggested by many authors (e.g. [HOL 83, MAR 84]), load variations can be expected to be smoothed by using the recycle flow rate as a control input. We shall consider the above control algorithm under the following conditions: 1) the controlled outputs are the effluent BOD (Biological Oxygen Demand) and the dissolved oxygen concentrations, which are assumed to be measured online (see [KHO 85] for an online measuring device of the BOD in activated sludge processes); 2) the control inputs are the recycle flow rate and the air flow rate. Therefore, input u and output y are defined as follows:   S F u= R , y= W C

(7.34)

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

185

The expressions of f and g in (7.2) are readily obtained from model (7.31):    S  0 −V −k1 μX + Din Sin − S   , g= (7.35) f= −k2 μX − Din C − VC a0 CS − C g is invertible (i.e. g −1 exists) as long as S and CS − C are different from zero. Under these physically realistic conditions, linearizing controller (7.5) can be applied. The remaining problem is how to deal with the “unknown” parameters k1 , k2 and μ, and state variable X. The solution proceeds as follows. Let us first define the auxiliary variables ζ: ⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 k1 S ζ1 (7.36) z = ⎣ζ2 ⎦ = ⎣k2 ⎦ X + ⎣0 1 0 ⎦ ⎣ C ⎦ ζ3 0 0 0 k2 XR The dynamic equation of ζ is derived from model (7.31) and definitions (7.36): ⎤ ⎡ ⎤ ⎡ k1 ⎤ ⎡ ⎤ ⎡ ζ1 Din Sin ζ1 −D1 0 D2 d ⎣ ⎦ ⎢ k2 ⎥ ⎣ζ ⎦ ⎣ ΔQ ⎦ ζ2 = ⎣ (7.37) O2 0 −D1 D2 ⎦ 2 + dt ζ3 ζ3 −D3 C 0 D3 −D4 One important feature of the above dynamics (7.37) is that these are independent of the (unknown) kinetics. Equation (7.37) is a dynamic system linear-in-the-state-ζ. We can check that its state matrix is asymptotically stable if Fin − FR > 0 and FW > 0. Therefore equation (7.37) can be used to calculate the value of ζ online from the knowledge of the flow rates Fin , FR and FW , the feedrates Fin Din and ΔQO2 , the dissolved oxygen concentration C and the ratio k1 /k2 . Let now rewrite the specific growth rate μ as follows: μ = αSC

(7.38)

where α is an (unknown) positive function of the process components: equation (7.38) simply implies that there is no growth in the absence of one of the limiting substrates. By introducing (7.36), (7.38), function f in (7.35) becomes:      −αSC ζ1 − S + Din Sin − S   f= (7.39) −αSC z2 − C − Din C This expression of f is used in the calculation of the control law (7.5):       V ˆ 1 SC ζ1 − S FR = − C11 S ∗ − S − Din Sin − S + α S     ∗  C11 S ∗ − S C 1  C12 C − C − W =  S a0 CS − C   Din Sin C + +α ˆ 2 C Sζ2 − Cζ1 S

(7.40)

(7.41)

186

Bioprocess Control

C11 and C12 (> 0) are the control design parameters, i.e. we have considered here a decoupling diagonal matrix C1 :  C11 C1 = 0

0 C12

(7.42)

Variables ζ1 and ζ2 are calculated via the use of equations (7.37) (which only require the knowledge of the ratio k1 /k2 ) and parameter α is estimated online by using e.g. a recursive least squares (RLS) algorithm. In discrete-time, if α is estimated via the substrate concentration equation S (α1 , to be used in equation (7.40)) or via the dissolved oxygen concentration C (α2 , to be used in equation (7.41)), the RLS algorithm is written as follows: θˆi,t+1 = θˆi,t + gi,t φi,t ei,t+1 gi,t =

gi,t−1 γi + gi,t−1 φ2i,t

i = 1, 2

0 < γi ≤ 1

(7.43) (7.44)

with: θˆ1,t = α ˆ 1,t ,

  φ1,t = T St Ct St − z1,t

ˆ 1,t φ1,t e1,t+1 = St+1 − St + T D1,t St − T Din,t Sin,t − α   θˆ2,t = α ˆ 2,t , φ2,t = T St Ct Ct − z2,t   ˆ 2,t φ2,t e2,t+1 = Ct+1 − Ct + T D1,t Ct − T a0 Wt Cs − Ct − α

(7.45) (7.46) (7.47) (7.48)

and t the time index, and γi (i = 1, 2) a forgetting factor. Note that now because of variables ζ, biomass concentration X no longer appears explicitly in the controller (7.40)-(7.41). A typical simulation result is shown in Figure 7.2 (see also [DOC 92]). The activated sludge process has been simulated by numerical integration of the basic dynamic model equations (7.31) with the following (Monod-type) model for the specific growth rate: μ = μmax

C S KS + S KC + C

(7.49)

and a decay rate (−kd X) which has been added to the biomass equation (in order to take into account biomass mortality). The specific growth rate model and the decay rate term are (obviously) completely ignored by the control algorithm. The decay rate term can be viewed as an unmeasured disturbance. The model parameters have been

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

187

Figure 7.2. Control of an activated sludge process in the presence of an unknown biomass death rate

set to the following values (inspired from data from the literature, in particular from [HOL 83, MAR 89]): k1 = 1.2, k2 = 0.565, μmax = 0.2 h−1 , KS = 75 mg/l kd = 0.001 h−1 , a0 = 0.018 m−3 , CS = 10 mg/l, V = 100 m3 VS = 50 m3 , KC = 2 mg/l In Figure 7.2, a square wave signal of the influent BOD concentration (disturbance input) Sin (from 150 mg/l to 200 mg/l) (in order to simulate the periodical variation of the pollutant load) has been applied over a period of 20 days (480 hours). The sampling period has been set to 3 minutes following the constraints of the commercially available BOD measuring device proposed by [KHO 85]. The following initial process conditions have been considered in the simulation: S = 5 mg/l, C = 6 mg/l, X = 1225 mg/l, XR = 2333 mg/l

(7.50)

The controller parameters have been set to the following values: S ∗ = 5 mg/l, C ∗ = 6 mg/l, C11 = 1 h−1 , C12 = 10 h−1

(7.51)

188

Bioprocess Control

The auxiliary variables ζ, the unknown parameters and tuning estimation variables have been initialized as follows: ζ1 = 1400 mg/l, ζ2 = 750 mg/l, ζ3 = 1400 mg/l, g1,0 = g2,0 = 10−3 γ1 = γ2 = 0.9, α ˆ 1,0 = α ˆ 2,0 = 0.00025 l2 /mg2 /h Note the ability of the controller to maintain the controlled outputs S and C close to their desired values in spite of the unknown disturbance. 7.3. Adaptive extremum-seeking control of bioprocesses Most adaptive control schemes documented in the literature (e.g. [AST 95, GOO 84, KRS 95, NAR 89]) are developed for regulation to known set-points or tracking known reference trajectories. In some applications, however, the control objective could be to optimize an objective function which can be a function of unknown parameters, or to select the desired values of the state variables to keep a performance function at its extremum value. Self-optimizing control and extremum-seeking control are two methods of handling these types of optimization problems. The task of extremum seeking is to find the operating set-points that maximize or minimize an objective function. Since the early research work on extremum control in the 1920s [LEB 22], several applications of extremum control approaches have been reported (e.g. [AST 95, DRK 95, STE 80, VAS 57]). Krstic et al. [KRS 00a, KRS 00b] presented several extremum control schemes and stability analysis for extremum-seeking of linear unknown systems and a class of general nonlinear systems [KRS 00a, KRS 00b, KRS 98]. A neural network-based approach has been proposed in [LI 95]. The implications for the biochemical industry are clear. In this sector, it is recognized that even small performance improvements in key process control variables may result in substantial economic gains. As an example, the potential benefits of extremum-seeking techniques in the maximization of biomass production rate in wellmixed biological processes has been demonstrated in [WAN 99]. The proposed scheme utilizes explicit structure information of the objective function that depends on system states and unknown plant parameters. This scheme is based on Lyapunov’s stability theorem. As a result, global stability is ensured during the seeking of the extremum of the nonlinear stirred tank bioreactors. It is also shown that once a certain level of persistence of excitation (PE) condition is satisfied, the convergence of the extremum-seeking mechanism can be guaranteed. In this section we concentrate on the adaptive extremum-seeking control of bioreactors operating in the fed-batch mode (see also [TIT 03a, TIT 03b]) but several other alternatives of the similar schemes (including the use of universal approximation like artificial neural networks (ANN)) have been proposed in the literature for different bioprocess configurations [GUA 04, HAR 06, MAR 04, MAR 03, ZHA 02].

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

189

7.3.1. Fed-batch reactor model Fed-batch bioreactors represent an important class of bioprocesses, mainly in the food industry and in the pharmaceutical industry but also e.g. for biopolymer applications (PHB). One of the key issues in the operation of fed-batch reactors is to optimize the production of a synthesis product (e.g. penicillin, enzymes, etc.) or biomass (e.g. baker’s yeast). They are therefore ideal candidates for optimal control strategies. A intensive research activity was devoted to optimal control of (fed-batch) bioreactors mainly in the 1970s and in the 1980s (see e.g. [OHN 76, CHE 79, PAR 85, PER 79]). However, in practice, because of the large uncertainty related to the modeling of the process dynamics [BAS 90], poor performance may be expected from such control strategies, and although a priori attractive, optimal control has not been largely applied to industrial bioprocesses. Alternative approaches have been proposed for handling the process uncertainties with an adaptive control scheme (e.g. [DOC 89, VAN 93]). In this approach, we propose to go a step further by including a static optimum search in the adaptive control scheme. Consider the following dynamic model of a simple microbial growth process with one gaseous product in a fed-batch reactor: dX = μX − DX dt dS = −k1 μX + D(S0 − S) dt y = k2 μX

(7.52) (7.53) (7.54)

dv = Dv dt

(7.55)

where states X (g/l) and S (g/l) hold for biomass and substrate concentrations, respectively. μ (h−1 ) is the specific growth rate, D (h−1 ) is the dilution rate, y (g/l/h) is the production rate of the reaction product, S0 (g/l) denotes the concentration of the substrate in the feed, k1 , k2 are yield coefficients, and v (l) is the volume of liquid medium in the tank. A typical situation in bioprocess applications is when the biomass concentration is not available for online measurement while the gaseous outflow rate y (e.g. CO2 ) is easier to measure online. We consider here that only S and y are measurable while the biomass concentration X is not available for feedback control. In this work, we consider the extremum-seeking problem for the bioprocess model (7.52)-(7.55) with a specific growth rate μ expressed by the Haldane model. This model (see Figure 7.3) is given by: μ=

μ0 S KS + S +

S2 KI

(7.56)

190

Bioprocess Control

Figure 7.3. Haldane model

where μ0 is a parameter related to the maximum value of the specific growth rate as -

S follows: μ0 = μmax (1 + 2 K KI ). Coefficients KS and KI denote the saturation constant and the inhibition constant, respectively. The Haldane model is a growth model commonly used in situations where substrate inhibition is important. This situation is typical of fed-batch bioreactors. The control objective is to design a controller with D as the control action such that the biomass production V X achieves its maximum at the end of the fed-batch operation. It is well-known (e.g. [BAS 90]) that the maximization will be completed if the specific growth rate is kept at its maximum value:

(7.57) S ∗ = KS KI

From the above considerations, we know that if the substrate concentration S can be stabilized at the set-point S ∗ then the production of biomass is maximized. However, since the exact values of the Haldane model parameters KS , μ0 and KI are usually unknown, the desired set-point S ∗ is not available. In this work, an adaptive extremum-seeking algorithm is developed to search this unknown set-point such that the biomass production at the end of the reactor operation, i.e. v(tf )X(tf ) (with tf the final time of the fed-batch operation) is maximized. In the technical developments here below, we shall consider the following assumption for parameters KS and KI of the Haldane model.

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

191

Assumption: KS and KI are known to be bounded as follows: KS,min ≤ KS ≤ KS,max , 0 < KI ≤ KI,max . This assumption is only important for the technical developments in order to avoid singularities in the extremum-seeking controller. It should not be interpreted as a “microbial” constraint on the kinetic model. 7.3.2. Estimation and controller design The design of the adaptive extremum-seeking controller will proceed in different steps. First of all, we shall start with the estimation equation for y, then include the controller equations and the estimation equations for the unknown parameters in a Lyapunov-based derivation framework, and end up with the stability and convergence analysis arguments. 7.3.2.1. Estimation equation for the gaseous outflow rate y Let us start with the parameter estimation algorithm for the unknown parameters KS , KI and μ0 . The ratio of the yield coefficients kk12 is assumed to be known. It follows from (7.54) that: y (7.58) μX = k2 Then equations (7.52)-(7.53) can be reformulated as follows: 1 dX = y − DX dt k2

(7.59)

dS k1 = − y + D(S0 − S) dt k2

(7.60)

By considering equations (7.54)-(7.56) and (7.59)-(7.60), the time derivative of y is equal to:  dy KS − S 2 /KI k1 = k2 μ0 X  2 − y + D(S0 − S) dt k2 KS + S + S 2 /KI (7.61)  1 + k2 μ y − DX k2 Since the biomass concentration X is not accessible for online measurement, we y reformulate dy dt by replacing X with k2 μ as follows: 

2

S Ks − K dy I =  dt S KS + S +

S2 KI



−

  k1 2 y + D S0 − S y + μy − Dy k2

(7.62)

192

Bioprocess Control

μ0 Let θ = [θs θμ θi ]T with θμ = K , θs = K1S , θi = KI1KS , and define θk = kk12 . S Equations (7.60) and (7.62) can then be rewritten as follows:   dS = −θk y + D S0 − S (7.63) dt     dy 1 − θi S 2 = D S0 − S − θ k y y 2 dt S(1 + θs S + θi S ) (7.64) θμ Sy + − uy 1 + θs S + θi S 2

Let θˆ denote the estimate of the true parameter θ, and yˆ be the prediction of y by ˆ The predicted state yˆ is generated by the following using the estimated parameter θ. equation:     dˆ y 1 − θˆi S 2 =   D S0 − S − θ k y y 2 ˆ ˆ dt S 1 + θs S + θi S θˆμ Sy + − Dy + ky ey 1 + θˆs S + θˆi S 2

(7.65)

with ky > 0 and the prediction error ey = y − yˆ. It follows from (7.63)-(7.65) that:     dey 1 − θi S 2  D S0 − S − θ k y y = −ky ey +  2 dt s 1 + θs S + θi S +

    1 − θˆi S 2 θμ Sy −  D S0 − S − θ k y y  2 2 ˆ ˆ 1 + θs S + θi S S 1 + θs S + θi S

−

θˆμ Sy 1 + θˆs S + θˆi S 2

(7.66)

7.3.2.2. Design of the adaptive extremum-seeking controller The desired set-point (7.57) can be re-expressed as follows: 1 S∗ = √ θi

(7.67)

Since parameter θi is unknown, we design a controller to drive the substrate concentration S to: 1

θˆi i.e. an estimate of the unknown optimum S ∗ . An excitation signal is then designed and injected into the adaptive system such that the estimated parameter θˆi converges to its true value. The extremum-seeking control objective can be achieved when the substrate concentration S is stabilized at the optimal operating point S ∗ .

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

193

Define the “control” error zs : 1 zs = S − − d(t) θˆi

(7.68)

where d(t) ∈ C 1 is a dither signal that will be assigned later. The time derivative of zs is given by:   1 − 3 dθˆi dzs ˙ = Γ1 = D S0 − S − θk y + θˆi 2 − d(t) dt 2 dt We consider a Lyapunov function candidate:   e2y  1 θ˜μ2 zs2 θ˜s2 θ˜i2 + V = + + + 1 + θs S + θi S 2 2 2 γμ γs γi 2

(7.69)

(7.70)

with constants γμ , γs , γi > 0. Taking the time derivative of V and substituting (7.63), (7.69) and (7.66) leads to: dV θ˜s dθˆs θ˜i dθˆi θ˜μ dθˆμ = z s Γ1 − − − dt γμ dt γs dt γi dt     1 − θi S 2   D S0 − S − θk y y + ey θμ Sy + ey S      1 − θˆi S 2 − ey  D S0 − S − θ k y y  2 S 1 + θˆs S + θˆi S   θˆμ Sy 1 + θs S + θi S 2 + 2 ˆ ˆ 1 + θs S + θi S      θs + 2θi S S0 − S D  2  1 + θ − e2y ky − S + θ S s i 2(1 + θs S + θi S 2   1 + θs + 2θi S θk y 2 Defining the functions:       ˜ = −e2y ky − θs + 2θi S S0 − S D 1 + θs S + θi S 2 Γ 2 1 + θs S + θi S 2  1  − e2y θs + 2θi S θk y 2

(7.71)

(7.72)

Ψi = Γ3 D + Γ4

(7.73)

Ψμ = ey Sy

(7.74)

Ψs = Γ5 D + Γ6

(7.75)

194

Bioprocess Control

where: Γ3 Γ4 Γ5 Γ6 we can write

dV dt

   ey S 1 − θˆi S 2 S0 − S y = −ey S S0 − S y − 1 + θˆs S + θˆi S 2   ey S 1 − θˆi S 2 θk y 2 ey S 3 θˆμ y 2 = ey Sθk y + − 1 + θˆs S + θˆi S 2 1 + θˆs S + θˆi S 2    ey 1 − θˆi S 2 S0 − S y =− 1 + θˆs S + θˆi S 2   ey 1 − θˆi S 2 θk y 2 ey θˆμ S 2 y = − 1 + θˆs S + θˆi S 2 1 + θˆs S + θˆi S 2 



(7.76) (7.77) (7.78) (7.79)

as follows:

    dV 1 − 3 dθˆi 1 dθˆi ˜ ˙ = zs D S0 − S − θk y + θˆi 2 − d(t) + Ψi − θi dt 2 dt γi dt   1 dθˆμ ˜ 1 dθˆs ˜ ˜ + Ψμ − θμ + Ψs − θs + Γ γμ dt γs dt

(7.80)

For the solution of the extremum-seeking problem, we pose the dynamic state feedback: ˆ 3 ˙ = a(t) + 1 θˆ− 2 dθi − kd d(t) d(t) i 2 dt   1 D= − kz zs + θk y + a(t) − kd d(t) , S0 − S

(7.81) kz > 0

(7.82)

where a(t) acts as a dither signal on the closed-loop process and kd is a strictly positive constant. The substitution of (7.82) in (7.80) yields:    ˙ ˙ ˙ dV θˆi ˜ θˆμ ˜ θˆs ˜ ˜ ˜ (7.83) 2 = −kz zs + Ψi − θi + Ψμ − θμ + Ψs − θs Γ + Γ dt γi γμ γs Using the definition of Ψi and Ψs , we express (7.83) as:   ˙ dV θˆi ˜ = −kz zs2 + Γ3 D + Γ4 − θi + Ψμ − dt γi  ˙ θˆs ˜ ˜ + Γ5 D + Γ6 − θs Γ γs

˙ θˆμ ˜ θμ γμ

(7.84)

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

We propose the following update law as follows: .     γi Γ3 D + Γ4 , if θˆi > i or θˆi = i and Γ3 D + Γ4 > 0 ˙ˆ θi = 0 otherwise .     γs Γ5 D + Γ6 , if θˆs > s or θˆs = s and Γ5 D + Γ6 > 0 ˙ ˆ θs = 0 otherwise ˙ θˆμ = γμ Ψμ

195

(7.85)

(7.86) (7.87)

1 > 0, and θˆi (0) ≥ i = Ks,max1KI,max > 0. with the initial condition θˆs (0) ≥ s = KS,max The update laws (7.85)-(7.86) are projection algorithms that ensure that θˆs (t) ≥ s > 0 and θˆi (t) ≥ i > 0. They also ensure that:



  ˙ ˙ ˙ θˆi ˜ θˆμ ˜ θˆs ˜ Γ3 D + Γ4 − θi + Ψμ − θμ + Γ5 D + Γ6 − θs ≤ 0, γi γμ γs

(7.88)

7.3.2.3. Stability and convergence analysis Substituting the update laws (7.85)-(7.86) into (7.84), we obtain: dV ˜ ≤ −kz zs2 + Γ dt

(7.89)

˜ is negative. Using (7.72), We then assign the gain function ky such that the term Γ we consider:   S0 − S |D| (7.90) ky = ky0 + (S + ) with positive constants ky0 > 0 and 0 <   1. As a result, we obtain:   dV ≤ −kz zs2 − ky0 1 + θs S + θi S 2 e2y dt

(7.91)

ˆ zs and as required. Following LaSalle-Yoshizawa’s Theorem, it is concluded that θ, ey are bounded, and: lim zs = 0,

t→∞

lim ey = 0

t→∞

This implies that: ˙ lim θˆi (t) = 0,

t→∞

˙ lim θˆμ (t) = 0,

t→∞

˙ lim θˆs (t) = 0.

t→∞

(7.92)

196

Bioprocess Control

Hence, the auxiliary variable d(t) is bounded if a(t) is bounded and d(t) tends towards zero if a(t) does. Thus, all signals of the closed-loop system are bounded. It should be noted that the convergence of the state error ey does not mean that the estimated parameters converge to their true values as t → ∞. In section 7.4, we investigate the condition that guarantees the parameter convergence. 7.3.2.4. A note on dither signal design The results of the previous section confirm the convergence of the extremumseeking scheme if the persistency of excitation (PE) condition (7.116) is encountered. It remains fairly difficult to ensure that the dither signal employed is sufficiently rich. One of the main difficulties is that the calculation of the PE condition criteria depends on the value of the unknown parameters. In this study, we use the asymptotic value of the PE condition (7.116) to test whether a given dither signal is sufficiently exciting. The approach can be summarized as follows. As t → ∞, the analysis detailed above confirms that limt→∞ zs = 0. This means that asymptotically, the substrate concentration S converges to √1ˆ + d(t). Since θˆi θi

converges to a constant value, the asymptotic dynamics of the substrate concentration, S∞ , coincide with the dynamics of the dither signal given (7.81) which converge to dS∞ = d˙ = a(t) − kd d(t) dt as t → ∞. The asymptotic dynamics of the production rate y∞ are given by:    dy∞ 1 − θ i S∞   a(t) − kd d(t) = 2 dt S∞ 1 + θ s S∞ + θ i S∞    ∞ θ μ S∞ 1  a(t) − kd d(t) y + + S0 − S∞ 1 + θ s S∞ + θ i S∞ 2 −

(7.93)

(7.94)

θk y∞ 2 . S0 − S∞

If we assume that a set of possible values of parameters θi , θμ and θs can be obtained, (7.93)-(7.94) can be solved for a given set of initial conditions and external signal a(t). The corresponding trajectories are estimates of the asymptotic trajectories of the system subject to the extremum-seeking control. For each specific trajectory, we can calculate the corresponding value of the matrix: 

t+T0

Ψ∞ (τ )dτ. t

(7.95)

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

197

1 where Ψ∞ (t) = Φa (S∞ , y∞ , u∞ , θ)Φa (S∞ , y∞ , u∞ , θ)T with u∞ = (S0 −S (θk y∞ ∞) + a(t) − kd d(t)): ⎡ ⎤ −2γu S∞ 2 − γu S∞ 3 θs − S∞ 4 y∞ θμ   ⎢ ⎥ Φa S∞ , y∞ , θ, u∞ = ⎣ −γu S∞ + γu S∞ 3 θi − θμ S∞ 3 y∞ ⎦ (7.96)

S∞ 2 y ∞ + θ s S∞ 3 y ∞ + θ i S∞ 4 y ∞ and γu = (a(t) − kd d(t))y∞ . The strategy for testing any specific dither signal consists of evaluating the minimum eigenvalue of matrix (7.95) over a specific time horizon. If the minimum eigenvalue is positive for all t > 0, the dither signal is deemed sufficiently rich and its use can be justified on the closed-loop extremum-seeking control system. It is important to note that the corresponding conclusion depends entirely on the specific choice of parameter values that are used in the calculations. More work is required to investigate how this assessment can be conducted in a manner that is invariant of the choice of parameter values. The use of this technique will be considered in the simulation study presented in the next section. 7.3.3. Simulation results The aim of this section is to illustrate the performance of an adaptive extremumseeking controller in a number of simulations, performed using a realistic example of a fed-batch process. The kinetic model parameters, yield coefficients and initial states used during numerical simulations are: μ0 = 0.53 h−1 , KS = 1.2 g/l, KI = 0.22 g/l, k1 = 0.4, k2 = 1 X(0) = 7.2 g/l, S(0) = 2 g/l, S0 = 20 g/l

(7.97)

For the Haldane model, from Figure 7.3, the maximum on the growth specific rate occurs at S ∗ = √1θ = 0.52 g/l. The control objective is to design a controller i for the dilution rate, u, to regulate substrate S at S ∗ . The controller requires online measurements of variables S and y, as well as the knowledge of the kinetic parameters, determining the S ∗ . These values are obtained using the estimation algorithm previously presented, using the measurements of y. For the simulation study, we consider the following initial estimates of the kinetic parameters:   ˆ S = 10, K ˆ I = 0.03 ˆ0 = 10, K (7.98) θˆμ = 1, θˆS = 0.1, θˆI = 3 μ

198

Bioprocess Control

The design parameters for the extremum-seeking controller are set to: γμ = 10, γS = 200, γi = 200 ky,0 = 20, kz = 0.5, kd,0 = 1,  = 0.2

Dither signal a(t) is chosen as follows: a(t) =

5

  A1i sin 0.001 + (5 − 0.001)i/4 t

i=1

+

5

(7.99)   A2i cos 0.01 + (5 − 0.01)i/4 t

i=1

where A1i and A2i are normally distributed random numbers in the interval [−0.1, 0.1]. To test the richness of the dither signal, we calculate the smallest eigenvalue of matrix √ (7.95) over the interval [0, 100] using the initial conditions, x(0) = 7.2, s(0) = 1/ 3 and parameter estimates given by (7.98). The simulation shows that the signal is sufficiently exciting in a region of these initial conditions and parameter values. The resulting dither signal was used in the subsequent simulation of the extremum-seeking control scheme. It should be noted that it is relatively easy in practice to provide a dither signal such that matrix (7.95) is positive definite. However, the convergence of the parameter estimates can also depend on the conditioning of this matrix. The calculations demonstrate that the condition number of the matrix remains around 103 , a value that is relatively high. This indicates that parameter convergence may remain quite slow. In fact, a closer look at the spectral decomposition of this matrix indicates that the poor conditioning is associated with the adaptation of θs . The convergence properties of the extremum-seeking control scheme are shown in Figure 7.4. We consider the initial conditions, x(0) = 7.2 and s(0) = 2.0. It is shown from Figure 7.4 that the extremum-seeking scheme converges to the intended growth rate value. The substrate concentration converges to the unknown optimum as well. The dilution rate manipulation resulting from the extremum-seeking control is also shown. The dilution rate is seen to reach its lower bound as a result of the dither signal, a(t). Convergence of the kinetic parameters to their true values is achieved. It is important to note that the control performance is strongly dependent on the convergence of parameter θI , determining the set point for the control law, as illustrated by the substrate evolution, from Figure 7.4. Overall, extremum-seeking is shown to perform satisfactorily for this case.

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses 0.1

0.12 0.1

Dilution Rate hr−1

μ(S)gl−1 hr−1

0.09 0.08 0.07 0.06 0.05 0.04 0.03 0

199

20

30

40

50

60

70

80

90

0.06

0.04 0.02

μ(S) μ(S)∗ 10

0.08

0

100

0

10

20

30

Time (hr)

40

50

60

70

80

90 100

Time (hr)

3

6

S S∗

Estimated θ True θ

5

2.5

4

θs , θ i , θ μ

Sgl−1

2

1.5 1

3 2 1 0

0.5 0 0

−1 10

20

30

40

50

60

70

80

90

100

−2

0

10

20

Time (hr)

30

40

50

60

70

80

90 100

Time (hr)

Figure 7.4. Illustration of the convergence properties

In Krstic et al., an extremum-seeking scheme was proposed to optimize the production rate for a class of bioreactors governed by Monod kinetics and Haldane kinetics [WAN 99]. An extremum-seeking control was developed following the design procedure proposed in [WAN 99]. The main difference is that we use the procedure to optimize the growth rate μ(S). In this case, the extremum-seeking scheme yields the closed-loop system: x˙ = (μ − D)x S˙ = −k1 μx + (S0 − S)D ˆ˙ = k(μ − η) + a sin(ωt) D

(7.100)

η˙ = ωh (μ − η) ˆ + a sin(ωt) D=D where μ is given by (7.56). Note that the structure assumes that the actual value of μ is measured in this case. This is extremely unlikely since this would require detailed structural information about the parameter values of μmax , Ks and Ki entering the expression for μ.

Bioprocess Control 0.1

0.1

0.09

0.09

0.08

μ(S)gl−1 hr−1

μ(S)gl−1 hr−1

200

0.07 0.06 0.05

0.07 0.06 0.05

0.04 0.03 0.02 0

0.08

0.04

μ(S) μ(S)∗ 20

40

60

80

100 120 140 160 180 200

0.03 0

μ(S) μ(S)∗ 20

40

60

Time (hr)

80

100 120 140 160 180 200

Time (hr)

0.06

0.2 0.18 0.16

Dilution Rate hr−1

Dilution Rate hr−1

0.05 0.04

0.03 0.02

0.14 0.12 0.10 0.08 0.06 0.04

0.01

0.02 0 0

20

40

60

80

100 120 140 160 180 200

0

0

20

Time (hr)

40

60

80

100 120 140 160 180 200

Time (hr)

Figure 7.5. Performance of the proposed scheme (right) and the controller proposed in [WAN 99] for the case x(0) = 7.2, s(0) = 2.0

In order to facilitate the comparison, the tuning parameters of the extremumseeking scheme are set to the following values: a = 0.01,

ω = 0.75,

k = 5,

ωh = 0.1

The tuning parameters were assigned following the guidelines outlined in [KRS 00a]. Two simulations were performed. The first simulation was initiated at X(0) = 7.2, S(0) = 2.0, as above. Figure 7.5 shows the comparison between the performance of the two extremum-seeking schemes. On the left hand side, we show the dilution rate and specific growth rate μ for the proposed extremum-seeking controller. On the right side, we show the dilution rate and specific growth rate resulting from the application of the extremum-seeking scheme proposed in [WAN 99]. The results demonstrate that the two control schemes provide comparatively equivalent convergence properties. The scheme of [WAN 99] provides a slower convergence which could be improved by further adjustments of the design parameters. The values employed were the values that were found to provide the best performance for this system.

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses 0.095

0.09

0.09

0.08

0.085

μ(S)gl−1 hr−1

μ(S)gl−1 hr−1

0.1

0.07 0.06 0.05 0.04

0.08 0.075 0.07 0.065

0.03

0.06

0.02

0.055 μ(S) μ(S)∗

0.01 0 0

50

μ(S)

0.05

0.045 100 150 200 250 300 350 400 450 500 0

μ(S)∗ 50

0.05

0.1

0.04

0.08

0.03 0.02

100 150 200 250 300 350 400 450 500

Time (hr) 0.12

μ(S)gl−1 hr−1

Dilution Rate hr−1

Time (hr) 0.06

0.06 0.04 0.02

0.01 0 0

201

50

100 150 200 250 300 350 400 450 500

Time (hr)

0

0

50

100 150 200 250 300 350 400 450 500

Time (hr)

Figure 7.6. Performance of the proposed scheme (right) and the controller proposed in [WAN 99] for the case x(0) = 1.2, s(0) = 2.0

In the second simulation, the initial concentration of biomass was set to X(0) = 1.2. The simulation results are shown in Figure 7.6. As above, we show the dilution rate and specific growth rate for the proposed scheme on the right hand side and the results for the controller proposed by [WAN 99] on the left hand side. The scheme proposed in [WAN 99] provides very poor convergence properties compared to the scheme proposed here. Overall, the results indicate that the proposed scheme provides very consistent performance for this system. In the next set of simulations, we wish to illustrate the controller performance for regulation (as illustrated in Figure 7.7) by introducing a disturbance in the S0 concentration between 60 and 70 hours. As expected, the controller quickly rejects the disturbance while attenuating its effect on S which is accurately regulated at the set point during the whole operation. The last simulation illustrates the performance of the extremum-seeking controller in the presence of noisy measurements. As in the previous case, we consider the

202

Bioprocess Control 3

0.1 0.09

2

0.07

Sgl−1

μ(S)gl−1 hr−1

0.08

0.06

1.5 1

0.05 0.04 0.03 0

S S∗

2.5

0.5

μ(S) μ(S)∗ 10

20

30

40

50

60

70

80

90

0

100

0

10

20

30

Time (hr)

40

50

60

70

80

90 100

Time (hr)

0.2 0.18

Dilution Rate hr−1

0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0 0

10

20

30

40

50

60

70

80

90 100

Time (hr)

Figure 7.7. Illustration of the robustness properties. S0 is changed from 20 g/l to 40 g/l from t = 60 hr to t = 70 hr

operation of the bioreactor in the presence of a step in inlet substrate concentration, S0 from 20 g/l to 40 g/l from t = 60 hr to t = 70 hr. In addition, it is assumed that the substrate measurement, S, and the production rate measurements, y, are corrupted by additive measurement noise. The measured substrate concentration is given by Sm = S + 0.1nS where nS is a unit variance normally distributed additive noise term. Similarly, the measured production rate is given by, ym = y + 0.005ny , where ny is a unit variance normally distributed additive noise term. The results of the simulation are given in Figure 7.8. The results demonstrate that the extremum-seeking controller performs adequately in the presence of measurement noise. 7.4. Appendix: analysis of the parameter convergence ˜ converges to the By LaSalle’s invariance principle, the error vector (zs , ey , θ) largest invariant set M of dynamic system (7.66) and (7.85)–(7.87) contained in the

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

203

3

0.1

μ(S)(gl−1 hr−1 )

0.08 0.07 0.06 0.05 μ(S)

0.04 0.03 0

μ(S)∗ 10

20

30

40

50

60

70

80

90

Substrate Concentration (gl−1 )

S 0.09

S∗

2.5 2 1.5 1

0.5 0

100

0

10

20

30

Time (hr)

40

50

60

70

80

90 100

Time (hr)

0.18

Dilution Rate (hr−1 )

0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0 0

10

20

30

40

50

60

70

80

90 100

Time (hr)

Figure 7.8. Illustration of the robustness properties. The measurements of S and y are corrupted by additive noise. S0 is changed from 20 g/l to 40 g/l from t = 60 hr to t = 70 hr

˜ ∈ R5 | zs = ey = 0}. The purpose of the following is to study set E = {(zs , ey , θ) the invariant set M to obtain the condition under which parameter convergence (∞ can be achieved. Since ey converges to zero, we know that 0 e˙ y dt = ey (∞) − ey (0) = −ey (0). This implies that e˙ y is integrable. It follows from error equation ˆ d and its time derivatives. Since θ, ˆ ey ∈ L∞ , (7.66) that e¨y is a function of y, S, yˆ, θ, and the excitation signal d and d˙ are bounded, we know that e¨y is bounded. This implies the uniform continuity of e˙ y . Using Barbalat’s Lemma [IOA 96], we conclude that e˙ y → 0 as t → ∞. On the invariant set M , we have ey ≡ 0 and e˙ y ≡ 0. By setting ey = e˙ s =e˙ y = 0, equation (7.66) leads to θ˜k y = 0 and:   ˆ D) θ˜i , θ˜s , θ˜μ Φa (S, y, θ,    = 0, 1 + θs S + θi S 2 1 + θˆs S + θi S 2



 zs , ey , θ˜ ∈ M

(7.101)

204

Bioprocess Control

where: ⎡ ⎤ −2γu S 2 − γu S 3 θˆs − S 4 y θˆμ ⎢ ⎥ ˆ D) = ⎢ −γu S + γu S 3 θˆi − θˆμ S 3 y ⎥ Φa (S, y, θ, ⎣ ⎦ S 2 y + θˆs S 3 y + θˆi S 4 y

(7.102)

and γu = (D(S0 − S) − θk y)y. Since S > 0 and θˆ are bounded, we know that: ˆ = 0, θ˜T Φa (S, y, θ) where: θ˜ = [θ˜i M=



 zs , ey , θ˜ ∈ M

θ˜s

θ˜μ ]T . Therefore, the largest invariant set M in E is:

/

0  ˆ =0 zs , ey , θ˜ ∈ R6 | zs = ey = 0, θ˜T Φa (S, y, u, θ)

(7.103)

˜ ∈ M: It follows from (7.103) that ∀(zs , ey , θ) ˆ T (s, y, u, θ) ˆ θ˜ = 0 θ˜T Ψ(t)θ˜ = θ˜T Φa (S, y, u, θ)Φ a

(7.104)

ˆ T (S, y, u, θ) ˆ is positive definite, then we may conclude that If Φa (S, y, u, θ)Φ a ˜ θ = 0. However, it is impossible to satisfy this condition because the matrix ˆ T (S, y, u, θ) ˆ is singular at any given time. We consider the condition: Φa (S, y, u, θ)Φ a 1 lim t→∞ T0



t+T0

 T  θ˜ Ψ(τ )θ˜ dτ = 0

(7.105)

t

with positive constant T0 . It can be shown from (7.85)-(7.87) and limt→∞ ey = 0 that ˙ limt→∞ θˆ = 0. However, this does not necessarily imply that θ˜ converges to a constant as t → ∞. However, it is possible to show that this is the case for the extremumseeking controller proposed. From (7.91)-(7.92) we see that the closed-loop signals zs and ey belong to L2 , the set of square integrable functions. Let us first prove the convergence of θˆi . The adaptation rate for this estimate is such that: ˙    θˆi  ≤ γi Γ3 D + Γ4  (7.106) Substituting for Γ3 , Γ4 and the control input D, we obtain:       3ˆ ˆi S 2 y  ˙   e S 1− θ e S y θ y   y μ θˆi  ≤ γi  − ey Sy− −kz zs +a−kd d −   1+ θˆs S + θˆi S 2 1+ θˆs S + θˆi S 2 

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

205

or        3ˆ ˆi S 2 y  ˙   S 1 − θ y θ S    μ θˆi  ≤ γi  Sy + a − kd d +  ey 2 2 ˆ ˆ ˆ ˆ  1 + θs S + θi S 1 + θs S + θi S       S 1 − θˆi S 2 y     + γi  Sy +  ey zs  1 + θˆs S + θˆi S 2 

(7.107)

Since all signals of the closed-loop process are bounded, the quantities: .    1 3ˆ ˆi S 2 y     S 1 − θ y θ S μ  (7.108) L3 = sup γi  Sy + a − kd d + t≥0 1 + θˆs S + θˆi S 2 1 + θˆs S + θˆi S 2  and 1 .    ˆi S 2 y   S 1 − θ  L4 = sup γi  Sy + t≥0 1 + θˆs S + θˆi S 2 

(7.109)

exist and are finite. As a result, we write inequality (7.107) as follows: ˙       θˆi  ≤ L3 ey  + L4 ey zs 

(7.110)

Since zs ∈ L2 and ey ∈ L2 it follows that the right hand side of inequality (7.110) ˙ ˙ belongs to L1 . The analysis shows that θˆi ∈ L1 . The integration of θˆi gives:  θˆi (t) = θˆi (0) +

t

˙ θˆi (σ)dσ

(7.111)

0

˙ Since θˆi ∈ L1 , the integral on the right hand side of (7.111) exists and is finite. This, in turn, demonstrates that θˆi converges to a constant. Consequently, this also guarantees that θ˜i converges to a constant. If we consider θˆs , a similar argument leads to the following inequality:       ˙    1 − θˆi S 2 y θˆμ S 2 y   θˆs  ≤ γs  a(t) − kd d(t) +  ey 2 2  1 + θˆs S + θˆi S 1 + θˆs S + θˆi S  (7.112)      1 − θˆi S 2 y     + γs  e z  1 + θˆs S + θˆi S 2  y s

206

Bioprocess Control

As above, we conclude that the constants: .     1 ˆi S 2 y  ˆμ S 2 y    1 − θ θ  a(t) − kd d(t) + L5 = sup γs  t≥0 1 + θˆs S + θˆi S 2 1 + θˆs S + θˆi S 2  (7.113) and

.    1  1 − θˆi S 2 y   L6 = sup γs  t≥0 1 + θˆs S + θˆi S 2 

(7.114)

exist and are finite. As a result, we obtain the inequality: ˙       θˆs  ≤ L5 ey  + L6 ey zs  ˙ confirming that θˆs ∈ L1 . Therefore, θˆs , and, as a result, θ˜s , are guaranteed to converge to constant values as t → ∞. The convergence of θˆμ can be established in a similar manner. From (7.87), we obtain:     ˙  θˆμ  ≤ |Sy|ey  ≤ L7 ey  where L7 = supt≥0 {|Sy|}. By the boundedness of the closed-loop signals, L7 is ˙ guaranteed to exist and to be finite. Therefore, θˆμ ∈ L1 and the convergence of θˆμ and ˜ θμ to constant values is guaranteed. ˜ as t → ∞. From (7.105), we Let θ˜∞ be the set of constant values reached by θ(t) ˜ can write, ∀(zs , ey , θ) ∈ M : 3 2  t+T0 1 T ˜ Ψ(τ )dτ θ˜∞ = 0 (7.115) θ∞ lim t→∞ T0 t As a result, we show that if the dither signal d(t) is designed such that the following condition holds:  t+T0 1 Ψ(τ )dτ ≥ c0 I (7.116) lim t→∞ T0 t for some c0 > 0 and: S ∈ Ωs =

2 3 1 S | S = √ + d(t), θ¯i ≥ i θ¯i

then, parameter error θ˜ converges to zero asymptotically.

(7.117)

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

207

7.5. Bibliography [ALV 88] J.G. A LVAREZ and J.G. A LVAREZ, “Analysis and control of fermentation processes by optimal and geometric methods”, Proc. ACC, vol. 2, pp. 1112–1117, 1988 [AST 95] K.J. A STROM and B. W ITTENMARK, Adaptive Control, 2nd Edition, AddisonWesley, Reading, MA, 1995. [BAS 90] G. BASTIN and D. D OCHAIN, On-line Estimation and Adaptive Control of Bioreactors, Elsevier, Amsterdam, 1990. [BIN 83] R. B INOT, T. B OL, H. NAVEAU and E.J. N YNS, “Biomethanation by immobilized fluidised cells”, Wat. Sci. Tech., vol. 15, pp. 103–115, 1983. [CHE 91] L. C HEN, G. BASTIN and V. VAN B REUSEGEM, “Adaptive nonlinear regulation of fed-batch biological reactors: an industrial application”, in Proc. 30th IEEE CDC, pp. 2130– 2135, 1991. [CHE 79] A. C HERUY and A. D URAND, “Optimization of Erytromycin biosynthesis by controlling pH and temperature: theoretical aspects and practical applications”, Biotechnol. Bioeng., vol. 9, pp. 303–320, 1979. [CHI 91] M. C HIDAMBARAM and Y.S.N. M ALLESWARARAO, “Nonlinear control of a mixed-culture fed-batch bioreactor”, Ind. Chem. Eng., vol. 33, no. 2, pp. 53–57, 1991. [DAH 91] B. DAHHOU, J. B ORDENEUVE and J.P. BABARY, “Multivariable long-range predictive control algorithm applied to a continuous flow fermentation process”, in Proc. World IFAC Congress, pp. 393–397, 1991. [DOC 84] D. D OCHAIN and G. BASTIN, “Adaptive identification and control algorithms for nonlinear bacterial growth systems”, Automatica, vol. 20, pp. 621–634, 1984. [DOC 86] D. D OCHAIN, On-line parameter estimation, adaptive state estimation and adaptive control of fermentation processes, PhD thesis, Université Catholique de Louvain, Belgium, 1986. [DOC 89] D. D OCHAIN and G. BASTIN, “Adaptive control of fedbatch bioreactors”, Chemical Engineering Communications, vol. 87, pp. 67–85, 1989. [DOC 91] D. D OCHAIN, M. P ERRIER and A. PAUSS, “Adaptive control of the hydrogen concentration in anaerobic digestion”, Ind. Eng. Chem. Res., vol. 30, pp. 129–136, 1991. [DOC 92] D. D OCHAIN and M. P ERRIER, “Adaptive linearizing control of activated sludge processes”, in Proc. Control Systems’92, pp. 211–215, 1992. [DRK 95] S. D RKUNOV, U. O ZGUNER, P. D IX and B. A SHRAFI, “ABS control using optimum search via sliding modes”, IEEE Trans. Contr. Syst. Tech., vol. 3, pp. 79–85, 1995. [FLA 91] J.M. F LAUS, A. C HERUY and J.M. E NGASSER, “An adaptive controller for batch feed bioprocess. Applicaton to lysine production”, J. Process Control, vol. 1, pp. 271–281, 1991.

208

Bioprocess Control

[FRI 84] J.L. F RIPIAT, T. B OL, R. B INOT, H. NAVEAU and E.J. N YNS, “A strategy for the evaluation of methane production from different types of substrate biomass”, in Biomethane, Production and Uses, R. B UVET, M.F. F OX and D.J. P ICKER (eds.), pp. 95– 105, Roger Bowskill ltd, Exeter, UK, 1984. [GOL 86] M.P. G OLDEN, B.J. PANGRIE and B.E. Y DSTIE, “Nonlinear adaptive optimization of a continuous bioreactor”, in Proc. AIChE 1986 National Meeting, Pap.125b, 1986. [GOO 84] G.C. G OODWIN and K.S. S IN, Adaptive Filtering Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ, 1984. [GUA 04] M. G UAY, D. D OCHAIN and M. P ERRIER, “Adaptive extremum seeking control of stirred tank bioreactors”, Automatica, vol. 40, no. 5, pp. 881–888, 2004. [HAM 75] R.P. H AMALAINEN, A. H ALME and A. G YLLENBERG, “A control model for activated sludge wastewater treatment process”, in Proc. 6th IFAC World Congress, Boston, Paper 61:6, 1975. [HAR 06] J. H ARMAND, D. D OCHAIN and M. G UAY, “Dynamical optimization of a configuration of multi-fed interconnected bioreactors by optimum seeking”, in Proc. CDC’06, pp. 2122–2127, 2006. [HEN 92] M.A. H ENSON and D. S EBORG, “Nonlinear control strategies for continuous fermenters”, Chem. Eng. Sci., vol. 47, no. 4, pp. 821-835, 1992. [HOL 82] A. H OLMBERG and J. R ANTA, “Procedures for parameter and state estimation of microbial growth process models”, Automatica, vol. 18, pp. 181–193, 1982. [HOL 83] A. H OLMBERG, “A microprocessor-based estimation and control system for the activated sludge process”, in Modelling and Control of Biotechnical Processes, A. H ALME (ed.), pp. 111–120, Pergamon Press, Oxford, UK, 1983. [HOO 86] K.A. H OO and J.C. K ANTOR, “Linear feedback equivalence and control of an unstable biological reactor”, Chem. Eng. Comm., vol. 46, pp. 385–399, 1986. [IOA 96] P.A. I OANNOU and J. S UN, Robust Adaptive Control, Prentice-Hall, Englewood Cliffs, NJ, 1996. [KHO 85] M. K HONE, “Practical experiences with a new online BOD measuring device”, Env. Technol. Letters, vol. 6, pp. 546–555, 1985. [KRS 95] M. K RSTIC, I. K ANELLAKOPOULOS and P. KOKOTOVIC, Nonlinear and Adaptive Control Design, John Wiley, New York, 1995. [KRS 98] M. K RSTIC and H. D ENG, Stabilization of Nonlinear Uncertain Systems, SpringerVerlag, New York, 1998. [KRS 00a] M. K RSTIC, “Performance improvement and limitations in extremum seeking control”, Systems & Control Letters, vol. 5, pp. 313–326, 2000. [KRS 00b] M. K RSTIC and H.H. WANG, “Stability of extremum seeking feedback for general dynamic systems”, Automatica, vol. 4, pp. 595–601, 2000.

Adaptive Linearizing Control and Extremum-Seeking Control of Bioprocesses

209

[LEB 22] M. L EBLANC, “Sur l’électrification des chemins de fer au moyen de courants alternatifs de fréquence élevée”, Revue Générale de l’Electricité, vol. 12, no. 8, pp. 275–277, 1922. [LI 95] P.Y. L I, Self optimizing control and passive velocity filed control of intelligent machines, PhD thesis, UC Berkeley, 1995. [MAR 03] N. M ARCOS, M. G UAY, D. D OCHAIN and T. Z HANG, “Adaptive extremum seeking control of a continuous bioreactor”, J. Process Control, vol. 14, no. 3, pp. 317–328, 2003. [MAR 04] N. M ARCOS, M. G UAY and D. D OCHAIN, “Output feedback adaptive extremum seeking control of a continuous stirred tank bioreactor with Monod’s kinetics”, J. Process Control, Special issue on Dynamics, Monitoring, Control and Optimization of Biological Systems, vol. 14, no. 7, pp. 807–818, 2004. [MAR 84] S. M ARSILI -L IBELLI, “Optimal control of the activated sludge process”, Trans. Inst. Meas. Control, vol. 6, pp. 146–152, 1984. [MAR 89] S. M ARSILI -L IBELLI, “Modelling, identification and control of the activated sludge process”, Advances in Biochemical Enginering/Biotechnology, vol. 38, pp. 90–148, 1989 [MOS 83] F.E. M OSEY, “Mathematical modelling of the anaerobic digestion process: regulatory mechanisms for the formation of short-chain volatile acids from glucose”, Water Sci. Technol., vol. 15, pp. 209–232, 1983. [NAR 89] K.S. NARENDRA and A.M. A NNASWAMY, Stable Adaptive Systems, PrenticeHall, Englewood Cliffs, NJ, 1989. [OHN 76] H. O HNO, E. NAKANISHI and T. TAKAMATSU, “Optimal control of a semibatch fermentation”, Biotechnol. Bioeng., vol. 28, pp. 847–864, 1976. [PAR 85] S.J. PARULEKAR, J.M. M ODAK and H.C. L IM, “Optimal control of fed-batch bioreactors”, Proc. ACC, Boston, vol. 2, pp. 849–854, 1985. [PAU 90] A. PAUSS, C. B EAUCHEMIN, R. S AMSON and S. G UIOT, “Continuous measurement of dissolved H2 in an anaerobic reactor using a commercial probe hydrogen/air fuel cell-based”, Biotechnol. Bioeng., vol. 35, pp. 492–501, 1990. [PER 79] P. P ERINGER and H.T. B LACHERE, “Modelling and optimal control of baker’s yeast production in repetead fed-batch culture”, Biotechnol. Bioeng., vol. 9, pp. 205–213, 1979. [PER 93] M. P ERRIER and D. D OCHAIN, “Evaluation of control strategies for anaerobic digestion processes”, Int. J. Adaptive Cont. Signal Proc., vol. 7, no. 4, pp. 309–321, 1993. [REN 88] P. R ENARD, D. D OCHAIN, G. BASTIN, H. NAVEAU and E.J. N YNS, “Adaptive control of anaerobic digestion processes. A pilot-scale application”, Biotechnol. Bioeng., vol. 31, pp. 287–294, 1988. [REN 91] P. R ENARD, V. VAN B REUSEGEM, N. N GUYEN, H. NAVEAU and E.J. N YNS, “Implementation of an adaptive controller for the start-up and steady-state running of a biomethanation process operated in the CSTR mode”, Biotechnol. Bioeng., vol. 8, pp. 805– 812, 1991.

210

Bioprocess Control

[STE 80] J. S TERNBY, “Extremum control systems: An area for adaptive control?”, Preprints of the Joint American Control Conference, San Francisco, CA, 1980. [TIT 03a] M. T ITICA, D. D OCHAIN and M. G UAY, “Real-time optimization of fed-batch bioreactors via adaptive extremum-seeking control”, Chemical Engineering Research and Design, vol. 81, no. A9, pp. 1289–1295, 2003. [TIT 03b] M. T ITICA, D. D OCHAIN and M. G UAY, “Adaptive extremum seeking control of fedbatch bioreactors”, European Journal of Control, vol. 9, no. 6, pp. 618–631, 2003. [VAN 91] V. VAN B REUSEGEM and G. BASTIN, “Reduced order dynamical modelling of reaction systems: a singular perturbation approach”, in Proc. 30th IEEE Conf. Dec. Cont., vol. 2, pp. 1049–1054, 1991. [VAN 93] J. VAN I MPE, Modelling and optimal adaptive control of biotechnological processes, PhD thesis, Katholieke Universiteit Leuven, Belgium, 1993. [VAS 57] G. VASU, “Experiments with optimizing controls applied to rapid control of engine presses with high amplitude noise signals”, Transactions of the ASME, pp. 481–488, 1957. [WAN 99] H. WANG, M. K RSTIC and G. BASTIN, “Optimizing bioreactors by extremum seeking”, Int. Journal Adaptive Control and Signal Processing, vol. 13, pp. 651–669, 1999. [ZHA 02] T. Z HANG, M. G UAY and D. D OCHAIN, “Adaptive extremum seeking control of continuous stirred tank bioreactors”, AIChE J., vol. 49, no. 1, pp. 113–123, 2002.

Chapter 8

Tools for Fault Detection and Diagnosis

8.1. Introduction Since the beginning of the industrial era, man has been confronting the problems posed by machines created by himself. He has even fallen victim to problems cropping up on account of design errors, diverse external disturbances or even a lack of maintenance of mechanisms. This rapidly led to concerns about the condition of machines and therefore man tried to minimize the consequences of the technical problems coming up in this connection. For a long time, direct human observation – i.e. without employing any dedicated instruments or specific sensors – was the sole means for detecting deviations from the normal operating conditions. Sensations, perceived directly by the human operator (noise, temperature, odors, vibrations, etc.) were – and in certain cases still are – the basic elements in the step for detecting malfunctions and for monitoring installations. Then, sensors and recording devices came into existence. They give direct information about important process variables, but the reliability of the information given out by them should be verified before being employed in a procedure for taking decisions and/or for choosing actions. Furthermore, these recordings on paper or on magnetic media have provided a long-term memory for the human operator and this has enabled him to understand the dynamic dimension of the phenomena observed by him. Industrial systems have also become more and more complex with the automatic control of control loops, introduction of multilevel microprocessors and, more

Chapter written by Jean-Philippe S TEYER, Antoine G ÉNOVÉSI and Jérôme H ARMAND.

211

212

Bioprocess Control

recently, prioritized and distributed information. In fact, this development has complicated the task of diagnosing industrial installations. Since the 1960s, considerable interest has been expressed on operational safety. This concern was initially focused on large systems such as those in the fields of nuclear energy, petrochemicals and aerospace engineering. This has resulted in the emergence of methods such as failure modes, effects and criticality analysis (FMECA) and also the appearance of an entire series of papers, whose main areas are described later in this chapter. The literature concerned with fault detection and isolation (FDI) and with their diagnosis thus shows a large diversity with respect to both the viewpoints and the methods. This is certainly due to many factors, namely: – scientists of diverse schools of thought, who approached this vast problem; – diversity of systems subjected to diagnosis; – wealth and heterogenity of the available information; – multiplicity of problems that can be detected and diagnosed Later, we will introduce some general terms and concepts employed in the field of FDI and diagnosis. We will then present the leading methods developed on the basis of a separation into three broad fields: – methods based on signals (connected with signal processing); – methods based on models (connected with automatic control); – methods based on expert knowledge (connected with Artificial Intelligence). Although this separation, rests on the historical evolution of the domain, it is relatively arbitrary. In practice, we actually come across an association of methods, the application of methods belonging to one branch to another or even, their use in series, i.e., in sequence form for detection and then diagnosis. 8.2. General definitions 8.2.1. Terminology In order to fix a common basis for the vocabulary used, first it will be necessary to make more explicit certain definitions. The majority of them have been discussed within the SAFEPROCESS Technical Committee of the International Federation of Automatic Control. A fault is defined as an unpermitted deviation of at least one characteristic property or parameter of the system from normal operating conditions. A breakdown or failure is a permanent interruption of a system’s ability to perform a required function under specified operating conditions. A residual is a fault indicator, based on deviation between measurements and model-equation-based calculations.

Tools for Fault Detection and Diagnosis

213

A symptom is a change of an observable quantity from normal behavior. Fault detection consists of the determination of faults present in a system and time of detection. Fault isolation is the determination of the type, location, and date of detection of a fault. Fault identification is the determination of the size and time-variant behavior of a fault. It follows fault isolation and is of interest only in the event of a possible reconfiguration of the control system. Diagnosis is defined as the determination of the type, size, location and time of a fault. It follows fault detection and includes fault isolation and identification. Monitoring is a continuous real-time task of determining the conditions of a system, by recording information, recognizing and indicating anomalies of behavior. Supervision represents the monitoring of a physical system and taking appropriate actions to maintain the operation in the case of faults. Reconfiguration of a system consists of the synthesis of new control rules in the presence of faulty elements in order to ensure the installation’s availability. In addition, a FDI procedure coupled with an online diagnostic procedure (i.e. FDD1), is called online, when it evaluates the data obtained from the process in real time. In this case, only the information from the past and predictions about the future are available. This imposes constraints on the availability of information and on the response time. When all the information concerning the history of an event are employed a posteriori, the system is said to be “offline”. 8.2.2. Fault types When applied to industrial systems, one of FDI’s difficulties lies in the large variety of possible faults from the viewpoint of the input channel as well as from their amplitude and frequency. Figure 8.1 represents a system with different fault input channels. From the viewpoint of the input channel, we can consider the defects as additive if they affect the process behavior in a manner independent of the known inputs. If they do so in a way that depends on the known inputs, we consider them as being multiplicative. Furthermore, a classification of faults based on their time-variant developments defines them as being: – abrupt: this type of fault is mainly characterized by the discontinuity in the variable’s time-variant development. The development, if it does not correspond to the 1. Fault Detection and Diagnosis.

214

Bioprocess Control

Faults Set point − +

Checking

Faults

Unknown inputs Output

Process

Measurement Faults Figure 8.1. Representation of a system and different fault input channels

normally expected dynamic developments for the variable, is characteristic of a blunt breakdown of the element in question: full or partial stop, disconnection, etc.; – intermittent: we deal here with a type of fault that is characteristic of bad contacts. It is a particular case of an abrupt fault with the property of a signal which returns randomly to normal values; – gradual: this type of fault is essentially characteristic of a clogging or wear of the part. These faults are very difficult to detect as their time-variant evolutions are the same as a slow parametric modification representing a non-stationarity of the process. Figure 8.2 recapitulates this fault classification based on their time-variant evolutions v

v

Abrupt fault

t

v

Intermittent fault t

Gradual fault

t

Figure 8.2. Time-variant evolutions of different fault types over a variable

Furthermore, the presence of multiple faults is a rather frequent situation in practice. This situation occurs when one or more variables (or parameters) exhibit abnormal developments because of a simultaneous malfunction of elements of an installation. In fact, the propagation of a breakdown in a process generally produces cascading events that can have these types of consequences. 8.3. Fault detection and diagnosis With developments in automatic control, technological progress has enabled the development of more and more complex processes. In the case of a technical

Tools for Fault Detection and Diagnosis

215

installation, the role of man has thus evolved from direct physical manipulation to a task of monitoring and even supervision. In case of the failure of a technical installation, the process operator has to deal with a cascade of information, which is a tough test to his intellectual capabilities, reactivity and stress management. Under these conditions, diagnosis and decision making are turned into tasks, which, depending on the subjectivity and expertise of the individual, are more or less easy to manage, increasing at times the risk of human errors. Furthermore, these tasks are not easy to carry out, as fault detection has to be accomplished under real conditions – thus online – of the operation of installations: – the time-variant characteristics of faults are unknown; – the nominal model of the system is uncertain; – state and measurement noises are present; – the detection signal should be obtained within a finite time, subject to the operating constraints of the process. Early detection of faults, automated diagnosis and assistance to operators are thus fields in which there is very active scientific research to remedy the failure situations and find solutions suited to each process. Figure 8.3 shows the general schematic diagram of an FDD structure. The two main stages, i.e. fault detection and diagnosis, constitute a procedure that is globally accepted today, while the residual generation and decision making phases are separate. The generation of residuals – an essential stage of FDI – consists of calculating signals that reflect the presence or absence of faults. In the absence of faults, their average is zero, but becomes non-zero when one (or more) fault(s) arises. The FDI procedures should therefore enable us to separate (i.e., uncouple) the effect of unknown inputs, noises, modeling errors and faults themselves. These procedures are carried out on the basis of knowledge about the normal-condition-installation to be monitored, as well as the available signals. This knowledge can be put up in different forms: statistics, rule bases, mathematical models, etc. 8.3.1. Methods based directly on signals Direct observation of variables measured online is intuitively the first solution that we can employ while seeking to evaluate the condition of a process. Thus the beginnings for the use of signals for FDD go back to the early days of application of sensors on processes. 8.3.1.1. Hardware redundancy Hardware redundancy is a method that is generally adopted in critical installations (e.g. aerospace, nuclear) in which the availability of information and means of action

216

Bioprocess Control

Acquisition of knowledge Acquisition of analytical knowledge

Engineers

Physical laws

Analytical level

Process

Analytical knowledge

Analytical solution to problems

Mathematical models

Dynamic mathematical models

Normal process behavior

Generation of residuals

Heuristic level Acquisition of heuristic knowledge

Heuristic knowledge

System expert

Fault trees Process history

Detection of faults Diagnosis

Generation of assumptions Assumptions

Other processes Statistics about the faults

Taking decision

Fault

Figure 8.3. General schematic of an FDD structure

is indispensable for the good functioning and the safety of equipment. Use of more than one sensor to obtain the same information about a variable will in fact enable us to detect deviations with respect to a nominal condition. Furthermore, the number of sensors exceeding the number of measured variables makes possible the localization of a sensor fault. This method has the following disadvantages: increase in the installation cost and in the probability of sensor failures – leading to an increased maintenance requirement. 8.3.1.2. Specific sensors Specific sensors can also be used to directly generate detection signals or to discover the condition of a component. For example, there is widespread use of limit sensors, in order to indicate the operating conditions of an engine or exceeding threshold values in industrial installations. The advantages of these sensors lie in their simplicity and robustness. In addition, they are also very useful for supervision solutions based on discrete events such as Petri networks. However, their discrete nature itself constitutes their limitation.

Tools for Fault Detection and Diagnosis

217

8.3.1.3. Comparison of thresholds An extension of the preceding method using a mathematical treatment of the signal represents a method that is used most frequently in the industry. It consists of comparing the measurements of variables and their properties (e.g., averages, tendencies) with pre-established thresholds. Generally, two types of thresholds are installed: one for safety and another for alarm. The first enables us to detect small changes in the behavior of variables and to take corrective measures sufficiently in advance in order to avoid more serious consequences. The second triggers a contingency sequence in order to safeguard equipment and human life. This type of detection, though simple, has the disadvantage of not being sufficiently robust in the face of measurement noises and possible variations in the working modes. In fact, a noise-added signal with a low threshold trips numerous false alarms and this renders the detection system useless if not totally ineffective. Conversely, an increase in the threshold implies a lowering of sensitivity to faults, resulting in very delayed detection and even in non-detection. We can propose solutions for customizing – in a dynamic manner – the detection thresholds depending on the situations encountered and thus minimizing the ratios of false alarms and non-detection (see for example [PUI 99]). 8.3.1.4. Spectral analysis The spectral analysis of the signals coming from the sensors also enables us to determine very effectively the condition of the installation under monitoring. At first, the signals are analyzed under normal operating conditions: high frequencies are associated with noise level and low frequencies with developments belonging to the process condition. Furthermore, any deviation of frequency characteristics of a signal is associated with a breakdown situation. This approach is seen to be very useful for analyzing signals showing oscillations with periods that are more or less long (e.g., electrical current, flow rates, pressures). In certain cases, the use of low-pass filters also helps detect small deviations with respect to the normal condition. This approach using spectral analysis has the advantage of being relatively simple with respect to practical implementation, but then it is rather inconveniently sensitive to measurement noises when they are within the frequency zone of interest. Furthermore, it will be necessary to carry out frequent sampling in order to be in a position to reconstitute the start signal, while at the same time minimizing the loss of frequencies. In the literature, there are several articles that refer to auto-correlation functions, spectral density of signals, Fourier transformation or even to the use of wavelets. These methods are quite appropriate when we know the frequencies that are representative of faults. Conversely, it will be preferable to use parametric models of signals, which enable online estimation of frequencies and average values of parameters.

218

Bioprocess Control

8.3.1.5. Statistical approaches Numerous applications employ statistical methods for detecting faults. Among the most used, we can include the methods using control charts (e.g., Shewhart’s control charts, simple or weighted moving averages) as well as CuSum-type algorithms, those based on likelihood relationships or even Bayesian algorithms. These approaches are based on the assumption of rapid changes in the characteristics of signals or parameters of models with respect to the dynamics considered as being slow (i.e., quasistationary processes). It is to be noted that these approaches refer to the rapidity of change and not to its amplitude; this authorizes their use for detecting gradual changes with low detection thresholds. In addition, in the last few years, we have seen the emergence of a considerable interest for multidimensional analysis employing principal component analysis (PCA) and partial least squares or projected latent structures (PLS). In fact, the information given by the increasing number of sensors installed with respect to the processes makes it very difficult to analyze the results. The PLS approach helps reduce the number of variables to be treated by determination of the linear relationships among them (latent variables), thus facilitating the explanation of variance in the data series. 8.3.2. Model-based methods Even though any online detection and diagnosis method is fundamentally based on the information about the condition of the process to be monitored (i.e., measurements), there are methods employing additional information apart from those brought in only by the physical sensors. This information can in particular be derived from knowledge of the input/output behavior of a process or of internal processes governing the evolution of the same. This knowledge is generally expressed in the form of mathematical models. The use of model-based methods depends on the existence of analytical redundancies. The principle consists of finding mathematical formulations, which help reveal these analytical redundancies by determination of the causal factors, which are inherent to the system to be monitored. Research works in this line started almost 50 years ago with the simultaneous use of a variety of techniques. This diversity led many authors to recently make a big push to unify the viewpoints, language and to show the equivalences among the methods (see [GER 98, CHE 99]). Among the different methods for detection and diagnosis using mathematical models, we find mainly parity space, observers and parametric estimation.

Tools for Fault Detection and Diagnosis

Faults Input

219

Unknown inputs Measurements

Process

+ Model

−

Residual V

Figure 8.4. Parity space approach in an input-output format

8.3.2.1. Parity space The parity space method was one of the early methods employed for FDI purposes. It derives its name from the field of informatics in which a parity check is made on logic circuits. The principle of the method consists of verifying the consistency that exists between the inputs and outputs of the monitored system (see Figure 8.4). Two main branches contributed to its development: – reconciliation of data with the detection of rough errors; – use of static models and then dynamic models through a representation of systems in the form of transfer and state space function. The detection of rough errors began with the works of Kuehn and Davidson by the application of linear constraints (i.e., least squares) on the balances produced from noise-added measurements [KUE 61]. This method, also called direct redundancy, consists of using the analytical redundancies expressed in the form of algebraic equations among the diverse measurements of the system to be monitored. Furthermore, statistical methods coupled with material balance sheets were employed on the basis of the assumption of normal distribution errors. A study presented by Serth and Heenan compares the atypical measurement values with the residuals obtained from the least squares, by employing statistical tests and tests on partial balance sheets [SER 86]. More recently, Dovi and colleagues used the likelihood approach generalized on the basis of abnormal data distributions [DOV 97]. Finally, a very interesting application relates to the validation of data using advanced fuzzy form techniques for supervising the water quality in a river [BOU 98]. Another branch of the parity space method is based on the principle of temporal redundancies. The dynamic relationships between the inputs and outputs are taken into account for generating the residuals with characteristics, which respond to a desired

220

Bioprocess Control

Faults Input

Unknown inputs Measurements

Process ym(t) Model

+

Residual

−

H Figure 8.5. Use of a full-order observer for the generation of residuals (taken from [FRA 96])

transient behavior and to a noise filtration [CHO 84, PAT 91]. Recently, Staroswiecki and Comtet-Varga approached this method from the viewpoint dealing with the properties of detectability and isolability for nonlinear systems [STA 99]. 8.3.2.2. Observers Subsequently, we will refer to the strategy called by certain authors observer methods or even closed loop methods as opposed to the open loop methods corresponding to those of parity space methods described earlier. The diagram in Figure 8.5 represents the use of an observer for the generation of residuals. Error signal e(t) between output y(t) of the process and ym (t) of the model is the basis for the creation of signals supporting information on the faults. The synthesis of matrix H should thus enable the uncoupling of the outputs from the unknown inputs. This matrix should therefore have the particular characteristics of stability and insensitivity to the perturbations in the output. Beard and Jones were in fact the first to propose the replacement of equipment redundancy with detection algorithms based on observers [BEA 71, JON 73]. Their papers related to linear systems and the method was called the Beard-Jones failure detection filter. Other papers emerged following Beard and Jones’s report, which were then extended to nonlinear systems (see for example [FRA 96, ALCO 97] and their references). Different types of observers were also developed depending on the nature of problems to be solved: the observers based on the use of a single output [CLA 78], those using all the system outputs [WIL 76] and the output observers [SCH 96].

Tools for Fault Detection and Diagnosis

Faults Input

221

Unknown inputs Measurements

Process

Estimation of parameters Physical parameters Residual Decision Figure 8.6. Parametric estimation for fault detection and diagnosis

The problem of the robustness of detection signals faced with unknown inputs and uncertainties about the model has been tackled for the past 20 years. In addition, the use of observers with unknown inputs enables verification by perfect detection. An interesting solution is proposed by [FRA 93], with a frequency approach for minimizing the detectability thresholds using the factorization of residual generator. While perfect detection cannot be produced, good results can be obtained by employing the H∞ method [CHE 99]. For obtaining good results, the current studies on detection based on analytical redundancy aim at increasing the robustness in the face of unknown inputs and improving the characteristics governing the isolability of the residuals using the so-called structural and directional improvements [GER 98]. 8.3.2.3. Parametric estimation The parametric estimation approach considers that the influence of faults is reflected on the parameters and not just, as was thought previously, on the variables of the physical system. The principle of this method consists of continuously estimating the process parameters by using the inlet/outlet measurements and evaluating the gap, which separates them from the reference values under the normal condition of the process (see Figure 8.6). Parametric estimation has the advantage of furnishing information about the size of deviations. In addition, online identification helps avoid the problem of slow nonstationarities. However, one of the major disadvantages of the method lies in the necessity of having a permanently excited physical system. Thus, this leads to practical

222

Bioprocess Control

problems in the case of processes that are dangerous, costly, or operated in stationary mode. Furthermore, the relationships between mathematical and physical parameters are not always inversible in a unitary manner, thus complicating the task of diagnosis based on residuals. 8.3.3. Methods based on expertise With extremely rapid developments in computer science (theory and hardware), the idea of creating intelligent machines to automate the reasoning processing has appeared – and at times still appears – to be realizable. In fact, since the emergence of artificial intelligence (AI) in the 1950s, the mechanistic postulate whereby the computer models and completely substitutes humans has guided numerous researchers. Over the course of time – and after numerous disillusions – more pragmatic approaches have emerged and have made considerable progress. Thus, the cognitive approach defends the thesis of computer programs imitating the working human mind. This approach focuses on the understanding of human cognitive functions in mental terms and in terms of the reasoning process. Another approach, i.e., the pragmatic approach, considers that the lessons provided by AI are not an end in themselves but are means for developing theories, enabling us to improve our capacity to “effectively” program computers. Finally, the connectionist approach makes a rough, imperfect and reduced model of the human brain’s working. The latter actually tries to reproduce behaviors observed on the basis of learning. This learning is then generalized in an empirical manner to other examples, without there being at any time a necessity to describe any symbolic representation of knowledge. In the industry, these “intelligent”2 systems have found many fields of applications: breakdown diagnosis, control, supervision, planning, scheduling, simulation and process design. However, on a finer analysis of these diverse fields of application, it is possible to make out two major trends for the implementation of expert knowledge: “fully automatic”, whereby the knowledge is modeled and automated and the other trend in which machines and men share the intelligence and action levels. In both these trends, the basic problem is that concerned with the acquisition and representation of knowledge. The latter can be envisaged in two forms: deep and superficial knowledge. The first reveals the knowledge about internal processes governing the procedure (e.g., material and energy balances, causality ratios among variables). On the other hand, superficial knowledge contains non-formalized probabilities and heuristics reflecting the know-how of the process operators.

2. Here, we are talking about systems derived from AI techniques.

Tools for Fault Detection and Diagnosis

223

8.3.3.1. AI models Diagnosis is a causal process. It consists of determining the defective components that can explain an observed malfunction. If it is based on knowledge about the normal condition of the process, we refer to a coherence-based diagnosis. On the other hand, if information is available only about the breakdowns and their symptoms, we refer to an abductive diagnosis. Overall, three types of models are employed in AI within the context of monitoring: – in associative models, situations during which significant malfunctioning occurs are associated with alarm schematics using direct expression of connections between symptoms and characteristic situations. In general, they are based on existing expertise, whether it is with process operators or with maintenance engineers. Such models are used in particular in expert systems and in the identification of scenarios; – as for predictive models, they enable the simulation of the possible behaviors of a system in its different modes: normal behavior, degraded behavior or behavior in case of breakdown. Due to their capacity to calculate the predicted behaviors step-by-step, these models can be directly used to detect faults (i.e., by comparing the predicted behaviors with the observations). They can be obtained by abstraction of numerical models, as suggested by the qualitative physics approach. We can also obtain them using Bayesian networks and time-variant probabilist networks, constituting additional examples of predictive models; – referring to explicative models, they are based on expertise already formulated in the enterprise in the form of a breakdown catalog, of AMDEC, or of graphs. In the influence graphs for example, the interactions among the process variables are described qualitatively. Their nodes thus represent the variables and the arcs represent the connections among them. These arcs can be labeled by a sign, by a gain or by periods. They constitute a qualitative model, which can then be used for detection (e.g., by simulation when the input variables are propagated through the influence graph to obtain qualitative responses) [BAS 96]. Another example is the bond graph, especially adapted to model dynamic physical systems. It is based on the conservation of energy and is backed up by a library of basic primitives and a composition operation, enabling us to build the system’s global model in a systematic manner [TRA 97]. As for the causal graphs, they describe the chains of causation between the operating conditions and their observed effects (see Figure 8.7). The causal graphs are by far the most used for modeling real applications of supervision systems. Their advantage consists of expressing the knowledge explicitly and enabling us to manipulate directly the indirect or direct causal antecedents. We can also employ the graphs to produce a hierarchical cut of systems or even jointly with a quantitative model. We draw the reader’s attention to [TRA 97] for numerous other examples.

224

Bioprocess Control Through causal graph

Qualitative modeling and/or Quantitative modeling

Informs about Affects

Regulating organ

Θ

Regulating organ

pH [O2]

μ

DAM pO2

[X]

DN

Through material balance

Criterion of functioning

DP

dX = (μ − D) X dt ⎛ μ ⎞ Q dS +mS /X⎟ X+ in (Sin – S) = –⎜ V dt ⎝ YX /S ⎠ ⎞ ⎛ μ dC C + mC /X⎟ X – (Qin – Qout ) = Kl a C * – C – ⎜ dt V ⎠ ⎝ YX /C

(

[P]

[CO2] DQg

)

dV = Qin – Qout dt

[S] D%O2 DQs

Color [AAc] Odor

Analysis of gases DQsels

Input signals (microscopic observations)

[Sels]

observations

Internal signals (microscopic interpretations)

Figure 8.7. Example of causal graph for a biological process [STEY 91]

8.3.3.2. Artificial neural networks When sufficient knowledge about the process to be monitored is not available and when it is not possible to develop a knowledge model of the process, so-called black box models can be employed. This refers to the case of employing artificial neural networks (ANN) whose application in the fields of modeling, control and diagnosis has been widely reported in the literature (e.g. [NAR 90] or [BIS 94]). An ANN is in fact a computer system made up of a number of basic processors (or nodes) interconnected among themselves and processes – in a dynamic manner – the information coming to it from external signals. In general, an ANN is employed in two phases. At first, the network synthesis is carried out, and this comprises many stages: the selection of a network type, neural type, number of layers and learning methods. The learning enables us, on the basis of optimization of a criterion, to reproduce the behavior of the system to be modeled. It consists of searching for a set of parameters (weights) and can be performed in two ways: supervised (network uses the input and output data of the system to be modeled) and unsupervised (only the input data of the system are given and the learning is carried out by comparison between examples). When the learning results obtained by the ANN are satisfactory, we can use it for generalization. Here we are dealing with the second phase, wherein new examples – that have not been used during the learning – are used to judge the capacity of ANN to predict the behaviors of the modeled system. As mentioned earlier, we can use the ANN to diagnose the failures. Their low sensitivity to measurement noises, their capacity to resolve nonlinear and multivariable problems, to store knowledge in a compact manner and to learn online and in real time

Tools for Fault Detection and Diagnosis

225

are in fact the features that make them attractive for diagnosing the failures. They can thus be used at three levels: – as a model for the system to be monitored under normal condition and generating an error residual between the observations and the predictions; – as a system of evaluation of residuals for the diagnosis; – or as a system of detection in one stage (as classifiers) or in two stages (for the generation of residuals and the diagnosis). 8.3.3.3. Fuzzy inference systems During the past 20 years, Fuzzy Inference Systems (FIS) – which are based on Zadeh’s fuzzy set theory [ZAD 65] – have become very popular. Applications in signal processing, modeling, control, process supervision and decision-making show in fact the capacity of FIS for handling highly nonlinear problems using expert knowledge. The basic structure of an FIS is made up of: 1) a universe of discourse containing the functions for the adherence of input and output variables to classes. These functions can have different forms, the most frequent ones being triangular, trapezoidal and Gaussian; 2) a knowledge base that contains the rules linking the input and output variables under the form “IF...THEN...”; 3) a reasoning mechanism basing its working on the generalized modus ponens logic. The FIS can be qualified by the “gray box” method. In fact, they express the expert knowledge in the form of inference rules while at the same time qualitatively classifying the inputs and outputs. They also carry out calculations on the basis of weights and functions that are fixed, so as to correspond to the observed behavior without there being any explicit physical significance. The most used formalisms for the FIS are those by Mamdani [MAM 74] and Takagi-Sugeno [TAK 85], the latter also being called T-S. Other formalisms exist in the literature, but their use is much more limited. The working mode of an FIS thus enables us to handle quantities and symbols in an indistinct manner, to manipulate them in order to make calculations and to explain the course traversed to obtain a result. Whereas the diagnostic tasks rest on the quantity of heuristics that are difficult to formalize in a mathematical model: – correlation among widely differing variables; – qualitative observations (e.g., color, noise); – intuitions, linked in fact to statistics (e.g., this apparatus poses more problems than that one), difficult to quantify but very effective.

226

Bioprocess Control

Faults Input

Unknown inputs

Process Qualitative model

Measurements

Disagreement detector Faults

Generator of prospects Figure 8.8. Knowledge based observer

Recently, fuzzy models were proposed by Frank for detecting faults by using “knowledge-based observers” (KBO) whose structure is represented in Figure 8.8 [FRA 96]. A KBO is in fact an expert system made up of the following modules: – a qualitative model of the process for determination of expected conditions; – a disagreement detector which compares the output of the model with that of the process; – a generator of prospects for proposing a list of possible faults. A very interesting approach was also set up by Evsukoff and collaborators [EVS 98]. Here, the residuals are generated from numerical simulations, taking into account the errors in modeling and the noises present in the system. These residuals are then evaluated by a fuzzy procedure, which will determine if the deviation is significant enough to generate a detection signal. Finally, all the “pertinent” residuals are included in a causal graph, in which each arc is represented by a transfer function. A numerical calculation is then made in order to define if the locally observed effects are justified by the residual or if its cause is upstream. Finally, a fuzzy procedure decides about the fault isolation. Recently, Simani has presented an approach to fault detection using sensors applied to an energy-generating station [SIM 99]. The station is simulated by a T-S FIS which, on the basis of a series of linear models representing the diverse working modes, enables the generation of residuals using the difference with respect to real measurements. These residuals are finally compared with the thresholds for diagnosing the malfunctioning sensor.

Tools for Fault Detection and Diagnosis

Layer 1

Layer 2

Layer 3

Layer 4 x y

227

Layer 5

A1 x

A2

Π

w1

N

w1

A5

w1+f1 Σ

y

A3

Π

A6

N w2

w2

A4

f

w2+f2

x y

Figure 8.9. Structure of a T-S neuro-fuzzy system (according to [JAN 97])

Furthermore, neuro-fuzzy hybrid methods have been proposed for the modeling, control and supervision of industrial processes. These methods depend on two main architectures: – those in which the learning of an ANN is optimized with the use of fuzzy rules; – those in which the FIS is the basic structure and whose adherence functions and rules are optimized by learning functions normally used for the ANN (see Figure 8.9). 8.3.4. Choice and combined use of diverse methods In fact, the methods presented previously are neither opposed, nor contrary; they are complementary with each other. It is therefore not surprising to see applications, wherein different handling levels (i.e., signals, models and expert knowledge) are integrated in the same structure. For example, [DAL 79] proposed the combined use of equipment redundancy along with the parity space method. Isermann and Ulieru have also developed a methodology for integrating heterogenous source symptoms (comparison of thresholds, analytical redundancy and signal models), which are then integrated into a fuzzy decision-making module [ISE 93]. This fuzzy model constitutes in fact a mapping between the symptoms and faults in a directed graph schematic diagram. More recently, [GIR 98] proposed the application of statistical methods for generating the residuals, followed by the use of a causal graph for representing the overall diagnosis of the condition of an installation. 8.4. Application to biological processes For the past 40 years, numerous studies applied to bioprocesses have been put to the test. Given their complexity and high economic interest, these processes have in

228

Bioprocess Control

fact monopolized the attention of specialists from many domains: geneticists, microbiologists, process engineers, control engineers, computer specialists, etc. Concerning specifically the modeling and control of bioprocesses, we can see numerous applications of methods based on analytical models based on the material balance. On the other hand, it is surprising to note that as for the FDD, it is especially the use of statistical techniques and AI techniques which prevail. In fact, this is not particularly amazing as the bioprocesses show highly nonlinear, multivariable and dynamic characteristics, which at times render the mathematical models null and void very rapidly. In addition, the lack of reliable and less expensive sensors has for a long time proved to be a serious disadvantage for the application of analytical methods. Later, we will illustrate the application of diverse techniques for fault detection and diagnosis in the case of relatively “simple” biological processes (i.e., mono-biomass, mono-substrate) and will enlarge the field of application to more complex processes (i.e., wastewater treatment processes). 8.4.1. “Simple” biological processes Since the 1980s, Georges and Gregory Stephanopoulos have been of the opinion that many important elements of biological processes had to be taken into account while developing an AI approach for supervision [STEP 86]: – multi-objective characteristics of bioprocesses; – use of knowledge coming from different domains; – need to build on human expertise; – possible use of analytical models. Later, Konstantinov and Yoshida prepared a similar balance sheet for the application of control systems to bioprocesses [KON 92b]. In fact, according to them, the high complexity and numerous uncertainties that exist with regard to these processes necessitate sophisticated logical operations that cannot be provided by the traditional control. As a result, they consider the use of new formalisms for exploiting the knowledge about these processes as much as possible. Thus, nonlinear time-scale change methods have for example been put in place for “normalizing” the time-variant evolution of measured variables [GOL 96]. This approach has particularly enabled the application of traditional methods for the detection of faults to processes of low reproducibility such as bioprocesses. It has also proved to be very effective for detecting discrete events such as the diverse phases of the fermentation of Escherichia coli in a semi-continuous reactor and the diauxic fermentation of Saccaromyces cerevisiae.

Tools for Fault Detection and Diagnosis

229

Diverse examples of ANN applications to bioprocesses can also be found in the literature (see for example [PAT 97, SHI 98]). In the later example, an auto-associative ANN was used to learn about the faults of temperature sensors and plasmidic instability in a discontinuous fermentation of α-amylase by recombining Saccaromyces cerevisiae. The results are compared in particular to statistical methods such as PCA, and the capacity of ANN to represent the nonlinearities of the process was highlighted. In addition, approaches based on AI have been put into concrete form by numerous expert systems applied to fault detection and bioprocess diagnosis (see for example [ASA 90, STEY 93, SUZ 97]). Also, the interest for considering the expert knowledge relating to the time-variant evolutions of bioprocess variables has been demonstrated with a view to determining the physiological state of micro-organisms [STEY 91, KON 92a]. Algorithms based on fuzzy logic were also used. As an illustration, the monitoring of cellular activity for the production of proteins in unicellular organisms was carried out by adjusting – using fuzzy logic – the set point of optical density [SOT 84]. Another fuzzy expert system enabled the determination of the sterility condition of fermentors producing β-galactosidase [OJA 94]. In the case of contaminations that are very difficult to diagnose, the system also provides information to the human operator about the most suitable purification process for minimizing the losses. Fuzzy formalism was also applied to the piloting of semi-continuous yeast fermentations [SII 95]. More recent studies called upon the recognition of forms using fuzzy classification for detecting sensor faults and validating the measurements [BOU 98]. 8.4.2. Wastewater treatment processes Wastewater treatment processes refer to particular cases of bioprocesses. However, with respect to “traditional” processes (i.e., those used in agri-food industries or pharmaceuticals), they are subjected to numerous, more considerable perturbations that are external as well as internal. They are also much more complex as we are generally dealing with ecosystems and not with pure cultures. Just to cite an example, if the input to a bioprocess involving agri-food or pharmaceuticals is controlled and known, the same cannot be assumed in the domain of wastewater treatment, far from it (see for example the effect of heavy rainfall on the working of urban wastewater treatment plants). The necessity of fault detection and diagnosis to develop systems has thus grown and numerous papers have started to tackle this subject. The earlier papers making use of human expertise were developed on the basis of the fuzzy approach around the late 1970s. By employing the profile of dissolved oxygen, an FIS has for example enabled the estimation of substrate consumption in an activated sludge process and provided information to the human operators about changing the set points of traditional controllers in the installation [FLA 80]. At the same time, 20 fuzzy rules were proposed for regulating the flow rate of an activated sludge process [TON 80]. More

230

Bioprocess Control

recently, a monitoring system based on adaptive fuzzy classification was put up for managing many anaerobic treatment plants [MAR 96]. With the same order of ideas, a hybrid procedure combining FIS and ANN was developed for diagnosing a fluidizedbed anaerobic digester [STEY 97]. Finally, the fuzzy approach was used successfully for checking the reduction in chemical oxygen demand in an urban purification plant [FU 98] and for managing an anaerobic digestion process [GEN 99, GEN 00]. More generally, a review carried out in 1992 about environment technologies bore witness to the vast range of the AI – environment approach [CLI 92]. The diversity of domains – more than 100 projects were counted in fact – illustrated the possibilities provided by AI in the matter of natural disaster management, industrial risks and water resources as well as toxic and dangerous wastes. The review further pointed out that France is at the forefront in the expertise connected with the water area by citing projects such as “Qualiteau” for optimizing the water quality, “Ozonor” for managing the working of an ozonization plant, “Castor” for supervising water exchanges in a crisis situation and the expert systems “Orage” and “Géant” for managing and detecting anomalies in purification plants. On a more international scale, diverse expert systems such as those covered in [HOR 86, WAT 91] for supervising activated sludge processes and that referred to in [JEN 87] for managing a catchment area were proposed. An anaerobic digester was also managed by an expert system based on fuzzy rules [BAR 92]. This system enabled the correction of hydraulic, organic or toxic overloads on the basis of online measurements and the calculation of the variation rates and trends. Furthermore, good results are obtained for the same type of biological process by [PUL 98] in the face of perturbations on the effluent quality. Statistical techniques for evaluating the sensors were also proposed by [WAT 93]. The process condition is evaluated by methods involving the recognition of forms of online microscopic images. Multi-dimensional analytical techniques such as PCA and PLS are also proposed in [ROS 98] and applied to the detection of changes in the characteristics of an effluent in a purification plant. These techniques also enable the evaluation of the condition of sensors and the process depending on the working mode. More recently, [PON 99] has used the PCA over the respirometric profile of the oxygen consumption rate to detect the presence of toxic substances or bad meteorological conditions (presence of thunder storms) in an activated sludge process. Detection of faults using parametric methods on nonlinear models was tackled by [FUE 96]. Here, the recursive least squares method (RLS) is employed to estimate the model parameters of process material balance. This approach uses the online data (such as flow rates and optical density) and offline data such as the substrate and biomass concentrations at the process output and in the recirculation loop. This enables the detection of changes in the rate of growth and decantation of biomass as well as fault detection in aeration turbines.

Tools for Fault Detection and Diagnosis

231

Furthermore, analytical methods are proposed by [KIU 96]. Their approach consists of generating the residuals using linear observers for the supervision of an anaerobic digester. The system is rendered robust through an evaluation stage of the residuals produced by FIS, and it enables the isolation of faults due to organic and hydraulic overloads, presence of toxic elements, acidification as well as sensor faults. Finally, quite encouraging diagnosis results from an anaerobic digester were obtained, thanks to a so-called interval or set analytical approach, enabling us to obtain the nonlinearities and uncertainties of purification processes (see [RAP 98, HAD 98, ALCA 99, HAD 99, GOU 00]). 8.5. Conclusion As we have seen throughout this chapter, there are a considerable number of methodologies available for the designer of systems intended for detecting and diagnosing industrial processes. Nevertheless, this apparent abundance offers a restricted scope of application because: – sensors give pertinent information; – there is rich historical data; – reliable models of installation under normal conditions or in failure mode are not always available or do not necessitate highly prohibitive investments in terms of money or personnel. The availability of sensors is the first most important point and as was expressed by Olsson and Newell, “measuring, is knowing” [OLS 97]. For FDD applications, the number of measurements available under actual installation conditions determine in fact the detectability and isolability of faults. It is therefore desirable to integrate the concept of diagnosability on the same level as any other performance criterion and this, right from the design of an industrial installation. This would enable in particular the establishment of the number and locations of sensors for carrying out a pertinent and effective diagnosis of the concerned processes. As for historical data, it has to be admitted that – if they are available for an industrial process at all – they are often very heterogenous and difficult to use. On the one hand, the type of equipment support (i.e., graphs printed on paper, tables, etc.) used renders it very difficult to look for elements that are relevant for the diagnosis. On the other hand, expert knowledge on installations under normal and failure conditions is volatile (e.g., the process operator going on retirement, elements that are subjective and difficult to formalize and at times missing from archives, etc.). The abundance and variety of this useful knowledge thus makes very difficult – if not impossible – the use of a unique procedure towards fault detection and diagnosis. Finally, the acquisition of a reliable model that is representative of the relevant variables of an industrial process is a long and difficult task. Furthermore, this at times

232

Bioprocess Control

calls for operating conditions that cannot be met on account of constraints in industrial production. The modeling would therefore often depend on historical data, which might lack exciter signals for certain outputs as well as representativity over the installation’s entire possible working range. In this context, it is not surprising to note that in spite of the variety of applications that exist in FDD and notwithstanding the large number of publications and scientific symposia on the subject, there is a gulf between the theoretical developments of methods and their effective application to industrial processes. However, we believe that this gulf will diminish in the near future. 8.6. Bibliography [ALCA 99] V. A LCARAZ -G ONZALEZ, A. G ENOVESI, J. H ARMAND, A.V. G ONZALEZ, A. R APAPORT and J.P. S TEYER, “Robust exponential nonlinear observers for a class of lumped models useful in chemical and biochemical engineering – application to a wastewater treatment process”, in International Workshop on Application of Interval Analysis to Systems and Control, MISC’99, Girona, Spain, pp. 225–235, 24–26 February, 1999. [ALCO 97] E. A LCORTA G ARCÍA and P.M. F RANK, “Deterministic nonlinear observer-based approaches to fault diagnosis: a survey”, Control Engineering Practice, vol. 5, no. 5, pp. 663–670, 1997. [ASA 90] H. A SAMA, T. NAGAMUNE, M. H IRATA, A. H IRATA and I. E NDO, “An expert system for cultivating operations”, Ann. N.Y. Acad. Sci., no. 589, pp. 569–579, 1990. [BAR 92] M.W. BARNETT and J.F. A NDREWS, “Expert system for anaerobic digestion process operation”, Journal f Environmental Engineering, vol. 118, no. 6, pp. 949–963, 1992. [BAS 96] M. BASSEVILLE and M.O. C ORDIER, “Surveillance et diagnostic de systèmes dynamiques: approches complèmentaires du traitement de signal et de l’intelligence artificielle”, Research report, no. 2861, Research Department INRIA Rennes, pp. 1–40, 1996. [BEA 71] R.V. B EARD, Failure accomodation in linear systems through self reorganization, PhD thesis, Massachussets Institute of Technology (MIT), 1971. [BEN 96] F. B EN H MIDA, J.-M. O LIVE and J.-C. B ERTRAND, “Statistical assesment of fault indicator signals for online detection and location”, in CESA’96 IMACS Multiconference, Lille, France, 1996. [BIS 94] C.M. B ISCHOP, “Neural networks and their applications”, Rev. Sci. Instrum., vol. 65, no. 6, pp. 1803–1832, 1994. [BOU 98] A.N. B OUDAOUD and M.H. M ASSON, “newblock Analytical redundancy and the design of robust detection systems”, in IFAC Workshop on On-line Fault Detection and Supervision in the Chemical Process Industries, Solaize, Lyon, France, IFP-IFAC, 1998. [BRI 92] D. B RIE, Méthodes statistiques de détection de rupture de modèle. Application au traitement du signal issu d’un capteur à courant de Foucault, PhD thesis, University of Nancy I, France, 1992.

Tools for Fault Detection and Diagnosis

233

[CHE 99] J. C HEN and R.J. PATTON, Robust Model-based Fault Diagnosis for Dynamic Systems, Kluwer Academic, 1999. [CHO 84] E.Y. C HOW and A.S. W ILLSKY, “Analytical redundancy and the design of robust failure detection systems”, IEEE Transactions on Automatic Control, vol. 29, no. 7, pp. 603– 614, 1984. [CLA 78] R.N. C LARK, “Instrument fault detection”, IEEE Trans. Aerospace Electron. Syst., vol. 14, no. 3, pp. 456–465, 1978. [CLI 92] C. C LICQUOT DE M ENTQUE, Technologies de l’Environnement, Collection Nouvelles Technologies de l’Information, A JOUR, Editor, 1992. [DAL 79] K.C. DALY, E. G AI and J.V. H ARRISON, “Generalized likelihood test for FDI in redundancy sensor configuration”, IEEE Transactions on Automatic Control, vol. 22, pp. 759–803, 1979. [DOV 97] V. D OVI, A. R EVERBERI and L. M AGA, “Reconciliation of process measurements when data are subject to detection limits”, Chemical Engineering Science, vol. 52, no. 17, pp. 3047–3050, 1997. [EVS 98] A. E VSUKOFF, J. M ONTMAIN and S. G ENTIL, “Causal model based supervising and training”, in IFAC Workshop on On-line Fault Detection and Supervision in the Chemical Process Industries, Solaize, Lyon, France, IFP-IFAC, 1998. [FLA 80] M.J. F LANAGAN, “On the application of approximate reasoning to control of the activated sludge process”, in Joint Automatic Control Conference, San Francisco, USA, 1980. [FRA 93] P.M. F RANK and X. D ING, “Frequency domain approach to minimizing detectable faults in FDI systems”, Applied Mathematics and Computer Science, vol. 3, no. 3, pp. 417– 443, 1993. [FRA 96] P.M. F RANK, “Analytical and qualitative model-based fault diagnosis – A survey and some new results”, European Journal of Control, vol. 2, pp. 6–28, 1996. [FU 98] C.S. F U AND M. P OCH, “Fuzzy model and decision of COD control for an activated sludge process”, Fuzzy Sets and Systems, vol. 93, no. 3, pp. 281–292, 1998. [FUE 96] M. F UENTE, P. V EGA, M. Z ARROP and M. P OCH, “Fault detection in a real wastewater plant using parameter estimation techniques”, Control Engineering Practice, vol. 4, no. 8, pp. 1089–1098, 1996. [GEN 99] A. G ENOVESI, J. H ARMAND and J.P. S TEYER, “A fuzzy logic based diagnosis system for the online supervision of an anaerobic digestor pilot-plant”, Biochemical Engineering Journal, vol. 3, pp. 171–183, 1999. [GEN 00] A. G ENOVESI, J. H ARMAND and J.P. S TEYER, “Integrated fault detection and isolation: Application to a winery’s wastewater treatment plant”, Applied Intelligence, vol. 13, pp. 59–76, 2000. [GER 98] J.J. G ERTLER, Fault Detection and Diagnosis in Engineering Systems, Marcel Dekker, Inc., 1998.

234

Bioprocess Control

[GIR 98] D. G IRAUD, Diagnostic des systèmes industriels complexes par agrégation de méthodes: Application à une station d’épuration, PhD thesis, Henri Poincaré University, Nancy I, 166 pages, 1998. [GOL 96] K. G OLLMER and C. P OSTEN, “Supervision of bioprocesses using a dynamic time warping algorithm”, Control Engineering Practice, vol. 4, no. 9, pp. 1287–1295, 1996. [GOU 00] J.-L. G OUZÉ, A. R APAPORT and M.Z. H ADJ -S ADOCK, “Interval observers for uncertain biological systems”, Ecological Modelling, vol. 133, no. 1–2, pp. 45–56, 2000. [HAD 98] M.Z. H ADJ -S ADOK and J.-L. G OUZÉ, “Bounds estimation for uncertain models of wastewater treatment”, in IEEE International Conf. on Control and Appliactions, Trieste, Italy, pp. 336–340, 1998. [HAD 99] M.Z. H ADJ -S ADOK and J.-L. G OUZÉ, “Comparison between two interval observers for wastewater treatment models”, European Control Conference ECC99, Karlsruhe, Germany, 8 pages (CD-ROM), September 1999. [HOR 86] N.J. H ORAN and C.R. E CCLES “The potential for expert systems in the operation and control of activated sludge plants”, Process Biochemistry, pp. 81–85, 1986. [ISE 93] R. I SERMANN and M. U LIERU, “Integrated fault detection and diagnosis”, in IEEE International Conference on Systems, Man and Cybernetics, Le Touquet, France, IEEE, 1993. [JAN 97] R.J.-S. JANG, C.-T. S UN and E. M IZUTANI, Neuro-fuzzy and soft computing. A computational approach to learning and machine intelligence, Prentice-Hall, 1997. [JEN 87] W.O. J ENKINS and P.W. J OWITT, “Expert systems in river basin management” Civil Engineering Systems, vol. 4, no. 1, pp. 31–38, 1987. [JOE 99] A. J OENTGEN, L. M IKENINA, R. W EBER, A. Z EUGNER and H.-J. Z IMMERMANN, “Automatic fault detection in gearboxes by dynamic fuzzy data analysis”, Fuzzy Sets and Systems, vol. 105, pp. 123–132, 1999. [JON 73] H.L. J ONES, Failure detection in linear systems, PhD thesis, Massachussets Institute of Technology (MIT), 1973. [KIU 96] N. K IUPEL and P.M. F RANK, “Fuzzy supervision for an anaerobic wastewater plant”, in CESA’96 IMACS Multiconference, Lille, France, 1996. [KON 92a] K. KONSTANTINOV and T. YOSHIDA, “Knowledge-based control of fermentation processes”, Biotechnology and Bioengineering, vol. 39, pp. 479–486, 1992. [KON 92b] K. KONSTANTINOV and T. YOSHIDA, “Real-time qualitative analysis of the temporal shapes of (bio)process variables”, AIChE Journal, vol. 38, no. 11, pp. 1703–1715, 1992. [KUE 61] D.R. K UEHN and H. DAVIDSON, “Computer control: Mathematics of control”, Chemical Engineering Progress, vol. 57, no. 6, pp. 44, 1961. [MAM 74] E.H. M AMDANI and P.D. M EM, “Application of fuzzy algorithms for control of simple dynamic plant”, Proceedings IEE, vol. 121, no. 12, 1974.

Tools for Fault Detection and Diagnosis

235

[MAR 96] S. M ARSILI -L IBELLI and A. M ULLER, “Adaptive fuzzy pattern recognition in the anaerobic digestion process”, Pattern Recognition Letters, vol. 17, no. 6, pp. 651–659, 1996. [NAR 90] K.S. NARENDRA and K. PARTHASARATHY, “Identification and control of dynamical systems using neural networks”, IEEE Transactions on Neural Networks, vol. 1, no. 1, pp. 4–27, 1990. [OJA 94] H. O JAMO, P. VAIJA and M. D OHNAL, “Intelligent assistance to the fermentation operation”, Acta Polytechnica Scandinavica, vol. 218 (Chemical Technology and Metallurgy Series), pp. 1–30, 1994. [OLS 97] G. O LSSON and R. N EWELL, “Reviewing, assessing and speculating”, Water Science Technology, vol. 37, no. 12, pp. 397–401, 1997. [PAT 91] R.J. PATTON and J. C HEN, “Deterministic nonlinear observer-based approaches to fault diagnosis: A survey”, in IFAC/IMACS Symposium on Fault Detection Supervision and Safety for Technical Processes, SAFEPROCESS’91, Baden-Baden, Germany, 1991. [PAT 97] P. PATNAIK, “A recurrent neural network for a fed batch fermentation with recombinant Escherichia coli subject to inflow disturbances”, Process Biochemistry, vol. 32, no. 5, pp. 391–400, 1997. [PON 99] M.N. P ONS, N. ROCHE, C. B LANC, O. P OTIER and C. P ROST, “Detection of critical situations in wastewater treatment plants”, in European Conference of Biotechnology, Brussels, Belgium, 1999. [PUI 99] V. P UIG, J. S ALUDES and J. Q UEVEDO, “A new algorithm for adaptive threshold generation in robust fault detection based on sliding window and global optimization”, in European Control Conference ECC’99, Karlsruhe, Germany, 8 pages on CD-ROM, 1999. [PUL 98] P.C. P ULLAMMANAPPALLIL, S.A. S VORONOS, D.P. C HYNOWETH and G. LYBERATOS, “Expert system for control of anaerobic digesters”, Biotechnology and Bioengineering, vol. 58, no. 1, pp. 13–22, 1998. [RAP 98] A. R APAPORT and J. H ARMAND, “Robust regulation of a bioreactor in a highly uncertain environment”, in IFAC-EurAgEng International Workshop on Decision and Control in Waste Bio-Processing, WASTE-DECISION’98, 8 pages (on CD-ROM), Narbonne, France, 25–27 February, 1998. [ROS 98] C. ROSEN and G. O LSSON, “Disturbance detection in wastewater treatment plants”, Water Science Technology, vol. 37, no. 12, pp. 197–205, 1998. [SCH 96] H. S CHNEIDER and P. F RANK, “Observer based supervision and fault detection in robots using nonlinear and fuzzy logic residual evaluation”, IEEE Transactions on control systems technology, vol. 4, no. 3, pp. 274–282, 1996. [SER 86] R.W. S ERTH and W.A. H EENAN, “Gross error detection and data reconciliation in steam-metering systems”, AIChE Journal, vol. 32, no. 5, pp. 733–742, 1986. [SHI 98] H. S HIMIZU, K. YASUOKA, K. U CHIYAMA and S. S HIOYA, “Bioprocess fault detection by nonlinear multivariate analysis: Application of an artificial autoassociative neural network and wavelet filter bank”, Biotechnology progress, vol. 14, no. 1, pp. 79– 87, 1998.

236

Bioprocess Control

[SII 95] T. S IIMES, P. L INKO, C. VON N UMERS, M. NAKAJIMA and I. E NDO, “Real-time fuzzy-knowledge-based control of baker’s yeast production”, Biotechnology and Bioengineering, vol. 45, pp. 135–143, 1995. [SIM 99] S. S IMANI, “Fuzzy multiple inference identification and its application to fault diagnosis of industrial processes”, in World Multiconference on Systemics, Cybernetics and Informatics SCI’99, Orlando, Florida, USA, 1999. [SON 90] Y. S ONODA, S. K ANEMOTO and H. I MARUOKA, “A diagnostic expert system for a boiling water reactor using a dynamic model”, in Artificial Intelligence in Nuclear Power Plants, Helsinki, Finland, 1990. [SOT 84] G. S OTIROV, D. F ILEV and K.B. KONSTANTINOV, “Hierarchical system for control of continuous fermentation process synthesized on the basis of linguistic approach”, in Lecture Notes in Control and Information Sciences, G. S CHMIDT, M. S INGH, A. T ITLI and S. T ZAFESTAS (eds.), pp. 638–643, Springer-Verlag, 1984. [STA 99] M. S TAROSWIECKI and G. C OMTET-VARGA, “Fault detectability and isolability in algebraic dynamic systems”, in European Control Conference ECC’99, Karlsruhe, Germany, 1999. [STEP 86] G. S TEPHANOPOULOS and G. S TEPHANOPOULOS, “Artificial intelligence in the development and design of biochemical processes”, Trends Biotechnology, vol. 4, pp. 241– 249, 1986. [STEY 91] J.P. S TEYER, Sur une approche qualitative des systèmes physiques: aide en temps réel à la conduite des procédés fermentaires, PhD thesis, Paul Sabatier University, Toulouse, France, 1991. [STEY 93] J.P. S TEYER, I. Q UEINNEC and D. S IMOES, “BIOTECH: a real-time application of artificial intelligence for fermentation processes”, Control Engineering Practice, vol. 1, no. 2, pp. 315–321, 1993. [STEY 97] J.P. S TEYER, D. ROLAND, J.C. B OUVIER and R. M OLETTA, “Hybrid fuzzy neural network for diagnosis – Application to the anaerobic treatment of wine distillery wastewater in a fluidized bed reactor”, Water Science Technology, vol. 36, no. 6-7, pp. 209–217, 1997. [SUZ 97] T. S UZUKI, Y. S AKINO, M. NAKAJIMA, H. A SAMA, T. F UJII, K. S ATO, H. K AETSU and I. E NDO, “A novel man machine interface for a bioprocess expert system constructed for cooperative decision making and operation”, Journal of Biotechnology, vol. 52, no. 3, pp. 277–282, 1997. [TAK 85] T. TAKAGI and M. S UGENO “Fuzzy identification of systems and its application to modelling and control”, IEEE Transactions on Systems, Man, and Cybernetics, vol. 15, pp. 116–132, 1985. [TON 80] R.M. T ONG, M.B. B ECK and A. L ATTEN, “Fuzzy control of the activated sludge wastewater treatment process”, Automatica, vol. 16, pp. 659–701, 1980. [TRA 97] L. T RAVÉ -M ASSUYÈS, P. DAGUE and F. G UERRIN, Le raisonnement qualitatif pour les sciences de l’ingénieur, Hermes, 501 pages, 1997.

Tools for Fault Detection and Diagnosis

237

[WAT 91] P. WATTS and B. K NIGHT, “Fault diagnosis in ASPEX: an expert system for the control of the activated sludge process”, Engineering Applications of Artificial Intelligence, vol. 4, no. 2, pp. 151, 1991. [WAT 93] S. WATANABE, K. BABA, M. YODA, W.C. W U, I. E NBUTSU, M. H IRAOKA and K. T SUMURA, “Intelligent operation support system for activated sludge process”, Water Science Technology, vol. 28, no. 11–12, pp. 325–332, 1993. [WIL 76] A.S. W ILLSKY, “A survey of design methods for failure detection systems”, Automatica, vol. 12, pp. 601–611, 1976. [XU 89] E. X U, G. X U and S. Z HANG, “Application of expert fuzzy controller in the penicillin fermentation process”, in IFAC Workshop on AI in Real-Time Control, Shenyang, China, 1989. [ZAD 65] L. Z ADEH, “Fuzzy sets”, Information and Control, vol. 8, pp. 338–353, 1965. [ZHA 97] J. Z HANG, E.B. M ARTIN and A.J. M ORRIS, “Process monitoring using nonlinear statistical techniques”, Chemical Engineering Journal, vol. 67, no. 3, pp. 181–189, 1997.

238

List of Authors

Olivier BERNARD COMORE Institut national de recherche en informatique et en automatique Sophia-Antipolis, France Denis DOCHAIN CESAME Université catholique de Louvain, Belgium Antoine GÉNOVÉSI CIRDC Groupe Danone, Le Plessis-Robinson, France Jean-Luc GOUZÉ COMORE Institut national de recherche en informatique et en automatique Sophia-Antipolis, France Martin GUAY Queen’s University Kingston, Canada

240

Bioprocess Control

Jérôme HARMAND LBE Institut national de recherche agronomique, Narbonne, France Michel PERRIER URCPC Ecole polytechnique de Montréal, Canada Isabelle QUEINNEC LAAS-CNRS Toulouse, France Jean-Philippe STEYER LBE Institut national de recherche agronomique, Narbonne, France Mariana TITICA GEPEA Saint-Nazaire, France Peter VANROLLEGHEM BIOMATH University of Ghent, Belgium

Index

A activated sludge, 229, 230 adaptive control, 135 adaptive linearizing control, 15, 162 adaptive observer, 133 anaerobic digestion, 68, 131, 163, 165, 230 anti-reset windup, 159 asymptotic observer, 118, 125

B

fed-batch, 62, 120, 124, 129, 145, 150, 161 fed-batch reactor, 228 Fisher information matrix, 53, 56, 59, 60

G–I general dynamic model, 116, 119, 129, 131 Haldane model, 63, 70, 146, 148, 149, 158 hybrid model, 14 identifiability, 47 instability, 70, 229

batch, 50, 61, 62, 120, 145, 150, 161 breakdown detection, 211

K–L

C

Kalman filter, 91 Kalman observer, 134 Luenberger observer, 118, 134

continuous reactor, 129, 145, 150 controller, 62, 144, 153, 155, 229 covariance matrix, 53, 55, 59, 65 culture of animal cells, 116, 119, 129

D, E diagnostic, 152, 211, 213, 214 equilibrium point, 70, 148, 149, 155 experiment design, 14, 48, 57, 58, 70, 72

F fault detection, 214 Fault Detection and Diagnosis (FDD), 228, 231 Fault Detection and Isolation (FDI), 212, 213

M maintenance reaction, 64 material balance, 13, 15, 64, 69, 123, 124, 128, 135, 145, 154, 165, 219, 228, 230 Monod’s model, 48, 50, 52, 56, 70, 135, 146, 156 mortality, 147

N neural network, 224 nitrification, 61 nonlinear PI controller, 160

242

Bioprocess Control

O

R, S

observability, 48 observer, 13, 15, 116, 218, 220, 226, 231 optimal adaptive control, 15

respirogram, 62 robust control, 162 sensitivity function, 53, 55, 73 software sensor, 13, 79, 144, 151 specific growth rate, 15, 73, 117, 119, 123, 145, 156 stability, 136, 137, 145, 147, 162, 167, 220 state transformation, 14, 120, 124, 136 structural identifiability, 47, 48 supervision, 15, 143, 213, 216, 225, 230

P parameter estimation, 15, 47, 54, 63, 115, 218, 221, 230 parameter identification, 13, 47, 54, 57, 63, 68 PI controller, 157, 160 PID controller, 158 pratical identifiability, 47, 52

V–Y variance, 54, 61, 62, 65, 66 yeast growth, 129