Stochastic Optimization Methods, Second Edition

Stochastic Optimization Methods Second edition Kurt Marti Stochastic Optimization Methods Second edition 123 Univ...

Author: Kurt Marti

98 downloads 1764 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Stochastic Optimization Methods Second edition

Kurt Marti

Stochastic Optimization Methods Second edition

123

Univ. Prof. Dr. Kurt Marti Federal Armed Forces University Munich Department of Aerospace Engineering and Technology 85577 Neubiberg/München Germany [email protected]

ISBN: 978-3-540-79457-8

e-ISBN: 978-3-540-79458-5

Library of Congress Control Number: 2008925523 c 2008 Springer-Verlag Berlin Heidelberg ° This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover design: WMXDesign GmbH, Heidelberg Printed on acid-free paper 9 8 7 6 5 4 3 2 springer.com

Preface

Optimization problems in practice depend mostly on several model parameters, noise factors, uncontrollable parameters, etc., which are not given ﬁxed quantities at the planning stage. Typical examples from engineering and economics/operations research are: Material parameters (e.g. elasticity moduli, yield stresses, allowable stresses, moment capacities, and speciﬁc gravity), external loadings, friction coeﬃcients, moments of inertia, length of links, mass of links, location of center of gravity of links, manufacturing errors, tolerances, noise terms, demand parameters, technological coeﬃcients in input-output functions, cost factors, etc.. Due to several types of stochastic uncertainties (physical uncertainty, economic uncertainty, statistical uncertainty, and model uncertainty) these parameters must be modelled by random variables having a certain probability distribution. In most cases at least certain moments of this distribution are known. To cope with these uncertainties, a basic procedure in engineering/economic practice is to replace ﬁrst the unknown parameters by some chosen nominal values, e.g. estimates or guesses, of the parameters. Then, the resulting and mostly increasing deviation of the performance (output, behavior) of the structure/system from the prescribed performance (output, behavior), i.e., the “tracking error”, is compensated by (online) input corrections. However, the online correction of a system/structure is often time-consuming and causes mostly increasing expenses (correction or recourse costs). Very large recourse costs may arise in case of damages or failures of the plant. This can be avoided to a large extent by taking into account at the planning stage the possible consequences of the tracking errors and the known prior and sample information about the random data of the problem. Hence, instead of relying on ordinary deterministic parameter optimization methods – based on some nominal parameter values – and applying some correction actions, stochastic optimization methods should be applied: By incorporating the stochastic parameter variations into the optimization process, expensive and increasing online correction expenses can be avoided or at least reduced to a large extent.

VI

Preface

Consequently, for the computation of robust optimal decisions/designs, i.e., optimal decisions which are insensitive with respect to random parameter variations, appropriate deterministic substitute problems must ﬁrst be formulated. Based on theoretical principles, these substitute problems depend on probabilities of failure/success and/or on more general expected cost/loss terms. Since probabilities and expectations are deﬁned by multiple integrals in general, the resulting often nonlinear and also nonconvex deterministic substitute problems can be solved only by approximative methods. Hence, the analytical properties of the substitute problems are examined, and appropriate deterministic and stochastic solution procedures are presented. The aim of this book is therefore to provide analytical and numerical tools – including their mathematical foundations – for the approximate computation of robust optimal decisions/designs as needed in concrete engineering/economic applications. Two basic types of deterministic substitute problems occur mostly in practice: *

Reliability-Based Optimization Problems: Minimization of the expected primary costs subject to expected recourse cost constraints (reliability constraints) and remaining deterministic constraints, e.g. box constraints. In case of piecewise constant cost functions, probabilistic objective functions and/or probabilistic constraints occur; * Expected Total Cost Minimization Problems: Minimization of the expected total costs (costs of construction, design, recourse costs, etc.) subject to the remaining deterministic constraints. Basic methods and tools for the construction of appropriate deterministic substitute problems of diﬀerent complexity and for various practical applications are presented in Chap. 1 and part of Chap. 2 together with fundamental analytical properties of reliability-based optimization/expected cost minimization problems. Here, the main problem is the analytical treatment and numerical computation of the occurring expectations and probabilities depending on certain input variables. For this purpose deterministic and stochastic approximation methods are provided, which are based on Taylor expansion methods, regression and response surface methods (RSM), probability inequalities, and diﬀerentiation formulas for probability and expectation value functions. Moreover, ﬁrst order reliability methods (FORM), Laplace expansions of integrals, convex approximation/deterministic descent directions/eﬃcient points, (hybrid) stochastic gradient methods, stochastic approximation procedures are considered. The mathematical properties of the approximative problems and iterative solution procedures are described. After the presentation of the basic approximation techniques in Chap. 2, in Chap. 3 derivatives of probability and mean value functions are obtained by the following methods: Transformation method, stochastic completion and transformation, and orthogonal function series expansion. Chapter 4 shows

Preface

VII

how nonconvex deterministic substitute problems can be approximated by convex mean value minimization problems. Depending on the type of loss functions and the parameter distribution, feasible descent directions can be constructed at non-eﬃcient points. Here, eﬃcient points, determined often by linear/quadratic conditions involving ﬁrst and second order moments of the parameters, are candidates for optimal solutions to be obtained with much larger eﬀort. The numerical/iterative solution techniques presented in Chap. 5 are based on hybrid stochastic approximation, methods using not only simple stochastic (sub) gradients, but also deterministic descent directions and/or more exact gradient estimators. More exact gradient estimators based on Response Surface Methods (RSM) are described in the ﬁrst part of Chap. 5. Convergence in the mean square sense is considered, and results on the speed up of the convergence by using improved step directions at certain iteration points are given. Various extensions of hybrid stochastic approximation methods are treated in Chap. 6 on stochastic approximation methods with changing variance of the estimation error. Here, second order search directions, based e.g. on the approximation of the Hessian, and asymptotic optimal scalar and matrix valued step sizes are constructed. Moreover, using diﬀerent types of probabilistic convergence concepts, related convergence properties and convergence rates are given. Mathematical deﬁnitions and theorems needed here are given in the Appendix. Applications to the approximate computation of survival/failure probabilities for technical and economical systems/structures are given in Chap. 7. The reader of this book needs some basic knowledge in linear algebra, multivariate analysis and stochastics. To complete this monograph, the author was supported by several collaborators: First of all I thank Ms Elisabeth L¨ oßl from the Institute for Mathematics and Computer Sciences for the excellent LATEX typesetting of the whole manuscript. Without her very precise and careful work, this project could not have been completed. I also owe thanks to my former collaborators, Dr. Andreas Aurnhammer, for further LATEX support and for providing English translation of some German parts of the original manuscript, and to Thomas Platzer for proof reading of some parts of the book. Last but not least I am indebted to Springer-Verlag for including the book in the Springerprogram. I thank, especially, the Publishing Director Economics and Management Science of Springer-Verlag, Dr. Werner A. M¨ uller, for his great patience until the completion of this book. Munich August 2004

Kurt Marti

VIII

Preface

Preface to the Second Edition The major change in the second edition of this book is the inclusion of a new, extended version of Chap. 7 on the approximate computation of probabilities of survival and failure of technical, economic structures and systems. Based on a representation of the state function of the structure/system by the minimum value of a certain optimization problem, approximations of the probability of survival/failure are obtained by means of polyhedral approximation of the socalled survival domain, certain probability inequalities and disretizations of the underlying probability distribution of the random model parameters. All the other chapters have been updated by including some more recent material and by correcting some typing errors. The author again thanks Ms. Elisabeth L¨ oßl for the excellent LaTeXtypesetting of all revisions and completions of the text. Moreover, I thank Dr. M¨ uller, Vice President Business/Economics and Statistics, Springer-Verlag, for making it possibile to publish a second edition of “Stochastic Optimization Methods”. Munich March 2008

Kurt Marti

Contents

Part I Basic Stochastic Optimization Methods 1

2

Decision/Control Under Stochastic Uncertainty . . . . . . . . . . . 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Deterministic Substitute Problems: Basic Formulation . . . . . . . . 1.2.1 Minimum or Bounded Expected Costs . . . . . . . . . . . . . . . 1.2.2 Minimum or Bounded Maximum Costs (Worst Case) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deterministic Substitute Problems in Optimal Decision Under Stochastic Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Optimum Design Problems with Random Parameters . . . . . . . . 2.1.1 Deterministic Substitute Problems in Optimal Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Deterministic Substitute Problems in Quality Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Basic Properties of Substitute Problems . . . . . . . . . . . . . . . . . . . . 2.3 Approximations of Deterministic Substitute Problems in Optimal Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Approximation of the Loss Function . . . . . . . . . . . . . . . . 2.3.2 Regression Techniques, Model Fitting, RSM . . . . . . . . . . 2.3.3 Taylor Expansion Methods . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Applications to Problems in Quality Engineering . . . . . . . . . . . . 2.5 Approximation of Probabilities: Probability Inequalities . . . . . . 2.5.1 Bonferroni-Type Inequalities . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Tschebyscheﬀ-Type Inequalities . . . . . . . . . . . . . . . . . . . . . 2.5.3 First Order Reliability Methods (FORM) . . . . . . . . . . . . .

3 3 5 6 8 9 9 13 16 18 19 20 22 26 29 30 31 32 37

X

Contents

Part II Diﬀerentiation Methods 3

Diﬀerentiation Methods for Probability and Risk Functions 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Transformation Method: Diﬀerentiation by Using an Integral Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Representation of the Derivatives by Surface Integrals . . 3.3 The Diﬀerentiation of Structural Reliabilities . . . . . . . . . . . . . . . 3.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 More General Response (State) Functions . . . . . . . . . . . . 3.5 Computation of Probabilities and its Derivatives by Asymptotic Expansions of Integral of Laplace Type . . . . . . . 3.5.1 Computation of Probabilities of Structural Failure and Their Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Numerical Computation of Derivatives of the Probability Functions Arising in Chance Constrained Programming . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Integral Representations of the Probability Function P (x) and its Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Orthogonal Function Series Expansions I: Expansions in Hermite Functions, Case m = 1 . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Integrals over the Basis Functions and the Coeﬃcients of the Orthogonal Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Estimation/Approximation of P (x) and its Derivatives . 3.7.3 The Integrated Square Error (ISE) of Deterministic Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Orthogonal Function Series Expansions II: Expansions in Hermite Functions, Case m > 1 . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Orthogonal Function Series Expansions III: Expansions in Trigonometric, Legendre and Laguerre Series . . . . . . . . . . . . . 3.9.1 Expansions in Trigonometric and Legendre Series . . . . . . 3.9.2 Expansions in Laguerre Series . . . . . . . . . . . . . . . . . . . . . . .

43 43 46 51 54 57 57 62 62

66 72 75 79 82 88 89 91 92 92

Part III Deterministic Descent Directions 4

Deterministic Descent Directions and Eﬃcient Points . . . . . . 95 4.1 Convex Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.1.1 Approximative Convex Optimization Problem . . . . . . . . . 99 4.2 Computation of Descent Directions in Case of Normal Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 4.2.1 Descent Directions of Convex Programs . . . . . . . . . . . . . . 105 4.2.2 Solution of the Auxiliary Programs . . . . . . . . . . . . . . . . . . 108 4.3 Eﬃcient Solutions (Points) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.3.1 Necessary Optimality Conditions Without Gradients . . . 116

Contents

XI

4.3.2 Existence of Feasible Descent Directions in Non-Eﬃcient Solutions of (4.9a), (4.9b) . . . . . . . . . . . . 117 4.4 Descent Directions in Case of Elliptically Contoured Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 4.5 Construction of Descent Directions by Using Quadratic Approximations of the Loss Function . . . . . . . . . . . . . . . . . . . . . . 121

Part IV Semi-Stochastic Approximation Methods 5

RSM-Based Stochastic Gradient Procedures . . . . . . . . . . . . . . . 129 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 5.2 Gradient Estimation Using the Response Surface Methodology (RSM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 5.2.1 The Two Phases of RSM . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.2.2 The Mean Square Error of the Gradient Estimator . . . . 138 5.3 Estimation of the Mean Square (Mean Functional) Error . . . . . 142 5.3.1 The Argument Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.3.2 The Criterial Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 5.4 Convergence Behavior of Hybrid Stochastic Approximation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 5.4.1 Asymptotically Correct Response Surface Model . . . . . . 148 5.4.2 Biased Response Surface Model . . . . . . . . . . . . . . . . . . . . . 150 5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 5.5.1 Fixed Rate of Stochastic and Deterministic Steps . . . . . . 158 5.5.2 Lower Bounds for the Mean Square Error . . . . . . . . . . . . 169 5.5.3 Decreasing Rate of Stochastic Steps . . . . . . . . . . . . . . . . . 173

6

Stochastic Approximation Methods with Changing Error Variances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 6.2 Solution of Optimality Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 178 6.3 General Assumptions and Notations . . . . . . . . . . . . . . . . . . . . . . . 179 6.3.1 Interpretation of the Assumptions . . . . . . . . . . . . . . . . . . . 181 6.3.2 Notations and Abbreviations in this Chapter . . . . . . . . . . 182 6.4 Preliminary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 6.4.1 Estimation of the Quadratic Error . . . . . . . . . . . . . . . . . . . 183 6.4.2 Consideration of the Weighted Error Sequence . . . . . . . . 185 6.4.3 Further Preliminary Results . . . . . . . . . . . . . . . . . . . . . . . . 188 6.5 General Convergence Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.5.1 Convergence with Probability One . . . . . . . . . . . . . . . . . . . 190 6.5.2 Convergence in the Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.5.3 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . . 195 6.6 Realization of Search Directions Yn . . . . . . . . . . . . . . . . . . . . . . . . 204 6.6.1 Estimation of G∗ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

XII

Contents

6.6.2 Update of the Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 6.6.3 Estimation of Error Variances . . . . . . . . . . . . . . . . . . . . . . . 216 6.7 Realization of Adaptive Step Sizes . . . . . . . . . . . . . . . . . . . . . . . . . 220 6.7.1 Optimal Matrix Step Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . 221 6.7.2 Adaptive Scalar Step Size . . . . . . . . . . . . . . . . . . . . . . . . . . 227 6.8 A Special Class of Adaptive Scalar Step Sizes . . . . . . . . . . . . . . . 236 6.8.1 Convergence Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 6.8.2 Examples for the Function Qn (r) . . . . . . . . . . . . . . . . . . . . 241 6.8.3 Optimal Sequence (wn ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 6.8.4 Sequence (Kn ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Part V Reliability Analysis of Structures/Systems 7

Computation of Probabilities of Survival/Failure by Means of Piecewise Linearization of the State Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 7.2 The State Function s∗ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 7.2.1 Characterization of Safe States . . . . . . . . . . . . . . . . . . . . . . 258 7.3 Probability of Safety/Survival . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 7.4 Approximation I of ps , pf : FORM . . . . . . . . . . . . . . . . . . . . . . . . . 262 7.4.1 The Origin of IRν Lies in the Transformed Safe Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 7.4.2 The Origin of IRν Lies in the Transformed Failure Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 7.4.3 The Origin of IRν Lies on the Limit State Surface . . . . . . 268 7.4.4 Approximation of Reliability Constraints . . . . . . . . . . . . . 269 7.5 Approximation II of ps , pf : Polyhedral Approximation of the Safe/Unsafe Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 7.5.1 Polyhedral Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 273 7.6 Computation of the Boundary Points . . . . . . . . . . . . . . . . . . . . . . 279 7.6.1 State Function s∗ Represented by Problem A . . . . . . . . . 280 7.6.2 State Function s∗ Represented by Problem B . . . . . . . . . 280 7.7 Computation of the Approximate Probability Functions . . . . . . 282 7.7.1 Probability Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 7.7.2 Discretization Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 7.7.3 Convergent Sequences of Discrete Distributions . . . . . . . 293

Part VI Appendix A

Sequences, Series and Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 A.1 Mean Value Theorems for Deterministic Sequences . . . . . . . . . . 301 A.2 Iterative Solution of a Lyapunov Matrix Equation . . . . . . . . . . . 309

Contents

XIII

B

Convergence Theorems for Stochastic Sequences . . . . . . . . . . . 313 B.1 A Convergence Result of Robbins–Siegmund . . . . . . . . . . . . . . . . 313 B.1.1 Consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 B.2 Convergence in the Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 B.3 The Strong Law of Large Numbers for Dependent Matrix Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 B.4 A Central Limit Theorem for Dependent Vector Sequences . . . . 319

C

Tools from Matrix Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 C.1 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 C.2 The v. Mises-Procedure in Case of Errors . . . . . . . . . . . . . . . . . . . 322

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

1 Decision/Control Under Stochastic Uncertainty

1.1 Introduction Many concrete problems from engineering, economics, operations research, etc., can be formulated by an optimization problem of the type min f0 (a, x)

(1.1a)

s.t. fi (a, x) ≤ 0, i = 1, . . . , mf

(1.1b)

gi (a, x) = 0, i = 1, . . . , mg

(1.1c)

x ∈ D0 .

(1.1d)

Here, the objective (goal) function f0 = f0 (a, x) and the constraint functionsfi = fi (a, x), i = 1, . . . , mf , gi = gi (a, x), i = 1, . . . , mg , deﬁned on a joint subset of IRν × IRr , depend on a decision, control or input vector x = (x1 , x2 , . . . , xr )T and a vector a = (a1 , a2 , . . . , aν )T of model parameters. Typical model parameters in technical applications, operations research and economics are material parameters, external load parameters, cost factors, technological parameters in input–output operators, demand factors. Furthermore, manufacturing and modeling errors, disturbances or noise factors, etc., may occur. Frequent decision, control or input variables are material, topological, geometrical and cross-sectional design variables in structural optimization [62], forces and moments in optimal control of dynamic systems and factors of production in operations research and economic design. The objective function (1.1a) to be optimized describes the aim, the goal of the modelled optimal decision/design problem or the performance of a technical, economic system or process to be controlled optimally. Furthermore, the constraints (1.1b)–(1.1d) represent the operating conditions guaranteeing a safe structure, a correct functioning of the underlying system, process, etc. Note that the constraint (1.1d) with a given, ﬁxed convex subset

4

1 Decision/Control Under Stochastic Uncertainty

D0 ⊂ IRr summarizes all (deterministic) constraints being independent of unknown model parameters a, as, e.g., box constraints: xL ≤ x ≤ xU

(1.1d )

with given bounds xL , xU . Important concrete optimization problems, which may be formulated, at least approximatively, this way are problems from optimal design of mechanical structures and structural systems [2, 62, 130, 138, 146, 148], adaptive trajectory planning for robots [3, 6, 27, 97, 111, 143], adaptive control of dynamic system [144,147], optimal design of economic systems [123], production planning, manufacturing [73, 118] and sequential decision processes [101], etc. A basic problem in practice is that the vector of model parameters a = (a1 , . . . , aν )T is not a given, ﬁxed quantity. Model parameters are often unknown, only partly known and/or may vary randomly to some extent. Several techniques have been developed in the recent years in order to cope with uncertainty with respect to model parameters a. A well known basic method, often used in engineering practice, is the following two-step procedure [6, 27, 111, 143, 144]: (1) Approximation: Replace ﬁrst the ν-vector a of the unknown or stochastic varying model parameters a1 , . . . , aν by some ﬁxed vector a0 of so-called nominal values a0l , l = 1, . . . , ν. Apply then an optimal decision (control) x∗ = x∗ (a0 ) with respect to the resulting approximate optimization problem min f0 (a0 , x)

(1.2a)

s.t. fi (a0 , x) ≤ 0, i = 1, . . . , mf

(1.2b)

gi (a0 , x) = 0, i = 1, . . . , mg

(1.2c)

x ∈ D0 .

(1.2d)

Due to the deviation of the actual parameter vector a from the nominal vector a0 of model parameters, deviations of the actual state, trajectory or performance of the system from the prescribed state, trajectory, goal values occur. (2) Compensation: The deviation of the actual state, trajectory or performance of the system from the prescribed values/functions is compensated then by online measurement and correction actions (decisions or controls). Consequently, in general, increasing measurement and correction expenses result in course of time. Considerable improvements of this standard procedure can be obtained by taking into account already at the planning stage, i.e., oﬄine, the mostly available a priori (e.g., the type of random variability) and sample information

1.2 Deterministic Substitute Problems: Basic Formulation

5

about the parameter vector a. Indeed, based, e.g., on some structural insight, or by parameter identiﬁcation methods, regression techniques, calibration methods, etc., in most cases information about the vector a of model/structural parameters can be extracted. Repeating this information gathering procedure at some later time points tj > t0 (= initial time point), j = 1, 2, . . . , adaptive decision/control procedures occur [101]. Based on the inherent random nature of the parameter vector a, the observation or measurement mechanism, resp., or adopting a Bayesian approach concerning unknown parameter values [10], here we make the following basic assumption: Stochastic Uncertainty: The unknown parameter vector a is a realization a = a(ω), ω ∈ Ω,

(1.3)

of a random ν-vector a(ω) on a certain probability space (Ω, A0 , P), where the probability distribution Pa(·) of a(ω) is known, or it is known that Pa(·) lies within a given range W of probability measures on IRν . Using a Bayesian approach, the probability distribution Pa(·) of a(ω) may also describe the subjective or personal probability of the decision maker, the designer. Hence, in order to take into account the stochastic variations of the parameter vector a, to incorporate the a priori and/or sample information about the unknown vector a, resp., the standard approach “insert a certain nominal parameter vector a0 , and correct then the resulting error” must be replaced by a more appropriate deterministic substitute problem for the basic optimization problem (1.1a)–(1.1d) under stochastic uncertainty.

1.2 Deterministic Substitute Problems: Basic Formulation The proper selection of a deterministic substitute problem is a decision theoretical task, see [78]. Hence, for (1.1a)–(1.1d) we have ﬁrst to consider the outcome map T e = e(a, x) := f0 (a, x), f1 (a, x), . . . , fmf (a, x), g1 (a, x), . . . , gmg (a, x) , (1.4a) a ∈ IRν , x ∈ IRr (x ∈ D0 ), and to evaluate then the outcomes e ∈ E ⊂ IR1+mf +mg by means of certain loss or cost functions γi : E → IR, i = 0, 1, . . . , m,

(1.4b)

with an integer m ≥ 0. For the processing of the numerical outcomes γi e(a, x) , i = 0, 1, . . . , m, there are two basic concepts:

6

1 Decision/Control Under Stochastic Uncertainty

1.2.1 Minimum or Bounded Expected Costs Consider the vector of (conditional) expected losses or costs ⎛ ⎞ ⎞ ⎛ F0 (x) Eγ0 (e(a(ω), x)) ⎜ F1 (x) ⎟ ⎜ Eγ1 (e(a(ω), x)) ⎟ ⎜ ⎟ ⎟ ⎜ F(x) = ⎜ . ⎟ := ⎜ ⎟ , x ∈ IRr , .. ⎝ .. ⎠ ⎠ ⎝ . Fm (x)

(1.5)

Eγm (e(a(ω), x))

where the (conditional) expectation “E” is taken with respect to the time history A = At , (Aj ) ⊂ A0 up to a certain time point t or stage j. A short deﬁnition of expectations is given in Sect. 2.1, for more details, see, e.g., [7, 43, 124]. Having diﬀerent expected cost or performance functions F0 , F1 , . . . , Fm to be minimized or bounded, as a basic deterministic substitute problem for (1.1a)–(1.1d) with a random parameter vector a = a(ω) we may consider the multi-objective expected cost minimization problem “min”F(x)

(1.6a)

x ∈ D0 .

(1.6b)

s.t.

Obviously, a good compromise solution x∗ of this vector optimization problem should have at least one of the following properties [25, 134]: Deﬁnition 1.1 (a) A vector x0 ∈ D0 is called a functional–eﬃcient or Pareto optimal solution of the vector optimization problem (1.6a), (1.6b) if there is no x ∈ D0 such that Fi (x) ≤ Fi (x0 ), i = 0, 1, . . . , m

(1.7a)

Fi0 (x) < Fi0 (x0 ) for at least one i0 , 0 ≤ i0 ≤ m.

(1.7b)

and (b) A vector x0 ∈ D0 is called a weak functional–eﬃcient or weak Pareto optimal solution of (1.6a), (1.6b) if there is no x ∈ D0 such that Fi (x) < Fi (x0 ), i = 0, 1, . . . , m

(1.8)

(Weak) Pareto optimal solutions of (1.6a), (1.6b) may be obtained now by means of scalarizations of the vector optimization problem (1.6a), (1.6b). Three main versions are stated in the following: (1) Minimization of primary expected cost/loss under expected cost constraints (1.9a) min F0 (x)

1.2 Deterministic Substitute Problems: Basic Formulation

7

s.t. Fi (x) ≤ Fimax , i = 1, . . . , m x ∈ D0 .

(1.9b) (1.9c)

Here, F0 = F0 (x) is assumed to describe the primary goal of the design/decision making problem, while Fi = Fi (x), i = 1, . . . , m, describe secondary goals. Moreover, Fimax , i = 1, . . . , m, denote given upper cost/loss bounds. Remark 1.1 An optimal solution x∗ of (1.9a)–(1.9c) is a weak Pareto optimal solution of (1.6a), (1.6b). (2) Minimization of the total weighted expected costs Selecting certain positive weight factors c0 , c1 , . . . , cm , the expected weighted total costs are deﬁned by F˜ (x) :=

m

ci Fi (x) = Ef a(ω), x ,

(1.10a)

i=0

where

m

f (a, x) :=

ci γi e(a, x) .

(1.10b)

i=0

Consequently, minimizing the expected weighted total costs F˜ = F˜ (x) subject to the remaining deterministic constraint (1.1d), the following deterministic substitute problem for (1.1a)–(1.1d) occurs: min

m

ci Fi (x)

(1.11a)

i=0

s.t. x ∈ D0 .

(1.11b)

Remark 1.2 Let ci > 0, i = 1, 1, . . . , m, be any positive weight factors. Then an optimal solution x∗ of (1.11a), (1.11b) is a Pareto optimal solution of (1.6a), (1.6b). (3) Minimization of the maximum weighted expected costs Instead of adding weighted expected costs, we may consider the maximum of the weighted expected costs: (1.12) F˜ (x) := max ci Fi (x) = max ci Eγi e a(ω), x . 0≤i≤m

0≤i≤m

Here c0 , c1 , . . . , m, are again positive weight factors. Thus, minimizing F˜ = F˜ (x) we have the deterministic substitute problem

8

1 Decision/Control Under Stochastic Uncertainty

min max ci Fi (x)

(1.13a)

x ∈ D0 .

(1.13b)

0≤i≤m

s.t. Remark 1.3 Let ci , i = 0, 1, . . . , m, be any positive weight factors. An optimal solution of x∗ of (1.13a), (1.13b) is a weak Pareto optimal solution of (1.6a), (1.6b). 1.2.2 Minimum or Bounded Maximum Costs (Worst Case) Instead of taking expectations, we may consider the worst case with respect to the cost variations caused by the random parameter vector a = a(ω). Hence, the random cost function (1.14a) ω → γi e a(ω), x is evaluated by means of Fisup (x) := ess sup γi e a(ω), x , i = 0, 1, . . . , m.

(1.14b)

Here, ess sup (. . .) denotes the (conditional) essential supremum with respect to the random vector a = a(ω), given information A, i.e., the inﬁmum of the supremum of (1.14a) on sets A ∈ A0 of (conditional) probability one, see, e.g., [124]. Consequently, the vector function F = Fsup (x) is then deﬁned by ⎞ ⎛ ⎞ ⎛ ess sup γ0 e a(ω), x F0 (x) ⎜ ⎟ ⎟ ⎜ ⎜ F1 (x) ⎟ ⎟ ⎜ e a(ω), x ess sup γ 1 ⎟ ⎜ ⎟. Fsup (x) = ⎜ . ⎟ := ⎜ (1.15) ⎟ ⎜ .. ⎝ .. ⎠ ⎟ ⎜ ⎝ . ⎠ Fm (x) ess sup γm e a(ω), x Working with the vector function F = Fsup (x), we have then the vector minimization problem (1.16a) “min”Fsup (x) s.t. x ∈ D0 .

(1.16b)

By scalarization of (1.16a), (1.16b) we obtain then again deterministic substitute problems for (1.1a)–(1.1d) related to the substitute problem (1.6a), (1.6b) introduced in Sect. 1.2.1. More details for the selection and solution of appropriate deterministic substitute problems for (1.1a)–(1.1d) are given in the next sections.

2 Deterministic Substitute Problems in Optimal Decision Under Stochastic Uncertainty

2.1 Optimum Design Problems with Random Parameters In the optimum design of technical or economic structures/systems two basic classes of criterions appear: First there is a primary cost function G0 = G0 (a, x).

(2.1a)

Important examples are the total weight or volume of a mechanical structure, the costs of construction, design of a certain technical or economic structure/ system, or the negative utility or reward in a general decision situation. For the representation of the structural/system safety or failure, for the representation of the admissibility of the state, or for the formulation of the basic operating conditions of the underlying structure/system certain state functions (2.1b) yi = yi (a, x), i = 1, . . . , my , are chosen. In structural design these functions are also called “limit state functions” or “safety margins.” Frequent examples are some displacement, stress, load (force and moment) components in structural design, or production and cost functions in production planning problems, optimal mix problems, transportation problems, allocation problems and other problems of economic design. In (2.1a), (2.1b), the design or input vector x denotes the r-vector of design or input variables, x1 , x2 , . . . , xr , as, e.g., structural dimensions, sizing variables, such as cross-sectional areas, thickness in structural design, or factors of production, actions in economic design. For the design or input vector x one has mostly some basic deterministic constraints, e.g., nonnegativity constraints, box constraints, represented by x ∈ D,

(2.1c)

10

2 Deterministic Substitute Problems in Optimal Decision

where D is a given convex subset of IRr . Moreover, a is the ν-vector of model parameters. In optimal structural design, p a= (2.1d) P is composed of the following two subvectors: P is the m-vector of the acting external loads, e.g., wave, wind loads, payload, etc. Moreover, p denotes the (ν − m)-vector of the further model parameters, as, e.g., material parameters (like strength parameters, yield/allowable stresses, elastic moduli, plastic capacities, etc.) of each member of the structure, the manufacturing tolerances and the weight or more general cost coeﬃcients. In production planning problems, a = (A, b, c)

(2.1e)

is composed of the m × r matrix A of technological coeﬃcients, the demand m-vector b and the r-vector c of unit costs. Based on the my -vector of state functions T y(a, x) := y1 (a, x), y2 (a, x), . . . , ymy (a, x) ,

(2.1f)

the admissible or safe states of the structure/system can be characterized by the condition y(a, x) ∈ B, (2.1g) where B is a certain subset of IRmy ; B = B(a) may depend also on some model parameters. In production planning problems, typical operating conditions are given, cf. (2.1e), by y(a, x) := Ax − b ≥ 0 or y(a, x) = 0, x ≥ 0.

(2.2a)

In mechanical structures/structural systems, the safety (survival) of the structure/system is described by the operating conditions yi (a, x) > 0 for all i = 1, . . . , my

(2.2b)

with state functions yi = yi (a, x), i = 1, . . . , my , depending on certain response components of the structure/system, such as displacement, stress, force, moment components. Hence, a failure occurs if and only of the structure/system is in the ith failure mode (failure domain) yi (a, x) ≤ 0

(2.2c)

for at least one index i, 1 ≤ i ≤ my . Note. The number my of safety margins or limit state functions yi = yi (a, x), i = 1, . . . , my , may be very large. E.g., in optimal plastic

2.1 Optimum Design Problems with Random Parameters

11

design the limit state functions are determined by the extreme points of the admissible domain of the dual pair of static/kinematic LPs related to the equilibrium and linearized convex yield condition, see Sect. 7. Basic problems in optimal decision/design are: (1) Primary (construction, planning, investment, etc.) cost minimization under operating or safety conditions min G0 (a, x)

(2.3a)

y(a, x) ∈ B

(2.3b)

s.t. x ∈ D.

(2.3c)

Obviously we have B = (0, +∞)my in (2.2b) and B = [0, +∞)my or B = {0} in (2.2a). (2) Failure or recourse cost minimization under primary cost constraints “min”γ y(a, x) (2.4a) s.t. G0 (a, x) ≤ Gmax x ∈ D.

(2.4b) (2.4c)

In (2.4a) γ = γ(y) is a scalar or vector valued cost/loss function evaluating violations of the operating conditions (2.3b). Depending on the application, these costs are called “failure” or “recourse” costs [58, 59, 98, 120, 137, 138]. As already discussed in Sect. 1, solving problems of the above type, a basic diﬃculty is the uncertainty about the true value of the vector a of model parameters or the (random) variability of a: In practice, due to several types of uncertainties such as, see [154]: –

Physical uncertainty (variability of physical quantities, like material, loads, dimensions, etc.) – Economic uncertainty (trade, demand, costs, etc.) – Statistical uncertainty (e.g., estimation errors of parameters due to limited sample data) – Model uncertainty (model errors) the ν-vector a of model parameters must be modeled by a random vector a = a(ω), ω ∈ Ω,

(2.5a)

on a certain probability space (Ω, A0 , P) with sample space Ω having elements ω, see (1.3). For the mathematical representation of the corresponding (conA of the random vector a = a(ω) ditional) probability distribution Pa(·) = Pa(·)

12

2 Deterministic Substitute Problems in Optimal Decision

(given the time history or information A ⊂ A0 ), two main distribution models are taken into account in practice: (1) Discrete probability distributions (2) Continuous probability distributions In the ﬁrst case there is a ﬁnite or countably inﬁnite number l0 ∈ IN ∪ {∞} of realizations or scenarios al ∈ IRν , l = 1, . . . , l0 , (2.5b) P a(ω) = al = αl , l = 1, . . . , l0 , taken with probabilities αl , l = 1, . . . , l0 . In the second case, the probability that the realization a(ω) = a lies in a certain (measurable) subset B ⊂ IRν is described by the multiple integral (2.5c) P a(ω) ∈ B = ϕ(a) da B

with a certain probability density function ϕ = ϕ(a) ≥ 0, a ∈ IR , ν

ϕ(a)da = 1.

The properties of the probability distribution Pa(·) may be described – fully or in part – by certain numerical characteristics, called parameters of Pa(·) . These distribution parameters θ = θh are obtained by considering expectations θh := Eh a(ω) (2.6a) of (measurable) functions (h ◦ a)(ω) := h a(ω)

(2.6b)

composed of the random vector a = a(ω) with certain (measurable) mappings h : IRν −→ IRsh , sh ≥ 1.

(2.6c)

According to the type of the probability distribution Pa(·) of a = a(ω), the expectation Eh a(ω) is deﬁned by ⎧ l0 ⎪ ⎪ ⎪ h al αl , in the discrete case (2.5b) ⎪ ⎨ Eh a(ω) = l=1 ⎪ ⎪ h(a)ϕ(a) da, in the continuous case (2.5c). ⎪ ⎪ ⎩

(2.6d)

IRν

Further distribution parameters θ are functions θ = Ψ (θh1 , . . . , θhs )

(2.7)

2.1 Optimum Design Problems with Random Parameters

13

of certain “h-moments” θh1 , . . . , θhs of the type (2.6a). Important examples of the type (2.6a), (2.7), resp., are the expectation a = Ea(ω) (for h(a) := a, a ∈ IRν ) and the covariance matrix T Q := E a(ω) − a a(ω) − a = Ea(ω)a(ω)T − a aT

(2.8a)

(2.8b)

of the random vector a = a(ω). Due to the stochastic variability of the random vector a(·) of model parameters, and since the realization a(ω) = a is not available at the decision making stage, the optimal design problem (2.3a)–(2.3c) or (2.4a)–(2.4c) under stochastic uncertainty cannot be solved directly. Hence, appropriate deterministic substitute problems must be chosen taking into account the randomness of a = a(ω), cf. Sect. 1.2. 2.1.1 Deterministic Substitute Problems in Optimal Design According to Sect. 1.2, a basic deterministic substitute problem in optimal design under stochastic uncertainty is the minimization of the total expected costs including the expected costs of failure (2.9a) min cG EG0 a(ω), x + cf pf (x) s.t. Here,

x ∈ D.

(2.9b)

pf = pf (x) := P y a(ω), x ∈B |

(2.9c)

is the probability of failure or the probability that a safe function of the structure, the system is not guaranteed. Furthermore, cG is a certain weight factor, and cf > 0 describes the failure or recourse costs. In the present deﬁnition of expected failure costs, constant costs for each realization a = a(ω) of a(·) are assumed. Obviously, it is pf (x) = 1 − ps (x)

(2.9d)

with the probability of safety or survival ps (x) := P y a(ω), x ∈ B .

(2.9e)

In case (2.2b) we have pf (x) = P yi a(ω), x ≤ 0 for at least one index i, 1 ≤ i ≤ my .

(2.9f)

14

2 Deterministic Substitute Problems in Optimal Decision

The objective function (2.9a) may be interpreted as the Lagrangian (with given cost multiplier cf ) of the following reliability based optimization (RBO) problem, cf. [2, 28, 88, 91, 99, 120, 138, 154, 162]: (2.10a) min EG0 a(ω), x s.t. pf (x) ≤ αmax x ∈ D,

(2.10b) (2.10c)

where αmax > 0 is a prescribed maximum failure probability, e.g., αmax = 0.001, cf. (2.3a)–(2.3c). The “dual” version of (2.10a)–(2.10c) reads

s.t.

min pf (x)

(2.11a)

EG0 a(ω), x ≤ Gmax

(2.11b)

x∈D

(2.11c)

with a maximum (upper) cost bound Gmax , see (2.4a)–(2.4c). Further substitute problems are obtained by considering more general expected failure or recourse cost functions Γ (x) = Eγ y a(ω), x (2.12a) arising from structural systems weakness or failure, or because of false operation. Here, T (2.12b) y a(ω), x := y1 a(ω), x , . . . , ymy a(ω), x is again the random vector of state functions, and γ : IRmy → IRmγ

(2.12c)

is a scalar or vector valued cost or loss function. In case B = (0, +∞)my or B = [0, +∞)my it is often assumed that γ = γ(y) is a nonincreasing function, hence, γ(y) ≥ γ(z), if y ≤ z, (2.12d) where inequalities between vectors are deﬁned componentwise. Example 2.1. If γ(y) = 1 for y ∈ B c (complement of B) and γ(y) = 0 for y ∈ B, then Γ (x) = pf (x).

2.1 Optimum Design Problems with Random Parameters

15

Example 2.2. Suppose that γ = γ(y) is a nonnegative scalar function on IRmy such that | (2.13a) γ(y) ≥ γ0 > 0 for all y ∈B with a constant γ0 > 0. Then for the probability of failure we ﬁnd the following upper bound 1 | ≤ Eγ y a(ω), x , pf (x) = P y a(ω), x ∈B γ0

(2.13b)

where the right hand side of (2.13b) is obviously an expected cost function of type (2.12a)–(2.12c). Hence, the condition (2.10b) can be guaranteed by the cost constraint (2.13c) Eγ y a(ω), x ≤ γ0 αmax . Example 2.3. If the loss function γ(y) is deﬁned by a vector of individual loss functions γi for each state function yi = yi (a, x), i = 1, . . . , my , hence, T γ(y) = γ1 (y1 ), . . . , γmy (ymy ) ,

(2.14a)

then

T Γ (x) = Γ1 (x), . . . , Γmy (x) , Γi (x) := Eγi yi a(ω), x , 1 ≤ i ≤ my , (2.14b)

i.e., the my state functions yi , i = 1, . . . , my , will be treated separately. Working with the more general expected failure or recourse cost functions Γ = Γ (x), instead of (2.9a)–(2.9c), (2.10a)–(2.10c) and (2.11a)–(2.11c) we have the related substitute problems: (1) Expected total cost minimization min cG EG0 a(ω), x + cTf Γ (x),

(2.15a)

s.t. x∈D

(2.15b)

(2) Expected primary cost minimization under expected failure or recourse cost constraints (2.16a) min EG0 a(ω), x s.t. Γ (x) ≤ Γ max

(2.16b)

x ∈ D,

(2.16c)

16

2 Deterministic Substitute Problems in Optimal Decision

(3) Expected failure or recourse cost minimization under expected primary cost constraints “min”Γ (x) (2.17a) s.t. EG0 a(ω), x ≤ Gmax (2.17b) x ∈ D.

(2.17c)

Here, cG , cf are (vectorial) weight coeﬃcients, Γ max is the vector of upper loss bounds, and “min” indicates again that Γ (x) may be a vector valued function. 2.1.2 Deterministic Substitute Problems in Quality Engineering In quality engineering [109,113,114] the aim is to optimize the performance of a certain product or a certain process, and to make the performance insensitive (robust) to any noise to which the product, the process may be subjected. Moreover, the production costs (development, manufacturing, etc.) should be bounded or minimized. Hence, the control (input, design) factors x are then to set as to provide – –

An optimal mean performance (response) A high degree of immunity (robustness) of the performance (response) to noise factors – Low or minimum production costs Of course, a certain trade-oﬀ between these criterions may be taken into account. For this aim, the quality characteristics or performance (e.g., dimensions, weight, contamination, structural strength, yield, etc.) of the product/process is described [109] ﬁrst by a function y = y a(ω), x (2.18) depending on a ν-vector a = a(ω) of random varying parameters (noise factors) aj = aj (ω), j = 1, . . . , ν, causing quality variations, and the r-vector x of control/input factors xk , k = 1, . . . , r. The quality loss of the product/process under consideration is measured then by a certain loss function γ = γ(y). Simple loss functions γ are often chosen [113, 114] in the following way: (1) Nominal-is-best Having a given, ﬁnite target or nominal value y ∗ , e.g., a nominal dimension of a product, the quality loss γ(y) reads γ(y) := (y − y ∗ )2 .

(2.19a)

2.1 Optimum Design Problems with Random Parameters

17

(2) Smallest-is-best If the target or nominal value y ∗ is zero, i.e., if the absolute value |y| of the product quality function y = y(a, x) should be as small as possible, e.g., a certain product contamination, then the corresponding quality loss is deﬁned by (2.19b) γ(y) := y 2 . (3) Biggest-is-best If the absolute value |y| of the product quality function y = y(a, x) should be as large as possible, e.g., the strength of a bar or the yield of a process, then a possible quality loss is given by γ(y) :=

2 1 . y

(2.19c)

Obviously, (2.19a) and (2.19b) are convex loss functions. Moreover, since the quality or response function y = y(a, x) takes only positive or only negative values y in many practical applications, also (2.19c) yields a convex loss function in many cases. Further decision situations may be modeled by choosing more general convex loss functions γ = γ(y). A high mean product/process level or performance at minimum random quality variations and low or bounded production/manufacturing costs is achieved then [109, 113, 114] by minimization of the expected quality loss function Γ (x) := Eγ y a(ω), x

,

(2.19d)

subject to certain constraints for the input x, as, e.g., constraints for the production/manufacturing costs. Obviously, Γ (x) can also be minimized by maximizing the negative logarithmic mean loss (sometimes called “signal-tonoise-ratio” [109]): SN := − log Γ (x).

(2.19e)

Since Γ = Γ (x) is a mean value function as described in the previous sections, for the computation of a robust optimal input vector x∗ one has the following stochastic optimization problem

s.t.

min Γ (x)

(2.20a)

EG0 a(ω), x ≤ Gmax

(2.20b)

x ∈ D.

(2.20c)

18

2 Deterministic Substitute Problems in Optimal Decision

2.2 Basic Properties of Substitute Problems As can be seen from the conversion of an optimization problem with random parameters into a deterministic substitute problem, cf. Sects. 2.1.1 and 2.1.2, a central role is played by expectation or mean value functions of the type (2.21a) Γ (x) = Eγ y a(ω), x , x ∈ D0 , or more general

Γ (x) = Eg a(ω), x , x ∈ D0 .

(2.21b)

Here, a = a(ω) is a random ν-vector, y = y(a, x) is an my -vector valued function on a certain subset of IRν × IRr , and γ = γ(z) is a real valued function on a certain subset of IRmy . Furthermore, g = g(a, x) denotes a real valued function on a certain subset of IRν ×IRr . In the following we suppose that the expectation in (2.21a), (2.21b) exists and is ﬁnite for all input vectors x lying in an appropriate set D0 ⊂ IRr , cf. [11]. The following basic properties of the mean value functions Γ are needed in the following again and again. Lemma 2.1. Convexity. Suppose that x → g a(ω), x is convex a.s. (almost sure) on a ﬁxed convex domain D0 ⊂ IRr . If Eg a(ω), x exists and is ﬁnite for each x ∈ D0 , then Γ = Γ (x) is convex on D0 . Proof. This property follows [58, 59, 78] directly from the linearity of the expectation operator. If g = g(a, x) is deﬁned by g(a, x) := γ y(a, x) , see (2.21a), then the above theorem yields the following result:

Corollary 2.1. Suppose that γ is convex and Eγ y a(ω), x exists and is ﬁnite for each x ∈ D0 . (a) If x → y a(ω), x is linear a.s., then Γ = Γ (x) is convex. (b) If x → y a(ω), x is convex a.s., and γ is a convex, monotonous nondecreasing function, then Γ = Γ (x) is convex. It is well known [72] that a convex function is continuous on each open subset of its domain. A general suﬃcient condition for the continuity of Γ is given next. Lemma 2.2. Continuity. Suppose that Eg a(ω), x exists and is ﬁnite for each x ∈ D0 , and assume that x → g a(ω), x is continuous at x0 ∈ D0 a.s. If there is a function ψ = ψ a(ω) having ﬁnite expectation such that

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

g a(ω), x ≤ ψ a(ω) a.s. for all x ∈ U (x0 ) ∩ D0 ,

19

(2.22)

where U (x0 ) is a neighborhood of x0 , then Γ = Γ (x) is continuous at x0 . Proof. The assertion can be shown by using Lebesgue’s dominated convergence theorem, see, e.g., [78]. For the consideration of the diﬀerentiability of Γ = Γ (x), let D denote an

open subset of the domain D0 of Γ . Lemma 2.3. Diﬀerentiability. Suppose that (1) Eg a(ω), x exists and is ﬁnite for each x ∈ D0 , (2) x → g a(ω), x is diﬀerentiable on the open subset D of D0 a.s. and (3) (2.23a) ∇x g a(ω), x ≤ ψ a(ω) , x ∈ D, a.s., where ψ = ψ a(ω) is a function having ﬁnite expectation. Then the expec tation of ∇x g a(ω), x exists and is ﬁnite, Γ = Γ (x) is diﬀerentiable on D and ∇Γ (x) = ∇x Eg a(ω), x = E∇x g a(ω), x , x ∈ D. (2.23b) ∆Γ , k = 1, . . . , r, of Γ at a ∆xk ﬁxed point x0 ∈ D, the assertion follows by means of the mean value theorem, inequality (2.23a) and Lebesgue’s dominated convergence theorem, cf. [58, 59, 78].

Proof. Considering the diﬀerence quotients

Example 2.4. In case (2.21a), under obvious diﬀerentiability assumptions conT cerning γ and y we have ∇x g(a, x) = ∇x y(a, x) ∇γ y(a, x) , where ∇x y(a, x) denotes the Jacobian of y = y(a, x) with respect to a. Hence, if (2.23b) holds, then T (2.23c) ∇Γ (x) = E∇x y a(ω), x ∇γ y a(ω), x .

2.3 Approximations of Deterministic Substitute Problems in Optimal Design The main problem in solving the deterministic substitute problems deﬁned above is that the arising probability and expected cost functions pf = pf (x), Γ = Γ (x), x ∈ IRr , are deﬁned by means of multiple integrals over a ν-dimensional space. Thus, the substitute problems may be solved, in practice, only by some approximative analytical and numerical methods [37, 165]. In the following

20

2 Deterministic Substitute Problems in Optimal Decision

we consider possible approximations for substitute problems based on general expected recourse cost functions Γ = Γ (x) according to (2.21a) having a real valued convex loss function γ(z). Note that the probability of failure function pf = pf (x) may be approximated from above, see (2.13a), (2.13b), by expected cost functions Γ = Γ (x) having a nonnegative function γ = γ(z) being bounded from below on the failure domain B c . In the following several basic approximation methods are presented. 2.3.1 Approximation of the Loss Function Suppose here that γ = γ(y) is a continuously diﬀerentiable, convex loss function on IRmy . Let then denote T y(x) := Ey a(ω), x = Ey1 a(ω), x , . . . , Eymy a(ω), x (2.24) the expectation of the vector y = y a(ω), x of state functions yi = yi a(ω), x , i = 1, . . . , my . For an arbitrary continuously diﬀerentiable, convex loss function γ we have T y a(ω), x − y(x) . (2.25a) γ y a(ω), x ≥ γ y(x) + ∇γ y(x) Thus, taking expectations in (2.25a), we ﬁnd Jensen’s inequality (2.25b) Γ (x) = Eγ y a(ω), x ≥ γ y(x) which holds for any convex function γ. Using the mean value theorem, we have y )T (y − y), (2.25c) γ(y) = γ(y) + ∇γ(ˆ where yˆ is a point on the line segment yy between y and y. By means of (2.25b), (2.25c) we get 0 ≤ Γ (x) − γ y(x) ≤ E ∇γ yˆ a(ω), x · y a(ω), x − y(x) . (2.25d) (a) Bounded gradient If the gradient ∇γ is bounded on the range of y = y a(ω), x , x ∈ D, i.e., if (2.26a) ∇γ y a(ω), x ≤ ϑmax a.s. for each x ∈ D, with a constant ϑmax > 0, then 0 ≤ Γ (x) − γ y(x) ≤ ϑmax E y a(ω), x − y(x) , x ∈ D.

(2.26b)

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

√ Since t → t, t ≥ 0, is a concave function, we get 0 ≤ Γ (x) − γ y(x) ≤ ϑmax q(x), where

2 q(x) := E y a(ω), x − y(x) = trQ(x)

21

(2.26c)

(2.26d)

is the generalized variance, and

Q(x) := cov y a(·), x

(2.26e)

denotes the covariance matrix of the random vector y = y a(ω), x . Consequently, the expected loss function Γ (x) can be approximated from above by Γ (x) ≤ γ y(x) + ϑmax q(x) for x ∈ D. (2.26f) (b) Bounded eigenvalues of the Hessian Considering second order expansions of γ, with a vector y˜ ∈ y¯y we ﬁnd 1 y )(y − y). γ(y) − γ(y) = ∇γ(y)T (y − y) + (y − y)T ∇2 γ(˜ 2

(2.27a)

Suppose that the eigenvalues λ of ∇2 γ(y) are bounded from below and above on the range of y = y a(ω), x for each x ∈ D, i.e., ≤ λmax < +∞, a.s., x ∈ D (2.27b) 0 < λmin ≤ λ ∇2 γ y a(ω), x with constants 0 < λmin ≤ λmax . Taking expectations in (2.27a), we get λmin λmax q(x) ≤ Γ (x) ≤ γ y(x) + q(x), x ∈ D. (2.27c) γ y(x) + 2 2 Consequently, using (2.26f) or (2.27c), various approximations for the deterministic substitute problems (2.15a), (2.15b), (2.16a)–(2.16c), (2.17a)– (2.17c) may be obtained. Based on the above approximations of expected cost functions, we state the following two approximates to (2.16a)–(2.16c), (2.17a)–(2.17c), resp., which are well known in robust optimal design: (1) Expected primary cost minimization under approximate expected failure or recourse cost constraints (2.28a) min EG0 a(ω), x s.t.

γ y(x) + c0 q(x) ≤ Γ max x ∈ D,

where c0 is a scale factor, cf. (2.26f) and (2.27c);

(2.28b) (2.28c)

22

2 Deterministic Substitute Problems in Optimal Decision

(2) Approximate expected failure or recourse cost minimization under expected primary cost constraints (2.29a) min γ y(x) + c0 q(x) s.t.

EG0 a(ω), x ≤ Gmax x ∈ D.

(2.29b) (2.29c)

Obviously, by means of (2.28a)–(2.28c) or (2.29a)–(2.29c) optimal designs x∗ are achieved which – – –

Yield a high mean performance of the structure/structural system Are minimally sensitive or have a limited sensitivity with respect to random parameter variations (material, load, manufacturing, process, etc.) Cause only limited costs for design, construction, maintenance, etc.

2.3.2 Regression Techniques, Model Fitting, RSM In structures/systems with variable quantities, very often one or several functional relationships between the measurable variables can be guessed according to the available data resulting from observations, simulation experiments, etc. Due to limited data, the true functional relationships between the quantities involved are approximated in practice – on a certain limited range – usually by simple functions, as, e.g., multivariate polynomials of a certain (low) degree. The resulting regression metamodel [64] is obtained then by certain ﬁtting techniques, such as weighted least squares or Tschebyscheﬀ minmax procedures for estimating the unknown parameters of the regression model. An important advantage of these models is then the possibility to predict the values of some variables from knowledge of other variables. In case of more than one independent variable (predictor or regressor) one has a “multiple regression” task. Problems with more than one dependent or response variable are called “multivariate regression” problems, see [33]. (a) Approximation of state functions. The numerical solution is simpliﬁed considerably if one can work with one single state function y = y(a, x). Formally, this is possible by deﬁning the function y min (a, x) := min yi (a, x). 1≤i≤my

(2.30a)

Indeed, according to (2.2b), (2.2c) the failure of the structure, the system can be represented by the condition y min (a, x) ≤ 0.

(2.30b)

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

23

Fig. 2.1. Loss function γ

Thus, the weakness or failure of the technical or economic device can be evaluated numerically by the function (2.30c) Γ (x) := Eγ y min a(ω), x with a nonincreasing loss function γ : IR → IR+ , see Fig. 2.1. However, the “min”-operator in (2.30a) yields a nonsmooth function y min = y min (a, x) in general, and the computation of the mean and variance function (2.30d) y min (x) := Ey min a(ω), x (2.30e) σy2min (x) := V y min a(·), x by means of Taylor expansion with respect to the model parameter vector a at a = Ea(ω) is not possible directly, cf. Sect. 2.3.3. According to the deﬁnition (2.30a), an upper bound for y min (x) is given by y min (x) ≤ min y i (x) = min Eyi a(ω), x . 1≤i≤my

Further approximations of y using the representation

min

1≤i≤my

(a, x) and its moments can be found by

1 a + b − |a − b| 2 of the minimum of two numbers a, b ∈ IR. E.g., for an even index my we have min min yi (a, x), yi+1 (a, x) y min (a, x) = min(a, b) =

i=1,3,...,my −1

=

1 yi (a, x) + yi+1 (a, x)− yi (a, x) − yi+1 (a, x) . i=1,3,...,my −1 2 min

24

2 Deterministic Substitute Problems in Optimal Decision

In many cases we may suppose that the state functions yi = yi (a, x), i = 1, . . . , my , are bounded from below, hence, yi (a, x) > −A, i = 1, . . . , my , for all (a, x) under consideration with a positive constant A > 0. Thus, deﬁning y˜i (a, x) := yi (a, x) + A, i = 1, . . . , my , and therefore y˜min (a, x) := min y˜i (a, x) = y min (a, x) + A, 1≤i≤my

we have y min (a, x) ≤ 0 if and only if y˜min (a, x) ≤ A. Hence, the survival/failure of the system or structure can be studied also by means of the positive function y˜min = y˜min (a, x). Using now the theory of power or H¨older means [24], the minimum y˜min (a, x) of positive functions can be represented also by the limit min

y˜

(a, x) = lim

λ→−∞

my 1 y˜i (a, x)λ my i=1

1/λ

of the decreasing family of power means M

[λ]

(˜ y ) :=

my 1 λ y˜ my i=1 i

1/λ ,

λ < 0. Consequently, for each ﬁxed p > 0 we also have min

y˜

p

(a, x) = lim

λ→−∞

my 1 y˜i (a, x)λ my i=1

p/λ .

p Assuming that the expectation EM [λ] y˜ a(ω) , x exists for an exponent λ = λ0 < 0, by means of Lebesgue’s bounded convergence theorem we get the moment representation min

E y˜

p a(ω), x = lim E λ→−∞

my λ 1 y˜i a(ω), x my i=1

p/λ .

Since t → tp/λ , t > 0, is convex for each ﬁxed p > 0 and λ < 0, by Jensen’s inequality we have the lower moment bound p E y˜min a(ω), x ≥ lim

λ→−∞

my λ 1 E y˜i a(ω), x my i=1

p/λ .

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

25

a(·), x we get the approxima-

Hence, for the pth order moment of y˜min tions my my p/λ λ p/λ 1 1 y˜i a(ω), x ≥ E y˜i a(ω), x E my i=1 my i=1 for some λ < 0.

Using regression techniques, Response Surface Methods (RSM), etc., for given vector x, the function a → y min (a, x) can be approximated [14, 23, 55,63,136] by functions y = y(a, x) being suﬃciently smooth with respect to the parameter vector a (Figs. 2.2 and 2.3). In many important cases, for each i = 1, . . . , my , the state functions (a, x) −→ yi (a, x) are bilinear functions. Thus, in this case y min = y min (a, x) is a piecewise linear function with respect to a. Fitting a linear or quadratic Response Surface Model [17, 18, 105, 106] y(a, x) := c(x) + q(x)T (a − a) + (a − a)T Q(x)(a − a)

(2.30f)

to a → y min (a, x), after the selection of appropriate reference points a(j) := a + da(j) , j = 1, . . . , p,

(2.30g)

Fig. 2.2. Approximate smooth failure domain [y(a, x) ≤ 0] for given x

26

2 Deterministic Substitute Problems in Optimal Decision

Fig. 2.3. Approximation y˜(a, x) of y min (a, x) for given x (j)

with “design” points da ∈ IRν , j = 1, . . . , p, the unknown coeﬃcients c = c(x), q = q(x) and Q = Q(x) are obtained by minimizing the mean square error p 2 y(a(j) , x) − y min (a(j) , x) (2.30h) ρ(c, q, Q) := j=1

with respect to (c, q, Q). Since the model (2.30f) depends linearly on the function parameters (c, q, Q), explicit formulas for the optimal coeﬃcients c∗ = c∗ (x), q ∗ = q ∗ (x), Q∗ = Q∗ (x),

(2.30i)

are obtained from this least squares estimation method, see Chap. 5. (b) Approximation of expected loss functions. Corresponding to the approximation (2.30f) of y min = y min (a, x),using again least squares techniques, a mean value function Γ (x) = Eγ y a(ω), x , cf. (2.12a), can be approximated at a given point x0 ∈ IRν by a linear or quadratic Response Surface Function Γ(x) := β0 + βIT (x − x0 ) + (x − x0 )T B(x − x0 ),

(2.30j)

with scalar, vector and matrix parameters β0 , βI , B. In this case estimates y (i) = Γˆ (i) of Γ (x) are needed at some reference points x(i) = x0 + d(i) , i = 1, . . . , p. Details are given in Chap. 5. 2.3.3 Taylor Expansion Methods As can be seen above, cf. (2.21a), (2.21b), in the objective and/or in the constraints of substitute problems for optimization problems with random data mean value functions of the type

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

27

Γ (x) := Eg a(ω), x occur. Here, g = g(a, x) is a real valued function on a subset of IRν × IRr , and a = a(ω) is a ν-random vector. (a) Expansion with respect to a: Suppose that on its domain the function g = g(a, x) has partial derivatives ∇la g(a, x), l = 0, 1, . . . , lg +1, up to order lg + 1. Note that the gradient ∇a g(a, x) contains the so-called sensitivities ∂g (a, x), j = 1, . . . , ν, of g with respect to the parameter vector a at ∂aj (a, x). In the same way, the higher order partial derivatives ∇la g(a, x), l > 1, represent the higher order sensitivities of g with respect to a at (a, x). Taylor expansion of g = g(a, x) with respect to a at a := Ea(ω) yields lg 1 1 l ∇a g(¯ ∇lg+1 g(ˆ a, x) · (a − a ¯)l + a, x) · (a − a ¯)lg +1 g(a, x) = l! (lg + 1)! a l=0

(2.31a) where a ˆ := a ¯ + ϑ(a − a ¯), 0 < ϑ < 1, and (a − a ¯)l denotes the system of lth order products ν (aj − a ¯j )lj j=1

with lj ∈ IN ∪ {0}, j = 1, . . . , ν, l1 + l2 + . . . + lν = l. If g = g(a, x) is deﬁned by g(a, x) := γ y(a, x) , see (2.21a), then the partial derivatives ∇la g of g up to the second order read: T (2.31b) ∇a g(a, x) = ∇a y(a, x) ∇γ y(a, x) T ∇2a g(a, x) = ∇a y(a, x) ∇2 γ y(a, x) ∇a y(a, x) +∇γ y(a, x) · ∇2a y(a, x), where (∇γ) ·

∇2a y

:= (∇γ)T

∂2y ∂ak ∂al

(2.31c)

·

(2.31d)

k,l=1,...,ν

Taking expectations in (2.31a), Γ (x) can be approximated, cf. Sect. 2.3.1, by lg l ∇la g(¯ a, x) · E a(ω) − a ¯ , (2.32a) Γ (x) := g(¯ a, x) + l=2

28

2 Deterministic Substitute Problems in Optimal Decision

l where E a(ω) − a ¯ denotes the system of mixed lth central moments T of the random vector a(ω) = a1 (ω), . . . , aν (ω) . Assuming that the domain of g = g(a, x) is convex with respect to a, we get the error estimate Γ (x) − Γ(x) ≤

1 E sup ∇alg +1 g a ¯ + ϑ a(ω) − a ¯ ,x (lg + 1)! 0≤ϑ≤1 lg +1 . (2.32b) ×a(ω) − a ¯

In many practical cases the random parameter ν-vector a = a(ω) has a l +1 convex, bounded support, and ∇ag g is continuous. Then the L∞ -norm 1 ess sup ∇alg +1 g a(ω), x (2.32c) r(x) := (lg + 1)! is ﬁnite for all x under consideration, and (2.32b), (2.32c) yield the error bound lg +1 . (2.32d) ¯ Γ (x) − Γ(x) ≤ r(x)E a(ω) − a Remark 2.1. The above described method can be extended T to the case of vector valued loss functions γ(z) = γ1 (z), . . . , γmγ (z) . (b) Inner expansions: Suppose that Γ (x) is deﬁned by (2.21a), hence, Γ (x) = Eγ y a(ω), x . Linearizing the vector function y = y(a, x) with respect to a at a ¯, thus, y(a, x) ≈ y(1) (a, x) := y(¯ a, x) + ∇a y(¯ a, x)(a − a ¯), the mean value function Γ (x) is approximated by Γ(x) := Eγ y(¯ a, x) + ∇a y(¯ a, x) a(ω) − a ¯ .

(2.33a)

(2.33b)

Corresponding to (2.24), (2.26e), deﬁne y (1) (x) := Ey(1) a(ω), x = y(a, x) (2.33c) Q(x) := cov y(1) a(·), x = ∇a y(a, x) cov a(·) ∇a y(a, x)T . (2.33d) In case of convex loss functions γ, approximates of Γ and the corresponding substitute problems based on Γ may be obtained now by applying the methods described in Sect. 2.3.1. Explicit representations for Γ are obtained in case of quadratic loss functions γ.

2.4 Applications to Problems in Quality Engineering

29

Error estimates can be derived easily for Lipschitz-continuous or convex loss function γ. In case of a Lipschitz-continuous loss function γ with Lipschitz constant L > 0, e.g., for sublinear [78, 81] loss functions, using (2.33c) we have (2.33e) Γ (x) − Γ(x) ≤ L · E y a(ω), x − y(1) a(ω), x . Applying the mean value theorem [29], under appropriate second order diﬀerentiability assumptions, for the right hand side of (2.33e) we ﬁnd the following stochastic version of the mean value theorem E y a(ω), x − y(1) a(ω), x 2 ≤ E a(ω) − a sup ∇2a y a + ϑ a(ω) − a , x . (2.33f) 0≤ϑ≤1

2.4 Applications to Problems in Quality Engineering Since the deterministic substitute problems arising in quality engineering are of the same type as in optimal design, the approximation techniques described in Sect. 2.3 can be applied directly to the robust design problem (2.20a)– (2.20c). This is shown in the following for the approximation method described in Sect. 2.3.1. Hence, the approximation techniques developed in Sect. 2.3.1 are applied now to the objective function Γ (x) = Eγ y a(ω), x of the basic robust design problem (2.20a)–(2.20c). If γ = γ(y) in an arbitrary continuously diﬀerentiable convex loss function on IR1 , then we ﬁnd the following approximations: (a) Bounded ﬁrst order derivative In case that γ has a bounded derivative on the convex support of the quality function, i.e., if (2.34a) γ y a(ω), x ≤ ϑmax < +∞ a.s. , x ∈ D, then where

Γ (x) ≤ γ y(x) + ϑmax q(x), x ∈ D,

(2.34b)

y(x) := Ey a(ω), x , (2.34c) 2 (2.34d) q(x) := V y a(·), x = E y a(ω), x − y(x) denote the mean and variance of the quality function y = y a(ω), x .

30

2 Deterministic Substitute Problems in Optimal Decision

(b) Bounded second order derivative If the second order derivative γ of γ is bounded from above and below, i.e., if (2.35a) 0 < λmin ≤ γ y a(ω), x ≤ λmax < +∞ a.s., x ∈ D, then λmin λmax q(x) ≤ Γ (x) ≤ γ y(x) + q(x), x ∈ D. γ y(x) + 2 2

(2.35b)

Corresponding to (2.28a)–(2.28c), (2.29a)–(2.29c), we then have the following “dual” robust optimal design problems: (1) Expected primary cost minimizationunder approximate expected failure or recourse cost constraints (2.36a) min EG0 a(ω), x s.t.

γ y(x) + c0 q(x) ≤ Γ max

(2.36b)

x ∈ D,

(2.36c)

where Γ max denotes an upper loss bound. (2) Approximate expected failure cost minimization under expected primary cost constraints (2.37a) min γ y(x) + c0 q(x) s.t.

EG0 a(ω), x ≤ Gmax x ∈ D,

(2.37b) (2.37c)

scale where c0 is again a certain factor, see (2.34a), (2.34b), (2.35a), (2.35b), and G0 = G0 a(ω), x represents the production costs having an upper bound Gmax .

2.5 Approximation of Probabilities: Probability Inequalities In reliability analysis of engineering/economic structures or systems, a main problem is the computation of probabilities

2.5 Approximation of Probabilities: Probability Inequalities

P

N

:= P

Vi

a(ω) ∈

i=1

⎛

or

P⎝

N

N

31

(2.38a)

Vi

i=1

⎞

⎛

N

Sj ⎠ := P ⎝a(ω) ∈

j=1

⎞ Sj ⎠

(2.38b)

j=1

of unions and intersections of certain failure/survival domains (events) Vj , Sj , j = 1, . . . , N . These domains (events) arise from the representation of the structure or system by a combination of certain series and/or parallel substructures/systems. Due to the high complexity of the basic physical relations, several approximation techniques are needed for the evaluation of (2.38a), (2.38b). 2.5.1 Bonferroni-Type Inequalities In the following V1 , V2 , . . . , VN denote arbitrary (Borel-)measurable subsets of the parameter space IRν , and the abbreviation P(V ) := P a(ω) ∈ V (2.38c) is used for any measurable subset V of IRν . Starting from the representation of the probability of a union of N events, ⎞ ⎛ N N Vj ⎠ = (−1)k−1 sk,N , (2.39a) P⎝ j=1

k=1

where

sk,N :=

1≤i1
P

k

Vil

,

we obtain [45] the well known basic Bonferroni bounds ⎛ ⎞ ρ N P⎝ Vj ⎠ ≤ (−1)k−1 sk,N for ρ ≥ 1, ρ odd j=1

⎛ P⎝

N

j=1

⎞ Vj ⎠ ≥

(2.39b)

l=1

(2.39c)

k=1 ρ

(−1)k−1 sk,N for ρ ≥ 1, ρ even.

(2.39d)

k=1

Besides (2.39c), (2.39d), a large amount of related bounds of diﬀerent complexity are available, cf. [45, 153, 155]. Important bounds of ﬁrst and second degree are given below:

32

2 Deterministic Substitute Problems in Optimal Decision

⎛ max qj ≤ P ⎝

1≤j≤N

⎛ Q1 − Q2 ≤ P ⎝

N j=1 N

⎞ Vj ⎠ ≤ Q1

(2.40a)

⎞ Vj ⎠ ≤ Q1 − max

1≤l≤N

j=1

⎞ ⎛ N Q21 ≤P⎝ Vj ⎠ ≤ Q1 . Q1 + 2Q2 j=1

qil

(2.40b)

i=l

(2.40c)

The above quantities qj , qij , Q1 , Q2 are deﬁned as follows: Q1 :=

N

qj with qj := P(Vj )

(2.40d)

j=1

Q2 :=

j−1 N

qij with qij := P(Vi ∩ Vj ).

(2.40e)

j=2 i=1

Moreover, deﬁning q := (q1 , . . . , qN ), Q := (qij )1≤i,j≤N , we have

⎛ P⎝

N

(2.40f)

⎞ Vj ⎠ ≥ q T Q− q,

(2.40g)

j=1

where Q− denotes the generalized inverse of Q, cf. [155]. 2.5.2 Tschebyscheﬀ-Type Inequalities In many cases the survival or feasible domain (event) S =

m

Si , is represented

i=1

by a certain number m of inequality constraints of the type yli < (≤)yi (a, x) < (≤)yui , i = 1, . . . , m,

(2.41a)

as, e.g., operating conditions, behavioral constraints. Hence, for a ﬁxed input, design or control vector x, the event S = S(x) is given by S := {a ∈ IRν : yli < (≤)yi (a, x) < (≤)yui , i = 1, . . . , m} .

(2.41b)

Here, yi = yi (a, x), i = 1, . . . , m,

(2.41c)

are certain functions, e.g., response, output or performance functions of the structure, system, deﬁned on (a subset of) IRν × IRr . Moreover, yli < yui , i = 1, . . . , m, are lower and upper bounds for the variables yi , i = 1, . . . , m. In case of one-sided constraints some bounds yli , yui are inﬁnite.

2.5 Approximation of Probabilities: Probability Inequalities

33

Two-Sided Constraints If yli < yui , i = 1, . . . , m, are ﬁnite bounds, (2.41a) can be represented by |yi (a, x) − yic | < (≤)ρi , i = 1, . . . , m,

(2.41d)

where the quantities yic , ρi , i = 1, . . . , m, are deﬁned by yic :=

yli + yui yui − yli , ρi := . 2 2

(2.41e)

Consequently, for the probability P(S) of the event S, deﬁned by (2.41b), we have (2.41f) P(S) = P yi a(ω), x − yic < (≤)ρi , i = 1, . . . , m . Introducing the random variables yi a(ω), x − yic , i = 1, . . . , m, y˜i a(ω), x := ρi

(2.42a)

and the set B := {y ∈ IRm : |yi | < (≤)1, i = 1, . . . , m} ,

(2.42b)

with y˜ = (˜ yi )1≤i≤m , we get P(S) = P y˜ a(ω), x ∈ B .

(2.42c)

Considering any (measurable) function ϕ : IRm → IR such that (i) ϕ(y) ≥ 0, y ∈ IRm (ii) ϕ(y) ≥ ϕ0 > 0, if y ∈B, |

(2.42d) (2.42e)

with a positive constant ϕ0 , we ﬁnd the following result: Theorem 2.1. For any (measurable) function ϕ : IRm → IR fulﬁlling conditions (2.42d), (2.42e), the following Tschebyscheﬀ-type inequality holds: P yli < (≤)yi a(ω), x < (≤)yui , i = 1, . . . , m 1 Eϕ y˜ a(ω), x , (2.43) ≥1− ϕ0 provided that the expectation in (2.43) exists and is ﬁnite. denotes the probability distribution of the random mProof. If P y˜ a(·),x vector y˜ = y˜ a(ω), x , then

34

2 Deterministic Substitute Problems in Optimal Decision

Eϕ y˜ a(ω), x = ϕ(y)Py˜(a(·),x) (dy) + y∈B c

y∈B

≥

ϕ(y)Py˜(a(·),x) (dy)

ϕ(y)Py˜(a(·),x) (dy) ≥ ϕ0 y∈B c

Py˜(a(·),x) (dy)

y∈B c

| = ϕ0 1 − P y˜ a(ω), x ∈ B , = ϕ0 P y˜ a(ω), x ∈B which yields the assertion, cf. (2.42c).

Remark 2.2. Note that P(S) ≥ αs with a given minimum reliability αs ∈ (0, 1] can be guaranteed by the expected cost constraint Eϕ y˜ a(ω), x ≤ (1 − αs )ϕ0 . Example 2.5. If ϕ = 1B c is the indicator function of the complement B c of B, then ϕ0 = 1 and (2.43) holds with the equality sign. Example 2.6. For a given positive deﬁnite m × m matrix C, deﬁne ϕ(y) := y T Cy, y ∈ IRm . Then, cf. (2.42b), (2.42d), (2.42e), (2.44a) min ϕ(y) = min min y T Cy, min y T Cy . y∈ |B

1≤i≤m yi ≥1

yi ≤−1

Thus, the lower bound ϕ0 follows by considering the convex optimization problems arising in the right hand side of (2.44a). Moreover, the expectation Eϕ(˜ y ) needed in (2.43) is given, see (2.42a), by Eϕ(˜ y ) = E y˜T C y˜ = EtrC y˜y˜T , T −1 = trC(diag ρ) cov y a(·), x + y(x) − yc y(x) − yc (diag ρ)−1 , (2.44b) where “tr” denotes the trace of a matrix, diag ρ is the diagonal matrix diag ρ := (ρi δij ), yc := (yic ), see (2.41e), and y = y(x) := Eyi a(ω), x . Since Q ≤ trQ ≤ mQ for any positive deﬁnite m × m matrix Q, an upper bound of Eϕ(˜ y ) reads (Fig. 2.4) T Eϕ(˜ y ) ≤ mC (diag ρ)−1 2 cov y a(·), x + y(x) − yc y(x) − yc . (2.44c) Example 2.7. Assuming in the above Example 2.6 that C = diag (cii ) is a diagonal matrix with positive elements cii > 0, i = 1, . . . , m, then min ϕ(y) = min cii > 0, y∈ |B

1≤i≤m

(2.44d)

2.5 Approximation of Probabilities: Probability Inequalities

35

Fig. 2.4. Function ϕ = ϕ(y)

and Eϕ(˜ y ) is given by

Eϕ(˜ y) =

m

cii

i=1

=

m i=1

cii

2 E yi a(ω), x − yic ρ2i 2 σy2i a(·), x + y i (x) − yic ρ2i

.

(2.44e)

One-Sided Inequalities Suppose that exactly one of the two bounds yli < yui is inﬁnite for each i = 1, . . . , m. Multiplying the corresponding constraints in (2.41a) by −1, the admissible domain S = S(x), cf. (2.41b), can be represented always by S(x) = {a ∈ IRν : y˜i (a, x) < (≤) 0, i = 1, . . . , m} ,

(2.45a)

where y˜i := yi − yui , if yli = −∞, and y˜i := yli − yi , if yui = +∞. If we set y˜(a, x) := y˜i (a, x) and

then

˜ := y ∈ IRm : yi < (≤) 0, i = 1, . . . , m , B

(2.45b)

˜ . P S(x) = P y˜ a(ω), x ∈ B

(2.45c)

Consider also in this case, cf. (2.42d), (2.42e), a function ϕ : IRm → IR such that

36

2 Deterministic Substitute Problems in Optimal Decision

(i) ϕ(y) ≥ 0, y ∈ IRm ˜ (ii)ϕ(y) ≥ ϕ0 > 0, if y ∈| B.

(2.46a) (2.46b)

Then, corresponding to Theorem 2.1, we have this result: Theorem 2.2 (Markov-type inequality). If ϕ : IRm → IR is any (measurable) function fulﬁlling conditions (2.46a), (2.46b), then 1 Eϕ y˜ a(ω), x , (2.47) P y˜ a(ω), x < (≤) 0 ≥ 1 − ϕ0 provided that the expectation in (2.47) exists and is ﬁnite. Remark 2.3. Note that a related inequality was used already in Example 2.2. Example 2.8. If ϕ(y) :=

m

wi eαi yi with positive constants wi , αi , i =

i=1

1, . . . , m, then inf ϕ(y) = min wi > 0

(2.48a)

m Eϕ y˜ a(ω), x = wi Eeαi y˜i a(ω),x ,

(2.48b)

˜ y∈ |B

and

1≤i≤m

i=1

where the expectation in (2.48b) can be computed approximatively by Taylor expansion: αi y˜i −˜ y i (x) Eeαi y˜i = eαi y˜i (x) Ee 2 α2 ≈ eαi y˜i (x) 1 + i E yi a(ω), x − y i (x) 2

α2 = eαi y˜i (x) 1 + i σy2i (a(·),x) . (2.48c) 2 Supposing that yi = yi a(ω), x is a normal distributed random variable, then 1 2 2 ¯ α σ Eeαi y˜i = eαi y˜i (x) e 2 i yi (a(·),x) . (2.48d) Example 2.9. Consider ϕ(y) := (y − b)T C(y − b), where, cf. Example 2.6, C is a positive deﬁnite m × m matrix and b < 0 a ﬁxed m-vector. In this case we again have min ϕ(y) = min min ϕ(y) ˜ y∈ |B

1≤i≤m yi ≥0

and T Eϕ(˜ y ) = E y˜ a(ω), x − b C y˜ a(ω), x − b T = trCE y˜ a(ω), x − b y˜ a(ω), x − b .

2.5 Approximation of Probabilities: Probability Inequalities

Note that

y˜i (a, x) − bi =

37

yi (a, x) − (yui + bi ), if yli = −∞ yli − bi − yi (a, x), if yui = +∞,

where yui + bi < yui and yli < yli − bi . Remark 2.4. The one-sided case can also be reduced approximatively to the two-sided case by selecting a suﬃciently large, but ﬁnite upper bound y˜ui ∈ IR, lower bound y˜li ∈ IR, resp., if yui = +∞, yli = −∞. For multivariate Tschebyscheﬀ inequalities see also [75]. 2.5.3 First Order Reliability Methods (FORM) Special approximation methods for probabilities were developed in structural reliability analysis [19, 21, 54, 104] for the probability of failure pf = pf (x) given by (2.9c) with B := (0, +∞)my . Hence, see (2.2b), pf = 1 − P y a(ω), x > 0 . Applying certain probability inequalities, see Sects. 2.5.1 and 2.5.2, the case with m(>1) limit state functions is reduced ﬁrst to the case with one limit state function only. Approximations of pf are obtained then by linearization a, x) or by quadratic approximation of a transformed state function y˜i = y˜i (˜ ˜∗i (x) for each i = 1, . . . , m. at a certain “design point” a ˜∗ = a Based on the above procedure, according to (2.9f) we have pf (x) = P yi a(ω), x ≤ 0 for at least one index i, 1 ≤ i ≤ my ≤

my i=1

where

my P yi a(ω), x ≤ 0 = pf i (x),

(2.49a)

i=1

pf i (x) := P yi a(ω), x ≤ 0 .

(2.49b)

Assume now that the random ν-vector a = a(ω) can be represented [104] by a(ω) = U a(ω) , (2.50a) a= a(ω) is where U : IRν → IRν is a certain 1–1-transformation in IRν , and an N (0, I)-normal distributed random ν-vector, i.e., with mean vector zero and covariance matrix equal to the identity matrix I. Assuming that the νvector a = a(ω) has a continuous probability distribution and considering the conditional distribution functions Hi = Hi (ai |a1 , . . . , ai−1 )

38

2 Deterministic Substitute Problems in Optimal Decision

of ai = ai(ω), given aj (ω) = aj , j = 1, . . . , i − 1, then the random ν-vector z(ω) := T a(ω) deﬁned by the transformation z1 := H1 (a1 ) z2 := H2 (a2 |a1 ) .. . zν := Hν (aν |a1 , . . . , aν−1 ), cf. [128], has stochastically independent components z1 = z1 (ω), . . . , zν = zν (ω), and z = z(ω) is uniformly distributed on the ν-dimensional hypercube [0, 1]ν . Thus, if Φ = Φ(t) denotes the distribution function of the univariate N (0, 1)-normal distribution, then the inverse a ˜ = U −1 (a) of the transformation U can be represented by ⎞ ⎛ −1 Φ (z1 ) ⎟ ⎜ .. a ˜ := ⎝ ⎠ , z = T (a). . Φ−1 (zν )

In the following, let x be a given, ﬁxed r-vector. Deﬁning then a), x , a, x) := yi U ( yi ( we get

pf i (x) = P yi a(ω), x ≤ 0 , i = 1, . . . , my .

(2.50b)

(2.50c)

In the First Order Reliability Method (FORM), cf. [54], the state function a, x) is linearized at a so-called design point a∗ = a∗(i) . A design point yi = yi ( a∗(i) is obtained by projection of the origin a = 0 onto the limit state a∗ = a, x) = 0, see Fig. 2.5. Since yi ( a∗ , x) = 0, we get surface yi (

Fig. 2.5. FORM

2.5 Approximation of Probabilities: Probability Inequalities

T yi ( a, x) ≈ yi ( a∗ , x) + ∇a yi ( a∗ , x) ( a− a∗ ) T a∗ , x) ( a− a∗ ) = ∇a yi (

39

(2.50d)

and therefore

T ∗ ∗ ∇a yi ( a , x) ( a(ω) − a ) ≤ 0 = Φ −βi (x) . pf i (x) ≈ P

(2.50e)

Here, the reliability index βi = βi (x) is given by T ∇a yi ( a∗ , x) a∗ . βi (x) := − ∇a yi ( a∗ , x)

(2.50f)

Note that βi (x) is the minimum distance from a = 0 to the limit state surface a, x) = 0 in the a-domain. yi ( For more details and Second Order Reliability Methods (SORM) based on second order Taylor expansions, see Chap. 7 and [54, 104].

2 Deterministic Substitute Problems in Optimal Decision Under Stochastic Uncertainty

2.1 Optimum Design Problems with Random Parameters In the optimum design of technical or economic structures/systems two basic classes of criterions appear: First there is a primary cost function G0 = G0 (a, x).

(2.1a)

Important examples are the total weight or volume of a mechanical structure, the costs of construction, design of a certain technical or economic structure/ system, or the negative utility or reward in a general decision situation. For the representation of the structural/system safety or failure, for the representation of the admissibility of the state, or for the formulation of the basic operating conditions of the underlying structure/system certain state functions (2.1b) yi = yi (a, x), i = 1, . . . , my , are chosen. In structural design these functions are also called “limit state functions” or “safety margins.” Frequent examples are some displacement, stress, load (force and moment) components in structural design, or production and cost functions in production planning problems, optimal mix problems, transportation problems, allocation problems and other problems of economic design. In (2.1a), (2.1b), the design or input vector x denotes the r-vector of design or input variables, x1 , x2 , . . . , xr , as, e.g., structural dimensions, sizing variables, such as cross-sectional areas, thickness in structural design, or factors of production, actions in economic design. For the design or input vector x one has mostly some basic deterministic constraints, e.g., nonnegativity constraints, box constraints, represented by x ∈ D,

(2.1c)

10

2 Deterministic Substitute Problems in Optimal Decision

where D is a given convex subset of IRr . Moreover, a is the ν-vector of model parameters. In optimal structural design, p a= (2.1d) P is composed of the following two subvectors: P is the m-vector of the acting external loads, e.g., wave, wind loads, payload, etc. Moreover, p denotes the (ν − m)-vector of the further model parameters, as, e.g., material parameters (like strength parameters, yield/allowable stresses, elastic moduli, plastic capacities, etc.) of each member of the structure, the manufacturing tolerances and the weight or more general cost coeﬃcients. In production planning problems, a = (A, b, c)

(2.1e)

is composed of the m × r matrix A of technological coeﬃcients, the demand m-vector b and the r-vector c of unit costs. Based on the my -vector of state functions T y(a, x) := y1 (a, x), y2 (a, x), . . . , ymy (a, x) ,

(2.1f)

the admissible or safe states of the structure/system can be characterized by the condition y(a, x) ∈ B, (2.1g) where B is a certain subset of IRmy ; B = B(a) may depend also on some model parameters. In production planning problems, typical operating conditions are given, cf. (2.1e), by y(a, x) := Ax − b ≥ 0 or y(a, x) = 0, x ≥ 0.

(2.2a)

In mechanical structures/structural systems, the safety (survival) of the structure/system is described by the operating conditions yi (a, x) > 0 for all i = 1, . . . , my

(2.2b)

with state functions yi = yi (a, x), i = 1, . . . , my , depending on certain response components of the structure/system, such as displacement, stress, force, moment components. Hence, a failure occurs if and only of the structure/system is in the ith failure mode (failure domain) yi (a, x) ≤ 0

(2.2c)

for at least one index i, 1 ≤ i ≤ my . Note. The number my of safety margins or limit state functions yi = yi (a, x), i = 1, . . . , my , may be very large. E.g., in optimal plastic

2.1 Optimum Design Problems with Random Parameters

11

design the limit state functions are determined by the extreme points of the admissible domain of the dual pair of static/kinematic LPs related to the equilibrium and linearized convex yield condition, see Sect. 7. Basic problems in optimal decision/design are: (1) Primary (construction, planning, investment, etc.) cost minimization under operating or safety conditions min G0 (a, x)

(2.3a)

y(a, x) ∈ B

(2.3b)

s.t. x ∈ D.

(2.3c)

Obviously we have B = (0, +∞)my in (2.2b) and B = [0, +∞)my or B = {0} in (2.2a). (2) Failure or recourse cost minimization under primary cost constraints “min”γ y(a, x) (2.4a) s.t. G0 (a, x) ≤ Gmax x ∈ D.

(2.4b) (2.4c)

In (2.4a) γ = γ(y) is a scalar or vector valued cost/loss function evaluating violations of the operating conditions (2.3b). Depending on the application, these costs are called “failure” or “recourse” costs [58, 59, 98, 120, 137, 138]. As already discussed in Sect. 1, solving problems of the above type, a basic diﬃculty is the uncertainty about the true value of the vector a of model parameters or the (random) variability of a: In practice, due to several types of uncertainties such as, see [154]: –

Physical uncertainty (variability of physical quantities, like material, loads, dimensions, etc.) – Economic uncertainty (trade, demand, costs, etc.) – Statistical uncertainty (e.g., estimation errors of parameters due to limited sample data) – Model uncertainty (model errors) the ν-vector a of model parameters must be modeled by a random vector a = a(ω), ω ∈ Ω,

(2.5a)

on a certain probability space (Ω, A0 , P) with sample space Ω having elements ω, see (1.3). For the mathematical representation of the corresponding (conA of the random vector a = a(ω) ditional) probability distribution Pa(·) = Pa(·)

12

2 Deterministic Substitute Problems in Optimal Decision

(given the time history or information A ⊂ A0 ), two main distribution models are taken into account in practice: (1) Discrete probability distributions (2) Continuous probability distributions In the ﬁrst case there is a ﬁnite or countably inﬁnite number l0 ∈ IN ∪ {∞} of realizations or scenarios al ∈ IRν , l = 1, . . . , l0 , (2.5b) P a(ω) = al = αl , l = 1, . . . , l0 , taken with probabilities αl , l = 1, . . . , l0 . In the second case, the probability that the realization a(ω) = a lies in a certain (measurable) subset B ⊂ IRν is described by the multiple integral (2.5c) P a(ω) ∈ B = ϕ(a) da B

with a certain probability density function ϕ = ϕ(a) ≥ 0, a ∈ IR , ν

ϕ(a)da = 1.

The properties of the probability distribution Pa(·) may be described – fully or in part – by certain numerical characteristics, called parameters of Pa(·) . These distribution parameters θ = θh are obtained by considering expectations θh := Eh a(ω) (2.6a) of (measurable) functions (h ◦ a)(ω) := h a(ω)

(2.6b)

composed of the random vector a = a(ω) with certain (measurable) mappings h : IRν −→ IRsh , sh ≥ 1.

(2.6c)

According to the type of the probability distribution Pa(·) of a = a(ω), the expectation Eh a(ω) is deﬁned by ⎧ l0 ⎪ ⎪ ⎪ h al αl , in the discrete case (2.5b) ⎪ ⎨ Eh a(ω) = l=1 ⎪ ⎪ h(a)ϕ(a) da, in the continuous case (2.5c). ⎪ ⎪ ⎩

(2.6d)

IRν

Further distribution parameters θ are functions θ = Ψ (θh1 , . . . , θhs )

(2.7)

2.1 Optimum Design Problems with Random Parameters

13

of certain “h-moments” θh1 , . . . , θhs of the type (2.6a). Important examples of the type (2.6a), (2.7), resp., are the expectation a = Ea(ω) (for h(a) := a, a ∈ IRν ) and the covariance matrix T Q := E a(ω) − a a(ω) − a = Ea(ω)a(ω)T − a aT

(2.8a)

(2.8b)

of the random vector a = a(ω). Due to the stochastic variability of the random vector a(·) of model parameters, and since the realization a(ω) = a is not available at the decision making stage, the optimal design problem (2.3a)–(2.3c) or (2.4a)–(2.4c) under stochastic uncertainty cannot be solved directly. Hence, appropriate deterministic substitute problems must be chosen taking into account the randomness of a = a(ω), cf. Sect. 1.2. 2.1.1 Deterministic Substitute Problems in Optimal Design According to Sect. 1.2, a basic deterministic substitute problem in optimal design under stochastic uncertainty is the minimization of the total expected costs including the expected costs of failure (2.9a) min cG EG0 a(ω), x + cf pf (x) s.t. Here,

x ∈ D.

(2.9b)

pf = pf (x) := P y a(ω), x ∈B |

(2.9c)

is the probability of failure or the probability that a safe function of the structure, the system is not guaranteed. Furthermore, cG is a certain weight factor, and cf > 0 describes the failure or recourse costs. In the present deﬁnition of expected failure costs, constant costs for each realization a = a(ω) of a(·) are assumed. Obviously, it is pf (x) = 1 − ps (x)

(2.9d)

with the probability of safety or survival ps (x) := P y a(ω), x ∈ B .

(2.9e)

In case (2.2b) we have pf (x) = P yi a(ω), x ≤ 0 for at least one index i, 1 ≤ i ≤ my .

(2.9f)

14

2 Deterministic Substitute Problems in Optimal Decision

The objective function (2.9a) may be interpreted as the Lagrangian (with given cost multiplier cf ) of the following reliability based optimization (RBO) problem, cf. [2, 28, 88, 91, 99, 120, 138, 154, 162]: (2.10a) min EG0 a(ω), x s.t. pf (x) ≤ αmax x ∈ D,

(2.10b) (2.10c)

where αmax > 0 is a prescribed maximum failure probability, e.g., αmax = 0.001, cf. (2.3a)–(2.3c). The “dual” version of (2.10a)–(2.10c) reads

s.t.

min pf (x)

(2.11a)

EG0 a(ω), x ≤ Gmax

(2.11b)

x∈D

(2.11c)

with a maximum (upper) cost bound Gmax , see (2.4a)–(2.4c). Further substitute problems are obtained by considering more general expected failure or recourse cost functions Γ (x) = Eγ y a(ω), x (2.12a) arising from structural systems weakness or failure, or because of false operation. Here, T (2.12b) y a(ω), x := y1 a(ω), x , . . . , ymy a(ω), x is again the random vector of state functions, and γ : IRmy → IRmγ

(2.12c)

is a scalar or vector valued cost or loss function. In case B = (0, +∞)my or B = [0, +∞)my it is often assumed that γ = γ(y) is a nonincreasing function, hence, γ(y) ≥ γ(z), if y ≤ z, (2.12d) where inequalities between vectors are deﬁned componentwise. Example 2.1. If γ(y) = 1 for y ∈ B c (complement of B) and γ(y) = 0 for y ∈ B, then Γ (x) = pf (x).

2.1 Optimum Design Problems with Random Parameters

15

Example 2.2. Suppose that γ = γ(y) is a nonnegative scalar function on IRmy such that | (2.13a) γ(y) ≥ γ0 > 0 for all y ∈B with a constant γ0 > 0. Then for the probability of failure we ﬁnd the following upper bound 1 | ≤ Eγ y a(ω), x , pf (x) = P y a(ω), x ∈B γ0

(2.13b)

where the right hand side of (2.13b) is obviously an expected cost function of type (2.12a)–(2.12c). Hence, the condition (2.10b) can be guaranteed by the cost constraint (2.13c) Eγ y a(ω), x ≤ γ0 αmax . Example 2.3. If the loss function γ(y) is deﬁned by a vector of individual loss functions γi for each state function yi = yi (a, x), i = 1, . . . , my , hence, T γ(y) = γ1 (y1 ), . . . , γmy (ymy ) ,

(2.14a)

then

T Γ (x) = Γ1 (x), . . . , Γmy (x) , Γi (x) := Eγi yi a(ω), x , 1 ≤ i ≤ my , (2.14b)

i.e., the my state functions yi , i = 1, . . . , my , will be treated separately. Working with the more general expected failure or recourse cost functions Γ = Γ (x), instead of (2.9a)–(2.9c), (2.10a)–(2.10c) and (2.11a)–(2.11c) we have the related substitute problems: (1) Expected total cost minimization min cG EG0 a(ω), x + cTf Γ (x),

(2.15a)

s.t. x∈D

(2.15b)

(2) Expected primary cost minimization under expected failure or recourse cost constraints (2.16a) min EG0 a(ω), x s.t. Γ (x) ≤ Γ max

(2.16b)

x ∈ D,

(2.16c)

16

2 Deterministic Substitute Problems in Optimal Decision

(3) Expected failure or recourse cost minimization under expected primary cost constraints “min”Γ (x) (2.17a) s.t. EG0 a(ω), x ≤ Gmax (2.17b) x ∈ D.

(2.17c)

Here, cG , cf are (vectorial) weight coeﬃcients, Γ max is the vector of upper loss bounds, and “min” indicates again that Γ (x) may be a vector valued function. 2.1.2 Deterministic Substitute Problems in Quality Engineering In quality engineering [109,113,114] the aim is to optimize the performance of a certain product or a certain process, and to make the performance insensitive (robust) to any noise to which the product, the process may be subjected. Moreover, the production costs (development, manufacturing, etc.) should be bounded or minimized. Hence, the control (input, design) factors x are then to set as to provide – –

An optimal mean performance (response) A high degree of immunity (robustness) of the performance (response) to noise factors – Low or minimum production costs Of course, a certain trade-oﬀ between these criterions may be taken into account. For this aim, the quality characteristics or performance (e.g., dimensions, weight, contamination, structural strength, yield, etc.) of the product/process is described [109] ﬁrst by a function y = y a(ω), x (2.18) depending on a ν-vector a = a(ω) of random varying parameters (noise factors) aj = aj (ω), j = 1, . . . , ν, causing quality variations, and the r-vector x of control/input factors xk , k = 1, . . . , r. The quality loss of the product/process under consideration is measured then by a certain loss function γ = γ(y). Simple loss functions γ are often chosen [113, 114] in the following way: (1) Nominal-is-best Having a given, ﬁnite target or nominal value y ∗ , e.g., a nominal dimension of a product, the quality loss γ(y) reads γ(y) := (y − y ∗ )2 .

(2.19a)

2.1 Optimum Design Problems with Random Parameters

17

(2) Smallest-is-best If the target or nominal value y ∗ is zero, i.e., if the absolute value |y| of the product quality function y = y(a, x) should be as small as possible, e.g., a certain product contamination, then the corresponding quality loss is deﬁned by (2.19b) γ(y) := y 2 . (3) Biggest-is-best If the absolute value |y| of the product quality function y = y(a, x) should be as large as possible, e.g., the strength of a bar or the yield of a process, then a possible quality loss is given by γ(y) :=

2 1 . y

(2.19c)

Obviously, (2.19a) and (2.19b) are convex loss functions. Moreover, since the quality or response function y = y(a, x) takes only positive or only negative values y in many practical applications, also (2.19c) yields a convex loss function in many cases. Further decision situations may be modeled by choosing more general convex loss functions γ = γ(y). A high mean product/process level or performance at minimum random quality variations and low or bounded production/manufacturing costs is achieved then [109, 113, 114] by minimization of the expected quality loss function Γ (x) := Eγ y a(ω), x

,

(2.19d)

subject to certain constraints for the input x, as, e.g., constraints for the production/manufacturing costs. Obviously, Γ (x) can also be minimized by maximizing the negative logarithmic mean loss (sometimes called “signal-tonoise-ratio” [109]): SN := − log Γ (x).

(2.19e)

Since Γ = Γ (x) is a mean value function as described in the previous sections, for the computation of a robust optimal input vector x∗ one has the following stochastic optimization problem

s.t.

min Γ (x)

(2.20a)

EG0 a(ω), x ≤ Gmax

(2.20b)

x ∈ D.

(2.20c)

18

2 Deterministic Substitute Problems in Optimal Decision

2.2 Basic Properties of Substitute Problems As can be seen from the conversion of an optimization problem with random parameters into a deterministic substitute problem, cf. Sects. 2.1.1 and 2.1.2, a central role is played by expectation or mean value functions of the type (2.21a) Γ (x) = Eγ y a(ω), x , x ∈ D0 , or more general

Γ (x) = Eg a(ω), x , x ∈ D0 .

(2.21b)

Here, a = a(ω) is a random ν-vector, y = y(a, x) is an my -vector valued function on a certain subset of IRν × IRr , and γ = γ(z) is a real valued function on a certain subset of IRmy . Furthermore, g = g(a, x) denotes a real valued function on a certain subset of IRν ×IRr . In the following we suppose that the expectation in (2.21a), (2.21b) exists and is ﬁnite for all input vectors x lying in an appropriate set D0 ⊂ IRr , cf. [11]. The following basic properties of the mean value functions Γ are needed in the following again and again. Lemma 2.1. Convexity. Suppose that x → g a(ω), x is convex a.s. (almost sure) on a ﬁxed convex domain D0 ⊂ IRr . If Eg a(ω), x exists and is ﬁnite for each x ∈ D0 , then Γ = Γ (x) is convex on D0 . Proof. This property follows [58, 59, 78] directly from the linearity of the expectation operator. If g = g(a, x) is deﬁned by g(a, x) := γ y(a, x) , see (2.21a), then the above theorem yields the following result:

Corollary 2.1. Suppose that γ is convex and Eγ y a(ω), x exists and is ﬁnite for each x ∈ D0 . (a) If x → y a(ω), x is linear a.s., then Γ = Γ (x) is convex. (b) If x → y a(ω), x is convex a.s., and γ is a convex, monotonous nondecreasing function, then Γ = Γ (x) is convex. It is well known [72] that a convex function is continuous on each open subset of its domain. A general suﬃcient condition for the continuity of Γ is given next. Lemma 2.2. Continuity. Suppose that Eg a(ω), x exists and is ﬁnite for each x ∈ D0 , and assume that x → g a(ω), x is continuous at x0 ∈ D0 a.s. If there is a function ψ = ψ a(ω) having ﬁnite expectation such that

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

g a(ω), x ≤ ψ a(ω) a.s. for all x ∈ U (x0 ) ∩ D0 ,

19

(2.22)

where U (x0 ) is a neighborhood of x0 , then Γ = Γ (x) is continuous at x0 . Proof. The assertion can be shown by using Lebesgue’s dominated convergence theorem, see, e.g., [78]. For the consideration of the diﬀerentiability of Γ = Γ (x), let D denote an

open subset of the domain D0 of Γ . Lemma 2.3. Diﬀerentiability. Suppose that (1) Eg a(ω), x exists and is ﬁnite for each x ∈ D0 , (2) x → g a(ω), x is diﬀerentiable on the open subset D of D0 a.s. and (3) (2.23a) ∇x g a(ω), x ≤ ψ a(ω) , x ∈ D, a.s., where ψ = ψ a(ω) is a function having ﬁnite expectation. Then the expec tation of ∇x g a(ω), x exists and is ﬁnite, Γ = Γ (x) is diﬀerentiable on D and ∇Γ (x) = ∇x Eg a(ω), x = E∇x g a(ω), x , x ∈ D. (2.23b) ∆Γ , k = 1, . . . , r, of Γ at a ∆xk ﬁxed point x0 ∈ D, the assertion follows by means of the mean value theorem, inequality (2.23a) and Lebesgue’s dominated convergence theorem, cf. [58, 59, 78].

Proof. Considering the diﬀerence quotients

Example 2.4. In case (2.21a), under obvious diﬀerentiability assumptions conT cerning γ and y we have ∇x g(a, x) = ∇x y(a, x) ∇γ y(a, x) , where ∇x y(a, x) denotes the Jacobian of y = y(a, x) with respect to a. Hence, if (2.23b) holds, then T (2.23c) ∇Γ (x) = E∇x y a(ω), x ∇γ y a(ω), x .

2.3 Approximations of Deterministic Substitute Problems in Optimal Design The main problem in solving the deterministic substitute problems deﬁned above is that the arising probability and expected cost functions pf = pf (x), Γ = Γ (x), x ∈ IRr , are deﬁned by means of multiple integrals over a ν-dimensional space. Thus, the substitute problems may be solved, in practice, only by some approximative analytical and numerical methods [37, 165]. In the following

20

2 Deterministic Substitute Problems in Optimal Decision

we consider possible approximations for substitute problems based on general expected recourse cost functions Γ = Γ (x) according to (2.21a) having a real valued convex loss function γ(z). Note that the probability of failure function pf = pf (x) may be approximated from above, see (2.13a), (2.13b), by expected cost functions Γ = Γ (x) having a nonnegative function γ = γ(z) being bounded from below on the failure domain B c . In the following several basic approximation methods are presented. 2.3.1 Approximation of the Loss Function Suppose here that γ = γ(y) is a continuously diﬀerentiable, convex loss function on IRmy . Let then denote T y(x) := Ey a(ω), x = Ey1 a(ω), x , . . . , Eymy a(ω), x (2.24) the expectation of the vector y = y a(ω), x of state functions yi = yi a(ω), x , i = 1, . . . , my . For an arbitrary continuously diﬀerentiable, convex loss function γ we have T y a(ω), x − y(x) . (2.25a) γ y a(ω), x ≥ γ y(x) + ∇γ y(x) Thus, taking expectations in (2.25a), we ﬁnd Jensen’s inequality (2.25b) Γ (x) = Eγ y a(ω), x ≥ γ y(x) which holds for any convex function γ. Using the mean value theorem, we have y )T (y − y), (2.25c) γ(y) = γ(y) + ∇γ(ˆ where yˆ is a point on the line segment yy between y and y. By means of (2.25b), (2.25c) we get 0 ≤ Γ (x) − γ y(x) ≤ E ∇γ yˆ a(ω), x · y a(ω), x − y(x) . (2.25d) (a) Bounded gradient If the gradient ∇γ is bounded on the range of y = y a(ω), x , x ∈ D, i.e., if (2.26a) ∇γ y a(ω), x ≤ ϑmax a.s. for each x ∈ D, with a constant ϑmax > 0, then 0 ≤ Γ (x) − γ y(x) ≤ ϑmax E y a(ω), x − y(x) , x ∈ D.

(2.26b)

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

√ Since t → t, t ≥ 0, is a concave function, we get 0 ≤ Γ (x) − γ y(x) ≤ ϑmax q(x), where

2 q(x) := E y a(ω), x − y(x) = trQ(x)

21

(2.26c)

(2.26d)

is the generalized variance, and

Q(x) := cov y a(·), x

(2.26e)

denotes the covariance matrix of the random vector y = y a(ω), x . Consequently, the expected loss function Γ (x) can be approximated from above by Γ (x) ≤ γ y(x) + ϑmax q(x) for x ∈ D. (2.26f) (b) Bounded eigenvalues of the Hessian Considering second order expansions of γ, with a vector y˜ ∈ y¯y we ﬁnd 1 y )(y − y). γ(y) − γ(y) = ∇γ(y)T (y − y) + (y − y)T ∇2 γ(˜ 2

(2.27a)

Suppose that the eigenvalues λ of ∇2 γ(y) are bounded from below and above on the range of y = y a(ω), x for each x ∈ D, i.e., ≤ λmax < +∞, a.s., x ∈ D (2.27b) 0 < λmin ≤ λ ∇2 γ y a(ω), x with constants 0 < λmin ≤ λmax . Taking expectations in (2.27a), we get λmin λmax q(x) ≤ Γ (x) ≤ γ y(x) + q(x), x ∈ D. (2.27c) γ y(x) + 2 2 Consequently, using (2.26f) or (2.27c), various approximations for the deterministic substitute problems (2.15a), (2.15b), (2.16a)–(2.16c), (2.17a)– (2.17c) may be obtained. Based on the above approximations of expected cost functions, we state the following two approximates to (2.16a)–(2.16c), (2.17a)–(2.17c), resp., which are well known in robust optimal design: (1) Expected primary cost minimization under approximate expected failure or recourse cost constraints (2.28a) min EG0 a(ω), x s.t.

γ y(x) + c0 q(x) ≤ Γ max x ∈ D,

where c0 is a scale factor, cf. (2.26f) and (2.27c);

(2.28b) (2.28c)

22

2 Deterministic Substitute Problems in Optimal Decision

(2) Approximate expected failure or recourse cost minimization under expected primary cost constraints (2.29a) min γ y(x) + c0 q(x) s.t.

EG0 a(ω), x ≤ Gmax x ∈ D.

(2.29b) (2.29c)

Obviously, by means of (2.28a)–(2.28c) or (2.29a)–(2.29c) optimal designs x∗ are achieved which – – –

Yield a high mean performance of the structure/structural system Are minimally sensitive or have a limited sensitivity with respect to random parameter variations (material, load, manufacturing, process, etc.) Cause only limited costs for design, construction, maintenance, etc.

2.3.2 Regression Techniques, Model Fitting, RSM In structures/systems with variable quantities, very often one or several functional relationships between the measurable variables can be guessed according to the available data resulting from observations, simulation experiments, etc. Due to limited data, the true functional relationships between the quantities involved are approximated in practice – on a certain limited range – usually by simple functions, as, e.g., multivariate polynomials of a certain (low) degree. The resulting regression metamodel [64] is obtained then by certain ﬁtting techniques, such as weighted least squares or Tschebyscheﬀ minmax procedures for estimating the unknown parameters of the regression model. An important advantage of these models is then the possibility to predict the values of some variables from knowledge of other variables. In case of more than one independent variable (predictor or regressor) one has a “multiple regression” task. Problems with more than one dependent or response variable are called “multivariate regression” problems, see [33]. (a) Approximation of state functions. The numerical solution is simpliﬁed considerably if one can work with one single state function y = y(a, x). Formally, this is possible by deﬁning the function y min (a, x) := min yi (a, x). 1≤i≤my

(2.30a)

Indeed, according to (2.2b), (2.2c) the failure of the structure, the system can be represented by the condition y min (a, x) ≤ 0.

(2.30b)

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

23

Fig. 2.1. Loss function γ

Thus, the weakness or failure of the technical or economic device can be evaluated numerically by the function (2.30c) Γ (x) := Eγ y min a(ω), x with a nonincreasing loss function γ : IR → IR+ , see Fig. 2.1. However, the “min”-operator in (2.30a) yields a nonsmooth function y min = y min (a, x) in general, and the computation of the mean and variance function (2.30d) y min (x) := Ey min a(ω), x (2.30e) σy2min (x) := V y min a(·), x by means of Taylor expansion with respect to the model parameter vector a at a = Ea(ω) is not possible directly, cf. Sect. 2.3.3. According to the deﬁnition (2.30a), an upper bound for y min (x) is given by y min (x) ≤ min y i (x) = min Eyi a(ω), x . 1≤i≤my

Further approximations of y using the representation

min

1≤i≤my

(a, x) and its moments can be found by

1 a + b − |a − b| 2 of the minimum of two numbers a, b ∈ IR. E.g., for an even index my we have min min yi (a, x), yi+1 (a, x) y min (a, x) = min(a, b) =

i=1,3,...,my −1

=

1 yi (a, x) + yi+1 (a, x)− yi (a, x) − yi+1 (a, x) . i=1,3,...,my −1 2 min

24

2 Deterministic Substitute Problems in Optimal Decision

In many cases we may suppose that the state functions yi = yi (a, x), i = 1, . . . , my , are bounded from below, hence, yi (a, x) > −A, i = 1, . . . , my , for all (a, x) under consideration with a positive constant A > 0. Thus, deﬁning y˜i (a, x) := yi (a, x) + A, i = 1, . . . , my , and therefore y˜min (a, x) := min y˜i (a, x) = y min (a, x) + A, 1≤i≤my

we have y min (a, x) ≤ 0 if and only if y˜min (a, x) ≤ A. Hence, the survival/failure of the system or structure can be studied also by means of the positive function y˜min = y˜min (a, x). Using now the theory of power or H¨older means [24], the minimum y˜min (a, x) of positive functions can be represented also by the limit min

y˜

(a, x) = lim

λ→−∞

my 1 y˜i (a, x)λ my i=1

1/λ

of the decreasing family of power means M

[λ]

(˜ y ) :=

my 1 λ y˜ my i=1 i

1/λ ,

λ < 0. Consequently, for each ﬁxed p > 0 we also have min

y˜

p

(a, x) = lim

λ→−∞

my 1 y˜i (a, x)λ my i=1

p/λ .

p Assuming that the expectation EM [λ] y˜ a(ω) , x exists for an exponent λ = λ0 < 0, by means of Lebesgue’s bounded convergence theorem we get the moment representation min

E y˜

p a(ω), x = lim E λ→−∞

my λ 1 y˜i a(ω), x my i=1

p/λ .

Since t → tp/λ , t > 0, is convex for each ﬁxed p > 0 and λ < 0, by Jensen’s inequality we have the lower moment bound p E y˜min a(ω), x ≥ lim

λ→−∞

my λ 1 E y˜i a(ω), x my i=1

p/λ .

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

25

a(·), x we get the approxima-

Hence, for the pth order moment of y˜min tions my my p/λ λ p/λ 1 1 y˜i a(ω), x ≥ E y˜i a(ω), x E my i=1 my i=1 for some λ < 0.

Using regression techniques, Response Surface Methods (RSM), etc., for given vector x, the function a → y min (a, x) can be approximated [14, 23, 55,63,136] by functions y = y(a, x) being suﬃciently smooth with respect to the parameter vector a (Figs. 2.2 and 2.3). In many important cases, for each i = 1, . . . , my , the state functions (a, x) −→ yi (a, x) are bilinear functions. Thus, in this case y min = y min (a, x) is a piecewise linear function with respect to a. Fitting a linear or quadratic Response Surface Model [17, 18, 105, 106] y(a, x) := c(x) + q(x)T (a − a) + (a − a)T Q(x)(a − a)

(2.30f)

to a → y min (a, x), after the selection of appropriate reference points a(j) := a + da(j) , j = 1, . . . , p,

(2.30g)

Fig. 2.2. Approximate smooth failure domain [y(a, x) ≤ 0] for given x

26

2 Deterministic Substitute Problems in Optimal Decision

Fig. 2.3. Approximation y˜(a, x) of y min (a, x) for given x (j)

with “design” points da ∈ IRν , j = 1, . . . , p, the unknown coeﬃcients c = c(x), q = q(x) and Q = Q(x) are obtained by minimizing the mean square error p 2 y(a(j) , x) − y min (a(j) , x) (2.30h) ρ(c, q, Q) := j=1

with respect to (c, q, Q). Since the model (2.30f) depends linearly on the function parameters (c, q, Q), explicit formulas for the optimal coeﬃcients c∗ = c∗ (x), q ∗ = q ∗ (x), Q∗ = Q∗ (x),

(2.30i)

are obtained from this least squares estimation method, see Chap. 5. (b) Approximation of expected loss functions. Corresponding to the approximation (2.30f) of y min = y min (a, x),using again least squares techniques, a mean value function Γ (x) = Eγ y a(ω), x , cf. (2.12a), can be approximated at a given point x0 ∈ IRν by a linear or quadratic Response Surface Function Γ(x) := β0 + βIT (x − x0 ) + (x − x0 )T B(x − x0 ),

(2.30j)

with scalar, vector and matrix parameters β0 , βI , B. In this case estimates y (i) = Γˆ (i) of Γ (x) are needed at some reference points x(i) = x0 + d(i) , i = 1, . . . , p. Details are given in Chap. 5. 2.3.3 Taylor Expansion Methods As can be seen above, cf. (2.21a), (2.21b), in the objective and/or in the constraints of substitute problems for optimization problems with random data mean value functions of the type

2.3 Approximations of Deterministic Substitute Problems in Optimal Design

27

Γ (x) := Eg a(ω), x occur. Here, g = g(a, x) is a real valued function on a subset of IRν × IRr , and a = a(ω) is a ν-random vector. (a) Expansion with respect to a: Suppose that on its domain the function g = g(a, x) has partial derivatives ∇la g(a, x), l = 0, 1, . . . , lg +1, up to order lg + 1. Note that the gradient ∇a g(a, x) contains the so-called sensitivities ∂g (a, x), j = 1, . . . , ν, of g with respect to the parameter vector a at ∂aj (a, x). In the same way, the higher order partial derivatives ∇la g(a, x), l > 1, represent the higher order sensitivities of g with respect to a at (a, x). Taylor expansion of g = g(a, x) with respect to a at a := Ea(ω) yields lg 1 1 l ∇a g(¯ ∇lg+1 g(ˆ a, x) · (a − a ¯)l + a, x) · (a − a ¯)lg +1 g(a, x) = l! (lg + 1)! a l=0

(2.31a) where a ˆ := a ¯ + ϑ(a − a ¯), 0 < ϑ < 1, and (a − a ¯)l denotes the system of lth order products ν (aj − a ¯j )lj j=1

with lj ∈ IN ∪ {0}, j = 1, . . . , ν, l1 + l2 + . . . + lν = l. If g = g(a, x) is deﬁned by g(a, x) := γ y(a, x) , see (2.21a), then the partial derivatives ∇la g of g up to the second order read: T (2.31b) ∇a g(a, x) = ∇a y(a, x) ∇γ y(a, x) T ∇2a g(a, x) = ∇a y(a, x) ∇2 γ y(a, x) ∇a y(a, x) +∇γ y(a, x) · ∇2a y(a, x), where (∇γ) ·

∇2a y

:= (∇γ)T

∂2y ∂ak ∂al

(2.31c)

·

(2.31d)

k,l=1,...,ν

Taking expectations in (2.31a), Γ (x) can be approximated, cf. Sect. 2.3.1, by lg l ∇la g(¯ a, x) · E a(ω) − a ¯ , (2.32a) Γ (x) := g(¯ a, x) + l=2

28

2 Deterministic Substitute Problems in Optimal Decision

l where E a(ω) − a ¯ denotes the system of mixed lth central moments T of the random vector a(ω) = a1 (ω), . . . , aν (ω) . Assuming that the domain of g = g(a, x) is convex with respect to a, we get the error estimate Γ (x) − Γ(x) ≤

1 E sup ∇alg +1 g a ¯ + ϑ a(ω) − a ¯ ,x (lg + 1)! 0≤ϑ≤1 lg +1 . (2.32b) ×a(ω) − a ¯

In many practical cases the random parameter ν-vector a = a(ω) has a l +1 convex, bounded support, and ∇ag g is continuous. Then the L∞ -norm 1 ess sup ∇alg +1 g a(ω), x (2.32c) r(x) := (lg + 1)! is ﬁnite for all x under consideration, and (2.32b), (2.32c) yield the error bound lg +1 . (2.32d) ¯ Γ (x) − Γ(x) ≤ r(x)E a(ω) − a Remark 2.1. The above described method can be extended T to the case of vector valued loss functions γ(z) = γ1 (z), . . . , γmγ (z) . (b) Inner expansions: Suppose that Γ (x) is deﬁned by (2.21a), hence, Γ (x) = Eγ y a(ω), x . Linearizing the vector function y = y(a, x) with respect to a at a ¯, thus, y(a, x) ≈ y(1) (a, x) := y(¯ a, x) + ∇a y(¯ a, x)(a − a ¯), the mean value function Γ (x) is approximated by Γ(x) := Eγ y(¯ a, x) + ∇a y(¯ a, x) a(ω) − a ¯ .

(2.33a)

(2.33b)

Corresponding to (2.24), (2.26e), deﬁne y (1) (x) := Ey(1) a(ω), x = y(a, x) (2.33c) Q(x) := cov y(1) a(·), x = ∇a y(a, x) cov a(·) ∇a y(a, x)T . (2.33d) In case of convex loss functions γ, approximates of Γ and the corresponding substitute problems based on Γ may be obtained now by applying the methods described in Sect. 2.3.1. Explicit representations for Γ are obtained in case of quadratic loss functions γ.

2.4 Applications to Problems in Quality Engineering

29

Error estimates can be derived easily for Lipschitz-continuous or convex loss function γ. In case of a Lipschitz-continuous loss function γ with Lipschitz constant L > 0, e.g., for sublinear [78, 81] loss functions, using (2.33c) we have (2.33e) Γ (x) − Γ(x) ≤ L · E y a(ω), x − y(1) a(ω), x . Applying the mean value theorem [29], under appropriate second order diﬀerentiability assumptions, for the right hand side of (2.33e) we ﬁnd the following stochastic version of the mean value theorem E y a(ω), x − y(1) a(ω), x 2 ≤ E a(ω) − a sup ∇2a y a + ϑ a(ω) − a , x . (2.33f) 0≤ϑ≤1

2.4 Applications to Problems in Quality Engineering Since the deterministic substitute problems arising in quality engineering are of the same type as in optimal design, the approximation techniques described in Sect. 2.3 can be applied directly to the robust design problem (2.20a)– (2.20c). This is shown in the following for the approximation method described in Sect. 2.3.1. Hence, the approximation techniques developed in Sect. 2.3.1 are applied now to the objective function Γ (x) = Eγ y a(ω), x of the basic robust design problem (2.20a)–(2.20c). If γ = γ(y) in an arbitrary continuously diﬀerentiable convex loss function on IR1 , then we ﬁnd the following approximations: (a) Bounded ﬁrst order derivative In case that γ has a bounded derivative on the convex support of the quality function, i.e., if (2.34a) γ y a(ω), x ≤ ϑmax < +∞ a.s. , x ∈ D, then where

Γ (x) ≤ γ y(x) + ϑmax q(x), x ∈ D,

(2.34b)

y(x) := Ey a(ω), x , (2.34c) 2 (2.34d) q(x) := V y a(·), x = E y a(ω), x − y(x) denote the mean and variance of the quality function y = y a(ω), x .

30

2 Deterministic Substitute Problems in Optimal Decision

(b) Bounded second order derivative If the second order derivative γ of γ is bounded from above and below, i.e., if (2.35a) 0 < λmin ≤ γ y a(ω), x ≤ λmax < +∞ a.s., x ∈ D, then λmin λmax q(x) ≤ Γ (x) ≤ γ y(x) + q(x), x ∈ D. γ y(x) + 2 2

(2.35b)

Corresponding to (2.28a)–(2.28c), (2.29a)–(2.29c), we then have the following “dual” robust optimal design problems: (1) Expected primary cost minimizationunder approximate expected failure or recourse cost constraints (2.36a) min EG0 a(ω), x s.t.

γ y(x) + c0 q(x) ≤ Γ max

(2.36b)

x ∈ D,

(2.36c)

where Γ max denotes an upper loss bound. (2) Approximate expected failure cost minimization under expected primary cost constraints (2.37a) min γ y(x) + c0 q(x) s.t.

EG0 a(ω), x ≤ Gmax x ∈ D,

(2.37b) (2.37c)

scale where c0 is again a certain factor, see (2.34a), (2.34b), (2.35a), (2.35b), and G0 = G0 a(ω), x represents the production costs having an upper bound Gmax .

2.5 Approximation of Probabilities: Probability Inequalities In reliability analysis of engineering/economic structures or systems, a main problem is the computation of probabilities

2.5 Approximation of Probabilities: Probability Inequalities

P

N

:= P

Vi

a(ω) ∈

i=1

⎛

or

P⎝

N

N

31

(2.38a)

Vi

i=1

⎞

⎛

N

Sj ⎠ := P ⎝a(ω) ∈

j=1

⎞ Sj ⎠

(2.38b)

j=1

of unions and intersections of certain failure/survival domains (events) Vj , Sj , j = 1, . . . , N . These domains (events) arise from the representation of the structure or system by a combination of certain series and/or parallel substructures/systems. Due to the high complexity of the basic physical relations, several approximation techniques are needed for the evaluation of (2.38a), (2.38b). 2.5.1 Bonferroni-Type Inequalities In the following V1 , V2 , . . . , VN denote arbitrary (Borel-)measurable subsets of the parameter space IRν , and the abbreviation P(V ) := P a(ω) ∈ V (2.38c) is used for any measurable subset V of IRν . Starting from the representation of the probability of a union of N events, ⎞ ⎛ N N Vj ⎠ = (−1)k−1 sk,N , (2.39a) P⎝ j=1

k=1

where

sk,N :=

1≤i1
P

k

Vil

,

we obtain [45] the well known basic Bonferroni bounds ⎛ ⎞ ρ N P⎝ Vj ⎠ ≤ (−1)k−1 sk,N for ρ ≥ 1, ρ odd j=1

⎛ P⎝

N

j=1

⎞ Vj ⎠ ≥

(2.39b)

l=1

(2.39c)

k=1 ρ

(−1)k−1 sk,N for ρ ≥ 1, ρ even.

(2.39d)

k=1

Besides (2.39c), (2.39d), a large amount of related bounds of diﬀerent complexity are available, cf. [45, 153, 155]. Important bounds of ﬁrst and second degree are given below:

32

2 Deterministic Substitute Problems in Optimal Decision

⎛ max qj ≤ P ⎝

1≤j≤N

⎛ Q1 − Q2 ≤ P ⎝

N j=1 N

⎞ Vj ⎠ ≤ Q1

(2.40a)

⎞ Vj ⎠ ≤ Q1 − max

1≤l≤N

j=1

⎞ ⎛ N Q21 ≤P⎝ Vj ⎠ ≤ Q1 . Q1 + 2Q2 j=1

qil

(2.40b)

i=l

(2.40c)

The above quantities qj , qij , Q1 , Q2 are deﬁned as follows: Q1 :=

N

qj with qj := P(Vj )

(2.40d)

j=1

Q2 :=

j−1 N

qij with qij := P(Vi ∩ Vj ).

(2.40e)

j=2 i=1

Moreover, deﬁning q := (q1 , . . . , qN ), Q := (qij )1≤i,j≤N , we have

⎛ P⎝

N

(2.40f)

⎞ Vj ⎠ ≥ q T Q− q,

(2.40g)

j=1

where Q− denotes the generalized inverse of Q, cf. [155]. 2.5.2 Tschebyscheﬀ-Type Inequalities In many cases the survival or feasible domain (event) S =

m

Si , is represented

i=1

by a certain number m of inequality constraints of the type yli < (≤)yi (a, x) < (≤)yui , i = 1, . . . , m,

(2.41a)

as, e.g., operating conditions, behavioral constraints. Hence, for a ﬁxed input, design or control vector x, the event S = S(x) is given by S := {a ∈ IRν : yli < (≤)yi (a, x) < (≤)yui , i = 1, . . . , m} .

(2.41b)

Here, yi = yi (a, x), i = 1, . . . , m,

(2.41c)

are certain functions, e.g., response, output or performance functions of the structure, system, deﬁned on (a subset of) IRν × IRr . Moreover, yli < yui , i = 1, . . . , m, are lower and upper bounds for the variables yi , i = 1, . . . , m. In case of one-sided constraints some bounds yli , yui are inﬁnite.

2.5 Approximation of Probabilities: Probability Inequalities

33

Two-Sided Constraints If yli < yui , i = 1, . . . , m, are ﬁnite bounds, (2.41a) can be represented by |yi (a, x) − yic | < (≤)ρi , i = 1, . . . , m,

(2.41d)

where the quantities yic , ρi , i = 1, . . . , m, are deﬁned by yic :=

yli + yui yui − yli , ρi := . 2 2

(2.41e)

Consequently, for the probability P(S) of the event S, deﬁned by (2.41b), we have (2.41f) P(S) = P yi a(ω), x − yic < (≤)ρi , i = 1, . . . , m . Introducing the random variables yi a(ω), x − yic , i = 1, . . . , m, y˜i a(ω), x := ρi

(2.42a)

and the set B := {y ∈ IRm : |yi | < (≤)1, i = 1, . . . , m} ,

(2.42b)

with y˜ = (˜ yi )1≤i≤m , we get P(S) = P y˜ a(ω), x ∈ B .

(2.42c)

Considering any (measurable) function ϕ : IRm → IR such that (i) ϕ(y) ≥ 0, y ∈ IRm (ii) ϕ(y) ≥ ϕ0 > 0, if y ∈B, |

(2.42d) (2.42e)

with a positive constant ϕ0 , we ﬁnd the following result: Theorem 2.1. For any (measurable) function ϕ : IRm → IR fulﬁlling conditions (2.42d), (2.42e), the following Tschebyscheﬀ-type inequality holds: P yli < (≤)yi a(ω), x < (≤)yui , i = 1, . . . , m 1 Eϕ y˜ a(ω), x , (2.43) ≥1− ϕ0 provided that the expectation in (2.43) exists and is ﬁnite. denotes the probability distribution of the random mProof. If P y˜ a(·),x vector y˜ = y˜ a(ω), x , then

34

2 Deterministic Substitute Problems in Optimal Decision

Eϕ y˜ a(ω), x = ϕ(y)Py˜(a(·),x) (dy) + y∈B c

y∈B

≥

ϕ(y)Py˜(a(·),x) (dy)

ϕ(y)Py˜(a(·),x) (dy) ≥ ϕ0 y∈B c

Py˜(a(·),x) (dy)

y∈B c

| = ϕ0 1 − P y˜ a(ω), x ∈ B , = ϕ0 P y˜ a(ω), x ∈B which yields the assertion, cf. (2.42c).

Remark 2.2. Note that P(S) ≥ αs with a given minimum reliability αs ∈ (0, 1] can be guaranteed by the expected cost constraint Eϕ y˜ a(ω), x ≤ (1 − αs )ϕ0 . Example 2.5. If ϕ = 1B c is the indicator function of the complement B c of B, then ϕ0 = 1 and (2.43) holds with the equality sign. Example 2.6. For a given positive deﬁnite m × m matrix C, deﬁne ϕ(y) := y T Cy, y ∈ IRm . Then, cf. (2.42b), (2.42d), (2.42e), (2.44a) min ϕ(y) = min min y T Cy, min y T Cy . y∈ |B

1≤i≤m yi ≥1

yi ≤−1

Thus, the lower bound ϕ0 follows by considering the convex optimization problems arising in the right hand side of (2.44a). Moreover, the expectation Eϕ(˜ y ) needed in (2.43) is given, see (2.42a), by Eϕ(˜ y ) = E y˜T C y˜ = EtrC y˜y˜T , T −1 = trC(diag ρ) cov y a(·), x + y(x) − yc y(x) − yc (diag ρ)−1 , (2.44b) where “tr” denotes the trace of a matrix, diag ρ is the diagonal matrix diag ρ := (ρi δij ), yc := (yic ), see (2.41e), and y = y(x) := Eyi a(ω), x . Since Q ≤ trQ ≤ mQ for any positive deﬁnite m × m matrix Q, an upper bound of Eϕ(˜ y ) reads (Fig. 2.4) T Eϕ(˜ y ) ≤ mC (diag ρ)−1 2 cov y a(·), x + y(x) − yc y(x) − yc . (2.44c) Example 2.7. Assuming in the above Example 2.6 that C = diag (cii ) is a diagonal matrix with positive elements cii > 0, i = 1, . . . , m, then min ϕ(y) = min cii > 0, y∈ |B

1≤i≤m

(2.44d)

2.5 Approximation of Probabilities: Probability Inequalities

35

Fig. 2.4. Function ϕ = ϕ(y)

and Eϕ(˜ y ) is given by

Eϕ(˜ y) =

m

cii

i=1

=

m i=1

cii

2 E yi a(ω), x − yic ρ2i 2 σy2i a(·), x + y i (x) − yic ρ2i

.

(2.44e)

One-Sided Inequalities Suppose that exactly one of the two bounds yli < yui is inﬁnite for each i = 1, . . . , m. Multiplying the corresponding constraints in (2.41a) by −1, the admissible domain S = S(x), cf. (2.41b), can be represented always by S(x) = {a ∈ IRν : y˜i (a, x) < (≤) 0, i = 1, . . . , m} ,

(2.45a)

where y˜i := yi − yui , if yli = −∞, and y˜i := yli − yi , if yui = +∞. If we set y˜(a, x) := y˜i (a, x) and

then

˜ := y ∈ IRm : yi < (≤) 0, i = 1, . . . , m , B

(2.45b)

˜ . P S(x) = P y˜ a(ω), x ∈ B

(2.45c)

Consider also in this case, cf. (2.42d), (2.42e), a function ϕ : IRm → IR such that

36

2 Deterministic Substitute Problems in Optimal Decision

(i) ϕ(y) ≥ 0, y ∈ IRm ˜ (ii)ϕ(y) ≥ ϕ0 > 0, if y ∈| B.

(2.46a) (2.46b)

Then, corresponding to Theorem 2.1, we have this result: Theorem 2.2 (Markov-type inequality). If ϕ : IRm → IR is any (measurable) function fulﬁlling conditions (2.46a), (2.46b), then 1 Eϕ y˜ a(ω), x , (2.47) P y˜ a(ω), x < (≤) 0 ≥ 1 − ϕ0 provided that the expectation in (2.47) exists and is ﬁnite. Remark 2.3. Note that a related inequality was used already in Example 2.2. Example 2.8. If ϕ(y) :=

m

wi eαi yi with positive constants wi , αi , i =

i=1

1, . . . , m, then inf ϕ(y) = min wi > 0

(2.48a)

m Eϕ y˜ a(ω), x = wi Eeαi y˜i a(ω),x ,

(2.48b)

˜ y∈ |B

and

1≤i≤m

i=1

where the expectation in (2.48b) can be computed approximatively by Taylor expansion: αi y˜i −˜ y i (x) Eeαi y˜i = eαi y˜i (x) Ee 2 α2 ≈ eαi y˜i (x) 1 + i E yi a(ω), x − y i (x) 2

α2 = eαi y˜i (x) 1 + i σy2i (a(·),x) . (2.48c) 2 Supposing that yi = yi a(ω), x is a normal distributed random variable, then 1 2 2 ¯ α σ Eeαi y˜i = eαi y˜i (x) e 2 i yi (a(·),x) . (2.48d) Example 2.9. Consider ϕ(y) := (y − b)T C(y − b), where, cf. Example 2.6, C is a positive deﬁnite m × m matrix and b < 0 a ﬁxed m-vector. In this case we again have min ϕ(y) = min min ϕ(y) ˜ y∈ |B

1≤i≤m yi ≥0

and T Eϕ(˜ y ) = E y˜ a(ω), x − b C y˜ a(ω), x − b T = trCE y˜ a(ω), x − b y˜ a(ω), x − b .

2.5 Approximation of Probabilities: Probability Inequalities

Note that

y˜i (a, x) − bi =

37

yi (a, x) − (yui + bi ), if yli = −∞ yli − bi − yi (a, x), if yui = +∞,

where yui + bi < yui and yli < yli − bi . Remark 2.4. The one-sided case can also be reduced approximatively to the two-sided case by selecting a suﬃciently large, but ﬁnite upper bound y˜ui ∈ IR, lower bound y˜li ∈ IR, resp., if yui = +∞, yli = −∞. For multivariate Tschebyscheﬀ inequalities see also [75]. 2.5.3 First Order Reliability Methods (FORM) Special approximation methods for probabilities were developed in structural reliability analysis [19, 21, 54, 104] for the probability of failure pf = pf (x) given by (2.9c) with B := (0, +∞)my . Hence, see (2.2b), pf = 1 − P y a(ω), x > 0 . Applying certain probability inequalities, see Sects. 2.5.1 and 2.5.2, the case with m(>1) limit state functions is reduced ﬁrst to the case with one limit state function only. Approximations of pf are obtained then by linearization a, x) or by quadratic approximation of a transformed state function y˜i = y˜i (˜ ˜∗i (x) for each i = 1, . . . , m. at a certain “design point” a ˜∗ = a Based on the above procedure, according to (2.9f) we have pf (x) = P yi a(ω), x ≤ 0 for at least one index i, 1 ≤ i ≤ my ≤

my i=1

where

my P yi a(ω), x ≤ 0 = pf i (x),

(2.49a)

i=1

pf i (x) := P yi a(ω), x ≤ 0 .

(2.49b)

Assume now that the random ν-vector a = a(ω) can be represented [104] by a(ω) = U a(ω) , (2.50a) a= a(ω) is where U : IRν → IRν is a certain 1–1-transformation in IRν , and an N (0, I)-normal distributed random ν-vector, i.e., with mean vector zero and covariance matrix equal to the identity matrix I. Assuming that the νvector a = a(ω) has a continuous probability distribution and considering the conditional distribution functions Hi = Hi (ai |a1 , . . . , ai−1 )

38

2 Deterministic Substitute Problems in Optimal Decision

of ai = ai(ω), given aj (ω) = aj , j = 1, . . . , i − 1, then the random ν-vector z(ω) := T a(ω) deﬁned by the transformation z1 := H1 (a1 ) z2 := H2 (a2 |a1 ) .. . zν := Hν (aν |a1 , . . . , aν−1 ), cf. [128], has stochastically independent components z1 = z1 (ω), . . . , zν = zν (ω), and z = z(ω) is uniformly distributed on the ν-dimensional hypercube [0, 1]ν . Thus, if Φ = Φ(t) denotes the distribution function of the univariate N (0, 1)-normal distribution, then the inverse a ˜ = U −1 (a) of the transformation U can be represented by ⎞ ⎛ −1 Φ (z1 ) ⎟ ⎜ .. a ˜ := ⎝ ⎠ , z = T (a). . Φ−1 (zν )

In the following, let x be a given, ﬁxed r-vector. Deﬁning then a), x , a, x) := yi U ( yi ( we get

pf i (x) = P yi a(ω), x ≤ 0 , i = 1, . . . , my .

(2.50b)

(2.50c)

In the First Order Reliability Method (FORM), cf. [54], the state function a, x) is linearized at a so-called design point a∗ = a∗(i) . A design point yi = yi ( a∗(i) is obtained by projection of the origin a = 0 onto the limit state a∗ = a, x) = 0, see Fig. 2.5. Since yi ( a∗ , x) = 0, we get surface yi (

Fig. 2.5. FORM

2.5 Approximation of Probabilities: Probability Inequalities

T yi ( a, x) ≈ yi ( a∗ , x) + ∇a yi ( a∗ , x) ( a− a∗ ) T a∗ , x) ( a− a∗ ) = ∇a yi (

39

(2.50d)

and therefore

T ∗ ∗ ∇a yi ( a , x) ( a(ω) − a ) ≤ 0 = Φ −βi (x) . pf i (x) ≈ P

(2.50e)

Here, the reliability index βi = βi (x) is given by T ∇a yi ( a∗ , x) a∗ . βi (x) := − ∇a yi ( a∗ , x)

(2.50f)

Note that βi (x) is the minimum distance from a = 0 to the limit state surface a, x) = 0 in the a-domain. yi ( For more details and Second Order Reliability Methods (SORM) based on second order Taylor expansions, see Chap. 7 and [54, 104].

4 Deterministic Descent Directions and Eﬃcient Points

4.1 Convex Approximation According to Sects. 1.2 and 2.1 we consider here deterministic substitute problems of the type min F (x) s.t. x ∈ D (4.1a) with the convex feasible domain D and the expected total cost function (4.1b) F (x) := EG0 a(ω), x + Γ (x), where

Γ (x) := Eγ y a(ω), x .

(4.1c)

Here, G0 = G0 (a, x) is the primary cost function (e.g., setup, construction costs), and y = y(a, x) is the my -vector of state functions of the underlying structure/system/process. The function G0 = G0 (a, x) and the vector function y = y(a, x) depend on the ν-vector a of random model parameters and the r-vector x of decision/design variables. Moreover, γ = γ(y) is a loss function evaluating violations of the operating conditions y(a, x) ∈ B, see (2.1g). We assume in this chapter that the expectations in (4.1b), (4.1c) exist and are ﬁnite. Moreover, we suppose that the gradient of F (x), Γ (x) exists, and can be obtained, see formulas (2.23b), (2.23c) by interchanging diﬀerentiation and expectation: (4.2a) ∇F (x) = E∇x G0 a(ω), x + ∇Γ (x) with

T ∇Γ (x) = E∇x y a(ω), x ∇γ y a(ω), x .

(4.2b)

The main goal of this chapter is to present several methods for the construction of (1) deterministic descent directions h = h(x) for F (x), Γ (x) at

96

4 Deterministic Descent Directions and Eﬃcient Points

certain points x ∈ IRr , and (2) so-called eﬃcient points x0 of the stochastic optimization problem (4.1a)–(4.1c). Note that a descent direction for the function F at a point x is a vector h = h(x) such that F (x + th) < F (x) for 0 < t ≤ t0 with a constant t0 > 0. The descent directions h = h(x) to be constructed here depend not explicitly on the gradient ∇F, ∇Γ of F, Γ , but on some structural properties of F, Γ . Moreover, eﬃcient points x0 ∈ D are feasible points of (4.1a)–(4.1c) not admitting any feasible descent directions h(x0 ) at x0 . Hence, eﬃcient points x0 are candidates for optimal solutions x∗ of (4.1a)–(4.1c). For the construction of a descent direction h = h(x) at a point x = x0 the objective function F = F (x) is replaced ﬁrst [78] by the mean value function

T Fx0 (x) := E G0 a(ω), x0 + ∇x G0 a(ω), x0 (x − x0 ) + Γx0 (x), (4.3a) with

Γx0 (x) := Eγ y a(ω), x0 + ∇x y a(ω), x0 (x − x0 ) .

(4.3b)

follows from F by linearization of the primary cost function Obviously, Fx 0 G0 = G0 a(ω), x with respect to x at x0 , and by “inner linearization” of the expected recourse cost function Γ = Γ (x) at x0 (Fig. 4.1). Corresponding to (4.2b), the gradient ∇Fx0 (x) is given by

Fig. 4.1. Convex approximation Γx0 of Γ at x0

4.1 Convex Approximation

∇Fx0 (x) = ∇x G0 a(ω, x0 + ∇Γx0 (x)

97

(4.3c)

with T ∇Γx0 (x) = E∇x y a(ω), x0 ∇γ y a(ω), x0 + ∇x y a(ω), x0 (x − x0 ) . (4.3d) Under the general assumptions concerning F and Γ , the following basic properties of Fx0 hold: Theorem 4.1. (1) If γ is a convex cost function, then Γx0 and Fx0 are convex functions for each x0 . (2) At point x0 the following equations hold: Γx0 (x0 ) = Γ (x0 ), Fx0 (x0 ) = F (x0 ) ∇Γx0 (x0 ) = ∇Γ (x0 ), ∇Fx0 (x0 ) = ∇F (x0 ). (3) Suppose that γ fulﬁlls the Lipschitz condition γ(y) − γ(z) ≤ Ly − z, y, z ∈ B

(4.4a) (4.4b)

(4.4c)

with a constant L > 0 and a convex set B ⊂ IRmy . If y = y(a, x) and its linearization at x0 fulﬁll y a(ω), x ∈ B, y a(ω), x0 + ∇x y a(ω), x0 (x − x0 ) ∈ B a.s. (4.4d) for a point x, then |Γ (x) − Γx0 | ≤ Lx − x0 · E sup ∇x y a(ω), λx + (1 − λ)x0 0≤λ≤1 −∇x y a(ω), x0 . (4.4e) Proof. (a) The ﬁrst term of Fx0 is linear in x, and also the argument of γ in Γx0 is linear in x. Hence, Γx0 , Fx0 are convex for each convex loss function γ such that the expectations exist and are ﬁnite. (b) The equations in (4.4a), (4.4b) follow from (4.1b), (4.1c), (4.3a), (4.3b) and from (4.2a), (4.2b), (4.3c), (4.3d) by putting x = x0 . The last part follows by applying the mean value theorem for vector valued functions, cf. [29].

Remark 4.1. (a) If γ = γ(y) is a convex function on IRmy , then (4.4c) holds on each bounded closed subset B ⊂ IRmy with a local Lipschitz constant L = L(B). (b) An important class of convex loss functions γ having a global Lipschitz constant L = Lγ are the sublinear functions on IRmy . A function γ is called sublinear if it is positive homogeneous and subadditive, hence,

98

4 Deterministic Descent Directions and Eﬃcient Points

γ(λy) = λγ(y), λ ≥ 0, y ∈ IRmy γ(y + z) ≤ γ(x) + γ(z), y, z ∈ IR

my

.

(4.5a) (4.5b)

Obviously, a sublinear function is convex. Furthermore, deﬁning the norm γ of a sublinear function γ on IRmy by γ := sup γ(y) : y ≤ 1 , (4.5c) we have

γ(y) ≤ γ · y, y ∈ IRmy γ(y) − γ(z) ≤ γ · y − z, y, z ∈ IRmy .

(4.5d) (4.5e)

Further properties of sublinear functions may be found in [78]. In the following we suppose now that in (4.1b), (4.1c) diﬀerentiation and expectation can be interchanged. Using Theorem 4.1, for the construction of descent directions of F at a point x0 we may proceed as follows: Theorem 4.2. (a) Let x0 ∈ IRr be a given point. If h = h(x0 ) is a descent direction for Fx0 at x0 , then h is also a descent direction for F at x0 . (b) Suppose that z is a vector such that Fx0 (z) < Fx0 (x0 ). Then h = z − x0 is a descent direction of Fx0 at x0 . (c) If Γx0 (z) = Γx0 (x0 ) for z = x0 , andΓx0 is not constant on the line segment x0 z := λz + (1 − λ)x0 : 0 ≤ λ ≤ 1 , then h = z − x0 is a descent direction of Γx0 at x0 . Proof. (a) Using (4.4b), we have ∇F (x0 )T h = ∇Fx0 (x0 )T h < 0. Hence, h is also a descent direction of F at x0 . (b) Since Fx0 is a diﬀerentiable, convex function on IRmy , we obtain ∇Fx0 (x0 )T h = ∇Fx0 (x0 )T (z − x0 ) ≤ Fx0 (z) − Fx0 (x0 ) < 0. Thus, h is a descent direction of Fx0 at x0 . (c) In this case there is a vector w = λz + (1 − λ)x0 = x0 + λ(z − x0 ), 0 < λ < 1, such that Γx0 (w) < Γx0 (x0 ). Thus, ∇Γx0 (x0 )T λh = ∇Γx0 (x0 )T (w − x0 ) ≤ Γx0 (w) − Γx0 (x0 ) < 0 and therefore ∇Γx0 (x0 )T h < 0.

Deterministic descent directions h of F at x0 can be obtained then in two ways: (1) Separate linear inequality: Corollary 4.1. For a given point x0 ∈ IRr , suppose that there is a vector z ∈ IRr such that

4.1 Convex Approximation

99

T E∇x G0 a(ω), x0 (z − x0 ) < (≤)0

(4.6a)

Γx0 (z) < (=)Γx0 (x0 ),

(4.6b)

where in case “=” the function Γx0 is not constant on x0 z, or (4.6a) holds with “<.” Then h = z − x0 is a descent direction for F at x0 . Proof. The assertion follows from T Fx0 (z) − Fx0 (x0 ) = E∇x G0 a(ω), x0 (z − x0 ) + Γx0 (z) − Γx0 (x0 ).

(2) Enlarged loss function: Deﬁning the enlarged state function y˜ = y˜(a, x) and the enlarged loss function γ˜ = y˜(˜ γ ) by

t G0 (a, x) , γ˜ (˜ y ) = γ˜ := t + γ(y), (4.7a) y˜(a, x) := y(a, x) y we have

F (x) = E˜ γ y˜ a(ω), x .

(4.7b)

Deterministic descent directions may be found then by applying Theorems 4.1 and 4.2 directly to the representation (4.7a), (4.7b) of F . Obviously, the convex approximation Fx0 of F at x0 can be represented by Fx0 (x) = c0 + c¯T x + Eγ A(ω)x − b(ω) , (4.8a) where T c0 := EG0 a(ω), x0 − E∇x G0 a(ω), x0 x0 c¯ := E∇x G0 a(ω), x0 .

(4.8b) (4.8c)

Furthermore, A(ω), b(ω) is the random my (r + 1) matrix given by A(ω) := ∇x y a(ω), x0 b(ω) := − y a(ω), x0 − ∇x y a(ω), x0 x0 .

(4.8d) (4.8e)

4.1.1 Approximative Convex Optimization Problem According to the above of deterministic descent direc results, the construction tions of F (x) = EG0 a(ω), x + Eγ y a(ω), x at a point x0 , as well as the

100

4 Deterministic Descent Directions and Eﬃcient Points

approximate solution of the (nonconvex) optimization problem (4.1a)–(4.1c) can be reduced to a convex optimization problem min F (x) s.t. x ∈ D

(4.9a)

F (x) := c¯T x + Eu A(ω)x − b(ω) .

(4.9b)

with the convex objective

Here, c¯ is a given, ﬁxed r-vector of mean cost coeﬃcients, and A(ω), b(ω) is a random m × (r + 1) matrix. Furthermore, u = u(y) is a convex loss function such that the expectation in (4.9b) exists and is ﬁnite for all x under consideration. We still suppose that u has the following partial monotonicity property: yI ≤ zI , yII = zII ⇒ u(y) ≤ u(z) yI ≤ zI , yI = zI , yII = zII ⇒ u(y) < u(z),

(4.10a) (4.10b)

with a certain index set where y, z ∈ IRm , yI := (yi )i∈J , yII := (yi )i ∈J | J ⊂ {1, . . . , m}. Note that inequalities a ≤ b, a < b for vectors a, b ∈ IRm are deﬁned componentwise. The construction of descent directions for point x0 depends on F at a the type of the probability distribution of A(ω), b(ω) and the functional properties of u. Thus, we need the following deﬁnitions: Deﬁnition 4.1. The loss function u is called (partly) monotonous increasing or strictly (partly) monotonous increasing if u fulﬁlls (4.10a), (4.10b), respectively. Let C J , C JJ denote the set of all convex functions u on IRm fulﬁlling (4.10a), (4.10b), respectively. Moreover, let CˆJ , CˆJJ be the set of all strictly convex functions fulﬁlling (4.10a), (4.10b) respectively. In order to guarantee that the expectations under consideration exist and are ﬁnite, we take loss functions u from the following subclasses: Deﬁnition 4.2. Let C J (P ), C JJ (P ), CˆJ (P ), CˆJJ (P ) designate the set of all u ∈ C J , C JJ , CˆJ , CˆJJ resp., such that the expectation, the integral with respect to the probability measure P , (4.11) F0 (x) := Eu A(ω)x − b(ω) = u A(ω)x − b(ω) P (dω) J JJ ˆJ JJ , Csep , Csep , Cˆsep and exists and is ﬁnite for all x ∈ IRr . Finally, let Csep J JJ J JJ ˆ ˆ Csep (P ), Csep (P ), Csep (P ), Csep (P ) denote the subset of all separable elements m ' ui (zi )) of C J , . . . , CˆJJ and C J (P ), . . . , CˆJJ (P ), respectively. (i.e., u(z) := i=1

4.2 Computation of Descent Directions in Case of Normal Distributions

101

Note that C J ⊃ C JJ ⊃ CˆJJ and C J ⊃ CˆJ ⊃ CˆJJ , where corresponding inclusions hold also for the subsets of separable or integrable u. Solving now problem (4.9a), (4.9b), besides the well known diﬃculties in minimizing mean value functions, the loss function u should be exactly known. However, in practice there is always some uncertainty about the appropriate selection of u, e.g., due to diﬃculties in assigning appropriate penalty costs u(z) to the deviation z = A(ω)x − b(ω) between the output A(ω)x of the system x → A(ω)x and the target or upper bound b(ω). Consequently, in the following we want to modify [76, 77, 81, 83, 84, 90, 93] the approximate problem (4.9a), (4.9b) in such a way that we need only some obvious functional properties (e.g., monotonicity, convexity) of u. Using

certain moments of the x c(ω) ˜ , we ﬁnd new substituting random vector A(ω)x − ˜b(ω) := A(ω)x − b(ω) problems for (4.9a), (4.9b) and then also for(4.1a)–(4.1c) having the following ˜ ˜ properties: (1) Only certain moments of A(ω), b(ω) are needed and (2) compromise solutions, called eﬃcient solutions of (4.1a)–(4.1c), are obtained which are valid for a large class of loss functions u.

4.2 Computation of Descent Directions in Case of Normal Distributions In the following we suppose that the random m × (n + 1) matrix A(ω), b(ω) has a normal distribution with mean ¯ ¯b) := E A(ω), b(ω) (A, (4.12a) and covariance matrix

Q = cov A(·), b(·) = (Qij )i,j=1,...,m .

(4.12b)

The (n + 1) × (n + 1) matrix Qij := cov Ai (·), bi (·) , Aj (·), bj (·) = Qji ,

(4.12c)

denotes the covariance matrix of the ith and the jth row Ai (ω), bi (ω) , Aj (ω), bj (ω) of A(ω), b(ω) . Consequently, δ(ω, x) := A(ω)x − b(ω) is ¯ − ¯b and covariance matrix normally distributed with mean Ax Qx := cov A(·)x − b(·) = gij (x)

i,j=1,...,m

.

(4.13a)

102

4 Deterministic Descent Directions and Eﬃcient Points

The functions gij (x) are given by ˆT Qij x ˆ=x ˆT Qji x ˆ=x ˆT gij (x) = x

Qij + Qji x ˆ with x ˆ := 2

x ; −1

especially, the variance of δi (ω, x) = Ai (ω)x − bi (ω) is given by ˆT Qii x ˆ. gii (x) = var A(·)x − bi (·) = x

(4.13b)

(4.13c)

Derivative-free descent directions of the objective function F can be obtained, as developed in [83, 84, 90], by using the relations given below. For J (P ). In this case, with sake of simplicity, we assume ﬁrst that u ∈ Csep ¯ ¯ ¯ AI := (Ai )i∈J , AII := (Ai )i∈J , the relations read: c¯T x ≥ c¯T y A¯I x ≥ A¯I y, A¯II x = A¯II y gii (x) ≥ gii (y), i = 1, . . . , m.

(4.14a) (4.14b), (4.14c) (4.14d)

Theorem 4.3. (a) If any vectors x, y ∈ IRr fulﬁll relations (4.14a)–(4.14d), J (P ), (b) Let x, y ∈ IRr be related according then F (x) ≥ F (y) for each u ∈ Csep J JJ J (P ), Csep (P ), Cˆsep (P ), to (4.14a)–(4.14d), then F (x) > F (y) for each u ∈ Csep resp., if still the following additional condition holds: c¯T x > c¯T y A¯i x > A¯i y for some i ∈ J gii (x) > gii (y) for some i, 1 ≤ i ≤ m.

(4.14a ) (4.14b ) (4.14d )

Proof. According to the assumptions, ui is a (strictly) monotonous nondecreasing, (strictly) convex function for each i ∈ J. Furthermore, the objective F reads: m Eui A¯i x − ¯bi + gii (x)ξi (ω) , F (x) = cT x + i=1

where ξi = ξi (ω) is a standard normal distributed random variable. This representation of F yields then the assertion F (x) ≥ (>)F (y) in case of the relations (4.14a)–(4.14d), with “=” in (4.14d), and (4.14a ), (4.14b ). The proof for the general case (4.14d) and case (4.14d ) is contained in the proof of the next theorem.

Remark 4.2. (a) Obviously, relations (4.14a)–(4.14d), (4.14a ), (4.14b), (4.14d ), involving only ﬁrst and second order moments of A(ω), b(ω), c(ω) , are linear or quadratic in y, (b) The constraint “x ∈ D” in (4.1a) is taken into account by adding to (4.14a)–(4.14d) the condition y ∈ D, where in the following D may be any convex subset of IRr .

(4.14e)

4.2 Computation of Descent Directions in Case of Normal Distributions

103

For more general loss functions u ∈ C J (P ), C JJ (P ), CˆJ (P ), CˆJJ (P ), resp., relations (4.14a)–(4.14e) must be replaced by the following stronger conditions: c¯T x ≥ c¯T y A¯I x ≥ A¯I y, A¯II x = A¯II y Qx Qy (i.e., Qx − Qy is positive semideﬁnite) y ∈ D.

(4.15a) (4.15b), (4.15c) (4.15d) (4.15e)

Corresponding to Theorem 4.3, here we know the following result [83, 84]: Theorem 4.4. (a) If r-vectors x, y fulﬁll relations (4.15a)–(4.15d), then F (x) ≥ F (y) for each u ∈ C J (P ). (b) Let x, y ∈ IRr be related according to (4.15a)–(4.15d). Then F (x) > F (y) for each u ∈ C J (P ), C JJ (P ), CˆJ (P ), resp., if still the following additional condition holds: (4.15a ) (4.15b ) (4.15d )

c¯T x > c¯T y A¯i x > A¯i y for some i ∈ J Qx Qy , Qx = Qy .

T Proof. According to (4.9b), (4.11) we have F (x) = c¯ x + F0 (x) with F0 (x) = Eu A(ω)x − b(ω) , where u ∈ C J (P ), C JJ (P ), CˆJ (P ), respectively. Thus, if

cT y + F0 (x), and in the rest of (4.15a), (4.15a ), resp., holds, then F (x) ≥ (>)¯ the proof we have to consider F0 only. Deﬁning ¯ − ¯b) = A(ω) − A¯ x − b(ω) − ¯b , Yx = Yx (ω) := A(ω)x − b(ω) − (Ax we ﬁnd that

¯ − ¯b + Yx (ω) ≥ (>)Eu Ay ¯ − ¯b + Yx (ω) =: F˜0 , F0 (x) = Eu Ax

provided that (4.15b), (4.15b ) holds and u ∈ C J (P ), u ∈ C JJ (P ), respectively. Since Yx (ω) is a N (0, Qx )-normal distributed random m-vector, the charac(0) (0) (0) (0) teristic functions Pˆx , Pˆy of the distributions Px , Py of Yx , Yy , resp., are given by (0) Pˆx (z) := E exp iYx (ω)T z = exp − 12 z T Qx z , (0) Pˆy (z) := E exp iYy (ω)T z = exp − 12 z T Qy z , z ∈ IRm .

Supposing now that (4.15d), (4.15d ), resp., holds, we consider the characteristic function

1 ˆ K(z) := exp − z T (Qx − Qy )z , z ∈ IRm , 2

104

4 Deterministic Descent Directions and Eﬃcient Points

of the N (0, Qx − Qy )-normal distribution K on IRm . We ﬁnd ˆ Pˆy(0) (z)K(z) = Pˆx(0) (z), z ∈ IRm , (0)

(0)

(0)

(0)

˜(z) := hence, Px is the convolution Px = K ∗Py of K and Py . Deﬁning u ¯ − ¯b + z), z ∈ IRm , we obtain u(Ay ˜ Yx (ω) = u ˜(z)Px(0) (dz) = u ˜(z)(K ∗ Py(0) )(dz) F˜0 = E u (0) = u(v + w)K(dw)Py (dv) = J(v)Py(0) (dv), where J(v) :=

)

u ˜(v + w)K(dw), v ∈ IRm . For any u ∈ C J (P ) we have that

u ˜(v + w) − u ˜(v) ≥ g T (v + w − v) = g T w, w ∈ IRm , ˜(v). Consequently, for all v ∈ IRm and a subgradient [78] g ∈ ∂ u ˜(v) for all v ∈ IRm , J(v) ≥ u ˜(v) + g T w K(dw) = u and therefore

F˜0 =

J(v)Py(0) (dv) ≥

u ˜(v)Py(0) (dv) = F0 (y).

If u ∈ CˆJ (P ), then u ˜(v + w) − u ˜(v) > g T w, w = 0, ˜(v). In case of (4.15d ) we get for all v ∈ IRm and a subgradient g ∈ ∂ u ˜(v) for all v ∈ IRm , J(v) = u ˜(v + w)K(dw) > u ˜(v) + g T w K(dw) = u and therefore

F˜0 =

J(v)Py(0) (dv) >

u ˜(v)Py(0) (dv) = F0 (y),

which concludes the proof now. There are several suﬃcient conditions for (4.15d) implying (4.14d). Using, e.g., Gersgorin’s circle theorem, we ﬁnd that (4.15d), (4.15d ), is implied by gii (x) − gii (y) ≥

m gij (x) − gij (y), i = 1, 2, . . . , m,

(4.16a)

j=1 j=i

(4.16a) holds with gii (x) > gii (y) at least once, respectively. Clearly, (4.16a ) implies (4.14d ).

(4.16a )

4.2 Computation of Descent Directions in Case of Normal Distributions

105

Though, for practical purposes, (4.16a), (4.16a ), with given x, can be solved for y by certain search techniques, for analytical considerations ' we need the following representation of (4.16a): For each i = 1, . . . , m, let i designate the set of all (m − 1)-vectors σi = (σi1 , . . . , σii−1 , σii+1 , . . . , σim )T such that σij ∈ {−1, +1} for all j = i. Obviously, (4.16a), (4.16a ) is satisﬁed if and only if , i = 1, . . . , m, (4.16b) giσi (x) ≥ giσi (y) for all σi ∈ i

(4.16b) holds with gii (x) > gii (y) for some 1 ≤ i ≤ m,

(4.16b )

resp., where, cf. (4.13b), (4.13c), giσi (x) := gii (x) −

ˆ σij gij (x) = x ˆT Qii − σji Qij x

j=i

⎛

j=i

=x ˆT ⎝Qii −

j=i

⎞ Qij + Qji ⎠ σij x ˆ. (4.16c) 2

With a given vector x, condition (4.16b), (4.16b ), resp., may contain at most m2m−1 inequalities for y. However, in practice (4.16b), (4.16b ) contain much fewer constraints for y, since mostly some of the covariance matrices Qij (= QTji ), j = i, are zero. A very interesting situation occurs certainly if for a given i, 1 ≤ i ≤ m, Qii (σi ) := Qii −

j=i

σij

Qij + Qji is positive (semi) deﬁnite for all σi ∈ . i 2 (4.16d)

If (4.16d) holds, then cf. (4.16c), y giσi (y) = yˆT Qii (σi )ˆ ' is (strictly) convex for each σi ∈ i , see (4.16b). 4.2.1 Descent Directions of Convex Programs If n-vectors x, y are related such that F (x) ≥ F (y), y = x, then h = y − x is a descent direction for F at x, provided that F (x) = F (x) or F is not constant on the line segment xy = {λx + (1 − λ)y : 0 ≤ λ ≤ 1} joining x and y. This suggests the following deﬁnition: Deﬁnition 4.3. C J (P, D) := {u ∈ C J (P ) : F (x) = c¯T x+Eu A(ω)x − b(ω) is J J (P, D) := Csep (P ) ∩ not constant on arbitrary line segments xy of D}. Csep J C (P, D).

106

4 Deterministic Descent Directions and Eﬃcient Points

Obviously, we have F (x) = Fu (x) := Ev

c(ω)T x A(ω)x − b(ω)

with v

t := t + u(z). z

(4.17a)

By Lemma 2.2 in [84] we know then that F is constant on a line segment xy, y = x, if and only if F (x) = F (y) and (“a.s.” means with probability one) u λ A(ω)x − b(ω) + (1 − λ) A(ω)y − b(ω) = λu A(ω)x − b(ω) + (1 − λ)u A(w)y − b(w) a.s., 0 ≤ λ ≤ 1. (4.17b) For convex functions u being strictly convex with respect to a certain variable zι , 1 ≤ ι ≤ m, (4.17b) has the following consequences: Lemma 4.1. Let u ∈ C J (P ) be a loss function such that for some index 1 ≤ ι≤m u λz + (1 − λ)w < λu(z) + (1 − λ)u(w) if zι = wι and 0 < λ < 1. (4.17c) If (4.17b) holds, then Aι (ω)(y − x) = 0 a.s. and Rιι (y − x) = 0, where Rιι denotes the covariance matrix of Aι (ω). Corollary 4.2. Let u ∈ C J (P ) be a loss function fulﬁlling (4.17c). If Rιι is regular for some 1 ≤ ι ≤ m, then F = Fu is strictly convex on IRr . Thus, we have the following inclusion: Corollary 4.3. C J (P, D) ⊃ {u ∈ C J (P ): There exists at least one index ι, 1 ≤ ι ≤ m, such that Rιι is regular and (4.17c) holds}. J (P, D). Of course, a related inclusion holds also for Csep Now we have the following consequences from Theorems 4.3, 4.4:

Corollary 4.4 (Construction of descent directions of F at x without using derivatives of F ). J (P ), u ∈ C J (P ), respectively. If x, y = x, are related according (a) Let u ∈ Csep to (4.14a)–(4.14d), (4.15a)–(4.15d), resp., then h := y − x is a descent direction for F at x, provided that F is not constant on xy (e.g., c¯T x > c¯T y). (b) If x, y = x, fulﬁll relations (4.14a)–(4.14d), (4.15a)–(4.15d) and u ∈ JJ , C JJ , resp., then h = y − x is a descent direction for F at x, proCsep vided that A¯i x > A¯i , y for some i ∈ J. (c) Let x, y = x, fulﬁll (4.14a)–(4.14d), (4.15a)–(4.15d), and consider u ∈ J , CˆJ , respectively. Then h = y − x is a descent direction for F at x, Cˆsep provided that gii (x) > gii (y) for some i, 1 ≤ i ≤ m, Qx = Qy , respectively.

4.2 Computation of Descent Directions in Case of Normal Distributions

107

Remark 4.3. (a) Feasible descent directions h = y − x for F at x ∈ D are obtained if besides (4.14a)–(4.14d), (4.15a)–(4.15d) also (4.14e), (4.15e), resp., is taken into consideration. (b) Besides descent directions for F , in the following we are also looking for necessary optimality conditions for (4.9a), (4.9b) without using derivatives of F . Obviously, if x∗ is an optimal solution of (4.9a), (4.9b), then F admits no feasible descent directions at x∗ . Because of Theorems 4.3, 4.4, Corollary 4.4 and the above remark, we are looking now for solutions of (4.14a)–(4.14d) and (4.15a)–(4.15d). 1 Representing ﬁrst the symmetric (r + 1) × (r + 1) matrix (Qij + Qji ), 2 by

1 Rij dij (Qij + Qji ) = , (4.18a) d 2 ij T qij where Rij is a symmetric r × r matrix, dij ∈ IRr and qij ∈ IR, we get gij (x) = xT Rij x − 2xT dij + qij ,

(4.18b)

see (4.13b). Since gii is convex for each i = 1, . . . , m, we ﬁnd gii (y) ≥ gii (x) + ∇gii (x)T (y − x),

(4.18c)

where in (4.18c) the strict inequality “>” holds for y = x, if Rii is positive deﬁnite. Thus, since (4.15a)–(4.15d) implies (4.14a)–(4.14d), we have the following lemma: Lemma 4.2 (Necessary conditions for (4.14a)–(4.14d), (4.15a)– (4.15d)). If y = x fulﬁlls (4.14a)–(4.14d) or (4.15a)–(4.15d) with given x, then h = y − x satisﬁes the linear constraints c¯T h ≤ 0 A¯I h ≤ 0, A¯II h = 0

(4.19a) (4.19b), (4.19c)

(Rii x − dii )T h ≤ 0(< 0, if Rii is positive deﬁnite), i = 1, . . . , m.

(4.19d)

Remark 4.4. If the cone Cx of feasible directions for D at x ∈ D is represented (1) (2) (1) (2) by Cx = {h ∈ IRr \{0} : Ux h ≤ 0, Ux h < 0}, where Ux , Ux , resp., are (1) (2) (1) (2) νx × r, νx × r matrices with νx , νx ≥ 0, then the remaining relations (4.14e), (4.15e) can be described by Ux(1) h ≤ 0, Ux(2) h < 0. (1)

Note that Ux

(4.19e), (4.19f)

(2)

= 0 and νx = 0 if x ∈ int D (:= interior of D).

Using now theorems of the alternatives [74], for the existence of solutions h = 0 of (4.19a)–(4.19f), we obtain this characterization:

108

4 Deterministic Descent Directions and Eﬃcient Points

Lemma 4.3. (a) Suppose that Rii is positive deﬁnite for i ∈ K, where ∅ = (2) K ⊂ {1, . . . , m} or νx > 0. Then, either (4.19a)–(4.19f) has a solution h(= 0), or there exist multipliers |J|

ν (1)

ν (2)

γi ≥ 0, 0 ≤ i ≤ m, λI ∈ IR+ , λII ∈ IRm−|J| , π (1) ∈ IR+x , π (2) ∈ IR+x (4.20a) with (γi )i∈K = 0 or π (2) = 0 such that γ0 c¯ +

m

T T γi (Rii x − dii ) + A¯TI λI + A¯TII λII + Ux(1) π (1) + Ux(2) π (2) = 0.

i=1

(4.20b) (b) Suppose that Rii is singular for each i = 1, 2, . . . , m and = 0 (i.e., (2) Ux is canceled). Then either (4.19a)–(4.19f) has a solution h(= 0) such that in (4.19a)–(4.19f) the strict inequality “<” holds at least once in the (1) 1 + |J| + m + νx inequalities of (4.19a)–(4.19f), or there exist multipliers (2) νx

γi > 0, i = 0, 1, . . . , m, λI > 0, λII ∈ IRM −|J| , π (1) > 0

(4.20a )

such that (4.20b) holds. (2) (c) If νx = 0, then (4.19a)–(4.19f) has a solution h = 0*such that “=” holds everywhere in (4.19a)–(4.19f) if and only if rank x < r for the * (1) (1 + 2m + νx ) × r matrix x having the rows cT0 , Ai , i = 1, . . . , m, (Rii x − (1) (1) dii )T , i = 1, . . . , m, Uxj , j = 1, . . . , νx . 4.2.2 Solution of the Auxiliary Programs For any given x ∈ D, solutions y = x of (4.14a)–(4.14e), (4.15a)–(4.15e) can be obtained by random search methods, modiﬁed Newton methods, and by solving certain optimization problems related to (4.14a)–(4.14e), (4.15a)– (4.15e). Solution of (4.14a)–(4.14e). Here we have the program

where

min f (y) s.t. (4.14a)–(4.14c), (4.14e),

(4.21a)

f (y) := max gii (y) − gii (x) : 1 ≤ i ≤ m .

(4.21b)

Note. Clearly, also any other selection of functions from {cT y − cT x, A¯i y − A¯i x, i ∈ J, gii (y) − gii (x), i = 1, . . . , m} can be used to deﬁne the objective function f in (4.21a). Obviously, (4.21a), (4.21b) is equivalent to min t

(4.21a )

4.2 Computation of Descent Directions in Case of Normal Distributions

109

s.t. c¯T y ≤ c¯T x A¯I y ≤ A¯I x, A¯II y = A¯II x

(4.21b ) (4.21c ), (4.21d )

gii (y) − gii (x) ≤ t, i = 1, . . . , m

(4.21e )

y ∈ D.

(4.21f )

It is easy to see that (4.21a), (4.21b), (4.21a )–(4.21f ) are convex programs having for x ∈ D always the feasible solution y = x, (y, t) = (x, 0). Optimal solutions y ∗ , f (y ∗ ) ≤ 0, (y ∗ , t∗ ), t∗ ≤ 0, resp., exist under weak assumptions. Further basic properties of the above program are shown next: Lemma 4.4. Let x ∈ D be any given feasible point. (a) If y ∗ , (y ∗ , t∗ ) is optimal in (4.21a), (4.21b), (4.21a )–(4.21f ), resp., then y ∗ solves (4.14a)–(4.14d). (b) If y, (y, t) is feasible in (4.21a), (4.21b), (4.21a )–(4.21f ), then y solves (4.14a)–(4.14d), provided that f (y) ≤ 0, t ≤ 0, respectively. If y solves (4.14a)–(4.14d), then y, (y, t), t0 ≤ t ≤ 0, is feasible in (4.21a), (4.21b), (4.21a )–(4.21f ), resp., where t0 := f (y) ≤ 0. (c) (4.14a)–(4.14d) has the unique solution y = x if and only if (4.21a )– (4.21f ) has the unique optimal solution (y ∗ , t∗ ) = (x, 0). Proof. Assertions (a) and (b) are clear. Suppose now that (4.14a)–(4.14d) yields y = x. Obviously, (y ∗ , t∗ ) = (x, 0) is then optimal in (4.21a )–(4.21f ). Assuming that there is another optimal solution (y ∗∗ , t∗∗ ) = (x, 0), t∗ ≤ 0, of (4.21a )–(4.21f ), we know that y ∗∗ solves (4.14a)–(4.14d). Hence, we have y ∗∗ = x and therefore t∗∗ < 0 which is a contradiction to y ∗∗ = x. Conversely, assume that (4.21a )–(4.21f ) has the unique optimal solution (y ∗ , t∗ ) = (x, 0), and suppose that (4.14a)–(4.14d) has a solution y = x. Since (y, t), f (y) ≤ t ≤ 0, is feasible in (4.21a )–(4.21f ), we ﬁnd t∗ ≤ t ≤ 0 and therefore f (y) = 0. Thus, (y, 0) = (y ∗ , t∗ ) is also optimal in (4.21a )–(4.21f ),

in contradiction to the uniqueness of (y ∗ , t∗ ) = (x, 0). According to Lemma 4.4, solutions y of system (4.14a)–(4.14d) can be determined by solving (4.21a )–(4.21f ). In the following we assume that D = x ∈ IRr : g(x) ≤ g0 , (4.22) T where g(x) = gi (x), . . . , gν (x) is a ν-vector of convex functions on IRr , g0 ∈ Rν . Furthermore, suppose that (4.21a )–(4.21f ) fulﬁlls the Slater condition for each x ∈ D. This regularity condition holds, e.g., (consider (y, t) := (x, 1)) if g(y) := Gy with a ν × r matrix G.

(4.23)

110

4 Deterministic Descent Directions and Eﬃcient Points

The necessary and suﬃcient optimality conditions for (4.21a )–(4.21f ) read: m '

γi = 1

(4.24a)

i=1

m ' i=1

γi (Rii y − dii ) = − 12 γ0 c + A¯TI λI + A¯TII λII +

∂g T ∂x (y) µ

(4.24b)

gii (y) − gii (x) − t ≤ 0, i = 1, . . . , m γi gii (y) − gii (x) − t = 0, γi ≥ 0, i = 1, . . . , m

(4.24d)

c¯T y − c¯T x ≤ 0, γ0 (¯ cT y − c¯T x) = 0, γ0 ≥ 0

(4.24e)

A¯I y − A¯I x ≤ 0, λTI (A¯I y − A¯I x) = 0, λI ≥ 0 (λI ∈ IR|J| )

(4.24f)

A¯II y − A¯II x = 0, λII ∈ IR g(y) − g0 ≤ 0, µT g(y) − g0 = 0, µ ≥ 0 (µ ∈ IRν ).

(4.24g)

m−|J|

(4.24c)

(4.24h)

Fundamental properties of optimal solutions of (4.21a )–(4.21f ): For given x ∈ D, let (y ∗ , t∗ ) be optimal in (4.21a )–(4.21f ). Hence, t∗ ≤ 0, (y ∗ , t∗ ) is characterized by (4.24a)–(4.24h), and y ∗ solves (4.14a)–(4.14d), see Lemma 4.4a. Moreover, one of the following cases will occur, see Corollary 4.4: Case i: t∗ < 0. Here we have gii (y ∗ ) < gii (x), 1 ≤ i ≤ m. Thus, y ∗ = x, and h = y ∗ −x is a J J (P ) or u ∈ Csep (P ) feasible descent direction for F at x, provided that u ∈ Cˆsep ∗ and F = Fu is not constant on xy . Case ii.1: t∗ = 0 and gii (y ∗ ) < gii (x) for some 1 ≤ i ≤ m or c¯T y ∗ < c¯T x ¯ or Ai y ∗ < A¯i x for some i ∈ J. Here, y ∗ = x, and h = y ∗ − x is a feasible descent direction for F at x if J J JJ (P ), u ∈ Csep (P ), u ∈ Csep (p), respectively. u ∈ C˜sep ∗ Case ii.2: t = 0 and gii (y ∗ ) = gii (x), 1 ≤ i ≤ m, c¯T y ∗ = c¯T x, A¯i y ∗ = A¯i x, i ∈ J, with y ∗ = x. J (P ) Then h = y ∗ −x is a feasible descent direction for F at x for all u ∈ Csep ∗ such that F = Fu is not constant on xy . Case ii.3: t∗ = 0 and (y ∗ , t∗ ) = (x, 0) is the unique optimal solution of (4.21a )–(4.21f ). According to Lemma 4.4c, in Case ii.3 we know here that (4.14a)–(4.14d) has only the trivial solution y = x. Hence, the construction of a descent direction h = y − x by means of (4.14a)–(4.14d) and Corollary 4.4 fails at x. Note that points x ∈ D having this property are candidates for optimal solutions of (4.9a), (4.9b). Solution of (4.15a)–(4.15e). Let x ∈ D be a given feasible point. Replacing for simplicity (4.15d) by (4.16a), for handling system (4.15a)–(4.15e), we get the following optimization problem min fˆ(y) s.t. (4.15a)–(4.15c), (4.15e),

(4.25a)

4.2 Computation of Descent Directions in Case of Normal Distributions

111

where

⎫ ⎧ ⎬ ⎨ fˆ(y) := max gii (y) − gii (x) + gij (x) − gij (y) : 1 ≤ i ≤ m (4.25b) ⎭ ⎩ j=i ' = max {giσi (y) − giσi (x) : σi ∈ i , 1 ≤ i ≤ m} ,

see the note to program (4.21a), (4.21b). In the above, giσi is deﬁned by (4.16c). An equivalent version of (4.25a), (4.25b) reads min t

(4.25a )

s.t. (4.25b ) c¯T y ≤ c¯T x ¯ (4.25c ), (4.25d ) A¯I y ≤ A¯I x, A¯II y = A II x ' gii (y) − gii (x) + gij (x) − gij (y) ≤ t, i = 1, . . . , m (4.25e ) j=i

y ∈ D,

where condition (4.25e ) has also the equivalent form ' giσi (y) − giσi (x) ≤ t for all σi ∈ i , i = 1, . . . , m.

(4.25f )

(4.25e )

It is easy to see that, for given x ∈ D, program (4.25a), (4.25b), (4.25a )– (4.25f ), resp., has always the trivial feasible solution y = x, (y, t) = (x, 0). Moreover, optimal solutions y ∗ , (y ∗ , t∗ ) of (4.25a), (4.25b), (4.25a )–(4.25f ), resp., exist under weak assumptions, where t∗ = fˆ(y ∗ ) ≤ 0. If gii (y ∗ ) = gii (x) for an index i, then gij (y ∗ ) = gij (x) for all j = 1, . . . , m and f (y ∗ ) = t∗ = 0. If condition (4.16d) holds for every i = 1, . . . , m, then (4.25a), (4.25b), (4.25a )– (4.25f ) are convex. Further properties of (4.25a), (4.25b), (4.25a )–(4.25f ) are given in the following: Lemma 4.5. Let x ∈ D be a given point and consider system (4.15a)–(4.15c), (4.15e), (4.16a). (a) If y ∗ , (y ∗ , t∗ ) is optimal in (4.25a), (4.25b), (4.25a )–(4.25f ), resp., then y ∗ solves (4.15a)–(4.15c), (4.15e), (4.16a). (b) If y, (y, t) is feasible in (4.25a), (4.25b), (4.25a )–(4.25f ), then y solves (4.15a)–(4.15c), (4.15e), (4.16a) provided that fˆ(y) ≤ 0, t ≤ 0, respectively. If y solves (4.15a)–(4.15c), (4.15e), (4.16a), then y, (y, t), t0 ≤ t ≤ 0, is feasible in (4.25a), (4.25b), (4.25a )–(4.25f ), resp., where t0 := fˆ(y) ≤ 0. (c) (4.15a)–(4.15c), (4.15e), (4.16a) has the unique solution y = x if and only if program (4.25a )–(4.25f ) has the unique optimal solution (y ∗ , t∗ ) = (x, 0). (d) If (y, t) is feasible in (4.25a )–(4.25f ), then (y, t) is also feasible in (4.21a )–(4.21f ).

112

4 Deterministic Descent Directions and Eﬃcient Points

Proof. Assertions (a), (b), (d) are clear, and (c) can be shown as the corresponding assertion (c) in Lemma 4.4.

As was shown above, (4.15a)–(4.15c), (4.15e), (4.16a) can be solved by means of (4.25a )–(4.25f ). If (4.25a )–(4.25f ) is convex, see (4.16d), then we may proceed as in (4.21a )–(4.21f ). Indeed, if D is given by (4.22), and we assume that the Slater condition holds, then the necessary and suﬃcient conditions for an optimal solution (y ∗ , t∗ ) of (4.25a )–(4.25f ) read: m '

m '

' ' γiσi

i=1 σi ∈

"

' ' γiσi = 1

i=1 σi ∈

Rii −

' j=i

(4.26a)

# ' σij Rij y − dii − σij dij i

j=i

∂g (y)T µ (4.26b) = − 12 γ0 c + A¯TI λI + A¯TII λII + ∂x ' (4.26c) giσi (y) − giσi (x) − t ≤ 0, σi ∈ i , i = 1, . . . , m ' γiσi giσi (y) − giσi (x) − t = 0, γiσi ≥ 0, σi ∈ i , i = 1, . . . , m (4.26d) i

cT y − c¯T x) = 0, γ0 ≥ 0 c¯T y ≤ c¯T x, γ0 (¯ A¯I y − A¯I x ≤ 0, λTI (A¯I y − A¯I x) = 0, λI ≥ 0 A¯II y = A¯II x, λII ∈ IRm−|J| g(y) − g0 ≤ 0, µT g(y) − g0 = 0, µ ≥ 0.

(4.26e) (4.26f) (4.26g) (4.26h)

In the non-convex case the conditions (4.26a)–(4.26h) are still necessary for an optimal solution (y ∗ , t∗ ) of (4.25a )–(4.25f ), provided that (y ∗ , t∗ ) fulﬁlls a certain regularity condition, see, e.g., [47]. Fundamental properties of optimal solutions of (4.25a )–(4.25f ): For given x ∈ D, let (y ∗ , t∗ ) be an optimal solution of (4.25a )–(4.25f ). According to Lemma 4.5(a) we know that y ∗ solves then (4.15a)–(4.15c), (4.15e), (4.16a) and therefore also (4.15a)–(4.15e). Moreover, one of the following diﬀerent cases will occur, cf. Corollary 4.4: Case i: t∗ < 0. This yields gii (y ∗ ) < gii (x), i = 1, . . . , m, and Qx Qy∗ . Thus, y ∗ = x, and h = y ∗ − x is a feasible descent direction for F at x, provided that u ∈ CˆJ (P ) or u ∈ C J (P ) and F = Fu is not constant on xy ∗ . Case ii.1: t∗ = 0 and gii (y ∗ ) < gii (x) for an index i, i = 1, . . . , m, or T ∗ c¯ y < c¯T x or A¯i y ∗ < A¯i x for some i ∈ J. Then y ∗ = x, and h = y ∗ − x is a feasible descent direction for F at x, provided that u ∈ CˆJ (P ), u ∈ C J (P ), u ∈ C JJ (P ), respectively. Case ii.2: t∗ = 0 and Qy∗ = Qx , c¯T y ∗ = c¯T x, A¯i y ∗ = A¯i x, i ∈ J, with ∗ y = x.

4.3 Eﬃcient Solutions (Points)

113

Here, h = y ∗ − x is a feasible descent direction for all u ∈ C J (P ) such that F = Fu is not constant on xy ∗ . Case ii.3: t∗ = 0 and (y ∗ , t∗ ) = (x, 0) is the unique optimal solution of (4.25a )–(4.25f ). According to Lemma 4.5(c) we know that (4.15a)–(4.15c), (4.15e), (4.16a) has then only the trivial solution y = x. In this case the construction of a descent direction h = y − x for F by means of (4.15a)–(4.15c), (4.15e), (4.16a) and Corollary 4.4 fails at x. Points of this type are candidates for optimal solutions of (4.9a), (4.9b), see Sect. 4.3. Lemmas 4.5(d) and (4.18c) yield still the auxiliary convex program min t

(4.27a)

s.t. (4.27b) c¯T y ≤ c¯T x ¯ ¯ ¯ (4.27c), (4.27d) AI y ≤ AI x, AII y = AII x (4.27e) 2(Rii x − dii )T (y − x) − t ≤ 0, i = 1, . . . , m y ∈ D. (4.27f) Obviously, for given x ∈ D, each feasible solution (y, t) of (4.21a )–(4.21f ) or (4.25a )–(4.25f ) is also feasible in (4.27a)–(4.27f). If (4.25a )–(4.25f ) is convex, then (4.27e) can be sharpened, see (4.16d), (4.18a), (4.25e ), as follows: , T ' ' σij Rij x − dii − σij dij (y − x) − t ≤ 0, 2 Rii − +

j=i

σi ∈

'

j=i i,

1 ≤ i ≤ m.

(4.27e )

4.3 Eﬃcient Solutions (Points) Instead of solving approximatively the complicated stochastic optimization problem (4.9a), (4.9b), where one has not always a unique loss function u, we may look for an alternative solution concept: Obvious alternatives for optimal J J JJ ˆJJ (Cˆsep , Csep , Csep ), u ∈ solution of (4.9a), (4.9b) with a loss function u ∈ Csep J ˆJ JJ ˆJJ 0 C (C , C , C ), resp., are just the points x ∈ D such that the above procedure for ﬁnding a feasible descent direction for F = Fu fails at x0 . Based on (4.14a)–(4.14e), (4.15a)–(4.15e), resp., and Corollary 4.4, eﬃcient solutions are deﬁned as follows: J -eﬃcient solution of (4.9a), Deﬁnition 4.4. A vector x0 ∈ D is called a Csep (4.9b) if there is no y ∈ D such that

114

4 Deterministic Descent Directions and Eﬃcient Points

c¯T x0 ≥ c¯T y A¯I x ≥ A¯I y, A¯II x0 = A¯II y gii (x0 ) ≥ gii (y), i = 1, . . . , m 0

(4.28a) (4.28b), (4.28c) (4.28d)

and (¯ cT x0 , A¯I x0 , gii (x0 ), 1 ≤ i ≤ m) = (¯ cT y, A¯I y, gii (y), 1 ≤ i ≤ m)

(4.28e)

J x0 ∈ D is called a strongly (or uniquely) Csep -eﬃcient solution of (4.9a), (4.9b) if there is no vector y ∈ D, y = x0 , such that (4.28a)–(4.28d) holds. st J J , resp., denote the set of all Csep -strongly Csep -eﬃcient solutions Let Esep , Esep of (4.9a), (4.9b). st are the Pareto-optimal solutions We observe that the elements of Esep , Esep [25, 134] of the vector optimization problem “min” c¯T y, A¯I y−¯bI , A¯II y−¯bII , ¯bII − A¯II y, gii (y), 1 ≤ i ≤ m . (4.28 ) y∈D

Deﬁnition 4.5. A vector x0 ∈ D is called a C J -eﬃcient solution of (4.9a), (4.9b) if there is no y ∈ D such that c¯T x0 ≥ c¯T y (4.29a) (4.29b), (4.29c) A¯I x ≥ A¯I y, A¯II x0 = A¯II y ' (4.29d) giσi (x0 ) ≥ giσi (y), σi ∈ i , i = 1, . . . , m 0

and (¯ cT x0 , A¯I x0 , gii (x0 ), 1 ≤ i ≤ m) = (¯ cT y, A¯I y, gii (y), 1 ≤ i ≤ m).

(4.29e)

x0 ∈ D is called a strongly (or uniquely) C J -eﬃcient solution of (4.9a), (4.9b) if there is no y ∈ D, y = x0 , such that (4.29a)–(4.29d) holds. Let E, E st , resp., denote the set of all C J -, strongly C J -eﬃcient solutions of (4.9a), (4.9b). Note that the elements of E, E st are the Pareto-optimal solutions of the multi-objective program ' “min” c¯T y, A¯I y − ¯bI , A¯II y − ¯bII , ¯bII − A¯II y, giσ (y), σi ∈ ,1 ≤ i ≤ m . i

y∈D

i

(4.29 )

Discussion of Deﬁnitions 4.4, 4.5: Relations (4.28a)–(4.28d) and (4.14a)–(4.14d) agree. Furthermore, (4.29a)– (4.49d) and (4.15a)–(4.15c), (4.16b) coincide, where (4.16b) is equivalent to (4.16a). We have the inclusions st ⊂ E , E st ⊂ E, Esep sep where equality in (4.30a) holds under the following conditions:

(4.30a)

4.3 Eﬃcient Solutions (Points)

115

Lemma 4.6. (a) If gii is strictly convex for at least one index i, 1 ≤ i ≤ m, st = E . then Esep sep ' (b) If giσi is convex for all σi ∈ i , i = 1, 2, . . . , m, and gii is strictly convex for at least one index i, 1 ≤ i ≤ m, then E st = E. (c) If for arbitrary x, y ∈ D ¯ = Ay, ¯ gii (x) = gii (y), i = 1, . . . , m, x = y ⇔ c¯T x = c¯T y, Ax

(4.30b)

st = E st then Esep sep and E = E. Proof. (a) We may suppose, e.g., that g11 is strictly convex. Let then x0 ∈ Esep st . Hence, there exists y ∈ D, y = x0 , such that | sep and assume that x0 ∈E (4.28a)–(4.28d) holds. Since all gii are convex and g11 is strictly convex, it is easy to see that η := λy 0 + (1 − λ)y fulﬁlls (4.28a)–(4.28e) for each 0 < λ < 1. st and therefore E st = E , cf. (4.30a). AsThis contradiction yields x0 ∈ Esep sep sep sertion (b) follows in the same way. In order to show (c), let x0 ∈ Esep , x0 ∈ E, st , x0 ∈E | sep | st . Consider y ∈ D, y = x0 , such that resp., and assume x0 ∈E (4.28a)–(4.28d), (4.29a)–(4.29d), resp., holds. Because of x0 ∈ Esep , x0 ∈ E, resp., we have c¯T x0 , A¯I x0 , gii (x0 ), 1 ≤ i ≤ m = c¯T y, A¯I y, gii (y), 1 ≤ i ≤ m . This equation, (4.28c), (4.29c), resp., and (4.30b) yield the contradiction y = st , x0 ∈ E st , resp., and therefore E st and E st = E.

x0 . Thus, x0 ∈ Esep sep J −, strongly According to Lemmas 4.4(c), 4.5(c), a point x0 ∈ D is strongly Csep J C -eﬃcient if and only if (4.21a )–(4.21f ), (4.25a )–(4.25f ), resp., with x := x0 , has the unique optimal solution (y ∗ , t∗ ) = (x0 , 0). Since (4.29d), being equivalent to (4.16a), implies (4.28d), we ﬁnd

st ⊂ E st . Esep ⊂ E, Esep A special subset of eﬃcient solutions of (4.9a), (4.9b) is given by E0 := x0 ∈ D : (4.19a)–(4.49f) has no solution h = 0

(4.30c)

(4.31)

Indeed, Lemma 4.2 and Remark 4.4 yield the following inclusion: st . Corollary 4.5. E0 ⊂ Esep By Lemma 4.3 we have this parametric representation for E0 : Corollary 4.6. (a) If Rii is positive deﬁnite for at least one index 1 ≤ i ≤ m, then x ∈ E0 if and only if x ∈ D and there are parameters γi , i = 0, 1, . . . , m, λI , λII and π (1) , π (2) such that (4.20a), (4.20b) holds.

116

4 Deterministic Descent Directions and Eﬃcient Points (2)

(b) Let Rii be singular for each i = 1, . . . , m. If νx > 0 for each x ∈ D, then x ∈ E0 if and only if x ∈ D and there exist γi , 0 ≤ i ≤ m, λI , λII , π (1) , π (2) (2) such that (4.20a), (4.20b) holds. If νx = 0, *x ∈ D, then x ∈ E0 if and * only if x ∈ D, rank x = n (with the matrix x deﬁned in Lemma 4.3(c)) or there are parameters γi , 0 ≤ i ≤ m, λI , λII and π (1) such that (4.20a ), (4.20b) holds. 4.3.1 Necessary Optimality Conditions Without Gradients As was already discussed in Sect. 4.2.2, Cases ii.3, eﬃcient solutions are candidates for optimal solutions x∗ of (4.9a), (4.9b). This will be made more precise in the following: JJ (P ), u ∈ CˆJJ (P ). Theorem 4.5. If x∗ ∈ argminx∈D F (x) with any u ∈ Cˆsep ∗ ∗ then x ∈ Esep , x ∈ E, respectively.

Proof. Supposing that x∗ ∈E | sep , x∗ ∈E, | resp., we ﬁnd y ∈ D such that (4.28a)– (4.28e), (4.29a)–(4.29e) holds, where x0 := x∗ . Thus, (4.14a)–(4.14e), (4.15a)– (4.15c), (4.15e), (4.16a), resp., hold. Moreover at least one of the relations (4.14a ), (4.14b ), (4.14d ), (4.15a ), (4.15b ) and (4.16b ), resp., hold with x := x∗ . Consequently, according to Corollary 4.4 and Remark 4.3a we have a feasible descent direction for F at x∗ . This contradiction to the optimality

of x∗ yields the assertion. J (P ), C J (P ) by certain elements of Approximating the elements of Csep JJ Csep (P ), C (P ), resp., we ﬁnd the next condition:

ˆJJ

Theorem 4.6. If D is compact or D is closed and F (x) → +∞ as x → J (P ), u ∈ C J (P ) there exists x∗ ∈ argmin F (x) +∞, then for each u ∈ Csep contained in the closed hull E¯sep , E¯ of Esep , E, respectively. J JJ (P ), u ∈ C J (P ), and deﬁne uρ ∈ Cˆsep (P ), uρ ∈ Proof. Consider any u ∈ Csep JJ ˆ C (P ), resp., by

uρ (z) := u(z) + ρ

m

vi (zi ),

z ∈ IRm ,

i=1

where ρ > 0 and vi = vi (zi ) is deﬁned by | vi (zi ) = exp(zi ), i ∈ J, and vi (zi ) = zi2 for i ∈J.

If D is compact, then for each ρ > 0 we ﬁnd x∗ρ ∈ argminx∈D Fρ (x), where Fρ (x) := c¯T x + Euρ A(ω)x − b(ω) , see (4.9b). Furthermore, since D is

4.3 Eﬃcient Solutions (Points)

117

vi A(ω)x − bi (ω) is convex and therefore continuous i=1 on D, there is a constant C > 0 such that F (x) − Fρ (x) ≤ ρC for all x ∈ D. This yields minx∈D F (x) − minx∈D Fρ (x) = minx∈D F (x) − Fρ (x∗ρ ) ≤ ρC. Hence, Fρ (x∗ρ ) → minx∈D F (x) as ρ ↓ 0. Furthermore, we have F (x∗ρ ) − Fρ (x∗ρ ) ≤ ρC, ρ > 0, and for any accumulation point x∗ of (x∗ρ ) we get compact and x → E

m '

F (x∗ ) − min F (x) ≤ F (x∗ ) − F (x∗ρ ) + F (x∗ρ ) − Fρ (x∗ρ ) + Fρ (x∗ρ ) x∈D − min F (x) ≤ F (x∗ ) − F (x∗ρ ) + 2ρC, ρ > 0. x∈D

Consequently, since x∗ρ ∈ Esep , x∗ρ ∈ E, resp., for all ρ > 0, cf. Theorem 4.5, ¯ for every accumulation point x∗ of (x∗ρ ) we ﬁnd that x∗ ∈ E sep , x∗ ∈ E, ∗ resp., and F (x ) = minx∈D F (x). Suppose now that D is closed and F (x) → +∞ as x → +∞. Since Fρ (x) ≥ F (x), x ∈ IRn , ρ > 0, we also have that Fρ (x) → +∞ for each ﬁxed ρ > 0. Hence, there exist optimal solutions x∗0 ∈ argminx∈D F (x) and x∗ρ ∈ argminx∈D Fρ (x) for every ρ > 0. For all 0 < ρ ≤ 1 and with an arbitrary, but ﬁxed x ˜ ∈ D we ﬁnd then x∗ρ ∈ Esep , x∗ρ ∈ E, resp., and x) ≤ F1 (˜ x), 0 < ρ ≤ 1. F (x∗ρ ) ≤ Fρ (x∗ρ ) ≤ Fρ (˜ Thus, (x∗ρ ) must lie in a bounded set. Let R := max x∗0 , R0 , where ˜ := x ∈ D : x ≤ R , then R0 denotes a norm bound of (x∗ρ ). If D ˜ minx∈D F (x) = minx∈D˜ F (x) and minx∈D Fρ (x) = minx∈D˜ Fρ (x). Since D is compact, the rest of the proof follows now as in the ﬁrst part. J (P, D), C J (P, D), see Deﬁnition 4.3, we have the folFor elements of Csep lowing condition: J (P, D), u ∈ Theorem 4.7. If x∗ ∈ argminx∈D F (x) with any u ∈ Csep J ∗ ∗ st st C (P, D), then x ∈ Esep , x ∈ E , respectively. J st . According to (P, D) and assume that x∗ ∈E Proof. Let u ∈ Csep | sep ∗ Deﬁnition 4.4 we ﬁnd then y ∈ D, y = x , such that (4.28a)–(4.28d) hold with x0 := x∗ . Corollary 4.4a yields that h = y − x∗ is a feasible descent direction for F at x∗ which is a contradiction to the optimality of x∗ . Hence, st . The rest follows in the same way.

x∗ ∈ Esep

4.3.2 Existence of Feasible Descent Directions in Non-Eﬃcient Solutions of (4.9a), (4.9b) The preceding Theorems 4.3, 4.4 and Corollary 4.4 yield this ﬁnal result:

118

4 Deterministic Descent Directions and Eﬃcient Points

Corollary 4.7. (a) Let x0 ∈E, | x0 ∈E | sep , respectively. Then there exists y ∈ D, y = x0 , such that (4.29a)–(4.29e), (4.28a)–(4.28e), resp., holds. ConseJ (P ), resp., and F (y) < quently, F (y) ≤ F (x0 ) for all u ∈ C J (P ), u ∈ Csep 0 JJ JJ F (x ) for all u ∈ Cˆ (P ), u ∈ Cˆsep (P ), respectively. Hence, h = y − x0 is a J (P ), and for feasible descent direction for F at x0 for all u ∈ CˆJJ (P ), u ∈ Cˆsep J J all u ∈ C (P ), u ∈ Csep (P ), resp., such that F is not constant on x0 y. (b) Let st , respectively. Then there is y ∈ D, y = x0 , such that (4.29a)– x0 ∈E | st , x0 ∈E | sep (4.29d), (4.28a)–(4.28d), resp., holds. Hence, F (y) ≤ F (x0 ), and h = y − x0 J (P ), such is a feasible descent direction for F at x0 for all u ∈ C J (P ), u ∈ Csep 0 that F is not constant on x y.

4.4 Descent Directions in Case of Elliptically Contoured Distributions In extension of the results in Sect. 4.2, here we suppose that the random m × (r + 1) matrix A(ω), b(ω) has an elliptically contoured probability distribution, cf. [56,119,155]. Members of this family are, e.g., the multivariate normal, Cauchy, stable, Student t-, inverted Student t-distribution as also their truncations to elliptically contoured sets in IRm(r+1) and the uniform distribution on elliptically contoured sets in IRm(r+1) . The parameters of this class can be described by an integer order parameter s ≥ 1 and an m(r + 1) location matrix Ξ = (Θ, θ) (4.32a) Moreover, there are positive semideﬁnite m(r + 1) × m(r + 1) scale matrices , σ = 1, . . . , s, (4.32b) Q(σ) = Qij (σ) i,j=1,...,m

having (r + 1) × (r + 1) submatrices Qij such that Qji = QTij , i, j = 1, . . . , m.

(4.32c)

In the following M ∈ IRm(r+1) is always represented by an m × (r + 1) matrix. =P (M ) of this class of - - The characteristic function P A(·),b(·)

A(·),b(·)

distributions is then given by - P

T (M ) := E exp itrM A(ω), b(ω)

A(·),b(·)

s ' = exp(itrM Ξ T ) exp − , g vecM Q(σ)(vecM )T σ=1

(4.33a)

4.4 Descent Directions in Case of Elliptically Contoured Distributions

119

where vecM := (M1 , M2 , . . . , Mm )

(4.33b)

denotes the m(r + 1)-vector of the rows M1 , . . . , Mm of M . Furthermore, g = g(t) is a certain nonnegative 1–1-function on IR+ . E.g., in the case of multivariate symmetric stable distributions of order s and characteristic exponent α, 1 0 < α ≤ 2, we have g(t) := tα/2 , t ≥ 0, see [119]. 2 Consequently, the probability distribution PA(·)x−b(·) of the random mvector x (4.34a) A(ω)x − b(ω) = A(ω), b(ω) x ˆ with x ˆ := −1 may be represented by the characteristic function -A(·)x−b(·) (z) = E exp iz T A(ω)x − b(ω) P T - =P = E exp itr(z x ˆT ) A(ω), b(ω)

(z x ˆT )

A(·),b(·)

s T = exp iz (Θx − θ) exp − g z Qx (σ)z . (4.34b) T

σ=1

In (4.34b) the positive semideﬁnite m × m matrices Qx (σ) are deﬁned by Qx (σ) := x ˆT Qij (σ)ˆ x , σ = 1, . . . , s, (4.34c) i,j=1,...,m

cf. (4.32b), (4.32c) and (4.13a)–(4.13c). Corresponding to Sect. 4.2, in the following we consider again the derivative-free construction of (feasible) descent directions h = y − x for the objective function F of the convex optimization problem (4.9a), (4.9b), hence, F (x) = c¯T x + F0 (x) where

F0 (x) = Eu A(ω)x − b(ω) with u ∈ C J (P ).

Following the proof of Theorem 4.4, for a given vector x and a vector y = x to be determined, we consider the quotient of the characteristic functions of A(ω)x − b(ω) and A(ω)y − bω), see (4.34a), (4.34b): -A(·)x−b(·) (z) P = exp iz T (Θx − Θy) ϕ(0) x,y (z), -A(·)y−b(·) (z) P where

s g z T Qx (σ)z − g z T Qy (σ)z ϕ(0) . x,y (z) := exp −

(4.35a)

σ=1

(4.35b)

120

4 Deterministic Descent Directions and Eﬃcient Points (0)

Thus, a necessary condition such that ϕx,y is the characteristic function m - (0) ϕ(0) x,y (z) = Kx,y (z), z ∈ IR ,

(4.35c)

(0)

of a symmetric probability distribution Kx,y reads Qx (σ) Qy (σ), i.e., Qx (σ) − Qy (σ) is positive semideﬁnite, σ = 1, . . . , s (4.36) for arbitrary functions g = g(t) having the above mentioned properties. Note that (4.35b), (4.35c) means that (0) ∗ PA0 (·)y−b0 (·) , (4.37) PA0 (·)x−b0 (·) = Kx,y where A0 (ω), b0 (ω) := A(ω), b(ω) − Ξ. Based on (4.35a)–(4.35c) and (4.36), corresponding to Theorem 4.4 for , here we have the following results: normal distributions P A(·),b(·)

Theorem 4.8. Distribution invariance. For given x ∈ IRr , let y = x fulﬁll the relations c¯T x ≥ c¯T y Θx = Θy Qx (σ) = Qy (σ), σ = 1, . . . , s.

(4.38a) (4.38b) (4.38c)

Then PA(·)x−b(·) = PA(·)y−b(·) , F0 (x) = F0 (y) for arbitrary u ∈ C(P ) := C J (P ) with J = ∅, and F (x) ≥ F (y). Proof. According to (4.34a), (4.34b), conditions (4.38b), (4.38c) imply -A(·)y−b(·) , hence, PA(·)x−b(·) = PA(·)y−b(·) , and the rest follows -A(·)x−b(·) = P P immediately.

Corollary 4.8. Under the above assumptions h = y − x is a descent direction for F at x, provided that F is not constant on xy. Theorem 4.9. Suppose that (4.36) is also suﬃcient for (4.35c). Assume that (0) the distribution Kx,y has zero mean for all x, y under consideration. For given r x ∈ IR , let y = x be a vector satisfying the relations c¯T x ≥ c¯T y (¯ cT x > c¯T y) ΘI x ≥ ΘI y (Θi x > Θi y for at least one i ∈ J)

(4.39a) (4.39b)

ΘII x = ΘII y

(4.39c) Qx (σ) Qy (σ), σ = 1, . . . , s Qx (σ) = Qy (σ) for at least one 1≤σ≤s , (4.39d) where Θi := (Θi )i∈J , ΘII := (Θi )i∈|J . Then F (x) ≥ (>)F (y) for all u ∈ C J (P ), C JJ (P ), CˆJ (P ), respectively.

4.5 Construction of Descent Directions

121

Proof. Under the above assumptions (4.37) holds and F0 (x) = Eu A(ω)x − b(ω) = Eu Θx − θ + Yx (ω) , Yx (ω) := A(ω)x − b(ω) − (Θx − θ). Now the proof of Theorem 4.4 can be transferred to the present case.

4.5 Construction of Descent Directions by Using Quadratic Approximations of the Loss Function Suppose that there exists a closed convex subset Z0 ⊂ IRm such that the loss function u fulﬁlls the relations dT V D ≤ dT ∇2 u(z)d ≤ dT W d for all z ∈ Z0 and d ∈ IRm ,

(4.40a)

where V, W are given ﬁxed symmetric m × m matrices. Moreover, suppose A(ω)x − b(ω) ∈ Z0 for all x ∈ D and ω ∈ Ω. ¯ ¯b) = E A(ω), b(ω) be the mean of A(ω), b(ω) , and let Let (A,

(4.40b)

(4.41) Qij = cov Ai (·), bi (·) , Aj (·), bj (·) T Aj (ω), bj (ω) − (A¯j , ¯bj ) = E Ai (ω), bi (ω) − A¯i , ¯bi denote the covariance matrix of the ith and jth row Ai (·), bi (·) , Aj (·), bj (·) of A(ω), b(ω) . For any symmetric m × m matrix U = (uij ) we deﬁne then ˆ by the symmetric (r + 1) × (r + 1) matrix U T ˆ = E A(ω) − A, ¯ b(ω) − ¯b U A(ω) − A, ¯ b(ω) − ¯b U =

m

(4.42)

uij Qij .

i,j=1

For a given r-vector x ∈ D we consider now the Taylor expansion of u at ¯ − ¯b = E A(ω)x − b(ω) , (4.43) z¯x = Ax hence, 1 z )T (z − z¯x ) + (z − z¯)T ∇2 u(w)(z − z¯x ), u(z) = u(¯ zx ) + ∇u(¯ 2

122

4 Deterministic Descent Directions and Eﬃcient Points

where w = z¯x + ϑ(z − z¯x ) for some 0 < ϑ < 1. Because of (4.40b) and the convexity of Z0 it is z¯x ∈ Z0 for all x ∈ D and therefore w ∈ Z0 for all z ∈ Z0 . Hence, (4.40a) yields zx )T (z − z¯x ) + 12 (z − z¯x )T V (z − z¯x ) ≤ u(z) u(¯ zx ) + ∇u(¯ ≤ u(¯ zx ) + ∇u(¯ zx )T (z − z¯x ) + 12 (z − z¯x )T W (z − z¯x )

(4.44)

for all x ∈ D and z ∈ Z0 . Putting z := A(ω)x − b(ω) into (4.44), because of (4.40b) we ﬁnd u(¯ zx ) + ∇u(¯ zx )T A(ω)x − b(ω) − z¯x T + 12 A(ω)x − b(ω) − z¯x V A(ω)x − b(ω) − z¯x ≤ u A(ω)x − b(ω) ≤ (4.45) zx )T A(ω)x − b(ω) − z¯x ≤ u(¯ zx ) + ∇u(¯ T + 12 A(ω)x − b(ω) − z¯x W A(ω)x − b(ω) − z¯x for all x ∈ D and all ω ∈ Ω.

x Taking now expectations on both sides of (4.45), using (4.42) and x ˆ = −1 we get ˆx u(¯ zx ) + 12 x ˆT Vˆ x ˆT W ˆ ≤ Eu A(ω)x − b(ω) ≤ u(¯ zx ) + 12 x ˆ (4.46a) for all x ∈ D. ¯ − ¯b, where y ∈ D, into the second inequality of Furthermore, putting z := Ay (4.44), we ﬁnd ¯ − ¯b) ≤ u(¯ ¯ − x) (4.46b) ¯ − x) + 1 (y − x)T A¯T W A(y u(Ay zx ) + ∇u(¯ zx )T A(y 2 T ¯ zx ) (y − x) + 12 (y − x)T (A¯T W A)(y − x) = u(¯ zx ) + A¯T ∇u(¯ for all x ∈ D and y ∈ D. Considering now F (y) − F (x) for vectors x, y ∈ D, from (4.46a) we get F (y) − F (x) = c¯T y + Eu A(ω)y − b(ω) − c¯T x + Eu A(ω)x − b(ω) 1 ˆ yˆ + u(Ay ¯ − ¯b) − ≤ c¯ (y − x) + yˆT W 2

T

Hence, we have the following result: Theorem 4.10. If D, A(ω), b(ω) , Z0 and V, W (4.40a), (4.40b) hold, then for all x, y ∈ D it is

1 Tˆ ¯ ¯ x ˆ Vx ˆ + u(Ax − b) . 2

are selected such that

1 T ˆ ¯ − ¯b)−u(Ax− ¯ ¯b). (4.47a) y W yˆ − x F (y)−F (x) ≤ c¯T (y −x)+ (ˆ ˆT Vˆ x ˆ)+u(Ay 2

4.5 Construction of Descent Directions

123

Using also inequality (4.46b), we get this corollary: Corollary 4.9. Under the assumption of Theorem 4.10 it holds 1 T ˆ F (y) − F (x) ≤ c¯T (y − x) + (ˆ y W yˆ − x ˆT Vˆ x ˆ) + 2 T ¯ − ¯b) (y − x) + 1 (y − x)T A¯T W A(y ¯ − x). + A¯T ∇u(Ax 2 ˆ by Represent the (r + 1) × (r + 1) matrix W ⎞ ⎛ . . .. w W ⎟ ⎜ ˆ = ⎜ ... ... ...⎟, W ⎠ ⎝ . T . w . γ

(4.47b)

(4.47c)

. is a symmetric r × r matrix, w is an r-vector and γ is a real number. where W Then ˆ yˆ = x ˆx . (y − x) + 2(W . x − w)T (y − x), ˆT W ˆ + (y − x)T W yˆT W and therefore 1 T ˆ 1 . + A¯T W A)(y ¯ ˆ (W − Vˆ )ˆ x + (y − x)T (W − x) F (y) − F (x) ≤ c¯T (y − x) + x 2 2 T ¯ − ¯b) + W . x − w (y − x) + A¯T ∇u(Ax T 1 T ˆ ¯ − ¯b) + W . x − w (y − x) (4.47d) x ˆ (W − Vˆ )ˆ x + c¯ + A¯T ∇u(Ax 2 1 . + A¯T W A)(y ¯ − x). + (y − x)T (W 2 Thus, F x + (y − x) − F (x) is estimated from above by a quadratic form in h = y − x. Concerning the construction of descent directions of F at x we have this ﬁrst immediate consequence from Theorem 4.10: =

Theorem 4.11. Suppose that the assumptions of Theorem 4.10 hold. If vectors x, y ∈ D, y = x, are related such that cT y c¯T x ≥ (>)¯ ¯ = Ay ¯ Ax ˆ yˆ, ˆ ≥ (>)ˆ yT W x ˆT Vˆ x

(4.48a) (4.48b) (4.48c)

then F (x) ≥ (>)F (y). Moreover, if F (x) > F (y) or F (x) = F (y) and F is not constant on xy, then h = y − x is a feasible descent direction of F at x.

124

4 Deterministic Descent Directions and Eﬃcient Points

Note. If the loss function u is monotone with respect to the partial order on IRm induced by IRm + , then (4.48b) can be replaced by one of the weaker relations ¯ ≥ Ay ¯ or Ax ¯ ≤ Ay. ¯ Ax

(4.48b )

If u has the partial monotonicity property (4.10a), (4.10b), then (4.48b) can be replaced by (4.14b), (4.14b ), (4.14c). The next result follows from (4.47d). Theorem 4.12. Suppose that the assumptions of Theorem 4.10 hold. If vectors x, y ∈ D, y = x, are related such that ¯ ¯ − ¯b) + W .x − w . + A¯T W A)(y − x) = − c¯ + A¯T ∇u(Ax (4.49a) (W T ¯ − ¯b) + W . x − w (y − x) ≤ (<) − x ˆ − Vˆ )ˆ c¯ + A¯T ∇u(Ax ˆT (W x, (4.49b) then F (y) ≤ (<)F (x). Furthermore, if F (y) < F (x) or F (y) = F (x) and F is not constant on xy, then h = y − x is a feasible descent direction for F at x. Proof. If (4.49a), (4.49b) holds, then (4.47d) yields T 1 T ˆ ¯ − ¯b) + W . x − w (y − x) x ˆ (W − Vˆ )ˆ x + c¯ + A¯T ∇u(Ax 2 1 . + A¯T W A)(y ¯ − x) + (y − x)T (W 2 T 1 T ˆ ¯ − ¯b) + W . x − x (y − x) ˆ (W − Vˆ )ˆ ≤ x x + c¯ + A¯T ∇u(Ax 2 1 .x − w +(−1) (y − x)T c¯ + A¯T ∇u(T¯x − ¯b) + W 2

T 1 T ˆ T ¯ ˆ ¯ ¯ . x ˆ (W − V )ˆ x + c¯ + A ∇u(Ax − b) + W x − w (y − x) = 2

F (y) − F (x) ≤

≤ (<)0.

Discussion of (4.49a), (4.49b): Suppose that ˜ + A¯T W A¯ is positive deﬁnite, C=W

(4.50a)

hence, there are positive numbers α, β such that αx2 ≤ xT Cx ≤ βx2 .

(4.50b)

4.5 Construction of Descent Directions

125

According to (4.49a) it is ¯ − ¯b) + W .x − w , y − x = −C −1 c¯ + A¯T ∇u(Ax and (4.49b) reads

¯ ¯b)+ W .x−w c¯+ A¯T ∇u(Ax−

or where

T

¯ ¯b)+ W ˆ − Vˆ )ˆ .x−w ≥ (>)ˆ C −1 c¯+ A¯T ∇u(Ax− xT (W x

ˆ − Vˆ )ˆ ˆT (W x, aT C −1 a ≥ x ¯ − ¯b) + W . x − w. a = c¯ + A¯T ∇u(Ax

Because of (4.50b) it is 1 1 a2 ≤ aT C −1 a ≤ a2 . β α Thus, we ﬁnd the following criterion for (4.49b): 2 1 ¯ + ¯b) + W . x − w ˆ − Vˆ )ˆ (1) If ¯ c + A¯T ∇u(Ax xT (W x, then (4.49b) ≥ (>)ˆ β holds. 2 1 ˆ − Vˆ )ˆ ¯ − ¯b) + W . x − w (2) If x ˆT (W c + A¯T ∇u(Ax x > ¯ , then (4.49b) is not α valid.

5 RSM-Based Stochastic Gradient Procedures

5.1 Introduction According to the methods for converting an optimization problem with a random parameter vector a = a(ω) into a deterministic substitute problem, see Chap. 1 and Sects. 2.1.1, 2.1.2, 2.2, a basic problem is the minimization of the total expected cost function F = F (x) on a closed, convex feasible domain D for the input or design vector x. of notation, here the random total cost function f = For simpliﬁcation f a(ω), x is denoted also by f = f (ω, x), hence, f (ω, x) := f a(ω), x . Thus, the expected total cost minimization problem minimize Ef (ω, x) s.t. x ∈ D

(5.1)

must be solved. In this chapter we consider therefore the approximate solution of the mean value minimization problem (5.1) by a semi-stochastic approximation method based on the response surface methodology (RSM), see, e.g., [17, 55, 106]. Hence, f = f (ω, x) denotes a function on IRr depending also on a random element ω, and “E” is the expectation operator with respect to the underlying probability space (Ω, A, P ) containing ω. Moreover, D designates a closed convex subset of IRr . We suppose that the objective function of (5.1) (5.2) F (x) = Ef (ω, x), x ∈ IRr , is deﬁned and ﬁnite for every x ∈ IRr . Furthermore, we assume that the derivatives ∇F (x), ∇2 F (x), . . . exist up to a certain order κ. In some cases, cf. [11] and Sect. 2.2, we may simply interchange diﬀerentiation and integration, hence, (5.3a) ∇F (x) = E∇x f (ω, x) or ∇F (x) = E∂x f (ω, x),

130

5 RSM-Based Stochastic Gradient Procedures

where ∇x f, ∂x f denotes the gradient, subgradient, resp., of f with respect to x. Corresponding formulas may hold also for higher derivatives of F . However, in some important practical situations, especially for probability-valued functions F (x), (5.3a) does not hold. According to Chap. 3 we know that gradient representation in integral form ∇F (x) = G(w, x)µ(dw) (5.3b) with a certain vector-valued function G = G(w, x), may be derived then, cf. also [149, 159]. One of the main method for solving problem (5.1) is the stochastic approximation procedure [36, 44, 46, 52, 69, 71, 112, 116] Xn+1 = pD (Xn − ρn Yns ),

n = 1, 2, . . . ,

(5.4a)

where Yns denotes an estimator of the gradient ∇F (x) at x = Xn . Important examples are the simple stochastic (sub)gradient or the ﬁnite diﬀerence gradient estimator Yns = ∇x f (ωn , Xn ) Yns ∈ ∂x f (ωn , Xn ) , or Yns = G(wn , Xn ) (5.4b) Yns =

r 1 (1) (2) f (ωn,j , Xn + cn ej ) − f (ωn,j , Xn − cn ej ) ej 2cn j=1

(5.4c) (k)

with the unit vectors e1 , . . . , er or IRr . Here, ωn (wn ), n = 1, 2, . . . , ωn,j , k = 1, 2, j = 1, 2, . . . , r, n = 1, 2, . . . , resp., are sequences of independent realizations of the random element ω(w). Using formula (5.4b), (5.4c), resp., algorithm (5.4a) represents a Robbins–Monro (RM)-, a Kiefer–Wolfowitz (KW)-type stochastic approximation procedure. Moreover, pD designates the projection of IRr onto D, and ρn > 0, n = 1, 2, . . . , denotes the sequence of step sizes. A standard condition for the selection of (ρn ) is given [34–36] by ∞ n=1

ρn = +∞,

∞

ρ2n < +∞,

(5.4d)

n=1

and corresponding conditions for (cn ) can be found in [164]. Due to its stochastic nature, algorithms of the type (5.4a) have only a very small asymptotic convergence rate in general cf. [164]. Considerable improvements of (5.4a) can be obtained [87] by (1) Replacing −Yns at certain iteration points Xn , n ∈ IN1 , by an improved step direction hn of F at Xn , see [79, 80, 84, 86, 90] (2) Step size control, cf. [85, 89, 117]

5.2 Gradient Estimation Using the Response Surface Methodology (RSM)

131

Using (I), we get the following hybrid stochastic approximation procedure pD (Xn + ρn hn ) for n ∈ IN1 Xn+1 = (5.5) pD (Xn − ρn Yns ) for n ∈ IN2 where IN1 , IN2 is a certain partition of the set IN of integers. Important types of improved step directions hn are: –

–

(feasible) Deterministic descent directions hn existing at nonstationary/ noneﬃcient points Xn in problems (5.1) having certain functional properties, see Sect. 4 and [81–84, 93]. hn = −Yne , where Yne is a more exact estimator of ∇F (Xn ), as, e.g., the arithmetic mean of a certain number M = Mn of simple stochastic s s = ∇x f (ωn,i , Xn ), Yn,i ∈ ∂x f (ωn,i , Xn ), resp., i = 1, . . . , M , gradients Yn,i of the type (5.4b), cf. [86], or the estimator Yne deﬁned by the extended central diﬀerence schemes given by Fabian [39, 41].

By an appropriate selection of the partition IN1 , IN2 we may guarantee that the stochastic approximation method is accelerated suﬃciently, without using the improved and therefore more expensive step direction hn too often. Several numerical experiments show that improved step directions hn are most eﬀective during an initial phase of the algorithm. Having several generators of deterministic descent directions hn in special situations, as, e.g., in stochastic linear programs with appropriate parameter distributions, see Sect. 4, in the following a general method for the computation of more exact gradient estimators Yne at an arbitrary iteration point Xn = x0 is described. This method, which includes as a special case the well known ﬁnite diﬀerence schemes, used, e.g., in the KW -type procedures [36,41,164], is based on the following fundamental principle of numerical diﬀerentiation, see, e.g., [22, 48, 51]: in a neighStarting from certain supporting points x(i) , i = 1, . . . , p, lying borhood S of x0 and the corresponding function values y (i) = F x(i) , i = 1, . . . p, by interpolation, ﬁrst an interpolation function (e.g., a polynomial of degree p−1) Fˆ = Fˆ (x) is computed. Then, the kth order numerical derivatives ˆ k F of F at x0 are deﬁned by the analytical derivatives ∇ ˆ k F (x0 ) := ∇k Fˆ (x0 ), k = 0, 1, . . . , κ ∇ of the interpolation function Fˆ at x0 . Clearly, in order to guarantee a certain ˆ k F (x0 ), a suﬃciently large accuracy of the higher numerical derivatives ∇ (i) number p of interpolation points x , i = 1, . . . , p, is needed.

5.2 Gradient Estimation Using the Response Surface Methodology (RSM) We consider a subdomain S of the feasible domain D such that x0 ∈ S ⊂ D,

(5.6a)

132

5 RSM-Based Stochastic Gradient Procedures

ˆ (x0 ) of ∇F where x0 is the point, e.g., x0 := Xn , at which an estimate ∇F must be computed. Corresponding to the principle of numerical diﬀerentiation described above, in order to estimate ﬁrst the objective F (x) on S, function (i) (i) , i = 1, . . . , p, are estimates y , i = 1, . . . , p, of the function values F x determined at so-called design points x(i) = x(i) (x0 ) ∈ S,

i = 1, 2, . . . , p,

chosen by the decision maker [17, 63, 106]. Simple estimators are i = 1, 2, . . . , p, y (i) := f ω (i) , x(i)

(5.6b)

(5.6c)

where ω (1) , ω (2) , . . . , ω (p) are independent realizations of the random element ω. The objective function F is then approximated – on S – mostly by a polynomial response surface model Fˆ (x) = Fˆ (x|βj , j ∈ J). In practice, usually ﬁrst and/or second order models are used. The coeﬃcients βj , j ∈ J, are determined by means of regression analysis (mainly least squares estimation), ˆ (x0 ) at x0 is see [23, 55, 63]. Having Fˆ (x), the RSM-gradient estimator ∇F deﬁned by the gradient (with respect to x) ˆ (x0 ) := ∇Fˆ (x0 ) ∇F

(5.6d)

of Fˆ at x0 . Using higher order polynomial response surface models Fˆ , the ˆ k F (x0 ) of higher order derivatives of F at x0 can be deﬁned in estimates ∇ the same way: ˆ k F (x0 ) := ∇k Fˆ (x0 ), ∇

k = 1, . . . , κ.

(5.6e)

Remark 5.1. The above gradient estimation procedure (5.6a)–(5.6e) is especially useful if the gradient representation (5.3a) or (5.3b) does not hold (or is not known explicitly) for every x ∈ IRr , or the numerical evaluation of ∇x f (ω, x), ∂x f (ω, x), G = G(w, x), resp., is too complicated. The following very important example is taken from structural reliability [28, 38, 156]: A decisive role and 3.1, by the failure is played, see Sects. 2.1 probability F (x) = 1 − P yl ≤ y(a(ω), x) ≤ yu that, for a given design vector x (e.g., structural dimensions), the structure does not fulﬁll the basic behavioral constraints yl ≤ y a(ω), x ≤ yu . Here, yl , yu are given lower and upper bounds, y = y a(ω), x is an m-vector of displacement and/or stress components of the structure, and a = a(ω) denotes the random vector of structural parameters, e.g., load parameters, yield stresses, elastic moduli. Obviously, F (x) = Ef (ω, x) with f (ω, x) := 1 − 1B y a(ω), x ,

5.2 Gradient Estimation Using the Response Surface Methodology (RSM)

133

where 1B designates the characteristic function of B = {y ∈ IRm : yl ≤ y ≤ yu }. Here, (5.3a) does not hold, and the numerical evaluation of diﬀerentiation formulas of the type (5.3b), see Chap. 3, may be very complicated – if they are available at all. Remark 5.2. In addition to Remark 5.1, the RSM-estimator has still the following advantages: (a) A priori knowledge on F can be incorporated easily in the selection of an appropriate response surface model. As an important example we mention here the use of so-called reciprocal variables 1/xk or more general intermediate variables tk = tk (xk ) besides the decision variables xk , k = 1, . . . , r, in structural optimization, see, e.g., [5, 42]. (b) The present method does not only yield estimates of the gradient (and some higher derivatives) of F at x0 , but also estimates Fˆ (x0 ) of the function value F (x0 ). However, besides gradient estimations, estimates of function values are the main indicators used to evaluate the performance of an algorithm, see [44]. Indeed, having these indicators, we may check whether the process runs into the “right direction,” i.e., decreases (increases, resp.) the mean performance criterion F (x), see [105]. Hence, appropriate stopping criterions can be formulated. Remark 5.3. In many practical situations in a certain neighborhood of an optimal solution x∗ of (5.1), the objective function F is nearly quadratic. Hence, F may be described then by means of second order polynomial models Fˆ with high accuracy. Based on second order models Fˆ of F , estimates x ˆ∗ of ∗ an optimal solution x of (5.1) may be obtained by solving analytically the approximate program min Fˆ (x) s.t. x ∈ D. Furthermore, Newton-type step directions hn := −∇2 Fˆ (Xn )−1 ∇Fˆ (Xn ), to be used in procedure (5.5), can be calculated easily. Remark 5.4. The above remarks show that RSM has many very interesting features and advantages. But we have also to mention that for problems (5.1) with large dimensions r, the computational expenses to ﬁnd a full second order model Fˆ or more accurate gradient estimates may become very large. However, an improved search direction hn is required in the semi-stochastic approximation procedure only at iteration points Xn with n ∈ IN1 , cf. Sect. 5.1. Furthermore, within RSM there are several suggestions to reduce the estimation expenses. E.g., by the group screening few variables xkl , l = 1, . . . , r1 (< r), can be identiﬁed [102]. On the other hand, by decomposition – e.g., in structural optimization – a large optimization problem that might be intractable because of the large number of unknowns and constraints combined with a computationally expensive evaluation of the objective function F can be converted into a set of much smaller and separate, but coordinated subproblems, see, e.g., [145]. Hence, in many cases the optimization problem (5.1) can be reduced again to a moderate size.

134

5 RSM-Based Stochastic Gradient Procedures

5.2.1 The Two Phases of RSM In Phase 1 of RSM, i.e., if the process (Xn ) is still far away from an optimal point x∗ of (5.1), then F is estimated on S by the linear empirical model Fˆ (x) = β0 + βIT (x − x0 ),

(5.7a)

β T = (β0 , βIT ) = (β0 , β1 , . . . , βr )T

(5.7b)

where is the (r + 1)-vector of unknown coeﬃcients of the linear model (5.7a). Having (i) estimates y of the function values F x(i) at the so-called design points x(i) , i = 1, . . . , p, in S, by least squares estimation (LSQ) the following estimate βˆ of β is obtained: βˆ = (W T W )−1 W T y. Here, the p × (r + 1) matrix W and the p-vector y are deﬁned by ⎛ ⎞ ⎛ (1) ⎞ T 1 d(1) y ⎜ (2)T ⎟ ⎜ y (2) ⎟ ⎜1 d ⎟ ⎟ ⎟, y = ⎜ W =⎜ ⎜ .. ⎟ . . ⎜. . ⎟ ⎝ . ⎠ ⎝. . ⎠ (p)T y (p) 1d

(5.8a)

(5.8b)

with d := x − x0 and therefore d(i) = x(i) − x0 ,

i = 1, 2, . . . , p.

(5.8c)

In order to guarantee the regularity of W T W , we require that d(2) − d(1) , . . . , d(p) − d(1) , p ≥ r + 1, span the IRr .

(5.8d)

Having βˆ by (5.8a), the gradient ∇F of F can be estimated on S by the gradient of (5.7a), hence, ˆ (x) = βˆI for all x ∈ S. ∇F

(5.9)

The accuracy of the estimator (5.9) on S, cf. (5.6a), is studied in the following way: Denote by (5.10a) ε(i) = ε(i) ω, x(i) , i = 1, 2, . . . , p, the stochastic error of the estimate y (i) of F x(i) , see, e.g., (5.6c). By means of ﬁrst order Taylor expansion of F at x0 we obtain, cf. (5.8c), (i) y (i) = F x(i) + ε(i) = F (x0 ) + ∇F (x0 )T d(i) + R1 + ε(i) ,

(5.10b)

5.2 Gradient Estimation Using the Response Surface Methodology (RSM)

135

where

1 (i)T 2 d ∇ F x0 + ϑi d(i) d(i) , 0 < ϑi < 1, (5.10c) 2 denotes the remainder of the ﬁrst order Taylor expansion. Obviously, if S is a convex subset of the feasible domain D of (5.1), and the Hessian ∇2 F (x) of the objective function F of (5.1) is bounded on S, then (i)

R1 =

(i) 1 (i) R1 ≤ d sup ∇2 F x0 + ϑd(i) 2 0≤ϑ≤1 ≤

1 (i) 2 d sup∇2 F (x). 2 x∈S

(5.10d)

Thus, representation (5.10b) yields the data model y = W β + R1 + ε,

(5.11a)

where W, y are given by (5.8b), and the vectors β, R1 and ε are deﬁned by ⎛ (1) ⎞ ⎛ (1) ⎞ R1 ε ⎜ (2) ⎟

⎜ ε(2) ⎟ ⎜ R1 ⎟ F (x0 ) ⎟ ⎜ ⎟ , R1 = ⎜ β= (5.11b) . ⎟. ⎜ .. ⎟ , ε = ⎜ ∇F (x0 ) ⎝ .. ⎠ ⎝ . ⎠ ε(p)

(p)

R1

Concerning the p-vector ε of random errors ε(i) , i = 1, . . . , p, we make the following (standard) assumptions E(ε|x0 ) = 0, cov(ε|x0 ) = E(εεT |x0 ) = Q := (σi2 δij ),

(5.11c), (5.11d)

where δij , i, j = 1, . . . , p, denotes the Kronecker symbol, and σi2 = σi2 (x0 , d(1) , d(2) , . . . , d(p) ),

i = 1, 2, . . . , p.

Putting now the data model (5.11a) into (5.8a), we get

F (x0 ) + (W T W )−1 W T (R1 + ε). βˆ = ∇F (x0 )

(5.11e)

(5.12a)

Hence, for the estimator βˆI of ∇F (x0 ), cf. (5.9), we ﬁnd βˆI = ∇F (x0 ) +

−1

1 (1) d − d, . . . , p d(p) − d (R1 + ε), (5.12b)

p 1 (i) d − d d(i) − d p i=1

136

5 RSM-Based Stochastic Gradient Procedures

where d is deﬁned by 1 (i) d , p i=1 p

d=

d(i) = x(i) − x0 .

(5.12c)

Clearly, (5.12b), (5.12c) represents the accuracy of the estimator (5.9) on S. Note. According to (5.12a), the accuracy of βˆ depends on the deterministic model error R1 , the stochastic observation error ε and also on the choice of the design points x(1) , . . . , x(p) , see, e.g., [17, 136]. In Phase 2 of RSM, i.e., if the process (Xn ) is approaching a certain neighborhood of a minimal point x∗ of (5.1), see Remark 5.3, then the linear model (5.7a) of F is no more adequate, in general. It is replaced therefore by the more general quadratic empirical model Fˆ (x) = β0 + βIT (x − x0 ) + (x − x0 )T B(x − x0 ),

x ∈ S.

(5.13a)

The unknown parameters β0 , βi , i = 1, . . . , r, in the r-vector βI , and βij , i, j = 1, . . . , r, in the symmetric r×r matrix B, are determined again by least squares estimation. The LSQ-estimate ˆ βˆ = (βˆ0 , βˆI , B) of the

1 2 (r + 3r + 2)-vector 2 β T = (β0 , β1 , β2 , . . . , βr , β11 , β12 , . . . , β1r , β22 , β23 , . . . , β2r , . . . , βr−1r−1 , βr−1r , βrr ),

(5.13b)

of unknown parameters in model (5.13a), denoted for simplicity also by (5.13b )

β = (β0 , β1 , B), is given – in correspondence with formula (5.8a) – again by βˆ = (W T W )−1 W T y. 1 Here the p × (r2 + 3r + 2) matrix W 2 ⎛ T T 1 d(1) z (1) ⎜ (2)T (2)T ⎜1 d z ⎜ W =⎜. . .. ⎜ .. .. . ⎝ T

1 d(p) z (p)

T

(5.14a)

and the p-vector y are deﬁned by ⎞ ⎟ ⎟ ⎟ ⎟, ⎟ ⎠

⎞ y (1) ⎜ y (2) ⎟ ⎟ ⎜ ⎟ y=⎜ ⎜ .. ⎟ , ⎝ . ⎠ ⎛

y (p)

(5.14b)

5.2 Gradient Estimation Using the Response Surface Methodology (RSM)

137

where d(i) = x(i) − x0 see (5.8c), and T (i)2 (i) (i) (i) (i)2 (i) (i) (i) (i) (i)2 z (i) = d1 , 2d1 d2 , . . . , 2d1 d(i) , d , 2d d , . . . , 2d d , . . . , d . r r r 2 2 3 2 (5.14c) Corresponding to the ﬁrst phase, the existence of (W W )−1 is guaranteed – for the full quadratic model – by the assumption

(p)

(2) 2 d − d(1) d − d(1) , . . . , span the IR(r +3r+2)/2 , (5.14d) z (2) − z (1) z (p) − z (1) see [17, 106], hence, p≥

(r + 1)(r + 2) + 1. 2

ˆ the gradient of F is then estimated on S by the Having βˆ = (βˆ0 , βˆI , B), gradient of (5.13a), thus ˆ (x) = βˆI + 2B(x ˆ − x0 ) ∇F

for

x∈S

(5.15a)

and therefore ˆ (x0 ) = βˆI . ∇F

(5.15b)

Note. In practice, in the beginning of Phase 2 only the pure quadratic terms βii (xi − x0i )2 , i = 1, . . . , r, may be added (successively) to the linear model (5.7a). For the consideration of the accuracy of the estimator (5.15b) we use now second order Taylor expansion. Corresponding to (5.10b), we obtain 1 (i) y (i) = F (x(i) ) + ε(i) = F (x0 ) + ∇F (x0 ) d(i) + d(i) ∇2 F (x0 )d(i) + R2 + ε(i) , 2 (5.16a) where 1 (i) (5.16b) R2 = ∇3 F x0 + ϑi d(i) · d(i)3 , 0 < ϑi < 1, 3! denotes the remainder of the second order Taylor expansion. Clearly, if S is a convex subset of D and the multilinear form ∇3 F (x) of all third order derivatives of F is bounded on S, then

(i)

1 (i) 3 d sup ∇3 F (x0 + ϑd(i) ) 6 0≤ϑ≤1 1 (i) 3 ≤ d sup ∇3 F (x). 6 x∈S

|R2 | ≤

(5.16c)

Representation (5.16a) yields now the data model y = W β + R2 + ε.

(5.17a)

Here, W, y are given by (5.14b), and the vectors β, R2 , ε are deﬁned, cf. (5.13b), (5.13b ) for the notation, by

138

5 RSM-Based Stochastic Gradient Procedures

⎛

1 β = F (x0 ), ∇F (x0 ), ∇2 F (x0 ) , 2

⎞ (1) R2 ⎜ (2) ⎟ ⎜ R2 ⎟ ⎟ R2 = ⎜ ⎜ .. ⎟ , ⎝ . ⎠

⎞ ε(1) ⎜ ε(2) ⎟ ⎟ ⎜ ε = ⎜ . ⎟. ⎝ .. ⎠ ⎛

(p)

R2

ε(p) (5.17b)

Corresponding to (5.11c)–(5.11e) we also suppose in Phase 2 that cov(ε|x0 ) = E(εεT |x0 ) = Q := (σi2 δij ) (5.17c), (5.17d) (5.17e) σi2 = σi2 x0 , d(1) , d(2) , . . . , d(p) , i = 1, . . . , p.

E(ε|x0 ) = 0,

Putting the data model (5.17a) into (5.14a), we get – see (5.13b), (5.13b ) concerning the notation –

1 βˆ = F (x0 ), ∇F (x0 ), ∇2 F (x0 ) + (W T W )−1 W T R2 + (W T W )−1 W T ε. 2 (5.18a) Especially, for the estimator βˆI of ∇F (x0 ), see (5.15a), (5.15b), we ﬁnd 1 βˆI = ∇F (x0 ) + U −1 (d(1) − d, . . . , d(p) − d) (5.18b) p p 1 (i) (i) T Z −1 (z (1) − z, . . . , z (p) − z) (R2 + ε). (d − d)(z − z) − p i=1 The vectors d(i) , d, z (i) are given by (5.8c), (5.12c), (5.14c), resp., and z, Z, U are deﬁned by 1 (i) z , p i=1

1 (i) (z − z)(z (i) − z)T , (5.18c), (5.18d) p i=1 p p 1 (i) 1 (i) (i) T (i) T U = Z −1 (5.18e) (d − d)(d − d) − (d − d)(z − z) p i=1 p i=1 p 1 (i) (i) T × . (z − z)(d − d) p i=1 p

z=

p

Z=

5.2.2 The Mean Square Error of the Gradient Estimator According to (5.9) and (5.12b), (5.15b) and (5.18b), resp., in both phases for ˆ (x0 ) we ﬁnd the gradient estimator ∇F ˆ (x0 ) = ∇F (x0 ) + H(R + ε) ∇F

(5.19)

with R = R1 , R = R2 . Furthermore, H designates the r × p matrix arising in ˆ (x0 ) = βˆ1 . the representation (5.12b), (5.18b), resp., of ∇F

5.2 Gradient Estimation Using the Response Surface Methodology (RSM)

ˆ (x0 ) The mean square error V = V (x0 ) of the estimator ∇F ˆ (x0 ) − ∇F (x0 )2 |x0 V := E ∇F

139

(5.20a)

is given, cf. see (5.11c), (5.17c), resp., by the sum V = edet 2 + E estoch 2 |x0

(5.20b)

ˆ (x0 ) of the squared bias edet 2 := HR2 and the variance of ∇F E estoch 2 |x0 := E Hε2 |x0 . Since w2 = tr(wwT ) = wwT for a vector w, where tr(M ), M , resp., denotes the trace, the operator norm of a square matrix M , we ﬁnd edet 2 = HRRT H T = sup wT (HRRT H T )w ≤ R2 HH T . (5.20c)

w =1

Moreover, using that tr(M ) is equal to the sum of the eigenvalues of M , we get E(estoch 2 |x0 ) = E tr(HεεT H T )|x0 = tr(HQH T ) ≤ r sup (H T w)T (σi2 δij )(H T w)

≤r

max

1≤i≤p

σi2

w =1

sup H w = r T

2

w =1

max

1≤i≤p

σi2

HH T , (5.20d)

cf. (5.11d), (5.17d), respectively. Consequently, inequalities (5.20c), (5.20d) yield in the ﬁrst and in the second phase

2 2 (5.20e) V ≤ R + r max σi HH T . 1≤i≤p

Here, the matrix product HH T can be computed explicitly: In Phase 1, due to (5.12b) we have 1 H = H0−1 (d(1) − d, d(2) − d, . . . , d(p) − d), p

(5.21a)

where d(i) , d are given by (5.8c), (5.12c), resp., and 1 (i) H0 = (d − d)(d(i) − d)T . p i=1 p

(5.21b)

140

5 RSM-Based Stochastic Gradient Procedures

By an easy calculation, from (5.21a), (5.21b) we get 1 −1 1 H0 H0 H0−1 = H0−1 . (5.21c) p p Assume now that the function value F x(i) is estimated ν times at HH T =

x(i) = x0 + d(i) for each i = 1, . . . , p0 .

(5.22a)

Hence, p = νp0 and d(kp0 +i) = d(i) , x(kp0 +i) = x(i) , i = 1, 2, . . . , p0 , k = 1, 2, . . . , ν − 1.

(5.22b)

Putting (0)

H0

:=

p0 1 (i) (0) (0) T d −d d(i) − d , p0 i=1

d

(0)

in case (5.22a), (5.22b) we easily ﬁnd

1 1 (0)−1 . HH T = H0 ν p0

:=

p0 1 (i) d , p0 i=1

(5.23a)

(5.23b)

In Phase 2 matrix H reads, see (5.18b), " 1 H = H0−1 d(1) − d, . . . , d(p) − d (5.24a) p p # T 1 (i) (i) −1 (1) (p) − d −d z −z z − z, . . . , z − z , Z p i=1 where d(i) , d, z (i) , z, Z are given by (5.8c), (5.12c), (5.14c), (5.18c), (5.18d), resp., and H0 is deﬁned, cf. (5.18e), here by H0 := U.

(5.24b)

Corresponding to (5.21c), from (5.24a), (5.24b), also in Phase 2 we get HH T =

1 −1 H . p 0

(5.24c)

In case (5.22a), (5.22b) of estimating F ν times at p0 diﬀerent points, one can easily see that an equation corresponding to (5.23b) holds also in Phase 2. Consequently, from (5.20e), and (5.21c), (5.24c), resp., for the mean square estimation error V we obtain in both phases

1 H −1 , (5.25a) V ≤ R2 + r max σi2 1≤i≤p p 0

5.2 Gradient Estimation Using the Response Surface Methodology (RSM)

141

where H0 is given by (5.21b), (5.24b), respectively. In case (5.22a), (5.22b) we get

1 (0)−1 2 2 V ≤ R + r max σi H0 , (5.25b) 1≤i≤p νp0 provided that 2 = σi2 , σ(kp 0 +i)

i = 1, 2, . . . , p0 ,

k = 1, . . . , ν − 1.

(5.26)

Finally, we have to consider R2 : First, in Phase 1 according to (5.10d) and (5.11b) we ﬁnd with certain numbers ϑi , 0 < ϑi < 1, i = 1, . . . , p, R1 2 ≤

p 2 1 (i) 4 d ∇2 F x0 + ϑi d(i) 4 i=1

p 2 1 (i) 4 d sup ∇2 F x0 + ϑd(i) 4 i=1 0≤ϑ≤1 p 2 1 (i) 4 ≤ d sup∇2 F (x) , 4 i=1 x∈S

≤

(5.27a)

where the last inequality holds if S, cf. (5.6a), (5.6b) is convex, and ∇2 F (x) is bounded on S. In Phase 2, with (5.16b), (5.17b), we ﬁnd R2 2 ≤

p 2 1 (i) 6 d ∇3 F x0 + ϑi d(i) 36 i=1

p 2 1 (i) 6 d sup ∇3 F x0 + ϑd(i) 36 i=1 0≤ϑ≤1 p 1 (i) 6 ≤ d sup ∇3 F (x)2 , 36 i=1 x∈S

≤

(5.27b)

where the last inequality holds under corresponding assumptions as used above. Summarizing (5.27a), (5.27b), we have Rl 2 ≤

p 2 1 l+1 (i) 2(l+1) (i) d sup F (x + ϑd ) ∇ , 0 ((l + 1)!)2 i=1 0≤ϑ≤1

l = 1, 2.

(5.27c) Putting (5.27c) into (5.25a), for l = 1, 2 we ﬁnd p 2 1 (i) 2(l+1) 1 l+1 (i) V ≤ d sup F (x + ϑd ) ∇ 0 ((l + 1)!)2 p i=1 0≤ϑ≤1 r + max σi2 H0−1 . (5.28) p 1≤i≤p

142

5 RSM-Based Stochastic Gradient Procedures

We consider now the case d(i) = µd˜(i) ,

i = 1, 2, . . . , p,

0 < µ ≤ 1,

(5.29a)

where d˜(i) , i = 1, . . . , p, are given r-vectors. Obviously ˜ 0, H0 = µ2 H

(5.29b)

˜ 0 follows from (5.21b), (5.24b), resp., by replacing where the r × r matrix H d(i) by d˜(i) , i = 1, . . . , p. Using (5.29a), from (5.28) we get for l = 1, 2 ⎛ p 2 1 ˜(i) 2(l+1) µ2l ⎜ l+1 (i) ˜ V ≤ ⎝ ∇ d sup F (x + ϑ d ) 2 0 p i=1 0≤ϑ≤1 (l + 1)! ⎞ +

rµ−2 ⎟ ˜ −1 max σi2 ⎠ H 0 . l≤i≤p p

(5.30)

5.3 Estimation of the Mean Square (Mean Functional) Error Considering now the convergence behavior of the hybrid stochastic approximation procedure (5.5) working with the RSM-gradient estimator Yne , we need the following notations. Let IN1 , IN2 be a given partition of IN, and suppose that Xn+1 = pD (Xn − ρn Yns ) with a simple gradient estimator Yns , see (5.4b), at each iteration point Xn with n ∈ IN2 . Furthermore, deﬁne for each Xn with n ∈ IN1 the improved step direction hn = −Yne by the following scheme: Following Sect. 5.2, at x0 = Xn , n ∈ IN1 , we select r-vectors d(i) , i = 1, . . . , p. In the most general case it is d(i) = d(i,n) = d(i,n) (Xn ),

i = 1, 2, . . . , p = p(n),

(5.31a)

i.e., the increments d(1) , . . . , d(p) and p may depend on the stage index n and the iteration point Xn . In the most simple case we have d(i,n) = d(i,0) ,

i = 1, 2, . . . , p = p¯ for all n = 1, 2, . . . ,

(5.31b)

where d(1,0) , . . . , d(p,0) are p given, ﬁxed r-vectors. The unknown objective function F is then estimated at x(i) = x(i,n) = Xn + d(i,n) ,

i = 1, 2, . . . , p(n),

(5.31c)

i.e., we have the estimates y (i) = y (i,n) = F (x(i,n) ) + ε(i,n) ,

i = 1, 2, . . . , p(n).

(5.31d)

5.3 Estimation of the Mean Square (Mean Functional) Error

According to (5.8b), (5.14b), resp., we deﬁne ⎛ (1,n) ⎞ y ⎜ y (2,n) ⎟ ⎜ ⎟ yn = ⎜ . ⎟ with p = p(n) ⎝ .. ⎠ ⎛

y (p,n) T

1 d(1,n) ⎜ (2,n)T ⎜1 d Wn = ⎜ .. ⎜ .. ⎝. . (p,n)T 1d

⎞

⎛

143

(5.31e)

T

T

1 d(1,n) z (1,n) ⎜ (2,n)T (2,n)T ⎟ z ⎜1 d ⎟ ⎟ , Wn = ⎜ . .. .. ⎜. ⎟ ⎝. ⎠ . . (p,n)T (p,n)T 1d z

⎞ ⎟ ⎟ ⎟ ⎟ ⎠

for Phase 1, Phase 2, resp., where (i,n)2 (i,n) (i,n) (i,n) (i,n)2 , 2d1 d2 , . . . , 2d1 d(i,n) , d2 , z (i,n) = d1 r T 2 (i,n) (i,n) , i = 1, 2, . . . , p, 2d2 d3 , . . . , d(i,n) r

(5.31f)

(5.31g)

cf. (5.14c). Note that in case (5.31b) the matrices Wn are ﬁxed. Finally, the RSM-step directions hn are deﬁned by ˆ I with β(n) ˆ ˆ (Xn ) := −β(n) = (WnT Wn )−1 WnT yn , (5.32) hn = −Yne = −∇F see (5.8a), (5.9) for Phase 1 and (5.14a), (5.15b) for Phase 2. Of course, we suppose that all matrices WnT Wn , n = 1, 2, . . . , are regular, see (5.8d), (5.14d). 5.3.1 The Argument Case Here we consider the behavior of the hybrid stochastic approximation method (5.5) by means of the mean square errors bn = EXn − x∗ 2 , n ≥ 1, with an optimal solution x∗ of (5.1).

(5.33)

Following [89], in the present case we suppose k0 x − x∗ 2 ≤ (x − x∗ )T ∇F (x) − ∇F (x∗ ) ≤ k1 x − x∗ 2

(5.34a)

k02 x − x∗ 2 ≤ ∇F (x) − ∇F (x∗ )2 ≤ k12 x − x∗ 2 (5.34b) for all x ∈ D where k0 , k1 are some constants such that k1 ≥ k0 > 0. Moreover, for the mean square estimation error

2 Vns (Xn ) = E Yns − ∇F (Xn ) |Xn

(5.34c)

(5.35a)

144

5 RSM-Based Stochastic Gradient Procedures

of the simple stochastic gradient Yns according to (5.4b), if (5.3a) holds true, we suppose that Vns (Xn ) ≤ σ1s2 + C1s Xn − x∗ 2 with constants σ12 , C1s ≥ 0. For n ∈ IN2 with Yns given by (5.4b) we obtain [89] bn+1 ≤ 1 − 2k0 ρn + (k12 + C1s )ρ2n bn + σ1s2 ρ2n .

(5.35b)

(5.36)

For n ∈ IN1 we have, see (5.32), Xn+1 = pD (Xn − ρn Yne ), and therefore [89] E Xn+1 − x∗ 2 |Xn ≤ Xn − x∗ 2 − 2ρn E (Xn − x∗ )T Yne − ∇F (x∗ ) |Xn + ρ2n E Yne − ∇F (Xn )2 |Xn T ∇F (Xn ) − ∇F (x∗ ) |Xn + 2E Yne − ∇F (Xn ) (5.37) + ∇F (Xn ) − ∇F (x∗ )2 . According to (5.19) in Phase 1 and Phase 2 we get E(Yne |Xn ) = ∇F (Xn ) + E Hn (Rn + εn )|Xn . In this equation, cf. (5.10c) and (5.31g), ⎛ 1 Rl (Xn , d(1,n) ) ⎜ R2 (Xn , d(2,n) ) ⎜ l Rn = ⎜ .. ⎝ . Rlp (Xn , d(p(n),n) )

(5.38a)

(5.11b), (5.16b) and (5.17b), resp., (5.31a)– ⎞

⎛

ε(1,n) ε(2,n) .. .

⎜ ⎟ ⎜ ⎟ ⎟ , l = 1, 2, εn = ⎜ ⎝ ⎠

⎞ ⎟ ⎟ ⎟ ⎠

(5.38b)

ε(p(n),n)

is the vector of remainders resulting from using ﬁrst, second order empirical models for F , the vector of stochastic errors in estimating F (x) at Xn + d(i,n) , i = 1, 2, . . . , p(n), respectively. Furthermore, Hn is the r × p matrix given by (5.21a), (5.24a), resp., if there we put d(i) := d(i,n) , z (i) := z (i,n) , i = 1, 2, . . . , p = p(n). Hence, E(Yne |Xn ) = ∇F (Xn ) + edet n , where, cf. (5.20a)–(5.20c), edet n = Hn Rn is the bias resulting from using a ﬁrst or second order empirical model for F . Consequently E (Xn − x∗ )T Yne − ∇F (x∗ ) |Xn = (Xn − x∗ )T ∇F (Xn ) − ∇F (x∗ ) + edet n ≥ k0 Xn − x∗ 2 + (Xn − x∗ )T edet n

(5.39a)

5.3 Estimation of the Mean Square (Mean Functional) Error

145

see (5.34a). Moreover, T T ∇F (Xn )−∇F (x∗ ) |Xn = edet ∇F (Xn )−∇F (x∗ ) , E Yne −∇F (Xn ) n (5.39b) and (5.34b) yields 2 ∇F (Xn ) − ∇F (x∗ ) ≤ k12 Xn − x∗ 2 . Finally, (5.25a) implies that

2 Vne (Xn ) := E Yne − ∇F (Xn ) |Xn

1 H −1 , ≤ Rn 2 + r max σi (Xn ) p(n) n0 1≤i≤p(n)

(5.39c)

(5.39d)

where Hn0 is given by (5.21b), (5.24b), respectively. From (5.37) and (5.39a)– (5.39d) we now have E Xn − x∗ 2 |Xn ≤ (1 − 2k0 ρn + k12 ρ2n )Xn − x∗ 2 T ∗ 2 det T ∗ ∇F (X −2ρn edet (X − x ) + 2ρ e ) − ∇F (x ) n n n n n 1 H −1 . (5.40) +ρ2n Rn 2 + r max σi (Xn )2 p(n) n0 1≤i≤p(n) T T ∇F (Xn ) − ∇F (x∗ ) , we get, Estimating next edet (Xn − x∗ ) and edet n n see (5.20c), (5.21c), (5.24c), (5.34b) / 1 det T −1 ∗ Rn · Xn − x∗ , (5.41a) en (Xn − x ) ≤ Hn0 p(n) / 1 det T −1 ∇F (Xn ) − ∇F (x∗ ) ≤ k1 Hn0 Rn · Xn − x∗ . (5.41b) en p(n) According to (5.27c) we ﬁnd ⎛ p(n) 1 1 (i,n) 2(ln +1) ⎝ Rn ≤ d (ln + 1)! p(n) i=1 p(n) 1

⎞1/2 2 × sup ∇ln +1 F (Xn + ϑd(i,n) ) ⎠ .

(5.41c)

0≤ϑ≤1

Here, (ln )n∈IN1 is a sequence with ln ∈ {1, 2} for all n ∈ IN1 such that {n ∈ IN1 : algorithm (5.5) is in Phase λ} = {n ∈ IN1 : ln = λ},

λ = 1, 2. (5.41d)

146

5 RSM-Based Stochastic Gradient Procedures

Note. For the transition from Phase 1 to Phase 2 statistical F tests may be applied. However, since a sequence of tests must be applied, the signiﬁcance level is not easy to calculate. Furthermore, suppose that representation (5.29a) holds, i.e., d(i,n) = µn d˜(i,n) ,

0 < µn ≤ 1,

i = 1, 2, . . . , p(n),

n0 , cf. where d˜(i,n) , i = 1, . . . , p(n), are given r-vectors. Then Hn0 = µ2n H (5.29b), and (5.41c) yields ⎛ p(n) / / ln 1 µ 1 ˜(i,n) 2(ln +1) n −1 −1 ⎝ Rn ≤ H Hn0 d n0 (ln + 1)! p(n) i=1 p(n) ⎞1/2 2 ln +1 × sup ∇ F (Xn + ϑd˜(i,n) ) ⎠ . (5.41e) 0≤ϑ≤1

Now we assume that 2 sup ∇ln +1 F (Xn + ϑd˜(i,n) ) ≤ γn2 + Γn2 Xn − x∗ 2 , 1 ≤ i ≤ p(n), n ∈ IN1 ,

0≤ϑ≤1

(5.42) where γn ≥ 0, Γn ≥ 0, n ∈ IN1 , are certain coeﬃcients. Then (5.41e) yields / 1 −1 Rn Hn0 (5.43) p(n) ⎛ ⎞1/2 p(n) / ln −1 µn ⎝ 1 ≤ H γn + Γn Xn − x∗ . d˜(i,n) 2(ln +1) ⎠ n0 (ln + 1)! p(n) i=1 Putting (5.41a), (5.41b), (5.41e) and (5.43) into (5.40), with (5.42) we get E Xn+1 − x∗ 2 |Xn ≤ 1 − 2[k0 − Γn µlnn Tn ]ρn + (k1 + Γn µlnn Tn )2 ρ2n Xn − x∗ 2 −2

µn −1 2 2 2 ln 2 H max σi (Xn ) + γn (µn Tn ) + ρn r p(n) n0 1≤i≤p(n) + 2γn (ρn + k1 ρ2n )µlnn Tn Xn − x∗ ,

n ∈ IN1 . (5.44a)

Here, Tn is given by 1 Tn = (ln + 1)!

/

⎞1/2 p(n) 1 −1 ⎝ H d˜(i,n) 2(ln +1) ⎠ . n0 p(n) i=1 ⎛

(5.44b)

5.4 Convergence Behavior of Hybrid Stochastic Approximation Methods

147

In the following we suppose, cf. (5.31a)–(5.31d), that

2 (i,n) (i,n) − F (x ) |Xn =: σi (Xn )2 ≤ σ1F 2 + C1F Xn − x∗ 2 , E y 1 ≤ i ≤ p(n),

(5.45)

with constants σ1F , C1F ≥ 0. Taking expectations on both sides of (5.44a), we ﬁnd the following result: Theorem 5.1. Assume that (5.34a)–(5.34c), (5.42), (5.45) hold, and use representation (5.29a). Then + bn+1 ≤

1 − 2[k0 − Γn µlnn Tn ]ρn + (k1 + Γn µlnn Tn )2 , −2 µ −1 ρ2 bn +rC1F n H n p(n) n0 ∗ +2γn (ρn + k1 ρ2n )µln n Tn EXn − x

µ−2 −1 2 ln 2 ρ2n for all n ∈ IN1 , (5.46) + rσ1F 2 n H + γ (µ T ) n n n p(n) n0

where Tn is given by (5.44b). Remark 5.5. Since the KW-gradient estimator (5.4c) can be represented as a special (Phase 1-)RSM-gradient estimator, for the estimator Yns according to (5.4c) we obtain again an error recursion of type (5.46). Indeed, we have only to set ln = 1 and to use appropriate (KW-)increments d(i,n) := d(i,0) , i = 1, . . . , p, n ≥ 1. 5.3.2 The Criterial Case Here the performance of (5.5) is evaluated by means of the mean functional errors bn = E F (Xn ) − F ∗ , n ≥ 1, with the minimum value F ∗ of (5.1). Under corresponding assumptions, see [89], also in the criterial case an inequality of the type (5.46) is obtained.

5.4 Convergence Behavior of Hybrid Stochastic Approximation Methods In the following we compare the hybrid stochastic approximation method s := (5.5) with the standard stochastic gradient procedure (5.4a), i.e., Xn+1 pD (Xn − ρn Yns ), n = 1, 2, . . . . For this purpose we suppose that (5.3a) holds and Yns is given by (5.4b).

148

5 RSM-Based Stochastic Gradient Procedures

5.4.1 Asymptotically Correct Response Surface Model Suppose that γn = 0 for all n ∈ IN1 . Hence, according to (5.42), the sequence of response surface models Fˆn is asymptotically correct. In this case inequalities (5.36) and (5.46) yield " e e2 2 e2 2 ρn + K1,n ρn )bn+ σ1,n ρn for n ∈ IN1 (1 − 2K0,n (5.47a) bn+1 ≤ 2 s 2 1 − 2k0 ρn + (k1 + C1 )ρn bn + σ1s2 ρ2n for n ∈ IN2 , where k0 , k12 + C1s , σ1s2 are the same constants as in (5.36) and e = k0 − Γn µlnn Tn K0,n

rC1F −1 H µ2n p(n) n0 rσ F 2 −1 = 2 1 H . µn p(n) n0

e2 = (k1 + Γn µlnn Tn )2 + K1,n e2 σ1,n

(5.47b) (5.47c) (5.47d)

A weak assumption is that e K0,n ≥ K0e > 0 for all n ∈ IN1 e2 K1,n e2 σ1,n

≤ ≤

(5.48a)

K1e2 < +∞ for all n ∈ IN1 σ1e2 < +∞ for all n ∈ IN1

(5.48b) (5.48c)

with constants K0e > 0, K1e2 < +∞ and σ1e2 < +∞; note that k0 ≥ K0e , see (5.47b). Remark 5.6. Suppose that (Γn )n∈IN1 , cf. (5.42), is bounded, i.e., 0 ≤ Γn ≤ Γ0 < +∞ for all n ∈ IN1 with a constant Γ0 . The assumptions (5.48a)–(5.48c) can be guaranteed, e.g., as follows: (a) If, for n ∈ IN1 , F (x) is estimated ν(n) times at each one of p0 diﬀerent n0 = H (0) points Xn + d(i) , i = 1, 2, . . . , p0 , see (5.22a), (5.22b), then H 0 (0) with a ﬁxed matrix H for all n ∈ IN1 . Since ln ∈ {1, 2} for all n ∈ IN1 , 0 we ﬁnd, cf. (5.44b), Tn ≤ T0 for all n ∈ IN1 with a constant T0 < +∞. Because of 0 < µn ≤ 1, p(n) = ν(n)p0 , (5.47b)–(5.47d) yield e ≥ k0 − Γ0 T0 µn K0,n e2 K1,n ≤ (k1 + Γ0 T0 )2 +

rC1F (0)−1 H0 , µ2n

e2 σ1,n ≤

rσ1F 2 (0)−1 H0 . µ2n

Thus, conditions (5.48a)–(5.48c) hold for arbitrary ν(n), n ∈ IN1 , if k0 − K0e ≥ µn ≥ µ0 > 0 for all n ∈ IN1 with Γ0 T 0 k0 − K0e 0 < K0e < k0 , 0 < µ0 < . Γ0 T 0

(5.48d)

5.4 Convergence Behavior of Hybrid Stochastic Approximation Methods

149

˜ n0 = H ˜0 (b) In case (5.31b), i.e., if d˜(i,n) = d˜(i,0) , i = 1, 2, . . . , p, n ≥ 1, then H ˜ 0 and therefore Tn ≤ T0 < +∞ for all for all n ∈ IN1 with a ﬁxed matrix H e e2 e2 , K1,n , σ1,n , n ∈ IN1 , with ﬁxed T0 , since ln ∈ {1, 2}, n ∈ IN1 . Thus, for K0,n (0) ˜ ˜ 0. resp., we obtain the same estimates as above if H is replaced by H 0 Hence, also in this case, conditions (5.48a)–(5.48c) hold for an arbitrary p if (µn )n∈IN1 is selected according to (5.48d). According to (5.36), for the mean square errors of (5.4a), (5.4b) bsn := EXns − x∗ 2 ,

n = 1, 2, . . . ,

we have the “upper” recurrence relation bsn+1 ≤ 1 − 2k0 ρn + (k12 + C1s )ρ2n bsn + σ1s2 ρ2n ,

n = 1, 2, . . . .

In addition to (5.35b) assume Vns (Xn ) = E Yns − ∇F (Xn )2 |Xn ≥ σ0s2 + C0s Xn − x∗ 2

(5.49)

(5.50)

with coeﬃcients 0 < σ0s ≤ σ1s , 0 ≤ C0s ≤ C1s . Using (5.34a), (5.34b), we ﬁnd, see [85, 89], for (bsn ) also the “lower” recursion (5.51a) bsn+1 ≥ 1 − 2k1 ρn + (k02 + C0s )ρ2n bsn + σ0s2 ρ2n for n ≥ n0 , provided that, e.g., Xns − ρn Yns ∈ D a.s. for all n ≥ n0 .

(5.51b)

Having the lower recursion (5.51a) for (bsn ), we can compare algorithms (5.4a), (5.4b) and (5.5) by means of the quotients bn bn ≤ s for n ≥ n0 , s bn Bn

(5.52a)

where the sequence (B sn ) of lower bounds B sn ≤ bsn , n ≥ n0 , is deﬁned by B sn+1 = 1 − 2k1 ρn + (k02 + C0s )ρ2n B sn + σ0s2 ρ2n , n ≥ n0 , with B n0 ≤ bsn0 . (5.52b) In all other cases we consider bn s

Bn

, n = 1, 2, . . . , s

(5.52c)

where the sequence of upper bounds B n ≥ bsn , n = 1, 2, . . . , deﬁned by s s s B n+1 = 1 − 2k0 ρn + (k12 + C12 )ρ2n B n + σ1s2 ρ2n , n = 1, 2, . . . , B 1 ≥ bs1 , (5.52d)

150

5 RSM-Based Stochastic Gradient Procedures

represents the “worst case” of (5.4a), (5.4b). For the standard step sizes ρn =

c with constants c > 0, q ∈ IN ∪ {0} n+q

(5.53)

and with k1 ≥ k0 > 0 we have, see [164], 1 c2 for all c > , n→∞ 2k1 c − 1 2k1 1 c2 c2 s ≥ σ0s2 for all c > lim n · B n = σ1s2 . n→∞ 2k0 c − 1 2k1 c − 1 2k0 lim n · B sn = σ0s2

(5.54a) (5.54b)

For the standard step sizes (5.53) the asymptotic behavior of (bn /bsn ), s (bn /B n ), resp., can be determined by means of the methods developed in the following Sect. 5.5: Theorem 5.2. Suppose that for the RSM-based stochastic approximation procedure (5.5) the upper error recursion (5.47a) holds, where (5.48a)–(5.48c) is fulﬁlled, and (ρn ) is deﬁned by (5.53). Moreover, suppose that groups of M iterations with the RSM-gradient estimator Yne and N iterations with the simple gradient estimator Yns are taken by turns. (1) If the lower estimates (5.51a) of (bsn ) hold, then lim sup n→∞

1 σ1e2 M + σ1s2 N 2k1 c − 1 bn · for all c > ≤ bsn σ0s2 (M + N ) 2K0e c − 1 2K0e

(5.55a)

s

(2) If (bn ) is compared with (B n ), then lim sup n→∞

bn s Bn

≤

1 σ1e2 M + σ1s2 N 2k0 c − 1 · for all c > σ1s2 (M + N ) 2K0e c − 1 2K0e

(5.55b)

Note. It is easy to see that the right hand side of (5.55a), (5.55b), resp., takes any value below 1, provided that σ1e2 and the rate N/(N + M ) of steps with a “simple” gradient estimator Yns are suﬃciently small. Instead of applying the standard step size rule (5.53) and having then Theorem 5.2, we may use the optimal step sizes developed in [89]. Here estimates of the coeﬃcients appearing in the optimal step size rules may be obtained from second order response surface models, see also the next Chap. 6. 5.4.2 Biased Response Surface Model Assume now that γn > 0 for all n ∈ IN1 . For n ∈ IN1 , according to Theorem 5.1 we have ﬁrst to estimate Exn − x∗ : For any given sequence (δn )n∈IN1 of positive numbers, we know [164] that EXn − x∗ ≤ δn +

1 bn for all n ∈ IN1 . δn

(5.56)

5.4 Convergence Behavior of Hybrid Stochastic Approximation Methods

151

Taking, see (5.47b), (5.48a), δn :=

4γn (1 + k1 ρn )Tn ln · µn , k0 − Γn µlnn Tn

n ∈ IN1 ,

(5.57)

according to (5.46), (5.56) and (5.57) we obtain, see (5.47b)–(5.47d),

bn+1 ≤

2 8 γn (1 + k1 ρn )Tn (µlnn )2

3 e e2 2 1 − K0,n ρn + K1,n ρn bn + 2 e2 + σ1,n + γn2 (µlnn Tn )2 ρ2n for all n ∈ IN1 .

e K0,n

ρn (5.58)

Inequalities (5.36) and (5.58) may be represented jointly by bn+1 ≤ un bn + vn ,

n = 1, 2, . . .

(5.59)

where the coeﬃcients un , vn can be taken easily from (5.36) and (5.58). We want now to select (ρn ), (µn )n∈IN1 and p(n) such that n∈IN1

un ≤ 1 − Aρn for n ≥ n0 with n0 ∈ IN and ﬁxed A > 0.

(5.60)

According to (5.36), (5.58) we have to select A > 0 such that −2k0 + (k12 + C1s )ρn ≤ −A for n ∈ IN2 , n ≥ n0 (5.61a) 3 ˜ −1 ρn ≤ −A − (k0 − Γn µlnn Tn ) + (k1 + Γn µlnn Tn )2 ρn + rC1F H n0 2 µ2n p(n) for n ∈ IN1 , n ≥ n0 . (5.61b) 2 ˜ 2 it is Assuming that with ﬁxed numbers K0e > 0, K11 ≥ 0 and K 12 e K0,n = k0 − Γn µlnn Tn ≥ K0e for all n ∈ IN1

(5.62a)

2 (k1 + Γn µlnn Tn )2 ≤ K11 < +∞ for all n ∈ IN1

(5.62b)

˜ −1 ≤ K ˜ 2 < +∞ for all n ∈ IN1 , rC1F H 12 n0

(5.62c)

see (5.47b)–(5.47d), (5.48a)–(5.48c) and Remark 5.6, we ﬁnd that (5.61b) is implied by 2 ˜2 ρn + K K11 12

3 ρn ≤ K0e − A for n ∈ IN1 , n ≥ n0 . µ2n p(n) 2

(5.61b )

Thus, selecting A > 0 such that 0

3 3 e K0 ≤ k0 < 2k0 , 2 2

(5.63a)

152

5 RSM-Based Stochastic Gradient Procedures

we ﬁnd that (5.61a), (5.61b) and therefore also (5.60) hold, provided that ρn → 0, n → ∞ ρn → 0, n → ∞ with n ∈ IN1 . µ2n p(n) Note that (5.63c) holds for any sequence p(n) if

(5.63b) (5.63c)

n∈IN1

ρn → 0, n → ∞ with n ∈ IN1 . µ2n

(5.63c )

Having (5.62a)–(5.62c) and (5.63a)–(5.63c), inequalities (5.59) and (5.60) yield bn+1 ≤ (1 − Aρn )bn + vn for all n ≥ n0 . Finally, supposing still that ∞

ρn = +∞ and

n=1

∞

vn < +∞,

(5.63d), (5.63e)

n=1

the above recursion implies bn → 0 as n → ∞. Condition (5.63e) demands, with respect to IN2 , that ρ2n < +∞. (5.63f) n∈IN2

Concerning IN1 we assume that ˜ −1 ≤ h0 for all n ∈ IN1 γn ≤ γ0 , Tn ≤ T0 , H n0

(5.64)

with certain constants c0 > 0, T0 > 0, h0 > 0. Under these assumptions condition (5.63e) holds if – besides (5.63f) – we still demand that

(µlnn )2 ρn < +∞,

n∈IN1

n∈IN1

ρ2n < +∞ µ2n p(n)

(5.65a), (5.65b)

(µlnn )2 ρ2n < +∞,

(5.65c)

n∈IN1

cf. (5.63b). Since 0 < µn ≤ 1, n ∈ IN1 , (5.63f) and (5.65c) hold if

∞ ' n=1

ρ2n < +∞.

In summary, in the present case we have the following result: Theorem 5.3. Suppose that, besides the general assumptions guaranteeing in˜ −1 equalities (5.36) and (5.46), (γn )n∈IN1 , (Γn )n∈IN1 , (Tn )n∈IN1 and H n0 n∈IN1 are bounded. If (ρn ), (µn )n∈IN1 , p(n) are selected such that n∈IN1

k0 −K0e Γ0 T0

for all n ∈ IN1 , where 0 < K0e < k0 and Γ0 > 0, T0 > 0 (1) 0 < µn < are upper bounds of (Γn )n∈IN1 , (Tn )n∈IN1

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

153

(2) Conditions (5.63b)–(5.63d), (5.63f ) and (5.65a)–(5.65c) hold, then EXn − x∗ 2 → 0 as n → ∞ In the literature on stochastic approximation [164] many results are available on the convergence rate of numerical sequences (βn ) deﬁned by recursions of the type cn dn βn + π+1 , βn+1 ≤ 1 − n n

n = 1, 2, . . . ,

with lim inf cn = c > π and lim sup dn = d > 0. Hence, the above recursions n→∞

n→∞

on (bn ) yield still the following result, cf. (5.54a), (5.54b): Theorem 5.4. Let the sequence of stepsizes (n ) be given by (5.53). Assume that the general assumptions of Theorem 5.3 are fulﬁlled. (1) If p(n) = 0(n) and µn = 0(n− 2 ), then bn = 0(n−1 ), provided that c > 3 . 2K0e 1 1 (2) If p(n) is bounded and µn = 0(n− 4 ), then bn = 0(n− 2 ) for all 1

3 . c> 2K0e

n∈IN1

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures In order to compare the speed of convergence of the pure stochastic algorithm (5.4a), (5.4b) and the hybrid stochastic approximation algorithm (5.5), we consider the mean square error bn = EXn − x∗ 2 , where x∗ is an optimal solution of (5.1). Some preliminaries are needed for the derivation of a recursion for the error sequence (bn ). First, we note that the projection operator pD from IRr on D is deﬁned by x − pD (x) = inf x − y for all x ∈ IRr . y∈D

It is known [16] that pD (x) − pD (y) ≤ x − y for all x, y ∈ IRr .

(5.66)

As mentioned in Sect. 5.1, see also Sect. 4, many stochastic optimization problems (5.1) have the following property:

154

5 RSM-Based Stochastic Gradient Procedures

Property (DD). At certain “nonstationary” or “noneﬃcient” points x∈D there exits a feasible descent direction (DD) h = h(x) of F which can be obtained with much less computational expenses than the gradient ∇F (x). Moreover, h = h(x) is stable with respect to changes of the loss function contained in f = f (ω, x), see [77, 82, 83]. Consider then the mapping A : IRr → IRr deﬁned by −h(x), if (DD) holds at x A(x) = (5.67) ∇F (x), else. In many cases [77, 82, 83] h(x) can be represented by h(x) = y − x with a certain vector y ∈ D. Hence, in this case we get pD (Xn + n hn ) = Xn + n hn , if 0 < n ≤ 1, and the projection onto D can be omitted then. We suppose now pD x∗ − A(x∗ ) = x∗ for all > 0.

(5.68)

for every optimal solution x∗ of (5.1). This assumption is justiﬁed, since for a convex, diﬀerentiable objective function F , the optimal solutions x∗ of (5.1) can be characterized [16] by pD x∗ − ∇F (x∗ ) = x∗ for all > 0, and in x∗ there is no feasible descent direction h(x∗ ) on the other hand. The hybrid algorithm (5.5) can be represented then by Xn+1 = pD Xn − n A(Xn ) + ξn , n = 1, 2, . . . ,

(5.69a)

where the noise r-vector ξn is deﬁned as follows: (1) Hybrid methods with deterministic descent directions h(Xn ) and standard gradient estimators Yns 0, if n ∈ IN1 (5.69b) ξn := Yns − ∇F (Xn ), if n ∈ IN2 , where A(x) is deﬁned by (5.67). (2) Hybrid methods with RSM-gradient estimators Yne and simple gradient estimators Yns e Yn − ∇F (Xn ), if n ∈ IN1 (5.69c) ξn := Yns − ∇F (Xn ), if n ∈ IN2 , where A(x) := ∇F (x). While for hybrid methods with mixed RSM-/standard gradient estimators Yne , Yns a recurrence relation for the error sequence (bn ) was given in Theorem 5.1 and in Sect. 5.4.1, for case (i) we have this result:

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

155

Lemma 5.1 (Mixed feasible descent directions A(Xn ) and standard gradient estimators Yns ). Let x∗ be an optimal solution. Assume that the optimality condition (5.68) holds and the following conditions are fulﬁlled 2 A(x) − A(x∗ ) ≤ γ1 + γ2 x − x∗ 2 , T A(x) − A(x∗ ) (x − x∗ ) ≥ αx − x∗ 2

(5.70) (5.71)

for every x ∈ D, where γ1 ≥ 0, γ2 > 0 and α > 0 are some constants. Moreover, for n ∈ IN2 assume that E(Yns |Xn ) = ∇F (Xn )a.s. (almost sure) EYns 2 < +∞,

(5.72) (5.73)

Then bn = EXn − x∗ 2 satisﬁes the recursion bn+1 ≤ (1 − 2αn + γ2 2n )bn + βn 2n , where βn is deﬁned by

βn = γ1 + Eξn 2 .

n = 1, 2, . . . ,

(5.74a) (5.74b)

Proof. Using (5.66) and (5.68)–(5.73), we ﬁnd 2 Xn+1 − x∗ 2 = pD Xn − n A(Xn ) + ξn − pD x∗ − n A(x∗ ) 2 ≤ (Xn − x∗ ) − n A(Xn ) − A(x∗ ) − n ξn T ≤ Xn − x∗ 2 − 2n A(Xn ) − A(x∗ ) (Xn − x∗ ) +2n A(Xn ) − A(x∗ )2 −2n ξnT Xn − x∗ − n A(Xn ) − A(x∗ ) + 2n ξn 2 ≤ Xn − x∗ 2 − 2αn Xn − x∗ 2 + 2n γ1 + γ2 Xn − x∗ 2 −2ξnT Xn − x∗ − n A(Xn ) − A(x∗ ) + 2n ξn 2 . From (5.69b) and (5.72) we obtain E(ξn |Xn ) = 0 a.s. for every n. Hence, from the above inequality we a.s. have E Xn+1 − x∗ 2 |Xn ≤ (1 − 2αn + γ2 2n )Xn − x∗ 2 +γ1 2n + 2n E ξn 2 |Xn . Taking expectations on both sides, we get the asserted recursion (5.74a), (5.74b).

156

5 RSM-Based Stochastic Gradient Procedures

We want to discuss in short the assumptions in the above lemma. First we remark that (5.70) holds if (5.75) A(x) − A(x∗ ) ≤ k1 + k2 x − x∗ for some constants k1 ≥ 0, k2 > 0. Indeed, (5.75) implies (5.70) with γ1 = k12 + k1 k2 , γ2 = k1 k2 + k22 . If inequality (5.71) is satisﬁed for A(x) = ∇F (x), then F is called strongly convex at x∗ . For A(x) = −h(x) being a feasible descent direction, condition (5.71) reads h(x) (x∗ − x) ≥ αx − x∗ 2 for x ∈ D. Since hopt (x) = x∗ − x is the best descent direction in X, the above inequality may be regarded as a condition on the quality of h(x) as a descent direction. The remaining conditions (5.72), (5.73) state that the error ξn in a stochastic step has mean zero and ﬁnite second order moments. Considering now the convergence behavior of (5.74a), (5.74b), in the literature, see, e.g., [132, 135, 160, 164], the following type of results is available. Lemma 5.2. Let (bn ) be a sequence of nonnegative numbers such that µn τ bn + π+1 , n ≥ n0 (5.76) bn+1 ≤ 1 − n n for some positive integer n0 . If lim µn = µ and τ > 0, then n→∞

bn = 0(n−π ), if µ > π > 0, n , if µ = π > 0, bn = 0 log nµ bn = 0(n

−µ

), if π > µ > 0.

If (bn ) satisﬁes the recursion µn τn bn + π+1 , n > n0 , bn+1 = 1 − n n

(5.77a) (5.77b) (5.77c)

(5.78)

where lim µn = µ > π > 0 and lim τn = τ ≥ 0, then n→∞

n→∞

lim nπ bn =

n→∞

τ . µ−π

(5.79)

Applying Lemmas 5.2–(5.74a), (5.74b), this result is obtained: Corollary 5.1. Suppose that the assumptions in Lemma 5.1 are fulﬁlled and, in addition, let (5.80) Eξn 2 ≤ σ 2 < +∞ for n ∈ IN2 . c , where c > 0 and If the step size n is chosen according to n = n+q 2 q ∈ IN ∪ {0}, then lim E Xn (ω) − x∗ = 0, where the asymptotic rate of n→∞

convergence is given by (5.77a)–(5.77c) with π = 1 and µ = 2αc.

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

157

c , it is easy to see that (5.74a), (5.74b) Proof. Having (5.80) and n = n+q n n − γ2 c2 implies (5.76) for µn = 2αc , µ = 2αc, τ = c2 (γ1 + σ 2 ) n+q (n + q)2 and π = 1. The corollary follows now from Lemma 5.2.

Assuming now that lim n = 0,

n→∞

there is an integer n0 > 0 such that 0 < 1 − 2αn + γ2 2n < 1 − an , n ≥ n0 ,

(5.81)

where a is a ﬁxed positive number such that 0 < a < 2α. Furthermore, assume that (5.80) holds. Then (5.74a), (5.74b) and (5.81) yield the recursion bn+1 ≤ (1 − an )bn + β¯n 2n , where β¯n is given by

β¯n =

n ≥ n0 ,

(5.82)

γ1 , if n ∈ IN1 γ1 + σ 2 , if n ∈ IN2 .

(5.83)

Next we want to consider the “worst case” in (5.82), i.e., the recursion Bn+1 = (1 − an )Bn + β¯n 2n ,

n ≥ n0 .

(5.84a)

If bn and Bn satisfy the recursions (5.82) and (5.84a), respectively, and if Bn0 ≥ bn0 , then by a simple induction argument we ﬁnd that 2 E Xn (ω) − x∗ = bn ≤ Bn for all n ≥ n0 . (5.84b) Corresponding to Corollary 5.1, for the upper bound Bn of the mean square error bn we have this Corollary 5.2. Suppose that all assumptions in Lemma 5.1 are fulﬁlled and c with c > 0 and q ∈ IN ∪ {0}, then for (Bn ) we (5.80) holds. If n = n+q have the asymptotic formulas Bn = 0(n−1 ), if ac > 1,

log n Bn = 0 , if ac = 1, n Bn = 0(n−ac ), if ac < 1 as n → ∞. Furthermore, if ac > 1 and IN1 = ∅, i.e., in the pure stochastic case, we have c2 . lim nBn = (γ1 + σ 2 ) n→∞ ac − 1

158

5 RSM-Based Stochastic Gradient Procedures

Proof. The assertion follows by applying Lemmas 5.2–(5.84a), (5.84b).

Since by means of Corollaries 5.1 and 5.2 we can not distinguish between the convergence behavior of the pure stochastic and the hybrid stochastic approximation algorithm, we have to consider recursion (5.84a), (5.84b) in more detail. It turns out that the improvement of the speed of convergence resulting from the use of both stochastic and deterministic directions depends on the rate of stochastic and deterministic steps taken in (5.69a)–(5.69c). Note. For simplicity of notation, an iteration step Xn → Xn+1 using an improved step direction, a standard stochastic gradient is called a “deterministic step,” a “stochastic step,” respectively. 5.5.1 Fixed Rate of Stochastic and Deterministic Steps Let us assume that the rate of stochastic and deterministic steps taken in M , i.e., M deterministic and N stochas(5.69a)–(5.69c) is ﬁxed and given by N tic steps are taken by turns beginning with M deterministic steps. Then Bn satisﬁes recursion (5.84a), (5.84b) with β¯n given by (5.83), i.e., γ1 for n ∈ IN1 ¯ βn = γ1 + σ 2 for n ∈ IN2 , where IN1 , IN2 are deﬁned by IN1= n ∈ IN : (m−1)(M + N )+1 ≤ n ≤ mM + (m − 1)N for some m ∈ IN , ˜ + (m ˜ − 1)N + 1 ≤ n ≤ m(M ˜ + N ) for some IN2=IN\IN1 = n ∈ IN : mM m ˜ ∈ IN . We want to compare Bn with the upper bounds Bns corresponding to the purely stochastic algorithm (5.4a), (5.4b), i.e., Bns shall satisfy (5.84a), (5.84b) with β¯n = γ1 + σ 2 for all n ≥ n0 . We assume that the step size 0 is chosen according to n =

c n+q

with a ﬁxed q ∈ IN ∪ {0} and c > 0 such that ac ≥ 1. Then the recursions to be considered are β˜n ϑ Bn + (n+q) Bn+1 = 1 − n+q 2 , n ≥ n0 , s ϑ B Bns + (n+q) = 1 − n+q Bn+1 2 , n ≥ n0 ,

(5.85) (5.86)

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

where

β˜n =

159

A for n ∈ IN1 B for n ∈ IN2 ,

A = c2 γ1 , B = c2 (γ1 + σ 2 ) and ϑ = ac. We need two lemmata (see [164] for the ﬁrst and [127] for the second). Lemma 5.3. Let ϑ ≥ 1 be a real number and

n ϑ Ck,n = for k, n ∈ IN, k < n. 1− m m=k+1

Then for every ε > 0 there is a kε ∈ IN such that ϑ ϑ k k (1 − ε) ≤ Ck,n ≤ (1 + ε) n n for all n > k ≥ kε . Lemma 5.4. Let the numbers yn satisfy the recursion yn+1 = Un yn + Vn for n ≥ ν, where (Un ), (Vn ) are given sequences of real numbers such that Un = 0 for n ≥ ν. Then ⎞ ⎛ n n V j ⎠ Um for n ≥ ν. yn+1 = ⎝yν + 'j m=ν m=ν Um j=ν We are going to prove the following result: Theorem 5.5. Let the sequence (Bn ) and (Bns ) be deﬁned by (5.85) and (5.86), respectively. If ϑ ≥ 1, then γ1 M + (γ1 + σ 2 )N AM + BN Bn = . = Q := s n→∞ Bn B(M + N ) (γ1 + σ 2 )(M + N ) lim

Proof. Using the notation of Lemma 5.3, we have j 1− m=ν

ϑ m+q

= Cν+q−1,j+q for j ≥ ν ≥ 1,

and as a simple consequence of (5.85), (5.86) and Lemma 5.4 we note Bn+1 = s Bn+1

Bν + Bνs +

n ' j=ν n ' j=ν

β˜j (j+q)2 Cν+q−1,j+q

, n≥ν B (j+q)2 C

ν+q−1,j+q

(5.87)

160

5 RSM-Based Stochastic Gradient Procedures

for every ﬁxed integer ν > n0 . By Lemma 5.3 for every 0 < ε < 1 there is an integer p = ν − 1, ν > n0 , such that (1 − ε)

p+q j+q

ϑ

≤ Cp+q,j+q ≤ (1 + ε)

p+q j+q

ϑ for j > p.

Consequently, for n ≥ ν we ﬁnd the inequality Bν +

1 (1+ε)(p+q)ϑ

Bνs +

n ' β˜j (j + q)ϑ−2 j=ν n '

B (1−ε)(p+q)ϑ

Bν + ≤

Bνs +

(5.88)

β˜j (j + q)ϑ−2

j=ν n '

B (1+ε)(p+q)ϑ

'

Since ϑ ≥ 1, the series

(j + q)ϑ−2

j=ν n '

1 (1−ε)(p+q)ϑ

Bn+1 s Bn+1

≤

. (j + q)ϑ−2

j=ν

(j + q)ϑ−2 diverges as n → ∞. So the numbers

j=ν

Bν and Bνs have no inﬂuence on the limit of both sides of the above inequalities (5.88) as n → ∞. Suppose now that we are able to show the equation lim

n→∞

AM + BN S(ν, n) = = L, T (ν, n) M +N

(5.89)

where S(ν, n) =

n

β˜j (j + q)ϑ−2 ,

j=ν

T (ν, n) =

n

(j + q)ϑ−2 .

j=ν

Taking the limit in (5.88) for n → ∞, for every ε > 0 we obtain 1+ε L Bn+1 Bn+1 1−ε L · ≤ lim inf s · . ≤ lim sup s ≤ j→∞ Bn+1 1+ε B B 1−ε B n→∞ n+1 But this can be true only if lim

n→∞

Bn+1 exists and s Bn+1

AM + BN L Bn+1 = = s n→∞ B B B(M + N ) n+1 lim

which proves (5.87). Hence, it remains to show that the limit (5.89) exists and is equal to L. For this purpose we may assume without loss of generality that

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

161

ν = m0 (M + N ) + 1 for some integer m0 . Furthermore, we assert that from the existence of S ν, m(M + N ) lim (5.90) m→∞ T ν, m(M + N ) the existence of the limit (5.89) follows. To see this, let n ≥ ν be given and let m = mn be the uniquely determined integer such that m(M + N ) + 1 ≤ n < (m + 1) (M + N ). Since

S ν, (m + 1)(M + N ) = S(ν, n) + S n + 1, (m + 1)(M + N ) ,

for m(M + N ) + 1 ≤ n < (m + 1)(M + N ) a simple computation yields S ν, (m + 1)(M + N ) S(ν, n) − (5.91) T (ν, n) T ν, (m + 1)(M + N ) S n + 1, (m + 1)(M + N ) T n + 1, (m + 1)(M + N ) · S(ν, n) − = . T ν, (m + 1)(M + N ) T ν, (m + 1)(M + N ) · T (ν, n) Clearly, from the deﬁnition of S(ν, n), T (ν, n) and β˜j ≤ B for all j ∈ IN follows that S n + 1, (m + 1)(M + N ) T m(M + N ) + 1, (m + 1)(M + N ) ≤B T ν, (m + 1)(M + N ) T ν, (m + 1)(M + N ) (5.92a) and T m(M + N )+1, (m + 1)(M + N ) T n + 1, (m + 1)(M + N ) ·S(ν, n) ≤B . T ν, (m + 1)(M + N ) · T (ν, n) T ν, (m + 1)(M + N ) (5.92b) Assume now that

T m(M + N ) + 1, (m + 1)(M + N ) lim = 0. n→∞ T ν, (m + 1)(M + N )

(5.93)

Since n → ∞ implies that m = mn → ∞, from (5.91), the inequalities (5.92a), (5.92b) and assumption (5.93) we obtain

162

5 RSM-Based Stochastic Gradient Procedures

S ν, (mn + 1)(M + N ) S(ν, n) − = 0. lim n→∞ T (ν, n) T ν, (mn + 1)(M + N ) Therefore, if the limit (5.90) exists, then also the limit (5.89) exists and both limits are equal. For 1 ≤ ϑ ≤ 2 the limit (5.93) follows from T m(M + N ) + 1, (m + 1)(M + N ) ≤ M + N and the divergence of T ν, (m + 1)(M + N ) as m → ∞. For ϑ > 2 we have ϑ−2 T m(M + N ) + 1, (m + 1)(M + N ) ≤ (M + N ) (m + 1)(M + N ) + q and with ν = m0 (M + N ) + 1 m T ν, (m + 1)(M + N ) =

(k+1)(M +N )

(j + q)ϑ−2

k=m0 j=k(M +N )+1

≥ (M + N )

m ϑ−2 k(M + N ) + q . k=m0

Consider the monotonously increasing function ϑ−2 u(x) = x(M + N ) + q , x ≥ 0. Because of

k+1

k+1

u(x − 1) dx ≤ k

u(k + 1 − 1)dx = u(k), k

we have that m T ν, (m + 1)(M + N ) ≥ (M + N ) u(k) k=m0 k+1 m+1 m u(x − 1)dx = (M + N ) u(x − 1)dx ≥ (M + N )

1 = ϑ−1

k=m0 k

m0

ϑ−1 m(M + N ) + q

− (m0 − 1)(M + N ) + q

ϑ−1 .

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

163

Therefore, for ϑ > 2 we ﬁnd T m(M + N ) + 1, (m + 1)(M + N ) T ν, (m + 1)(M + N ) ⎧ ⎪ ⎨ m(M + N ) + q ϑ−2 m(M + N ) + q ≤ (ϑ − 1)(M + N ) ⎪ (m + 1)(M + N ) + q ⎩ ϑ−1 ⎫−1 ⎪ ⎬ (m0 − 1)(M + N ) + q − ϑ−2 ⎪ . ⎭ (m + 1)(M + N ) + q

This yields then the limit (5.93). It remains to show that (5.90) and consequently also (5.89) holds. Again with ν = m0 (M + N ) + 1 we have m−1 S ν, m(M + N ) =

=

m−1

⎛

=

m−1

β˜j (j + q)ϑ−2

(5.94)

k=m0 j=k(M +N )+1

(k+1)M +kN

⎝A

k=m0

(k+1)(M +N )

⎞

(k+1)(M +N )

(j + q)ϑ−2 + B

j=k(M +N )+1

(j + q)ϑ−2 ⎠

j=(k+1)M +kN +1

σk ,

k=m0

where σk is deﬁned by σk = A

M M +N ϑ−2 ϑ−2 k(M + N ) + i + q k(M + N ) + i + q +B . i=1

i=M +1

In the same way we ﬁnd m−1 τk , T ν, m(M + N ) =

(5.95)

k=m0

where τk is given by τk =

M +N

ϑ−2

k(M + N ) + i + q

.

i=1

Deﬁne now the functions f (x) = A

M M +N ϑ−2 ϑ−2 x(M + N ) + i + q x(M + N ) + i + q +B i=1

i=M +1

164

5 RSM-Based Stochastic Gradient Procedures

and g(x) =

M +N

ϑ−2

x(M + N ) + i + q

.

i=1

Then σk = f (k) and τk = g(k). For 1 ≤ ϑ ≤ 2 the functions f and g are monotonously nonincreasing. Hence, k+1

k+1

f (x)dx ≤ f (k) = σk ≤

f (x − 1)dx

k

and therefore

k

m f (x)dx ≤

m−1

m σk ≤

k=m0

m0

f (x − 1)dx. m0

Analogously we ﬁnd m g(x)dx ≤

m−1

m τk ≤

k=m0

m0

g(x − 1)dx. m0

By (5.94) and (5.95), in the case 1 ≤ ϑ ≤ 2 we have now m

m

f (x − 1)dx S ν, m(M + N ) m0 m ≤ 0 m ≤ . m T ν, m(M + N ) g(x − 1)dx g(x)dx f (x)dx

m0

m0

If ϑ > 2, then f and g are monotonously increasing and therefore k+1

k+1

f (x)dx ≥ f (k) = σk ≥ k

f (x − 1)dx, k

hence, m f (x)dx ≥

m−1

m σk ≥

k=m0

m0

f (x − 1)dx, m0

and in the same way we get m g(x)dx ≥ m0

m−1 k=m0

m τk ≥

g(x − 1)dx. m0

(5.96a)

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

Consequently, if ϑ > 2, then by (5.94) and ( 5.95) we have )m )m f (x − 1)dx f (x)dx S ν, m(M + N ) m0 m0 ≤ ≤ )m )m T ν, m(M + N ) g(x)dx g(x − 1)dx m0

165

(5.96b)

m0

Hence, in both cases we have the same type of upper and lower bounds for S ν, m(M + N ) . T ν, m(M + N ) We have to discuss two cases: (i) Let ϑ = 1. In this case we ﬁnd m f (x − 1)dx = m0

M A log (m − 1)(M + N ) + i + q M + N i=1

− log (m0 − 1)(M + N ) + i + q

+

B M +N

M +N

log (m − 1)(M + N ) + i + q

i=M +1

A·M log(m − 1) M +N

M A i+q + − log (m0 − 1)(M +N ) + i + q log M +N + M + N i=1 m−1

M +N B·N B i+q + log(m − 1) + log M + N + M +N M +N m−1 i=M +1 − log (m0 − 1)(M + N ) + i + q − log (m0 − 1)(M + N ) + i + q

and in the same way m g(x)dx = log m + m0

=

M +N i+q 1 log M + N + M + N i=1 m − log m0 (M + N ) + i + q .

Since in the above integral representations all terms besides log(m − 1), have now

AM M +N

BN log(m − 1) and log m are bounded as m → ∞, we M +N

166

5 RSM-Based Stochastic Gradient Procedures

m f (x − 1)dx lim

m0

m→∞

= lim

m

m→∞

log(m − 1) BN AM · + M +N log m M +N

g(x)dx m0

× Analogously

log(m − 1) log m

=

AM + BN . M +N

m f (x)dx m0 lim m→∞ m

=

AM + BN , M +N

g(x − 1)dx m0

and the theorem is proved for ϑ = 1. (ii) Let now ϑ > 1. Here we ﬁnd m f (x − 1)dx = m0

=

M ϑ−1 A (m − 1)(M + N ) + i + q (M + N )(ϑ − 1) i=1 ϑ−1 − (m0 − 1)(M + N ) + i + q

M +N ϑ−1 B (m − 1)(M + N ) + i + q + (M + N )(ϑ − 1) i=M +1 ϑ−1 − (m0 − 1)(M + N ) + i + q

ϑ−1 M i+q Amϑ−1 1 (M + N ) + = 1− (M + N )(ϑ − 1) i=1 m m

ϑ−1 i+q m0 − 1 (M + N ) + − m m

ϑ−1 M +N i+q 1 Bmϑ−1 (M + N ) + 1− + (M + N )(ϑ − 1) m m i=M +1

ϑ−1 i+q m0 − 1 (M + N ) + − m m

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

as well as )m m0

167

ϑ−1 M +N i+q mϑ−1 M +N + g(x)dx = (M + N )(ϑ − 1) i=1 m

ϑ−1 i+q m0 (M + N ) + − m m

and therefore m f (x − 1)dx lim

m→∞

m0

=

m

AM (M + N )ϑ−1 + BN (M + N )ϑ−1 AM + BN . = ϑ−1 (M + N )(M + N ) M +N

g(x)dx m0

Analogously

m f (x)dx m0 lim m→∞ m

=

AM + BN , M +N

g(x − 1)dx m0

which completes now the proof of our theorem.

Discussion of Theorem 5.5 If M deterministic an N stochastic steps are taken by turns and the step 1 c size n is chosen such that n = with q ∈ IN and c ≥ , where 0
σ2 Bn ≈ 1 − w Bns as n → ∞, γ1 + σ 2 M is the percentage of deterministic steps in one complete M +N turn of M + N deterministic and stochastic steps. Hence, here we ﬁnd that −1 2 times faster asymptotically the semi-stochastic procedure is 1 − γ1σ+σ2 w than the pure stochastic approximation procedure. This conclusion is studied in more detail in Sect. 5.5.2. where w =

Comparison of Hybrid Stochastic Approximation Procedures By the above results we are also able to compare two hybrid stochastic approximation procedures characterized by the parameters A, B, M, N, ϑ and

168

5 RSM-Based Stochastic Gradient Procedures

˜ respectively. Corresponding to these two tuples of parameters ˜ B, ˜ M ˜,N ˜ , ϑ, A, ˜ respectively, let Bn and B ˜n , respectively, be ˜ B, ˜ M ˜,N ˜ , ϑ, A, B, M, N, ϑ and A, the upper bounds of the mean square errors of the hybrid procedures de˜ s , respectively, be the ﬁned by the recursion (5.85). Analogously, let Bns and B n solution of recursion (5.86) corresponding to the above tuples of parameters. If ϑ ≥ 1 and ϑ˜ ≥ 1, then by Theorem 5.5 we know that AM + BN Bn as n → ∞ → s Bn B(M + N ) and

˜ +B ˜N ˜ ˜n A˜M B → as n → ∞. s ˜ ˜ ˜ ˜ Bn B(M + N )

Moreover, by Corollary 5.2 we know that for ϑ > 1, ϑ˜ > 1 n · Bns Bns = → s ˜n ˜ns B n·B

B ϑ−1 ˜ B ˜ ϑ−1

=

B ϑ˜ − 1 as n → ∞. · ˜ ϑ−1 B

Consequently, we ﬁnd this Corollary 5.3. For any two hybrid stochastic approximation procedures char˜ resp., where ˜ B, ˜ M ˜,N ˜ , ϑ, acterized by the parameters A, B, M, N, ϑ and A, ˜ ϑ > 1 and ϑ > 1, it holds Bn = ˜n n→∞ B lim

Proof. For

AM +BN M +N ˜M ˜ +B ˜N ˜ A ˜ +N ˜ M

·

ϑ˜ − 1 . ϑ−1

Bn we may write ˜n B Bn B s 1 Bn = s · n· ˜ . ˜n ˜ns Bn n→∞ B Bn B ˜s lim

Bn

Hence, by the above considerations we get 1 Bn Bn Bs = lim s · lim n · ˜n ˜ns lim n→∞ B n→∞ Bn n→∞ B lim

˜n B ˜s n→∞ B n

=

˜ M ˜ +N ˜) AM + BN B ϑ˜ − 1 B( · · · ˜ ϑ − 1 A˜M ˜ +B ˜N ˜ B(M + N ) B

which proves our corollary.

˜ = 0, B ˜ = B and ϑ˜ = ϑ, i.e., if B ˜n = Bns , then from Corollary 5.2 Note. If M we gain back Theorem 5.5.

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

169

5.5.2 Lower Bounds for the Mean Square Error The results obtained above may be used now to ﬁnd also lower bounds for the mean square error of our hybrid procedure, provided that some further hypotheses are fulﬁlled. To this end assume that for an optimal solution x∗ of (5.1) (5.97) pD Xn − n A(Xn ) + ξn − pD x∗ − n A(x∗ ) = Xn − n A(Xn ) + ξn − x∗ − n A(x∗ ) for all n ∈ IN. This holds, e.g., if ∇F (x∗ ) = 0 and Xn − n A(Xn ) + ξn ∈ D for all n ∈ IN, see (5.67) and (5.69a). Now, if (5.97) holds, then we ﬁnd 2 (5.98) Xn+1 − x∗ 2 = (Xn − x∗ ) − n A(Xn ) − A(x∗ ) − n ξn T = Xn − x∗ 2 − 2n A(Xn ) − A(x∗ ) (Xn − x∗ ) 2 +2n A(Xn ) − A(x∗ ) − 2n ξnT Xn − x∗ −n A(Xn ) − A(x∗ ) + 2n ξn2 2 . Assume now that for all x ∈ D the inequalities 2 A(x) − A(x∗ ) ≥ δ1 + δ2 x − x∗ 2 , x = x∗ , T A(x) − A(x∗ ) (x − x∗ ) ≤ βx − x∗ 2

(5.99) (5.100)

hold, and for all n ∈ IN the inequality Eξn 2 ≥ η 2 > 0

(5.101)

is fulﬁlled, where δ1 ≥ 0, δ2 > 0, β > 0 and η > 0 are given constants. Note that these inequalities are opposite to (5.70), (5.71) and (5.80), respectively. Furthermore, (5.99), (5.100) and (5.101) together with (5.70), (5.71) and (5.80) imply that δ1 ≤ γ1 , δ2 ≤ γ2 , α ≤ β and η 2 ≤ σ 2 . If again (5.72) is valid, i.e., if E(ξn |Xn ) = 0 a.s. for every n ∈ IN2 , then, using equality (5.98), for bn = EXn − x∗ 2 we ﬁnd in the same way as in

170

5 RSM-Based Stochastic Gradient Procedures

the proof of Lemma 5.1 the following recursion being opposite to (5.74a), (5.74b) and (5.82) (5.102a) bn+1 ≥ (1 − 2βn + δ2 2n )bn + 2n δ1 + Eξn 2 ≥ (1 − 2βn )bn + β n 2n ,

where βn =

n = 1, 2, . . . ,

δ1 , if n ∈ IN1 δ1 + η 2 , if n ∈ IN

(5.102b)

In contrast to the “worst case” in (5.82), which is represented by the recursion (5.84a), (5.84b), here we consider the “best case” in (5.102a), i.e., the recursion B n+1 = (1 − 2βn )B n 2n + β n 2n , n = 1, 2, . . . .

(5.103)

Assuming again that lim n = 0, there is an integer n0 such that (5.81) n→∞ holds, i.e., 0 < 1 − 2αn + γ2 2n < 1 − an for all n ≥ n0 , where 0 < a < 2α, as well as 1 − 2βn > 0 for all n ≥ n0 . Let Bn0 = bn0 and B n0 = bn0 . Then, according to Sect. 5.5, inequality (5.84a), we know that bn ≤ Bn for all n ≥ n0 , and by a similar simple induction argument we ﬁnd B n ≤ bn ≤ Bn for all n ≥ n0 .

(5.104a)

Since the pure stochastic procedure is characterized simply by IN1 = ∅, we also have (5.104b) B sn ≤ bsn ≤ Bns , for all n ≥ n0 , where bsn is the mean square error of the pure stochastic algorithm. Furthermore, Bns and B sn are the solutions of (5.84a) and (5.103), resp., with β¯n = γ1 + σ 2 and β n = δ1 + η 2 for all n ∈ IN, where, of course, Bns 0 = bsn0 and B sn0 = bsn0 . Thus, the inequalities (5.104a), (5.104b) yield the estimates bn Bn Bn ≤ s ≤ s for all n ≥ n0 . s Bn bn Bn Let now the step size n be deﬁned by n =

c , n+q

n = 1, 2, . . .

where c > 0 and q ∈ IN ∪ {0} are ﬁxed numbers.

(5.105)

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

171

The (hybrid) stochastic algorithms having the lower and upper mean square error estimates (B n ) and (Bns ) are characterized – see Sect. 5.5.1 – by the parameters A = c2 δ1 ,

B = c2 (δ1 + η 2 ),

M, N, ϑ = 2βc,

and ˜ = c2 (γ1 + σ 2 ), B

˜ = 0, ϑ˜ = ac, respectively. M

Hence, if c > 0 is selected such that ac > 1 and 2βc > 1, then Corollary 5.3 yields B lim ns = n→∞ Bn

c2 δ1 M +c2 (δ1 +η 2 )N M +N c2 (γ1 + σ 2 )

·

ac − 1 . 2βc − 1

(5.106a)

Furthermore, the sequences (Bn ), (Bns ), resp., are estimates of the mean square error of the (hybrid) stochastic algorithms represented by the parameters A = c2 γ1 , B = c2 (γ1 + σ 2 ), M, N, ϑ = ac and ˜ = 0, ϑ˜ = 2βc, respectively. ˜ = c2 (δ1 + η 2 ), M B Thus, if ac > 1 and 2βc > 1, then Corollary 5.3 yields Bn lim s = n→∞ B n

c2 γ1 M +c2 (γ1 +σ 2 )N M +N c2 (δ1 + η 2 )

·

2βc − 1 . ac − 1

(5.106b)

Consequently, we have now this next result: Theorem 5.6. Suppose that the minimization problem (5.1) fulﬁlls the inequalities (5.70), (5.71) and (5.99), (5.100). Furthermore, assume that the c with q ∈ noise conditions (5.72), (5.80) and (5.101) hold. If n = n+q IN ∪ {0} and c > 0 such that ac > 1, where 0 < a < 2α, and if (5.97) is fulﬁlled, then bn bn (5.107) Q1 ≤ lim inf s ≤ lim sup s ≤ Q2 , n→∞ bn n→∞ bn where Q1 =

δ1 M + (δ1 + η 2 )N ac − 1 · , (M + N )(γ1 + σ 2 ) 2βc − 1

(5.108a)

Q2 =

γ1 M + (γ1 + σ 2 )N 2βc − 1 · . (M + N )(δ1 + η 2 ) ac − 1

(5.108b)

Proof. Since 0 < a < 2α and α ≤ β, we have ac < 2βc for every c > 0 and therefore 2βc − 1 > ac − 1 > 0. The assertion follows now from (5.106a) and (5.106b).

172

5 RSM-Based Stochastic Gradient Procedures

Discussion of Theorem 5.6 (a) Obviously, (5.107) yields Q1 ≤ Q2 , and, using formulas (5.108a), (5.108b), we ﬁnd Q1 < Q2 . Moreover, by means of Theorem 5.5 and because of δ1 ≤γ1 , α≤β, η 2 ≤σ 2 as also 1 < ac < 2αc ≤ 2βc, we obtain Q1 ≤

ac − 1 γ1 M + (γ1 + σ 2 )N ac − 1 Bn Bn · = · lim s < lim s 2 n→∞ n→∞ (M + N )(γ1 + σ ) 2βc − 1 2βc − 1 Bn Bn (5.109)

as also Q2 ≥

2βc − 1 δ1 M + (δ1 + η 2 )N 2βc − 1 B B · = · lim ns > lim ns . 2 n→∞ n→∞ (M + N )(δ1 + η ) ac − 1 ac − 1 Bn Bn (5.110)

(b) According to (5.107) we have Q1 bsn ≤ bn ≤ Q2 bsn as n → ∞. This means that asymptotically the hybrid stochastic approximation procedure is 1 1 times faster and at least times faster at most Q1 Q2 than the pure stochastic procedure. 1 > 1 for the best case. Moreover, Q2 < 1 According to (5.109) we have Q1 holds if and only if ac − 1 γ1 M + (γ1 + σ 2 )N < f := (< 1). (M + N )(δ1 + η 2 ) 2βc − 1

(5.111)

Since always γ1 + σ 2 − f (δ1 + η 2 ) > γ1 + σ 2 − δ1 − η 2 ≥ 0, condition (5.111) is equivalent to f (δ1 + η 2 ) − γ1 N < M γ1 + σ 2 − f (δ1 + η 2 )

(5.112a)

which can be fulﬁlled if and only if δ1 + η 2 >

γ1 2βc − 1 = γ1 f ac − 1

(5.112b)

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

173

1 , where a0 is a ﬁxed number such that a0 0 < a0 < 2α, then ac ≥ a0 c > 1 for all a with a0 ≤ a < 2α. Hence, if 1 c > , then Theorem 5.6 holds for all numbers a with a0 ≤ a < 2α. a0 1 , where 0 < a0 < 2α, then the relations (5.107) Consequently, if c > a0 with (5.108a) and (5.108b), (5.109) and (5.110) as also (5.111), (5.112a) and (5.112b) hold also if a is replaced by 2α.

(c) If c is selected according to c >

5.5.3 Decreasing Rate of Stochastic Steps A further improvement of the speed of convergence is obtained if we have a decreasing rate of stochastic steps taken in (5.69a)–(5.69c). In order to handle this case, we consider the sequence cn = nλ EXn − x∗ 2 ,

n = 1, 2, . . .

¯ where λ ¯ is a given ﬁxed positive number. Then, for some λ with 0 ≤ λ < λ, under the same conditions and using the same methods as in Lemma 5.1, corresponding to (5.74a), (5.74b) we ﬁnd

λ 1 (1 − 2αn + γ2 2n )cn + (n + 1)λ βn 2n , n = 1, 2, . . . , cn+1 ≤ 1 + n (5.113a) where again (5.113b) βn = γ1 + Eξn 2 and ξn is deﬁned by (5.69b). We claim that for n ≥ n1 , n1 ∈ IN, suﬃciently large,

λ 1 1+ (1 − 2αn + γ2 2n ) ≤ 1, n

(5.114)

provided that the step size n is suitably chosen. Indeed, if n → 0 as n → ∞, then there is an integer n0 such that (see (5.81)) 0 < 1 − 2αn + γ2 2n < 1 − an , n ≥ n0 for any number a such that 0 < a < 2α. According to (5.114) and (5.81) we have to choose now n such that

λ 1 (1 − an ) ≤ 1, n ≥ n1 (5.115) 1+ n with an integer n1 ≥ n0 . Write

λ 1 pn (λ) 1+ =1+ n n

174

5 RSM-Based Stochastic Gradient Procedures

with pn (λ) = n

1 1+ n

λ

− 1 . Hence, condition (5.115) has the form

1 − an ≤

n , n + pn (λ)

n ≥ n1 .

¯ and therefore pn (λ) ≤ pn (λ) ¯ for all n ∈ IN, the latter is true if Since 0 ≤ λ < λ 1 − an ≤

n ¯ , n + pn (λ)

n ≥ n1 .

Hence, for the step size n we ﬁnd the condition n ≥

¯ ¯ 1 1 pn (λ) 1 pn (λ) = ¯ ¯ p n a n + pn (λ) n a 1 + (λ)

(5.116)

n

for all n suﬃciently large. Writing λ¯ 1 + n1 − 1 ¯ pn (λ) = , 1 n

we see that ¯ = lim pn (λ)

n→∞

Therefore

¯ pn (λ)

lim

n→∞

d ¯ ¯ (1 + x)λ |x=0 = λ. dx

1+

¯ pn (λ) n

¯ = λ,

which implies that ϑ1 ≤

¯ pn (λ) 1+

≤ ϑ2 ,

¯ pn (λ) n

n = 1, 2, . . . ,

(5.117)

¯ ≤ ϑ2 . If λ ¯ ∈ IN, then an easy with some constants ϑ1 , ϑ2 such that 0 < ϑ1 < λ consideration shows that ¯ pn (λ) 1+

¯ pn (λ) n

,

n = 1, 2, . . . ,

¯ ∈ IN the bounds in a monotonously increasing sequence. Hence, for every λ ¯ λ 2 −1 ¯ Consequently, if and ϑ2 = λ. ϑ1 , ϑ2 can be selected according to ϑ1 = 2λ¯ n is deﬁned by ¯ 1 1 pn (λ) n = (5.118a) ¯ n a 1 + pn (λ) n

or n is given by

5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures

n =

1 c c = n+q n1+

q n

with q ∈ IN and c >

ϑ2 , a

175

(5.118b)

then there is an integer n1 ≥ n0 such that (5.116) holds for n ≥ n1 . Let ϑ = ϑ2 , ϑ = ac, respectively. Then in both cases we have n ≤ and from (5.113a), (5.113b)–(5.115) follows

ϑ 1 · , a n

2 ϑ 1 a n2 2 ϑ ¯ ≤ cn + 2λ βn nλ−2 a

cn+1 ≤ cn + (n + 1)λ βn 2n ≤ cn + (n + 1)λ βn ≤ cn +

n+1 n

λ¯ βn

2 ϑ nλ−2 a

for n ≥ n1 and therefore cN +1 − cn1 =

N n=n1

2 N ϑ (cn+1 − cn ) ≤ 2 βn nλ−2 , N ≥ n1 . a n=n ¯ λ

(5.119)

1

Suppose now that also (5.80) holds, hence Eξn 2 ≤ σ 2 for all n ∈ IN2 . Therefore βn ≤ β¯n , where (see (5.83)) β¯n = γ1 for n ∈ IN1 and β¯n = γ1 + σ 2 for n ∈ IN2 . Inequality (5.119) yields now ⎛ ⎞ 2 N N 2 ϑ ⎜ γ1 γ1 + σ ⎟ ¯ cN +1 − cn1 ≤ 2λ + ⎝ ⎠. 2−λ a n n2−λ n=n1 n=n1 n∈IN1

n∈IN2

¯ ≥ 2, we ﬁnd this result: Choosing λ Theorem 5.7. Let the assumptions of Lemma 5.1 be fulﬁlled, and suppose that (5.80) holds. Moreover, let the step size n be deﬁned by (5.118a) or (5.118b). If γ1 = 0 and IN1 = IN\IN2 with IN2 = {nk = k p : k = 1, 2, . . .} for some ﬁxed integer p = 1, 2, . . ., then EXn − x∗ 2 = 0(n−λ ) as n → ∞ 1 for all λ such 0 ≤ λ < 2 − . p Proof. Under the given hypotheses we have 2 ϑ 1 ¯ λ cN +1 − cn1 ≤ 2 σ2 p a (k )2−λ n1 ≤kp ≤N 2 ∞ ϑ 1 ¯ ≤ 2λ σ2 , N ≥ n1 , p(2−λ) a k k=1

(5.120)

176

5 RSM-Based Stochastic Gradient Procedures

¯ and a ﬁxed λ ¯ ≥ 2. Since 2− where cn = nλ EXn −x∗ 2 with 0 ≤ λ < λ

1 ¯ < λ, p

1 is equivalent to p(2 − λ) > 1, the above series converges. p Hence, the sequence (cn ) is bounded which yields the assertion of our theorem.

and 0 ≤ λ < 2 −

Discussion of Theorem 5.7 The condition γ1 = 0 means that the operator A(x) deﬁned by (5.67) must satisfy (see (5.70)) the Lipschitz condition √ A(x) − A(x∗ ) ≤ γ2 x − x∗ . Deﬁnition (5.120) of IN2 obviously means that we have a decreasing rate n p −1 of stochastic steps taken in (5.69a), (5.69b). According to Lemma 5.2, for a pure stochastic approximation procedure we know that EXn − x∗ 2 = 0(n−κ ) with 0 < κ ≤ 1. Hence, if stochastic and deterministic steps are taken in (5.69a), (5.69b) as described by (5.120), then the speed of convergence may be increased from O(n−κ ) with 0 < κ ≤ 1 1 in the pure stochastic case to the order of O(n−λ ) with 1 < λ < 2 − in the p hybrid stochastic case. If p = 1, p → ∞, respectively, then we obtain the pure stochastic, pure deterministic case, respectively. Denoting by sn the number of stochastic steps taken in (5.69a), (5.69b) up to the nth stage, in the case (5.120) we have 1

sn ≈ n1/p sn and therefore 2 ≈ n1/p−2 . Hence, the estimate of the convergence rate given n by Theorem 5.7 can also be represented in the form r s n n = O EXn − x∗ 2 ≈ O n2 n if rn = n1/p−1 is the rate of stochastic steps taken in (5.69a), (5.69b), cf. the example in [80].

6 Stochastic Approximation Methods with Changing Error Variances

6.1 Introduction As already discussed in the previous Chap. 5, hybrid stochastic gradient procedures are very important tools for the iterative solution of stochastic optimization problems as discussed in Chaps. 1–4: min F (x) s.t. x ∈ D

with the expectation value function F (x) = Ef a(ω), x having gradient G(x) = ∇F (x). Moreover, quite similar stochastic approximation procedures may be used for the solution of systems of nonlinear equations G(x) = 0 involving a vectorial so-called “regression function” G = G(x) on IRν . These algorithms generating a sequence (Xn ) of estimators Xn of a solution x∗ of one of the above mentioned related problems are based on estimators Yn = G(Xn ) + Zn , n = 0, 1, 2, . . . of the gradient G of the objective function F or the regression function G at an iteration point Xn . The resulting iterative procedures of the type X(n+1) := pD (Xn − Rn Yn ), n = 0, 1, 2, . . . , cf. (5.5), are well studied in the literature. Here, pD denotes the projection of IRν onto the feasible domain D known to contain an (optimal) solution, and Rn , n = 0, 1, 2, . . . is a sequence of scalar or matrix valued step sizes. In the extensive literature on stochastic approximation procedures, cf. [39–41, 60, 61, 125, 164], various suﬃcient conditions can be found for the underlying optimization problem, for the regression function, resp., the sequence of estimation errors (Zn ) and the sequence of step sizes Rn , n = 0, 1, 2, . . ., such that

178

6 Stochastic Approximation Methods with Changing Error Variances

Xn → x∗ , n → ∞, in some probabilistic sense (in the (quadratic) mean, almost sure (a.s.), in distribution). Furthermore, numerous results concerning the asymptotic distribution of the random algorithm (Xn ) and the adaptive selection of the step sizes for increasing the convergence behavior of (Xn ) are available. As shown in Chap. 5, the convergence behavior of stochastic gradient procedures may be improved considerably by using More exact gradient estimators Yn at certain iteration stages n ∈ N1 and/or – Deterministic (feasible) descent directions hn = h(Xn ) available at certain iteration points Xn

–

While the convergence of hybrid stochastic gradient procedures was examined in Chap. 5 in terms of the sequence of mean square errors bn = EXn − x∗ 2 , n = 0, 1, 2, . . . , in the present Chap. 6, for stochastic approximation algorithms with changing error variances further probabilistic convergence properties are studied in detail: Convergence almost sure (a.s.), in the mean, in distribution. Moreover, the asymptotic distribution is determined, and the adaptive selection of scalar and matrix valued step sizes is considered for the case of changing error variances. The results presented in this chapter are taken from the doctoral thesis of E. Pl¨ ochinger [115]. Remark 6.1. Based on the RSM-gradient estimators considered in Chap. 5, implementations of the present stochastic approximation procedure, especially the Hessian approximation given in Sect. 6.6, are developed in the thesis of J. Reinhart [121], see also [122], for stochastic structural optimization problems.

6.2 Solution of Optimality Conditions For a given set ∅ = D ⊂ IRν and a measurable function G : D −→ IRν , consider the following problem P: Find a vector x∗ ∈ D such that (6.1) x∗ = pD x∗ − r · G(x∗ ) for all r > 0 . Here, D is a convex and compact set, and for a vector y ∈ IRν , let pD (y) denote the projection of y onto D, determined uniquely by the relation pD (y) ∈ D and y − pD (y) = min y − x. x∈D

Problems of this type have to be solved very often. E.g., in the special case that G = ∇F , where F : D −→ IR is a convex and continuously diﬀerentiable function, according to [16] condition (6.1) is equivalent to

6.3 General Assumptions and Notations

179

F (x∗ ) = min F (x). x∈D

˚ (:= interior of D), condition (6.1) is equivFurthermore, for a vector x∗ ∈ D alent with the system of (nonlinear) equations G(x∗ ) = 0. In many practical situations, the function G(x) can not be determined directly, but we may assume that for every x ∈ D a stochastic approximate (estimate) Y of G(x) is available. Hence, for the numerical solution of problem (P ) we consider here the following stochastic approximation method: Select a vector X1 ∈ D, and for n = 1, 2, . . . determine recursively Xn+1 := pD (Xn − Rn · Yn ) with Rn := ρn · Mn and Yn := G(Xn ) + Zn .

(6.2a) (6.2b)

The matrix step size Rn = ρn Mn , where ρn is a positive number and Mn is a real ν ×ν matrix, can be chosen by the user in order to control the algorithm (6.2a), (6.2b). Yn denotes the search direction, and Zn is the estimation error occurring at the computation of G(Xn ) in step n. Furthermore, ρn , Mn and Zn are random variables deﬁned on an appropriate probability space (Ω, A, P).

6.3 General Assumptions and Notations In order to derive diﬀerent convergence properties of the sequence (Xn ) deﬁned by (6.2a), (6.2b), in this part we suppose that the following assumptions, denoted by (U), are fulﬁlled. Note that these conditions are the standard assumptions which can be found in similar way in all relevant work on stochastic approximation: (a) D ⊂ IRν is a nonempty, convex and compact stet with diameter δ0 . (b) Let G : D0 −→ IRν denote a vector function, where D0 ⊃ {x + e : x ∈ D and e < ε0 } with a positive number ε0 . (c) There exist positive numbers L0 and µ0 , as well as functions H : D −→ IRν × IRν and δ : D × IRν −→ IRν such that G(x + e) = G(x) + H(x) · e + δ(x; e), δ(x; e) ≤ L0 · e1+µ0 , H(x) is continuous in x for all x ∈ D and e < ε0 . (d) There is a unique x∗ ∈ D such that x∗ = pD x∗ − r · G(x∗ )

for all r > 0.

(U1a) (U1b) (U1c)

(U2)

180

6 Stochastic Approximation Methods with Changing Error Variances

(e) For x∗ ∈ ∂D (:= boundary of D) a scalar step size is selected, i.e., Rn = ρn · Mn = rn · I

with rn ∈ IR+ .

(U3)

(f) Concerning the selection of the matrices Mn in the step sizes Rn = ρn ·Mn , suppose that (U4a) sup Mn < ∞ a.s. (almost sure), n

and there is a partition 0n + ∆Mn , Mn = M

(U4b)

lim ∆Mn = 0 a.s.,

(U4c)

0n H(x)u ≥ au2 uT M

(U4d)

where n→∞

for each n ∈ IN, x ∈ D and u ∈ IRν . Here, a is a positive constant. (g) There exists a probability space (Ω, A, P) and σ-algebras A1 ⊂ . . . ⊂ 0n and Zn−1 for n = 1, 2, . . . An ⊂ A such that the variables Xn , ρn , Mn , M are An -measurable. (h) For the estimation error Zn in (6.2a), (6.2b) we suppose, cf. (5.35a), (5.35b), that √ nEn Zn −→ 0 a.s., (U5a) 2 2 2 ∗ 2 (U5b) En Zn − En Zn ≤ σ0,n + σ1 Xn − x for all n = 1, 2, . . . . Here En (·) := E(· | An ) denotes the conditional expectation with respect to An , σ1 is a constant and σ0 ,n denotes a An measurable random variable with 2 ≤ σ02 < ∞ sup E σ0,n

(U5c)

n

and a further constant σ0 . ˙ (κ) such that for k = (i) There exists a disjoint partition IN = IN(1) ∪˙ . . . ∪IN 1, . . . , κ the limits | {1, . . . , n} ∩ IN(k) | >0 n→∞ n

qk := lim

(U6)

exist and are positive, see Sect. 5.5.1. In the later Sects. 6.5–6.8 additional assumptions are needed, generating further convergence properties or implying some of the above assumptions. Assumptions of that type are marked by one of the letters V,W,X and Y, and hold always for the current section.

6.3 General Assumptions and Notations

181

6.3.1 Interpretation of the Assumptions According to condition (c), the mapping G : D −→ IRν is continuously diﬀerentiable and has the Jacobian H(x). Because of (a) and (U1a)–(U1c) there is a number β < ∞ with (6.3) H(x) ≤ β for all x ∈ D. Using the mean value theorem, we have then G(x) − G(y) ≤ β · x − y

(6.4)

for all x, y ∈ D. Hence, G is Lipschitz continuous with the Lipschitz constant β. Since the projection operator pD (y) and the map G(x) are continuous, because of a) there is a vector x∗ ∈ D fulﬁlling (U2). Conditions (U2) and (U3) guarantee, that for each n we have (6.5) x∗ = pD x∗ − Rn · G(x∗ ) . Consequently, in the deterministic case, i.e., if Zn ≡ 0, then algorithm (6.2a), (6.2b) stays at x∗ provided that Xn = x∗ for a certain index n. 0n are bounded, and According to condition (f), the matrices Mn and M (U4d) and the mean value theorem imply that 1 2 0n G(x) − G(x∗ ) ≥ ax − x∗ 2 x − x∗ , M (6.6) for all n and x ∈ D, where < ·, · > denotes the scalar product in IRν . Especially, for each n ∈ IN we have 1 2 0n G(Xn ) − G(x∗ ) ≥ Xn − x∗ , M an Xn − x∗ 2 a.s. (6.7) an ≥ a. Furthermore, (U4d) with certain An -measurable random variables yields: 0n , H(x) are regular for each n ∈ IN and x ∈ D. (6.8) M E.g., condition (U4d) holds in case of scalar step sizes, provided that 0n = m - n · I with m - n ≥ d, M T H(x) + H(x) ≥ 2αI for all n ∈ IN and x ∈ D, where d and α are positive numbers. These conditions hold in Sects. 6.7.2 and 6.8. The second of the above conditions implies also (U2), and means in the gradient case G(x) = ∇F (x) that the function F (x) is strong convex on D.

182

6 Stochastic Approximation Methods with Changing Error Variances

According to (g), the random variables ρn , Mn and Zn may depend on the history X1 , . . . , Xn , Z1 , . . . , Zn−1 , ρ1 , . . . , ρn , M1 , . . . , Mn of the process (6.2a), (6.2b). Condition (U5a) concerning the estimation error Zn holds, e.g., if the search direction Yn is an unbiased estimation of G(Xn ), hence, if En Yn = G(Xn ). From (U5a)–(U5c) and (a) we obtain that the estimation error has a bounded variance. Since for diﬀerent indices n the estimation error Zn may vary considerably, according to (i) the set of integers IN will be partitioned such that the variances of Zn are equal approximatively for large indices n lying in the same part IN(k) . Moreover, the numbers q (1) , . . . , q (κ) represent the portion of the sets IN(1) , . . . , IN(κ) with respect to IN. 6.3.2 Notations and Abbreviations in this Chapter The following notations and abbreviations are used in the next sections: an a(k) := lim inf -

(6.9)

n∈IN(k)

a(1) + . . . + qκ a(κ) ≥ a a := q1 ∆n := Xn − x∗ = actual argument error ∆Zn := Zn − En Zn 2 σk2 := lim sup Eσ0,n n∈IN(k)

τn+1 := τn (1 + γn ) weight factor, where τ1 := 1 and 0 ≤ γn is a nonnegative number ∆n := τn (Xn − x∗ ) weighted argument error G∗ := G(x∗ ), Gn := G(Xn ),

H ∗ := H(x∗ ) Hn := H(Xn )

λmin (A) := minimum eigenvalue of a symmetric matrix A λmax (A) := maximum eigenvalue of a symmetric matrix A AT := transpose of A / A := λmax (AT A) = norm of the matrix A A ≤ B : ⇐⇒ λmin (B − A) ≥ 0 1 α∗ := λmin (H ∗ + H ∗T ) 2 IMm,n := {m + 1, . . . , n} ∩ IM = (m, n)-section of a set IM ⊂ IN a+ := max (a, 0),

a− := a+ − a

6.4 Preliminary Results

183

¯0 := {ω ∈ Ω : ω ∈ / Ω0 } = complement of the set Ω0 ⊂ Ω Ω IΩ0 := characteristic function of Ω0 ⊂ Ω EΩ0 T := E(T IΩ0 ) = expectation of the random variable T restricted to the set Ω0 limextr := either

lim sup

or

lim inf

6.4 Preliminary Results In order to shorten the proofs of the results given in the subsequent sections, here some preliminary results are given. 6.4.1 Estimation of the Quadratic Error For the proof of the convergence of the quadratic argument error Xn − x∗ 2 the following auxiliary statements are needed. Lemma 6.1. For all n ∈ IN we have 2 En ∆n+1 2 ≤ ∆n − Rn (Gn − G∗ ) + Rn 2 En Zn 2 + En Zn − En Zn 2 + 2Rn En Zn ∆n − Rn (Gn − G∗ ).

(6.10)

Proof. Because of (6.5), for arbitrary n we get 2 ∆n+1 2 = pD (Xn − Rn Yn ) − pD (x∗ − Rn G∗ ) 2 ≤ Xn − Rn (Gn + Zn ) − (x∗ − Rn G∗ ) 3 2 4 = ∆n − Rn (Gn − G∗ ) − Rn En Zn − Rn (Zn − En Zn ) 2 2 ≤ ∆n − Rn (Gn − G∗ ) + Rn En Zn + Rn (Zn − En Zn ) +2∆n − Rn (Gn − G∗ ) − Rn En Zn , Rn (Zn − En Zn ). Inequality (6.10) is obtained then by taking expectations.

The ﬁrst term on the right hand side of inequality (6.10) can be estimated as follows: Lemma 6.2. For each n we ﬁnd ∆n − Rn (Gn − G∗ )2 ≤ 1 − 2ρn an − β∆Mn + ρ2n Mn 2 β 2 ∆n 2 . (6.11)

184

6 Stochastic Approximation Methods with Changing Error Variances

Proof. According to (U4b) we have ∆n − ρn Mn (Gn − G∗ )2 ≤ ∆n 2 + ρ2n Mn 2 Gn − G∗ 2 5 6 0n (Gn − G∗ ) + 2ρn ∆n ∆Mn Gn − G∗ . −2ρn ∆n , M Inequality (6.11) follows now by applying (6.4) and (6.7).

-n = τn (Xn − x∗ )) we have now an For the sequence of weighted errors (∆ upper bound which is used several times in the following: Lemma 6.3. For arbitrary ε > 0 and n ∈ IN we have (2) -n+1 2 ≤ (1 − ρn tn + t(1) - 2 En ∆ n )∆n + tn ,

where

(6.12)

gn γn tn := (1 + γn )2 1 + un − (2 + γn ), ε ρn g 2 n 2 + ρn Mn σ12 , t(1) n := (1 + γn ) ε 2 2 (2) 2 , tn := (1 + γn ) gn (ε + gn ) + τn ρn Mn σ0,n

and the following deﬁnitions are used: un := 2 an − β∆Mn − ρn Mn 2 β 2 , gn := τn ρn Mn En Zn . Proof. Because of the inequality 2 τ 2 ∆n − Rn (Gn − G∗ ) 2τn ∆n − Rn (Gn − G∗ ) ≤ ε + n , ε 2 , inequality (6.10) yields after multiplication with τn+1

-n+1 2 En ∆ 7 τn Rn En Zn ∆n − Rn (Gn − G∗ )2 τn2 ≤ (1 + γn )2 1 + ε 8 2 + τn Rn En Zn 2 + En Zn − En Zn 2 + ετn Rn En Zn . Taking into account (6.11) and (U5b), we ﬁnd 7 -n+1 2 ≤ (1 + γn )2 1 + gn (1 − ρn un )∆ -n 2 En ∆ ε 8 2

2 +gn (ε + gn ) + (τn Rn ) (σ0,n + σ12 ∆n 2 ) 7 gn gn -n 2 un + + Rn 2 σ12 ∆ = (1 + γn )2 1 − ρn 1 + ε ε 2 2 8 . +gn (ε + gn ) + τn Rn σ0,n

This proves now (6.12).

6.4 Preliminary Results

185

6.4.2 Consideration of the Weighted Error Sequence Suppose that ρn −→ 1 a.s., sn Mn − Dn −→ 0 a.s.

(6.13) (6.14)

for given deterministic numbers sn and matrices Dn . The weighted error -n := τn (Xn − x∗ ) can be represented then as follows: ∆ Lemma 6.4. For each integer n: -n + τn sn Dn (En Zn − Zn ) -n+1 = (I − sn Bn )∆ ∆ -n + τn sn ∆Dn (En Zn − Zn ) + sn ∆Bn ∆ + τn+1 −Rn (En Zn + G∗ ) + ∆Pn , where the variables in Lemma 6.4 are deﬁned as follows: γn I, Bn := Dn H ∗ − sn 1 - n − γn I , ∆Bn := Bn − (1 + γn )Rn H sn 1 (1 + γn )Rn − Dn , ∆Dn := sn ∆Pn := pD (Xn − Rn Yn ) − (Xn − Rn Yn ), 1 - n := H H x∗ + t(Xn − x∗ ) dt.

(6.15)

(6.16) (6.17) (6.18) (6.19) (6.20)

0

Proof. Because of assumption (U1a)–(U1c), and using the Taylor formula we get - n ∆n + Zn . Yn = Gn + Zn = G∗ + H Hence, (6.2a), (6.2b) yields ∆n+1 = ∆n − Rn Yn + ∆Pn - n )∆n + Rn (En Zn − Zn ) − Rn (En Zn + G∗ ) + ∆Pn . = (I − Rn H

Multiplying then with τn+1 , (6.15) follows, provided that the following relations are taken into account: - n ) = I − (1 + γn )Rn H - n − γn I = I − sn (Bn − ∆Bn ), (1 + γn )(I − Rn H (1 + γn )Rn = sn (Dn + ∆Dn ). For the further consideration of (6.15) some additional auxiliary lemmas are needed.

186

6 Stochastic Approximation Methods with Changing Error Variances

Lemma 6.5. For each n ∈ IN we have (1) (2) -n = ∆(0) ∆ n + ∆n + ∆n

(6.21)

(i)

with An -measurable random variables ∆n , i = 0, 1, 2, where (0)

∆n+1 = (I − sn Bn )∆(0) n + (τn sn )Dn (En Zn − Zn ),

(6.22)

(1) ∆n+1 ≤ I − sn Bn ∆(1) n + sn ∆Bn ∆n + τn+1 ρn Mn En Zn + G∗ + ∆Pn ,

(6.23)

(2)

2 En ∆n+1 2 ≤ I − sn Bn 2 ∆(2) n 2 +(τn sn )2 ∆Dn 2 σ0,n + σ12 ∆n 2 .

(6.24) (i)

Proof. For i = 0, 1, 2, select arbitrary A1 -measurable random variables ∆1 such that -1 = ∆(0) + ∆(1) + ∆(2) . ∆ 1

For n ≥ 1, deﬁne

(0) ∆n+1

1

1

then by (6.22) and put

(1) ∗ ∆n+1 := (I − sn Bn )∆(1) n + sn ∆Bn ∆n + τn+1 −Rn (En Zn + G ) + ∆Pn , (2) ∆n+1

:= (I −

sn Bn )∆(2) n

+ τn sn ∆Dn (En Zn − Zn ).

(i) (ii)

Relation (6.21) holds then because of Lemma 6.4, and (6.23) follows directly from (i) by taking the norm. Moreover, (ii) yields (2)

2 2 2 2 ∆n+1 2 ≤ I − sn Bn 2 ∆(2) n + (τn sn ) ∆Dn En Zn − Zn 5 6 + 2τn sn (I − sn Bn )∆(2) n , ∆Dn (En Zn − Zn ) . (2)

The random variables ∆n and ∆Dn are An -measurable. Thus, taking expectations in the above inequality, by means of (U5b) we get then inequality (6.24). Suppose now that the matrices Dn from (6.14) fulﬁll the following relations: Dn H ∗ + (Dn H ∗ )T ≥ 2an I, Dn H ∗ ≤ bn ,

(6.25) (6.26)

with certain numbers an , bn ∈ IR. The matrices Bn from (6.16) fulﬁll then the following inequalities:

6.4 Preliminary Results

187

Lemma 6.6. For each n ∈ IN:

γn T Bn + Bn ≥ 2 an − I =: 2 an I, sn 2 γn γn − 2 an + b2n =: b2n , Bn 2 ≤ sn sn

1 2 2 an − sn bn =: 1 − 2sn un , I − sn Bn ≤ 1 − 2sn 2 I − sn Bn ≤ 1 − sn un . Proof. The assertions follow by simple calculations from the relations (6.16), (6.25) and (6.26), where: √

1 − 2t ≤ 1 − t for

t≤

1 . 2

Some useful properties of the random variables ∆Bn , ∆Dn and ∆Pn arising in (6.17), (6.18) and (6.19) are contained in the next lemma: Lemma 6.7. Assuming that ˚ a.s., γn −→ 0 and Xn −→ x∗ ∈ D we have (a) ∆Bn , ∆Dn −→ 0 a.s. (b) G∗ = 0 (c) For each ω ∈ Ω, with exception of a null-set, there is an index n0 (ω) such that ∆Pn (ω) = 0

for every

n ≥ n0 (ω)

Proof. Because of Rn = ρn Mn , (6.13), (6.14), (6.18) and (U4a) we have ∆Dn = (1 + γn )

ρn Mn − Dn −→ 0 sn

a.s.

Furthermore, Xn → x∗ a.s. and (U1a)–(U1c) yield - n := H

1

H x∗ + t(Xn − X ∗ ) dt −→ H ∗

a.s.

0

Hence, because of (6.13), (6.14), (6.16) and (6.17) we ﬁnd ∆Bn = Dn H ∗ − (1 + γn ) ˚ with (U2) we get G∗ = 0. Since x∗ ∈ D,

ρn - n −→ 0 a.s. Mn H sn

188

6 Stochastic Approximation Methods with Changing Error Variances

˚ Consider now an element ω ∈ Ω with Xn (ω) −→ x∗ . Because of x∗ ∈ D ˚ for all n ≥ n0 (ω). Hence, for there is a number n0 (ω) such that Xn+1 (ω) ∈ D ˚ and therefore n ≥ n0 (ω) we ﬁnd pD (Xn (ω) − Rn (ω)Yn (ω)) =: Xn+1 (ω) ∈ D, pD (Xn (ω) − Rn (ω)Yn (ω)) = Xn (ω) − Rn (ω)Yn (ω), hence, ∆Pn (ω) = 0. 6.4.3 Further Preliminary Results Suﬃcient conditions for Xn+1 − Xn → 0 a.s. are given in the following: Lemma 6.8. (a) Xn+1 − Xn ≤ ρn Mn Yn for n = 1, 2, .... (b) Yn 2 ≤ 4 G∗ 2 + Gn − G∗ 2 + Zn − En Zn 2 + En Zn 2 for n = 1, 2, . . . . ∞ ' s2n (c) For each deterministic sequence of positive numbers (sn ) such that < ∞ we have

∞ ' n=1

n=1

s2n Yn 2 < ∞ a.s.

(d) Let (sn ) denote a sequence according to (c), and assume that (ρn ) fulﬁlls ∞ ' lim sup ρsnn ≤ 1 a.s. Then Xn+1 − Xn 2 < ∞ a.s., and (Xn+1 − Xn ) n→∞

converges to zero a.s.

n=1

Proof. For each n ∈ IN, according to (6.2a), (6.2b) we ﬁnd Xn+1 − Xn = pD (Xn − Rn Yn ) − pD (Xn ) ≤ Rn Yn = ρn Mn Yn , 13 ∗ 42 2 ∗ Yn = 16 G + (Gn − G ) + (Zn − En Zn ) + En Zn 4 ≤ 4 G∗ 2 + Gn − G∗ 2 + Zn − En Zn 2 + En Zn 2 .

(i)

(ii)

Due to assumption (a) in Sect. 6.3, inequality (6.4) and (U5a)–(U5c), for each n ∈ IN we ﬁnd Gn − G∗ 2 ≤ β 2 Xn − x∗ 2 ≤ β 2 δ02 , En Zn 2 −→ 0

a.s., EZn − En Zn 2 = E En Zn − En Zn 2 2 ≤ E σ0,n + σ12 Xn − x∗ 2 ≤ σ02 + σ12 δ02 .

6.4 Preliminary Results

Hence, for a sequence (sn ) according (c) we get s2n G∗ 2 + Gn − G∗ 2 + En Zn 2 < ∞ a.s.

189

(iii)

n

and

' n

s2n EZn − En Zn 2 < ∞. This yields then

s2n Zn − En Zn 2 < ∞ a.s.

(iv)

n

Now relations (ii), (iii) and (iv) imply s2n Yn 2 < ∞ a.s.

(v)

n

Because of (U4a)–(U4d) the sequence Mn is bounded a.s. Hence, under the assumptions in (d), relation (v) yields ρ2n Mn 2 Yn 2 < ∞ a.s. n

Thus, relation (i) implies now the assertion in (d). Conditions for the selection of sequence (an ) in inequality (6.7) are given in the next lemma: Lemma 6.9. Suppose that (i) Xn −→ x∗ a.s. 0n = D(k) a.s. with deterministic matrices D(k) for k = 1, . . . , κ (ii) lim M n∈IN(k)

Then (an ) can be selected such that T 1 a.s., k = 1, . . . , κ. an ≥ λmin D(k) H(x∗ ) + D(k) H(x∗ ) a(k) := lim inf 2 n∈IN(k) Proof. Consider an arbitrary k ∈ {1, . . . , κ}, put H ∗ := H(x∗ ) and and for n ∈ IN(k) deﬁne " an :=

b :=

1 λmin D(k) H ∗ + (D(k) H ∗ )T , 2

-n (G(Xn )−G(x∗ )) ∆n ,M ,

∆n 2 b

for ∆n = 0 , for ∆n = 0.

190

6 Stochastic Approximation Methods with Changing Error Variances

According to assumption (U1a), for ∆n = 0 we ﬁnd then T 0n − D(k) )H ∗ ∆n + M 0n δ(x∗ ; ∆n ) ( M ∆ T (k) ∗ n ∆ D H ∆n an = n + . ∆n 2 ∆n 2 Applying condition (U1b), we get 0n − D(k) H ∗ + M 0n L0 ∆n µ0 . an ≥ b − M Finally, due to the assumptions (i), (ii) and (U4a)–(U4d) we ﬁnd an ≥ b lim inf n∈IN(k)

a.s.

6.5 General Convergence Results In this section the convergence properties of the sequence (Xn ) generated by the algorithm (6.2a), (6.2b) are considered in case of “arbitrary” matrix 1 step sizes of order . Especially, theorems are given for the convergence with n probability one (a.s.), the rate of convergence, the convergence in the mean and the convergence in distribution. These convergence results are applied then to the algorithms having step sizes as given in Sects. 6.7 and 6.8. As stated in Sect. 6.3, here the assumptions (U1a)–(U1c) up to (U6) hold, and all further assumptions will be marked by the letter V. Here, we use the step size Rn = ρn Mn with a scalar factor ρn such that lim n ρn = 1

n→∞

a.s.

(V1)

According to condition (U4a), sequence (Mn ) is bounded a.s. Hence, with 1 probability 1, the step size Rn is of order . n 6.5.1 Convergence with Probability One A theorem about the convergence with probability 1 and the convergence rate of the sequence (Xn ) is given in the following. Related results can be found in the literature, see, e.g., [107] and [108]. The rate of convergence depends mainly on the positive random variable a, deﬁned in (6.9). Because of (6.7), this random variable can be interpreted as a mean lower bound of the quotients 0n · (G(Xn ) − G(x∗ )) Xn − x∗ , M . Xn − x∗ 2

6.5 General Convergence Results

191

9 1 we have Due to the assumptions (U5a)–(U5c), for each number λ ∈ 0, 2 the relations 1 σ 2 < ∞, 2 1 n n 1 2 σ0,n < ∞ a.s., 2(1−λ) n n 1 En Zn < ∞ a.s. n1−λ n

(6.27) (6.28) (6.29)

Theorem 6.1. If λ is selected such that 0 ≤ λ < min lim nλ (Xn − x∗ ) = 0

n→∞

$ 1 ,a a.s., then 2

a.s.

Proof. Because of (V1), with τn := nλ we ﬁnd lim τn ρn n1−λ = 1

n→∞

a.s. and τn+1 = (1 + γn )τn

with

nγn −→ λ.

Hence, due to (V1), (6.27), (6.28), (6.29) and (U4a) we get 2 ρn Mn σ12 < ∞ a.s., n

2 2 τn ρn Mn σ0,n < ∞ a.s., n

τn ρn Mn En Zn < ∞ a.s. n (1)

(2)

For the random variables tn , tn and tn occurring in Lemma 6.3 we have therefore t(i) (i) n < ∞ a.s. for i = 1, 2, n

tn − 2(an − λ) −→ 0 a.s. (k)

This yields t

:= lim inf tn = 2(a(k) − λ) f.s. for k = 1, . . . , κ. According to n∈IN(k)

the assumptions in this theorem and because of (6.9) we obtain t := q1 t(1) + . . . + qκ t(κ) = 2(a − λ) > 0

a.s.

(ii)

Because of (V1), (i) and (ii), the assertion follows now from Lemma 6.3 and Lemma B.3 in the Appendix. Replacing in Theorem 6.1 the bound a by the number a from (U4d), then a weaker result is obtained:

192

6 Stochastic Approximation Methods with Changing Error Variances

Corollary 6.1. If 0 ≤ λ < min

1 2, a

, then lim nλ (Xn − x∗ ) = 0 a.s. n→∞

Proof. According to (6.9) we have a ≥ a. The assertion follows now from Theorem 6.1.

Since a > 0, in Corollary 6.1 we may select also λ = 0. Hence, we always have Xn −→ x∗

a.s.

(6.30)

6.5.2 Convergence in the Mean We consider now the convergence rate of the mean square error EXn − x∗ 2 . For the estimation of this error the following lemma is needed: Lemma 6.10. If n ∈ IN, then En (n + 1)∆n+1 2

1 (1) 1 2 (nρn Mn )2 σ0,n ≤ 1 − sn n∆n 2 + + s(2) n n n (1)

(6.31)

(2)

with An -measurable random variables sn and sn , where an − 1) −→ 0 a.s., s(1) n − (2s(2) n −→ 0 a.s. Proof. Putting τn :=

√

(6.32) (6.33)

n, we ﬁnd

τn+1 = (1 + γn )τn

with nγn −→

1 . 2

According to Lemma 6.3 we select (1) s(1) n := n(ρn tn − tn ),

2 2 (2) s(2) n := ntn − nρn Mn σ0,n .

Then (U4a)–(U4d), (U5a)–(U5c) and (V1) yield √ ngn = n(nρn )Mn En Zn −→ 0 a.s., 2 n ρn Mn σ12 −→ 0 a.s. Consequently, an − 1) −→ 0 a.s., tn − (2nt(1) n −→ 0 a.s., 2 2 (2) ntn − nρn Mn σ0,n −→ 0 a.s. Thus, due to (V1) also (6.32) and (6.33) hold.

6.5 General Convergence Results

193

The following theorem shows that the mean square error EXn − x∗ 2 is 1 n , and an upper bound is given for the limit of the sequence essentially of∗order n EXn −x 2 . This bound can be found in the literature in the special case of scalar step sizes and having the trivial partition IN = IN(1) (cf. condition (i) from Sect. 6.3), see [26], [164, page 28] and [152]. Also in this result, an important role is played by the random variable a deﬁned in (6.9). Theorem 6.2. Let a> such that (a) (b)

a.s. Then for each ε0 > 0 there is a set Ω0 ∈ A

1 2

¯0 ) < ε0 , P(Ω

κ '

lim sup EΩ0 n∆n

2

≤

qk (-b(k) σk )2

k=1

2a−1

n∈IN

.

(6.34)

Here, -b(1) , . . . , -b(κ) and σ1 , . . . , σκ are constants such that 0n ≤ -b(k) lim sup M

a.s. and

n∈IN(k)

2 σk2 = lim sup E σ0,n n∈IN(k)

for each k ∈ {1, . . . , κ}. Proof. Because of Lemma 6.10 and the above assumptions, the assertions in this theorem follow directly from Lemma B.4 in the Appendix, provided there we set: 2 Tn := n∆n 2 , An := s(1) n , Bn := nρn Mn , 2 vn := σ0,n , Cn := s(2) n .

The following corollary is needed later on: Corollary 6.2. If a > such that

1 2

a.s., then for each ε0 > 0 there is a set Ω0 ∈ A

(a)

¯0 ) < ε0 P(Ω

(b)

lim sup EΩ0 n∆n 2 ≤ n∈IN

(-bσ)2 2a−1

provided the numbers -b and σ are selected such that 0n ≤ -b a.s. and lim sup M n∈IN

2 σ 2 = lim sup Eσ0,n . n∈IN

194

6 Stochastic Approximation Methods with Changing Error Variances

A convergence result in case of scalar step sizes is contained in this corollary: Corollary 6.3. Select numbers d(1) , . . . , d(κ) such that 0n = d(k) I lim M

n∈IN(k)

f.s. for k = 1, . . . , κ,

(6.35)

2dα∗ > 1,

(6.36)

where d := q1 d(1) + . . . + qκ d(κ) and α∗ := 12 λmin (H ∗ + H ∗T ). Then for each ε0 > 0 there is a set Ω0 ∈ A such that ¯0 ) < ε0 P (Ω

(a)

κ '

lim sup EΩ0 n∆n

2

(b)

≤

qk (d(k) σk )2

k=1

(6.37)

2dα∗ − 1

n∈IN

2 where again σk2 = lim sup Eσ0,n for each k ∈ {1, . . . , κ}. n∈IN(k)

Proof. Because of Corollary 6.1 and (6.35) we may apply Lemma 6.9. Hence, we may assume that the sequence (an ) in (6.7) fulﬁlls an ≥ d(k) α∗ a(k) := lim inf n∈IN(k)

for k = 1, . . . , κ

a.s.

Due to (6.9) and (6.36) we get therefore the inequality a ≥ dα∗ > 12 a.s. The assertion of the above Corollary 6.3 follows now from Theorem 6.2.

For given positive numbers α∗ , q1 , . . . , qκ , σ1 , . . . , σκ , the right hand side of (6.37) is minimal for d

(k)

:−1 9 q1 qκ 2 ∗ = σk α + ... + 2 σ12 σκ

for k = 1, . . . , κ.

(6.38)

Using these limits, Corollary 6.3 yields d=

1 , α∗

(6.39)

:−1 9 q1 qκ ∗2 + ... + 2 . lim sup EΩ0 n∆n ≤ α σ12 σκ n∈IN 2

(6.40)

For scalar step sizes this yields now some guidelines for an asymptotic optimal selection of the factors Mn in the step sizes Rn = ρn Mn . In the special case Mn −→ d · I

a.s.,

6.5 General Convergence Results

195

the most favorable selection reads d = α∗ −1 which yields then the estimate κ '

lim sup EΩ0 n∆n 2 ≤

k=1

n

qk σk2 .

α∗ 2

(6.41)

The right hand side of (6.40), (6.41), resp., contains, divided by α∗2 , the (q1 , . . . , qκ )-weighted harmonic resp. arithmetic mean of the variance limits σ12 , . . . , σκ2 of the estimation error sequence (Zn ). Hence, the bound in (6.40) is smaller than that in (6.41). 6.5.3 Convergence in Distribution √ In this section it is shown that the weighted error n(Xn − x∗ ) converges in distribution if further assumptions on the matrices Mn in step size Rn = ρn Mn and on the estimation error Zn are made. Hence, choose the following weighting factors: √ τn = n.

(6.42)

Now τn+1 = (1 + γn )τn holds again, where γn =

λn , n

λn −→

1 . 2

(6.43)

Furthermore, according to Corollary 6.1 it holds without further assumptions Xn −→ x∗

a.s.

(6.44)

Now we demand that matrices Mn fulﬁll certain limit conditions and, due to condition (U3), that x∗ is contained in the interior of the feasible set: Hence, (a) For all k ∈ {1, . . . , κ} there exists a deterministic matrix D(k) having lim Mn = D(k) a.s.

(V2)

n∈IN(k)

For n ∈ IN(k) let Dn := D(k) . (b) There exist a(1) , . . . , a(κ) ∈ IR such that a := q1 a(1) + . . . + qκ a(κ) >

1 , 2

D(k) H ∗ + (D(k) H ∗ )T ≥ 2a(k) I.

(V3a) (V3b)

(c) For any solution x∗ of (P), cf. (6.1), it holds ˚ x∗ ∈ D.

(V4)

196

6 Stochastic Approximation Methods with Changing Error Variances

Now some preliminary considerations are made which are useful for showing the already mentioned central limit theorem. According to (6.44) and assumptions (V2) and (V3a), (V3b) as well as Lemma 6.9, for bound a in (6.9) we may assume that a≥a>

1 2

a.s.

(6.45)

Moreover, due to (V1), in Sect. 6.4.2 sn :=

1 n

for

n = 1, 2, . . .

(6.46)

can be chosen, and (6.43) yields 1 γn = λn −→ . sn 2

(6.47)

Now, according to these relations and Lemma 6.6, for the matrices Bn := Dn H ∗ −

γn I = Dn H ∗ − λn I sn

(6.48)

arising in (6.16), the inequalities 2un , n un , I − sn Bn ≤ 1 − n

I − sn Bn 2 ≤ 1 −

(6.49) (6.50)

hold, where un := (an − λn ) −

1 2 (λ − 2λn an + b2n ), 2n n

(6.51)

and for n ∈ IN(k) we put an := a(k) and bn := b(k) := D(k) H ∗ . Hence, because of (6.47), for any k = 1, . . . , κ the limit lim un = u(k) := a(k) −

n∈IN(k)

1 2

(6.52)

exists. Moreover, using (6.45), we get u ¯ := q1 u(1) + . . . + qκ u(κ) = a −

1 > 0. 2

(6.53)

Stochastic Equivalence -n = √n(Xn − x∗ ) we ﬁnd that it can By further examination of sequence ∆ be replaced by a simpler sequence. This is guaranteed by the following result:

6.5 General Convergence Results

Theorem 6.3.

√

(0)

∆n+1 = (I − (0)

where ∆1

197

n(Xn − x∗ ) is stochastically equivalent to sequence 1 1 Bn )∆(0) n + √ Dn (En Zn − Zn ), n = 1, 2, . . . , n n

denotes an arbitrary A1 -measurable random variable.

Two stochastic sequences are called stochastically equivalent, if their difference converges stochastically to zero. √ The reduction onto the basic parts of the sequence ( n(Xn − x∗ )) in Theorem 6.3 is already mentioned in a similar form in [40] (Eq. (2.2.10), α = 1). But there the matrix Bn ≡ Λ is independent of index n. Because of Lemma 6.5 it is suﬃcient for the proof of the theorem to show (1) (2) that sequences (∆n ) and (∆n ) in (6.21) converge stochastically to 0. (1)

Lemma 6.11. (∆n ) converges stochastically to 0. Proof. According to the upper remarks and Lemma 6.5 it holds Tn+1 ≤ (1 −

un 1 -n + Cn ) )Tn + (∆Bn ∆ n n

(i)

with the nonnegative random variables Tn := ∆(1) n ,

√ Cn := n n + 1 ρn Mn En Zn + G∗ + ∆Pn .

Furthermore, according to (V1), (V4), (6.44), (U4a)–(U4d), (U5a)–(U5c) and Lemma 6.7 it holds (ii) ∆Bn , Cn −→ 0 a.s. Let ε, δ > 0 be arbitrary positive numbers. According to (ii) and Lemma B.5 in the Appendix there exists a set Ω1 ∈ A having ¯1 ) < P(Ω

δ , 4

∆Bn , Cn −→ 0

(iii) uniformly on Ω1 .

(iv)

Due to (6.45) and Corollary 6.2 there exists a constant K < ∞ and a set Ω2 ∈ A having ¯2 ) < P(Ω

δ , 4

-n ≤ K. sup EΩ2 ∆ n

(v) (vi)

198

6 Stochastic Approximation Methods with Changing Error Variances

For ε0 := δε¯ u[4(K + 1)]−1 > 0, with u ¯ from (6.53), according to (iv) there exists an index n0 such that for any n ≥ n0 on Ω0 := Ω1 ∩ Ω2 we have ∆Bn , Cn ≤ ε0 . Hence, with (i) and (iv) for n ≥ n0 it follows un ε0 (K + 1) E Ω0 T n + , EΩ0 Tn+1 ≤ 1 − n n and therefore because of (6.52), (6.53) and Lemma A.7, Appendix, δε ε0 (K + 1) = . u ¯ 4

lim sup EΩ0 Tn ≤ n

Thus, there exists an index n1 ≥ n0 having EΩ0 Tn ≤

δε 2

for n ≥ n1 .

(vii)

Finally, from (iii), (v), the Markov inequality and (vii) for indices n ≥ n1 holds ¯0 ) + P {Tn > ε} ∩ Ω0 P(Tn > ε) ≤ P(Ω ¯0 ) + ≤ P(Ω

δ δ EΩ0 Tn < + = δ. ε 2 2 (2)

Now the corresponding conclusion for the sequence (∆n ) is shown. (2)

Lemma 6.12. (∆n ) converges stochastically to 0. Proof. According to above remarks and Lemma 6.5 we have En Tn+1 ≤

1−

2un n

Tn +

1 2 (∆Dn 2 σ0,n + Cn ), n

(2)

where Tn := ∆n 2 and Cn := ∆Dn 2 σ12 ∆n 2 . Due to Lemma 6.7 it holds ∆Dn , Cn −→ 0

f.s.

Let ε, δ > 0 be arbitrary ﬁxed. Here, choosing B (k) ≡ C (k) ≡ 0, Lemma B.4, Appendix, can be applied. Hence, there exists a set Ω0 ∈ A where ¯0 ) < P(Ω

δ 2

and

lim sup EΩ0 Tn = 0. n

(i)

6.5 General Convergence Results

199

Hence, there exists an index n0 with EΩ0 Tn ≤

εδ 2

for

n ≥ n0 .

(ii)

Like in the previous proof from (i) and (ii) follows now P(Tn > ε) < δ

for all n ≥ n0 .

(0)

For sequence (∆n ) in Theorem 6.3 the following representation can be given n 1 (0) (0) √ Bm,n Dm ∆Zm (6.54) ∆n+1 = B0,n ∆1 − m m=1 for n = 1, 2, . . . . Here, Bn,n := I,

Bm,n

1 1 Bm+1 . := I − Bn · . . . · I − n m+1

(6.55)

is deﬁned for all indices m < n. Finally, according to (6.49) and (6.50) for matrices Bm,n it follows

n uj Bm,n ≤ =: bm,n , 1− (6.56) j j=m+1

n 2uj =: ¯bm,n . 1− Bm,n 2 ≤ (6.57) j j=m+1 Asymptotic Distribution Assume for the estimation error Zn the following conditions: (a) There are symmetric deterministic matrices W (1) , . . . , W (κ) such that for any k = 1, . . . , κ it holds either T ), W (k) = lim E(∆Zm ∆Zm m∈IN(k) T T lim E Em (∆Zm ∆Zm ) − E(∆Zm ∆Zm ) =0

m→∞

(V5a) (V5b)

or T ) W (k) = lim Em (∆Zm ∆Zm m∈IN(k)

a.s.

(V6)

(b) It holds either lim E ∆Zm 2 I{ ∆Zm 2 >tm} = 0 for all t > 0

m→∞

(V7)

or lim Em ∆Zm 2 I{ ∆Zm 2 >tm} = 0 a.s.

m→∞

for all t > 0.

(V8)

200

6 Stochastic Approximation Methods with Changing Error Variances

√ The convergence in distribution of n(Xn − x∗ ) is guaranteed then by the following result: √ n(Xn − x∗ ) is asymptotically N (0, V )Theorem 6.4. The sequence distributed. The covariance matrix V is the unique solution of the Lyapunovmatrix equation

T 1 1 ∗ ∗ = C, (6.58) DH − I V + V DH − I 2 2 where D := q1 D(1) + . . . + qκ D(κ) , C := q1 D(1) W (1) D(1)T + . . . + qκ D(κ) W (κ) D(κ)T

(6.59) (6.60)

and matrices D(1) , . . . , D(κ) from condition (V2). Central limit theorems of the above type can be found in many papers on stochastic approximation (cf., e.g., [1,12,40,70,107,108,131,132,161,163]). But in these convergence theorems it is always assumed that matrices Mn or the covariance matrices of the estimation error Zn converge to a single limit D, W , respectively. This special case is enclosed in Theorem 6.4, if assumptions (V2), (V3a), (V3b) and (V5a), (V5b)/(V6) for κ = 1 are fulﬁlled (cf. Corollary 6.4). Before proving Theorem 6.4, some lemmas are shown ﬁrst. For the deterministic matrices 1 Am,n := √ Bm,n Dm m

for m ≤ n,

(6.61)

arising in the important equation (6.54) the following result holds: Lemma 6.13. (a) For m = 1, 2, . . . it holds lim Am,n = 0. n→∞ n ' 2 Am,n < ∞. (b) lim sup n→∞ m=1

2L (c) It exists n0 ∈ IN with Am,n ≤ √ m sup Dn < ∞.

for

n0 ≤ m ≤ n, where L :=

n

(d) For indices m ∈ IN(k) , let Wm := W (k) . Then lim

n→∞

n

Am,n Wm ATm,n = V

m=1

with the matrix V according to (6.58). Proof. Because of (6.56) and (6.57) for m ≤ n it holds: L Am,n ≤ √ bm,n , m

(i)

6.5 General Convergence Results

Am,n 2 ≤

L2 ¯ bm,n . m

201

(ii)

Hence, according to (6.52) and (6.53) Lemma A.6, Appendix, can be applied. Therefore lim bm,n = 0 for

n→∞

lim

n→∞

m = 0, 1, 2, . . . ,

n

¯bm,n 1 = 1 < ∞, m 2¯ u m=1

(iii) (iv)

and because of Lemma A.5(a), Appendix, for any 0 < δ < u ¯ we have bm,n φm,n (¯ u − δ) ∼

m u¯−δ n

1.

(v)

From (i) to (v) propositions (a), (b) and (c) are obtained easily. Symmetric matrices V1 := 0 and Vn+1 :=

n

Am,n Wm ATm,n

for n = 1, 2, . . .

(6.62)

m=1

by deﬁnition fulﬁll equation Vn+1 =

T 1 1 1 I − Bn Vn I − Bn + Cn , n ≥ 1, n n n

(vi)

where Cn := Dn Wn DnT . Hence, because of (6.47) and (6.48) for all k the following limits exist: 1 B (k) := lim Bn = D(k) H ∗ − I. 2 n∈IN(k)

Furthermore, according to (V3b) it holds

1 (k) (k)T (k) B +B I = 2u(k) I. ≥2 a − 2 Therefore and because of (6.53), Lemma A.9, Appendix, can be applied to equation (vi). Hence, Vn −→ V , where V is unique solution of BV + V B T = C. Here, B := q1 B (1) + . . . + qκ B (κ) , and C is given by (6.60). Henceforth, B = DH ∗ − 12 I, where D is obtained from (6.59), and part (d) is proven.

202

6 Stochastic Approximation Methods with Changing Error Variances

Lemma 6.14. (a) If (V5a), (V5b) or (V6) holds, then n

stoch

T Am,n Em (∆Zm ∆Zm )ATm,n −→ V

(KV)

m=1

with matrix V from (6.58) (b) If (V7) or (V8) is fulﬁlled, then for each ε > 0 n

stoch Am,n 2 Em ∆Zm 2 I{ Am,n ∆Zm >ε} −→ 0

(KL)

m=1

holds. Proof. Let V-0 := 0, and for n = 1, 2, . . . V-n+1 :=

n

T Am,n Em (∆Zm ∆Zm )ATm,n .

m=1

Having matrix Vn+1 according to (6.62), for each n V-n+1 − V ≤ V-n+1 − Vn+1 + Vn+1 − V ,

(i)

EV-n+1 − V ≤ EV-n+1 − E V-n+1 + E V-n+1 − Vn+1 + Vn+1 − V (ii) holds. Furthermore, V-n+1 − Vn+1 ≤

n

T Am,n 2 Em (∆Zm ∆Zm ) − Wm ,

(iii)

m=1

EV-n+1 − E V-n+1 ≤

n

T T Am,n 2 E Em (∆Zm ∆Zm ) − E(∆Zm ∆Zm ) (iv)

m=1

and E V-n+1 − Vn+1 ≤

n

T Am,n 2 E(∆Zm ∆Zm ) − Wm .

(v)

m=1

For any real stochastic sequence (bn ) with bn −→ 0 a.s., Lemma 6.13(a),(b) and Lemma A.2, Appendix, provide n

Am,n 2 bm −→ 0 a.s.

(vi)

m=1

Besides, according to Lemma 6.13(d) it is Vn+1 − V −→ 0.

(vii)

6.5 General Convergence Results

203

Hence, under condition (V5a), (V5b) we get because of (ii), (iv), (v), (vi) and (vii) EV-n+1 − V −→ 0. In addition, with (V6), because of (i), (iii), (vi) and (vii) we have V-n+1 − V −→ 0

a.s.

Therefore, (KV) is fulﬁlled in both cases.

Select an arbitrary ε > 0. Due to Lemma 6.13(c), ∆Zm 2 I{ Am,n ∆Zm >ε} ≤ ∆Zm 2 I% √2L

m

&

∆Zm >ε

holds for all indices n0 ≤ m ≤ n. Hence, for n > n0 , where t := Un :=

n

ε 2 >0 2L

Am,n 2 Em (∆Zm 2 I{ Am,n Zm >ε} )

m=1

≤ Un0 +

n

Am,n 2 · Em ∆Zm 2 I{ ∆Zm 2 >tm} .

(viii)

m=n0 +1

Therefore, from (vi) and (viii) follows EUn −→ 0 at assumption (V7) and Un −→ 0 a.s. under assumption (V8). Hence, in any case (KL) is fulﬁlled. Now we are able to show Theorem 6.4. (0)

Proof (of Theorem 6.4). Let (∆n ) be deﬁned according to (6.54) with (0) ∆1 := 0. Thus, (0)

∆n+1 = −

n

Am,n ∆Zm

for n = 1, 2, . . . .

m=1

Due to Lemma 6.14 all conditions for the limit theorem Lemma B.7 in (0) the Appendix are fulﬁlled for this sequence. Hence, (∆n ) is asymptotically N (0, V )-distributed. According to Theorem 6.3 this holds also true for se √ n(Xn − x∗ ) . quence In the special case that the matrix sequence (Mn ) converges a.s. to a unique limit matrix D, we get the following corollary: Corollary 6.4. Let Mn −→ Θ

a.s.,

(6.63)

where Θ denotes a deterministic matrix having ΘH ∗ + (ΘH ∗ )T > I.

(6.64)

204

6 Stochastic Approximation Methods with Changing Error Variances

√

Then, n(Xn − x∗ ) is asymptotically N (0, V )-distributed. The covariancematrix V is unique solution of

T 1 1 ∗ ∗ ΘH − I V + V ΘH − I = ΘW ΘT , (6.65) 2 2 if W := q1 W (1) + . . . + qκ W (κ) .

(6.66)

Proof. In the present case, Θ(k) = Θ and a(k) = a for k = 1, . . . , κ, where a fulﬁlls ΘH ∗ + (ΘH ∗ )T ≥ 2aI. Due to (6.64) choose a > 12 . Hence, conditions (V2) and (V3) are fulﬁlled for

matrix sequence (Mn ). The assertion is now obtained from Theorem 6.4. Special limit theorems are obtained for diﬀerent limit-matrices D in Corollary 6.4: (a) For Θ = H ∗ −1 condition (6.64) is always fulﬁlled. Moreover, due to (6.65) matrix V is given by V = H ∗ −1 W (H ∗ −1 )T . This special case was already discussed for κ = 1 in [12, 107, 108, 161] (Corollary 9.1). T (b) Let Θ = d · I, where d is chosen to fulﬁll d · (H ∗ + H ∗ ) > I. Hence, according to (6.65) for V the following formula is obtained: d · (H ∗ V + V H ∗ ) − V = d2 W. T

This equation for the covariance matrix V was discussed, in case κ = 1, in [132] (Theorem 5) and in [40]. Matrix W in (6.66) can be interpreted as “mean limit covariance matrix” of the estimation error sequence (Zn ).

6.6 Realization of Search Directions Yn The approach used in this section for constructing search directions Yn of method (6.2a), (6.2b), allows adaptive estimation of important unknown items like Jacobi matrix H(x∗ ) and limit covariance matrix of error Zn . These estimators will be used in Sects. 6.7 and 6.8 for the construction of adaptive stepsizes. J.H. Venter (cf. [161]) takes in the one-dimensional case (ν = 1) at stage n the search direction 1 Yn := (Yn(1) + Yn(2) ). 2

6.6 Realization of Search Directions Yn (1)

205

(2)

Here, Yn and Yn are given stochastic approximations of G(Xn + cn ) and G(Xn − cn ), and (cn ) is a suitable deterministic sequence converging to zero. The advantage of this approach is that using the stochastic diﬀerence quotient 1 (Y (1) − Yn(2) ), hn := 2cn n a known approximation for the derivative of G(Xn ) is obtained. This diﬀerence quotient plays a fundamental role in the adaptive step size of Venter. In [12, 107] and [108] this approach for the determination of search directions was generalized to the multi-dimensional case. There, Yn :=

2ν 1 (j) Y 2ν j=1 n (i)

(ν+i)

is assumed as search direction, where vectors Yn and Yn are given stochastic approximations of vectors G(Xn + cn ei ) and G(Xn − cn ei ), and e1 , . . . , eν denote the canonic unit vectors of IRν . Then, n := 1 (Y (1) − Y (ν+1) , . . . , Y (ν) − Y (2ν) ) H n n n 2cn n is a known estimator of the Jacobian Hn := H(Xn ) of G at stage n. The main disadvantage of this approach for Yn is that at each iteration (1) (2ν) have to be determined. This can be n exactly 2ν realizations Yn , . . . , Yn avoided corresponding to the Venter approach if at stage n only two measure(1) (2) ments Yn and Yn are determined. Hence, only one column vector 1 (Y (1) − Yn(2) ) hn := 2cn n of the matrix H(Xn ) is estimated. Cycling through all columns, after ν iterations a new estimator for the whole matrix H(Xn ) is obtained. This possibility is contained in the following approach by means of choosing ln ≡ 1 and εn ≡ 0: For indices n = 1, 2, . . . , let ln ∈ IN0 and εn ∈ {0, 1} be integers with εn + ln > 0.

(W1a)

Furthermore, use at each iteration stage n ∈ IN Yn :=

1 2ln + εn

2ln

Yn(j)

(W1b)

j=1−εn (j)

as stochastic approximation of Gn := G(Xn ). Here, Yn (j) stochastic estimator of G at Xn , that is (j) Yn(j) = G(j) n + Zn ,

denotes a known

(6.67)

206

6 Stochastic Approximation Methods with Changing Error Variances (j)

(j)

(j)

where Gn := G(Xn ), and Zn denotes the estimation error for j = (0) (2l ) 0, . . . , 2ln . The arguments Xn , . . . , Xn n are given by (ln +i) := Xn − cn d(i) Xn(0) := Xn , Xn(i) := Xn + cn d(i) n , Xn n

(W1c)

for i = 1, . . . , ln , where (cn ) is a positive sequence converging to zero. As (i) direction vectors dn one after another the unit vectors e1 , e2 , . . . , eν , e1 , . . . of ν IR are chosen. Formally, this leads to d(i) n = el(n,i)

for

i = 1, . . . , ln ,

(W1d)

where indices l(n, i) are calculated as follows: l(1, 0) := 0, l(n, i) := l(n, i − 1) mod ν + 1

i = 1, . . . , ln ,

for

(W1e)

l(n + 1, 0) := l(n, ln ). Using the above choice of Yn , we get Yn = Gn + Zn

(6.68)

with estimation error Zn :=

1 2ln + εn

2ln

(j) (G(j) n + Zn ) − Gn .

(6.69)

j=1−εn

In order to guarantee that this total error Zn fulﬁlls conditions (U5a)–(U5c) (0) (2l ) in Sect. 6.3, let the error terms Zn , . . . , Zn n denote An+1 -measurable and stochastically independent random variables with given An . Furthermore, for j = 0, . . . , 2ln assume that 1 , 2

(W2a)

En Zn(j) − En Zn(j) 2 ≤ s20,n + s21 Xn(j) − x∗ 2

(W2b)

En Zn(j) ≤

L1 nµ1

with

µ1 >

with positive constants L1 , µ1 and s1 and An -measurable random variables s20,n ≥ 0, for which with an additional constant s0 sup Es20,n ≤ s20

(W2c)

n

is fulﬁlled. Hence, due to (W2b) and (W1c) En Zn(j) − En Zn(j) 2 ≤ t20,n + t21 Xn − x∗ 2 ,

(6.70)

sup Et20,n ≤ s20 + 2s21 sup c2n =: t20 ,

(6.71)

n

n

6.6 Realization of Search Directions Yn

207

if we denote t20,n := s20,n + 2c2n s21 , t21

:=

(6.72)

2s21 .

(6.73)

The error Zn has further properties: Lemma 6.15. For each n ∈ IN, Zn is An+1 -measurable, and the following properties hold: 1 (a) En Zn = 2ln + εn (b) En ∆Zn ∆ZnT = (c) En ∆Zn 2 =

2ln

(j) (G(j) n + En Zn ) − Gn ,

j=1−εn

1 (2ln + εn )2

1 (2ln + εn )2

2ln

En ∆Zn(j) ∆Zn(j)T ,

j=1−εn 2ln

En ∆Zn(j) 2

j=1−εn

1 t20,n + t21 Xn − x∗ 2 , ≤ 2ln + εn L1 0 (d) En Zn ≤ L0 c1+µ + µ1 with L0 and µ0 from (U1b). n n Here, the following abbreviations for n = 1, 2, . . . and j = 0, . . . , 2ln are used in Lemma 6.15: ∆Zn := Zn − En Zn , ∆Zn(j) := Zn(j) − En Zn(j) .

(6.74)

Proof. Formula (a) is a direct consequence of (6.69). Hence, it holds ∆Zn :=

1 2ln + εn

2ln

∆Zn(j) ,

j=1−εn (0)

(2ln )

and therefore due to the conditional independence of Zn , . . . , Zn En ∆Zn ∆ZnT =

1 (2ln + εn )2

2ln

we get

En ∆Zn(j) ∆Zn(j)T .

j=1−εn

Taking the trace of the matrix, we get En ∆Zn 2 = En trace (∆Zn ∆ZnT ) = trace (En ∆Zn ∆ZnT ) =

1 (2ln + εn )2

2ln j=1−εn

En ∆Zn(j) 2 ,

208

6 Stochastic Approximation Methods with Changing Error Variances

and using (6.70) assertion (c) follows. Due to condition (W2a) we have 2ln L1 1 (j) En Zn ≤ µ1 , 2ln + εn n j=1−εn and therefore by applying (a) we ﬁnd

ln 1 (i) L1 2 (ln +i) En Zn ≤ (Gn + Gn ) − Gn + µ1 n 2ln + εn 2 i=1 ln 1 (i) 2 L1 (ln +i) (G + Gn ≤ ) − Gn + nµ1 . 2ln + εn i=1 2 n Because of conditions (U1a)–(U1c) and (W1a), (W1b), for each i = 1, . . . , ln 1 (i) 1 (i) (i) n +i) (Gn + G(l δ(X ) − G ; c d ) + δ(X ; −c d ) = n n n n n n n n 2 2 1 1+µ0 1+µ0 0 = L0 c1+µ + cn d(i) ≤ L0 cn d(i) n n n 2 holds. Using the previously obtained inequality, assertion (d) follows. Due to Lemmas 6.15(c), (6.70) and (6.71), conditions (U5b) and (U5c) are fulﬁlled by means of 2 := σ0,n

1 t2 , σ0 := t0 , σ1 := t1 . 2ln + εn 0,n

(6.75)

For given positive numbers c0 and µ, choose for any n = 1, 2, . . . cn =

c0 , nµ

where

µ>

1 2(1 + µ0 )

(W3a)

with µ0 from condition (U1a)–(U1c). Due to Lemma 6.15(d) and (W2a) we get √ nEn Zn −→ 0 a.s. (6.76) Therefore the estimation error sequence (Zn ) fulﬁlls all conditions (U5a), (U5b). For convergence of the covariance matrices of the estimation error sequence (Zn ) (cf. Lemma 6.18), we need the existence of l(k) := lim ln < ∞,

(W4a)

ε(k) := lim εn ∈ {0, 1}

(W4b)

n∈IN(k)

n∈IN(k)

6.6 Realization of Search Directions Yn

209

for sequences (ln ) and (εn ) for each k = 1, . . . , κ. In particular this leads to l := sup ln < ∞,

(6.77)

n∈IN

¯l := lim l1 + . . . + ln = q1 l(1) + . . . + qκ l(κ) . n→∞ n

(6.78)

By means of approach (W1a), (W1b) for search directions Yn it is easy to get convergent estimators for G(x∗ ), H(x∗ ) as well as the limit covariance matrix of (Zn ). 6.6.1 Estimation of G∗ Vectors G∗n+1 :=

1 (Y1 + . . . + Yn ) for n

n = 1, 2, . . .

(6.79)

can be interpreted as An+1 -measurable estimators of G∗ := G(x∗ ). This is shown in the next result: Theorem 6.5. If Xn −→ x∗ a.s., then G∗n −→ G∗ a.s. Proof. For n = 1, 2, . . . we have G∗n+1

=

1 1− n

G∗n +

1 Yn . n

(i)

Due to (U5a), our assumption and (6.4) it holds En Yn − Gn = En Zn −→ 0, Gn −→ G∗

a.s.,

and therefore we get En Yn −→ G∗

a.s.

(ii)

Moreover, due to (U5b), (U5c) and condition (a) in Sect. 6.3 for each n EYn − En Yn 2 = EZn − En Zn 2 ≤ σ02 + (σ1 δ0 )2 holds and therefore 1 E Yn − En Yn 2 < ∞ a.s. 2 n n n

(iii)

Using (i), (ii) and (iii), the proposition follows from Lemma B.6 in the Appendix.

210

6 Stochastic Approximation Methods with Changing Error Variances

6.6.2 Update of the Jacobian For numbers ln in deﬁnition (W1a)–(W1e) we demand now ln > 0 for

n = 1, 2, . . . .

(W5)

Furthermore, for exponent µ in (W3a) µ<

1 2

(W3b)

is assumed. Due to the following lemma, the “stochastic central diﬀerence quotients” 1 h(i) (Y (i) − Yn(ln +i) ) for i = 1, . . . , ln , (6.80) n := 2cn n (i)

known at iteration stage n, are estimators of Hn · dn , where Hn := H(Xn ). (i)

Lemma 6.16. For each n ∈ IN and i ∈ {1, . . . , ln }, hn is An+1 -measurable, and it holds: 1 1 n +i) (G(i) − G(l )+ (En Zn(i) − En Zn(ln +i) ) n 2cn n 2cn 1 2 (i) 2 2 ∗ 2 t + t X − x (b) En h(i) n n − En h n ≤ 0,n 1 2c2n

(a) En h(i) n =

(i) (c) En h(i) n − Hn dn −→ 0 (j)

(j)

(j)

Proof. Part (a) holds because of En Yn = Gn + En Zn for j = 1, . . . , 2ln . (j) (j) (j) Furthermore, with ∆Zn := Zn − En Zn we have 1 (i) 2 En h(i) (∆Zn(i) − ∆Zn(ln +i) )2 n − En h n = E n 2cn 1 1 = 2 En ∆Zn(i) 2 + En ∆Zn(ln +i) 2 ≤ 2 t20,n + t21 Xn − x∗ 2 . 4cn 2cn

(j)

Here, the independence of the variables Zn was taken into account, and (6.70) was used. Now, because of (U1a)–(U1c) (i) (i) 0G(Xn ± cn d(i) n ) = G(Xn ) ± cn Hn dn + δ(Xn ; ±cn dn ),

and therefore due to (a), (W2a) (U1b) and (W3a) we ﬁnd 1 L1 (i) (i) (i) (i) δ(Xn ; cn dn ) − δ(Xn ; −cn dn ) + En hn − Hn dn = 2cn cn nµ1 L1 L0 L1 1+µ0 1+µ0 cn d(i) + ≤ + − cn d(i) = L0 cµn0 + . n n 2cn cn nµ1 c0 nµ1 −µ Finally, since cn −→ 0, µ0 > 0 and µ1 >

1 2

> µ, proposition (c) follows.

6.6 Realization of Search Directions Yn

211

(i)

Since for directions dn we use cyclically all unit vectors e1 , . . . , eν , the (i) Jacobian H ∗ := DG(x∗ ) can be estimated easily by means of vectors hn . For this recursive estimate we start with a A1 -measurable ν × ν-Matrix (0) H1 . In an iteration step n ≥ 1 we calculate for i = 1, . . . , ln An+1 -measurable matrices 1 (i)T − Hn(i−1) d(i) (6.81a) Hn(i) := Hn(i−1) + (h(i) n ) dn n n and put (0) (6.81b) Hn+1 := Hn(ln ) . According to (W1d), (6.81a), (6.81b) leads to ⎧ ⎨ 1 − 1 h(i−1) + 1 h(i) (i) n,l n n n , for hn,l := (i−1) ⎩h , for n,l

(i)

(i)

l = l(n, i) l = l(n, i),

(i)

if Hn = (hn,1 , . . . , hn,ν ). (i−1)

(i)

Hence, at transition of Hn to Hn only the l(n, i)th column vector is (i) changed (updated) by averaging with vector hn of (6.80). (0) Due to the following theorem, matrices Hn are known approximations of ∗ ∗ H , if Xn converges to x . (i) ¯ (0) denote an arbiBesides Hn this theorem uses further matrices: Let H 1 trary matrix, and for n = 1, 2, . . . and i = 1, . . . , ln let (i)T ¯ n(i) := H ¯ n(i−1) + 1 (Hn − H ¯ n(i−1) )d(i) H n dn , n

¯ (0) := H ¯ (ln ) . H n n+1

(6.82a) (6.82b)

¯ n(i) ) is bounded, and it holds Hn(0) − H ¯ n(0) −→ 0 a.s. Theorem 6.6. (a) (H (0) (b) If “Xn −→ x∗ a.s.,” then Hn −→ H ∗ a.s. Proof. To arbitrary ﬁxed l ∈ {1, . . . , ν} let (n1 , i1 ), (n2 , i2 ), . . . denote all or(i) dered index pairs (n, i), for which dn = el is chosen. Hence, 1 ≤ n1 ≤ n2 ≤ . . ., and for each k it holds 1 ≤ ik ≤ lnk as well as ik < ik+1 if nk = nk+1 . Moreover, for n ≥ 1 and i ≤ ln we have = e ⇐⇒ (n, i) ∈ (n , i ), (n , i ), . . . . d(i) l 1 1 2 2 n Due to (6.81a), (6.81b) and (6.82a), (6.82b), for k = 1, 2, . . . we get Vk+1 = (1 − αk )Vk + αk Uk ,

(i)

¯k , V¯k+1 = (1 − αk )V¯k + αk U

(ii)

212

6 Stochastic Approximation Methods with Changing Error Variances

if αk :=

1 , nk

k) Uk := h(i nk ,

¯k := Hn el , U k

k−1 ) Vk := Hn(ik−1 el ,

¯ (ik−1 ) el . V¯k := H nk−1

Because of (W1e), at latest after each νth iteration stage n, direction vector el is chosen. Therefore it holds nk+1 ≤ nk + ν and nk ≤ kν for each index k. Consequently, we get αk = ∞. (iii) k

Because of (W1e), and (6.77), at each iteration stage n, direction vector el is chosen at most l0 := [ -l−1 ν ] + 1 times and therefore k ≤ l0 nk for k = 1, 2, . . . . Hence, using condition (W3a),(W3b) we get α2 1 1 l02−2µ 1 k = ≤ < ∞. c2nk c20 c20 k 2−2µ n2−2µ k k k k

(iv)

Now, for k ≥ 1 let denote Bk the σ-algebra generated by Ank and U1 , . . . , Uk−1 . Obviously now holds: B1 ⊂ B2 ⊂ . . . ⊂ Bk ⊂ A ,

(v)

¯k ∆Uk := Uk − U

(vi)

is

Bk+1 -measurable,

k) E(∆Uk | Bk ) = Enk (∆Uk ) = Enk h(i (vii) nk − Hnk el , 2 (ik ) 2 k) E ∆Uk − E(∆Uk | Bk ) | Bk = Enk h(i nk − Enk hnk . (viii)

From (iv), (viii) and Lemma 6.16(b) we have αk2 E ∆Uk − E(∆Uk | Bk )2 | Bk k

≤

α2 2 ∗ 2 k t < ∞ a.s. + t X − x 0,n n 1 k k 2c2nk

(ix)

k

From (vii) and Lemma 6.16(c) we obtain E(∆Uk | Bk ) −→ 0

a.s.

Equations (i) and (ii) yield Vk+1 − V¯k+1 = (1 − αk )(Vk − V¯k ) + αk ∆Uk .

(x)

6.6 Realization of Search Directions Yn

213

Due to (iii), (v), (vi), (ix) and (x), Lemma B.6 in the Appendix can be applied. Hence, it holds ¯ n(ik ) )el −→ 0 a.s. Vk+1 − V¯k+1 = (Hn(ikk ) − H k Since l ∈ {1, . . . , ν} was chosen arbitrarily, we have shown proposition (a). In case of “Xn −→ x∗ a.s.” follows according to (U1c) also Hn −→ H ∗ a.s. Therefore, because of (ii), (iii) and Lemma B.6 (Appendix) ¯ (ik ) el −→ H ∗ el V¯k+1 = H nk

a.s.

¯ n(0) −→ H ∗ a.s. and therefore because of (a) also Hn(0) −→ Hence, it holds H ∗ H a.s. (0) Matrices Hn+1 from (6.81a), (6.81b) can also be approximated by means of special convex combinations of Jacobians H1 , . . . , Hn This is shown next. Theorem 6.7. Having “ Xn+1 − Xn −→ 0 a.s.,” we get - (0) −→ 0 Hn(0) − H n

a.s.

- n(i) are chosen as follows: Choose H - (0) := Here, An -measurable matrices H 1 (0) H1 , and for n = 1, 2, . . . as well as i = 1, . . . , ln let - n(i) := H - n(i−1) + 1 (Hn − H - n(i−1) ), H nν - (0) := H - (ln ) . H n n+1

(6.83a) (6.83b)

Proof. According to Theorem 6.6(a) only the following property has to be shown: - n(0) −→ 0 a.s. ¯ n(0) − H (∗) H Simple computation yields for n = 1, 2, . . . and 1 ≤ i ≤ ln

(i)T ¯ n(i−1) − H - n(i) = 1 − 1 (H - n(i−1) ) + 1 (Hn − H ¯ n(i−1) )(νd(i) ¯ n(i) − H − I). H n dn nν nν (i) Deﬁning now L(n, i) , nν ¯ (i) − H - (i) , − I) , VL(n,i) := H n n

L(n, i) := l1 + . . . + ln−1 + i, αL(n,i) := ¯ (i−1) )(νd(i) d(i)T UL(n,i) := (Hn − H n n n (i) can be rewritten as:

αL αL VL−1 + UL VL = 1 − L L

(ii)

for L = 1, 2, . . . . Due to (6.3) and Theorem 6.6(a) we have (UL ) bounded a.s.,

(iii)

214

6 Stochastic Approximation Methods with Changing Error Variances

and according to (6.78) we get lim αL =

L→∞

¯l =: α > 0. ν

(iv)

Hence, because of (ii), (iii), (iv) and Lemma A.4, Appendix, it is (VL ) bounded and VL − VL−1 −→ 0

a.s.

(v)

Because of (ii), for k = 0, 1, 2, . . . it holds V(k+1)ν = p0,k Vkν +

ν

pm,k

m=1

if we deﬁne pm,k :=

⎧ ⎨ ⎩

ν * j=m+1

1−

αkν+j kν+j

α α¯ αkν+m Ukν+m = 1 − Vkν + U k, kν + m k k

(vi)

, for m = 0, . . . , ν − 1

1 , for m = ν,

ν kν k αkν+m ¯k := 1 U · Ukν+m + (p0,k − 1) + 1 Vkν . pm,k ν m=1 α kν + m α Now we show ¯k −→ 0 U

a.s.

(∗∗)

Because of (vi) and Lemma A.4, Appendix, from this follows Vkν −→ 0 a.s. Hence, if (∗∗) holds, then due to (v), also (∗) holds true.

¯k , for each k = 0, 1, 2, . . . the following First of all, from the deﬁnition of U approximation is obtained: + ν , ν kν 1 αkν+m ¯ · − 1 Uk ≤ Ukν+m + Ukν+m · pm,k ν m=1 α kν + m m=1 k + (p0,k − 1) + 1 · Vkν . α

(vii)

For matrices ¯ n(i−1) , TL(n,i) := Hn − H due to “Xn+1 − Xn −→ 0 a.s.,” inequality (6.3) and Theorem 6.6(a), we have TL+1 − TL −→ 0

a.s. (i)

(viii)

Since for an index L = kν + m direction dn = em is chosen, for each k = 0, 1, 2, . . . holds

6.6 Realization of Search Directions Yn

215

ν ν Tkν + (Tkν+m − Tkν ) · (νem eTm − I) Ukν+m = m=1 ν

=

m=1

ν (Tkν+m − Tkν ) · (νem eTm − I) ≤ ν Tkν+m − Tkν .

m=1

(ix)

m=1

Now, let ε ∈ (0, α) denote a given arbitrary number. According to (iv) and (viii) there exists an index k0 ∈ IN such that α − ε ≤ αkν+m ≤ α + ε, ν Tkν+m − Tkν ≤ ε,

(x) (xi)

m=1

pm,k αkν+m · kν − 1 ≤ ε α kν + m

(xii)

for all k ≥ k0 and m = 1, . . . , ν. From (x), for k ≥ k0 we ﬁnd

ν

ν α−ε α+ε ≤ p0,k ≤ 1 − . 1− kν (k + 1)ν Since for each t ∈ [0, 1] the inequality 0 ≤ (1 − t)ν − (1 − νt) ≤ for indices k ≥ k0 we get −

ν(ν−1) 2 t 2

holds,

ν−1 α−ε α+ε α−ε ≤ p0,k − 1 ≤ (−1 + · ) k k+1 2 (k + 1)ν

or α+ε k k − + 1 ≤ (p0,k − 1) + 1 ≤ α α α(k + 1)

k+1 α−ε (α − ε)2 α − + , k α 2(k + 1)

respectively. Hence, there exists an index k1 ≥ k0 with (p0,k − 1) k + 1 ≤ 2 ε for k ≥ k1 . α α

(xiii)

Using now in (vii) the approximations (ix), (xi), (xii) and (xiii), we ﬁnally get for all k ≥ k1 ν 8 2ε 17 νε + ε Ukν+m + Vkν ν α m=1 : 9 2 a.s. ≤ ε · 1 + sup UL + sup VL α L L

¯k ≤ U

Since ε > 0 was chosen arbitrarily, relation (∗∗) follows now by means of (iii) and (v).

216

6 Stochastic Approximation Methods with Changing Error Variances

6.6.3 Estimation of Error Variances Special approach (W1a)–(W1e) for search directions Yn also allows estimation of limit covariance matrices W (1) , . . . , W (κ) of error sequence (Zn ) in procedure (6.2a), (6.2b). Again the following is assumed: ln > 0 for (1)

n = 1, 2, . . . . (2ln )

Since at stage n the vectors Yn , . . . , Yn

(W5)

are known, also

ln -n := 1 (Y (i) − Yn(ln +i) ) Z 2ln i=1 n

(6.84)

can be calculated. These random vectors behave similarly to the estimation error Zn . This can be seen by comparing Lemma 6.15 with the following result: Lemma 6.17. For each n ∈ IN holds ln 1 n +i) (a) En Zn = (G(i) − G(l + En Zn(i) − En Zn(ln +i) ) n 2ln i=1 n

(b) En (Z-n − En Z-n )(Z-n − En Z-n )T = (c) En Z-n − En Z-n 2 =

2ln 1 En (∆Zn(j) ∆Zn(j)T ) (2ln )2 j=1

2ln 1 En ∆Zn(j) 2 (2ln )2 j=1

(d) En Z-n −→ 0 -n − En Z-n . In the following we put ∆Z-n := Z (j)

(j)

(j)

(j)

Proof. From relation Yn = Gn + En Zn + ∆Zn we obtain representation in (a) and the following formula -n = ∆Z

ln 1 (∆Zn(i) − ∆Zn(ln +i) ). 2ln i=1

(1)

(2ln )

Hence, due to the conditional independence of Zn , . . . , Zn -T = En ∆Z-n ∆Z n

we get

ln 1 En (∆Zn(i) − ∆Zn(ln +i) )(∆Zn(i) − ∆Zn(ln +i) )T (2ln )2 i=1

ln 1 = (En ∆Zn(i) ∆Zn(i)T + En ∆Zn(ln +i) ∆Zn(ln +i)T ). (2ln )2 i=1

6.6 Realization of Search Directions Yn

217

This is proposition (b), and by taking the trace we also get (c). According to (U1a)–(U1c) and (6.3), for i = 1, . . . , ln follows (ln +i) (i) (i) = 2cn Hn d(i) G(i) n − Gn n + δ(Xn ; cn dn ) − δ(Xn ; −cn dn ) 0 . ≤ 2cn β + 2L0 c1+µ n ln 1 ' (i) (ln +i) Because of (W2a) we have 2ln (En Zn − En Zn ) ≤

i=1

L1 nµ1

and therefore

by means of (a) L1 . nµ1 The righthand side of this inequality converges to 0, since due to (W2a) and (W3a), (W3b) it holds cn −→ 0 and µ1 > 0. En Z-n ≤ cn (β + L0 cµn0 ) +

Now, for the existence of limit covariance matrices W (1) , . . . , W (κ) of (Zn ) required by (V6) we assume: Under assumption “Xn −→ x∗ a.s.” suppose lim En ∆Zn(jn ) ∆Zn(jn )T = V (k)

n∈IN(k)

a.s.

(W6)

exist for each k = 1, . . . , κ and for each sequence (jn ) with jn ∈ {0, . . . , 2ln }, n = 1, 2, . . . . Here, V (1) , . . . , V (k) are deterministic matrices. Due to the following lemma, sequence (Zn ) fulﬁlls condition (V6) from Sect. 6.5.3 with W (k) :=

2l(k)

1 V (k) + ε(k)

for k = 1, . . . , κ.

(6.85)

Lemma 6.18. If “ Xn −→ x∗ f.s.,” then for each k = 1, . . . , κ (a) lim En ∆Zn ∆ZnT = n∈IN(k)

(b) lim En Z-n Z-nT = n∈IN(k)

2l(k)

1 V (k) + ε(k)

1 V (k) 2l(k)

a.s.,

a.s.

Proof. Due to conditions (W4a), (W4b) and (W6), with Lemmas 6.15(b) and 6.17(b) we have lim En ∆Zn ∆ZnT =

n∈IN(k)

-T = lim En ∆Z-n ∆Z n

n∈IN(k)

1 V (k) 2l(k) + ε(k) 1 (k) 2ln

V (k)

a.s..

a.s.,

218

6 Stochastic Approximation Methods with Changing Error Variances

By means of -n ∆Z -T = En Z-n Z-T − (En Z-n )(En Z-n )T , E n ∆Z n n (En Z-n )(En Z-n )T = En Z-n 2 from the second equation and Lemma 6.17(d) we obtain proposition (b).

Because of Lemma 6.18 the following averaging method for the calculation of matrices W (1) , . . . , W (κ) is proposed: (1)

(κ)

At step n ∈ IN(k) with unique k ∈ {1, . . . , κ}, let matrices Wn , . . . , Wn already be calculated. Furthermore, let (k)

Wn+1 := (1 − (i)

Wn+1 := Wn(i)

2ln - -T 1 1 )W (k) + · Zn Zn n n n 2ln + εn

(6.86a)

for i = k, i ∈ {1, . . . , κ}.

(6.86b)

(1)

(κ)

Here, at the beginning let matrices W1 , . . . , W1 be chosen arbitrarily. Hence, if at stage n index n lies in IN(k) with a k ∈ {1, . . . , κ}, only matrix (k) (1) (κ) Wn of Wn , . . . , Wn is updated. (i)

For the convergence of matrix sequence (Wn )n the following assumption (j) on the fourth moments of sequence (Zn ) is needed: There exists µ2 ∈ [0, 1) and a random variable L2 < ∞ a.s. such that En ∆Zn(j) 4 ≤ L2 nµ2 a.s.

(W7)

for all n ∈ IN and j = 0, . . . , 2ln . Hence, initially we get ' 1 - -T - -T 2 Lemma 6.19. n2 En Zn Zn − En Zn Zn < ∞ a.s. n

-n Z-T we have Proof. For each n ∈ IN with matrix Cn := Z n En Cn − En Cn 2 ≤ En tr (Cn − En Cn )T (Cn − En Cn ) = tr En (Cn − En Cn )T (Cn − En Cn ) = tr En CnT Cn − (En Cn )T (En Cn ) ≤ tr(En CnT Cn ) = En trCnT Cn = En tr Z-n 2 Cn -n + En Z-n 4 = En Z-n 4 = En ∆Z 4 ln 1 = En (∆Zn(i) − ∆Zn(ln +i) ) + En Z-n 2ln i=1

6.6 Realization of Search Directions Yn

219

⎛

⎞4 2ln 1 ≤ En ⎝ ∆Zn(j) + En Z-n ⎠ 2ln j=1 ⎞ ⎛ 2ln 1 ≤ 8⎝ En ∆Zn(j) 4 + En Z-n 4 ⎠ . 2ln j=1 where “tr” denotes the trace of a matrix.

Hence, with (W7) and Lemma 6.17(d) the proposition follows. (1)

(κ)

Finally, we show that matrices Wn , . . . , Wn covariance matrices W (1) , . . . , W (κ) :

are approximations of limit

Theorem 6.8. If “ Xn −→ x∗ a.s.,” then for any k = 1, . . . , κ lim Wn(k) =

n∈IN(k)

2l(k)

1 V (k) = lim En ∆Zn ∆ZnT + ε(k) n∈IN(k)

a.s.

(6.87)

Proof. For given ﬁxed k ∈ {1, . . . , κ} let IN(k) = {n1 , n2 , . . .} with indices n1 < n2 < . . . . With abbreviations Ci :=

2lni , Bi := Ani , Z-n Z-T , Ti := Wn(k) i 2lni + εni i ni

by deﬁnition we have Ti+1 =

1 1 Ti + Ci 1− ni ni

for i = 1, 2, . . . .

Due to Lemma 6.18 and (W4a), (W4b) it holds lim E(Ci | Bi ) =

i→∞

Because of (U6) we have

1 V (k) = lim En ∆Zn ∆ZnT 2l(k) + ε(k) n∈IN(κ)

a.s.

(i)

i −→ qk > 0 for i −→ ∞ and therefore ni 1 = ∞. ni i

(ii)

1 Ci − E(Ci | Bi )2 | Bi < ∞ a.s. E n2i i

(iii)

Lemma 6.19 yields

Due to (i), (ii) and (iii) Lemma B.6, Appendix, can be applied to sequence

(Tn ). Hence, we have shown the proposition.

220

6 Stochastic Approximation Methods with Changing Error Variances (1)

(κ)

In a similar way approximations wn , . . . , wn errors of sequence (Zn ) can be given:

of the mean quadratic limit

At step n ∈ IN(k) with unique k ∈ {1, . . . , κ}, let

2ln 1 1 (k) -n 2 wn(k) + · Z wn+1 := 1 − n n 2ln + εn (i)

wn+1 := wn(i)

for i = k, i ∈ {1, . . . , κ}.

(1)

(κ)

Here, initially let arbitrary w1 , . . . , w1 vergence corollary can be shown:

(6.88a) (6.88b)

be given. Then, the following con-

Corollary 6.5. If “ Xn −→ x∗ a.s.,” then for any k = 1, . . . , κ lim wn(k) =

n∈IN(k)

(k)

Proof. If w1 1, . . . , κ

2l(k)

1 tr(V (k) ) = lim En ∆Zn 2 + ε(k) n∈IN(k)

a.s.

(6.89)

(k)

= tr(W1 ) for k = 1, . . . , κ , then for each n ∈ IN and k = wn(k) = tr(Wn(k) ) (k)

with matrix Wn from (6.86a), (6.86b). Hence, the proposition follows from Theorem 6.8 by means of taking the trace.

6.7 Realization of Adaptive Step Sizes Using the estimators for the unknown Jacobian H ∗ := H(x∗ ) and the limit covariance matrices W (1) , . . . , W (κ) of the sequence of estimation errors (Zn ), developed in Sect. 6.6, it is possible to select the step size Rn in algorithm (6.2a), (6.2b) adaptively such that certain convergence conditions are fulﬁlled. For the scalar ρn in the step size Rn = ρn Mn we suppose, as in Sect. 6.5, lim n ρn = 1

n→∞

a.s.

(X1)

Moreover, let the step directions (Yn ) be deﬁned by relations (W1a), (W1b) in Sect. 6.6. (0) Starting with a matrix H0 , and putting H1 := H0 , An -measurable ma(0) trices Hn can be determined recursively by (6.81a), (6.81b). (0) Depending on Hn , the factor Mn in the step size Rn is selected here such that conditions (U4a)–(U4d) from Sect. 6.3 are fulﬁlled. Independent of the convergence behavior of (Xn ), for large index n the (0) - n(0) given by (6.83a), (6.83b). matrices Hn do not deviate considerably from H This is shown next: - n −→ 0 a.s. Lemma 6.20. ∆Hn := Hn − H (0)

(0)

6.7 Realization of Adaptive Step Sizes

221

Proof. Because of (X1), the assumption of Lemma 6.8(d) holds. Hence, (Xn+1 − Xn ) converges to zero a.s. Using Theorem 6.7, the assertion follows.

- n ∈ H, where H denotes the set of According to the deﬁnition we have H all matrices H ∈ IM(ν × ν, IR) such that λi H (i) , I is ﬁnite, λi = 1, λi > 0 and H= (0)

i∈I

H

(i)

i∈I

% & ∈ H(x) : x ∈ D ∪ {H0 } for i ∈ I.

For the starting matrix H0 we assume H0 ≤ β with a number β according to (6.3). Then, for all H ∈ H we get H ≤ β.

(6.90)

6.7.1 Optimal Matrix Step Sizes If in (6.2a), (6.2b) a true matrix step size is used, then due to condition (U3) from Sect. 6.3 we suppose ˚ (X2) x∗ ∈ D. In case that the covariance matrix of the estimation error Zn converges towards a unique limit matrix W in the sense of (V5a), (V5b) or (V6), see Sect. 6.5, according to [117] for Mn := (H ∗ )−1

with

H ∗ := H(x∗ )

we √ obtain the minimum asymptotic covariance matrix of the process ( n(Xn − x∗ )). Since H ∗ is unknown in general, this approach is not possible directly. (0) However, according to Theorem 6.6, the matrix Hn is a known An measurable approximate to H ∗ at stage n, provided that (Xn ) converges to x∗ . Hence, similar to [12, 107, 108], for n = 1, 2, . . . we select (0) (0) (Hn )−1 , if Hn ∈ H0 Mn := (6.91) −1 H0 , else. Here, H0 is a set of matrices H ∈ IM(ν × ν, IR) with appropriate properties. Concerning H(x), H0 and H0 we suppose: (a) There is a positive number α such that H(x)H(y)T + H(y)H(x)T ≥ 2α2 I, H(x)H0T + H0 H(x)T ≥ 2α2 I, H0 H0T ≥ α2 I for all x, y ∈ D.

(X3a) (X3b) (X3c)

222

6 Stochastic Approximation Methods with Changing Error Variances

(b) There is a positive number α0 with H0 ⊂ H⊕ := H ∈ IM(ν × ν, IR) : HH T ≥ α02 I .

(X4)

Assumption (X3a)–(X3c) is also used in [108] and in [107]. As is shown later, see Lemma 6.22, conditions (X3a)–(X3c) and (X4) guarantee that sequence (Mn ) given by (6.91) fulﬁlls condition (U4a)–(U4d) in Sect. 6.3. If G(x) is a gradient function, hence, G(x) = ∇F (x) with a function F : D0 −→ IR, then (X3a)–(X3c) does not guarantee the convexity of F . Conversely, the Hessian H(x) of a strongly convex function F (x) does not fulﬁll condition (X3a)–(X3c) in general. Counter examples can be found in [115]. Some properties of the matrix sets H⊕ and H are described in the following: Lemma 6.21. (a) For a matrix H ∈ IM(ν × ν, IR) the following properties are equivalent (i) H ∈ H⊕ (ii) H is regular and H −1 ≤

1 α0

(b) For H ∈ H and x ∈ D we have (i) H(x)H T + HH(x)T ≥ 2α2 I (ii) HH T ≥ α2 I 1 (iii) H is regular and H −1 ≤ α (iv) H T H ≤ β 2 I 1 (v) (H −1 )(H −1 )T ≥ 2 I β T α2 (vi) H −1 H(x) + H −1 H(x) ≥ 2 2 I β Proof. If H is regular, then (HH T )−1 = (H −1 )T H −1 , H −1 2 = λmax (H −1 )T (H −1 ) , 1 HH T ≥ α02 I ⇐⇒ (HH T )−1 ≤ 2 I. α0

This yields the equivalence of (i) and (ii) in (a). ' λi H (i) ∈ H and x ∈ D be given arbitrarily. Because of Let now H = i∈I

(X3a)–(X3c) we have

6.7 Realization of Adaptive Step Sizes

H(x)H T + HH(x)T =

223

λi H(x)H (i)T + H (i) H(x)T ≥ 2α2 I

i∈I

and HH T =

λi λk H (i) H (k)T ≥ α2 I.

i,k∈I −1 T −1 T −1 ≤ α−2 I. This Hence, H is regular, and (H ) (H ) = (HH ) yields H −1 2 = λmax (H −1 )T (H −1 ) ≤ α−2 . Due to (6.90) we ﬁnd

H T H ≤ λmax (H T H)I = H2 I ≤ β 2 I and therefore β −2 I ≤ (H T H)−1 = (H −1 )(H −1 )T . Because of T H −1 H(x) + H −1 H(x) = (H −1 ) H(x)H T + HH(x)T (H −1 )T , assumptions (i) and (v) of (b) ﬁnally yield T α2 ≥ 2α2 (H −1 )(H −1 )T ≥ 2 2 I. H −1 H(x) + H −1 H(x) β Using Lemma 6.21, we obtain, as announced, the following result: Lemma 6.22. (a) sup Mn ≤ max{α0−1 , α−1 }. n∈IN

0n and ∆Mn such (b) For each n ∈ IN there are An -measurable matrices M that 0n + ∆Mn (i) Mn = M (ii) lim ∆Mn = 0 n→∞

a.s.

2 0n H(x) u ≥ α u2 (iii) uT M β2

for x ∈ D and u ∈ IRν

Proof. Since H0 ⊂ H⊕ and H0 ∈ H, Lemmas 6.21(a) (ii) and (b) (iii) for n = 1, 2, . . . yields (0) α0 −1 , if Hn ∈ H0 , Mn ≤ −1 α , else hence the ﬁrst assertion. Furthermore, according to Lemma 6.20 it is - n(0) −→ 0 a.s., ∆Hn := Hn(0) − H

(i)

- n ∈ H given by (6.83a), (6.83b). Because of H - n ∈ H for each with matrix H n ∈ IN, Lemma 6.21(b) (iii) yields 1 - (0) )−1 - n(0) is regular and (ii) H ≤ . (H n α (0)

(0)

224

6 Stochastic Approximation Methods with Changing Error Variances

Deﬁne now 0n := M

- n )−1 , if Hn ∈ H0 (H H0−1 , else, (0)

(0)

(6.92)

0n . Because of (i) and (ii), the assumptions of Lemma and put ∆Mn := Mn − M - n(0) and Bn := Hn(0) . Hence, C.2, Appendix, hold for An := H ∆Mn −→ 0

a.s.

- n(0) , H0 ∈ H Furthermore, according to Lemma 6.21(b) (vi), and because of H we get 2 0n H(x) u ≥ α u2 uT M 2 β for all n ∈ IN, x ∈ D and u ∈ IRν .

Due to Lemma 6.22, sequence (Mn ) fulﬁlls condition (U4a)–(U4d) from Sect. 6.3. Consequently, we have now this result: Theorem 6.9. (a) Xn −→ x∗ a.s., - n(0) −→ H ∗ := H(x∗ ) (b) Hn(0) , H

a.s.

Proof. The ﬁrst assertion follows immediately from Corollary 6.1. The second assertion follows then from (a), Theorem 6.6(b) and Theorem 6.7.

Supposing still

◦

H ∗ ∈ H0 := interior of H0 ,

(X5)

we get the main result of this section: Theorem 6.10. (a) Mn −→ (H ∗ )−1 a.s.

7 1 . (b) nλ (Xn − x∗ ) −→ 0 a.s. for each λ ∈ 0, 2 (c) For each 0 > 0 there is a set Ω0 ∈ A such that ¯0 ) < 0 (i) P(Ω κ 2 (ii) lim sup EΩ0 nXn − x∗ 2 ≤ (H ∗ )−1 qk σk2 , n∈IN

k=1

2 for k = 1, . . . , n where σk2 := lim sup E σ0,n n∈IN(k)

(d) If √the covariance matrices of (Zn ) fulﬁll (V6) and (V8) in Sect. 6.5.3, then n(Xn − x∗ ) has an asymptotic normal distribution with the asymptotic covariance matrix given by κ T qk W (k) (H ∗ )−1 . V = (H ∗ )−1 k=1

6.7 Realization of Adaptive Step Sizes

225

Proof. According to Theorem 6.9(b) and assumption (X5) there is a (stochas(0) tic) index n0 ∈ IN such that Hn ∈ H0 for all n ≥ n0 a.s.. Hence, according to the deﬁnition, for n ≥ n0 we get Mn = (Hn(0) )−1

0n = (H - (0) )−1 and M n

a.s.

Theorem 6.9(b) yields then 0n −→ Θ := (H ∗ )−1 Mn , M

a.s.

(i)

Because of Lemma 6.9 and (i) we suppose that an ≥ a = 1 a := lim inf -

(ii)

n∈IN

for sequence (an ) arising in (6.7). Assertion (b) follows by applying Theorem 6.1. Moreover, assertion (c) follows from (i), (ii) and Theorem 6.2. Because of (i), assertion (d) is obtained directly from Corollary 6.4.

Practical Computation of (Hn(0) )−1 In formula (6.91) deﬁning matrices Mn we have to compute (Hn )−1 . Accord(0) (l ) (0) ing to (6.81a), (6.81b), matrix Hn+1 := Hn n follows from Hn by an ln -fold (0) rank-1-update. Also the inverse of Hn can be obtain by rank-1-updates. (0)

(i−1)

Lemma 6.23. Let n ∈ IN and i ∈ {1, . . . , ln } with a regular Hn

. Then

(i)

δn (i−1) −1 (i) with δn(i) := n − 1 + d(i)T ) hn . n (Hn n (b) Hn(i) is regular if and only if δn(i) = 0. In this case 1 (i−1) −1 (i) (i−1) −1 d(i)T (Hn(i) )−1 = (Hn(i−1) )−1 + (i) d(i) − (H ) h ) . n n n n (Hn δn

(a) det(Hn(i) ) = det(Hn(i−1) )

Proof. The assertion follows directly from Lemma C.3, Appendix, if there we put 1 (i) (h − Hn(i−1) d(i) d := d(i) n , e := n ) n n (i)

and take into account that dn 2 = 1.

Of course, at stage n matrix (Hn+1 )−1 follows from (Hn )−1 at minimum expenses for ln = 1. Indeed, according to (6.81a), (6.81b) we have (0)

(0)

Hn+1 = Hn(0) +

(0)

1 (1) (1)T (h − Hn(0) d(1) . n )dn n n

226

6 Stochastic Approximation Methods with Changing Error Variances

Choice of Set H0 We still have to clarify how to select the set H0 occurring in (6.91) such that conditions (X4) and (X5) are fulﬁlled. Theoretically, H0 := H⊕ could be taken. However, this is not possible since the minimum eigenvalue of a matrix HH T cannot be computed exactly in general. Hence, for the choice of H0 , two practicable cases are presented: Example 6.1. Let denote · 0 any simple matrix norm with I0 = 1. Furthermore, deﬁne & % H0 := H ∈ IM(ν × ν, IR) : det(H) = 0 and H −1 0 ≤ b0 , where b0 is a positive number such that ∗ −1 (H ) < b0 . 0

(6.93)

For this set H0 we have the following result: Lemma 6.24. 1 and κ0 such that κ 0 b0 A ≤ κ0 A0 for each matrix A ∈ IM(ν × ν, IR).

(a) H0 ⊂ H⊕ with α0 := ◦

(b) H ∗ ∈ H0 . Proof. Each matrix H ∈ H0 is regular, and according to the above assumption we have 1 . H −1 ≤ κ0 H −1 0 ≤ κ0 b0 =: α0 Thus, Lemma 6.21(a) yields H ∈ H⊕ . Deﬁning a0 := (H ∗ )−1 0 and 0 := a0 1 b0 (1 − b0 ) > 0, formula (6.93) yields 0<

a0 ≤ 1 − a0 0 b0

and

0 <

1 . a0

If H − H ∗ 0 ≤ 0 , then according to Lemma C.1, Appendix, we know that H is regular and a0 ≤ b0 . H −1 0 ≤ 1 − a0 0 ◦ & %

Hence, H: H − H ∗ 0 ≤ 0 ⊂ H0 , or H ∗ ∈ H0 . In the above, the following norms · 0 can be selected: (a) A0 := max | ai,k | with κ0 := ν i,k≤ν ; < 1 1 2 T tr(A A) = ai,k , (b) A0 := ν ν i,k≤ν

where κ0 :=

√

ν

6.7 Realization of Adaptive Step Sizes

227

(0)

By means of Lemma 6.23 the determinant and the inverse of Hn can be (0) computed easily. Hence, the condition “Hn ∈ H0 ” arising in (6.91) can be veriﬁed easily. Example 6.2. For a given positive number β0 with β0 <

(det(H ∗ ))2 , (tr(H ∗ H ∗T ))ν−1

(6.94)

$ (det(H))2 ≥ β 0 . According to (tr(HH T ))ν−1 (0) Lemma 6.23 we easily can decide whether Hn ∈ H0 is fulﬁlled. Also in this case H0 fulﬁlls conditions (X4) and (X5):

deﬁne H0

H ∈ IM(ν × ν, IR) :

:=

Lemma 6.25. If ν > 1, then (a) H0 ⊂ H⊕ (b) H ∗ ∈

with

α0 :=

β0 (ν − 1)ν−1 > 0

◦ H0

Proof. If for given matrix H ∈ H0 Lemma C.4, Appendix, is applied to A := HH T , then

ν−1 ν−1 I ≥ β0 (ν − 1)ν−1 I =: α02 I, HH T ≥ det(HH T ) tr(HH T ) hence, H ∈ H⊕ . Since det(·) and tr(·) are continuous functions on IM(ν ×ν, IR), because of (6.94) there is a positive number 0 such that (det(H))2 ≥ β0 (tr(HH T ))ν−1 for each matrix H with H − H ∗ ≤ 0 . Consequently, also in the present case ◦

we have H ∗ ∈ H0 . 6.7.2 Adaptive Scalar Step Size We consider now algorithm (6.2a), (6.2b) with a scalar step size Rn = rn I = ρn Mn where rn ∈ IR+ , and ρn fulﬁlls (X1). Moreover, we suppose that Mn is deﬁned by πn (6.95) Mn = ∗ I for n = 1, 2, . . . . αn Here, πn and αn∗ are positive An -measurable random variables such that πn controls the variance of the estimation error Zn , and αn∗ takes into account the inﬂuence of the Jacobian H(x). In the following πn and αn are selected such that the optimal limit condition (5.83) is fulﬁlled. In this section we ﬁrst assume that the Jacobian H(x) and the random numbers πn and αn∗ fulﬁll the following conditions:

228

6 Stochastic Approximation Methods with Changing Error Variances

(a) Suppose that there is a positive number α such that H(x) + H(x)T ≥ 2αI

for x ∈ D.

(X6)

(b) There are positive numbers d and d¯ with d≤

πn ≤ d¯ a.s. αn∗

for all n = 1, 2, . . . .

(X7)

Corresponding to Lemma 6.22, here we have this result: Lemma 6.26. ¯ (i) sup Mn ≤ d, n

(ii) uT Mn H(x) u ≥ dαu2 for each x ∈ D, u ∈ IRν and n ∈ IN. Proof. According to the deﬁnition of Mn , the assertions (i), (ii) follow immediately from the above assumptions (a), (b).

0n :≡ Mn and ∆Mn :≡ 0. According to Lemma 6.26, also in the Put now M present case condition (U4a)–(U4d) from Sect. 5.3 is fulﬁlled. Thus, corresponding to the last section, cf. Theorem 6.9, we obtain the following result: Theorem 6.11. (a) Xn −→ x∗ a.s., (b) Hn(0) −→ H ∗ a.s. Further convergence results are obtained under additional assumptions on the random parameters πn and αn∗ : (c) There is a positive number α - with lim αn∗ = α -

n→∞

a.s.,

(X8)

α - < 2α∗ := λmin (H ∗ + H ∗T ).

(X9)

(d) There are positive numbers π (1) , . . . , π (κ) such that lim πn = π (k) a.s., k = 1, . . . , κ,

n∈IN(k)

q1 π (1) + . . . + qκ π (κ) = 1.

(X10a) (X10b)

Because of (c) and (d), for k = 1, . . . , κ we have lim

n∈IN(k)

π (k) πn =: d(k) = ∗ αn α -

a.s.,

(6.96a)

1 , α -

(6.96b)

d := q1 d(1) + . . . + qκ d(κ) = dα∗ >

1 , 2

(6.96c)

6.7 Realization of Adaptive Step Sizes

229

especially lim Mn = d(k) I

a.s.

n∈IN(k)

(6.97)

Results on the rate of convergence and the convergence in the quadratic mean of (Xn ) are obtained now: Theorem 6.12. (a) If λ ∈ [0, 12 ), then nλ (Xn − x∗ ) −→ 0 a.s. (b) For 0 > 0 there is a set Ω0 ∈ A such that ¯0 ) < 0 , (i) P(Ω

κ ' ∗ 2

(ii) lim sup EΩ0 nXn − x

≤

n∈IN

qk (π (k) σk )2

k=1

α -(2α∗ − α -)

,

(6.98)

where σ1 , . . . , σκ are selected as in Theorem 6.10(c). Proof. Because of (6.97) and Lemma 6.9 we may assume that a(k) ≥ d(k) α∗ ,

k = 1, . . . , κ.

Hence, due to (6.96b), (6.96c) we have a ≥ dα∗ > 12 . The ﬁrst assertion follows then from Theorem 6.1. Because of (6.96a) and (6.97), assertion (b) is obtained from Corollary 6.3.

As already mentioned in Sect. 6.5.2, the right hand side of (6.98) takes a minimum for α - = α∗ , π (k)

(6.99a)

q1 qκ = (σk2 ( 2 + . . . + 2 ))−1 , σ1 σκ

k = 1, . . . , κ.

(6.99b)

Corresponding to Theorem 6.10(d), also in the present case we have a central limit theorem: Theorem 6.13. In addition to the above assumptions, suppose ˚ (i) x∗ ∈ D (ii) The covariance matrices of (Zn ) fulﬁll conditions (V6) and (V8) in Sect. 6.5.3 √ Then n(Xn − x∗ ) is asymptotically N (0, V )-distributed, where the matrix V is determined by the equation κ 1 1 ∗ (H V + V H ∗T ) − V = 2 qk (π (k) )2 W (k) . α α k=1

(6.100)

230

6 Stochastic Approximation Methods with Changing Error Variances

Proof. For k = 1, . . . , κ we have d(k) (H ∗ + H ∗ )T ≥ 2(d(k) α∗ )I, and because of conditions (6.96b), (6.96c) we obtain a := q1 (d(1) α∗ ) + . . . + qκ (d(κ) α∗ ) = dα∗ >

1 . 2

Hence, according to (6.96a) and (6.97), conditions (V2) and (V3a), (V3b) in Sect. 6.5 hold. The assertion follows now from Theorem 6.4.

Choice of Sequence (πn ) Of course, the standard deﬁnition πn :≡ 1,

n = 1, 2, . . . ,

(6.101)

fulﬁlls condition (X10), but does not yield the minimum asymptotic bound of (EΩ0 nXn − x∗ 2 ). Using the assumptions of Sect. 6.6.3 concerning the estimation error (Zn ), (1) (κ) the An -measurable random variables wn , . . . , wn , given by (6.88a), (6.88b), are approximations of σ12 , . . . , σκ2 . Indeed, according to Corollary 6.5 we have lim wn(k) = lim En ∆Zn 2 a.s.,

n∈IN(k)

n∈IN(k)

k = 1, . . . , κ.

(6.102)

¯ with For given positive numbers w and w w ≤ min σk2 ≤ max σk2 ≤ w, ¯ k≤κ

(X11)

k≤κ

for n ∈ IN and k = 1, . . . , κ we deﬁne ⎧ (k) (k) ⎪ ¯ ⎨ wn , for wn ∈ [w, w] (k) (k) . w -n := w , for wn < w ⎪ ⎩ (k) w ¯ , for wn > w ¯

(6.103)

Then, for indices n ∈ IN(k) with k ∈ {1, . . . , κ} we consider

−1 q1 qκ (k) πn := w -n + . . . + (κ) . (1) w -n w -n

(6.104)

Because of (6.102) and (X11), for each n ∈ IN and k = 1, . . . , κ we ﬁnd w w ¯ ≤ πn ≤ , w ¯ w

−1 q1 qκ + . . . + lim πn = σk2 σ12 σκ2 n∈IN(k)

(6.105a) a.s.

(6.105b)

Hence, condition (X10a), (X10b) holds, and because of the optimality condition (6.99b), this choice of (πn ) yields the minimum asymptotic bound for EΩ0 nXn − x∗ 2 .

6.7 Realization of Adaptive Step Sizes

231

Choice of Sequence (α∗n ) Obviously, at stage n the factor αn∗ in the step size (6.95) may be chosen as a (0) function of Hn . Hence, for n = 1, 2, . . . we put (0) (0) (0) ¯ ϑn (Hn ), if Hn ∈ H1 and ϑn (Hn ) ∈ [ϑ, ϑ] (6.106) αn∗ := , else ϑ0 Here, H1 is an appropriate subset of IM(ν×ν, IR), ϑn (H) is a real value function ¯ on H1 , and ϑ0 , ϑ, ϑ¯ denote positive constants with ϑ0 ∈ [ϑ, ϑ]. If sequence (πn ) is chosen according to (6.104), then (X7) holds with d :=

w w ¯ · ϑ¯

and d¯ :=

w ¯ . w·ϑ

(6.107)

Consequently, without any further assumptions, the convergence properties according to Theorem 6.11 hold. Now we still require the following: (e) Suppose

◦

H ∗ ∈ H1 = interior of H1 . (f) In case

(0) “Hn

(X12)

−→ H ∗ a.s.” we assume that there is number α - such that ¯ - < ϑ, ϑ<α

(X13)

ϑn (Hn(0) ) −→ α -

a.s.

(X14)

In the important case (see the following examples) ϑn (H) ≡ ϑ(H) is continuous on

H1 ,

condition (X14) holds with α - = ϑ(H ∗ ). In the following we show that (X12), (X13) and (X14) imply condition (X8): Lemma 6.27. lim αn∗ = α

a.s.

n→∞

Proof. According to Theorem 6.11(b) Hn(0) −→ H ∗

a.s.

(i)

Thus, because of (X12), (X13) and (X14) there is an index n0 such that Hn(0) ∈ H1

¯ and ϑn (Hn(0) ) ∈ [ϑ, ϑ]

for all n ≥ n0 a.s. By deﬁnition for n ≥ n0 we have αn∗ = ϑn (Hn(0) ) a.s. - a.s. Hence, because of (i) and (X14) we get lim αn∗ = α n→∞

232

6 Stochastic Approximation Methods with Changing Error Variances

Examples for ϑn (H) We consider now several examples for the function ϑn (H) to be needed in (6.106) fulﬁlling conditions (X14) and (X9). In these cases the convergence properties of Theorem 6.12 and 6.13 hold. According to (X6) we have ◦

H ∗ ∈ H+ ,

(6.108)

provided that H+ := {H ∈ IM(ν × ν, IR) : H + H > 0}. In the following examples we take H1 := H+ . Hence, condition (X12) holds. T

Example 6.3. The function ϑ(1) (H) :=

1 λmin (H + H T ) 2

is continuous on H+ and ϑ(1) (H ∗ ) = α∗ . Therefore, ϑn (H) :≡ ϑ(1) (H) is an appropriate choice still guaranteeing the optimality condition (6.99a). Example 6.4. ϑ(2) (H) := (tr((H +H T )−1 ))−1 is also continuous on H+ , where ϑ(2) (H) <

1 = λmin (H + H T ) λmax ((H + H T )−1 )

for each matrix H ∈ H+ , especially ϑ(2) (H ∗ ) < 2α∗ . Hence, also ϑn (H) :≡ ϑ(2) (H) is a possible approach. Example 6.5. According to C.4, Appendix, on H+ the inequality ϑ(3) (H) < λmin (H + H T ) holds with the continuous function

ν−1 ν−1 . ϑ(3) (H) := det(H + H T ) · 2tr(H) Thus, also ϑn (H) :≡ ϑ(3) (H) is possible. Example 6.6. For given An -measurable random vectors un from IRν such that un = 1, n = 1, 2, . . . , for H ∈ H+ we deﬁne −1 T T −1 un . ϑ(4) n (H) := 2un (H + H ) If un is an approximate eigenvector to the maximum eigenvalue (2α∗ )−1 of (4) the matrix (H ∗ + H ∗T )−1 , then ϑn (H ∗ ) is an approximation of α∗ . ◦

Since H ∗ ∈ H+ , according to Theorem 6.11 there is a stochastic index n0 such that (6.109) Hn(0) ∈ H+ , n ≥ n0 a.s., and

(Hn(0) + Hn(0)T )−1 −→ (H ∗ + H ∗T )−1

a.s.

(6.110)

Regarding to the v. Mises-algorithm for the computation of an eigenvector related to the maximum eigenvalue of (H ∗ + H ∗T )−1 , the sequence of vectors (un ) is deﬁned recursively as follows (see Appendix C.2):

6.7 Realization of Adaptive Step Sizes

233

(a) Select a vector 0 = v0 ∈ IRν and put u1 := v0 −1 v0 . (b) For a given An -measurable vector un at stage n, deﬁne (0) (0)T (0) vn (Hn + Hn )−1 un , if Hn ∈ H+ . , un+1 := vn := vn v0 , else From (6.109), with probability one for n ≥ n0 we get 1 vn , un+1 = . 2 < un , vn > vn

(0) vn = (Hn(0) + Hn(0)T )−1 un , ϑ(4) n (Hn ) =

If the matrices A := (H ∗ + H ∗T )−1 and An := (Hn + Hn )−1 fulﬁll a.s. the assumptions in Lemma C.5(b), cf. Appendix, then due to (6.110) and (4) (0) Lemma C.5 sequence ϑn (Hn ) converges a.s. towards (0)

(0)T

1 = α∗ . 2λmax (H ∗ + H ∗T )−1 Therefore, ϑn (H) :≡ ϑn (H) fulﬁlls (X14) with α - = α∗ . Hence, also (X9) holds in this case. (4)

Some Remarks on the Computation of ϑn (H) If sequence (αn∗ ) in (6.95) is chosen according to the above examples, in each iteration step condition Hn(0) ∈ H+ ⇐⇒ Hn(0) + Hn(0)T > 0 (0)

(0)T

must be veriﬁed. This holds if and only if (see [142]) the matrix Hn + Hn has a Cholesky-decomposition. Hence, there is a matrix Sn such that Hn(0) + Hn(0)T = Sn SnT , ⎛ (n) ⎞ 0 s11 ⎜ ⎟ .. ⎟, Sn = ⎜ . ⎝ ⎠ ∗ (n)

(6.111a)

(6.111b)

(n)

sνν

s11 , . . . , s(n) νν > 0.

(6.111c)

In [142] an algorithm for the decomposition (6.111a)–(6.111c) is presented. If (0) this algorithm stops without result, then Hn ∈ H+ . The decomposition (6.111a)–(6.111c) is also useful for the computation of (0) ϑn (Hn ). This is shown in the following lemma:

234

6 Stochastic Approximation Methods with Changing Error Variances (0)

Lemma 6.28. For Hn ∈ H+ , let Sn be given by (6.111b). Then (n)

2 (a) det(Hn(0) + Hn(0)T ) = (s11 · . . . · s(n) νν )

(b) (Hn(0) + Hn(0)T )−1 = (Sn−1 )T Sn−1 , where ⎛ ⎞ (n) 0 t11 ⎜ ⎟ .. ⎟ Sn−1 = ⎜ . ⎝ ⎠ (n) ∗ tνν (n) (tij )2 (c) tr((Hn(0) + Hn(0)T )−1 ) = i≥j

(d) For given un ∈ IR determine vn := (Hn(0) + Hn(0)T )−1 un as follows (i) ﬁnd wn with Sn wn = un ν

(ii) determine vn such that

SnT vn = wn

Symmetric H ∗ If (e.g., in case of a gradient G(x) = ∇F (x)): H ∗ := H(x∗ ) is symmetric,

(X15)

then

1 λmin (H ∗ + H ∗T ) = λmin (H ∗ ). (6.112) 2 Hence, in the present case the function ϑn (H) in (6.106) can be chosen to depend only on H. In addition, conditions (X14) and (X9) can be fulﬁlled. Due to (X6) we have α∗ :=

◦

H ∗ ∈ H++ ,

(6.113)

provided that

% & H++ := H ∈ IM(ν × ν, IR) : det(H) > 0 and tr(H) > 0 .

In the following examples we take H1 := H++ , hence, condition (X12) holds. Example 6.7. (see Example 6.4). The function ϑ(5) (H) := is continuous on H++ and ϑ(5) (H ∗ ) <

2 tr(H −1 ) 2 λmax ((H ∗ )−1 )

= 2α∗ . Thus, ϑn (H) :≡

ϑ(5) (H) is a suitable function. Example 6.8. (see Example 6.5) ϑ(6) (H) := 2 det(H) · (

ν − 1 ν−1 ) tr(H)

is also continuous on H++ , and Lemma C.4, Appendix, yields ϑ(6) (H ∗ ) < 2α∗ . Hence, also ϑn (H) :≡ ϑ(6) (H) is an appropriate choice.

6.7 Realization of Adaptive Step Sizes

235

Example 6.9. (see Example 6.6) For H ∈ H++ deﬁne ϑ(7) n (H) :=

1 , uTn H −1 un

where u1 := v0 −1 v0 and (un ) is given recursively by (0) (0) vn (Hn )−1 un , for Hn ∈ H++ , vn := , un+1 := vn v0 , else n = 1, 2, . . . . Here, 0 = v0 ∈ IRν is an appropriate start vector. According to C.5, Appendix, in many cases the following equation holds: (0) lim ϑ(7) n (Hn ) =

n→∞

1 = α∗ a.s. lim < un , vn >

n→∞ (7)

Thus, also ϑn (H) :≡ ϑn (H) is a suitable function. Remark 6.2. Computation of ϑn (H): In the above Examples 6.7–6.9 the expressions tr (Hn(0) )−1 , det(Hn(0) ), (Hn(0) )−1 must be determined. This can be done very eﬃciently by means of Lemma 6.23. Non Optimal ϑn (H) (0)

The quantities ϑn (Hn ) in the Examples 6.3–6.9 can be determined only with larger expenses, since these examples were chosen such that condition (X9) holds. If this is not demanded, then simpler functions ϑn (H) can be taken into account. However, in this case the convergence properties in Theorems 6.12 and 6.13 can not be guaranteed. On the other hand, if all conditions hold, expect (X9), then we have this result: Theorem 6.14. It is nλ (Xn − x∗ ) −→ 0 a.s. for each λ such that $ 1 α∗ , . 0 ≤ λ < min 2 α Proof. We may assume (see the proof of Theorem 6.12) that a ≥ α∗ α -−1 . The assertion follows then from Theorem 6.1. According to Sect. 6.5.2, it would be favorable if the limit α - in (X14) fulﬁlls α - ≈ α∗ . This condition helps to ﬁnd appropriate functions ϑn (H). Among many other examples, the following one shows how such “non

optimal” functions ϑn (H) can be selected.

236

6 Stochastic Approximation Methods with Changing Error Variances

Example 6.10. The function ϑ(8) (H) := min hii is continuous on IM(ν × ν, IR), i≤ν

where hii denotes the ith diagonal element of H. Hence, deﬁne ϑn (H) := ϑ(8) (H),

H1 := IM(ν × ν, IR).

According to (6.106) we have (0) (0) ¯ ϑ(8) (Hn ), for ϑ(8) (Hn ) ∈ [ϑ, ϑ] ∗ , αn = , else ϑ0 ¯ we get and in case α - := min h∗ii ∈ (ϑ, ϑ) i≤ν

lim αn∗ = α

n→∞

a.s.

Here, inequality α - ≥ α∗ , holds, but condition (X9) does not hold in general. Remark 6.3. Reduction of the expenses in the computation of (αn∗ ). Formula (6.106) for sequence (αn∗ ) can be modiﬁed such that αn∗ is changed only at stages n ∈ IM. Here, IM is an arbitrary inﬁnite subset of IN. The advantage of (0) this procedure is that the time consuming Cholesky-decomposition of Hn + (0)T must be calculated only for “some” indices n ∈ IM. Hn Using a sequence (αn∗ ) as described above, the above convergence properties hold true.

6.8 A Special Class of Adaptive Scalar Step Sizes In contrary to the step size rule considered in Sect. 6.7.2, in the present section scalar step sizes of the type Rn = rn I

with

rn ∈ IR+ ,

(6.114a)

are studied for algorithm (6.2a), (6.2b), where the An -measurable random variables rn are deﬁned recursively as follows: For n = 1 we select positive A1 -measurable random variables r1 , w1 , and for n > 1 we put wn−1 rn−1 ) Kn . (6.114b) rn = r-n−1 Qn (wn Here, rn , rn−1 }, r-n−1 := min{¯ 1 if r ∈ [0, r¯n ], Qn (r) = 1 + r vn (r)

(6.114c) (6.114d)

with positive An -measurable random variables r¯n , wn , Kn and vn (r), r ∈ [0, r¯n ]. We assume that the following a priori assumptions hold:

6.8 A Special Class of Adaptive Scalar Step Sizes

237

(A1) For each n > 1 and r ∈ [0, r¯n ] suppose r¯0 ≤ inf r¯n ≤ ∞ n

a.s.,

(Y1)

v ≤ vn (r) ≤ v¯ a.s.,

(Y2)

w ≤ wn ≤ w ¯

(Y3)

a.s.,

lim (K2 · . . . · Kn ) = P ∗ ∈ (0, ∞)

n→∞

a.s.

(Y4)

with positive numbers r¯0 , v , v¯ , w , w ¯ and a random variable P ∗ . As can be seen from the following example, the deﬁnition (6.114a)–(6.114d) is motivated by the standard step size rn = nd . Example 6.11. Obviously, rn = rn =

d n

can be represented by

rn−1 , n > 1. 1 + rn−1 d1

However, this corresponds to (6.114a)–(6.114d) with r¯n := ∞, vn (r) := d1 and wn = Kn := 1. In (6.114a)–(6.114d), the factor Qn (r) guarantees the convergence of rn to 0, the variance of the estimation error Zn is taken into consideration by wn , and the starting behavior of rn can be inﬂuenced by Kn . Concrete examples for Qn (r), wn and Kn are given later on. Next to the convergence behavior of (rn ) and (Xn ) is studied. We suppose here that the step directions Yn in algorithm (6.2a), (6.2b) are given by (W1a)–(W1e) in Sect. 6.6. 6.8.1 Convergence Properties The following simple lemma is a key tool for the analysis of the step size rule (6.114a)–(6.114d). Lemma 6.29. Let t1 , u, u2 , u3 , . . . be given positive numbers. t (a) The function ϕ(t) := 1+tu is monotonous increasing on [0, ∞) and ϕ(t) ∈ [0, t) for each t > 0. , n = 2, 3, . . . , (b) If the sequence (tn ) is deﬁned recursively by tn := 1+ttn−1 n−1 un t1 then tn = 1+t1 (u2 +...+un ) , n = 2, 3, . . . .

Proof. The ﬁrst part is clear, and the formula in (b) follows by induction with respect to n. We show now that the step size rn according to (6.114a)–(6.114d) is of

order n1 .

238

6 Stochastic Approximation Methods with Changing Error Variances

Lemma 6.30. limextr n · rn ∈ n

7

w w ¯v ¯

, ww¯v

8 a.s.

Proof. Putting r-n wn Pn−1 , un := vn (rn−1 ), Pn wn−1

Pn := K2 · . . . · Kn , tn :=

after multiplication with wn Pn−1 , deﬁnition (6.114a)–(6.114d) yields tn−1 , n = 2, 3, . . . . 1 + tn−1 un

tn ≤

Hence, (Y2) and (Y3) Because of Lemma 6.29 we get tn ≤ yield −1 P Pn Pn v n ≤ ntn ≤ nt1 1 + t1 (P1 + . . . + Pn−1 ) nrn = ntn wn w w ¯ w

−1 1 v Pn (P1 + . . . + Pn−1 ) = + a.s. nt1 w ¯n w t1 1+t1 (u2 +...+un ) .

Therefore, by (Y4) and Lemma A.8, Appendix, we have lim sup n · r-n ≤ n

w ¯ wv

a.s.,

(i)

and (rn ) converges to zero a.s. Because of (Y1) there exists a stochastic index n0 with (ii) rn = r-n , n ≥ n0 a.s. From (i), (ii) and with (6.114a)–(6.114d) we get lim sup n · rn ≤ n

tn =

w ¯ wv

a.s.,

tn−1 , n ≥ n0 1 + tn−1 un

a.s.

Thus, for n ≥ n0 , Lemma 6.29 yields tn =

tn0 1 + tn0 (un0 +1 + . . . + un )

a.s.

(6.115)

Applying now (ii), (Y2) and (Y3), we get

−1 Pn v¯ Pn nrn = ntn ≥ ntn0 1 + tn0 (Pn0 + . . . + Pn−1 ) wn w w ¯ =

−1 v¯ 1 Pn (Pn0 + . . . + Pn−1 ) + ntn0 wn w ¯

a.s.

6.8 A Special Class of Adaptive Scalar Step Sizes

239

Finally, condition (Y4) and Lemma A.8, Appendix, yield lim inf (n · rn ) ≥ n

w v¯ w ¯

a.s.

Obviously, the step size Rn in (6.114a) can be represented by Rn = ρn Mn , where 1 and Mn := (nrn ) I . (6.116) ρn := n Concerning the Jacobian H(x) of G(x) we demand as in Sect. 6.7.2 (cf. condition (X6)): (A2) There is a positive number α such that H(x) + H(x)T ≥ 2α I

for all x ∈ D.

(Y5)

The sequence (Mn ) fulﬁlls condition (U4a)–(U4d) from Sect. 6.3: Lemma 6.31. (a) sup Mn < ∞ a.s. n

0n and ∆Mn such that (b) For each n ∈ IN there are An -measurable matrices M 0n + ∆Mn (i) Mn = M (ii)

lim ∆Mn = 0

n→∞

a.s.

0n H(x) u ≥ α w u2 for each x ∈ D and u ∈ IRν (iii) uT M w ¯ v¯ Proof. For n ∈ IN deﬁne 0n := max nrn , w I M w ¯ v¯

and

0n . ∆Mn := Mn − M

(6.117)

Because of condition (Y5), the assertions follows then immediately from Lemma 6.30.

Because of Lemma 6.31, corresponding to Theorems 6.9 and 6.11, here we have the following result: Theorem 6.15. (a) Xn −→ x∗ a.s. (b) Hn(0) −→ H ∗ a.s. In order to derive further convergence properties, one needs again additional a posteriori assumption on the random functions vn (r) and random quantities wn : ¯ such that for all n ∈ IN (A3) There is a constant L ¯r | vn (r) − vn (0) |≤ L

for r ∈ [0, r¯n ] a.s.

(Y6)

240

6 Stochastic Approximation Methods with Changing Error Variances

(A4) If “Xn −→ x∗ a.s.,” then there are positive numbers α - and w(1) , . . . , w(κ) with lim vn (0) = α -

n→∞

lim wn = w(k)

n∈IN(k)

a.s., a.s.,

(Y7) k = 1, . . . , κ.

(Y8)

Now, a stronger form of Lemma 6.30 can be given: Lemma 6.32. For each k ∈ {1, . . . , κ} we have q qκ −1 1 lim n · rn = α -w(k) + . . . + w(1) w(κ) n∈IN(k)

a.s.

Proof. According to (6.115) and (ii) from the proof of Lemma 6.30 we obtain, for n ≥ n0 , ntn0 Pn Pn = wn 1 + tn0 (un0 +1 + . . . + un ) wn

−1 1 1 Pn = + (un0 +1 + . . . + un ) . ntn0 n wn

nrn = ntn

(i)

Because of Lemma 6.30, (rn ) converges to zero a.s., and by Theorem 6.15 we have Xn −→ x∗ a.s. Hence, (Y4), (Y6), (Y7) and (Y8) yield

∗ ∗ (k) P lim (Pn , wn , un ) = P , w , (k) α a.s. (ii) w n∈IN(k) Having (i) and (ii), with Lemma A.8, Appendix, we obtain q qκ −1 P ∗ 1 lim nrn = P ∗ α + . . . + (κ) (1) w w w(k) n∈IN(k)

a.s.

According to Lemma 6.32 we have now lim Mn = d(k) I a.s., k = 1, . . . , κ ,

n∈IN(k)

d := q1 d(1) + . . . + qκ d(κ) =

1 , α -

(6.118) (6.119)

where

q qκ −1 1 d(k) := α -w(k) + . . . + , k = 1, . . . , κ (6.120) w(1) w(κ) Hence, the constant α - has the same meaning as in Sect. 6.7.2, see (6.97) and (6.96b). Thus, corresponding to condition (X9), here we demand: α - < 2α∗ := λmin (H ∗ + H ∗T ).

Corresponding to Theorem 6.12, here we have the following result:

(Y9)

6.8 A Special Class of Adaptive Scalar Step Sizes

241

Theorem 6.16. (a) If λ ∈ [0, 12 ), then nλ (Xn − x∗ ) −→ 0 a.s. (b) For each 0 > 0 there is a set Ω0 ∈ A such that ¯0 ) < 0 , (i) P(Ω (ii) lim sup EΩ0 nXn − x∗ 2 n

:−1 κ σ 2 q qκ 2 1 k ≤ α -(2α − α -) + . . . + q . (6.121) k (k) w(1) w(κ) w k=1 9

∗

Proof. Because of (6.119), (6.120) and (Y9), as in the proof of Theorem 6.12, for the number α -, d according to (6.9), (6.120), resp., we have α - ≥ dα∗ >

1 . 2

Based on (6.118)–(6.120), the assertions follow now from Theorem 6.1 and Corollary 6.3.

According to Sect. 6.5.2, the right hand side of (6.121) takes a minimum at the values α - = α∗ , 2 , k = 1, . . . , κ. w = σk2 := lim sup E σ0,n (k)

(6.122) (6.123)

n∈IN(k)

Also the limit properties in Theorem 6.13 can be transferred easily to the present situation: Theorem 6.17. Under the additional assumptions ◦

(i) x∗ ∈ D (ii) The covariance matrices of (Zn ) fulﬁll conditions (V6), (V8) in Sect. 6.5.3 √ sequence ( n(Xn − x∗ )) has an asymptotic N (0, V )-distribution, where the matrix V is determined by the equation q qκ −2 qk 1 ∗ 1 (H V + V H ∗T ) − V = α + . . . + W (k) . (k) )2 α w(1) w(κ) (w k=1 (6.124) κ

Similar to Theorem 6.13, the above result can be shown by using Theorem 6.4, where the numbers d(k) are not given by (6.96a), but by (6.118). 6.8.2 Examples for the Function Qn (r) Several formulas are presented now for the function Qn (r) in (6.114a)–(6.114d) such that conditions (Y1), (Y2) and (Y6) for the sequences (¯ rn ) and (vn (r))

242

6 Stochastic Approximation Methods with Changing Error Variances

hold true. In each case the function Qn (r) depends on two An -measurable random parameters αn∗ and Γn . Suppose that αn∗ , Γn fulﬁll the following condition: (A5) There are positive constants ϑ, ϑ¯ and γ¯ such that for n > 1 ϑ ≤ αn∗ ≤ ϑ¯ a.s., | Γn |≤ γ¯ a.s.

(Y10) (Y11)

Example 6.12. Standard step size Corresponding to Example 6.11 we take 1 , r ∈ [0, r¯n ] 1 + rαn∗ − r2 Γn $ ¯ q¯ K ∗ r¯n := αn min . , Γn − Γn + Qn (r) =

(6.125a) (6.125b)

¯ > 0 and q¯ ∈ (0, 1) are arbitrary constants. The following lemma Here, K shows the feasibility of (6.125a), (6.125b): Lemma 6.33. According to (6.125a), (6.125b) function Qn (r) has the form (6.114d), and for each r ∈ [0, r¯n ] we have (a) vn (r) = αn∗ − rΓn , ¯ (b) ϑ (1 − q¯) ≤ vn (r) ≤ ϑ¯ (1 + K) a.s., ϑ ¯ q¯} > 0 a.s. min{K, (c) r¯n ≥ r¯0 := γ¯ Proof. Because of (Y10), (Y11) and (6.125b), the inequality in (c) holds for each n > 1 and r ∈ [0, r¯n ]. Furthermore, ϑ (1 − q¯) ≤ αn∗ (1 − q¯) ≤ αn∗ − r¯n Γn + ≤ αn∗ − rΓn ¯ ≤ ϑ¯ (1 + K) ¯ a.s., ≤ αn∗ + r¯n Γn − ≤ αn∗ (1 + K) hence, also (b) holds.

Example 6.13. “Optimal” step size If En Zn = 0, n = 1, 2, . . . (cf. Robbins–Monro process), then by Lemma 6.1 and assumption (U5b) for each n ∈ IN we have 2 En ∆n+1 2 ≤ ∆n − rn (Gn − G∗ ) + rn2 En Zn 2 2 ≤ 1 − 2αrn + (β 2 + σ12 )rn2 ∆n 2 + rn2 σ0,n . Here, α and β are positive numbers with Xn − x∗ , Gn − G∗ ≥ αXn − x∗ 2 , Gn − G∗ ≤ βXn − x∗ (see (6.4)).

6.8 A Special Class of Adaptive Scalar Step Sizes

243

Based on the above estimate for the quadratic error ∆n 2 , in [89] an “optimal” deterministic step size rn is given which is deﬁned recursively as follows: rn =

1 − rn−1 α rn−1 for n = 2, 3, . . . . 2 1 − rn−1 (β 2 + σ12 )

This suggests the following function Qn (r): Let q¯ ∈ (0, 1), and for n > 1 deﬁne 1 − rαn∗ , r ∈ [0, r¯n ] 1 − r 2 Γn $ ∗ αn 1 . , r¯n := q¯ min Γn + αn∗ Qn (r) =

(6.126a) (6.126b)

The feasibility of this approach is shown next: Lemma 6.34. If Qn (r) and r¯n are deﬁned by (6.126a), (6.126b), then Qn (r) has the form (6.114d), and for each r ∈ [0, r¯n ] we have αn∗ − rΓn Γn − αn∗2 ∗ = α − r, n 1 − rαn∗ 1 − rαn∗ q¯ ϑ¯2 + γ¯ (b) (1 − q¯) ϑ ≤ vn (r) ≤ ϑ¯ + a.s., 1 − q¯ ϑ Γn − αn∗2 ϑ¯2 + γ¯ ≤ a.s., (c) 1 − rαn∗ 1 − q¯ $ ϑ 1 , > 0 a.s. (d) r¯n ≥ r¯0 := q¯ min γ¯ ϑ¯ (a) vn (r) =

Proof. Assertion (a) follows by a simple computation. If αn∗2 < Γn , then (6.126b) yields 0≤

Γn − αn∗2 Γn − αn∗2 ≤ ≤ Γn . ∗ 1 − rαn 1 − q¯ αn∗2 Γn−1

Hence, because of (a) we get αn∗ ≥ vn (r) ≥ αn∗ − rΓn ≥ αn∗ − r¯n Γn + ≥ αn∗ (1 − q¯). Let Γn ≤ αn∗2 . Then (6.126b) yields 0≤

αn∗2 − Γn αn∗2 + Γn − , ≤ 1 − rαn∗ 1 − q¯

and therefore with (a) αn∗ ≤ vn (r) ≤ αn∗ +

αn∗2 + Γn − q¯ . 1 − q¯ αn∗

From the above relations and assumptions (Y10), (Y11) we obtain now the assertions in (b), (c) and (d).

244

6 Stochastic Approximation Methods with Changing Error Variances

Example 6.14. A further approach reads Qn (r) = 1 − rαn∗ + r2 Γn , r ∈ [0, r¯n ] $ ¯ ∗ q¯ q¯αn∗ Kαn r¯n := min , , , ¯ Γn − αn∗ (1 + K) Γn +

(6.127a) (6.127b)

¯ > 0 and q¯ ∈ (0, 1) are arbitrary constants. In this case we have the where K following result: Lemma 6.35. Let Qn (r) and r¯n be given by (6.127a), (6.127b). Then function Qn (r) can be represented by (6.114d), and for each r ∈ [0, r¯n ] we have αn∗ − rΓn Γn − αn∗ (αn∗ − rΓn ) r, = αn∗ − ∗ 2 1 − rαn + r Γn 1 − r(αn∗ − rΓn ) ¯ ϑ¯ (1 + K) a.s., (b) (1 − q¯) ϑ ≤ vn (r) ≤ 1 − q¯ ¯ Γn − αn∗ (αn∗ − rΓn ) γ¯ + ϑ¯2 (1 + K) ≤ a.s., (c) ∗ 1 − r(αn − rΓn ) 1 − q¯ $ ¯ q¯ q¯ ϑ Kϑ (d) r¯n ≥ r¯0 := min , ¯ > 0 a.s. , ¯ γ¯ γ¯ ϑ (1 + K)

(a) vn (r) =

Proof. Obviously (a) holds. Using (6.127b) we ﬁnd un := αn∗ − rΓn ≥ αn∗ − r¯n Γn + ≥ αn∗ (1 − q¯), ¯ un ≤ αn∗ + r¯n Γn − ≤ αn∗ (1 + K), ∗ ¯ ≥ 1 − q¯. Qn (r) = 1 − run ≥ 1 − r¯n αn (1 + K) Hence, with (a) we get ¯ un α∗ (1 + K) , = vn (r) ≤ n αn∗ (1 − q¯) ≤ un ≤ 1 − run 1 − q¯ ¯ Γn − αn∗ un | Γn | +αn∗ un | Γn | +αn∗2 (1 + K) ≤ . 1 − run ≤ 1 − q¯ 1 − q¯ Applying now assumptions (Y10) and (Y11), the assertions in (b), (c) and (d) follow.

Example 6.15. Consider Qn (r) given by Qn (r) = exp(−rαn∗ + r2 Γn ), r ∈ [0, r¯n ] $ α∗ r¯n := min r¯, q¯ n+ . Γn

(6.128a) (6.128b)

In (6.128b) r¯ > 0 and q¯ ∈ (0, 1) are arbitrary constants. The feasibility of this approach is guaranteed by the next lemma:

6.8 A Special Class of Adaptive Scalar Step Sizes

245

Lemma 6.36. Let Qn (r) and r¯n be given by (6.128a), (6.128b). Then Qn (r) can be represented by (6.114d), where for each r ∈ [0, r¯n ] we have 1 exp(rαn∗ − r2 Γn ) − 1 = αn∗ + ∆αn∗ (r) r with r

1 exp(rαn∗ − r2 Γn ) − 1 ∗ ∆αn (r) := (αn∗ − rΓn ) − 1 − Γn , r rαn∗ − r2 Γn r¯ u ¯ exp(¯ ru ¯)) a.s., where u ¯ := ϑ¯ + r¯ γ¯ , ¯(1 + (b) ϑ (1 − q¯) ≤ vn (r) ≤ u 2 u ¯2 (c) | ∆αn∗ (r) |≤ exp(¯ ru ¯) + γ¯ a.s., 2 $ ϑ > 0 a.s. (d) r¯n ≥ r¯0 := min r¯, q¯ γ¯

(a) vn (r) =

Proof. Assertion (a) follows by an elementary calculation. According to (6.128b) we have αn∗ (1 − q¯) ≤ αn∗ − r¯n Γn + ≤ αn∗ − rΓn =: un ≤ αn∗ + r¯ | Γn | .

(i)

Furthermore, for each t > 0 we may write exp(t) = 1 + t +

t2 exp(τ (t)) with τ (t) ∈ [0, t]. 2

(ii)

Hence, with (a) we get vn (r) = un

run exp(run ) − 1 exp τ (run ) . = un 1 + run 2

Together with (i) we ﬁnd r¯un exp(¯ r un ) , αn∗ (1 − q¯) ≤ vn (r) ≤ un 1 + 2

(iii)

and (i), (ii) yield also the relation

un exp(run ) − 1 | ∆αn∗ (r) | = − 1 − Γn r run u2 u ru n n exp τ (run ) − Γn ≤ n exp(¯ run )+ | Γn | . (iv) = r 2 2 The assertion in (b), (c) and (d) follow now from (iii), (iv) and the assumptions (Y10), (Y11).

Choice of (α∗n ) In the above examples for Qn (r) condition (Y6) holds, and vn (0) = αn∗ a.s., for all n > 1.

(6.129)

246

6 Stochastic Approximation Methods with Changing Error Variances

Hence, assumption (Y7) is equivalent to lim αn∗ = α

n→∞

a.s.

(Y12)

Note that (Y12) and (Y9) are exactly the conditions which are required for (αn∗ ) in Sect. 6.7.2 (see (X8) and (X9)). Theorem 6.15 guarantees also in the present case that Xn −→ x∗ and (0) Hn −→ H ∗ a.s. Thus, sequence (αn∗ ) can be chosen also according to (6.106). Then (Y10) holds because of (X13), and due to Lemma 6.27 also conditions (Y12) and (Y9) are fulﬁlled. Selection of Sequence (Γn ) The An -measurable factors Γn in the above examples for Qn (r) can be chosen arbitrarily, provided that condition (Y11) holds. According to Lemma 6.32, Theorems 6.16 and 6.17,√(Γn ) does not inﬂuence the limit of (nrn ) and the asymptotic variance of ( n(Xn − x∗ )). Of course, the most simple choice is Γn ≡ 0, n > 1.

(6.130)

A positive Γn means that the step size rn has a slower convergence to zero than in the standard case (6.130). Example 6.13 suggest the following choice for (Γn ): 2 βn , if βn2 ≤ γ¯ for n = 2, 3, . . . , (6.131) Γn := γ0 , else ∗ where βnis an An -measurable estimate of the maximum eigenvalue β of the 1 ∗ ∗ T matrix 2 H(x ) + H(x ) , and γ0 is a constant such that | γ0 |≤ γ¯ . Since (6.132) Hn(0) −→ H ∗ := H(x∗ ) a.s.,

see Theorem 6.15(b), the following approximates βn of β ∗ can be used: (a) βn = ψ (1) (Hn(0) ) with ψ (1) (H) := tr(H), (b) βn = ψ (2) (Hn(0) ) with ψ (2) (H) := max hii , i≤ν

(6.133) (6.134)

where hii denotes the ith diagonal element of H. Because of (6.132) we have βn −→ ψ (i) (H ∗ ) a.s.,

(6.135)

where i = 1 for (6.133) and i = 2 for (6.134). The bound γ¯ in (6.131) is chosen such that (i) ∗ 2 ψ (H ) ≤ γ¯ . (6.136) Instead of (6.131) we can also take Γn := αn∗2 , n > 1.

(6.137)

In this case very simple expressions are obtained for the step size bound r¯n in Examples 6.12–6.15.

6.8 A Special Class of Adaptive Scalar Step Sizes

247

6.8.3 Optimal Sequence (wn ) The factors wn in (6.114a)–(6.114d) are deﬁned in the literature often by wn ≡ 1, n ≥ 1,

(6.138)

cf. Examples 6.11, 6.13. However, √ this approach does not yield the minimum asymptotic variance bound for n(Xn − x∗ ), see Theorem 6.16(b). (1) -n , . . . , According to Sect. 6.7.2, the An -measurable random variables w (κ) w -n , deﬁned by (6.103), fulﬁll for k = 1, . . . , κ the following conditions: (k)

-n ≤ w, ¯ w≤w

(6.139)

(k) w -n

(6.140)

lim

n∈IN(k)

= lim En Zn − En Zn 2 a.s. n∈IN(k)

Since the limits w(1) , . . . , w(κ) of (wn ) from (Y8) should fulﬁll in the best case (6.123), because of (6.140) we prefer the following choice of (wn ): For an integer n ≥ 1 belonging to the set IN(k) with k ∈ {1, . . . , κ}, put -n(k) . wn := w

(6.141)

Then, because of (6.139), (6.140) also conditions (Y3) and (Y8) hold. 6.8.4 Sequence (Kn ) According to Lemma 6.32, Theorems 6.16, 6.17, the positive An -measurable factors Kn in the step size rule (6.114b) play no √ role in the formulas for the limit of (nrn ) and the asymptotic variance of ( n(Xn − x∗ )). Selecting (Kn ), we have only to take into account the convergence condition (Y4). This requires Kn · Kn+1 · . . . ≈ 1 for “large” n. Hence, the factors Kn are responsible only for the initial behavior of the step size rn and the iterates Xn . By the factor Kn (as well as by wn ) the step size can be enlarged at rn−1 → rn . This is necessary, e.g., in case of a too small starting step size r1 . An obvious deﬁnition of Kn reads Kn ≡ 1,

n > 1.

(6.142)

However, it is preferable to take Kn > 1 as long as algorithm (6.2a), (6.2b) shows an “almost deterministic” behavior. This holds as long as the inﬂuence of the estimation error Zn−1 to the step direction Yn−1 = Gn−1 + Zn−1 is small. Based on the above considerations, some deﬁnitions for (Kn ) are presented:

248

6 Stochastic Approximation Methods with Changing Error Variances

Adaptive Selection of (Kn ) Suppose Kn = exp(s2n kn ),

n > 1,

(6.143)

where (sn ) is a deterministic sequence and kn denotes an An -measurable random number such that s2n < ∞ , (Y13) n

sup E | kn | < ∞.

(Y14)

n

Consequently, s2n | kn | < ∞ and especially E n

s2n | kn |< ∞ a.s.

n

Hence, due to (6.143), sequence (Kn ) fulﬁlls condition (Y4). E.g., (sn ) can be chosen as follows: sn =

s1 , n

n > 1,

with

s1 > 0.

(6.144)

In the following some examples are given for sequence (kn ). Example 6.16. According to assumption (a) in Sect. 6.3, the norm of the An measurable diﬀerence ∆Xn := Xn − Xn−1 is bounded by δ0 . Hence, (a) kn := ∆Xn , ∆Xn−1 , (b) kn := ∆Xn , ∆Xn−1 + , (c) kn := cos (∆Xn , ∆Xn−1 ) := (d) kn := cos (∆Xn , ∆Xn−1 )+

(6.145) (6.146) ∆Xn , ∆Xn−1 , ∆Xn ∆Xn−1

(6.147) (6.148)

fulﬁll condition (Y14) and are therefore appropriate deﬁnitions for (kn ). Example 6.17. The factors kn can be chosen also as functions of the approx(2l ) (0) imates Yn−1 , . . . , Yn−1n−1 for the step direction Yn−1 , cf. (W1a)–(W1e) in Sect. 6.6. Thereby we need the following result: Lemma 6.37. For arbitrary index n ∈ IN and j, k ∈ {0, . . . , 2ln } we have EYn(j) Yn(k) ≤ L2 := 4 G∗ 2 + (β 2 + t21 ) δ02 + t20 + L21 < ∞. Here, δ0 denotes the diameter of D, and β, t0 , t1 and L1 are given by (6.3), (6.71), (6.73) and (W2a).

6.8 A Special Class of Adaptive Scalar Step Sizes

249

Proof. Corresponding to the proof of Lemma 6.8 we ﬁnd ∗ (j) (j) (j) 2 Yn(j) 2 = G∗ + (G(j) n − G ) + (Zn − En Zn ) + En Zn ∗ 2 (j) (j) 2 (j) 2 ≤ 4 G∗ 2 + G(j) n − G + Zn − En Zn + En Zn . By (6.4), (6.70), (6.71) and (W2a) we have ∗ (j) ∗ G(j) n − G ≤ βXn − x ,

EZn(j) − En Zn(j) 2 ≤ t20 + t21 E∆n 2 , En Zn(j) ≤ L1 . Hence, due to Xn − x∗ , ∆n ≤ δ0 and the above inequalities we get (j)

EYn(j) 2 ≤ L2

EYn(k) 2 ≤ L2 .

as well as

Applying now Schwarz’ inequality, the assertion follows.

Having this lemma, we ﬁnd that condition (Y14) holds in the following cases: (a) kn :=

(b) kn := (c) kn :=

1

ln−1

ln−1 1

(i)

(l

n−1 Yn−1 , Yn−1

,

(6.149)

+i) +

(6.150)

i=1

ln−1

ln−1

(i)

(l

n−1 Yn−1 , Yn−1

i=1 Yn−1 2 I {

min i=1,...,ln−1

(d) kn :=

+i)

1 2ln−1

,

(i)

(l

n−1 Yn−1 , Yn−1

+i)

>0}

,

(6.151)

2ln−1 (0)

(j)

Yn−1 , Yn−1 + .

j=1

Similar examples may be found easily.

(6.152)

7 Computation of Probabilities of Survival/Failure by Means of Piecewise Linearization of the State Function

7.1 Introduction In reliability analysis [31,98,138] of technical or economic systems/structures, the safety or survival of the underlying technical system or structure S is described ﬁrst by means of a criterion of the following type: ⎧ < 0, a safe state σ of S exists, ⎪ ⎪ ⎪ ⎪ S is in a safe state, survival of ⎪ ⎪ ⎪ ⎪ S is guaranteed. ⎪ ⎪ ⎪ ⎪ ⎨ > 0, a safe state σ of S does not exist, S is in an unsafe state, s∗ (R, L) (7.1a) ⎪ ⎪ failure occurs (may occur) ⎪ ⎪ ⎪ ⎪ ⎪ = 0, S is in a safe or unsafe state, ⎪ ⎪ ⎪ resp. (depending on the special ⎪ ⎪ ⎩ situation) Here,

s∗ = s∗ (R, L)

(7.1b)

denotes the so-called (limit) state or performance function of the system/structure S depending basically on an – mR -Vector R of material or structural resistance parameters – mL -Vector L of external load factors In case of economics systems, the “resistance” vector R represents, e.g., the transfer capacity of an input–output system, and the “load” vector L may be interpreted as the vector of demands. The resistance and load vectors R, L depend on certain noncontrollable and controllable model parameters, hence, R = R a(ω), x , L = L a(ω), x (7.1c)

254

7 Computation of Probabilities

with a ν-vector a = a(ω) of random model parameters and an r-vector x of (nominal) design or decision variables. Hence, according to (7.1b), (7.1c), we also have the more general relationship (7.1d) s∗ = s∗ a(ω), x . Remark 7.1. Note that system/structure S = Sa(ω),x is represented here parametrically by the random vector a = a(ω) of model parameters and the vector x of design or decision variables. For the deﬁnition of the state function s∗ = s∗ a(ω), x , in static technical or economic problems one has, after a possible discretization (e.g., by FEM), the following two basic relations [100]: (I) The state equation (equilibrium, balance) ' T σ, a(ω), x = 0, σ ∈ 0 .

(7.2a)

S, e.g., a stress Here, σ is the state mσ -vector of the system/structure' mσ . Furstate vector of a mechanical structure, lying in a range 0 ⊂ IR thermore, for given (a, x), T = T σ, (a, x) denotes a linear or nonlinear mapping from IRmσ into IRmT representing, e.g., the structural equilibrium in elastic or plastic structural design [50, 53, 57, 66, 129]. Remark 7.2. We assume that for each a(ω), x under consideration, relation (7.2a) has at least one solution σ. Depending on the application, (7.2a) may have a unique solution σ = σ a(ω), x , as in the elastic case [129], or multiple solutions σ, as in the plastic case [57]. con Note that dition (7.2a) includes also inequality state conditions T σ, a(ω), x ≤ 0. In case of a linear(ized) system/structure the state equation (7.2a) reads C a(ω), x σ = L a(ω), x (7.2b) with the equilibrium matrix C = C a(ω), x . (II) Admissibility condition (operating condition) In addition to the state equation (7.2a), for the state vector σ we have the admissibility or operating condition ' ' (7.3a) a(ω), x a(ω), x) or σ ∈ int σ∈ with the admissible (feasible) state domain ' a(ω), x := σ ∈ IRn : g σ, a(ω), x ≤ 0

(7.3b)

depending on the model parameter vector a = a(ω) and the design vector x. Here,

7.1 Introduction

255

g = g σ, a(ω), x

(7.3c) denotes a vector function of σ, which depends on a(ω), x , and has appropriate analytical properties. In many applications or after linearization the function g is aﬃne–linear, hence, g σ, a(ω), x := H a(ω), x σ − h a(ω), x (7.3d) with a matrix (H, h) = H a(ω), x , h a(ω), x . We assume that zero state σ = 0 is an interior point of the admissible ' the state domain a(ω), x , i.e., ' 0 ∈ int a(ω), x or g 0, a(ω), x < 0. (7.3e) Thus, in case (7.3d) we have h a(ω), x > 0. (7.3f) ' a(ω), x is a closed convex set In many cases the feasible state domain containing the zero state as an interior point. In these cases the admissibility condition (7.3a) can be represented also by π σ, a(ω), x ≤ 1, π σ, a(ω), x < 1, resp., (7.4a) where π = π σ, a(ω), x denotes the Minkowski or distance functional of ' ' a(ω), x : the admissible state domain = σ ' a(ω), x . (7.4b) π σ, a(ω), x = inf λ > 0 : ∈ λ By decomposition, discretization, the state vector σ is often partitioned, σ = (σi )1≤i≤nG ,

(7.5a)

ofn0 -subvectors σi , i = 1, . . . , nG ; furthermore, the into a certain number nG' a(ω), x is represented then by the product admissible state domain ' ' G a(ω), x = ⊗ni=1 (7.5b) i a(ω), x of nG factor domains

' a(ω), λ := σ σ : g , a(ω), x ≤ 0 i i i i

with the functions

gi = gi σi , a(ω), x , i = 1, . . . , nG .

(7.5c)

(7.5d)

256

7 Computation of Probabilities

In mentioned convex case, due to (7.5b), the distance functional of ' the above a(ω), x can be represented by π σ, a(ω), x = max πi σi , a(ω), x , (7.5e) 1≤i≤nG

σi ' ∈ i a(ω), x πi σi , a(ω), x := inf λ > 0 : λ ' denotes the Minkowski functional of the factor domain i a(ω), x . where

(7.5f)

7.2 The State Function s∗ Based on the relations described in the for a given pair of vectors ﬁrst section, a(ω), x , the state function s∗ = s∗ a(ω), x can be deﬁned by the minimum value function of one of the following optimization problems: Problem A min s s.t.

(7.6a)

T σ, a(ω), x = 0 g σ, a(ω), x ≤ s1 ' σ ∈ 0,

(7.6b) (7.6c) (7.6d)

where 1 := (1, . . . , 1)T . Note that due to Remark 7.2, problem (7.6a)–(7.6d) always has feasible solutions (s, σ) = (˜ s, σ ˜ ) with any solution σ ˜ of (7.2a) and some suﬃciently large s˜ ∈ IR. Obviously, in case of a linear equation (7.2b), problem (7.6a)–(7.6d) state is convex, provided that g = g σ, a(ω), x is a convex vector function of σ, ' and σ has a convex range 0 . Problem B ' If T and g are linear with respect to σ, and the range 0 of σ is a closed convex polyhedron ' (7.7) 0 := {σ : Aσ ≤ b} with a ﬁxed matrix (A, b), then (7.6a)–(7.6d) can be represented, see (7.2b) and (7.3d), by the linear program (LP):

s.t.

min s

(7.8a)

C a(ω, x σ = L a(ω), x H a(ω), x σ − h a(ω), x ≤ s1

(7.8b)

Aσ ≤ b.

(7.8d)

(7.8c)

7.2 The State Function s∗

The dual representation of the LP (7.8a)–(7.8d) reads: T T max L a(ω), x u − h a(ω), x v − bT w s.t.

T T C a(ω), x u − H a(ω), x v − AT w = 0 T

1 v

=1

257

(7.9a)

(7.9b) (7.9c)

v, w ≥ 0. (7.9d) ' n Remark 7.3. If the state vector σ has full range 0 = IR , then (7.8d) is T T canceled, and in (7.9a)–(7.9d) the expressions b w, A w, w are canceled. ' In case of a closed convex admissible state domain a(ω), x , we also have condition (7.4a) with the distance functional π = the admissibility ' a(ω), x . Then, the state function s∗ = s∗ a(ω), x can π σ, a(ω), x of be deﬁned also by the minimum value function of the following optimization problem: Problem C min s s.t.

T σ, a(ω), x = 0 π σ, a(ω), x − 1 ≤ s ' σ ∈ 0.

(7.10a)

(7.10b) (7.10c) (7.10d)

Special forms of (7.10a)–(7.10d) are obtained ' for a linear state equation a(ω), x , as, e.g., closed con(7.2b) and/or for special admissible domains vex polyhedrons, cf. (7.3b), (7.3d), % & ' a(ω), x = σ : H a(ω), x σ ≤ h a(ω), x . (7.11) If (Hi , hi ), i = 1, . . . , mH , denote the rows of (H, h), then, because of property (7.3f), the distance functional of the set in (7.11) is given by σ ≤ hi a(ω), x , i = 1, . . . , mH π σ, a(ω), x = inf λ > 0 : Hi a(ω), x λ # " 1 Hi a(ω), x σ ≤ λ, i = 1, . . . , mH = inf λ > 0 : hi a(ω), x i a(ω), x σ, (7.12a) = max H 0≤i≤mH

where ˜ i a(ω), x := H

"

0, i = 0 1 Hi a(ω), x , 1 ≤ i ≤ mH . hi a(ω),x

(7.12b)

258

7 Computation of Probabilities

7.2.1 Characterization of Safe States Based on the state equation (I) and the admissibility condition (II), the safety or structure S = Sa(ω),x represented by the pair of vec of a system tors a(ω), x can be deﬁned as follows: Deﬁnition 7.1. Let a = a(ω), x, resp., be given vectors of model parameters, design variables. A system Sa(ω),x having conﬁguration a(ω), x ' is in a safe state (admits a safe state) if there exists a “safe” state vector σ ˜ ∈ 0 fulﬁlling (i) the state equation (7.2a) or (7.2b), hence, T σ ˜ , a(ω), x = 0, C a(ω), x σ ˜ = L a(ω), x , resp., (7.13a) and (ii) the admissibility condition (7.3a), (7.3b) or (7.4a), (7.4b) in the standard form g σ ˜ , a(ω), x ≤ 0, π σ ˜ , a(ω), x ≤ 1, resp., (7.13b) or in the strict form g σ ˜ , a(ω), x < 0, π σ ˜ , a(ω), x < 1, resp.,

(7.13c)

depending on the application. System Sa(ω),x is in an unsafe state if no safe state vector σ ˜ exists for Sa(ω),x . Remark 7.4. If Sa(ω),x is in an unsafe state, then, due to violations of the basic safety conditions (I) and (II), failures and corresponding damages of Sa(ω),x may (will) occur. E.g., in mechanical structures (trusses, frames) high external loads may cause very high internal stresses in certain elements which damage then the structure, at least partly. Thus, in this case failures and therefore compensation and/or repair costs occur. or failure of By means of the state function s∗ = s∗ a(ω), x , the safety a system or structure Sa(ω),x having conﬁguration a(ω), x can be described now as follows: Theorem 7.1. Let s∗ = s∗ a(ω), x be the state function of Sa(ω),x deﬁned by the minimum function of one of the above optimization problems (A)–(C). Then, the following basic relations hold: ˜ exists fulﬁlling the strict (a) If s∗ a(ω), x < 0, then a safe state vector σ admissibility condition (7.13c). Hence, S a(ω),x is in a safe state. ∗ (b) If s a(ω), x > 0, then no safe state vector exists. Hence, Sa(ω),x is in an unsafe state and failures and damages may occur. (c) If a safe state vector σ ˜ exists, then s∗ a(ω), x ≤ 0, s∗ a(ω), x < 0, resp., corresponding to the standard, the strict admissibility condition (7.13b), (7.13c).

7.3 Probability of Safety/Survival

259

∗

(d) If s a(ω), x ≥ 0, then no safe state vector exists in case of the strict admissibility condition (7.13c). (e) Suppose that the minimum (s∗ , σ ∗ ) is attained in the selected optimization problem deﬁning the state function s∗ . If s∗ a(ω), x = 0, then a safe state vector exists in case of the standard admissibility condition (7.13b), and no safe state exists in case of the strict condition (7.13c). Proof. (a) If s∗ a(ω), x < 0, then according to the deﬁnition of the minimum value function s∗ , there exists a feasible point (˜ s, σ ˜ ) in Problem (A), (B), (C), resp., such that, cf. (7.6c), (7.8c), (7.10c), g σ ˜ , a(ω), x ≤ s˜1, s˜ < 0, π σ ˜ , a(ω), x − 1 ≤ s˜, s˜ < 0.

(b)

(c)

(d) (e)

' Consequently, σ ˜ ∈ 0 fulﬁlls the state equation (7.13a) and the strict admissibility condition (7.13c). Hence, according to Deﬁnition 7.1, the vector σ ˜ is a safe state vector fulﬁlling the strict admissibility condition (7.13c). Suppose that s∗ a(ω), x > 0, and assume that there exists a safe state vector σ ˜ for Sa(ω),x . Thus, according to Deﬁnition 7.1, σ ˜ fulﬁlls the state equation (7.13a) and the admissibility condition (7.13b). Consequently, (˜ s,σ ˜ ) = (0, ˜ ) is feasible in Problem (A), (B) σ or (C), and we obtain s∗ a(ω), x ≤ 0 in contradiction to s∗ a(ω), x > 0. Thus, no safe state vector exists in this case. Suppose that σ ˜ is a safe state for Sa(ω),x . According to Deﬁnition 7.1, vector σ ˜ fulﬁlls (7.13a) and condition (7.13b), (7.13c), respectively. Thus, (˜ s, σ ˜ ) with s˜ = 0, with a certain s˜ < 0, resp., is feasible in Problem (A), ∗ a(ω), x ≤ 0 in case of the standard and (B) or (C). Hence, we get s ∗ s a(ω), x < 0 in case of the strict admissibility condition. Thisassertion follows directly from assertion (c). ∗ a(ω), x = 0, then under the present assumption there exists σ ∗ ∈ If s ' ∗ ∗ ∗ 0 such that (s , σ ) = (0, σ ) is an optimal solution of Problem (A), (B) ∗ or (C). Hence, σ fulﬁlls the state equation (7.13a) and the admissibility condition (7.13b). Thus, σ ∗ is a safe state vector with respect to the standard admissibility condition. On the other hand, because of (7.13c), no safe state vector σ ˜ exists in case of the strict admissibility condition.

7.3 Probability of Safety/Survival In the following we assume that for all design vectors x under consideration the limit state surface % & (7.14a) LSx := a ∈ IRν : s∗ (a, x) = 0

260

7 Computation of Probabilities

in IRν has probability measure zero, hence, P s∗ a(ω), x = 0 = 0 for x ∈ D,

(7.14b)

where D ⊂ IRr denotes the set of admissible design vectors x. Let x ∈ IRr be an arbitrary, but ﬁxed design vector. According to Theorem 7.1, the so-called safe (parameter) domain Bs,x of the system/structure Sa(ω),x is deﬁned by the set % & Bs,x := a ∈ IRν : s∗ (a, x) < 0 (7.15a) in the space IRν of random parameters a = a(ω). Correspondingly, the unsafe or failure (parameter) domain Bf,x of Sa(ω),x is deﬁned by (Fig. 7.1) % & Bf,x := a ∈ IRν : s∗ (a, x) > 0 . (7.15b) Thus, the probability of safety or survival ps = ps (x) and the probability of failure pf = pf (x) are given, cf. (7.14a), (7.14b), by ps (x) := P Sa(ω),x is in a safe state = P a(ω) ∈ Bs,x (7.16a) = P s∗ a(ω), x < 0 = P s∗ a(ω), x ≤ 0 , pf (x) = P Sa(ω),x is in an unsafe state = P a(ω) ∈ Bf,x (7.16b) = P s∗ a(ω), x > 0 = P s∗ a(ω), x ≥ 0 .

Fig. 7.1. Safe/unsafe domain

7.3 Probability of Safety/Survival

261

For a large class of probability distributions Pa(·) of the random vector a = a(ω) of noncontrollable model parameters, e.g., for continuous distributions, a = a(ω) can be represented by a certain 1–1-transformation (7.17) a(ω) = Γ −1 z(ω) or z(ω) = Γ a(ω) of a random ν-vector z = z(ω) having a standard normal distribution N (0, I) with zero mean and unit covariance matrix. Known transformations Γ : IRν → IRν are the Rosenblatt transformation, see Sect. 2.5.3, the orthogonal transformation [104], the Normal–Tail approximation [30]. Inserting (7.17) into deﬁnitions (7.16a), (7.16b), we get, cf. (7.14a), (7.14b), (7.18a) ps (x) = P s∗Γ z(ω), x < 0 = P s∗Γ z(ω), x ≤ 0 , (7.18b) pf (x) = P s∗Γ z(ω), x > 0 = P s∗Γ z(ω), x ≥ 0 , where the transformed state function s∗Γ = s∗Γ (z, x) is deﬁned by s∗Γ (z, x) := s∗ Γ −1 (z), x . Introducing the transformed safe, unsafe domains (Fig. 7.2). % & Γ := z ∈ IRν : s∗Γ (z, x) < 0 Bs,x = z ∈ IRν : s∗ Γ −1 (z), x < 0 = Γ (Bs,x ), % & Γ Bf,x := z ∈ IRν : s∗Γ (z, x) > 0 = z ∈ IRν : s∗ Γ −1 (z), x > 0 = Γ (Bf,x ),

Fig. 7.2. Transformed safe/unsafe domains

(7.18c)

(7.19a)

(7.19b)

262

7 Computation of Probabilities

we also have Γ , ps (x) = P z(ω) ∈ Bs,x Γ pf (x) = P z(ω) ∈ Bf,x .

(7.20a) (7.20b)

Furthermore, the transformed limit state surface LSxΓ is given by % & LSxΓ := z ∈ IRν : s∗Γ (z, x) = 0 = z ∈ IRν : s∗ Γ −1 (z), x = 0 = Γ (LSx ), (7.21a) where, see (7.14a), (7.14b), P z(ω) ∈ LSxΓ = P s∗ Γ −1 z(ω) , x = 0 = P s∗ a(ω), x = 0 = 0. (7.21b)

7.4 Approximation I of ps, pf : FORM Let x ∈ D be again an arbitrary, but ﬁxed design vector. The well known approximation method “FORM” is based [54] on the linearization of the state function s∗Γ = s∗Γ (z, x) at a so-called β-point zx∗ : Deﬁnition 7.2. A β-point is a vector zx∗ lying on the limit state surface s∗Γ (z, x) = 0 and having minimum distance to the origin 0 of the z-space IRν . For the computation of zx∗ three diﬀerent cases have to be taken into account. 7.4.1 The Origin of IRν Lies in the Transformed Safe Domain Here, we suppose thus Since in many cases

Γ or s∗Γ (0, x) < 0, 0 ∈ Bs,x

(7.22a)

Γ −1 (0) ∈ Bs,x .

(7.22b)

a ¯ = Ea(ω) = Γ −1 (0),

(7.22b) can be represented often by a ¯ ∈ Bs,x or s∗ (¯ a, x) < 0.

(7.22c)

Remark 7.5. Condition (7.22c) means that for the design vectors x under consideration the expected parameter vector a ¯ = Ea(ω) lies in the safe parameter domain Bs,x which is, of course, a minimum requirement for an eﬃcient operating of a system/structure under stochastic uncertainty.

7.4 Approximation I of ps , pf : FORM

263

Obviously, in the present case a β-point is the projection of the origin ¯ Γ of the failure domain B Γ . Hence, zx∗ is a solution of the onto the closure B f,x f,x projection problem (7.23) min z2 s.t. s∗Γ (z, x) ≥ 0. In the following we suppose that all derivatives of s∗Γ (·, x) under consideration exist. The Lagrangian L = L(z, λ) = L(z, λ; x) of the above projection problem reads (7.24) L(z, λ) := z2 + λ − s∗Γ (z, x) . Furthermore, for a projection zx∗ one has the following necessary optimality conditions: (7.25a) ∇z L(z, λ) = 2z + λ − ∇z s∗Γ (z, x) = 0 ∂L (z, λ) = −s∗Γ (z, s) ≤0 (7.25b) ∂λ ∂L (z, λ) = λ − s∗Γ (z, x) λ =0 (7.25c) ∂λ λ ≥ 0. (7.25d) Based on the above conditions, we get this result: Theorem 7.2 (Necessary optimality conditions). Let zx∗ be a β-point, i.e., an optimal solution of the projection problem (7.23). If zx∗ is a regular point, i.e., if s∗Γ (zx∗ , x) > 0 or if ∇z s∗Γ (zx∗ , x) = 0 for s∗Γ (zx∗ , x) = 0,

(7.26a) (7.26b)

then there exists a Lagrange multiplier λ∗ ∈ IR such that (zx∗ , λ∗ ) fulﬁlls the K.–T. conditions (7.25a)–(7.25d). The discussion of the two cases (7.26a), (7.26b) yields the following auxiliary result: Corollary 7.1. Suppose that zx∗ is a regular β-point. Then, in the present case (7.22a) we have s∗Γ (zx∗ , x) = 0 λ∗ ∇z s∗Γ (zx∗ , x) = 0 zx∗ = 2

(7.27a) (7.27b)

with a Lagrange multiplier λ∗ > 0. Proof. Let zx∗ be a regular β-point. If (7.26a) holds, then (7.25c) yields λ∗ = 0. Thus, from (7.25a) we obtain zx∗ = 0 and therefore s∗Γ (0, x) > 0 which contradicts assumption (7.22a) in this Sect. 7.4.1. Hence, in case (7.22a) we must

264

7 Computation of Probabilities

have s∗Γ (zx∗ , x) = 0. Since case (7.26b) is left, we have ∇z s∗Γ (zx∗ , x) = 0 for a regular zx∗ . A similar argument as above yields λ∗ > 0, since otherwise we have again zx∗ = 0 and therefore s∗Γ (0, x) = 0 which again contradicts (7.22a). Thus, relation (7.27b) follows from (7.25a).

Suppose now that zx∗ is a regular β-point for all x ∈ D. Linearizing the state function s∗Γ = s∗Γ (z, x) at zx∗ , because of (7.27a) we obtain s∗Γ (z; x) = s∗Γ (zx∗ , x) + ∇z s∗Γ (zx∗ , x)T (z − zx∗ ) + . . . = ∇z s∗Γ (zx∗ , x)T (z − zx∗ ) + . . . .

(7.28)

Using (7.16a) and (7.28), the probability of survival ps = ps (x) can be approximated now by (7.29a) ps (x) = P s∗Γ z(ω), x < 0 ≈ p˜s (x), where

p˜s (x) := P ∇z s∗Γ (zx∗ )T z(ω) − zx∗ < 0 = P ∇z s∗Γ (zx∗ , x)T z(ω) < ∇z s∗Γ (zx∗ , x)T zx∗ .

(7.29b)

Since z = z(ω) is an N (0, I)-distributed random vector, the random variable in (7.29b), hence, ξ(ω) := ∇z s∗Γ (zx∗ , x)T z(ω) has an N 0, ∇z s∗Γ (zx∗ , x) -normal distribution. Consequently, for p˜s we ﬁnd p˜s (x) = Φ

∇z s∗Γ (zx∗ , x)T zx∗ − 0 ∇z s∗ (zx∗ , x)

=Φ

Γ

∇z s∗Γ (zx∗ , x)T zx∗ ∇z s∗ (zx∗ , x) Γ

,

(7.29c)

where Φ = Φ(t) denotes the distribution function of the standard N (0, 1)normal distribution. Inserting (7.27b) into (7.29c), we get the following representation: ∗ ∇z s∗Γ (zx∗ , x)T λ2 ∇z s∗Γ (zx∗ , x) . (7.29d) p˜s (x) = Φ ∇z s∗ (zx∗ , x) Γ

Using again (7.27b), for the approximative probability p˜s = p˜s (x) we ﬁnd the following ﬁnal representation: Γ , Theorem 7.3. Suppose that the origin 0 of IRν lies in the safe domain Bs,x and assume that the projection problem (7.23) has regular optimal solutions zx∗ only. Then, ps (x) ≈ p˜s (x), where p˜s (x) = Φ zx∗ . (7.30)

7.4 Approximation I of ps , pf : FORM

265

As can be seen from the Deﬁnition (7.29b) of p˜s , the safe domain, cf. (7.19a), % & Γ = z ∈ IRν : s∗Γ (z, x) < 0 Bs,x is approximated by the tangent half space % & Hx := z ∈ IRν : ∇z s∗Γ (zx∗ , x)T (z − zx∗ ) < 0

(7.31)

Γ lies in Hx , hence, if (Fig. 7.4) at zx∗ . If Bs,x Γ ⊂ Hx , Bs,x

(7.32a)

ps (x) ≤ p˜s (x).

(7.32b)

Γ Hx ⊂ Bs,x ,

(7.33a)

p˜s (x) ≤ ps (x).

(7.33b)

then On the contrary, if as indicated by Fig. 7.3, then

Γ Fig. 7.3. Origin 0 lies in Bs,x

266

7 Computation of Probabilities

Fig. 7.4. The safe domain lies in the tangent half space

7.4.2 The Origin of IRν Lies in the Transformed Failure Domain Again let x be an arbitrary, but ﬁxed design vector. Opposite to (7.22a), here we assume now that Γ or s∗Γ (0, x) > 0, (7.34a) 0 ∈ Bf,x which means that

Γ −1 (0) ∈ Bf,x . zx∗

(7.34b) ν

is the projection of the origin 0 of IR onto the In this case, the β-point Γ Γ ¯s,x of the safe domain Bs,x , cf. Fig. 7.5. Thus, zx∗ is a solution of the closure B projection problem (7.35) min z2 s.t. s∗Γ (z, x) ≤ 0. Here, we have the Lagrangian L(z, λ) = L(z, λ; x) := z2 + λs∗Γ (z, x),

(7.36)

Γ ¯s,x reads, cf (7.25a)– and the K.-T. conditions for a projection zx∗ of 0 onto B (7.25d),

∇z L(z, λ) = 2z + λ∇z s∗Γ (z, x) = 0 ∂L (z, λ) = s∗Γ (z, x) ≤0 ∂λ ∂L (z, λ) = λs∗Γ (z, x) λ =0 ∂λ λ ≥ 0. Related to Theorem 7.2, in the present case we have this result:

(7.37a) (7.37b) (7.37c) (7.37d)

7.4 Approximation I of ps , pf : FORM

267

Γ Fig. 7.5. Origin 0 lies in Bf,x

Theorem 7.4 (Necessary optimality conditions). Let zx∗ be a β-point, i.e., an optimal solution of the projection problem (7.35). If zx∗ is regular, hence,

if

∇z s∗Γ (zx∗ , x)

if s∗Γ (zx∗ , x) < 0 or = 0 for s∗Γ (zx∗ , x) = 0,

(7.38a) (7.38b)

then there exists a Lagrange multiplier λ∗ ∈ IR such that (zx∗ , λ∗ ) fulﬁlls the K.–T. conditions (7.37a)–(7.37d). Discussing (7.38a), (7.38b), corresponding to Corollary 4.4 we ﬁnd this auxiliary result: Corollary 7.2. Suppose that zx∗ is a regular point of (7.35). Then, in case (7.34a) we have s∗Γ (zx∗ , x) = 0 zx∗ = −

(7.39a) ∗

λ ∇z s∗Γ (zx∗ , x) 2

(7.39b)

with λ∗ > 0. Proof. Let zx∗ be a regular β-point. If (7.38a) holds, then from (7.37c) we again get λ∗ = 0 and therefore zx∗ = 0, hence, s∗Γ (0, x) < 0 which contradicts assumption (7.34a) in this subsection. Hence, under condition (7.34a) we also have s∗Γ (zx∗ , x) = 0. Since case (7.38b) is left, we get ∇z s∗Γ (zx∗ , x) = 0. Moreover, a similar argument as above yields λ∗ > 0, since otherwise we get zx∗ = 0 and therefore s∗Γ (0, x) = 0. However, this contradict again assumption (7.34a). Equation (7.39b) follows now from (7.37a).

268

7 Computation of Probabilities

Proceeding now as in Sect. 7.4.1, we linearize the state function s∗Γ at zx∗ . Consequently, ps (x) is approximated again by means of formulas (7.29a), (7.29b), hence, ps (x) = P s∗Γ z(ω), x < 0 ≈ p˜s (x), where

p˜s (x) := P ∇z s∗Γ (zx∗ , x)T z(ω) − zx∗ < 0 .

In the same way as in Sect. 7.4.1, in the present case we have the following result: Theorem 7.5. Suppose that the origin 0 of IRν is contained in the unsafe doΓ , and assume that the projection problem (7.35) has regular optimal main Bf,x solutions zx∗ only. Then, ps (x) ≈ p˜s (x), where (7.40) p˜s (x) := Φ − zx∗ . 1 Γ or s∗Γ (0, x) > 0 is of minor interest Since Φ − zx∗ ≤ , the case 0 ∈ Bf,x 2 in practice. Moreover, in most practical cases we may assume, cf. (7.22a)– Γ . (7.22c), that 0 ∈ Bs,x 7.4.3 The Origin of IRν Lies on the Limit State Surface For the sake of completeness we still consider the last possible case that, for a given, ﬁxed design vector x (Fig. 7.6), we have 0 ∈ LSx or s∗Γ (0, x) = 0.

(7.41)

Obviously according to Deﬁnition 7.2 the β-point is given here by zx∗ = 0. Hence, the present approximation method yields p˜s (x) := P s∗Γ (zx∗ , x) + ∇z s∗Γ (zx∗ , x)T z(ω) − zx∗ < 0 = P ∇z s∗Γ (0, x)T z(ω) < 0 " 1 ∗ 2 , for ∇z sΓ (0, x) = 0 = 0, else,

(7.42)

since ξ(ω) = ∇z s∗Γ (0, x)T z(ω) has an N 0, ∇z s∗Γ (0, x) -normal distribution. Remark 7.6. Comparing all cases (7.22a), (7.34a) and (7.41), we ﬁnd that for approximation purposes only the ﬁrst case (7.22a) is of practical interest.

7.4 Approximation I of ps , pf : FORM

269

(a)

(b) Fig. 7.6. (a) 0 ∈ LSx , Case 1 (b) 0 ∈ LSx , Case 2

7.4.4 Approximation of Reliability Constraints In reliability-based design optimization (RBDO) for the design vector or load factor x we have probabilistic constraints of the type ps (x) ≥ αs

(7.43)

270

7 Computation of Probabilities

with a prescribed minimum reliability level αs ∈ (0, 1]. Replacing the probabilistic constraint (7.43) by the approximative condition p˜s (x) ≥ αs ,

(7.44a)

in case (7.22a) we obtain, see Theorem 7.3, Φ zx∗ ≥ αs and therefore the deterministic norm condition zx∗ ≥ βs := Φ−1 (αs )

(7.44b)

for the β-point zx∗ .

7.5 Approximation II of ps, pf : Polyhedral Approximation of the Safe/Unsafe Domain In the following an extension of the “FORM” technique is introduced which works with multiple linearizations of the limit state surface LSx . According to the considerations in Sects. 7.4.1–7.4.3, we may suppose here that (7.22a)– (7.22c) holds, hence, Γ or s∗Γ (0, x) < 0. 0 ∈ Bs,x As can be seen from Sect. 7.4, the β-point zx∗ lies on the boundary of the Γ Γ , Bf,x . Now, further boundary points b transformed safe/unsafe domain Bs,x Γ Γ of Bs,x , Bf,x , can be determined by minimizing or maximizing of a linear form f (z) := cT z,

(7.45)

Γ ¯s,x ¯ Γ of the safe, ,B with a given nonzero vector c ∈ IRν , on the closure B f,x Γ Γ unsafe domain Bs,x , Bf,x . Hence, instead of the projection problems (7.23) or (7.35), in the following we consider optimization problems of the following type: Γ Γ (7.46a) min(max) cT z s.t. z ∈ B s,x (z ∈ B f,x , resp.)

or

min(max) cT z s.t. s∗Γ (z, x) ≤ 0 (≥ 0, resp.).

(7.46b)

Clearly, multiplying the vector c by (−1), we may conﬁne to minimization problems only: (7.47) min cT z s.t. s∗Γ (z, x) ≤ 0 s∗Γ (z, x) ≥ 0, resp. . Deﬁnition 7.3. For a given vector c ∈ IRν and a ﬁxed design vector x, let bc,x denote an optimal solution of (7.47).

7.5 Polyhedral Approximation of the Safe/Unsafe Domain

271

Fig. 7.7. Minimization of a linear form f on the safe domain

As indicated in Fig. 7.7, the boundary point bc,x can also be interpreted as a β-point with respect to a shifted origin 0. The Lagrangian of (7.47) reads – for both cases – (7.48) L(z, λ) = L(z, λ; x) := cT z + λ ± s∗Γ (z, x) , and we have the K.–T. conditions ∇z L(z, λ) = c ± λ∇z s∗Γ (z, x) = 0 ∂L (z, λ) = s∗Γ (z, x) ≤ 0 (≥ 0, resp.) ∂λ ∂L λ (z, λ) = λs∗Γ (z, x) = 0 ∂λ λ ≥ 0.

(7.49a) (7.49b) (7.49c) (7.49d)

Supposing again that the solutions bc,x of (7.47) are regular, related to Sect. 7.4, here we have this result: Theorem 7.6 (Necessary optimality conditions). Let c = 0 be a given νvector. If bc,x is a regular optimal solution of (7.47), then there exists λ∗ ∈ IR such that s∗Γ (bc,x , x) = 0 ∇z s∗Γ (bc,x , x) = ∓

(7.50a) c with λ∗ > 0. λ∗

(7.50b)

272

7 Computation of Probabilities

Proceeding now in similar way as in Sect. 7.4, the state function s∗Γ = s∗Γ (z, x) is linearized at bc,x . Hence, with (7.50a) we get s∗Γ (z, x) = s∗Γ (bc,x , x) + ∇z s∗Γ (bc,x , x)T (z − bc,x ) + . . . = ∇z s∗Γ (bc,x , x)T (z − bc,x ) + . . . .

(7.51a)

Then, a ﬁrst type of approximation ps (x) ≈ p˜s (x) is deﬁned again by p˜s (x) := P ∇z s∗Γ (bc,x , x)T z(ω) − bc,x < 0 . As in Sect. 7.4 we obtain p˜s (x) = P ∇z s∗Γ (bc,x , x)T z(ω) < ∇z s∗Γ (bc,x , x)T bc,x ∇z s∗Γ (bc,x , x)T bc,x . =Φ ∇z s∗ (bc,x , x) Γ

(7.51b)

(7.51c)

Inserting now (7.50b) into (7.51c), for p˜s (x) we ﬁnd the following representation: Theorem 7.7. Let c = 0 be a given ν-vector and assume that (7.47) has regular optimal solutions bc,x only. Then the approximation p˜s (x) of ps (x), deﬁned by (7.51b), can be represented by

(∓c)T bc,x . (7.52) p˜s (x) = Φ c Proof. From (7.51c) and (7.50b) we obtain (λ∗ is deleted!) T

∓ λc∗ bc,x (∓c)T bc,x c . = Φ p˜s (x) = Φ ∓ ∗ c λ

Remark 7.7. We observe that for an appropriate selection of the vector c, approximation (7.52) coincides with approximation (7.30) of ps (x) based on a β-point. Obviously, also in the present case only an optimal solution bc,x of the minimization problem (7.47) is needed replacing the projection problem (7.23). In the following we assume that for any design vector x and any cost vector c = 0, the vector bc,x indicates an ascent direction for s∗Γ = s∗Γ (z, x) at bc,x , i.e., (7.53a) ∇z s∗Γ (bc,x , x)T bc,x > 0.

7.5 Polyhedral Approximation of the Safe/Unsafe Domain

273

Depending on the case taken in (7.47), this is equivalent, cf. (7.50b), to the condition (7.53b) (∓c)T bc,x > 0, involving the following two cases: Case 1 (“–”): Problem (7.47) with “≤,” Case 2 (“+”): Problem (7.47) with “≥.” Illustrations are given in the Fig. 7.8a,b below. 7.5.1 Polyhedral Approximation In the following we consider an arbitrary, but ﬁxed design vector x. Selecting now a certain number J of c-vectors c1 , c2 , . . . , c j , . . . , cJ ,

(7.54a)

and solving the related minimization problem (7.47) for c = cj , j = 1, . . . , J, we obtain the corresponding boundary points bc1 ,x , bc2 ,x , . . . , bcj ,x , . . . , bcJ ,x .

(7.54b)

Remark 7.8 (Parallel computing of the b-points). Computing the boundary points bcj ,x , j = 1, . . . , J, for a given design vectors x and c-vectors c1 , . . . , cJ , we have to solve optimization problem (7.47) for J diﬀerent cost vectors cj , j = 1, . . . , J, but with an identical constraint. Thus, the boundary points bcj ,x , j = 1, . . . , J, and the corresponding constraints ∇z s∗Γ (bcj ,x , x)T (z − bcj ,x ) < 0

(7.55a)

(∓cj )T (z − bcj ,x ) < 0,

(7.55b)

or, cf. (7.50b) j = 1, . . . , J, can be determined simultaneously by means of parallel computing. Considering the constraints (7.55a) or (7.55b) ﬁrst as joint constraints, we ﬁnd the following more accurate approximations of the probability of survival/failure ps , pf . Γ Case 1: Minimization of Linear Forms on Bs,x Γ In this case, see Fig. 7.9, the transformed safe domain Bs,x is approximated from outside by a convex polyhedron deﬁned by the linear constraints (7.55a) or (7.55b) with “–.” Hence, for ps we get the upper bound

ps (x) ≤ p˜s (x),

(7.56a)

274

7 Computation of Probabilities

(a)

(b) Fig. 7.8. (a) Condition (7.53a), (7.53b) in Case 1. (b) Condition (7.53a), (7.53b) in Case 2

where

p˜s (x) = P ∇z s∗Γ (bcj ,x , x)T z(ω) − bcj ,x < 0, j = 1, . . . , J (7.56b) = P (−cj )T z(ω) − bcj ,x < 0, j = 1, . . . , J .

7.5 Polyhedral Approximation of the Safe/Unsafe Domain

275

Fig. 7.9. Polyhedral approximation (in Case 1)

Deﬁning then the J × ν matrix A = A(c1 , . . . , cJ ) and the J-vector b = b(x; c1 , . . . , cJ ) by A(c1 , . . . , cJ ) := (−c1 , −c2 , . . . , −cJ )T , b(x; c1 , . . . , cJ ) := (−cj )T bcj ,x 1≤j≤J

p˜s can be represented also by p˜s (x; c1 , . . . , cJ ) = P A(c1 , . . . , cJ )z(ω) ≤ b(x; c1 , . . . , cJ ) .

(7.57a) (7.57b)

(7.57c)

Treating the constraints (7.55a) or (7.55b) separately, for ps we ﬁnd the larger, but simpler upper bound ps (x) ≤ p˜s (x),

(7.58a)

p˜s (x) := min p˜s (x; cj )

(7.58b)

where 1≤j≤J

and, cf. (7.51b), (7.51c), (7.52), p˜s (x; cj ) = Φ

(−cj )T bcj ,x c

.

(7.58c)

276

7 Computation of Probabilities

Γ Case 2: Minimization of Linear Forms on Bf,x Γ Here, see Figs. 7.5 and 7.8b, the transformed failure domain Bf,x is approximated from outside by a convex polyhedron based on the contrary of the constraints (7.55a) or (7.55b). Thus, for the probability of failure pf we get, cf. (7.33a), (7.33b), the upper bound

pf (x) ≤ p˜f (x) with

(7.59a)

p˜f (x) := P ∇z s∗Γ (bcj ,x , x)T z(ω) − bcj ,x ≥ 0, f = 1, . . . , J (7.59b) = P cjT z(ω) − bcj ,x ≥ 0, j = 1, . . . , J .

Corresponding to (7.57a), (7.57b), here we deﬁne the J × ν matrix A = A(c1 , . . . , cJ ) and the J-vector b = b(x; c1 , . . . , cJ ) by A(c1 , . . . , cJ ) := (c1 , c2 , . . . , cJ )T b(x; c1 , . . . , cJ ) := (cjT bcj ,x )1≤j≤J . Then, p˜f = p˜f (x) can be represented by p˜f (x) = P A(c1 , . . . , cJ )z(ω) ≥ b(x; c1 , . . . , cJ ) .

(7.60a) (7.60b)

(7.60c)

Obviously, in the present Case 2 p˜s (x) := 1 − p˜f (x)

(7.61a)

p˜s (x) ≤ ps (x)

(7.61b)

deﬁnes a lower bound for the probability of survival ps . Working with separate constraints, for pf we obtain the larger, but simpler upper bound pf (x) ≤ p˜f (x) with

(7.62a)

p˜f (x) := min P cjT z(ω) − bcj ,x ≥ 0 1≤j≤J = min 1 − p˜s (x; cj ) = 1 − max p˜s (x; cj ). 1≤j≤J

1≤j≤J

Here,

j

p˜s (x; c ) := Φ

cjT bcj ,x cj

(7.62b)

.

(7.62c)

7.5 Polyhedral Approximation of the Safe/Unsafe Domain

277

Remark 7.9. Since z = z(ω) has an N (0, I)-normal distribution, the random vector y = Az(ω) (7.63a) with the matrix A = A(c1 , . . . , cJ ) given by (7.57a), (7.60a), resp., has an N (0, Q)-normal distribution with the J × J covariance matrix Q = AAT = (ciT cj )i,j=1,...,J .

(7.63b)

Example 7.1 (Spherical safe domain). Γ is given by Here we assume that the transformed safe domain Bs,x % & Γ = z ∈ IRν : zE − R(x) =: s∗Γ (z, x) < 0 , Bs,x

(7.64)

where ·E denotes the Euclidean norm, and the radius R = R(x) is a function of the design vector x (Fig. 7.10). In the present case for each c = 0 we obtain, see (7.47) with “≤,” bc,x = −R(x)

c . c

(7.65a)

Consequently, for a single vector c, according to (7.58b) we get for each vector c

Fig. 7.10. Spherical safe domain (Case 1)

278

7 Computation of Probabilities

⎞ c (−c)T −R(x) c

⎠ p˜s (x; c) = Φ ⎝ c ⎛

(7.65b)

= Φ R(x) , where, cf. (7.32b),

ps (x) ≤ p˜s (x; c) = Φ R(x) .

(7.65c)

Working with J vectors c1 , . . . , cJ , we get, see (7.56a), (7.56b), p˜s (x, c1 , . . . , cJ ) = P (−cj )T z(ω) − bcj ,x < 0, j = 1, . . . , J = P (−cj )T z(ω) < R(x)cj , j = 1, . . . , J . In the special case J = 2ν and c1 = e1 , c2 = −e1 , c3 = e2 , c4 = −e2 , . . . , c2ν−1 = eν , c2ν = −eν ,

(7.66a)

where e1 , e2 , . . . , eν are the unit vectors of IRν (Fig. 7.11), we obtain

Γ Fig. 7.11. Spherical safe domain Bs,x with c-vectors given by the unit vectors

7.6 Computation of the Boundary Points

279

p˜s (x; ±e1 , . . . , ±eν ) = P zj (ω) < R(x), j = 1, . . . , ν ν

=

P − R(x) < zj (ω) < R(x)

j=1

ν = Φ R(x) − Φ − R(x) ν = 2Φ R(x) − 1 .

(7.66b)

In the present case of the spherical safe domain (7.64), the exact probability ps (x) can be determined analytically: ps (x) = P z(ω) ≤ R(x)

1 1 2 z dz. (7.67a) ... exp − = 2 (2π)ν/2

z ≤R(x)

Introducing polar coordinates in IRν , by change of variables in (7.67a) we obtain R(x) 2−ν r2 2 2 e− 2 rν−1 dr, (7.67b) ps (x) = ν Γ 2 0

where Γ = Γ (t) denotes the Γ -function. In the case ν = 2 we have Γ (1) = 1 and therefore 2 1 (7.67c) ps (x) = 1 − e− 2 R(x) . Comparing the above approximations, for ν = 2 we have 2 2 1 ps (x) = 1 − e− 2 R(x) ≤ p˜s (x; ±e1 , ±e2 ) = 2Φ R(x) − 1 ≤ p˜s (x; c) = Φ R(x) . (7.67d)

7.6 Computation of the Boundary Points Let x denote again a given, but ﬁxed design vector. In this section we consider the properties and possible numerical solution of the optimization problem (7.47) for the computation of the boundary (b-)points bc,x . According to Sect. 7.5, this problem has the following two versions: Case 1 (7.68) min cT z s.t. s∗Γ (z, x) ≤ 0 Case 2 min cT z

s.t. s∗Γ (z, x) ≥ 0.

(7.69)

Thus, solving (7.68) or (7.69), a central problem is that the constraint of these problems is not given by an explicit formula, but by means of the minimum value function of a certain optimization problem. However, in the

280

7 Computation of Probabilities

following we obtain explicit representations of (7.68) and (7.69), containing an inner optimization problem for the representation of the constraint, by taking into account problems (A), (B) and (C) in Sect. 7.2 describing the state function s∗ . 7.6.1 State Function s∗ Represented by Problem A For simplicity we assume that Problem A, i.e., (7.6a)–(7.6d), has an optimal solution (s∗ , σ ∗ ) for each pair (a, x) of a parameter vector a and a design vector x under consideration. According to the minimum value deﬁnition (7.6a)– (7.6d) of s∗ = s∗ (a, x), condition s∗ (a, x) ≤ 0 holds if and only if there exists a feasible point (s, σ) of (7.6a)–(7.6d) such that s ≤ 0. Since (7.6c) with

condition s ≤ 0 is equivalent to g˜ σ, (z, x) ≤ s1 ≤ 0, problem (7.68) can be described explicitly by (7.70a) min cT z σ,z

s.t.

T σ, (z, x) = 0 (7.70b) g˜ σ, (z, x) ≤ 0 (7.70c) ' (7.70d) σ ∈ 0. Here, the functions T = T σ, (z, x) and g˜ = g˜ σ, (z, x) are deﬁned, cf. (7.17), by T σ, (z, x) := T σ, Γ −1 (z), x , (7.71a) (7.71b) g˜ σ, (z, x) := g σ, Γ −1 (z), x . For a similar representation of the related problem (7.69), a representation of Problem A by means of a dual maximization problem is needed, see the next Sect. 7.6.2. 7.6.2 State Function s∗ Represented by Problem B For a given, ﬁxed design vector x we assume again that the LP (7.8a)–(7.8d) has an optimal solution (s∗ , σ ∗ ) for each (a, x) under consideration. Due to the minimum value representation of s∗ by (7.8a)–(7.8d), problem (7.68) can be described explicitly by (7.72a) min cT z σ,z

s.t. x)σ = L(z, ˜ x) C(z, ˜ x) ≤ 0 x)σ − h(z, H(z, Aσ ≤ b,

(7.72b) (7.72c) (7.72d)

7.6 Computation of the Boundary Points

281

where x) = L Γ −1 (z), x x) := C Γ −1 (z), x , L(z, C(z, ˜ x) = h Γ −1 (z), x . x) := H Γ −1 (z), x , h(z, H(z,

(7.73a) (7.73b)

Since (7.8a)–(7.8d) is an LP, the state function s∗ can be described also by the maximum value function of the dual maximization problem (7.9a)–(7.9d). Hence, the second problem (7.69) with the “≥” constraint can be described explicitly by (7.74a) min cT z u,v,w,z

s.t. ˜ x)T v − bT w ≥ 0 x)T u − h(z, L(z,

(7.74b)

x) u − H(z, x) v − A w = 0 C(z,

(7.74c)

1T v = 1

(7.74d)

v, w ≥ 0.

(7.74e)

T

T

T

The results in Sects. 7.6.1 and 7.6.2 can be summarized as follows: Theorem 7.8. Explicit representation of the programs for the computation of the b-points. (a) Problem (7.47) with the constraint “≤,” i.e., problem (7.68), can be represented explicitly by the optimization problem (7.70a)–(7.70d) or (7.72a)– (7.72d). (b) In the linear case, problem (7.47) with the “≥” constraint, i.e., problem (7.69), can be represented explicitly by the optimization problem (7.74a)– (7.74e). In practice, often the following properties hold: x) = C (independent of (z, x)) C(z,

(7.75a)

x) = L(z) L(z, (independent of x)

(7.75b)

x) = H (independent of (z, x)) H(z,

(7.75c)

˜ x) = h(x) (independent of z). h(z,

(7.75d)

The above conditions mean that we have a ﬁxed equilibrium matrix C, a random external load L = L a(ω) independent of the design vector x and ﬁxed material or strength parameters. Under the above conditions (7.75a)–(7.75d), problem (7.72a)–(7.72d) takes the following simpler form: (7.76a) min cT z σ,z

282

7 Computation of Probabilities

s.t. ˜ Cσ − L(z) =0 Hσ ≤ h(x) Aσ ≤ b.

(7.76b) (7.76c) (7.76d)

Remark 7.10. Normal distributed parameter vector a = a(ω). In this case ˜ = L(z) ˜ L is an aﬃne–linear function. Consequently, (7.76a)–(7.76d) is then an LP in the variables (σ, z).

7.7 Computation of the Approximate Probability Functions Based on polyhedral approximation, in Sect. 7.5.1 several approximations p˜s of the survival probability ps are derived. For the simpler case of separate treatment of the constraints (7.55a) or (7.55b), explicit representations of p˜s are given already in (7.58a)–(7.58c) and (7.61a), (7.61b), (7.62a)–(7.62c). For the computation of the more accurate approximative probability functions p˜s , p˜f given by (7.56b) or (7.57c), (7.59b) or (7.60c), resp., several techniques are available, such as (I) Probability inequalities (II) Discretization of the joint normal distribution N (0, I) and the related (III) Monte Carlo simulation In the following we consider methods (I) and (II). 7.7.1 Probability Inequalities In Case 1 we have, cf. (7.57c), p˜s (x) = P A(c1 , . . . , cJ )z(ω) ≤ b(x; c1 , . . . , cJ ) , where the J ×ν matrix A = A(c1 , . . . , cJ ) and the J-vector b = b(x; c1 , . . . , cJ ) are given by (7.57a), (7.57b); furthermore,z =z(ω) is a normal distributed random ν-vector with Ez(ω) = 0 and cov z(·) = I (identity matrix). Due to condition (7.53a), (7.53b) we have b = b(x; c1 , . . . , cJ ) > 0.

(7.77)

p˜s (x; c1 , . . . , cJ ) = P Az(ω) ≤ b Az(ω) ≤ 1 = P u(ω) ≤ 1 , = P b−1 d

(7.78a)

Thus, we get

7.7 Computation of the Approximate Probability Functions

283

where 1 := (1, . . . , 1)T ∈ IRJ bd := (bj δij )1≤i,j≤J , b−1 d =

1 δij bj

(7.78b)

u(ω) := b−1 d Az(ω),

(7.78c) 1≤i,j≤J

(7.78d)

where δij denotes the Kronecker symbol. Obviously, the random J-vector u = u(ω) has a joint normal distribution with T −1 (7.78e) Eu(ω) = 0, cov u(·) = b−1 d AA bd . Inequalities for p˜s of the Markov-type can be obtained, see, e.g., [100], by selecting a measurable function ϕ = ϕ(u) on IRJ such that (1)

ϕ(u) ≥ 0, u ∈ IRJ

(7.79a)

(2)

ϕ(u) ≥ ϕ0 > 0 for u ≤ 1

(7.79b)

with a positive constant ϕ0 . If ϕ = ϕ(u) is selected such that (7.79a), (7.79b) holds, then for p˜s we get the lower bound 1 Eϕ b−1 Az(ω) . (7.80) p˜s (x; c1 , . . . , cJ ) ≥ 1 − d ϕ0 We suppose that expectations under consideration exist and are ﬁnite. The approximate probability constraint (7.44a), i.e., p˜s (x; c1 , . . . , cJ ) ≥ αs can be guaranteed then by the condition Eϕ b−1 Az(ω) ≤ ϕ0 (1 − αs ). d

(7.81a)

By means of this technique, lower probability bounds for p˜s and corresponding approximative chance constraints may be obtained very easily, provided that the expectation Eϕ b−1 Az(ω) can be evaluated eﬃciently. Obd viously, the accuracy of the approximation (7.80) increases with the reliability level αs used in (7.44a): 1 −1 0 ≤ p˜s − 1 − Eϕ bd Az(ω) ≤ 1 − αs . (7.81b) ϕ0 Moreover, best lower bounds in (7.80) follow by maximizing the right hand side of (7.80). Some examples for the selection of a function ϕ = ϕ(u), the related positive constant ϕ0 and the consequences for the expectation term in inequality (7.80) are given next.

284

7 Computation of Probabilities

Example 7.2 (Positive deﬁnite quadratic forms). Here we suppose that ϕ = ϕ(u) is given by (7.82a) ϕ(u) := uT Qu, u ∈ IRJ , with a positive deﬁnite J × J matrix Q such that uT Qu ≥ ϕ0 > 0 for u ≤ 1

(7.82b)

where ϕ0 is a certain positive constant. If the positive deﬁnite quadratic form (7.82a) fulﬁlls (7.82b), then for p˜s = p˜s (x; c1 , . . . , cJ ) we have the lower probability bound (7.80). Moreover, in this case (7.78b)–(7.78e) yields Az(ω) = Eϕ u(ω) = Eu(ω)T Qu(ω) Eϕ b−1 d = EtrQu(ω)u(ω)T = trQcov u(·) T −1 = trQb−1 d AA bd .

(7.83)

Then, with (7.57a), (7.57b), (7.80) and (7.83) we have the following result: Theorem 7.9. Suppose that ϕ = ϕ(u) is given by (7.82a), (7.82b) with a positive deﬁnite matrix Q and a positive constant ϕ0 . In this case p˜s (x; c1 , . . . , cJ ) ≥ 1 −

where T −1 b−1 d AA bd

:=

1 T −1 trQb−1 d AA bd , ϕ0

ciT cj (ciT bci ,x )(cjT bcj ;x )

(7.84a)

.

(7.84b)

i,j=1,...,J

For a given positive deﬁnite matrix Q, the best, i.e., largest lower bound ϕ∗0 > 0 in (7.82b) and therefore also in (7.84a), is given by ϕ∗0 = min{uT Qu : u ≤ 1}.

(7.85a)

We also have, cf. Fig. 7.12, ϕ∗0 = min min{uT Qu : ul ≥ 1}, 1≤l≤J

(7.85b)

where ul denotes the lth component of u. Thus, ϕ∗0 is the minimum ϕ∗0 = min ϕ∗0l 1≤l≤J

(7.85c)

of the minimum values ϕ∗0l of the convex minimization problems min uT Qu s.t. 1 − ul ≤ 0,

(7.85d)

l = 1, . . . , J. If (Q−1 )ll denotes the lth diagonal element of the inverse Q−1 of Q, then

7.7 Computation of the Approximate Probability Functions

285

Fig. 7.12. One-sided inequalities in u-domain

ϕ∗0l =

1 (Q−1 )ll

and therefore ϕ∗0 = ϕ∗0 (Q) = min

1≤l≤J

(7.85e)

1 . (Q−1 )ll

(7.85f)

In the special case of a diagonal matrix Q = (qjj δij ) with positive diagonal elements qjj > 0, j = 1, . . . , J, from (7.85f ) we get ϕ∗0 = min qjj . 1≤l≤J

(7.85g)

The largest lower bound in the probability inequality (7.84a) can be obtained now by solving the optimization problem T −1 trQb−1 d AA bd , Q0 ϕ∗0 (Q)

min

(7.86)

where the constraint in (7.86) means that Q is a positive deﬁnite J ×J matrix. T −1 In the special case of a diagonal matrix Q, with W := b−1 d AA bd we get trQW =

J

qjj wjj

j=1

and therefore, W is at least positive semideﬁnite, J '

trQW = ϕ∗0 (Q)

qjj wjj

j=1

min qjj

1≤j≤J

≥ trW,

286

7 Computation of Probabilities

cf. (7.85g). Thus, if we demand in (7.86) that Q is diagonal, then min

Q0 Qdiagonal

T −1 trQb−1 T −1 d AA bd = trb−1 d AA bd . ∗ ϕ0 (Q)

(7.87)

Example 7.3 (Convex quadratic functions). In extension of (7.82a), (7.82b) we consider now the class of convex quadratic functions given by ϕ(u) := (u − u0 )T Q(u − u0 ), u ∈ IRJ ,

(7.88a)

with a positive semideﬁnite J × J matrix Q and a J-vector u0 such that u0 ≤ 1. Dividing the inequality in (7.79b) by the constant ϕ0 > 0, it is easy to see that we may suppose (7.88b) ϕ0 = 1. Thus, if the matrix/vector-parameter (Q, u0 ) is selected such that ϕ(u) ≥ 1 for all u ≤ 1,

(7.88c)

then corresponding to (7.80) we have p˜s (x; c1 , . . . , cJ ) ≥ 1 − Eϕ b−1 Az(ω) . d

(7.89a)

Using (7.78c)–(7.78e) and (7.83), in the present case (7.88a) we get Eϕ b−1 Az(ω) = Eϕ u(ω) d = E u(ω)T Qu(ω) − 2uT0 Qu(ω) + uTo u0 T −1 T = trQb−1 d AA bd + u0 u0 .

(7.89b)

Corresponding to (7.85a), we deﬁne ϕ∗0 = ϕ∗0 (Q, u0 ) by ϕ∗0 (Q, u0 ) := min (u − u0 )T Q(u − u0 ) : u ≤ 1 .

(7.90a)

We have, cf. (7.85b), ϕ∗0 (Q, u0 ) = min ϕ∗0l (Q, u0 ),

(7.90b)

1≤l≤J

where ϕ∗0l = ϕ∗0l (Q, u0 ) denotes the minimum value of the convex program min(u − u0 )T Q(u − u0 )s.t. 1 − ul ≤ 0,

(7.90c)

l = 1, . . . , J. As in (7.85d)–(7.85f) we get ϕ∗0 (Q, u0 ) = min

1≤l≤J

(1 − u0l )2 for Q (Q−1 )ll

0.

(7.90d)

7.7 Computation of the Approximate Probability Functions

Fixing a set

B ⊂ (Q, u0 ) : Q 0, u0 ≤ 1 ,

287

(7.91a)

the best probability bound in (7.89a) is represented by " # ∗ p˜s (x; c1 , . . . , cJ ) ≥ 1 − inf Eϕ b−1 d Az(ω) : ϕ0 (Q, u0 ) ≥ 1, (Q, u0 ) ∈ B , (7.91b) cf. (7.86). Hence, in the present case the following minimization problem is obtained: T −1 T (7.92a) min trQb−1 d AA bd + u0 u0 s.t. ϕ∗0 (Q, u0 ) ≥ 1

(7.92b)

(Q, u0 ) ∈ B.

(7.92c)

Approximation by Means of Two-Sided Probability Inequalities Selecting a J-vector d < 1 := (1, . . . , 1)T ∈ IRJ , we may approximate p˜s = p˜s (x; c1 , . . . , cJ ) by p˜s (x; c1 , . . . , cJ ) = P u(ω) ≤ 1 ≥ P d ≤ u(ω) ≤ 1 , (7.93) where the J-random vector u = u(ω) is again deﬁned by (7.78b)–(7.78e). T −1 Since u = u(ω) has an N (0, b−1 d AA bd )-normal distribution we may select d < 1 such that the error of the approximation (lower bound) is small. Corresponding to (7.79a), (7.79b), in the present case we consider a function ϕ = ϕ(u) on IRJ such that ϕ(u) ≥ 0 for u ∈ IRJ

(7.94a)

ϕ(u) ≥ 1 for all u ∈ IRJ such that u ∈ [d, 1].

(7.94b)

Then, from (7.93) and (7.78d) we get [100] p˜s (x; c1 , . . . , cJ ) ≥ 1 − Eϕ b−1 d Az(ω) .

(7.94c)

Example 7.4 (Convex quadratic functions for bounded u-domains). Suppose that the function ϕ = ϕ(u) is deﬁned again by (7.88a), where in the present case (Q, u0 ) must be selected such that ϕ(u) ≥ 1 for all u ∈ [d, 1]

(7.95a)

288

7 Computation of Probabilities

Fig. 7.13. Two-sided inequalities in u-domain

and d < u0 < 1. As in the above examples we deﬁne now, cf. Fig. 7.13, ϕ∗0 (Q, u0 ) = min (u − u0 )T Q(u − u0 ).

(7.95b)

ϕ∗0 (Q, u0 ) = min min ϕ∗0la (Q, uo ), ϕ∗0lb (Q, u0 ) ,

(7.95c)

u∈[d,1]

We get

1≤l≤J

where ϕ∗0la = ϕ∗0la (Q, u0 ), ϕ∗0lb = ϕ∗0lb (Q, u0 ), resp., l = 1, . . . , J, denote the minimum values of the convex optimization problems min(u − u0 )T Q(u − u0 ) s.t. ul − dl ≤ 0, min(u − u0 )T Q(u − u0 ) s.t. 1 − ul ≤ 0.

(7.95d) (7.95e)

If the matrix Q is positive deﬁnite, then (7.90d) yields ϕ∗0lb (Q, u0 ) =

(1 − u0l )2 , l = 1, . . . , J, (Q−1 )ll

(7.95f)

(dl − u0l )2 , l = 1, . . . , J. (Q−1 )ll

(7.95g)

and in similar way we get ϕ∗0la (Q, u0 ) =

Thus, in case of a positive deﬁnite J × J matrix Q we ﬁnally get 1 2 2 . min (1 − u ) , (d − u ) ϕ∗0 (Q, u0 ) = min 0l l 0l 1≤l≤J (Q−1 )ll

(7.95h)

Maximum lower probability bounds in (7.94c) can be obtained then by the same techniques as described in Example 7.3.

7.7 Computation of the Approximate Probability Functions

289

7.7.2 Discretization Methods The problem is again the computation of the approximate probability function p˜s given by (7.57c), hence, (7.96a) p˜s (x; c1 , . . . , cJ ) = P z(ω) ∈ C(x; c1 , . . . , cJ ) . Here, the convex set C(x; c1 , . . . , cJ ) in IRν is deﬁned by C(x; c1 , . . . , cJ ) := {z ∈ IRν : Az ≤ b},

(7.96b)

where the matrix (A, b) is given by (7.57a), (7.57b). Furthermore, z = z(ω) is a random ν-vector having the standard normal distribution N (0, I) with zero mean and unit covariance matrix I. However, in the present case it is also possible to consider directly the basic probability function ps given by (7.18a). Hence, ps (x) = P s∗Γ z(ω), x ≤ 0 Γ = P z(ω) ∈ Bs,x with the transformed limit state function s∗Γ = s∗Γ (z, x) or the transformed Γ . safe domain Bs,x Based on the representation of ps = ps (x) by a multiple integral [150] in IRν , the N (0, I)-distributed random ν-vector z = z(ω) is approximated now by a random ν-vector z˜ = z˜(ω) having a discrete distribution Pz˜(·) = µ :=

N

αj εz(j) ,

(7.97a)

j=1

where z (j) , j = 1, . . . , N , are the realizations or scenarios of z˜(·) taken in IRν with probabilities αj , j = 1, . . . , N , where N

αj = 1, αj > 0, j = 1, . . . , N.

(7.97b)

j=1

As an appropriate approximation of the standard normal distribution N (0, I) in IRν , the discrete distribution µ should have the following obvious properties: zµ(dz) = E z˜(ω) = Ez(ω) = 0 (7.97c)

zz T µ(dz) = cov z˜(·) = cov z(·) = I.

(7.97d)

290

7 Computation of Probabilities

Thus, we have to select the set of realizations and corresponding probabilities

(1) (2) z , z , . . . , z (j) , . . . , z (N ) (7.97e) α1 , α2 , . . . , αj , . . . , αN such that N

αj z (j) = 0

(7.98a)

αj z (j) z (j)T = I

(7.98b)

αj = 1

(7.98c)

j=1 N j=1 N j=1

αj > 0, j = 1, . . . , N.

(7.98d)

Remark 7.11. In order to increase the accuracy of the approximation µ of the N (0, I)-distribution in IRν , conditions for the third and higher order moments of z˜ = z˜(ω) can be added to (7.98a)–(7.98d). Moreover, known symmetry properties of N (0, I) can be used in the selection of the realizations z (j) and probabilities αj , j = 1, . . . , N , in (7.97e). Because of the symmetry of the covariance matrix, in system (7.98a)– (7.98d) we have ν(ν − 1) + 1 equalities 2 (b) N inequalities (a) ν +

for the N ν + N unknowns (z (j) , αj ), j = 1, . . . , N , in IRν × IR. Deﬁning α := (α1 , α2 , . . . , αN )T Z := (z (1) , z (2) , . . . , z (N ) ) ⎛√ ⎞ α1 0 √ ⎜ ⎟ α2 √ ⎜ ⎟ ( α)d := ⎜ ⎟ . .. ⎝ ⎠ √ αN 0 1 := (1, 1, . . . , 1)T ,

(7.99a) (7.99b)

(7.99c)

(7.99d)

conditions (7.98a)–(7.98d) can be represented also in the following form: √ Zα = 0 (Z αd )(Z αd )T = I αT 1 = 1 α > 0. √

(7.98a ) (7.98b ) (7.98c ) (7.98d )

7.7 Computation of the Approximate Probability Functions

291

In the special case αj =

1 , j = 1, . . . , N, N

(7.100a)

we get 1 1 N √ 1 αd = √ I, N

(7.100b)

α=

(7.100c)

and (98a )–(98d ) is reduced then to

Z1 = 0

T 1 1 √ Z √ Z = I. N N

(7.100d) (7.100e)

Example 7.5. For ν = 2 and N = 3, a possible selection of z (j) , αj , j = 1, 2, 3, reads 1 1 √ √ −√2 (2) 2 / /2 (1) (3) , z = z = 0 ,z = 3 3 − 2 2 1 1 1 α2 = , α3 = . α1 = , 3 3 3 1 If N = ν, then (7.100e) means that √ Z is an orthogonal matrix which N contradicts (7.100d). On the other hand, we can take N = 2ν and select then (j) the vectors zαj , j = 1, . . . , N , as follows: z˜(1) , z˜(2) , . . . , z˜(ν) , −˜ z (1) , −˜ z (2) , . . . , −˜ z (ν) ˜2 α ˜ν α ˜1 α ˜2 α ˜ν α ˜1 α , , ..., , , , ..., . 2 2 2 2 2 2

(7.101a) (7.101b)

Then, N

αj z (j) =

αj z (j) z (j)T =

j=1

=

ν α ˜j j=1 ν

2

z˜(j) +

ν α ˜j

(−˜ z (j) ) 2 2 j=1 j=1

ν α ˜j α ˜j − z˜(j) = 0, = 2 2 j=1

j=1

N

ν α ˜j

z˜(j) z˜(j)T +

ν α ˜j j=1

2

(7.101c)

(−˜ z (j) )(−˜ z (j) )T

α ˜ j z˜(j) z˜(j)T ,

(7.101d)

j=1 N j=1

αj =

ν α ˜j j=1

2

+

ν α ˜j j=1

2

=

ν j=1

α ˜j .

(7.101e)

292

7 Computation of Probabilities

Thus, in order to fulﬁll conditions (7.98a)–(7.98d), the data z˜(1) , . . . , z˜(ν) , ˜ ν must be selected such that α ˜1, . . . , α ν

α ˜ j z˜(j) z˜(j)T = I

(7.102a)

α ˜j = 1

(7.102b)

j=1 ν j=1

α ˜ j > 0, j = 1, . . . , ν.

(7.102c)

Corresponding to (98b ), (7.102a) means that

α ˜ 1 z˜(1) , . . . ,

α ˜ ν z˜(ν)

α ˜ 1 z˜(1) , . . . ,

α ˜ ν z˜(ν)

T = I.

(7.103a)

Hence, we have 1 z˜(j) = d(j) , j = 1, 2, . . . , ν, α ˜j

(7.103b)

where (d(1) , d(2) , . . . , d(ν) ) is an orthogonal ν × ν matrix.

(7.103c)

Example 7.6. In case ν = 2, 1 1 1 −1 , d(2) = √ d(1) = √ 1 2 1 2 are the columns of an orthogonal 2 × 2 matrix. Taking α ˜ 1 = α˜2 = (7.103b) and (7.101a), (7.101b) we get 1 1 1 1 = z (1) = z˜(1) = / √ 1 1 2 1 2

−1 1 1 −1 = z (2) = z˜(2) = / √ 1 1 1 2 2

z (3) = −˜ z (1) z (4) = −˜ z (2) where αj =

−1 = −1 1 , = −1

1 1 1 · = , j = 1, . . . , 4. 2 2 4

1 , from 2

7.7 Computation of the Approximate Probability Functions

293

Example 7.7. Let ν > 1 be an arbitrary integer and deﬁne d(j) := ej , α ˜j =

1 , j = 1, . . . , ν, ν

where e1 , e2 , . . . , eν are the unit vectors of IRν . Then, with (7.103b), distribution (7.101a), (7.101b) reads √ √ √ √ √ √ νe1 , νe2 , . . . , νeν , − νe1 , − νe2 , . . . , − νeν 1 2ν ,

1 2ν

,...,

1 2ν ,

1 2ν ,

1 2ν

,...,

1 2ν .

7.7.3 Convergent Sequences of Discrete Distributions Let z˜(·) denote again a random vector in IRν having a discrete probability distribution Pz˜(·) = µ according to (7.97a), (7.97b). We consider then sequences of i.i.d. (independent and identically distributed) random ν-vectors Z1 , Z2 , . . . , Zl , . . .

(7.104a)

PZl = Pz˜(·) = µ, l = 1, 2, . . . .

(7.104b)

such that Hence, due to (7.97c), (7.97d) we get EZl = E z˜ = zµ(dz) = 0 z )) = zz T ν(dz) = I. cov(Zl ) = cov(˜

(7.104c) (7.104d)

Starting from the discrete distribution µ, we obtain better approximations of the N (0, I)-distribution in IRν by considering the distribution of the ν-random vector 1 S (L) := √ (Z1 + Z2 + . . . + ZL ) for L = 1, 2, . . . . L

(7.105)

Due to (7.104c), (7.104d) we have 1 EZl = 0, ES (L) = √ L l=1 L

cov(S

(L)

L 1 ) = E√ Zl L l=1

(7.106a)

1 √ Zl L l=1

1 EZl ZlT = I. L

L

T

L

=

l=1

(7.106b)

294

7 Computation of Probabilities

Moreover, the distribution µ(L) of S (L) is given by the convolution µ(L) := PS (L) = PZ (L) ∗ PZ (L) ∗ . . . ∗ PZ (L) , 1

2

(7.106c)

L

(L)

where Zj , j = 1, . . . , L, are i.i.d. random ν-vectors having, see (7.97a), (7.97b), (7.97e), the discrete distribution (1) (2) (N ) z√ z√ z√(j) z√ , , . . . , , . . . , L L L L . (7.106d) α1 , α2 , . . . , αj , . . . , αN Consequently, if j1 , j2 , . . . , jl , . . . , jL are arbitrary integers such that jl ∈ {1, . . . , N }, l = 1, . . . , L,

(7.107a)

then the distribution µ(L) of S (L) is the discrete distribution taking the realizations (atoms) L 1 (jl ) z (7.107b) s = s(j1 , j2 , . . . , jL ) := √ L l=1 with the probabilities L

αs :=

αjl .

(7.107c)

s(j1 ,...,jL )=s l=1

Thus, µ(L) , l = 1, 2, . . . , is a sequence of discrete probability distributions of the type (7.97a)–(7.97d). According to the central limit theorem [13,110] it is known that (7.107d) µ(L) weak −→ N (0, I), L → ∞, where “ weak −→ ” means “weak convergence” of probability measures [13, 110]. An important special case arises if one starts with a random ν-vector z˜ having independent components z˜ = (˜ z1 , . . . , z˜ν )T . In this case also the components Zl1 , . . . , Zli , . . . , Zlν , l = 1, 2, . . . of the sequence (Zl ) of i.i.d. random ν-vectors according to (7.104a), (7.104b) are independent. Thus, from (7.105) we obtain that 1 Zl S (L) = √ L l=1 L

7.7 Computation of the Approximate Probability Functions

295

has independent components 1 =√ Zli , i = 1, . . . , ν. L l=1 L

(L)

Si

However, this means that ν

µ(L) = PS (L) = ⊗ PS (L) i=1

(7.108)

i

(L)

is the product of the distributions PS (L) of the components Si , i = 1, . . . , ν, i

of S (L) . If we assume now that the i.i.d. components z˜i , i = 1, . . . , ν, of z˜ have the common distribution 1 1 (7.109a) Pz˜i = ε−1 + ε+1 , 2 2 where εz denotes the one-point measure at a point z, then the i.i.d. compo(L) nents Si , i = 1, . . . , ν, of S (L) have, see (7.106a)–(7.106d), (7.107a)–(7.107c), the Binomial-type distribution PS (L) = i

L L 1 ε 2l−L , i = 1, . . . , ν. l 2L √L

(7.109b)

l=0

The Approximate Probability Functions Γ denotes again the transformed safe domain (in z-space), then the probIf Bs,x ability function ps and its approximate p˜s can be represented by Γ (7.110a) = P s∗Γ z(ω), x ≤ 0 , ps (x) = P z(ω) ∈ Bs,x

Γ = P s∗Γ z˜(ω), x ≤ 0 . (7.110b) p˜s (x) = p˜s (x; µ) := P z˜(ω) ∈ Bs,x Γ If, besides the random ν-vector z = z(ω), also the safe domain Bs,x is approxΓ s,x such that imated, e.g., by a polyhedral set B

Γ ⊂ B Γ or B Γ ⊂ B Γ , B s,x s,x s,x s,x then we have the approximate probability function s,x , µ) := P z˜(ω) ∈ B Γ . p˜s (x) = p˜s (x; B s,x

(7.110c)

296

7 Computation of Probabilities

Fig. 7.14. Discrete approximation of probability distributions

In the present case of approximate random ν-vectors z˜ with a discrete distribution Pz˜ = µ, according to (7.97a), (7.97b) the approximate probability functions read (Fig. 7.14) αj = αj , (7.111a) p˜s (x; µ) = s∗ (z (j) ,x)≤0 Γ Γ s,x , µ) = p˜s (x, B

Γ z (j) ∈Bs,x

αj .

(7.111b)

Γ z (j) ∈Bs,x

Using the central limit theorem [13, 49, 110], we get the following convergence result: Theorem 7.10. Suppose that the approximate discrete distribution µ(L) is deﬁned as described in Sect. 7.3. (a) If P s∗Γ z(ω), x = 0 = 0, then p˜s (x; µ(L) ) → ps (x), L → ∞.

(7.112a)

7.7 Computation of the Approximate Probability Functions

297

L be a convex approximation of B Γ such that P z(ω) ∈ ∂ B Γ = (b) Let B s,x s,x s,x 0, where ∂B denotes the boundary of a set B ⊂ IRν . Then Γ Γ s,x s,x ps (x; B (7.112b) , µ(L) ) − ps (x; B ) ≤ 0 √1L , L → ∞, ˜ where

Γ ) := P z(ω) ∈ B Γ . ps (x, B s,x s,x

A Sequences, Series and Products

In this appendix let IN = IN(1) ∪ . . . ∪ IN(κ) denote a partition of IN fulﬁlling condition (U6) from Chapter 6.3.

A.1 Mean Value Theorems for Deterministic Sequences Here, often used statements about generalized mean values of scalar sequences are given. In this context the following double sequences are of importance. We use the following notations: F := {(an ) : an ∈ IR for

n ∈ IN},

D := {(am,n )m,n : am,n ∈ IR for

0 ≤ m ≤ n},

D+ := {(am,n )m,n ∈ D : it exists n0 ∈ IN am,n > 0

with

for n0 ≤ m ≤ n},

D0 := {(am,n )m,n ∈ D+ : lim am,n = 0 for n→∞

m = 0, 1, 2, . . .}.

Two sequences (am,n ) and (bm,n ) in D+ can be compared if we deﬁne (a) (am,n ) is not greater than (bm,n ), denoted as am,n bm,n , if to any number ε > 0 there exists n0 ∈ IN with 0 < am,n ≤ (1 + ε)bm,n

for all n0 ≤ m ≤ n.

(b) (am,n ) and (bm,n ) are equivalent, denoted as am,n ∼ bm,n , if am,n bm,n

and

bm,n am,n .

Hence, “” is an order, and “∼” is an equivalence relation on D+ having following properties:

302

A Sequences, Series and Products

Lemma A.1. Let (am,n ), (bm,n ), (αm,n ) and (βm,n ) denote sequences from D+ . (a) If am,n αm,n and bm,n βm,n , then am,n + bm,n αm,n + βm,n , am,n bm,n αm,n βm,n , am,n αm,n , am,n s αm,n s for s > 1. βm,n bm,n (b) If am,n ∼ αm,n and bm,n ∼ βm,n , then am,n + bm,n ∼ αm,n + βm,n , am,n bm,n ∼ αm,n βm,n , αm,n am,n ∼ , am,n s ∼ bm,n s for s ∈ IR. bm,n βm,n Proof. In case of (a) there exists to any ε > 0 an index n0 ∈ IN with: 0 < am,n ≤ (1 + ε)αm,n , 0 < bm,n ≤ (1 + ε)βm,n for n0 ≤ m ≤ n. Hence, for n0 ≤ m ≤ n we get 0 < am,n + bm,n ≤ (1 + ε)(αm,n + βm,n ), 0 < am,n bm,n ≤ (1 + ε)2 (αm,n βm,n ), am,n αm,n 0< ≤ (1 + ε)2 , βm,n bm,n 0 < am,n s ≤ (1 + ε)s αm,n s for s ≥ 1. Thus, assertion (a) follows and therefore also assertion (b). The following mean value theorem is a modiﬁcation of a theorem by O. Toeplitz (cf. [65, Sect. 8, Theorem 5]).

Lemma A.2. Let (am,n ), (αm,n ) ∈ D0 and (βm ) ∈ F, and assume lim

n→∞

n

am,n =: a ∈ (0, ∞).

m=1

Deﬁning β := lim inf βn and β¯ := lim sup βn , we get for each index k ∈ IN: n

n

n ' ¯ (a) If αm,n am,n and β¯ ≥ 0, then lim sup αm,n βm ≤ aβ. n

m=k

(b) If am,n αm,n and β > 0, then aβ ≤ lim inf n

(c) If αm,n ∼ am,n , then limextr n

n '

n '

αm,n βm .

m=k

¯ αm,n βm ∈ [aβ, aβ].

m=k

In Lemma A.2 limextr denotes lim sup or lim inf.

A.1 Mean Value Theorems for Deterministic Sequences

303

Proof. Due to (am,n ), (αm,n ) ∈ D0 , we have for any k ∈ IN lim n

k−1

am,n = 0 and

m=1

lim n

k−1

αm,n βm = 0.

(i)

m=1

Hence, all assertions have only to be shown for k = 1.

For the proof of (a) assume β¯ < ∞. Hence, for any ε > 0 there exists an index n0 such that βm ≤ β¯ + ε > 0

and

0 < αm,n ≤ (1 + ε)am,n

for all indices m, n with n0 ≤ m ≤ n. Hence, for n ≥ n0 we ﬁnd n

αm,n βm ≤

m=n0

n

αm,n (β¯ + ε) ≤ (1 + ε)(β¯ + ε)

m=n0

n

am,n

m=n0

n n 0 −1 = (1 + ε)(β¯ + ε) am,n − am,n . m=1

Now, by means of (i) we get lim sup n

n '

m=1

αm,n βm ≤ (1 + ε)(β¯ + ε)a. Since

m=1

ε > 0 was chosen arbitrarily, this shows assertion (a). Using the assumptions of (b), there exists to any ε ∈ (0, β] an index n0 with 0 ≤ β − ε ≤ βm and 0 ≤ am,n ≤ (1 + ε)αm,n for n0 ≤ m ≤ n. Hence, for n ≥ n0 n

αm,n βm ≥

m=n0

n n 0 −1 β−ε ( am,n − am,n ) 1 + ε m=1 m=1

holds. Therefore again by means of (i) we have lim inf n

(1 + ε)−1 a. Thus, we obtain now statement (b).

n '

αm,n βm ≥ (β − ε)

m=1

In the proof of (c) we consider the following three possible cases: Case 1: β > 0. The relation in (c) is fulﬁlled due to (a) and (b). Case 2: β¯ < 0. This case is reduced to the previous case by means of the transition ¯ Here, from (a) we get from βn to −βn . Case 3: β ≤ 0 ≤ β. lim sup n

n

αm,n βm ≤ aβ¯ and

m=k

Hence, also statement (c) holds.

lim sup n

n m=k

αm,n (−βm ) ≤ a(−β).

304

A Sequences, Series and Products

Given a sequence (an ) ∈ F, for m, n ∈ IN0 , m ≤ n, we deﬁne now ⎧ n ⎨ * (1 − a ), for m < n j bm,n := j=m+1 ⎩ 1 , for m = n. About this sequence the following statement can be made: Lemma A.3. (a) If ' (α) there exists an index n0 with an < 1 for n ≥ n0 and (β) an = ∞, n

n '

then (bm,n ) ∈ D0 and lim bm,n am = 1 for k = 1, 2, . . . . n→∞ m=k ' 2 an < ∞ it is bm,n ∼ exp(−(am+1 + . . . + an )). (b) In case n ' (c) If | an |< ∞, then there exists an index n2 with n

limextr bm,n ∈ (0, ∞) n

for

m ≥ n2 .

Proof. In all cases (a), (b) and (c) there exists an index n0 with am < 1 for m ≥ n0 . Hence, it holds

(i) 0 < bm,n ≤ exp(−(am+1 + . . . + an )) for n0 ≤ m < n. ' an = ∞ we have lim bm,n = 0 for m = 0, 1, 2, . . . Consequently, in case n→∞

n

and therefore (bm,n ) ∈ D0 . Hence, for any index k n

bm,n am =

m=k

n

bm,n (1 − (1 − am )) = 1 − bk−1,n −→ 1

for n −→ ∞.

m=k

In case

' n

a2n < ∞ exists to each ε > 0 an index n1 ≥ n0 having 0 < exp(−2aj ) ≤ 1 − a2j for j ≥ n1 , ⎞ ⎛ a2j ⎠ ≤ 1 + ε. exp ⎝2 j≥n1

Hence, for n1 ≤ m < n we ﬁnd bm,n exp(am+1 + . . . + an ) ≥ bm,n (1 + am+1 ) · . . . · (1 + an ) = (1 − a2m+1 ) · . . . · (1 − a2n ) ≥ exp −2(a2m+1 + . . . + a2n ) ≥

1 , 1+ε

or exp(−(am+1 + . . . + an )) ≤ (1 + ε)bm,n , respectively. Therefore (b) follows by means of (i).

A.1 Mean Value Theorems for Deterministic Sequences

Now, let converges, if

'

| an |< ∞. Then (an ) converges to zero and series

n bn , n

305

'

bn

n

= 1, 2, . . . , is deﬁned by 2a , for an ≥ 0 bn := 1 n 2 an , for an < 0.

There exists an index n2 ≥ n0 with 0 < exp(−bn ) ≤ 1 − an

for n ≥ n2 .

Hence, using (i), assertion (c) follows. A simple application of Lemmas A.2 and A.3 is given next: Lemma A.4. Let T1 ∈ IR and (an ), (bn ) ∈ F with following ' properties: (i) an = ∞. Then There exists an index n0 with an ∈ [0, 1) for n ≥ n0 , (ii) n

limextr Tn ∈ [lim inf bn , lim sup bn ] for sequence n

n

n

Tn+1 := (1 − an )Tn + an bn ,

n = 1, 2, . . . .

Proof. By means of complete induction, formula Tn+1 = b0,n T1 +

n

(bm,n am )bm

for n = 1, 2, . . .

m=1

is obtained. Hence, the assertion follows from Lemmas A.3(a) and A.2(c).

Now, consider to given number a ∈ IR the following double sequence in D+ : ⎧ n ⎨ * (1 − a ), if m < n j Φm,n (a) := j=m+1 ⎩ 1 , if m = n. Moreover, let IM denote a subset of IN such that the limit q := lim

n→∞

| {1, . . . , n} ∩ IM | n

exists and is positive. Then we get the following result: Lemma A.5. For any a ∈ IR holds: m a ) , n | IM0,m | m ∼ , (b) n | IM0,n | (c) Φm,n (a) ∼ Φ|IM0,m |,|IM0,n | (a), q 1 Φm,n (a) = (d) If a > 0, then lim n→∞ m a

(a) Φm,n (a) ∼ (

m∈ IMk,n

for each k ∈ IN0 .

306

A Sequences, Series and Products

Proof. Due to representation exp(−a(

1 m 1 + . . . + )) = exp(a(cm − cn ))( )a m+1 n n

with the convergent sequence (cf. [65]) ck :=

1 1 + . . . + − ln (k), 1 k

k = 1, 2, . . . ,

by means of Lemma A.3(b) we have

m a 1 1 + ... + ∼ Φm,n (a). ∼ exp −a n m+1 n

Since the limit q is positive, to any ε > 0 there exists an index n0 such that 0<

| IM0,k | q ≤ ≤ (1 + ε)q 1+ε k

for

k ≥ n0 .

Hence, for n0 ≤ m ≤ n we get | IM0,m | m m ≤ ≤ (1 + ε)2 , (1 + ε)2 n | IM0,n | n and therefore (b) holds. Due to (a), (b), and Lemma A.1(b) we have m a |IM | a 0,m Φm,n (a) ∼ ∼ ∼ Φ|IM0,m |,|IM0,n | (a). n |IM0,n |

(i)

For a > 0, due to Lemma A.3(a) we have lim

n→∞

=

Φ|IM0,m |,|IM0,n | (a)

m∈ IMk,n

1 lim a n→∞

1 | IM0,m |

|IM0,n |

ΦM,|IM0.n | (a)

M =|IM0,k |+1

1 a = . M a

(ii)

Note that the index transformation in (ii) is feasible. Indeed, m ∈ IMk,n := {k + 1, . . . , n} ∩ IM ⇐⇒ | IM0,m |∈ {| IM0,k | +1, . . . , | IM0,n |}. Hence, by means of (i) and the deﬁnition of q it is Φ|IM0,m |,|IM0,n | (a)

1 1 ∼ Φm,n (a) . | IM0,m | qm

Therefore, (ii) and Lemma A.2(c) yields now assertion (d).

A.1 Mean Value Theorems for Deterministic Sequences

307

Now let (An ) ∈ F denote a given sequence with lim An = A(k)

for k = 1, . . . , κ,

n∈IN(k)

A := q1 A(1) + . . . + qκ A(κ) > 0. Here again q1 , . . . , qκ denote the limit parts of sets IN(1) , . . . , IN(κ) in IN according to (U6). Then double sequence ⎧ n ⎨ * (1 − Aj ), for m < n j bm,n := j=m+1 ⎩ 1 , for m = n is in D+ and has the following properties: Lemma A.6. (a) For any δ ∈ (0, A) we have Φm,n (A − δ). (b) lim bm,n = 0 for m = 0, 1, 2, . . .. n→∞ ' 1 (c) For each k ∈ IN0 it is lim bm,n m = n→∞ m∈ IM k,n

Φm,n (A + δ) bm,n

q A.

Proof. We have representation ⎛ ⎞ ⎛ (k) ⎞ |IN0,l | κ A ⎜ ⎝1 − l l ⎠⎟ bm,n = ⎝ ⎠, (k) | IN0,l | (k) k=1

(i)

l∈ INm,n

and for arbitrary numbers b1 , . . . , bκ ∈ IR it is due to Lemmas A.1 and A.5 ⎞ ⎛ ⎛ ⎞ (k) |IN0,n |

κ κ bk ⎟ bk ⎜ ⎜ ⎟ 1− 1− ⎠ ⎝ ⎝ ⎠= (k) L | IN0,l | (k) (k) k=1 k=1 L=|IN0,m |+1

l∈ INm,n

= ∼

κ

Φ|IN(k)

k=1 κ

(k) 0,m |,|IN0,n |

(

k=1

(bk ) ∼

κ

Φm,n (bk )

k=1

m bk ) ∼ Φm,n (b1 + . . . + bk ).

n

For each δ ∈ (0, A) there exists an index n0 with (k)

bk := qk A(k) −

|IN0,l | δ δ ≤ Al ≤ qk A(k) + =: ¯bk k l k

for all n0 ≤ l ∈ IN(k) and k = 1, . . . , κ. Hence, due to (i) and (ii) Φm,n (A + δ) bm,n Φm,n (A − δ).

(ii)

308

A Sequences, Series and Products

Since A ± δ > 0, Lemma A.5(a) yields lim bm,n = 0

for

n→∞

m = 0, 1, 2, . . . ,

and from Lemmas A.5(d) and A.2 one obtains ' q 1 bm,n m ≤ A+δ ≤ limextr n

m∈ IMk,n

q A−δ .

This holds true for any δ ∈ (0, A), and therefore (c) holds. Applying now Lemma A.6, the limit of sequence Tn+1 := (1 −

An fn )Tn + + gn n n

for n = 1, 2, . . .

can be calculated. Here, T1 ∈ IR and (fn ), (gn ) ∈ F with gn convergent. lim fn = f (k) for k = 1, . . . , κ, n∈IN(k)

n

Lemma A.7. For sequence (Tn ) equation lim Tn =

n→∞

κ qk f (k) k=1

A

holds. Proof. With abbreviations g0 := 0 and Fn := fn − An (g0 + . . . + gn−1 ), Sn := Tn − (g0 + . . . + gn−1 ), we get Sn+1 = Tn+1 − (g0 + . . . + gn ) 1 = Tn + (fn − An Tn ) + gn − (g0 + . . . + gn ) n 1 = Tn − (g0 + . . . + gn−1 ) + (Fn − An Sn ) n

1 An Sn + Fn for n = 1, 2, . . . . = 1− n n Hence, by means of complete induction we get Sn+1 = b0,n S1 +

κ

k=1 m∈ IN(k) 0,n

bm,n

1 Fm . m

A.2 Iterative Solution of a Lyapunov Matrix Equation

Due to the assertion we have with G :=

'

309

gn

n

lim Fm = f (k) − A(k) G

m∈IN(k)

for k = 1, . . . , κ.

Now, from Lemmas A.6(b), (c) and A.2(c) follows lim Sn =

n→∞

κ qk k=1

A

(f (k) − A(k) G) =

κ qk f (k) k=1

A

− G.

Finally, we get our assertion by means of the relation Tn = Sn + (g0 + . . .

+gn−1 ). The following simple mean value theorem is used a few times: Lemma A.8. Let (fn ) be selected as above. Then 1 (f1 + . . . + fn ) −→ qk f (k) , n → ∞. n κ

k=1

Proof. Due to representation Tn+1

1 := (f1 + . . . + fn ) = n

the assertion follows from Lemma A.7.

1 fn Tn + , 1− n n

A.2 Iterative Solution of a Lyapunov Matrix Equation In Sect. 6.5 (or more precisely in the proof of Lemma 6.13) the following recursive matrix equation is of interest: For n = 1, 2, . . . let Vn+1 =

I−

T 1 1 1 Bn Vn I − Bn + Cn n n n

where V1 and Cn denote symmetric and Bn real ν × ν-matrices with the following properties: lim Cn = C (k) , lim Bn = B (k) , B (k) + B (k)T ≥ 2u(k) I

n∈IN(k)

n∈IN(k)

for k = 1, . . . , κ, where u(1) , . . . , u(κ) denote reals with u ¯ := q1 u(1) + . . . + qκ u(κ) > 0.

310

A Sequences, Series and Products

Convergence of matrix sequence (Vn ) is guaranteed by the following result: Lemma A.9. The limit V := lim Vn exists and is unique solution of Lyan→∞ punov equation BV + V B T = C,

(∗)

where we put B := q1 B (1) + . . . + qκ B (κ)

C := q1 C (1) + . . . + qκ C (κ) .

and

For the proof of this theorem we need further auxiliary results. First of we state the following remark: Remark A.1. If for n ∈ IN holds an I Bn + BnT ≥ 2

for Bn ≤ bn

with numbers an , bn , then (cf. Lemma 6.6) 2 I − 1 Bn ≤ 1 − 2un with n n

un := an −

1 2 b . 2n n

an ) and (bn ) can be chosen such that Due to the assumptions about (Bn ), ( lim un = u(k)

n∈IN(k)

for

k = 1, . . . , κ.

Lemma A.10. For n ∈ IN let T1 , En and Fn denote symmetric ν ×ν-matrices having -n+1 := 1 (E1 + . . . + En ) −→ 0 and Fn < ∞. E n n Then Tn → 0, n → ∞, if the sequence (Tn ) is deﬁned by

T 1 1 1 Tn+1 := I − Bn Tn I − Bn + En + Fn , n = 1, 2, . . . . n n n (1)

(2)

Proof. With matrices T1 := T1 , T1 := 0 and

T 1 1 (1) (1) Tn+1 := I − Bn Tn I − Bn + Fn , n n

T 1 1 1 (2) + En Tn+1 := I − Bn Tn(2) I − Bn n n n (1)

(2)

we can write Tn = Tn + Tn for n = 1, 2, . . . . Because of the above remark we get now

2 (1) Tn+1 ≤ 1 − un Tn(1) + Fn . n (1)

Hence, Lemma A.7 yields Tn −→ 0 .

A.2 Iterative Solution of a Lyapunov Matrix Equation

-n+1 Due to E

311

-n + 1 En it is = 1 − n1 E n

T 1 1 -n -n ) + E -n I − 1 Bn Bn ) (Tn(2) − E −1 E + n n n

T 1 1 (2) = I − Bn (Tn − En ) I − Bn n n 3 4 1 1 - T -n B T . Bn E + E n − (Bn En + En Bn ) + n n n Hence, the above remark gives us

(2) -n+1 ≤ 1 − 2 un T (2) − E -n Tn+1 − E n n : -n 9 E 1 2 1 + 2Bn + Bn . + n n -n+1 = (I − Tn+1 − E (2)

-n −→ 0 follows. Finally, because of the Again, using Lemma A.7, Tn − E (2) assumptions about En also Tn converges to 0. (2)

Proof (of Lemma A.9). Having a solution V of (∗) it holds

T 1 1 1 (Vn − V ) + V I − Bn + Cn − V Vn+1 − V = I − Bn n n n

T 1 1 = I − Bn (Vn − V ) I − Bn n n 8 17 1 + Cn − (Bn V + V BnT ) + 2 Bn V BnT . n n Moreover, due to Lemma A.8 and the above assumptions we obtain 1 (B1 + . . . + Bn ) −→ qk B (k) =: B, n κ

k=1 κ

1 (C1 + . . . + Cn ) −→ qk C (k) =: C. n k=1

Hence, due to Lemma A.10 we have Vn −→ V, n → ∞.

The following theorem guarantees the uniqueness of the solution V of (∗). Lemma A.11 (Lyapunov). If −B is stable, that is Re(λ) > 0 for any eigenvalue λ of B, then (∗) has an unique solution V . This theorem can be found, e.g., in [8]. Due to relations 1 λmin (B + B T ) ≥ u ¯ > 0 for all eigenvalues λ of B, 2 in our situation −B is stable. Re(λ) ≥

B Convergence Theorems for Stochastic Sequences

The following theorems giving assertions about “almost sure convergence,” “convergence in the mean” and “convergence in distribution” of random sequences, are of fundamental importance for Chap. 6. Let, as already stated in Sect. 6.3, (Ω, A, P ) denote a probability space, A1 ⊂ A2 ⊂ . . . ⊂ A σ-algebras, En (·) := E(· | An ) the expectation with respect to An and IN = IN(1) ∪ . . . ∪ IN(κ) a disjunct partition of IN fulﬁlling condition (U6).

B.1 A Convergence Result of Robbins–Siegmund The following lemma due to H. Robbins and D. Siegmund, cf. [126], gives an assertion about almost sure convergence of nonnegative random sequences (Tn ). (1) (2) Let Tn , αn , αn and αn denote nonnegative An -measurable random variables with En Tn+1 ≤ (1 + αn(1) )Tn − αn + αn(2) for n = 1, 2, . . . , where

'

a.s.

(i)

αn < ∞ a.s. for i = 1, 2.

n

Lemma B.1. The limit lim Tn < ∞ exists, and it is n→∞

'

αn < ∞ a.s.

n

B.1.1 Consequences In special case αn = an Tn

for n = 1, 2, . . .

with nonnegative An -measurable random variables an , fulﬁlling

' n

a.s., we get this result:

an = ∞

314

B Convergence Theorems for Stochastic Sequences

Lemma B.2. Tn −→ 0, n → ∞ a.s. Proof. According to Lemma B.1 we know that T := lim Tn exists a.s. and n→∞ ' ' an Tn < ∞ a.s. Hence, due to an = ∞ a.s. we obtain the proposition. n

n

Now we abandon the nonnegativity condition of an . To n ∈ IN, let 0 ≤ Sn and An denote An -measurable random numbers with

An (1) + αn Sn + αn(2) a.s. En Sn+1 ≤ 1 − n for n = 1, 2, . . ., where we demand for (An ): (a)

lim An = A(k) a.s. for k = 1, . . . , κ with numbers A(1) , . . . , A(κ) ∈ IR,

n∈IN(k)

(b) A := q1 A(1) + . . . + qκ A(κ) > 0. Since for large n sequence (An ) is positive “in the mean,” (Sn ) converges a.s. to zero. Lemma B.3. Sn −→ 0, n → ∞ a.s. Proof. To arbitrary ε ∈ (0, A), let Bn := qk (A(k) − ε)

n (k)

| IN0,n |

for indices n ∈ IN(k) , where k ∈ {1, . . . , κ}. Then, for all k = 1, . . . , κ and n ∈ IN we get lim Bn = A(k) − ε, An ≥ Bn − ∆Bn , n∈IN(k)

with the An -measurable random variable ∆Bn := max{0, Bn −An } ≥ 0. Since Sn is nonnegative, it follows

Bn + βn(1) Sn + αn(2) , En Sn+1 ≤ 1 − n (1)

(1)

n . For suﬃciently large indices n it is ∆Bn = 0 a.s., where βn := αn + ∆B ' ∆Bn n and therfore < ∞ a.s. Hence, it holds n

n

0 ≤ βn(1) ,

βn(1) < ∞

a.s.

n

Furthermore, there exists an index n0 with 1−

Bn 1 A−ε , 1− ≥ n n 2

for

n ≥ n0 .

B.1 A Convergence Result of Robbins–Siegmund

315

Hence, for n ≥ n0 exists the An -measurable positive random variable (1) 1 − A−ε 1 − A−ε βn n n pn := 1− . = (1) (1) 1 − Bnn 1 − Bnn + βn 1 − Bnn + βn Now, let Pn0 := 1 and for n ≥ n0 Pn+1 := pn0 · . . . · pn , S¯n := Pn Sn , βn(2) := Pn+1 αn(2) . (2) Then Pn+1 , S¯n and βn are nonnegative An -measurable random variables, where

A−ε ¯ ¯ Sn + βn(2) , n ≥ n0 . En Sn+1 ≤ 1 − n

Now, only the following relations have to be veriﬁed: lim sup Pn < ∞,

lim inf Pn > 0 n

n

a.s.

Using the ﬁrst inequality it follows βn(2) < ∞ a.s., n

and therefore (S¯n ) converges to zero a.s. due to Lemma B.2. Hence, due to the second inequality and the deﬁnition also (Sn ) converges a.s. to 0. For n ≥ n0 we have 1− 1

A−ε n − Bnn

(1 − 2βn(1) ) ≤ pn ≤

1− 1

A−ε n − Bnn

.

As in the proof of Lemma A.6 it can be shown that for n0 ≤ m < n it holds ⎛ ⎞ ⎛ (k) ⎞

|IN0,l | n κ B Bj ⎜ ⎝1 − l l ⎠⎟ = 1− ⎝ ⎠ (k) j | IN | (k) j=m+1 k=1

l∈INm,n

0,l

⎞ ⎛ κ (k) qk (A − ε) ⎟ ⎜ 1− = ⎠ ∼ Φm,n (A − ε), ⎝ (k) IN0,l (k) k=1 l∈INm,n

or stated in another way n

1−

j=m+1

1

A−ε j B − jj

∼ 1.

316

B Convergence Theorems for Stochastic Sequences

Moreover, according to Lemma A.3(c) for an index n2 we have n

lim inf n

(1)

(1 − 2βj ) > 0

a.s.

j=n2 +1

The last two relations and the above inequality for pn ﬁnally yield limextr Pn ∈ (0, ∞) n

a.s.

B.2 Convergence in the Mean For real An -measurable random variables Tn , An , Bn , Cn and vn assume for any n ∈ IN and k = 1, . . . , κ En Tn+1 ≤ (1 −

An 1 )Tn + (Bn vn + Cn ) a.s., n n

as well as Tn , Bn , vn ≥ 0, lim inf An ≥ A(k) , lim sup Bn ≤ B (k) , lim sup Cn ≤ C (k) n∈IN(k)

n∈IN(k)

a.s.,

n∈IN(k)

lim sup Evn ≤ w(k) , A := q1 A(1) + . . . + qκ A(κ) > 0 n∈IN(k)

with real numbers A(1) , . . . , A(κ) , B (1) , . . . , B (κ) , C (1) , . . . , C (κ) and w(1) , . . . , w(κ) . Now, for sequence (Tn ) the following convergence theorem holds: Lemma B.4. To any ε0 > 0 there exists a set Ω0 ∈ A such that ¯0 ) < ε0 , (a) P(Ω (b) lim sup ETn IΩ0 ≤ n∈IN

κ qk (B (k) w(k) + C (k) )

A

k=1

.

For the proof of this theorem the following theorem of Egorov (cf., e.g., [67] page 285) is needed: Lemma B.5. Let U, U1 , U2 , . . . denote real random variables having lim sup Un ≤ U

a.s.

n

Then to any positive number ε0 there exists a set Ω0 ∈ A with following properties: ¯0 ) < ε0 , (a) P(Ω

B.2 Convergence in the Mean

317

(b) lim sup Un ≤ U is fulﬁlled uniformly on Ω0 , that is: To any number ε > 0 n

there exists an index n0 ∈ IN such that Un (ω) ≤ U (ω) + ε

for all n ≥ n0 and ω ∈ Ω0 .

Proof (of Lemma B.4). Let ε0 > 0 be chosen arbitrarily. According to Lemma B.5 there exists a set Ω0 ∈ A such that inequalities lim inf An ≥ A(k) , lim sup Bn ≤ B (k) , lim sup Cn ≤ C (k) n∈IN(k)

n∈IN(k)

n∈IN(k)

for all k = 1, . . . , κ hold uniformly on Ω0 . Choose now an arbitrary number ε ∈ (0, A). Then there exists an index n0 such that for all n ≥ n0 and n ∈ IN(k) on Ω0 holds: An ≥ A(k) − ε =: an , Bn ≤ B (k) + ε =: bn , Cn ≤ C (k) + ε =: cn . Hence, for the An -measurable sets {An ≥ an } ∩ {Bn ≤ bn } ∩ {Cn ≤ cn }, Ωn := Ω ,

for for

n ≥ n0 n < n0

we have Ω0 ⊂ Ω1 ∩ . . . ∩ Ωn =: Ω (n) ∈ An . Let now S1 := T1 , and for n ≥ 1 deﬁne 7 an an 8 Sn + Tn+1 − 1 − Tn IΩ (n) . Sn+1 := 1 − n n Then, due to Ω (1) ⊃ Ω (2) ⊃ . . . ⊃ Ω (n) we get T , on Ω (n) Sn+1 = n+1 an 1 − n Sn , otherwise. It is Ωn ⊃ Ω (n) ∈ An and therefore 7 an an 8 Sn + En Tn+1 − 1 − Tn IΩ (n) En Sn+1 = 1 − n n an 1 an 1 Sn + (Bn vn + Cn )IΩ (n) ≤ 1 − Sn + (bn vn + cn ). ≤ 1− n n n n Taking expectations, for n = 1, 2, . . . we get an 1 ESn + (bn Evn + cn ). ESn+1 ≤ 1 − n n Now, applying Lemma A.7 it follows (k) κ q + ε)w(k) + C (k) + ε k (B . lim sup ESn ≤ A−ε n k=1

318

B Convergence Theorems for Stochastic Sequences

Moreover, because of the above statements, due to Sn ≥ 0 and Ω0 ⊂ Ω (n) we ﬁnd ESn+1 ≥ ESn+1 IΩ0 = ETn+1 IΩ0 . Since ε ∈ (0, A) was chosen arbitrarily, last two relations yield lim sup ETn IΩ0 ≤ n

κ qk (B (k) w(k) + C (k) )

A

k=1

.

B.3 The Strong Law of Large Numbers for Dependent Matrix Sequences For all n ∈ IN let denote an real random numbers and Tn , Cn real random µ × ν-matrices with the following properties: T1 A1 -measurable, an An -measurable, Cn An+1 -measurable, an = ∞ a.s., an ∈ [0, 1),

n

a2n En Cn

− En Cn 2 < ∞, En Cn −→ C, n → ∞ a.s.,

n

with a further matrix C. For the averaged sequence Tn+1 := (1 − an )Tn + an Cn , n = 1, 2, . . . , the following strong law of large numbers holds: Lemma B.6. Tn −→ C, n → ∞ a.s. (1)

Proof. With abbreviations T1

(2)

:= T1 , T1

:= 0 and

(1)

Tn+1 := (1 − an )Tn(1) + an (Cn − En Cn ), (2)

Tn+1 := (1 − an )Tn(2) + an En Cn , (1)

(2)

we write Tn = Tn + Tn for n = 1, 2, . . .. Hence, according to Lemma A.4 (2) it is lim Tn = C a.s. Let now 0 = h ∈ IRν be chosen arbitrarily. With n

(1)

Sn := Tn h2 we then get

2 Sn+1 = (1 − an )2 Sn + a2n (Cn − En Cn )h + 2(1 − an )an Tn(1) h, (Cn − En Cn )h.

Therefore, by taking expectations we ﬁnd

2 En Sn+1 = (1 − an )2 Sn + a2n En (Cn − En Cn )h ≤ (1 − an )Sn + h2 a2n En Cn − En Cn 2 .

Now, due to Lemma B.2 we get Sn −→ 0 a.s. Since h is chosen arbitrarily, it (1)

follows Tn −→ 0 a.s. and therefore due to the above also Tn −→ C a.s.

B.4 A Central Limit Theorem for Dependent Vector Sequences

319

B.4 A Central Limit Theorem for Dependent Vector Sequences For indices m ≤ n, let Am,n denote real deterministic µ × ν-matrices, and let Un : Ω −→ IRν be An+1 -measurable random vectors with the following properties: (a) For n = 1, 2, . . ., assume En Un ≡ 0 a.s.

(K0)

(b) It exists a deterministic matrix V with n

stoch

T Am,n Em (Um Um )ATm,n −→ V.

(KV)

m=1

(c) For arbitrary > 0 assume n

stoch Am,n 2 Em Um 2 I{ Am,n Um > } −→ 0.

(KL)

m=1

Then, sequence (

n '

Am,n Um )n holds an asymptotic normal distribution:

m=1

Lemma B.7. (

n '

Am,n Um )n converges in distribution to a Gaussian dis-

m=1

tributed random vector U with EU = 0 and cov(U ) = V . Proof. Let U denote an arbitrary N (0, V )-distributed random vector, and let 0 = h ∈ IRµ be chosen arbitrarily. Now, for m ≤ n deﬁne Ym,n := hT Am,n Um , Yn :=

n

Ym,n , Y := hT U.

m=1

Due to the “Cram´er-Wold-Devise” (cf., e.g., [43], Theorem 8.7.6), only the convergence of (Yn ) in distribution to Y has to be shown. Due to (K0), (Ym,n , Am+1 )m≤n is a martingale diﬀerence scheme. Therefore, according to the theorem of Brown (cf. [43], Theorem 9.2.3), the following conditions have to be veriﬁed: n

stoch

2 Em Ym,n −→ hT V h,

(KV1)

m=1 n

stoch 2 Em Ym,n I{|Ym,n |>δ} −→ 0 for all

δ > 0.

(KL1)

m=1 2 T = hT Am,n Em (Um Um )ATm,n h. (KV1) follows from (KV), since Em Ym,n Moreover, it is | Ym,n |≤ hAm,n Um ≤ hAm,n Um , and therefore for δ > 0 we get

320

B Convergence Theorems for Stochastic Sequences 2 Em (Ym,n I{|Ym,n |>δ} ) ≤ h2 Am,n 2 Em Um 2 I{ Am,n Um >

Hence, (KL) is suﬃcient for (KL1).

δ } h

.

C Tools from Matrix Calculus

We consider here real matrices of the size ν × ν.

C.1 Miscellaneous First we give inequalities about the error occurring in the inversion of a perturbed matrix, cf. [68]. Lemma C.1. Perturbation lemma. Let A and ∆A denote matrices such that A is regular and A−1 0 · ∆A0 < 1, where · 0 is an arbitrary matrix norm such that I0 = 1. Then, also A + ∆A is regular and A−1 20 · ∆A0 , 1 − A−1 0 · ∆A0 A−1 0 . (b) (A + ∆A)−1 0 ≤ 1 − A−1 0 · ∆A0

(a) (A + ∆A)−1 − A−1 0 ≤

A consequence of Lemma C.1 is given next: Lemma C.2. Let (An ) and (Bn ) denote two matrix sequences with the following properties: (i) An regular, A−1 n ≤ L < ∞ for n ∈ IN, (ii) Bn − An −→ 0. Then there is an index n0 such that (a) Bn is regular for n ≥ n0 , (b) Bn−1 − A−1 n −→ 0.

322

C Tools from Matrix Calculus

Proof. Due to assumption (ii) there is an index n0 such that Bn − An <

1 , n ≥ n0 . L

Hence, because of (i) we get A−1 n Bn − An < 1. According to Lemma C.1, matrix Bn is regular and Bn−1 − A−1 n ≤

L2 Bn − An . 1 − LBn − An

Using (ii), assertion (b) follows.

If a rank-1-matrix is added to a regular matrix, then the determinant and the inverse of the new matrix can be computed easily. This is contained in the Sherman–Morrison-lemma: Lemma C.3. If e, d ∈ IRν and A is a regular matrix, then (a) det(A + edT ) = det(A)α, (b) If α = 0, then (A + edT )−1

α := 1 + dT A−1 e. 1 = A−1 − (A−1 e)(dT A−1 ). α

where

The proof of Lemma C.3 can be found, e.g., in [68]. A lower bound for the minimum eigenvalue of a positive deﬁnite matrix is given next, cf. [9], page 223: Lemma C.4. If A is a symmetric, positive deﬁnite ν × ν-matrix, then λmin (A) > det(A)(

ν − 1 ν−1 ) . tr(A)

Proof. If 0 < λ1 ≤ . . . ≤ λν denotes all eigenvalues of A, then tr(A) = λ1 + . . . + λn , det(A) = λ1 · . . . · λn . Hence det(A) = λ 2 · . . . · λν ≤ λ1

λ 2 + . . . + λν ν−1

ν−1

<

tr(A) ν−1

ν−1 .

C.2 The v. Mises-Procedure in Case of Errors For a symmetric, positive semideﬁnite ν×ν matrix A, the maximum eigenvalue ¯ := λmax (A) of A can be determined recursively by the v. Mises procedure, λ cf. [151], page 203:

C.2 The v. Mises-Procedure in Case of Errors

323

Given a vector 0 = u1 ∈ IRν , deﬁne recursively for n = 1, 2, . . . vn vn := Aun and un+1 := . vn For a suitable start vector u1 , sequence (un ) converges to an eigenvector of ¯ ¯ and un , vn converges to λ. the eigenvalue λ Suppose now that approximates A1 , A2 , . . . of A are given. Corresponding to the above procedure, for a start vector u1 ∈ IRν with u1 = 1, the sequences (un ) and (vn ) are determined by the following rule: vn for n = 1, 2, . . . . (∗) vn := An un and un+1 := vn We suppose that A and An fulﬁll the following properties: (a) ν > 1, and 0 = A symmetric and positive semideﬁnite, (b) An −→ A. ¯ provided that an Based on the following lemma, un , vn converges to λ, additional condition holds. ¯ the eigenspace of the maximum Let denote next U := {x ∈ IRν : Ax = λx} ν ¯ eigenvalue λ of A. Each vector z ∈ IR can then be decomposed uniquely as follows: z = z (1) + z (2) , where z (1) ∈ U and z (2) ∈ U ⊥ := {y ∈ IRν : y, x = 0 for all x ∈ U }. Now the already mentioned main result of this part of the appendix can be presented: Lemma C.5. Let (un ) and (vn ) be determined according to (∗). Then the following assertions are equivalent: ¯ (a) un , vn −→ λ. (b) There is an index n0 ∈ IN such that sup An − A < n≥n0

¯ (1) λ−µ 3 un0 .

Here, µ := λν−1 (cf. the proof of Lemma C.4) or µ := 0 if U = IRν . The (1) vector un0 denotes the projection of un0 onto U . In the standard case An ≡ A the second condition in Lemma C.5 means that u(1) n0 > 0 for an index n0 ∈ IN. ¯ This is the known condition for the convergence of un , vn to λ. For the proof of Lemma C.5 we need several auxiliary results. Put q := µ ¯ ∈ [0, 1), and deﬁne for n ∈ IN λ " (2) (1)

un

, if un = 0 tn := u(1) n

∞ , else, 1 dn := ¯ An − A and en := sup dk . λ k≥n

324

C Tools from Matrix Calculus

(1) Then, un = √ 1

1+t2n

, and the following lemma holds:

Lemma C.6. If dn

¯ (1) , 1 + t2n < 1 ⇐⇒ An − A < λu n

then tn+1

qtn + dn 1 + t2n ≤ . 1 − dn 1 + t2n

Proof. Because of vn = An un , decomposition of vn yields: ¯ (1) + P (An − A)un , vn(1) = P (An un ) = P Aun + (An − A)un = λu n vn(2) = Q(An un ) = Au(2) n + Q(An − A)un . Here, P is the projection matrix describing the projection of z ∈ IRν onto z (1) ∈ U , and Q := I − P . Taking the norm, one obtains ¯ (1) − An − A, vn(1) ≥ λu n (2) vn ≤ µu(2) n + An − A,

(i) (ii)

if the following relations are taken into account: (2) Au(2) n ≤ µun ,

P , Q ≤ 1,

un = 1.

¯ (1) From (i), (ii) and the assumption λu n − An − A > 0 we ﬁnally get (1) vn = 0 and (2)

tn+1 := =

un+1 (1)

un+1

(2)

=

vn (1)

vn

(2)

≤

µun + An − A ¯ (1) λu n − An − A

qtn + dn 1 + t2n . 1 − dn 1 + t2n

In the next step conditions are given such that (tn ) converges to zero: Lemma C.7. Assume that there is a number T ∈ (0, ∞) and an index n0 ∈ IN with (a) en0 ≤ (T ) := (b) tn0 ≤ T. Then tn → 0, n → ∞.

(1 − q)T √ , (1 + T ) 1 + T 2

C.2 The v. Mises-Procedure in Case of Errors

325

Proof. By induction we show ﬁrst that tn ≤ T,

n ≥ n0 .

(∗∗)

Given n ≥ n0 , suppose tn ≤ T . Because of (a) we have ; (1 − q)T (1 − q)T 1 + t2n 2 2 < 1. dn 1 + tn ≤ en0 1 + tn ≤ ≤ 2 1+T 1+T 1+T Hence, Lemma C.6 yields tn+1

qtn + dn 1 + t2n qT + dn 1 + t2n ≤ ≤ ≤T 1 − dn 1 + t2n 1 − dn 1 + t2n

and (∗∗) holds. Since (dn ) converges to zero, there is an index n1 ≥ n0 with 1 − dn

1 + T2 ≥

1+q , 2

n ≥ n1 .

By means of Lemma C.6 and (∗∗), for n ≥ n1 we get √ √ qtn + dn 1 + T 2 2q 2 1 + T2 t . = + d tn+1 ≤ n n 1+q 1+q 1+q 2 Because of

2q 1+q

∈ [0, 1) and dn −→ 0, we ﬁnally have tn −→ 0.

The meaning of “tn −→ 0” is shown in the next lemma: Lemma C.8. The following assertions are equivalent: ¯ (a) un , vn −→ λ, (b) u(1) n −→ 1, (c) tn −→ 0. Proof. Because of un = (1 + t2n )− 2 the equivalence of (b) and (c) is clear. For n ∈ IN we have un , vn = uTn A + (An − A) un (1)

1

(2) T (1) (2) T = (u(1) n + un ) A(un + un ) + un (An − A)un

¯ (1) 2 + u(2)T Au(2) + uT (An − A)un , = λu n n n n and T un (An −A)un ≤ An −A

,

(2) 2 0 ≤ u(2)T Au(2) n n ≤ µun .

326

C Tools from Matrix Calculus

Thus, for each n ∈ IN we get 2 u(1) n − dn ≤

un , vn 2 (2) 2 ≤ u(1) n + qun + dn ¯ λ 2 = q + (1 − q)u(1) n + dn .

Due to dn −→ 0 and q ∈ [0, 1) we obtain now also the equivalence of (a) and (b).

Now Lemma C.5 can be proved. Proof (of Lemma C.5). If ¯ un , vn −→ λ,

(i)

then because of An −→ A and Lemma C.8 there is an index n0 such that sup An − A <

n≥n0

¯−µ 1 − q (1) λ u(1) un0 . n0 ⇐⇒ en0 < 3 3

(ii)

Conversely, suppose that this condition holds. Then there are two possibilities: (1) Case 1: en0 = 0. Then un0 = 0, tn0 < ∞, and also en0 = 0 < (tn0 ) with function (t) from Lemma C.7. According to this Lemma we get then tn−→ 0. Case 2: en0 > 0. Due to (ii) it is 0 < en0 ≤

1 1−q < √ (1 − q) = (1). 3 2 2

Since (t) is continuous on [1, ∞) and decreases monotonically to zero, there is a number T ∈ [1, ∞) such that en0 = (T ). Because of (t) · t ≥

1−q 3

(iii)

for t ≥ 1, with (ii) and (iii) we ﬁnd

1−q 1 − q (1) 1 − q (2) un0 ≤ ≤ (T )T = en0 T < un0 T, 3 3 3 and therefore

(2)

tn0 :=

un0 (1)

un0

≤ T.

(iv)

From (iii) and (iv) with Lemma C.7 we obtain again tn −→ 0. Due to Lemma C.8, condition “lim tn = 0” implies (i) which concludes n now the proof.

References

1. Anbar D. (1978) A Stochastic Newton–Raphson Method. Journal of Statistical Planing and Inference 2: 153–163 2. Augusti G., Baratta A., Casciati F. (1984) Probabilistic Methods in Structural Engineering. Chapman and Hall, London, New York 3. Aurnhammer A. (2004) Optimale Stochastische Trajektorienplanung und Regelung von Industrierobotern. Fortschrittberichte VDI, Reihe 8, Nr. 1032. VDI-Verlag GmbH, D¨ usseldorf 4. Barner M., Flohr F. (1996) Analysis II. Walter de Gruyter, Berlin, New York 5. Barthelemy J.-F., Haftka R. (1991) Recent Advances in Approximation Concepts for Optimum Structural Design. In: Rozvany G. (ed.), Optimization of Large Structural Systems. Lecture Notes, Vol. 1: 96–108, NATO/DFG ASI, Berchtesgaden, Sept. 23–Oct. 4 6. Bastian G. et al. (1996) Theory of Robot Control. Springer, Berlin Heidelberg New York 7. Bauer H. (1996) Probability Theory. De Gruyter, Berlin 8. Bellman R. (1970) Introduction to Matrix Analysis. McGraw-Hill, New York 9. Beresin I.S., Shidkow N.P. (1971) Numerische Methoden. 2. VEB Deutscher Verlag der Wissenschaften, Berlin 10. Berger J. (1985) Statistical Decision Theory and Bauyesian Analysis. Springer, Berlin Heidelberg New York 11. Bertsekas D.P. (1973) Stochastic Optimization Problems with Nondiﬀerentiable Cost Functionals. JOTA 12: 218–231 12. Betro’ B., de Biase L. (1978) A Newton-Like Method for Stochastic Optimization. In: Dixon L.C.W., Szeg¨ o G.P. (eds.), Towards Global Optimization 2, North-Holland, Amsterdam 269–289 13. Bhattacharya R.N., Rao R.R. (1976) Normal Approximation and Asymptotic Expansion. Wiley, New York 14. Biles W.E., Swain J.J. (1979) Mathematical Programming and the Optimization of Computer Simulations. Mathematical Programming Studies 11: 189–207 15. Bleistein N., Handelsmann R. (1975) Asymptotic Expansions of Integrals. Holt, Rinehart and Winston, New York 16. Blum E., Oettli W. (1975) Mathematische Optimierung. Springer, Berlin Heidelberg New York

328

References

17. Box G.E., Draper N.R. (1987) Empirical Model-Building and Response Surfaces. Wiley, New York, Chichester, Brisbane, Toronto, Singapore 18. Box G.E.P., Wilson K.G. (1951) On the Experimental Attainment of Optimum Conditions. Journal of the Royal Statistical Society 13: 1–45 19. Breitung K., Hohenbichler M. (1989) Asymptotic Approximations for Multivariate Integrals with an Application to Multinormal Probabilities. Journal of Multivariate Analysis 30: 80–97 20. Breitung K. (1984) Asymptotic Approximations for Multinormal Integrals. ASCE Journal of the Engineering Mechanics Division 110(1): 357–366 21. Breitung K. (1990) Parameter Sensitivity of Failure Probabilities. In: Der Kiureghian A., Thoft-Christensen P. (eds.), Reliability and Optimization of Structural Systems ’90. Lecture Notes in Engineering, Vol. 61, 43–51. Springer, Berlin Heidelberg New York 22. Bronstein I.N., Semendjajew K.A. (1980) Taschenbuch der Mathematik. Verlag Harri Deutsch, Thun, Frankfurt/Main 23. Bucher C.G., Bourgund U. (1990) A Fast and Eﬃcient Response Surface Approach For Structural Reliability Problems. Structural Safety 7: 57–66 24. Bullen P.S. (2003) Handbook of Means and Their Inequalities. Kluwer, Dordrecht 25. Chankong V., Haimes Y.Y. (1983) Multiobjective Decision Making. Oxford, North Holland, New York, Amsterdam 26. Chung K.L. (1954) On a Stochastic Approximation Method. Annals of Mathematical Statistics 25: 463–483 27. Craig J.J. (1988) Adaptive Control of Mechanical Manipulators. AddisonWesley, Reading, MA 28. Der Kiureghian A., Thoft-Christensen P. (eds.) (1990) Reliability and Optimization of Structural Systems ’90. Lecture Notes in Engineering, Vol. 61. Springer, Berlin Heidelberg New York 29. Dieudonn´e J. (1960) Foundations of Modern Analysis. Academic, New York 30. Ditlevsen O. (1981) Principle of Normal Tail Approximation. Journal of the Engineering Mechanics Division, ASCE, 107: 1191–1208 31. Ditlevsen O., Madsen H.O. (1996) Structural Reliability Methods. Wiley, Chichester 32. Dolinski K. (1983) First-Order Second Moment Approximation in Reliability of Systems: Critical Review and Alternative Approach. Structural Safety 1: 211–213 33. Draper N.R., Smith H. (1980) Applied Regression Analysis. Wiley, New York 34. Ermoliev Y. (1983) Stochastic Quasigradient Methods and Their Application to System Optimization. Stochastics 9: 1–36 35. Ermoliev Y., Gaivoronski A. (1983) Stochastic Quasigradient Methods and Their Implementation. IIASA Workpaper, Laxenburg 36. Ermoliev Yu. (1988) Stochastic Quasigradient Methods. In: Ermoliev Yu., Wets R.J.-B. (eds.), Numerical Techniques for Stochastic Optimization, Springer, Berlin Heidelberg New York, 141–185 37. Ermoliev Yu., Wets R. (eds.) (1988) Numerical Techniques for Stochastic Optimization. Springer Series in Computational Mathematics Vol. 10. Springer, Berlin Heidelberg New York 38. Eschenauer H.A. et al. (1991) Engineering Optimization of Design Processes. Lecture Notes in Engineering, Vol. 63. Springer, Berlin Heidelberg New York

References

329

39. Fabian V. (1967) Stochastic Approximation of Minima with Improved Asymptotic Speed. Annals of Mathematical Statistics 38: 191–200 40. Fabian V. (1968) On Asymptotic Normality in Stochastic Approximation. Annals of Mathematical Statistics 39: 1327–1332 41. Fabian V. (1971) Stochastic Approximation. In: Rustagi J.S. (ed.), Optimizing Methods in Statistics, Academic, New York, London, 439–470 42. Fleury C. (1989) First and Second Order Approximation Strategies in Structural Optimization. Structural Optimization 1: 3–10 43. G¨ anssler P., Stute W. (1977) Wahrscheinlichkeitstheorie. Springer, Berlin Heidelberg New York 44. Gaivoronski A. (1988) Stochastic Quasigradient Methods and Their Implementation. In: Ermolief Yu., Wets R.J.-B. (eds.), Numerical Techniques for Stochastic Optimization, Springer, Berlin Heidelberg New York, 313–351 45. Galambos J., Simonelli I. (1996) Bonferroni-Type Inequalities with Applications. Springer, Berlin Heidelberg New York 46. Gates T.K., Wets R.J.-B., Grismer M.E. (1989) Stochastic Approximation Applied to Optimal Irrigation and Drainage Planning. Journal of Irrigation and Drainage Engineering, 115(3): 488–502 47. Gill P.E., Murray W., Wright M.H. (1981) Practical Optimization. Academic, New York, London 48. Graf Fink von Finkenstein (1977) Einf¨ uhrung in die Numerische Mathematik, C. Hanser Verlag, M¨ unchen 49. Grenander U. (1968) Probabilities on Algebraic Structures. Wiley, New York, London 50. Han R., Reddy B.D. (1999) Plasticity-Mathematical Theory and Numerical Analysis. Springer, Berlin Heidelberg New York 51. Henrici P. (1964) Elements of Numerical Analysis. Wiley, New York 52. Hiriart-Urruty J.B. (1977) Contributions a la programmation mathematique: Cas deterministe et stochastique, Thesis, University of Clermont-Ferrand II 53. Hodge P.G. (1959) Plastic Analysis of Structures. McGraw-Hill, New York 54. Hohenbichler M., Rackwitz R., et al. (1987) New Light on First and Second Order Reliability Methods. Structural Safety 4: 267–284 55. Jacobsen Sh.H., Schruben Lee W. (1989) Techniques for Simulation Response Optimization. Operations Research Letters 8: 1–9 56. Johnson N.L., Kotz S. (1976) Distributions in Statistics: Continuous Multivariate Distributions. Wiley, New York, London 57. Kaliszky R. (1989) Plasticity: Theory and Engineering Applications. Elsevier, Amsterdam 58. Kall P. (1976) Stochastic Linear Programming. Econometrics and Operations Research, Vol. XXI. Springer, Berlin Heidelberg New York 59. Kall P., Wallace S.W. (1994) Stochastic Programming. Wiley, Chichester 60. Kesten H. (1958) Accelerated Stochastic Approximation. Annals of Mathematical Statistics 29: 41–59 61. Kiefer J., Wolfowitz J. (1952) Stochastic Estimation of the Maximum of a Regression Function. Annals of Mathematical Statistics 23: 462–466 62. Kirsch U. (1993) Structural Optimization. Springer, Berlin Heidelberg New York 63. Kleijnen J.P.C. (1987) Statistical Tools for Simulation Practitioners. Marcel Decker, New York

330

References

64. Kleijnen J.P.C., van Groenendaal W. (1992) Simulation: A Statistical Perspective. Wiley, Chichester 65. Knopp K. (1964) Theorie und Anwendung der unendlichen Reihen. Springer, Berlin Heidelberg New York 66. K¨ onig J.A. (1987) Shakedown of Elastic–Plastic Structures. Elsevier, Amsterdam 67. Kolmogorof A.N., Fomin S.V. (1975) Reelle Funktionen und Funktionananalysis. VEB Deutscher Verlag der Wissenschaften, Berlin 68. Kosmol P. (1989) Methoden zur numerischen Behandlung nichtlinearer Gleichungen und Optimierungsaufgaben. B.G. Teubner, Stuttgart 69. Kushner H.J., Clark D.S. (1978) Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer, Berlin Heidelberg New York 70. Lai T.L., Robbins H. (1979) Adaptive Design and Stochastic Approximation. The Annals of Statistics 7(6): 1196–1221 71. Ljung L. (1977) Analysis of Recursive Stochastic Algorithms. IEEE Transactions on Automatic Control, AC 22: 551–575 72. Luenberger D.G. (1969) Optimization by Vector Space Methods. Wiley, New York 73. Luenberger D.G. (1973) Introduction to Linear and Nonlinear Programming. Addison-Wesley, Reading, MA 74. Mangasarian O. L. (1969) Nonlinear Programming. McGraw-Hill, New York, London, Toronto 75. Marshall A.W., Olkin I. (1960) Multivariate Chebyshev Inequalities. The Annals of Mathematical Statistics 31: 1001–1014 76. Marti K. (1977) Stochastische Dominanz und stochastische lineare Programme. Methods of Operations Research 23: 141–160 77. Marti K. (1978) Stochastic Linear Programs with Stable Distributed Random Variables. In: Stoer J. (ed.), Optimization Techniques Part 2. Lecture Notes in Control and Information Sciences, Vol. 7, 76–86 78. Marti K. (1979) Approximationen Stochastischer Optimierungsprobleme. A. Hain, K¨ onigsstein/Ts. 79. Marti K. (1979) On Solutions of Stochastic Programming Problems by Descent Procedures with Stochastic and Deterministic Directions. Methods of Operations Research 33: 281–293 80. Marti K. (1980) Solving Stochastic Linear Programs by Semi-Stochastic Approximation Algorithms. In: Kall P., Prekopa A. (eds.), Recent Result in Stochastic Programming. Lecture Notes in Economics and Mathematical Systems, 179: 191–213 81. Marti K. (1983) Descent Directions and Eﬃcient Solutions in Discretely Distributed Stochastic Programs. LN in Economics and Mathematical Systems, Vol. 299. Springer, Berlin Heidelberg New York 82. Marti K. (1984) On the Construction of Descent Directions in a Stochastic Program Having a Discrete Distribution. ZAMM 64: T336–T338 83. Marti K. (1985) Computation of Descent Directions in Stochastic Optimization Problems with Invariant Distributions. ZAMM 65: 355–378 84. Marti K., Fuchs E. (1986) Computation of Descent Directions and Eﬃcient Points in Stochastic Optimization Problems Without Using Derivatives. Mathematical Programming Study 28: 132–156

References

331

85. Marti K., Fuchs E. (1986) Rates of Convergence of Semi-Stochastic Approximation Procedures for Solving Stochastic Optimization Problems. Optimization 17: 243–265 86. Marti K. (1986) On Accelerations of Stochastic Gradient Methods by Using more Exact Gradient Estimations. Methods of Operations Research 53: 327–336 87. Marti K. (1987) Solving Stochastic Optimization Problems By Semi-Stochastic Approximation Procedures. In: Borne/Tzafestas (eds.), Applied Modelling and Simulation of Technological Systems, 559–568. Elsevier, North Holland, Amsterdam, New York, Oxford, Tokyo 88. Marti K. (1990) Stochastic Optimization Methods in Structural Mechanics. ZAMM 70: T 742–T745 89. Marti K., Pl¨ ochinger E. (1990) Optimal Step Sizes in Semi-Stochastic Approximation Procedures, I and II, Optimization 21(1): 123–153 and (2): 281–312 90. Marti K. (1990) Stochastic Programming: Numerical Solution Techniques by Semi-Stochastic Approximation Methods. In: Slowinski R., Teghem J. (eds.), Stochastic Versus Fuzzy Approaches to Multiobjective Programming Under Uncertainty, 23–43. Kluwer, Boston, Dordrecht, London 91. Marti K. (1990) Stochastic Optimization in Structural Design. ZAMM 72: T452–T464 92. Marti K. (1992) Approximations and Derivatives of Probabilities in Structural Design. ZAMM 72 6: T575–T578 93. Marti K. (1992) Computation of Eﬃcient Solutions of Discretely Distributed Stochastic Optimization Problems. ZOR 36: 259–296 94. Marti K. (1994) Approximations and Derivatives of Probability Functions. In: Anastassiou C., Rachev S.T. (eds.), Approximation, Probability and Related Fields, 367–377. Plenum, New York 95. Marti K. (1995) Diﬀerentiation of Probability Functions: The Transformation Method. Computers and Mathematics with Applications 30(3–6): 361–382 96. Marti K. (1995) Computation of Probability Functions and Its Derivatives by Means of Orthogonal Function Series Expansions. In: Marti K., Kall P. (eds.), Stochastic Programming: Numerical Techniques and Engineering Applications, 22–53, LNEMS 423. Springer, Berlin Heidelberg New York 97. Marti K. (1999) Path Planning for Robots Under Stochastic Uncertainty. Optimization 45: 163–195 98. Marti K. (2003) Stochastic Optimization Methods in Optimal Engineering Design Under Stochastic Uncertainty. ZAMM 83(11): 1–18 99. Marti K. (2003) Plastic Structural Analysis Under Stochastic Uncertainty. Mathematical and Computer Modelling of Dynamical Systems (MCMDS) 9(2): 303–325 100. Marti K. (2005) Stochastic Optimization Methods. Springer, Berlin Heidelberg New York 101. Marti K., Ermoliev Y., Pﬂug G. (eds.) (2004) Dynamic Stochastic Optimization. LNEMS 532. Springer, Berlin Heidelberg New York 102. Mauro C.A. (1984) On the Performance of Two-Stage Group Screening Experiments. Technometrics 26(3): 255–264 103. McGuire W., Gallagher R.H. (1979) Matrix Structural Analysis. Wiley, New York, London 104. Melchers R.R. (1999) Structural Reliability: Analysis and Prediction, 2nd edn. Ellis Horwood, Chichester (England)

332

References

105. Montgomery C.A. (1984) Design and Analysis of Experiments. Wiley, New York 106. Myers R.H. (1971) Response Surface Methodology. Allyn and Bacon, Boston 107. Nevel’son M.B., Khas’minskii R.Z. (1973) An Adaptive Robbins–Monro Procedure. Automation and Remote Control 34: 1594–1607 108. Pantel M. (1979) Adaptive Verfahren der Stochastischen Approximation. Dissertation, Universit¨ at Essen-Gesamthochschule, FB 6-Mathematik 109. Park S.H. (1996) Robust Design and Analysis for Quality Engineering. Chapman and Hall, London 110. Paulauskas V., Rackauskas A. (1989) Approximation Theory in the Central Limit Theorem. Kluwer, Dordrecht 111. Pfeiﬀer F., Johanni R. (1987) A Concept for Manipulator Trajectory Planning. IEEE Journal of Robotics and Automation RA-3: 115–123 112. Pﬂug G.Ch. (1988) Step Size Rules, Stopping Times and Their Implementation in Stochastic Quasigradient Algorithms. In: Ermoliev Yu., Wets R.J.-B. (eds.), Numerical Techniques for Stochastic Optimization, 353–372, Springer, Berlin Heidelberg New York 113. Phadke M.S. (1989) Quality Engineering Using Robust Design. P.T.R. Prentice Hall, Englewood Cliﬀs, NJ 114. Pahl G., Beitz W. (2003) Engineering Design: A Systematic Approach. Springer, Berlin Heidelberg New York London 115. Pl¨ ochinger E. (1992) Realisierung von adaptiven Schrittweiten f¨ ur stochastische Approximationsverfahren bei unterschiedlichem Varianzverhalten des Sch¨ atzfehlers. Dissertation, Universit¨ at der Bundeswehr M¨ unchen 116. Polyak B.T., Tsypkin Y.Z. (1980) Robust Pseudogradient Adaption Algorithms. Automation and Remote Control 41: 1404–1410 117. Polyak B.G., Tsypkin Y.Z. (1980) Optimal Pseudogradient Adaption Algorithms. Automatika i Telemekhanika 8: 74–84 118. Prekopa A., Szantai T. (1978) Flood Control Reservoir System Design Using Stochastic Programming. Mathematical Programming Study 9: 138–151 119. Press S.J. (1972) Applied Multivariate Analysis. Holt, Rinehart and Winston, New York, London 120. Rackwitz R., Cuntze R. (1987) Formulations of Reliability-Oriented Optimization. Engineering Optimization 11: 69–76 121. Reinhart J. (1997) Stochastische Optimierung von Faserkunststoﬀverbundplatten. Fortschritt–Berichte VDI, Reihe 5, Nr. 463. VDI Verlag GmbH, D¨ usseldorf 122. Reinhart J. (1998) Implementation of the Response Surface Method (RSM) for Stochastic Structural Optimization Problems. In: Marti K., Kall P. (eds.), Stochastic Programming Methods and Technical Applications. Lecture Notes in Economics and Mathematical Systems, Vol. 458, 394–409 123. Review of Economic Design, ISSN: 1434–4742/50. Springer, Berlin Heidelberg New York 124. Richter H. (1966) Wahrscheinlichkeitstheorie. Springer, Berlin Heidelberg New York 125. Robbins H., Monro S. (1951) A Stochastic Approximation Method. Annals of Mathematical Statistics 22: 400–407 126. Robbins H., Siegmund D. (1971) A Convergence Theorem for Non Negative Almost Supermaringales and Some Applications. Optimizing Methods in Statistics. Academic, New York, 233–257

References

333

127. Rommelfanger H. (1977) Diﬀerenzen- und Diﬀerentialgleichungen. Bibliographisches Institut Mannheim 128. Rosenblatt M. (1952) Remarks on a Multivariate Transformation. The Annals of Statistics 23: 470–472 129. Rozvany G.I.N. (1989) Structural Design via Optimality Criteria. Kluwer, Dordrecht 130. Rozvany G.I.N. (1997) Topology Optimization in Structural Mechanics. Springer, Berlin Heidelberg New York 131. Ruppert D. (1985) A Newton–Raphson Version of the Multivariate Robbins– Monro Procedure. The Annals of Statistics 13(1): 236–245 132. Sacks J. (1958) Asymptotic Distributions of Stochastic Approximation Procedures. Annals of Mathematical Statistics 29: 373–405 133. Sansone G. (1959) Orthogonal Functions. Interscience, New York, London 134. Sawaragi Y., Nakayama H., Tanino T. (1985) Theory of Multiobjective Optimization. Academic, New York, London, Tokyo 135. Schmetterer L. (1960) Stochastic Approximation. Proceedings 4th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 587–609 136. Schoofs A.J.G. (1987) Experimental Design and Structural Optimization. Doctoral Thesis, Technical University of Eindhoven 137. Schu¨eller G.I. (1987) A Critical Appraisal of Methods to Determine Failure Probabilities. Journal of Structural Safety 4: 293–309 138. Schu¨eller G.I., Gasser M. (1998) Some Basic Principles of Reliability-Based Optimization (RBO) of Structure and Mechanical Components. In: Marti K., Kall P. (eds.), Stochastic Programming-Numerical Techniques and Engineering. LNEMS Vol. 458: 80–103. Springer, Berlin Heidelberg New York 139. Sch¨ uler L. (1974) Sch¨ atzung von Dichten und Verteilungsfunktionen mehrdimensionaler Zufallsvariabler auf der Basis orthogonaler Reihen. Dissertation Naturwissenschaftliche Fakult¨ at der TU Braunschweig, Braunschweig 140. Sch¨ uler L., Wolﬀ H. (1976) Sch¨ atzungen mehrdimensionaler Dichten mit Hermiteschen Polynomen und lineare Verfahren zur Trendverfolgung. Forschungsbericht BMVg-FBWT 76–23, TU Braunschweig, Institut f¨ ur Angwandte Mathematik, Braunschweig 141. Schwartz S.C. (1967) Estimation of Probability Density by an Orthogonal Series. Annals of Mathematical Statistics 38: 1261–1265 142. Schwarz H.R. (1986) Numerische Mathematik. B.G. Teubner, Stuttgart 143. Sciavicco L., Siciliano B. (2000) Modeling and Control of Robot Manipulators. Springer, Berlin Heidelberg New York London 144. Slotine J.-J., Li W. (1991) Applied Nonlinear Control. Prentice-Hall, Englewood Chiﬀs, NJ 145. Sobieszczanski-Sobieski J.G., Dovi A. (1983) Structural Optimization by MultiLevel Decomposition. AIAA Journal 23: 1775–1782 146. Spillers W.R. (1972) Automated Structural Analysis: An Introduction. Pergamon, New York, Toronto, Oxford 147. Stengel R.F. (1986) Stochastic Optimal Control: Theory and Application. Wiley, New York 148. St¨ ockl G. (2003) Optimaler Entwurf elastoplastischer mechanischer Strukturen unter stochastischer Unsicherheit. Fortschritt-Berichte VDI, Reihe 18, Nr. 278. VDI-Verlag GmbH, D¨ usseldorf 149. Streeter V.L., Wylie E.B. (1979) Fluid Mechanics. McGraw-Hill, New York

334

References

150. Stroud, A.H. (1971) Approximate Calculation of Multiple Integrals. PrenticeHall, Englewood Cliﬀs, NJ 151. Stummel F., Hainer K. (1982) Praktische Mathematik. B.B. Teubner, Stuttgart 152. Syski W. (1987) Stochastic Approximation Method with Subgradient Filtering and On-Line Stepsize Rules for Nonsmooth, Nonconvex and Unconstrained Problems. Control and Cybernetics 16(1): 59–76 153. Szantai T. (1986) Evaluation of a Special Multivariate Gamma Distribution Function. Mathematical Programming Study 27: 1–16 154. Thoft-Christensen P., Baker M.J. (1982) Structural Reliability Theory and Its Applications. Springer, Berlin Heidelberg New York 155. Tong Y.L. (1980) Probability Inequalities in Multivariate Distributions. Academic, New York, London 156. Toropov V.V. (1989) Simulation Approach to Structural Optimization. Structural Optimization 1(1): 37–46 157. Tricomi F.G. (1970) Vorlesungen u ¨ ber Orthogonalreihen. Springer, Berlin Heidelberg New York 158. Triebel H. (1981) Analysis und mathematische Physik. BSB B.B. Teubner Verlagsgesellschaft, Leipzig 159. Uryas’ev St. (1989) A Diﬀerentiation Formula for Integrals over Sets Given by Inclusion. Numerical Functional Analysis and Optimization 10: 827–841 160. Venter Y.H. (1966) On Dvoretzky Stochastic Approximation Theorems. Annals of Mathematical Statistics 37: 1534–1544 161. Venter Y.H. (1967) An Extention of the Robbings–Monro Procedure. Annals of Mathematical Statistics 38: 181–190 162. Vietor T. (1994) Optimale Auslegung von Strukturen aus spr¨ oden Wirkstoﬀen. TIM-Forschungsberichte Nr. T04-02.94, FOMAAS, Unviersit¨ at Siegen 163. Walk H. (1977) An Invariance Principle for the Robbings–Monro Process in a Hilbert Space. Z. Wahrscheinlichkeitstheorie verw. Gebiete 39: 135–150 164. Wasan M.T. (1969) Stochastic Approximation. University Press, Cambridge 165. Wets R. (1983) Stochastic Programming: Solution Techniques and Approximation Schemes. In: Stochastic Programming Bachem A., Gr¨ otschel M., Korte B. (eds.), Mathematical Programming: The State of the Art, 566–603. Springer, Berlin Heidelberg New York

Index

β-point, 262 1–1-transformation, 261 a priori, 4, 5 actual parameter, 4 adaptive control of dynamic system, 4 adaptive trajectory planning for robots, 4 admissibility condition, 254 admissibility of the state, 9 approximate discrete distribution, 296 Approximate expected failure cost minimization, 30 approximate expected failure or recourse cost constraints, 21 Approximate expected failure or recourse cost minimization, 22 approximate optimization problem, 4 approximate probability function, 295 Approximation, 4 Approximation of expected loss functions, 26 Approximation of state function, 22 asymptotically, 200 back-transformation, 49, 56 Bayesian approach, 5 behavioral constraints, 32, 43 Biggest-is-best, 17 bilinear functions, 25 Binomial-type distribution, 295 Bonferroni bounds, 31 Bonferroni-type bounds, 45 boundary (b-)points), 279

boundary point, 271 boundary points, 273 Bounded 1st order derivative, 29 Bounded 2nd order derivative, 30 Bounded eigenvalues of the Hessian, 21 Bounded gradient, 20 box constraints, 4 calibration methods, 5 chance constrained programming, 46 changing error variances, 178 Compensation, 4 compromise solution, 6 concave function, 21 constraint functions, 3 Continuity, 18 Continuous probability distributions, 12 continuously diﬀerentiable, 20 control (input, design) factors, 16 control volume, 46 control volume concept, 46 Convex Approximation, 95 convex loss function, 17, 20 convex polyhedron, 273, 276 Convexity, 18 correction expenses, 4 cost(s) approximate expected failure or recourse ... constraints, 21 expected, 6 expected weighted total, 7 factors, 3 failure, 13 function, 5, 14

336

Index

loss function, 11 maximum, 8 maximum weighted expected, 7 minimum production, 16 of construction, 9 primary ... constraints, 11 recourse, 11, 13 total expected, 13 covariance matrix, 13, 21 cross-sectional areas, 9 curvature, 66 decision theoretical task, 5 decreasing rate of stochastic steps, 173 demand m-vector, 10 demand factors, 3 descent directions, 95 design optimal, 13 optimal ... of economic systems, 4 optimal ... of mechanical structures, 4 optimal structural, 10 optimum, 9 point, 37 problem, 3 robust ... problem, 29 robust optimal, 21 structural, 9 structural reliability and, 44 variables, 3 vector, 9, 44 deterministic constraint, 7, 9 substitute problem, 5–8, 13, 18, 19, 21 Diﬀerentiability, 19 diﬀerentiation, 58 diﬀerentiation formula, 46, 48, 52 diﬀerentiation of parameter-dependent multiple integrals, 46 diﬀerentiation with respect to xk , 49, 56 discrete distribution, 289, 293 Discrete probability distributions, 12 displacement vector, 54 distance functional, 255 distribution probability, 5 distribution parameters, 12 disturbances, 3

divergence, 49 divergence theorem, 51, 52, 59 domain of integration, 52, 53 dual representation, 257 economic uncertainty, 11 eﬃcient points, 96 elastic moduli, 54 elliptically contoured probability distribution, 118 essential supremum, 8 expectation, 6, 12, 13 expected approximate ... failure cost minimization, 30 approximate ... failure or recourse cost minimization, 22 approximate ... failure or recourse cost constraints, 21, 30 Approximation of ... loss functions, 26 cost, 6 cost minimization problem, 6 failure or recourse cost functions, 15 failure or recourse cost constraints, 15 failure or recourse cost minimization, 16 maximum weighted... costs, 7 primary cost constraints, 16, 22, 30 primary cost minimization, 15, 21, 30 quality loss, 17 recourse cost functions, 20 total ... costs, 13 total cost minimization, 15 total weighted ... costs, 7 weighted total costs, 7 explicit representations, 280 explicitly, 281 external load parameters, 3 external load factor, 253 extreme points, 11 factor control (input,design), 16 cost, 3 demand, 3 noise, 3 of production, 3, 9 weight, 7

Index failure, 9 approximate expected ... or recourse cost constraints, 21, 30 costs, 11, 13 domain, 10 expected costs of, 13 mode, 10 of the structure, 22 probability, 45 probability of, 13, 15 failure/survival domains, 31 ﬁnite diﬀerence gradient estimator, 130 First Order Reliability Method (FORM), 38 ﬂuid dynamics, 46 FORM, 262 function series expansions, 75 function(s) Approximation of expected loss, 26 Approximation of state, 22 bilinear, 25 concave, 21 constraint, 3 convex loss, 17 cost, 14 cost/loss, 11 hermite, 75 Laguerre, 75 Legendre, 75 limit state, 9, 10 loss, 14, 20 mean value, 17, 26, 28 objective, 3 output, 32 performance, 32 primary cost, 9 product quality, 17 quality, 29 recourse cost, 14 response, 32 state, 9, 10, 14, 15, 25 trigonometric, 75 functional determinant, 55 functional–eﬃcient, 6 gradient, 48, 55 bounded, 20 H¨ older mean, 24 Hermite functions, 75

337

higher order partial derivatives, 51 hybrid stochastic approximation procedure, 131 hypersurface, 52, 53, 60 inequality Jensen’s, 20 Markov-type, 36 Tschebyscheﬀ-type, 33 initial behavior, 247 Inner expansions, 28 inner linearization, 96 input r-vector, 43 input vector, 3, 9, 18 integrable majorant, 49, 50, 55 integral transformation, 46, 48, 58 interchanging diﬀerentiation and integration, 48, 55, 59 Jensen’s inequality, 20 Lagrangian, 14, 263 Laguerre functions, 75 least squares estimation, 26 Legendre functions, 75 limit state function, 9, 10, 45 limit state surface, 259 limited sensitivity, 22 linear form, 270 linear programs with random data, 46 linearization, 37 linearize the state function, 268 Linearizing the state function, 264 Lipschitz-continuous, 29 Loss Function, 20 Lyapunov equation, 310 manufacturing, 4 manufacturing errors, 3, 54 Markov-type, 283 Markov-type inequality, 36 material parameters, 3, 54 matrix step size, 179 maximum costs, 8 Mean Integrated Square Error, 83 mean limit covariance matrix, 204 mean performance, 16 mean square error, 139 mean value function, 17, 26, 28

338

Index

mean value theorem, 20, 29 measurement and correction actions, 4 mechanical structure, 9, 10 minimum reliability, 34 minimum asymptotic covariance matrix, 221 minimum distance, 39 model parameters, 3, 10 model uncertainty, 11 modeling errors, 3 more exact gradient estimators, 131 multiple integral, 12, 48 multiple linearizations, 270 Necessary optimality conditions, 263, 267, 271 noise, 16 noise factors, 3 nominal vector, 4 Nominal-is-best, 16 normal distributed random variable, 36 objective function, 3 operating condition, 254 operating conditions, 3, 9, 11 operating, safety conditions, 43 optimal control, 3 decision, 3 design of economic systems, 4 design of mechanical structures, 4 optimal design, 13 optimal structural design, 10 optimum design, 9 Orthogonal Function Series Expansion, 46 outcome map, 5 outcomes, 5 parallel computing, 273 parameter identiﬁcation, 5 parameters distribution, 12 external load, 3 material, 3, 54 model, 3, 4, 10 stochastic structural, 54 technological, 3

Pareto optimal, 6 optimal solution, 7 weak ... optimal solution, 7, 8 weak Pareto optimal, 6 Pareto-optimal solutions, 114 partial derivatives, 27 partial monotonicity, 100 performance, 16 performance function, 253 performance functions, 6 performance variables, 43 Phase 1 of RSM, 134 Phase 2 of RSM, 136 physical uncertainty, 11 Polyhedral approximation, 273 power mean, 24 primary cost constraints, 11 primary cost function, 9 primary goal, 7 probabilistic constraints, 269 probability continuous ... distributions, 12 density, 47, 54 density function, 12, 44 discrete ... distributions, 12 distribution, 5 function, 43, 46, 48 of survival, 54 of failure, 13, 15 of safety, 13 space, 5 subjective or personal, 5 probability density function, 12 probability inequalities, 282 probability of failure, 260 probability of safety, 260 product quality function, 17 production planning, 4 production planning problems, 10 projection of the origin, 64 projection problem, 263 quality quality quality quality quality

characteristics, 16 engineering, 16, 29 function, 29 loss, 16 variations, 16

Index

339

random matrix, 47 random parameter vector, 6, 8, 44 random variability, 11 random vector, 13 rate of stochastic and deterministic steps, 158 realizations, 12 recourse cost functions, 14 recourse costs, 11, 13 reference points, 25 region of integration, 48 regression techniques, 5, 25 regular point, 263 reliability First Order Reliability Method (FORM), 38 index, 39 of a stochastic system, 43 reliability analysis, 30, 253 reliability based optimization, 14, 45 Reliability constraints, 45 Reliability maximization, 45 Response Surface Methods, 25 Response Surface Model, 25 response variables, 43 response, output or performance functions, 32 robust design problem, 29 robust optimal design, 21 robustness, 16 RSM-gradient estimator, 132

sequential decision processes, 4 sizing variables, 9 Smallest-is-best, 17 stable distributions, 119 state equation, 254 state function, 9, 10, 14, 15, 25, 253, 256 statistical uncertainty, 11 stochastic approximation method, 179 Stochastic Completion and Transformation Method, 46 stochastic completion technique, 73 stochastic diﬀerence quotient, 205 stochastic structural parameters, 54 Stochastic Uncertainty, 5 structural optimization, 3 structural dimensions, 9 structural mechanics, 54 structural reliability analysis, 37 structural reliability and design, 44, 54 structural resistance parameters, 253 structural systems, 4, 10 structural systems weakness, 14 structure mechanical, 9 safe, 3 subjective or personal probability, 5 sublinear functions, 97 surface element, 53 Surface Integrals, 51 systems failure domain, 44

safe domain, 260 safe state, 258 safe structure, 3 safety, 9 safety conditions, 11 safety margins, 9 sample information, 4, 5 scalarization, 6, 8 scenarios, 12 search directions, 204 second order expansions, 21 secondary goals, 7 semi-stochastic approximation method, 129 sensitivities, 27 Sensitivity of reliabilities, 45 separable, 100

Taylor expansion, 27, 36 Taylor polynomials, 57 technological coeﬃcients, 10 technological parameters, 3 thickness, 9 total weight, 9 Transformation Method, 46 trigonometric functions, 75 Tschebyscheﬀ-type inequality, 33 two-step procedure, 4 uncertainty, 4 economic, 11 model, 11 physical, 11 statistical, 11 stochastic, 5

340

Index

unsafe domain, 260 unsafe state, 258 v. Mises procedure, 322 variable design, 3 vector input, 3

nominal, 4 optimization problem, 6 volume, 9 weak convergence, 294 weak functional–eﬃcient, 6 weight factors, 7 Worst Case, 8