Introduction to Sensitivity and Stability Analysis in Nonlinear Programming (Mathematics in Science and Engineering, 165)

Introduction to Sensitivity and Stability Analysis in Nonlinear Programming

This is Volume 165 in MATHEMATICS IN SCIENCE AND ENGINEERING A Series of Monographs and Textbooks Edited by RICHARD BELLMAN, University o f Southern California The complete listing of books in this series is available from the Publisher upon request.

Introduction to Sensitivity and Stability Analysis in Nonlinear Programming ANTHONY

V.

FIACCO

Operations Research Department Institute for Management Science and Engineering School of Engineering and Applied Science The George Washington University Washington, D.C.

1983

A C A D E M I C PRESS A Subsidiary of Harcourt Brace Jovanovich, Publishers

Paris San Diego

New York London O Sydney Tokyo Toronto San Francisco S ~ Paul0

Academic Press Rapid Manuscript Reproduction

COPYRIGHT @ 1983, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED. NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.

ACADEMIC PRESS, INC.

111 Fifth Avenue, New York. New York 10003

Uniied Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD.

24/28 Oval Road, London

NWI 7DX

Library of Congress Cataloging in Publication Data

Fiacco. Anthony V . Introduction to sensitivity and stability analysis in nonlinear programming. (Mathematics in science and engineering) Bibliography: p . Includes index. I . Nonlinear programming. I . Title. I I . Series. T57.8. F53 1982 519.7'6 82-1 I642 ISBN 0-12-254450-1

PRINTED IN THE UNITED STATES OF AMERICA 83 84 85 86

9 8 7 6 5 4 3 2 1

To Sarah, Alicia, Tonino, Ma, Pa, and Clara

This Page Intentionally Left Blank

Contents

Preface

xi

I . Overview Chapter I . Motivation and Perspective

3

Chapter 2 . Basic Sensitivity and Stability Results 2.1 2.2 2.3 2.4 2.5 2.6 2.7

Introduction 8 Objective Function and Solution Set Continuity Differential Stability 22 36 Implicit Function Theorem Results 54 Optimal Value and Solution Bounds 60 General Results from RHS Results Summary 62

11

II. Theory and Calculation of Solution Parameter Derivatives Chapter 3 . Sensitivity Analysis under Second- Order Assumptions 3.1 Introduction 67 3.2 First-Order Sensitivity Analysis of a Second-Order Local Solution 68 80 3.3 Examples 3 . 4 First- and Second-Order Parameter Derivatives of the Optimal Value Function 82

Chapter 4 . Computational Aspects of Sensitivity Calculations: The General Problem 4.1 Introduction

91 vii

viii

Contents

4.2 Formulas for the Parameter First Derivatives of a Karush-KuhnTucker Triple 91 4.3 Applications and Examples 106

Chapter 5 . Computational Aspects: RHS Perturbations 5.1 Introduction I16 5.2 The Use and lnitial Interpretation of Lagrange Multipliers 117 5.3 Examples of Early Sensitivity Interpretations of Lagrange Multipliers 123 125 5.4 Supporting Theory 5.5 Formulas for the Parameter First Derivatives of a Karush-KuhnTucker Triple and Second Derivatives of the Optimal Value Function 127 I36 5 . 6 Examples and Applications

ZIZ. Algorithmic Approximations Chapter 6 . Estimates of Sensitivity Information Using Penalty Functions

6.1 Introduction 155 6.2 Approximation of Sensitivity Information Using the Logarithmic156 Quadratic Mixed Barrier-Penalty Function Method 6.3 Examples of Estimates of Solution Point and Lagrange Multiplier Parameter Derivatives 168 6.4 Extensions 171 6.5 Sensitivity Calculations Based on the Perturbed Karush-KuhnTucker System 173 179 6 . 6 Optimal Value Function Sensitivity Estimates 6.7 Example of Estimates of Optimal Value and First- and SecondParameter Derivatives 182 185 6.8 Sensitivity Approximations for RHS Perturbations 6.9 Recapitulation I90

Chapter 7 . Calculation of Sensitivity Information Using Other Algorithms

7.1 Introduction I94 7.2 Connections between Algorithmic and Sensitivity 196 Calculations 7.3 Algorithmic Calculations of the Inverse of the Jacobian of the Karush-Kuhn-Tucker System 20 1 206 7.4 Sensitivity Results for Augmented Lagrangians 7.5 Conclusions and Extensions 218

ix

Contents

N. Applications and Future Research Chapter 8. An Example of Computational Implementations: A Multi-Item Continuous Review Inventory Model 8. I 8.2 8.3 8.4 8.5

Introduction 225 226 Screening of Sensitivity Information 229 Example Sensitivity Calculations by SENSUMT 232 A Multi-Item Inventory Model Additional Computational Experience with Applications

243

Chapter 9. Computable Optimal Value Bounds and Solution Vector Estimates for Parametric NLP Programs 9.1 Introduction 246 9.2 Computable Piecewise Linear Upper and Lower Optimal Value Bounds 247 9.3 Estimates of a Parametric Solution Vector and a Sharper Convex 249 Upper Bound 9.4 Connections between Optimal Value Bounds and Duality 255 9.5 Nonlinear Dual Lower Bounds 259 9.6 Extensions 265 268 9.7 Bounds on a Solution Point 273 9.8 Further Extensions and Applications

Chapter 10. Future Research and Applications 10. I Recapitulation and Other Research Directions 10.2 Future Research Directions and Applications 10.3 Conclusions 285

Appendix I . Notation, Conventions, and Symbols Appendix II. References

Lemmas, Theorems, Corollaries, Definitions, and 32 1 Examples 329

Selected Bibliography of Works Not Cited Author Index Subject Index

289

357 36 1

346

277 28 1

This Page Intentionally Left Blank

Preface

Mathematical programming sensitivity and stability results are now abundant and far-reaching, but they are scattered in the literature. This has seriously diluted the impact of these results and impaired their systematic implementation, a deficiency compounded by the relatively small effort that has been made to make these results intelligible and accessible to nonmathematicians. Our purpose with this volume is to initiate seriously the process of unification and explication and to communicate the practical applicability and usefulness of many of these results to practitioners. It is now feasible to develop software that allows for the calculation of sensitivity and stability information whenever a solution to a nonlinear programming (NLP) problem is calculated. It seems clear that nonlinear programming will not be a widely used or practical decision-making technique without this capability. (Witness the routine use of post-optimality analysis in linear programming.) Further, sensitivity and stability calculations and solution algorithm calculations often involve similar data and the same manipulations, so it appears natural and efficient to calculate the former as a by-product of the latter. We strongly support this approach and illustrate it via the detailed calculation of sensitivity and stability information by a classical penalty function method. Similar exploitation is indicated for Newton and projected- and reduced-gradient methods and for augmented Lagrangian algorithms. It should be stressed, however, that the basic sensitivity and stability results are presented in terms of the given problem functions and independently of any algorithmic manipulations. Our goal is therefore the advancement of both theoretical unification and practical implementation. Numerous results and treatments are published here for the first time. This book will therefore be of equal interest to practical problem solvers and developers of methodology. The treatment is sweeping with respect to general developments and detailed for results that follow under well-known smoothness assumptions. The latter results are highly structured and readily implemented, also providing a rich foundation and a clear perspective for results under more general assumptions.

xi

xii

Preface

This book is directed primarily, but certainly not exclusively, to researchers and practitioners who utilize mathematical programming models. Both mathematicians and operations researchers should find the developments of particular interest in applications. The numerous references, supplemented by the bibliography of works not cited in the text, should be of great value. The book has been used by the author as the text for a graduate course in NLP sensitivity and stability analysis and could also be utilized as a supporting text for mathematical programming courses or seminars on advanced topics. It should provide a valuable reference for anyone interested in the sensitivity and stability analysis of nonlinear algebraic equations. In this treatment, the term sensitiviry information is often taken to mean program parameter derivatives of the optimal value function or a “Karush-Kuhn-Tucker triple” of a parametric nonlinear program. Srabiliry information here refers primarily to parametric bounds on the optimal value function or solution points. To our knowledge this is the first book devoted entirely to general sensitivity and stability results in nonlinear programming. Others will surely follow as the results are further expounded and unified. Important results in mathematical programming theory and algorithms have been and undoubtedly will continue to be developed using perturbation methods. It is a pleasure to acknowledge several vital sources of encouragement and support. Richard Bellman originally suggested the idea of collecting these results into a single treatment. My dissertation students have contributed significantly, as collaborators, implementers, and constructive critics. Section 3.4, Chapters 4 and 5, Sections 6.5-6.8, and Sections 8.1-8.4 are based on papers coauthored with Robert Armacost. Sections 2.1-2.4 are based largely on a survey coauthored with William Hutzler. The optimal value bounds results reported in Section 9.2 were first implemented in collaboration with Abolfazl Ghaemi. Several valuable suggestions and numerous corrections were obtained from Jerzy Kyparisis, who carefully read the entire manuscript. He also collaborated in the development of the results reported in Section 9.7 and in collecting the list of references in the selected bibliography. Continued support has been forthcoming from the Operations Research Department and the Institute for Management Science and Engineering of the George Washington University. This work was also partially supported by the Army Research Office, the Office of Naval Research, and the National Science Foundation.

Overview

This Page Intentionally Left Blank

Chapter I

Motivation and Perspective

A

methodology for conducting a local perturbation

(sensitivity) analysis and finite perturbation (stability) analysis of solution behavior with respect to problem changes is a well established requirement of any scientific discipline A sensitivity and stability analysis should be an integral

part of any solution methodology.

The status of a solution

cannot be understood without such information.

This has been

well recognized since the inception of scientific inquiry and has been explicitly addressed from the beginning of mathematics.

Important branches of mathematics have been inspired

by perturbation and approximation questions, and entire journals and mathematical treatises are devoted to such subjects.

In mathematical programming, sensitivity and stability

techniques have been used to obtain optimality conditions, duality results, solution algorithms, convergence and rate of convergence proofs, and acceleration of convergence of algorithms, in addition to their more obvious and immediate applications in estimating near solutions with different data. Such techniques are also essential in more mundane applications, e.g., in model validation and cost-benefit analysis.

3

4

I Motivation and Perspective

In linear programming (LP), the theoretical and computational results regarding the effects of parameter changes have been reasonably standardized under appropriate nondegeneracy assumptions, with the now well known and routinely used "postoptimality" (parametric) sensitivity and "range" (solution invariance) analysis being standard options in most LP computer codes.

In the evolution of linear programming method-

ology, the concern with a parametric analysis of a solution came early, and the theoretical and computational aspects were developed and refined along with the solution procedures. This did not happen in nonlinear programing (NLP). Concentration of effort was initially devoted to the development of solution algorithms.

Unlike linear programming, where the

simplex method soon dominated and unified theoretical and computational efforts, no "optimal" computational approach was dominant in nonlinear programming.

Numerous solution strate-

gies were proposed, and a host of theoretical and computational problems arose in developing first- and second-order optimality conditions, competitive numerical algorithms, and convergence and rate of convergence characterizations. The fragmentation of effort in the development of nonlinear programming methodology was reflected in the neglect of the development of sensitivity and stability results, except sporadically and somewhat abstractly.

Certainly, a unified

theory of stability analysis required some unification in the characterization of optimality and convergence theory.

Fortu-

nately, this unification began about a decade ago and with it came a serious concentrated effort to develop a sensitivity and stability analysis methodology.

The references corrob-

orating this fact are now, happily, too numerous to cite, but

5

I Motivation and Perspective

it is obvious to anyone who has followed developments in nonlinear programming that sensitivity and stability analysis has reached a sophisticated and respectable level. An impressive body of theoretical results has been developed for nonlinear programming.

As for computational imple-

mentations, many of the most incisive results are too abstract and general to be of immediate use in practical applications, and much work is needed to interpret and specialize these to obtain useful computable methods.

We feel there is a great

need for a systematic exploitation of problem structure and for the efficient interfacing of perturbation calculations with "algorithmic" (solution algorithm) calculations.

The

development and computational implementation of standard accessible techniques for conducting nontrivial sensitivity analyses in NLP has not been widespread and lags appreciably behind the present level of sophistication cf the theory. Nonetheless, nontrivial implementations do exist, and it is time to collect and unify the basic procedures as well as the existing rich body of theory. It is hoped that this book will contribute to this unification.

It is only a first step, and much remains to be done.

We begin by providing an overview of many of the important basic results in Chapter 2, using a common terminology and notation and making some efforts to indicate important relationships.

Given this perspective, Chapters 3

-

8 are con-

cerned essentially with the calculation of the parameter derivatives of the optimal value solution vector and associated optimal Lagrange multipliers of general classes of parametric nonlinear programs satisfying standard ideal second-order

I Motivation and Perspective

6

(twice continuous differentiability) assumptions.

Nonalgo-

rithmic (problem-oriented) and algorithmic (solution-methodoriented) results are given, the emphasis being ultimately on the provision of simple computable algorithmic methods and results that might be useful in practical applications. Chapter 9 provides a simple but effective approach for calculating parametric bounds on the optimal value function and a parametric feasible solution vector estimate, and also presents preliminary results that utilize the optimal value bounds to derive parametric solution vector bounds.

These re-

sults are based essentially on the exploitation of convexity properties, particularly the convexity (or concavity) of the optimal value function, under appropriate assumptions.

As

with the calculation of the parameter derivatives, the bounds results are computable with information already typically generated by most NLP algorithms.

Thus, sensitivity and stabil-

ity calculations can invariably be closely tailored to solution algorithm calculations and performed as side calculations as progress is made towards a solution.

This approach and its

practical applicability are demonstrated in detail using a well known classical penalty-function algorithm.

The idea is

readily applied to any solution algorithm and adaptations to augmented Lagrangian methods, reduced gradient methods, and projected gradent methods are given, in addition to immediate connections with recursive techniques based on Newton’s method. Chapter 10 concludes with a brief survey of potentially fruitful research directions and applications. It is hoped that the results presented in Chapter 2 will convince the reader that the theory of sensitivity and stability in mathematical programming is sophisticated, incisive,

I Motivation and Perspective

7

and c o m p r e h e n s i v e .

T h e r e i s no a s p e c t o f t h e s u b j e c t t h a t t h e

r e s u l t s do n o t touch

-

i n d e e d , it i s c l e a r t h a t t h e e n t i r e

s u b j e c t c a n be d e v e l o p e d from t h e p e r t u r b a t i o n a l p o i n t of view. T h i s i s n o t s u r p r i s i n g , o n c e it i s r e a l i z e d t h a t many fundamental r e s u l t s i n mathematics can be proved u s i n g p e r t u r b a t i o n techniques.

The p o i n t w e w i s h t o make i s t h a t s e n s i t i v i t y and

s t a b i l i t y r e s u l t s cannot and w i l l n o t be c o n f i n e d simply t o t h e c l a s s i c a l c o n c e r n of w h e t h e r " t h e s o l u t i o n c h a n g e s c o n t i n u o u s l y w i t h continuous changes i n t h e d a t a . " w i l l o f c o u r s e h a v e t o be a d d r e s s e d .

Such q u e s t i o n s

But a c o m p l e t e p e r t u r b a -

t i o n t h e o r y i s completely encompassing, and hence w e e x p e c t t h a t e v e r y i m p o r t a n t r e s u l t i n m a t h e m a t i c a l programming w i l l be r e v i s i t e d from t h i s p e r s p e c t i v e . W e a l s o hope t h a t C h a p t e r s 2

-

1 0 convince t h e r e a d e r t h a t

u s e f u l c o m p u t a b l e t e c h n i q u e s a r e known a n d c a n now be implemented a n d i n t e r f a c e d w i t h any s t a n d a r d NLP a l g o r i t h m i c c a l c u lations.

T h e r e would a p p e a r t o b e no t h e o r e t i c a l o b s t a c l e t o

d e v e l o p i n g t h e s o f t w a r e , f o r t h e i n d i c a t e d s e n s i t i v i t y and s t a b i l i t y calculations, t h a t interfaces with.the calculations r e q u i r e d by a n y o f t h e s t a n d a r d NLP c o d e s .

Chapter 2

Basic Sensitivity and Stability Results

2.1.

INTRODUCTION Comprehensive treatments of linear programming sensitivity

analysis and parametric linear programming (i.e., obtaining a solution as a function of problem parameters) have been forthcoming (Dinkelbach, 1969; Noiieka, Guddat, Hollatz, and Bank, 1974; Gal, 1979) along with basic sensitivity results (parameter derivatives) in quadratic programming (Boot, 1963). Unified treatments of nonlinear parametric programming methodology have only very recently begun to emerge (Bank, Guddat, Klatte, Kummer, and Tamer, 1982; Brosowski, 1982). Activity in the study of sensitivity and stability of solutions of general mathematical programs to problem perturbations has been sporadic and results are scattered in the literature.

This chapter endeavors to give a flavor of the

state of the art in this area.

Many important contributions

are necessarily omitted in this overview, which is confined to parametric perturbations of nonlinear programming problems. This and the following section, and most of Sections 2.3 and 2.4, are based on Fiacco and Hutzler (1979b), though numerous definitions, additional results, and refinements have been introduced. X

9

2. I Introduction

the next two examples demonstrate, the solution of very

As

simple mathematical programs may vary smoothly or change drastically for arbitrarily small perturbations of the problem parameters. Example 2 . 1 . 1

Consider the nonlinear program minx(xl

-

s.t. x2

+ (x, + 1) 2

€1 xl,

x2 2 -x1' The problem may be viewed as seeking x(f) region) that minimizes the distance from

E

(E,

R(E)

(the feasible

-1) to R ( f ) .

A

depiction and three solutions are illustrated in Fig. 2.1.1.

(c

=-2)

Fig. E

= -2

J

2.1.1. 0, 2.

D e p i c t i o n o f Example 2 . 1 . 1

and s o l u t i o n s f o r

10

2 Basic Sensitivity and Stability Results

The a n a l y t i c a l s o l u t i o n of t h i s p r o b l e m is e a s i l y s e e n t o be

1

( ( 1+ E ) / 2 ,

T

~ ( € 1

[ X l ( E ) 1 x ~ ( E ) I=

=

X(E)

-

1)/2,

< -1

E

- 1 ( E ( 1

(0, 0) r ((E

It is clear that

-(1 + E ) / 2 ) ,

(E

-

1)/2),

> 1.

E

i s p i e c e w i s e l i n e a r , c o n t i n u o u s , and

d i f f e r e n t i a b l e everywhere e x c e p t f o r

E

= 21.

I t is r e a d i l y

shown t h a t t h e o p t i m a l v a l u e f u n c t i o n o f t h i s p r o b l e m , f

*

(E)

= f[x(E),

€1,

i s convex ( d e f i n e d j u s t a f t e r Theorem

and t w i c e d i f f e r e n t i a b l e everywhere e x c e p t f o r

2.2.4),

E

= 21,

where it i s o n l y o n c e d i f f e r e n t i a b l e . U n f o r t u n a t e l y , as t h e n e x t example i l l u s t r a t e s , t h e s o l u t i o n s o f m a t h e m a t i c a l p r o g r a m s do n o t a l w a y s b e h a v e s o n i c e l y . E x a m p l e 2 . I. 2 min

X

EX

s . t . x 1. -1. The s o l u t i o n o f t h i s p r o b l e m i s g i v e n by E

> 0;

and i f

X(E) E

X(E)

c a n b e c h o s e n a s any v a l u e i n [-1,

m)

= -1 i f

if

E =

0;

< 0 t h e r e i s no f i n i t e s o l u t i o n of t h i s p r o b l e m .

Thus, f * ( E ) =

-E,

v a l u e s of

Thus, a s

E.

0 and h a s no l o w e r bound, f o r t h e r e s p e c t i v e E

v a r i e s i n a s m a l l neighborhood o f t h e

o r i g i n i n E1 t h e s o l u t i o n may b e f i n i t e and u n i q u e , it may b e unbounded, o r t h e r e may b e i n f i n i t e l y many s o l u t i o n s . I t s h o u l d b e c l e a r from t h e s e two s i m p l e e x a m p l e s t h a t

v e r y s m a l l p e r t u r b a t i o n s of t h e parameters o f a mathematical program c a n c a u s e a w i d e v a r i e t y o f r e s u l t s .

The p u r p o s e o f

t h i s c h a p t e r i s t o summarize and i l l u s t r a t e t h e work t h a t h a s b e e n d o n e t o d a t e i n p r o v i d i n g c o n d i t i o n s u n d e r which t h e

II

2.2 Objective Function and Solution Set Continuity

s o l u t i o n s o f n o n l i n e a r programs a r e l o c a l l y w e l l behaved and i n e s t i m a t i n g s o l u t i o n p r o p e r t i e s a s a f u n c t i o n o f problem parameters. The r e a d e r n o t f a m i l i a r w i t h o t h e r s p a c e s and norms may s u b s t i t u t e En f o r a l l m o r e g e n e r a l s p a c e s and t h e u s u a l E u c l i d e a n norm f o r o t h e r norms i n t h e f o l l o w i n g r e s u l t s . 2.2

O B J E C T I V E FUNCTION AND SOLUTION SET C O N T I N U I T Y

Some o f t h e e a r l i e s t work i n s t a b i l i t y a n a l y s i s f o r nonl i n e a r programming was c o n c e r n e d w i t h t h e v a r i a t i o n of t h e o p t i m a l v a l u e f u n c t i o n w i t h changes i n a parameter appearing i n t h e r i g h t hand s i d e o f t h e c o n s t r a i n t s , i . e . ,

involving

problems of t h e form:

s . t . g ( x ) 1.

min X f ( x ) , where f : E n point-to-set

-+

E

1

,

g: En

+

P1

E,

Em and

i s i n Em.

E

(E)

The t h e o r y of

maps ( B e r g e , 1963) h a s been u s e d f o r much o f t h e

a n a l y s i s o f t h i s problem.

Hogan (1973d) h a s p r o v i d e d a n

e x c e l l e n t development o f t h o s e p r o p e r t i e s of p o i n t - t o - s e t

maps

t h a t are e s p e c i a l l y u s e f u l i n d e r i v i n g such r e s u l t s . Next w e p r e s e n t s e v e r a l d e f i n i t i o n s and p r o p e r t i e s rel a t i n g t o point-to-set important r e s u l t s .

maps t h a t are needed i n a number o f

The terms "map," "mapping," and " p o i n t - t o -

s e t map" a r e u s e d i n t e r c h a n g e a b l y i n t h e s e q u e l . Given t w o t o p o l o g i c a l s p a c e s T and X I a p o i n t - t o - s e t mapping

r

from T t o X i s a f u n c t i o n which associates w i t h

e v e r y p o i n t i n T a s u b s e t o f X. say t h a t t h e point-to-set

mapping

F o l l o w i n g Berge ( 1 9 6 3 ) , w e

r

i s c o n t i n u o u s a t to

E

T

i f it i s b o t h u p p e r s e m i c o n t i n u o u s and lower s e m i c o n t i n u o u s a t

12

2 B a w Sensitivity and Stability Reaulta

These l a s t two n o t i o n s a r e e s t a b l i s h e d by t h e f o l l o w i n g

to.

d e f i n i t i o n , which w e i n c l u d e f o r c o m p l e t e n e s s .

r

Let

Definition 2.2.1.

be a p o i n t - t o - s e t

mapping from T

t o s u b s e t s of X.

r

(i)

open s e t S

i s lower s e m i c o n t i n u o u s a t t o e T i f , f o r e a c h

c

X satisfying S

n

r ( t0) #

t h e r e e x i s t s a neigh-

borhood N of t o , N ( t O ) , s u c h t h a t f o r e a c h t i n N ( t O ) ,

r(t)

n

s

(ii)

z

pr.

r

i s upper s e m i c o n t i n u o u s a t t o E T i f , f o r e a c h

open s e t S C X c o n t a i n i n g

r ( t0 )

t h e r e e x i s t s a neighborhood N

of t o , N ( t O ) , s u c h t h a t f o r e a c h t i n " t o ) ,

r

Furthermore, i f

r ( t )C

S.

i s lower s e m i c o n t i n u o u s a t e a c h p o i n t o f

t h e n it i s s a i d t o be lower s e m i c o n t i n u o u s i n T; and i f

T,

i s upper s e m i c o n t i n u o u s a t e a c h p o i n t of T w i t h

f o r each t , t h e n

r

r(t)

r

compact

i s s a i d t o be upper s e m i c o n t i n u o u s i n T.

The mapping l' is s a i d t o be c o n t i n u o u s i n T i f i t is b o t h upper and lower s e m i c o n t i n u o u s i n T. D e f i n i t i o n s , s i m p l e r and g e n e r a l l y easier t o a p p l y , t h a t a r e closely r e l a t e d t o these semicontinuity properties f o r point-to-set

r

Let

maps a r e based on t h e p r o p e r t i e s o f t h e i r g r a p h s .

be a p o i n t - t o - s e t

t h e g r a p h G of

r

is

mapping from T t o s u b s e t s o f X.

G 5

UteTUxEr(t)( t , x ) , i . e . ,

Then

G = { ( t , x)

T x X ~ EX r ( t ) ) .

L e t tn e T be s u c h t h a t tn + to. Then

D e f i n i t i o n 2.2.2.

(i)

r

i s open a t to e T i f , f o r e a c h xo

E

r ( t O ) there

e x i s t s a v a l u e m and a sequence {xn) C X s u c h t h a t xn e T ( t n ) f o r e a c h n > m and xn (ii)

xn

.+

r

-f

xo.

i s c l o s e d a t to e T i f xn

xo t o g e t h e r imply t h a t xo

E

E

l'(tn) f o r e a c h n and

r (to).

E

13

2.2 Objective Function and Solution Sct Continuity

The map

r

i s s a i d t o b e o p e n ( c l o s e d ) i n T i f i t i s open

( c l o s e d ) a t e a c h p o i n t o f T.

r

I t may a l s o b e n o t e d t h a t

is

c l o s e d i n T i f and o n l y i f i t s graph is a c l o s e d set i n T

x

X.

An a l t e r n a t i v e t o t h e Berge d e f i n i t i o n o f c o n t i n u i t y o f t h e map

r

i s g i v e n i n t h e s e t t i n g o f open a n d c l o s e d maps by

Hogan ( 1 9 7 3 d ) .

The map

r

i s s a i d t o be c o n t i n u o u s a t t o E T

i f it i s b o t h open a n d c l o s e d a t t o , and i t i s s a i d t o b e c o n t i n u o u s i n T i f i t i s b o t h o p e n a n d c l o s e d i n T.

The d e f i -

n i t i o n t h a t i s a p p l i c a b l e i n t h e f o l l o w i n g r e s u l t s w i l l be

c l e a r from t h e c o n t e x t , i . e . ,

t h e framework f o r a g i v e n r e s u l t

w i l l i n v o l v e e i t h e r s e m i c o n t i n u i t y c o n d i t i o n s o r open-closed c o n d i t i o n s , b u t n o t both.

I n t h e f o r m e r case, t h e Berge

d e f i n i t i o n s w i l l a p p l y a n d i n t h e l a t t e r case t h e Hogan d e f i n i t i o n s w i l l apply. The r e l a t i o n s h i p between t h e s e d e f i n i t i o n s may b e found i n Hogan ( 1 9 7 3 d ) , a n d i s summarized a s f o l l o w s :

( i )t h e d e f i n i -

t i o n s o f l o w e r s e m i c o n t i n u o u s a n d o p e n a t a p o i n t are e q u i v a l e n t , a n d ( i i )i f

r

i s u n i f o r m l y compact n e a r t o , i . e . ,

if

t h e r e i s a n e i g h b o r h o o d N of to s u c h t h a t t h e c l o s u r e o f t h e

setuteNI'(t) i s compact, t h e n l' i s c l o s e d a t t o i f a n d o n l y i f

r ( t0 )

is compact a n d

r (t)is

I t is clear t h a t i f

upper semicontinuous a t to.

r(tO)is

upper semicontinuous a t to, t h e n

a c l o s e d s e t and

r

e v e r , t h e converse does n o t hold.

r(t)

is

m u s t b e c l o s e d a t t o . HowThe f o l l o w i n g v e r y s i m p l e

example e x h i b i t s t h i s d i s t i n c t i o n between u p p e r s e m i c o n t i n u o u s and c l o s e d , i n t h e absence of t h e compactness assumptions. C o n s i d e r f : E1 0 i f x = 0.

-t

E1 d e f i n e d by f ( x ) = l / x i f

x # 0 and f (x) =

I t may r e a d i l y b e v e r i f i e d t h a t f i s c l o s e d b u t

n o t upper semicontinuous a t x = 0.

14

2 Basic Sensitivity and Stability Results

F o l l o w i n g Berge ( 1 9 6 3 ) , f o r r e a l - v a l u e d t h e n o t a t i o n l s c and

USC

functions we use

f o r lower and u p p e r s e m i c o n t i n u i t y ,

These a r e d e f i n e d , a c c o r d i n g t o t h e c u s t o m a r y

respectively.

convention, a s follows.

L e t @ be a r e a l - v a l u e d

f i n e d on t h e t o p o l o g i c a l s p a c e X.

f u n c t i o n de-

Then

( i ) @ i s s a i d t o be l s c a t a p o i n t x o

E

X if

E

X if

@ ( x o ,)

l i m inf $(x)

x'x 0 ( i i ) @ i s s a i d t o be u s c a t a p o i n t x o

lim

SUP

X'Xo

@ ( X I 2 @(xo).

The f u n c t i o n @ i s s a i d t o b e c o n t i n u o u s a t xo i f i t i s b o t h lsc and

USC

e a c h p o i n t of X.

a t xo and l s c ( u s c ) on X i f @ i s l s c ( u s c ) a t

I f @ i s continuous a t xo, then

l i m i n f @ ( x ) = l i m s u p @ ( x )= l i m @ ( X I . X'X0 x+xo x+xO Using t h e s e d e f i n i t i o n s , t h e f o l l o w i n g r e s u l t s a r e e a s i l y established f o r real-valued

f u n c t i o n s (Berge, 1 9 6 3 ) .

Assume

X and T a r e t o p o l o g i c a l s p a c e s i n t h e n e x t two r e s u l t s .

Theorem 2.2.1.

d e f i n e d on X

x

I f f is a real-valued

USC

(lsc) function

T and i f R i s a l o w e r s e m i c o n t i n u o u s

(upper

s e m i c o n t i n u o u s ) mapping from T i n t o X s u c h t h a t f o r e a c h T, R ( E )

#

PI,

then t h e (real-valued) function f

f * ( E ) = i n f x { f ( x , € 1 ~x

is

USC

E

*

,

E

in

d e f i n e d by

~ ( € I1

(lsc).

Theorem 2 . 2 . 2 .

If f is a continuous real-valued

d e f i n e d on t h e s p a c e X i n t o X such t h a t R ( E ) f

x T

function

and R i s a c o n t i n u o u s mapping o f T f o r each

E

i n T, then t h e

15

2.2 Objective Function and Solution Set Continuity

(real-valued) function f f*(E) = infx{f(x,

i s c o n t i n u o u s i n T. s(E)

= {X

E)

* , defined ~x e

R(E)I

F u r t h e r m o r e , t h e mapping S , d e f i n e d by

R(E)I~(X,

E

by

E)

= f*(E)},

i s a n u p p e r s e m i c o n t i n u o u s mapping of T i n t o X. C o n d i t i o n s t h a t imply t h e c o n t i n u i t y o f t h e s o l u t i o n o f a m a t h e m a t i c a l program h a v e b e e n g i v e n by D a n t z i g , Folkman, and S h a p i r o ( 1 9 6 7 ) a n d by Robinson a n d Day ( 1 9 7 4 ) .

L e t t i n g f be a

f u n c t i o n from a m e t r i c s p a c e X t o E l , w i t h R ( E ) C X , and d e f i n i n g S ( E ) = {x D a n t z i g e t aZ.

E

R(E) If(x) = infz{f(z)I z E

(1967) o b t a i n c o n d i t i o n s f o r

S

R(E)}}

t o vary i n a

When R i s d e f i n e d by l i n e a r i n e q u a l i t i e s , t h e y

c l o s e d manner.

o b t a i n u n d e r a p p r o p r i a t e c o n d i t i o n s t h a t S i s a c l o s e d mapping. Under t h i s same h y p o t h e s i s , l e t t i n g S

*

d e n o t e t h e mapping S

when it i s a s i n g l e t o n , t h e y o b t a i n c o n d i t i o n s which y i e l d t h e c o n t i n u i t y of S domain.

*

a s a function of t h e parameter

The domain D o f a p o i n t - t o - s e t

defined a s D = { t e T I T ( t ) #

map

r:

in its

E,

T

-+

X is

PI.

The f o l l o w i n g c l a s s i c a l c o n c e p t of p o i n t w i s e t o p o l o g i c a l

l i m i t s of a s e q u e n c e {R 1 o f s u b s e t s o f a m e t r i c s p a c e X i s n utilized. D e f i n e t h e i n n e r l i m i t and t h e o u t e r l i m i t o f {Rn}, respectively, a s

9Rn n-

and

{

: x

-

l i m Rn : { x n+m

If

9Rn n+m

-

E

XI 3 xn

E

XI 3 xn

= l i m Rn = R ,

n+m

Rn

E

E

j

Rnj

f o r n 2 n* and xn

x}

+.

f o r j 1. j* and x n

+

j

x}.

t h e n t h e l i m i t R o f {Rn} i s s a i d t o

16

2 Basic Sensitivity and Stability Results

e x i s t and w e w r i t e

l i m Rn = R n+m

or

Rn

R.

+

A c o n n e c t e d s e t M i s s u c h t h a t t h e r e do n o t e x i s t d i s j o i n t

open sets A1 and A2 s u c h t h a t M C A1 U A 2 , M

# 9, a n d

A1

8.

M n A 2 #

L e t R be a point-to - s et

Theorem 2 . 2 . 3 .

mapping from t h e

metric s p a c e T t o t h e s e t o f s u b s e t s o f E n , w i t h R ( E ) a c l o s e d set f o r each

E

i n T.

L e t D be t h e domain o f R.

is a connected s e t f o r each

*

R ( E ) i s compact.

{ c n } C D,

-+

t h e mapping S

*

E

i n D, a n d f o r some

+

E

*

i n D,

F u r t h e r m o r e , assume t h a t f o r e v e r y s e q u e n c e i m p l i e s R(cn)

E*

is continuous a t

+

E

*

R(E*).

,

if n

I f g i s a n a f f i n e f u n c t i o n from E Ax

Suppose R ( E )

b , where A i s a n m

E

*

Then, i f f

c o n s t a n t v e c t o r , a n d i f F$,(g)

= {x

C(En),

i s i n i t s domain.

t o E ~ i, . e . ,

n c o n s t a n t m a t r i x and b

x

E

i f g(x) = E

Em i s a

Mlg(x) 2 0 ) where M C En,

E

t h e f u n c t i o n g is s a i d t o be nondegenerate w i t h r e s p e c t t o t h e set M i f %(g)

h a s a nonempty i n t e r i o r and no component of g

i s i d e n t i c a l l y zero. SM(g) = { x

E

The c o n t i n u i t y o f t h e map d e f i n e d by

F$,(g) I f ( x ) = i n f z { f ( z ) I z

E

F $ , ( g ) ) } and S i ( t h e

mapping SM when i t i s a s i n g l e t o n ) a s f u n c t i o n s o f g i s g i v e n by t h e f o l l o w i n g t h e o r e m s . x, y

M imply t h a t a x

E

+

(1

The s e t M i s s a i d t o b e convex i f

-

cx)y

E

M f o r any

CL E

[o, 11.

I f f i s c o n t i n u o u s , g i s a f f i n e , and M i s

Theorem 2 . 2 . 4 .

c l o s e d and c o n v e x , t h e n SM i s c l o s e d a t e v e r y n o n d e g e n e r a t e p o i n t g. A f u n c t i o n 4 i s s a i d t o be convex o n a convex s e t M i f

@ ( a x+ (1 any a

E

-

10,

a @ ( x )+ (1

a)y) 11.

-

a ) @ ( y )f o r any x , y

A f u n c t i o n @ i s quasi-convex

i f and o n l y i f { x

E

E

M and

on a convex s e t M

M l @ ( x )5 k} i s convex f o r a n y r e a l number

17

2.2 Objective Function and Solution Set Continuity

E q u i v a l e n t l y , $ ( x ) i s q u a s i - c o n v e x on M i f m a x { $ ( x ) , $ ( y ) }

k.

>

$(Ax

+

(1

-

X)y) f o r e v e r y x , y

M and every X

E

[O,

E

11.

It

f o l l o w s t h a t a convex f u n c t i o n i s q u a s i - c o n v e x . I f t h e a s s u m p t i o n s o f Theorem 2 . 2 . 4

Theorem 2 . 2 . 5 .

hold

*

and f i s quasi-convex or % ( g )

i s bounded, t h e n SM i s con-

t i n u o u s a t a n o n d e g e n e r a t e p o i n t g of i t s domain. M o t i v a t e d by a p p l i c a t i o n s t o t h e t h e o r y o f economic c h o i c e , Robinson and Day ( 1 9 7 4 ) , c o n s i d e r i n g a g e n e r a l c o n s t r a i n t s e t R ( E ) , p r o v i d e c o n d i t i o n s t h a t g u a r a n t e e t h e c o n t i n u i t y of t h e

point-to-set

mapping whose v a l u e a t

is t h e set of solutions

E

o f t h e m a t h e m a t i c a l program minx f ( x ,

E)

s.t. x

E

R(E)

E

E

T,

where R ( E ) r e p r e s e n t s a c o n s t r a i n t s e t as a f u n c t i o n o f t h e parameter

E,

a n d X a n d T are t o p o l o g i c a l v e c t o r s p a c e s .

t h a t end, l e t f mopping S: T

+

*

(€1

=

infx{f (x,

X by S ( E ) = { x E

E)

Ix

E

To

R ( E ) ~ ,and d e f i n e t h e

R ( E ) If ( x , E ) = f* ( E )

1.

A f u n c t i o n $ is s a i d t o be s t r i c t l y quasi-convex on a

convex s e t M i f , f o r a l l xl, and a l l h

E

( 0 , l), $(Ax,

+

x2 e M such t h a t $(x,) (1

-

A)xZ) < $ ( x 2 ) .

$(x2)

A convex

f u n c t i o n i s also s t r i c t l y quasi-convex. A s s u m e t h a t t h e s p a c e X i s l o c a l l y convex

Theorem 2.2.6.

and t h a t R ( E ) # fl f o r e a c h convex-valued o n T ( i . e . f (x,

E)

= min{f

1

(x,

E)

,

,

E

T and t h a t R i s continuous and

R ( E ) i s convex f o r e a c h E i n T ) .

If

f 2 ( E ) 1 , where f l i s c o n t i n u o u s o n X x

T and s t r i c t l y q u a s i - c o n v e x

c o n t i n u o u s o n T.

E

i n x f o r each f i x e d

E,

a n d f 2 is

t h e n S i s c o n t i n u o u s a n d convex-valued on T.

18

2 Basic Sensitivity and Stability Results

A s might be expected,

stronger s t a b i l i t y characterizations

w e r e i n i t i a l l y found f o r t h e o s t e n s i b l y s i m p l e r r i g h t hand s i d e ( r h s ) problems P1(€)

and P 2 ( € ) .

(However, i t i s n o t e d i n

may be r e f o r m u l a t e d S e c t i o n 2 . 6 t h a t t h e g e n e r a l problem P 3 ( ~ ) a s a n e q u i v a l e n t r h s problem o f t h e form P2(€). ) R e c a l l t h a t t h e i n e q u a l i t y - c o n s t r a i n e d r h s p r o b l e m is

min X f ( x ) where now f : En P

(E)

1

are:

s . t . g ( x ) 1. E1

+

P1 ( € 1

E

a n d g: En

-+

A s s o c i a t e d w i t h problem

Em.

( i )t h e f e a s i b l e r e g i o n , R ( E ) = { x

(ii)t h e set D =

E E m l R ( ~ )#

{E

t h e e f f e c t i v e domain o f f * ;

g } , i.e.,

E

n

E Ig(x) 2 E);

t h e domain o f R a n d

( i i i )a s e t I ( E )= { x E E n l g ( x ) >

€ 1 a s s o c i a t e d w i t h t h e i n t e r i o r of t h e f e a s i b l e r e g i o n ; and ( i v ) t h e o p t i m a l v a l u e f u n c t i o n f * ( E ) = i n f x I f ( x ) Ix

R ( E ) ~ ,

E

a l s o c a l l e d t h e " m a r g i n a l f u n c t i o n " when t h e p e r t u r b a t i o n s are rhs perturbations. The f o l l o w i n g t h r e e t h e o r e m s , d u e t o Evans a n d Gould

(1970), p r o v i d e c o n d i t i o n s f o r t h e s t a b i l i t y o f t h e c o n s t r a i n t set, i.e.,

t h e f e a s i b l e r e g i o n R ( E ) , a s w e l l a s f o r t h e con-

t i n u i t y of the optimal value function.

I n the statements

of t h e f o l l o w i n g r e s u l t s i n t h i s s e c t i o n w e w i l l d e n o t e by R a point-to-set

mapping from t h e s e t D C Em t o t h e s e t o f a l l

s u b s e t s of En, w i t h t h e v a l u e a t

E

i n D g i v e n by R ( E ) .

The

i n t e r i o r o f t h e s e t D i s d e n o t e d by D 0 , a n d t h e c l o s u r e of I (E)

-

is d e n o t e d by I ( E )

.

A s s u m e g is continuous.

T h e o r e m 2.2.7. ( i ) The mapping R i s u p p e r s e m i c o n t i n u o u s a t

only i f t h e r e e x i s t s a vector

E'

<

E

E

i f and

s u c h t h a t R ( E ' ) i s compact.

( ii) I f R ( E ) i s compact a n d I ( € )# g, t h e n R i s l o w e r semicontinuous a t

E

-

i f and o n l y i f I ( E ) = R ( E )

.

19

2.2 Objective Function and Solution Set Continuity

Theorem 2.2.8.

*

f

(i) If f is lsc and R is upper semicontinuous at is lsc at

*

then f

E,

then

E.

(ii) If I ( € )# at

E,

is

USC

a,

f is usc and R is lower semicontinuous

at

E.

The following theorem is an immediate consequence of these two results, which give important realizations of Theorem 2.2.1. Theorem 2 . 2 . 9 . Suppose E is in Do and that f is continn Also assume that there exists a vector E ' E uous in E

.

-

such that R ( E ' ) is compact, and that I(€) = R ( E ) .

*

a continuous mapping and f

Then, R is

a continuous function at

E.

Theorem 2.2.9 is related to Theorem 2.2.2, giving conditions that imply the satisfaction of the hypotheses of Berge's theorem.

The question of the stability of the set of optimal

solutions and the stability of the optimal value function has been addressed by a number of authors.

Greenberg and

Pierskalla (1972), referring to problem P1, have shown that the solution set point-to-set mapping at

E

at

E.

if R is upper semicontinuous at

et al.

S E

is upper semicontinuous

*

and if f

is continuous

This result is very similar to one given by Dantzig (1967) and by Berge (1963).

The essential differences

among these results lie in the use of semicontinuity in Greenberg and Pierskalla (1972) and the closedness of'maps in Dantzig e t a l .

(1967), while the conclusion drawn in Berge

(1963) is based on the continuity of R.

Greenberg and

Pierskalla (1972) also extend the Evans-Gould results to allow for general constraint perturbations.

20

2 Basic Sensitivity and Stability Results

C o n s i d e r i n g t h e m a r g i n a l f u n c t i o n f o r convex programming p r o b l e m s , which i s c e n t r a l t o t h e c o n s t r u c t i o n o f decomp o s i t i o n a l g o r i t h m s f o r l a r g e - s c a l e n o n l i n e a r programs, Hogan ( 1 9 7 3 ~ )e s t a b l i s h e d c o n d i t i o n s f o r i t s c o n t i n u i t y .

T h e s e re-

s u l t s , i n c l u d i n g t h e c a s e i n which t h e p a r a m e t e r o f t h e probl e m a p p e a r s i n t h e o b j e c t i v e f u n c t i o n a s w e l l a s i n t h e con-

s t r a i n t s , a r e g i v e n i n t h e n e x t t w o theorems.

The f u n c t i o n @

was s a i d t o b e convex on a convex s e t M i f @ ( a x + (1 a @ ( x ) + (1

-

a ) $ ( y ) f o r any x , y

f o r x # y and a

E

M and a n y a

11.

[Or

E

a)y) 5 If

( 0 , 1) t h e i n e q u a l i t y i s s t r i c t , t h e n @ i s

s a i d t o be s t r i c t l y convex. ( s t r i c t l y ) concave i f - @ Tlieorem 2 . 2 . 1 0 .

E

-

A function Q i s s a i d t o be

i s ( s t r i c t l y ) convex.

L e t f*(E) = infx{f(x)Ix

E

M,

g ( x ) L €1.

If ( i ) M i s a compact convex s e t i n E

n

,

( i i ) f i s c o n t i n u o u s on M, (iii) gi i s u s c on M,

(iv) then f

*

e a c h gi i s s t r i c t l y c o n c a v e on M,

i s c o n t i n u o u s on i t s e f f e c t i v e domain i n E

Theorem 2 . 2 . 1 1 .

g(x,

and

.

L e t f * ( E ) = i n f x { f ( x , E ) ~x e M,

If

01.

E)

m

( i ) M i s a compact convex s e t i n E

n

,

( i i ) f and g a r e b o t h c o n t i n u o u s o n M

x

E

k

,

and

( i i i ) e a c h component o f g is s t r i c t l y c o n c a v e on M f o r

each

E,

then f

*

i s c o n t i n u o u s on i t s e f f e c t i v e domain.

R e f e r r i n g back t o problem P1(€) f ( x ) 5 f*(E) Em t o En.

+

6 for 6

,

l e t S & ( E )= { X

0) d e f i n e a p o i n t - t o - s e t

E

R(E)

I

map S g from

21

2.2 Objective Function and Solution Set Continuity

S t e r n a n d T o p k i s (1976), d e f i n i n g a n o t i o n of l i n e a r c o n t i n u i t y , e s t a b l i s h c o n d i t i o n s u n d e r which f continuous.

*

i s Lipschitz

Under c o n v e x i t y a s s u m p t i o n s on t h e problem func-

t i o n s t h e y a l s o show t h a t t h e map S6 whose v a l u e i s S & ( E ) , t h e

set of &-optimal s o l u t i o n s , i s continuous. i n t r o d u c e d a b o v e f o r P1(&) i s u s e d .

The n o t a t i o n

I t i s assumed t h a t f and

g are c o n t i n u o u s . The r e a l - v a l u e d f u n c t i o n 4 is s a i d t o

D e f i n i t i o n 2.2.3.

b e L i p s c h i t z c o n t i n u o u s o n a s e t M C X , where X i s a normed s p a c e , i f t h e r e e x i s t s a v a l u e A > 0 s u c h t h a t (I$(x) xIIx

-

yII f o r a l l x, y

@ ( y )1 5

M.

E

Suppose

Definition 2.2.4.

-

from T C Em t o subsets o f En.

r

is a p o i n t - t o - s e t

Then

r

mapping

is s a i d t o be uniformly

l i n e a r l y c o n t i n u o u s o n To C T i f t h e r e e x i s t s a v a l u e X > 0 such t h a t

inf))z xpr ( t ) f o r a l l t , t E To. Theorem 2 . 2 . 1 2 .

- XI)

Ilt

5 A

-

TI), f o r a l l z

E

r(t)

and

L e t Do C D ( t h e domain o f R ) a n d s u p p o s e

R ( 7 ) i s bounded f o r some

T in

D.

I f R is uniformly l i n e a r l y

c o n t i n u o u s w i t h c o n s t a n t k o n To = Do n

{ E ( E

-

2 € 1 , and i f f i s

L i p s c h i t z continuous w i t h c o n s t a n t c on R ( E ) , t h e n f * i s L i p s c h i t z continuous w i t h c o n s t a n t k c on T

0' S t e r n and T o p k i s show t h a t t h e f o l l o w i n g c o n d i t i o n s imply

t h a t R i s u n i f o r m l y l i n e a r l y c o n t i n u o u s on a s u b s e t T1 of D: ( i ) Do i s a c l o s e d s u b s e t of D , I ( & )# fi f o r e a c h E i n Do, R(E)

bounded and convex, and g

j

E

C1 a n d convex o n R ( E ) f o r a l l j ,

w i t h T1 = To; or ( i i ) t h e C o t t l e C o n s t r a i n t Q u a l i f i c a t i o n (see ( C Q 4 ) i n t h e n e x t s e c t i o n ) h o l d s a t some

e x i s t s E < E^ s u c h t h a t R ( E ) i s bounded a n d g w i t h T1 = D fl

{El

IIE -

211 5 a ) f o r some

CI

j

i n D, t h e r e E

> 0.

C2 f o r a l l j , They a l s o

show t h a t t h e c o n d i t i o n s g i v e n i n ( i ) ,a l o n g w i t h f

E

C'

and

22

2 Basic Sensitivity and Stability Results

convex imply t h a t t h e map S 6 of t h e s e t S 6 ( € ) o f & - o p t i m a l s o l u t i o n s i s u n i f o r m l y l i n e a r l y c o n t i n u o u s on To f o r e a c h

6 > 0 and a l s o o b t a i n t h e n e x t c o n t i n u i t y r e s u l t f o r S 6 . Theorem 2 . 2 . 1 3 .

quasi-convex, continuous a t

I f f a n d -gi,

i = 1,

R ( F ) i s bounded, a n d I ( T )

..., m,

# 0 , then

are s t r i c t l y S6

is

f o r each 6 > 0.

Hager (1979) r e c e n t l y o b t a i n e d L i p s c h i t z r e s u l t s f o r q u a d r a t i c programs w i t h u n i q u e s o l u t i o n s , assuming t h a t t h e s e e x i s t f c r s m a l l p e r t u r b a t i o n s a n d t h a t t h e p e r t u r b a t i o n parame t e r v a r i e s o v e r a convex s e t .

Lipschitz c o n t i n u i t y of t h e

s o l u t i o n and m u l t i p l i e r v e c t o r o f s u c h programs f o l l o w s u n d e r t h e hypothesis t h a t t h e g r a d i e n t s of t h e binding c o n s t r a i n t s s a t i s f y t h e l i n e a r i n d e p e n d e n c e c r i t e r i o n (CQ3, S e c t i o n 2 . 3 ) . I n both i n s t a n c e s an e s t i m a t e of t h e Lipschitz constant is provided. L i p s c h i t z p r o p e r t i e s of t h e o p t i m a l v a l u e f u n c t i o n and e s t i m a t e s o f t h e C l a r k e ( 1 9 7 5 ) g e n e r a l i z e d g r a d i e n t w e r e obt a i n e d by Gauvin ( 1 9 7 9 ) f o r problem P 2 ( € ) ( S e c t i o n 2.3)

and

e x t e n d e d by Gauvin a n d Dubeau ( 1 9 7 9 ) t o problem P 3 ( € ) ( S e c t i o n 2.3).

E x t e n s i o n s i n a more g e n e r a l s e t t i n g h a v e b e e n p r o v i d e d

r e c e n t l y by G o l l a n ( 1 9 8 1 a , b ) and R o c k a f e l l a r ( 1 9 8 2 a , b ) . 2.3.

DIFFERENTIAL STABILITY I n t h i s s e c t i o n w e c o n c e n t r a t e on t h e o r y t h a t h a s b e e n

developed t o a n a l y z e v a r i o u s s t a b i l i t y p r o p e r t i e s of t h e o p t i mal v a l u e f u n c t i o n , e v o l v i n g from a n a n a l y s i s of t h e r a t e o f change of t h i s f u n c t i o n a t a s o l u t i o n p o i n t .

W e begin w i t h a

b r i e f d i s c u s s i o n o f s e v e r a l w e l l known c o n s t r a i n t q u a l i f i c a t i o n s t h a t a r e t y p i c a l l y i n v o k e d i n o b t a i n i n g many o f t h e results.

23

2.3 Differential Stability

The constraint qualifications used in mathematical programming are regularity conditions which are generally imposed to insure that the set of Karush-Kuhn-Tucker multipliers (Karush, 1939; Kuhn and Tucker, 1951) corresponding to an optimal solution of a mathematical program is nonempty.

We

present here five qualifications that are frequently applied. These and a number of others are treated in some detail by Mangasarian (1969).

Throughout this discussion we shall assume

the constraint set R E {x g : En

Em and h: En

-+

vector functions. V' and , ' V

-+

En( g(x) 2 0, h(x) = GI, where

E

Ep are once continuously differentiable

Define B(x) E {il g 1 . (x) = 0).

The operators

without subscripts, denote differentiation with

respect to x. The Mangasarian-Fromovitz constraint qualification

CQl.

(Mangasarian and Fromovitz, 1967) is said to hold at a point

*

x

R if:

E

(i) there exists a vector z for all i

E

En such that Vgi(x*)z > 0

..., p, and 1, ..., are linearly

B(x*), Vh. (x*)z = 0 for j = 1,

E

7

(ii) the gradients {Vh.(x*), j = 3

independent. This condition is equivalent to (CQ1)': C

+ CP

3=1

x

E

*

ieB(x ) w.Vh.(x*) = 0 has no nonzero solution ui 2 0, w 3

3

*

uivgi(x 1 1'

for

R. If the gi are concave (or even pseudo-concave) functions

and the h . are affine, then CQ1 is equivalent to the well known 3

"Slater condition" (Slater, 1950), a general form of which we give as CQ2.

24

2 Basic Sensitivity and Stability Results

If the function 4: En

Definition 2.3.1.

+

E1 is differ-

entiable on the convex set M and @(y) 2 $(XI for all x, y with V@(x) (y

-

E

M,

x) 1. 0, then 4 is said to be pseudo-convex on M.

(The function 4 is said to be pseudo-concave if - 4 is pseudoconvex. ) A differentiable convex function is pseudo-convex and a

pseudo-convex function is both quasi-convex and strictly quasi-convex. The Slater constraint qualification is satisfied at

CQ2.

*

x

R if h

is affine for each j, gi is pseudo-concave for j each i and t.here exists a point jT e R with gi(x) > 0 for each E

i in B(x CQ3.

*

at x j

=

E

*

).

The linear independence assumption is said to hold

*

R if the gradients cVgi(x 1 ,

l,...,p} CQ4.

c

*

i

E

*

B(x 1; Vhj(x*),

are linearly independent.

If there are no equality constraints and uiVgi(x

*

)

=

*

0 has no nonzero solution ui L 0, for x

ieB(x ) * R, the Cottle constraint qualification is said to hold at x

B

.

(In the absence of equality constraints, CQ1 is equivalent to CQ4 1 CQ5.

The Kuhn-Tucker constraint qualification (Kuhn and

*

Tucker, 1951) is satisfied at x E R if, for each nonzero * * vector z e En satisfying Vgi(x ) z 2 0 for each i E B(x ) and

*

Vh.(x ) z = 0, j = 1, 3

..., p,

tiable arc originating at x

*

z is tangent to a once-differen-

and contained in R.

The relationships that hold among these qualifications, in addition to those already mentioned, are that CQ3 implies CQ1, which, in turn, is sufficient for CQ5. which also implies CQ5.

Also, CQ2 implies CQ4,

For a proof and further discussion of

25

2.3 Differential Stability

the relationships among these constraint qualifications see Arrow, Hurwicz, and Uzawa (1961), Bazaraa, Goode, and Shetty (1972), Mangasarian (1969), and Peterson (1.973). Robinson (1976b) has shown the equivalence of CQ1 and a form of local stability of the set of solutions of a system of inequalities.

Gauvin (1977) has shown that CQ1 is both neces-

.

sary and sufficient for the set of Lagrange multiplier vectors corresponding to a given local solution of a general NLP problem to be nonempty, compact, and convex.

In addition, Gauvin

and Tolle (1977) established that CQ1 is preserved under rhs perturbations.

If CQ3 holds, it is well known that the

Lagrange multiplier vector, corresponding to a given solution, exists and is unique. One of the earliest characterizations of the differential stability of the optimal value function of a mathematical program was provided by Danskin (1966, 1967). problem of minimizing f(x,

E)

Addressing the

subject to x e R, R some topo-

logical space, Danskin derived conditions under which the directional derivative of the optimal value function exists, and also determined its representation. D e f i n i t i o n 2. 3.2.

The (one-sided) directional derivative

of the function $ at the point t in the direction z is defined to be: DZ$(t) = lim [@(t + B z ) B+O+

-

$(t)l/B

if the limit exists. Theorem 2 . 3 . 1 .

Let R be nonempty and compact and let f

and the partial derivatives af/aEi be continuous. point

E

Then at any

in Ek and for any direction z e Ek the directional

26

2 Basic Sensitivity and Stability Results

d e r i v a t i v e of f D f

*

Z

(E)

=

*

e x i s t s a n d i s g i v e n by

min VEf(x, X€S(E)

where S ( E ) = cx E R l f ( x ,

E)

E)Z,

= f*(E)).

This r e s u l t has wide a p p l i c a b i l i t y i n t h e s e n s e t h a t t h e c o n s t r a i n t s p a c e R c a n b e any compact t o p o l o g i c a l s p a c e .

It

h a s b e e n e x t e n d e d by a number o f a u t h o r s , i n c l u d i n g Dem'yanov a n d Rubinov ( 1 9 6 8 ) , t o o t h e r s p a c e s a n d a v a r i e t y o f f u n c t i o n a l forms.

The p r i n c i p a l r e s t r i c t i o n o f t h i s r e s u l t i s t h a t t h e

set R does n o t v a r y w i t h t h e parame-ter

E.

However, s i n c e i n -

e q u a l i t y a n d e q u a l i t y c o n s t r a i n t s c a n be " a b s o r b e d " i n t o t h e o b j e c t i v e f u n c t i o n o f a program t h r o u g h t h e u s e of a n a p p r o p r i a t e a u x i l i a r y function, (Lagrangian, p e n a l t y f u n c t i o n , e t c . )

,

Danskin's r e s u l t is o f t e n a p p l i c a b l e t o a u x i l i a r y f u n c t i o n I t c a n a l s o be r e a d i l y a p p l i e d t o t h e o b j e c t i v e

methods.

f u n c t i o n o f t h e d u a l o f a convex program w i t h r h s p e r t u r b a tions. E

T u r n i n g t o t h e p r o b l e m where R d e p e n d s o n a p a r a m e t e r

through t h e i n e q u a l i t y g ( x ,

-gi

E)

L

0 , a n d f a n d t h e components

a r e convex, Hogan (197333) h a s shown t h a t D z f ^ ( e ) e x i s t s k The f o l l o w i n g t h e o r e m p r e s e n t s and i s f i n i t e f o r a l l z e E

.

t h e d e t a i l s of t h i s r e s u l t f o r f*(E) = i n f x { f ( x , g(xi

€ 1 2 0 1 , where

M is a s u b s e t of En.

t h i s problem i s d e f i n e d a s L ( x , u ,

E)

E)

Ix

E

M,

The L a g r a n g i a n f o r

:f ( x ,

E)

-

T

u g ( x , E).

F o r c o n v e n i e n c e a n d w i t h o u t l o s s o f g e n e r a l i t y , w e s h a l l assume t h a t t h e parameter v a l u e of i n t e r e s t i s

E

= 0,

unless otherwise

stated. Theorem 2 . 3 . 2 .

f a n d t h e -gi

L e t M be a c l o s e d and convex s e t .

are convex on M f o r e a c h f i x e d

t i n u o u s l y d i f f e r e n t i a b l e on M borhood o f

E

k = 0 in E

.

x

N(0)

,

E

Suppose

and a r e con-

where N ( 0 ) i s a n e i g h -

I f S(0) : { x E MI g ( x , 0) 1. 0 a n d

27

2.3 Differential Stability

f * ( O ) = f ( x , 0)) i s nonempty and bounded, there is a point

X

E

M s u c h t h a t g ( Z , 0) > 0 , t h e n D Z f * ( 0 )

e x i s t s and i s f i n i t e f o r a l l z

*

DZf (0) =

f * ( O ) i s f i n i t e , and

E

E

k , and

min max VJEL(x,u , 0 ) z x e S ( 0 ) ueK(x,O) min

-

max

{ ( v E f ( x , 0)

-

uTv E g ( x , o ) ) z } ,

x e S ( 0 ) ueK(x,O) where K ( x , 0 ) i s t h e s e t o f o p t i m a l L a g r a n g e m u l t i p l i e r s f o r t h e given x

E

S(0).

F o r convex programs t h e s e t K ( x , 0 ) i s t h e same f o r e x h x

This does n o t allow us t o drop t h e minimization

S(0).

E

over x

E

S(0)

*

i n t h e e x p r e s s i o n f o r DZf (0), however, s i n c e

t h e q u a n t i t y VEL(x, u , 0)z b e i n g m i n i m i z e d w i l l g e n e r a l l y deHowever, it d o e s a l l o w f o r a c o n s i d e r a b l e economy

pend on x .

i n t h e c a l c u l a t i o n of DZf

*

s i n c e t h e c o n s t r a i n t s e t K ( x , 0)

a s s o c i a t e d w i t h t h e i n n e r m a x i m i z a t i o n p r o b l e m d o e s n o t depend on x . Some r e c e n t i n v e s t i g a t i o n s of t h i s s o r t h a v e f o c u s e d on t h e extremal value function inequality-equality o p t i m i z a t i o n problems w i t h r h s p e r t u r b a t i o n s ,

of t h e form

c o n s i s t e n t w i t h o u r n o t a t i o n f o r P 3 ( ~ ,) where x E~

E

EpI

E

=

( E ~ ,E

~ E

)E

k

( ~k = m

+

constrained

E

En,

E~

E

Eml

p) i n t h e following r e s u l t s

i n t h i s section, unless otherwise specified.

28

2 Basic Sensitivity and Stability Results

W e a l s o define, f o r R ( E ) #

8,

S ( E ) = {X E R ( E ) If(x) =

f * ( E ) l , and t h e L a g r a n g i a n L ( x , u , w,

+ wT [ h ( x )

-

E

*I .

f(x)

E)

- uT [ g ( x ) -

= 0 , and i f CQ1 h o l d s f o r some x* E

1

*

.

I f R ( 0 ) # jlf, w i t h R u n i f o r m l y compact n e a r

Theorem 2 . 3 . 3 .

continuous a t

1

Given t h e s e d e f i n i t i o n s , Gauvin and T o l l e

(1977) have proved t h e f o l l o w i n g c o n t i n u i t y p r o p e r t y o f f

E

E

E

S(O),

then f * is

= 0.

F i a c c o (1980b,c) and Gauvin and Dubeau (1979) showed t h a t t h i s r e s u l t h o l d s f o r t h e g e n e r a l problem minx f ( x ,

E)

s.t. g ( x ,

E)

2 0,

h(x,

E)

= 0,

where t h e problem f u n c t i o n s a r e C1 i n ( x ,

E),

R ( E ) , and S ( E )

a r e d e f i n e d t o be t h e f e a s i b l e r e g i o n and s o l u t i o n sets o f P 3 ( ~ ) ,r e s p e c t i v e l y , and t h e p a r a m e t e r

E

is a vector i n E

k

.

I n t h e a b s e n c e o f e q u a l i t y c o n s t r a i n t s , R o c k a f e l l a r (1974) h a s shown t h a t , under c e r t a i n second-order c o n d i t i o n s , t h e function f

*

o f P 2 ( ~ )s a t i s f i e s a s t a b i l i t y o f d e g r e e two

condition, i.e.,

i n a neighborhood o f

twice-differentiable and f * ( O ) = Cp(0).

f u n c t i o n Cp:

Em+'

E

= 0 there e x i s t s a

-t

E1 w i t h f * ( E ) 2 Cp ( E )

Under t h i s s t a b i l i t y p r o p e r t y , bounds on t h e

d i r e c t i o n a l d e r i v a t i v e s o f f * (when t h e y e x i s t ) c a n be d e r i v e d . For convex programming problems o f t h e form P 3 ( ~ ) ,G o l ' s t e i n (1972) h a s shown t h a t a s a d d l e - p o i n t c o n d i t i o n i s s a t i s f i e d by t h e d i r e c t i o n a l d e r i v a t i v e of f*.

Gauvin and T o l l e ( 1 9 7 7 ) ,

n o t assuming c o n v e x i t y , b u t l i m i t i n g t h e i r a n a l y s i s t o problem P * ( E ) , e x t e n d t h e work o f G o l ' s t e i n and p r o v i d e s h a r p bounds

on t h e D i n i upper and lower d e r i v a t i v e s o f f

*

, also

r e q u i r i n g t h e e x i s t e n c e o f second-order d e r i v a t i v e s .

without These

29

2.3 Differential Stability

r e s u l t s w e r e e x t e n d e d by F i a c c o a n d H u t z l e r ( 1 9 7 9 a ) t o t h e general inequality-constrained

problem a n d by F i a c c o ( 1 9 8 0 ~ )

and Gauvin and Dubeau (1979) t o t h e more g e n e r a l problem The Gauvin-Dubeau e x t e n s i o n

P 3 ( ~ ) ,a n d a r e p r e s e n t e d n e x t .

follows t h e approach of Auslender (1979), w h i l e t h e a u t h o r ' s a p p r o a c h i s b a s e d o n e l i m i n a t i n g t h e e q u a l i t y c o n s t r a i n t and using elementary arguments t o d e a l w i t h t h e r e s u l t i n g s i m p l e r i n e q u a l i t y - c o n s t r a i n e d problem. Let L(x, u, w,

E)

:f ( x ,

E)

-

T

u g(x,

E)

+ wTh ( x ,

denote

E)

t h e u s u a l L a g r a n g i a n of p r o b l e m P 3 ( ~ )a n d l e t K ( x , 0) d e n o t e t h e s e t o f Karush-Kuhn-Tucker a s o l u t i o n x of P

3

f * ( O ) ] / B , where z

(E) E

at

E

v e c t o r s (u, w) corresponding t o

= 0,

*

l e t Q ( z , B ) = [f (62)

-

A s above, R ( E )

Ek i s a u n i t v e c t o r .

d e n o t e s t h e f e a s i b l e r e g i o n , S ( E ) t h e s o l u t i o n s e t , and t h e f u n c t i o n s f , g , h a r e assumed j o i n t l y o n c e c o n t i n u o u s l y d i f f e r e n t i a b l e i n (x, problem P 3 ( E )

E).

The f o l l o w i n g r e s u l t s h o l d f o r

.

The D i n i u p p e r a n d l o w e r d e r i v a t i v e s are d e f i n e d a s

l i m sup B+O+

Q ( z , B ) and l i m i n f Q ( z , B ) B+0+

, respectively.

I f R ( 0 ) # g, R i s u n i f o r m l y compact n e a r k E = 0 , a n d C Q 1 h o l d s f o r some x* E S(0) t h e n , f o r any z E E , Theorem 2 . 3 . 4 .

l i m inf Q(z, 8) 2 min VEL(X B +O+ ( u , w ) E K ( x * ,0) The n e x t c o r o l l a r y ,

*

,

u , w,

0)z.

(2.3.1)

f o l l o w i n g i m m e d i a t e l y from t h e theorem,

g i v e s a r e s u l t t h a t i s weaker, b u t u s e f u l i n t h e s e q u e l . I f R ( 0 ) # 8, R i s u n i f o r m l y compact n e a r k = 0 , and CQ1 h o l d s a t e a c h x E S O), t h e n , f o r any z E E , CoroZZary 2.3.5.

E

l i m inf Q ( z , B) 2 B+O+

inf min V E L ( X , u , w , 0)z. ( 2 . 3 . 2 ) x e S ( 0 ) ( u , w ) e K ( x 0)

30

Z Basic Sensitivity and Stability Results

Under the conditions of Corollary 2.3.5,

T h e o r e m 2.3.6.

for any z

E

Ek ,

lim s u p Q(z, 0 ) 5 0+0+

inf max VEL(xf u , w, 0 ) z . x e S ( 0 ) (u,w)EK(x,O)

CoroZZary 2.3.7.

(2.3.3)

If, in the hypotheses of Corollary 2.3.5,

CQ1 is replaced with CQ3, then for each z

E

Ek , DZf*(0) exists

and (2.3.4) where [u(x), w(x) I is the unique optimal Lagrange multiplier vector associated with x

E

S(0).

Auslender (1979) also obtained these bounds for the rhs perturbation problem P2(€), extending the results of Gauvin and Tolle (1977) by using a weaker form of the MangasarianFromovitz constraint qualification and adapting implicit func tion theorem results due to Hestenes (1975).

This allows him

to replace the differentiability assumption on the objective and inequality functions with the weaker requirement that they be locally Lipschitz.

Fontanie (1980) used the author's re-

duction approach in addressing an extension of Auslender's results to allow general perturbations.

Utilizing concepts of

subdifferential analysis and generalized derivatives, Rockafellar (1982a) has provided general bounds and sharper results under weaker assumptions than those given heretofore. Further significant extensions of these results have been provided by Rockafellar (1982b), utilizing a second-order constraint qualification and saddle-point properties of augmented Lagrangians.

31

2.3 Differential Stability

The following theorem (Fiacco and Hutzler, 1979a; Fiacco, 1980b) corresponds, under slightly different assumptions, to results obtained by Gol'stein (1972) and Hogan (197333) (Theorem 2.3.2) for a general class of problems of the form P 3 ( ~ )that are convex in x.

..., m be convex functions in x, and let the functions h j = 1, ..., p be jr Corollary 2 . 3 . 8 .

Let f and -gi, i = 1,

affine in x, with all functions jointly C1 in (x,

# pI, R is uniformly compact near for each x

*

DZf (0) =

E

*

E

E).

If R(0)

= 0, and CQ1 is satisfied

S(O), then DZf (0) exists for each z

E

Ek , and

inf max VEL(x, u, w, 0)z. xeS(0) (u,w)~K(x,O)

(2.3.5)

As noted following Theorem 2.3.2, the expression for (2.3.5) is theoretically and computationally simplified by noting that K(x, 0)

Z

K(O), a set that is the same for each x

E

S(0).

Although we are focusing attention on programs for which the spaces involved are finite dimensional, we note that most of these sensitivity results have been extended to infinite dimensional programs.

For example, Maurer (1977a,b) obtained

a characterization of the directional derivative of the extremal value function subgradient for problem P 3 ( ~ , ) and has applied his results to a class of optimal control problems. Lempio and Maurer (1980) have obtained similar bounds under analogous assumptions that are required to handle general perturbed infinite-dimensional programs of the form minimize f(x,

E),

subject to x

E

M1 and g(x,

are arbitrary closed convex sets.

E)

E

M2, where M1 and M2

Other extensions in

32

2 Basic Sensitivity and Stability Results

infinite-dimensional spaces may be found in the works of Dem'yanov and Pevnyi (1972), Gollan (1981a,b), Levitin (1976), Rockafellar (1982a,b), and others. If the inequalities (2.3.2) and (2.3.3) are applied to problem P2(€), then we obtain the result

(

inf min uTzl xeS(0) (u,w)~K(x,O)

< lim sup Q(z, B ) B+O+

5

+

wTz2) 5 lim inf ~ ( z ,B ) +O'B

inf

xeS(0)

max (uTzl (u,w)eK(x,O)

+ wTz 2 )

obtained by Gauvin and Tolle (19771, where we have defined z = (zl, z2)T The equation (2.3.4) resulting from the addi-

.

tional assumption of linear independence (CQ3) becomes DZf*(0) =

T T inf (u(x) z1 + w(x) z2). xes (0)

These relationships explicitly link the rate of change of the optimal value function f * with the optimal Lagrange multipliers (u, w).

In particular, they show that, under the given

*

assumptions, the rate of change of f

in any direction z is

bounded if the union of the set of optimal Lagsange multipliers associated with S(0) is bounded.

It is clear that many addi-

tional valuable insights might be obtained from this connection, in deducing optimal value stability properties from optimal multiplier properties and conversely. This line of inquiry has been extremely fertile.

We next

indicate some results, relating to this connection, that provide an elegant unification of important necessary optimality conditions. For the problem min

X

f(x)

s.t. g(x) 2 x e M

E,

Pi(€)

33

2.3 Differential Stability

where M i s a convex s u b s e t o f En,

f i s convex and t h e gi a r e

concave, G e o f f r i o n (1971) g i v e s a clear p r e s e n t a t i o n of s o m e basic results. f

*

H e f i r s t notes t h a t t h e optimal value function

i s convex.

For t h e r e s u l t of i n t e r e s t , t h e following d e f i -

n i t i o n s are needed.

$(W)

A v e c t o r v i s s a i d t o be a s u b g r a d i e n t o f 6 a t t h e

is finite. point

L e t 6 be a convex f u n c t i o n a n d assume

W if @ ( Z )+ (x

@(x)

-

W)Tv f o r a l l x .

The program P ' ( E ) i s c a l l e d " s t a b l e " i f f 1 t h e r e e x i s t s c > 0 such t h a t

*

(E)

i s f i n i t e and

T h i s c o n d i t i o n was a p p a r e n t l y f i r s t i n t r o d u c e d by G a l e (1967) t o obtain duality results.

Assuming P i ( € ) h a s a s o l u t i o n ,

G e o f f r i o n shows t h a t P i ( € ) i s s t a b l e i f and o n l y i f a n o p t i m a l Karush-Kuhn-Tucker

multiplier u

*

e x i s t s , and u

m u l t i p l i e r f o r P i ( € ) i f and o n l y i f u f

*

at

*

*

is an optimal

i s a subgradient of

E.

V a r i a n t s o f t h e l a s t r e s u l t h a v e a p p a r e n t l y been known f o r

some t i m e .

F o r example, R o c k a f e l l a r ( 1 9 7 0 ) shows t h a t i f

P i ( € ) i s a convex program s a t i s f y i n g S l a t e r ' s c o n d i t i o n a n d

has a s o l u t i o n , t h e n t h e s t a t e d r e s u l t h o ld s . R o c k a f e l l a r (1967) u s e d a n e q u i v a l e n t n o t i o n of s t a b i l i t y o f a problem o f t h e f o r m min

X

-

f (x)

s.t. x

E

M

g(Ax) and

Ax

f

To.

Here, A i s a l i n e a r t r a n s f o r m a t i o n from a r e a l ( f i n i t e o r

infinite-dimensional) vector space X t o another similar space T,

f i s a f i n i t e - v a l u e d convex f u n c t i o n o n M C X I where M Z

34

2 Basic Sensitivity and Stability Results

and c o n v e x , a n d g i s a f i n i t e - v a l u e d c o n c a v e f u n c t i o n o n To C T f o r To # fl a n d convex. be c a l l e d

The p e r t u r b e d p r o b l e m c o n s i d e r e d w i l l

F ( E )a n d i s t h e above problem w i t h t h e o b j e c t i v e

-

function f ( x )

g(Ax

-

E),

where

E

f i r s t t h a t t h e o p t i m a l v a l u e f* o f Suppose t h a t f * ( O ) i s f i n i t e .

i s i n T.

R o c k a f e l l a r shows

F ( E ) i s convex o n

T.

H e t h e n d e f i n e s P(0) t o be

"stably set" i f the d i r e c t i o n a l derivative

and t h e c o n s t r a i n t s o f by a t l e a s t o n e x ) .

P(0) a r e

"consistent"

(i.e., s a t i s f i e d

(Geoffrion ( 1 9 7 1 ) n o t e s t h a t t h i s condi-

t i o n i s equivalent to G a l e ' s condition, given i n t h e l a s t Rockafellar proves t h a t i f t h e r e e x i s t s a t least

paragraph.) one x

E

X where f i s f i n i t e a t x and g i s f i n i t e a n d c o n t i n u -

ous a t Ax, t h e n P(0) i s s t a b l y s e t a n d f * i s c o n t i n u o u s n e a r c

=

0.

H e p r o c e e d s t o p r o v e many s t r o n g d u a l i t y r e l a t i o n s h i p s

between t h e g i v e n convex program P ( 0 ) and a c o n c a v e program

s i m i l a r i n form t o B ( 0 ) a n d c o n s t r u c t e d by means o f t h e conjugate f u n c t i o n theory of Fenchel (1949). C l a r k e ( 1 9 7 6 ) h a s shown t h a t i f X i s a Banach s p a c e and f

i s l o c a l l y L i p s c h i t z , t h e n programs o f t h e form P 1 ( € ) , w i t h x

E

M C X a r e "normal" i n t h e s e n s e t h a t g e n e r a l i z e d Karush-

Kuhn-Tucker c o n d i t i o n s c a n be shown t o h o l d , e v e n i n t h e a b s e n c e o f d i f € e r e n t i a b i l i t y and c o n v e x i t y a s s u m p t i o n s .

t e r m s t h e program min

X

f (x)

s . t . g ( x ) 2. x

E

M,

E,

Clarke

35

2.3 Differential Stability

where M is a closed subset of En, normal if Karush-Kuhn-Tucker type multipliers exist for any solution x.

*

is said to be "calm" if f lim inf[f

*

(E')

- f

*

(E)

The problem P i ( € )

is finite and

E)l/llE'

-

Ell

>

(2.3.6)

-m

E'+E

where as usual

x

E

M, g(x) 1. € 1 ,

R(E) # B R(E) = 8 .

(We note that the limit quotient in Eq. (2.3.6) stability also used by Rockafellar (1967).)

is a form of

Using these no-

tions, Clarke showed that if the problem is calm it is also

*

E

=

0,

then the problem is calm and normal for almost all

E

in a

normal, and if f

is finite in a neighborhood of

Conditions sufficient for the calmness

neighborhood of 0.

(and hence the normality) of the problem are given in the following theorem. Th eor em 2 . 3 . 9 .

(i) -gi, i

If =

1,

..., m, are convex,

(ii) M is convex and bounded, (iii) f is bounded and Lipschitz continuous on M, and (iv) there exists a point x

E

M such that g(x) >

E,

then

P ' ( E ) is calm.

1

Extensions, refinements, and generalizations of many of the results in this section may be found in Hiriart-Urruty (1979) and Gollan (1981a,b), and a further sharpening and more general and unified treatment of these and related results in Rockafellar (1982a,b).

36 2.4.

2 Basic Sensitivity and Stability Results

IMPLICIT FUNCTION

THEOREM RESULTS

There a r e many forms of i m p l i c i t f u n c t i o n theorems t h a t have found e x t e n s i v e a p p l i c a t i o n i n f u n c t i o n a l a n a l y s i s . These theorems t r e a t t h e g e n e r a l problem o f s o l v i n g a n e q u a t i o n o f t h e form $(x, Y) = 0

(2.4.1)

f o r x i n t e r m s o f y.

w e l l known.

The c l a s s i c a l r e s u l t s i n t h i s a r e a a r e

For c o m p l e t e n e s s w e p r e s e n t two forms o f t h e A more complete d i s c u s s i o n o f

i m p l i c i t f u n c t i o n theorem.

t h e s e and o t h e r theorems i s c o n t a i n e d i n Bochner and M a r t i n ( 1 9 4 8 ) , H e s t e n e s ( 1 9 6 6 , 1 9 7 5 ) , and i n most advanced t e x t s i n functional analysis.

These r e s u l t s e x t e n d t o more g e n e r a l

spaces. Theorem 2 . 4 . 1 .

Function. 1

[ I m p l i c i t F u n c t i o n Theorem f o r Ck

Suppose $: En+m

+

En is a k t i m e s c o n t i n u o u s l y

d i f f e r e n t i a b l e mapping whose domain i s T. $

(F, y )

Suppose (E, 7)E T,

= 0 , and t h e J a c o b i a n w i t h r e s p e c t t o x , Vx$(T,

TI, is

Then t h e r e e x i s t s a neighborhood of y , N ( 7 ) C Em, k and a u n i q u e f u n c t i o n y E C [ N ( y ) ] , y: N(y) + En, w i t h y(y) = nonsingular.

x

and $ [ y ( y ) , yI = 0 for a l l y e

N(y).

The f u n c t i o n y i s s a i d t o be d e f i n e d i m p l i c i t l y by t h e e q u a t i o n $ [Y (y), y1 = 0.

i s used t o d e n o t e t h e j Ek w i t h r e s p e c t t o i t s j - t h argu-

I n t h e n e x t theorem, t h e n o t a t i o n $ j-th component of 9: Ek+'

..., $,)/a(x,, ..., x k ) d e n o t e s the J a c o b i a n ..., $ k ) w i t h r e s p e c t t o (xl, ..., x k ) .

ment, and

of

($I~,

-+

a($l,

Theorem 2 . 4 . 2 .

Function. 1

[ I m p l i c i t F u n c t i o n Theorem f o r A n a l y t i c

I f $ j (xl,

..., xk;

yl,

..., y,)

neighborhood o f t h e o r i g i n w i t h $ . ( O ,

I

is a n a l y t i c i n a

0) = 0 f o r j = 1,

..., k ,

37

2.4 Implicit Function Theorem Results

..., 4Q/a(x1, ..., xk)l -1 exists at x = y = 0, then the system of equations @.(xl, ..., Xki y1r Y,) = 7 for j = 1, ... k, has a unique solution x = xj(y,, ..., y , ) , j

and [a(@,,

. - . r

which vanishes for y

=

O r

0 and which is analytic in a neighbor-

hood of the or gin. Results of this type have particular applicability to sensitivity analysis in nonlinear optimization and have only recently been exploited in NLP.

Hildebrandt and Graves (1927)

have provided results on the existence and differentiability of solutions of Eq. 2.4.1.

Cesari (1966) has established condi-

tions under which the equation @(y, y) = 0 has at least one solution, and discusses the continuous dependence of y on parameters of the equation.

Rheinboldt (1969) has given

global existence theorems for the solution of (2.4.1), which leads to a "continuation property."

This continuation property

has been applied to the solution of parametric optimization programs.

Recent applications of implicit function theory re-

sults due to Hestenes (1975) have been made in differential stability by Auslender (1979) and Gauvin and Dubeau (19791, as noted in the previous section. Fiacco and McCormick (1968) and Duffin, Peterson, and Zener (1967) (for geometric programming (GP)) provided some of the first applications of an implicit function theorem to obtaining sensitivity information about the solution of a mathematical program.

Since then, additional results in this

area have been obtained by Bigelow and Shapiro (1974), Armacost and Fiacco (1974, 1975, 1976, 1977, 1978), Armacost (1976a,b), Fiacco (1976), Jittorntrum (1978, 1981) , Robinson (1974), and Spingarn (1977).

38

2 Basic Sensitivity and Stability Results

Robinson (1979) provided an implicit function theorem for a "generalized equation" where we seek a "solution" of 0

F(Y, € 1 + T(y)

E

where F: Y

x

P

-+

(2.4.2)

ES is a vector function, Y is an open set in

ES, P is a topological space, T is a closed point-to-set mapping from E Given

E,

S

into itself, and

E

is a parameter vector.

the problem is to find y such that the point-to-set

map on the rhs of ( 2 . 4 . 2 ) contains 0, or equivalently, such that the vector -F(y,

E)

T(y).

E

It is assumed that F(y,

E)

is (jointly) continuous and the partial Frgchet derivative

V F is continuous on Y x P. Y Given 5 E P , the linearization of F at T around yo

-

E

Y is

-

given by LF (y) = F ( y o , E ) + VyF(yO, E ) (y - yo). It is YO assumed that Yo is a nonernpty bounded convex set and that Y = Yo + y B C Y for some y > 0, where B is the (open) unit Y ball in ES. Conditions are given such that: (i) the set of

solutions

h

S(E)

of ( 2 . 4 . 2 ) in Yo

+ 6B is upper semicontinuous

in some neighborhood N(F) of T for some 6 = C$

if

E

#

;(El

(0, y l

(with

N(T)); (ii) g ( F ) = Yo; and (iii) for each

some neighborhood Na(F) and each

0 #

E

E

E

c1

h

S(E)

> 0 and

N,(T),

c 2CS)

+ ( A + a) max (IIF(y, Y

E)

-

F(y, T )11 Iy

E

YO}B

(2.4.3)

for some A > 0. Nonlinear complementarity and equilibrium problems and the Karush-Kuhn-Tucker conditions for P 3 ( ~ )can be expressed as realizations of ( 2 . 4 . 2 ) .

The latter constitute our interest

here and are expressed in the form ( 2 . 4 . 2 ) when we take T T s = n + m + p, y = (x, u, w) , F = (CxL, g , -hTIT, and

39

2.4 Implicit Function Theorem Results

T =

a$,,

w h e r e JIC d e n o t e s t h e i n d i c a t o r f u n c t i o n o f C d e f i n e d

by

and

a d e n o t e s t h e " s u b d i f f e r e n t i a l o p e r a t o r " (i.e., a$c(y) is

t h e s e t o f a l l s u b g r a d i e n t s o f $, a t y ) [ R o c k a f e l l a r (197O)J and C = E n x EY x Ep, where = { u E ?lu 1. 0 ) . The a d d i t i o n a l c o n d i t i o n s r e q u i r e d f o r t h e s e r e s u l t s a r e t h e e x i s t e n c e o f rl > 0 , a l o n g w i t h t h e f o l l o w i n g a s s u m p t i o n s f o r each yo (i) Y (ii) Y (iii) Y

E

Yo:

n

G-'(o)

yo

= yo;

n ~ - l ( y )c y Y n yo

G-'(o)

YO

+

~ l l y l lf o~r e a c h y e

n G - l ( y ) # 0 a n d convex f o r e a c h y yo

E

n ~ ;

nB,

where G = LF + T, a n d t h e i n v e r s e r-l o f a p o i n t - t o - s e t YO yo -1 map r: X + 2 a t Z E 2 i s d e f i n e d t o b e r (?) = { x E X ~ Z E r(x)1. Robinson a l s o shows t h a t a s s u m p t i o n (iii) may be r e p l a c e d by ( i i i ) 'VyF(yO,

E)

i s p o s i t i v e s e m i d e f i n i t e and T i s a

maximal monotone o p e r a t o r .

An o p e r a t o r T i s c a l l e d "monotone"

i f f o r e a c h (yl, v l ) , ( y 2 , v 2 ) i n t h e g r a p h o f T, i t f o l l o w s T t h a t (y, - y,) (v, v,) 2 0. I t i s c a l l e d "maximal monotone"

-

i f i t i s monotone a n d i t s g r a p h i s n o t p r o p e r l y c o n t a i n e d i n t h a t o f any o t h e r monotone o p e r a t o r . Many o t h e r r e s u l t s f o r p a r a m e t r i c n o n l i n e a r programs a r e d e v e l o p e d by Robinson, making u s e o f t h e g e n e r a l i z e d e q u a t i o n framework b r i e f l y i n d i c a t e d h e r e .

40

2 Basic Sensitivity and Stability Results

An i l l u s t r a t i o n o f a n a p p l i c a t i o n o f t h e above r e s u l t s i s p r o v i d e d by Robinson t o a n a l y z e t h e s o l u t i o n s e t o f t h e l i n e a r generalized equation 0

E

+

Ay

+

a

T(y)

(2.4.4)

where A i s a p o s i t i v e s e m i d e f i n i t e s T =

aqc a n d

x

s matrix, a

E

ES,

C i s a nonempty p o l y h e d r a l convex s e t i n E

i s shown t h a t t h e s o l u t i o n s e t o f

s

.

It

i s nonempty and

(2.4.4)

bounded i f a n d o n l y i f t h e r e e x i s t s 6 o > 0 s u c h t h a t f o r e a c h

s

x

[la' =

8.

s m a t r i x A ' and each a

-

a l l } < 60,

E

ES w i t h 6 '

t h e set ;(A',

:max(llA'

a ' ) : {y10

E

+

A'y

-

All,

a'

+

Under t h e a s s u m p t i o n t h a t p b e a bound o n ; ( A ,

a$JC(y)}

a) # 8,

i t is shown t h a t 3X > 0 s u c h t h a t

+ SA(A', +

a')

h6'(1

-

n

M

A6')

c S(A, a) -1

(1

+

(2.4.5)

p)B

where M i s any open bounded s e t c o n t a i n i n g

a ) and 6 ' < 61

(A,

for some 6

1 > 0 . L i n e a r a n d q u a d r a t i c programming and l i n e a r c o m p l e m e n t a r i t y p r o b l e m s can b e p o s e d i n t h e f o r m ( 2 . 4 . 4 ) . A p p l i e d t o a q u a d r a t i c program, f o r example,

(2.4.5)

provides an e x t e n s i o n of a r e s u l t due t o Daniel ( 1 9 7 3 ) .

It

concerns t h e s o l u t i o n s t a b i l i t y f o r p o s i t i v e d e f i n i t e q u a d r a t i c p r o g r a m s , which i s g i v e n n e x t f o r c o m p a r i s o n w i t h ( 2 . 4 . 5 ) . D a n i e l c o n s i d e r s a program i n E T

m i n x ( l / 2 ) x Kx

-

n

o f t h e form

k Tx

s.t. cx 5 c Dx = d , where K i s p o s i t i v e d e f i n i t e a n d s y m m e t r i c w i t h

smallest e i g e n v a l u e . c a s e of

(2.4.5).

7

> 0 its

Daniel obtained t h e f o llo win g s p e c i a l

41

2.4 Implicit Function Theorem Results

where xo solves the program above and x;J solves that program when K' and k' replace K and k, respectively. Using additional assumptions, a number of stronger results have been obtained which characterize more completely the relationship between a solution set and the optimal value function of a mathematical program to general perturbations appearing simultaneously in the objective function and anywhere in the constraints.

These problems generally have the

form min

X

f(x,

E)

s.t. g(x, € 1 2 0 h(x, where f: En

x

E)

Ek

= -+

0, El, g: En

x

Ek

+

Em, and h: En

x

Ek

-+

EpI

unless specifically stated otherwise. McCormick (Fiacco and McCormick, 1968) obtained conditions that guarantee the existence of a differentiable function of

E

that locally solves a particular form (problem P;(E), Section 3.4) of P3(€).

Robinson (1974) and Fiacco (1976) extended

this result to programs in which the perturbations appear generally, as in P 3 ( ~ ) . These results are established by applying an appropriate implicit function theorem to the firstorder necessary conditions that must hold at a local solution of P 3 ( € ) to prove the existence of a continuous or differentiable Karush-Kuhn-Tucker triple.

The author's extension,

given in the next theorem, establishes the existence of a once continuously differentiable (local) solution of P 3 ( ~ ) ,along

42

2 Basic Sensitivity and Stability Results

with associated unique differentiable optimal Lagrange multipliers.

This is the basic theorem for the sensitivity results

developed in this book, and is proved (with slightly weaker differentiability assumptions) in Chapter 3 (Theorem 3.2.2). T h e o r e m 2.4.4.

If

(1) f, g , h are C2 in (x, E) in a neighborhood of (x*, O),

*

[x

(2) the second order sufficiency conditions hold at

* * , u ,w I

(see Lemma 3.2.11,

*

(3) the linear independence assumption holds at x

complementary slackness with respect to u

*

holds at x

*

, and

,

then

*

(i) x

is a local isolated (i.e., locally unique)

minimizing point of P3(0) with unique Lagrange multipliers

*

*

U i W r

(ii) for

E

near 0, there exists a unique C1 function

[x(E), U(E), W(E) I T satisfying the second-order suffi* * * T ciency conditions for problem P 3 ( ~ ,) with y(D) = [x , u , w I , Y(E)

=

hence

X(E)

is an isolated local minimizing point of P 3 ( ~ )with

associated unique Lagrange multipliers u (E) and w ( E) (iii) for

E

,

and

near 0, the gradients of the binding con-

straints are linearly independent, and strict complementary slackness holds for U(E) and g(x, E). Fiacco (1976) also provided an explicit formula for the partial derivatives, as well as approximations based on classical penalty functions (Theorem 2 . 4 . 9 ) .

These results

provide the basis for the methodology presented in Part I1 of this book.

43

2.4 Implicit Function Theorem Results

Jittorntrum (1978, 1981), essentially complecing the development of results pursued by Bigelow and Shapiro (1974), obtained the following results, which do not require the strict complementarity condition (4) of Theorem 214.4.

How-

ever, the second-order part of the second-order sufficiency conditions (2) of the theorem must be strengthened to: v2L(x

ZT

*

,

*

u

* ,w ,

0)z > 0 for all z # 0 such that Vgi(x*, 0)z

= 0, for all i such that gi(x*) = 0 and u; > 0 and such that

*

Vh.(x 7

,

0)z = 0 for all j .

Following Robinson (1980a), we call

this condition, taken together with the first-order KarushKuhn-Tucker conditions (see Lemma 3.2.11,

“the strong second-

order sufficient conditions for problem P 3 (O).“ (i)

-

Conclusions

(v) of the next theorem were obtained by Jittorntrum in

the cited 1978 doctoral dissertation, Conclusion (vi) in the cited 1981 paper. Theoraern 2 . 4 . 5 .

*

hood of (x

,

If f, g, h are C L in (x,

*

in a neighbor-

0), if the strong second-order sufficient condi-

*

*

tions for problem P (0) hold at (x , u , w 3 * independence condition holds at x , then (i) x

E)

*

)

,

and if the linear

is a local isolated minimizing point of P3(0)

and the associated Lagrange multipliers u (ii) for

E

*

and w

*

are unique,

in a neighborhood of 0, there exists a unique

c o n t i n u o u s vector function y(E) = [x(E), u(E) , w(E) l T satis-

fying the strong second-order sufficiency condition for a local * * * T r minimum of the problem P 3 ( ~ )such that- y ( 0 ) = (x , u , w and hence, X(E) is a locally unique minimizer of P 3 ( € ) with associated unique Lagrange multipliers U(E) and w(E), (iii) linear independence of the binding constraint gradients holds at X(E) for

E

near 0,

44 (iv) every

E

t h e r e e x i s t 0 < a,

( 1 ~ 1 1<

with

B,

y <

6,

-

2 Basic Sensitivity and Stability Results

and 6 > 0 s u c h t h a t f o r

and i n any d i r e c t i o n z # 0 , t h e ( u n i q u e l y d e t e r m i n e d ,

(vi)

o n e - s i d e d ) d i r e c t i o n a l d e r i v a t i v e DZy( E ) o f ( t h e components o f ) [x(E),

u(E),

w(E)]

exists a t

= 0.

E

W e n o t e t h a t , under t h e a s s u m p t i o n s o f Theorem 2.4.5,

may be f u r t h e r concluded t h a t f d i f f e r e n t i a b l e and [ x ( E )

v and v i o f t h e theorem. near

E

(E)

E

it

i s once c o n t i n u o u s l y

, U(E) , w ( E ) ]

t i v e s i n any d i r e c t i o n n e a r

E]

*

has d i r e c t i o n a l deriva-

= 0 , thus extending Conclusions

Furthermore, s i n c e V E f

*

(€1 = VEL[y(~),

= 0 and t h e problem f u n c t i o n s a r e C2, w e n o t e t h a t

*

t h e (component by component) d i r e c t i o n a l d e r i v a t i v e DZ o f VEf e x i s t s near

Also,

E

=

0 i n any d i r e c t i o n z and i s g i v e n by

t h e d i r e c t i o n a l se'cond d e r i v a t i v e DI of f * ( E ) e x i s t s ,

g i v e n by 2 * DZf ( € 1 = z DZ

[(VEf* )

45

2.4 Implicit Function Theorem Results

These expressions are consistent with the formula we obtain in 2 *

Theorem 3.4.1 for VEf

(E)

*

when f

E

2

C

.

Jittorntrum also provides an approach for computing the directional derivatives, based on solving a collection of equations and inequalities derived from necessary optimality conditions, that must hold at a solution.

Using the strong

second-order sufficient conditions for P 3 ( ~ ) Robinson (1980a) subsequently obtained general results that also dispense with strict complementarity and essentially subsume Theorem 2.4.5 (Conclusions (i)

-

(v)) as particular realizations.

Kojima

(1980) proved additional stability results for P 3 ( ~ ) rwithout strict complementary slackness, using the degree theory of continuous maps.

It is also relevant to note that some efforts

have been made to relax the linear independence assumption, as well as the strict complementarity assumption, obtaining similar results.

Notably, Kojima (1980) showed that X(E) will

be continuous and an i s o Z a t e d local minimum of E

P(E)

near

= 0 if the Mangasarian-Fromovitz constraint qualification

(CQ1) holds at x

*

,

rather than linear independence of the

binding constraint gradients, p r o v i d i n g the strong second-order sufficient conditions hold at x

*

(u, w) associated with x

.

*

for a11 optimal multipliers

T h i s use of the strong second-order

sufficient conditions is consistent with Robinson's finding (1980a,b) that CQ1 and the usuaZ second-order sufficient con-

*

ditions (Lemma 3.2.1) holding at x not sufficient that for

E

X(E)

for aZZ optimal (u, w) are

be an isolated local minimum of P ( E )

near 0 (although these assumptions suffice to conclude

that x

*

= x(0) i s an isolated local minimum of P ( O ) , as noted

in the remarks following Lemma 3.2.1).

46

2 Basic Sensitivity and Stebility Results

S p i n g a r n (1977) e x t e n d e d Theorem 2.4.4 problem P

C

2

3

(E)

by c o n s i d e r i n g t h e

w i t h a d d i t i o n a l c o n s t r a i n t s t h a t restrict

to a

E

s u b m a n i f o l d T i n Ek a n d r e s t r i c t x t o a " c y r t o h e d r o n " H o f H e h a s shown a c e r t a i n s e t o f s e c o n d - o r d e r

c l a s s C2 i n En.

c o n d i t i o n s t o b e n e c e s s a r y f o r o p t i m a l i t y , and t h a t t h e s e cond i t i o n s a l s o i m p l y t h e r e s u l t s o b t a i n e d i n Theorem 2.4.4. Before s t a t i n g t h i s r e s u l t , w e i n t r o d u c e S p i n g a r n ' s definitions. L e t A a n d B b e f i n i t e ( p o s s i b l y empty) i n d e x s e t s , and f o r

i e A and j

E

B,

l e t { g i } and { h . } be f i n i t e c o l l e c t i o n s o f C 1 7

f u n c t i o n s d e f i n e d on t h e open s e t U A' C A,

c

f o r x e U and

Also,

let

V(x, A ' )

I

= {Vgi(x) = {x

Z(A')

i e A ' ) U { V h . ( x )I j

and

B},

E

7

UI g i ( x ) = 0 and

E

h . ( x ) = 0, f o r a l l i

E

7

A'

and j

E

B).

L e t H be a nonempty c o n n e c t e d s u b s e t o f

Definition 2.4.1. En.

En.

Then, f o r k 1. 1, H i s a c y r t o h e d r o n o f class 'C

i f there

e x i s t sets o f Ck f u n c t i o n s {gi} f o r i E A a n d { h . ) f o r j 7

d e f i n e d o n a n e i g h b o r h o o d N of x* (i) x

*

E

Z(A)

g i ( x ) 1. 0 f o r a l l i ( i i ) i f CaiVgi

ai 2 0 , t h e n ai = b ( i i i ) A.

j

and f o r x e N ,

E

A and h . ( x ) = 0 f o r a l l j 7

Xb.Vh i

j

=

C A1 C A a n d V

x

E

H i f and o n l y i f

,

E

A1)

A and j

E

j r B,

C s p a n V(x*,

t h a t Z ( A o ) =. Z ( A 1 ) . C o n s i d e r now t h e problem minx f ( x ,

E)

s.t. g(x,

E)

1. 0 , h ( x ,

E)

E

0 f o r some a i r b

= 0 for all i

B,

En w i t h

E

, +

E

= 0, x

E

H,

E

e

T

B,

with

AO) implies

41

2.4 Implicit Function Theorem Results

which i s P3 ( 6 ) w i t h t h e a d d i t i o n a l c o n s t r a i n t s mentioned earlier.

The f o l l o w i n g d e f i n i t i o n c o n t a i n s c o n d i t i o n s which

a r e s u f f i c i e n t f o r o p t i m a l i t y f o r Pi(€). Definition 2.4.2.

*

The p o i n t y* = ( x

,

u

*

L e t H b e a c y r t o h e d r o n of class C

,

2

.

w * ) i~s s a i d t o s a t i s f y t h e s t r o n g

s e c o n d - o r d e r c o n d i t i o n s f o r Pi(€) i f ( i ) x* e 1x1 g ( x , € 1 2 ( i i ) -VL(x

*

*

*

,u ,w , *

t h e normal c o n e t o H a t x

E)

0 1 n {XI h ( x , €1

=

01,

i s i n t h e r e l a t i v e i n t e r i o r of

,

(iii) t h e g r a d i e n t s o f t h e c o n s t r a i n t s t h a t a r e b i n d i n g

at x

*

are l i n e a r l y i n d e p e n d e n t , f o r e a c h i = 1,

(iv) gi(x*,

E)

= 0,

T

(v)

and

*

2

z [V L ( x

f o r a l l nonzero z

u

I

..., m,

* , w* ,

E)

ui* > 0 i f and o n l y i f

*

+ K[VL(x

,

u

*

,

w

*

, ~ ) l l z>

0

En f o r which

E

z is i n t h e l a r g e s t l i n e a r subspace co n tain ed

(a)

i n t h e t a n g e n t cone t o H a t x Vgi(x

(b)

*

,

E)Z

{ i l gi(x*, Vh.(x*,

(c)

E)Z

3

= 0 for all i E B E)

* *

I

(E)

= 0 1 , and

= 0 f o r j = 1, . . . I pr

where K [ * ] , t h e c u r v a t u r e o f t h e f a c i a l s u b m a n i f o l d of H which contains x

*

,

is an n

x

n matrix.

I f t h e s e t H i s t a k e n t o be E n r t h e n Pi(€) r e d u c e s t o t h e program P 3 ( € ) and ( i i )a n d (v) above become t h e f a m i l i a r conditions

*

*

*

( i i ' ) VL(x I u , w , E ) = 0 , a n d T 2 * * * ( v ' ) z v L ( X r u , w r € 1 2 > o f o r a l l n o n z e r o z e E~ f o r which ( b ) and (c) above hold.

48

2 Basic Sensitivity and Stability Results

With t h e s e d e f i n i t i o n s , w e now s t a t e S p i n g a r n ' s r e s u l t . Consider t h e problem Pi(€).

Theorem 2 . 4 . 6 .

*

second-order c o n d i t i o n s h o l d a t y* := ( x Ep,

of

E

*

,u ,w

I f the strong * ) e~ H

x

Em x

* E Ek , t h e n t h e r e e x i s t neighborhoods N1 C Ek and N 2 C En

E*

u(E),

and a C1 f u n c t i o n

and x*, r e s p e c t i v e l y , W(E)

I T d e f i n e d on

Y(E)

= [x(E),

such t h a t :

N1

( i ) Y ( E ) s a t i s f i e s t h e s t r o n g second-order c o n d i t i o n s for P ~ ( E ) , (ii) f o r each

E

i n N1,

E N2

X(E)

i s an i s o l a t e d local

minimizer f o r P ' ( E ) , and 3

(iii) f o r e a c h w(E),

E

associated with

i n N1, X(E)

t h e Lagrange m u l t i p l i e r s u ( E ) , a r e uniquely determined.

Under s l i g h t l y weaker a s s u m p t i o n s t h a n t h o s e invoked by F i a c c o ( 1 9 7 6 ) , Robinson (1974) h a s o b t a i n e d r e s u l t s f o r probl e m P3(€) s i m i l a r t o t h o s e s t a t e d i n Theorem 2 . 4 . 4 , t h e c o n t i n u i t y of t h e Karush-Kuhn-Tucker

proving

t r i p l e , and u s i n g t h e

r e s u l t s t o d e r i v e bounds on t h e v a r i a t i o n o f Y ( E ) . L e t T b e a Banach s p a c e , To C T, M C En,

Theorem 2 . 4 . 7 .

w i t h M and To open sets.

L e t f , g, and h have second p a r t i a l

d e r i v a t i v e s w i t h r e s p e c t t o x which a r e j o i n t l y c o n t i n u o u s on For

M x To.

E

*

*

i n To, suppose (x

Tucker t r i p l e o f P 3 ( € ) .

* * ,u ,w )

i s a Karush-Kuhn-

Also assume t h a t t h e l i n e a r

independence, s t r i c t complementary s l a c k n e s s , and second-order

*

sufficiency conditions hold a t (x

*

= (x

* * , u , w ).

Then

*

(i) t h e r e e x i s t s a c o n t i n u o u s f u n c t i o n Y ( E ) w i t h Y ( E )

* , u , w*jT,

and f o r e a c h

Karush-KuhnLTucker [VL(X,

U,

W,

E

i n T o , Y ( E ) i s t h e unique

t r i p l e o f P 3 ( ~ and ) t h e unique z e r o o f

€1, U1gl(x, € 1 ,

..., Umg,,,(x,

€1, h l ( x , € 1 ,

49

2.4 Implicit Function Theorem Results (11)

for

near

E

E*,

X(E)

i s an i s o l a t e d l o c a l minimizing

p o i n t o f P 3 ( ~ ,) a n d (iii) l i n e a r i n d e p e n d e n c e , s t r i c t complementary s l a c k n e s s ,

and t h e s e c o n d - o r d e r s u f f i c i e n c y c o n d i t i o n s h o l d f o r

*

* 2 and N X of ( x , 2 i n N X w e have E

IIY

-

Y(E)

II

near

E

*

.

Under t h e h y p o t h e s e s o f t h e p r e v i o u s

Theorem 2 . 4 . 8 .

theorem, f o r any X

E

1 ( 0 , 1), t h e r e e x i s t n e i g h b o r h o o d s N X o f * * 1 u , w ) s u c h t h a t f o r a n y E i n NA and any y E

5 (1 - x ) - ’ I I M [ Y ( E * )I

E

* I -1 I I * I I F ( E~ ) , 1 1 ,

...,

...,

where M i s t h e J a c o b i a n o f F = [VL, ulgl, umgmI h l , T h 1 w i t h r e s p e c t t o y. Robinson a p p l i e s t h e s e r e s u l t s t o P d e t e r m i n e t h e r a t e of c o n v e r g e n c e o f a l a r g e f a m i l y o f

*

a l g o r i t h m s f o r s o l v i n g P3 ( E )

.

E x t e n s i o n s o f r e s u l t s a n a l o g o u s t o Theorem 2 . 4 . 4 i n f i n i t e dimensional s p a c e s have been o b t a i n e d .

but i n

I o f f e and

Tikhomirov (1980; R u s s i a n o r i g i n a l , 1974) u s e a n i m p l i c i t f u n c t i o n t h e o r e m t o p r o v e t h a t [ x ( E ) , w ( E ) ] e C’ Karush-Kuhn-Tucker

t r i p l e near a given

E

is t h e unique

f o r t h e problem

minx f ( x )

s . t . F(x) = where x

f

X,

F:

E,

X

+.

T , a n d X a n d T are Banach s p a c e s , and

a s s u m p t i o n s a n a l o g o u s t o t h o s e o f Theorem 2 . 4 . 4

are invoked. 1 2 g ( x ) 1. E , F ( x ) = E ,

S i m i l a r r e s u l t s f o r Pi(€): minx f ( x ) s . t . 1 E~ e Eml E~ e T, E = ( E , E ~ w e) r e ~o b t a i n e d by W i e r z b i c k i a n d

Kurcyusz ( 1 9 7 7 ) , a g a i n u s i n g a n i m p l i c i t f u n c t i o n , where X a n d T are H i l b e r t s p a c e s .

I n t h e r e m a i n d e r o f t h i s s e c t i o n w e summarize s e v e r a l a d d i t i o n a l i m p o r t a n t r e s u l t s a s s o c i a t e d w i t h Theorem 2 . 4 . 4 . These w i l l be t r e a t e d i n d e t a i l i n C h a p t e r s 3-8.

50

2 Basic Sensitivity and Stability Results

If we consider the logarithmic-quadratic mixed barrierpenalty function (Fiacco and McCormick, 1968) associated with problem P3(€) given by m

P

we have the following theorem due to Fiacco (1976). The proof of this result is given in Chapter 6 (Theorem 6.2.1). [Locally unique C1-KKT triple associated

Theorem 2 . 4 . 9 .

with a locally unique unconstrained local minimum of the barrier-penalty function W(x,

E,

of Theorem 2.4.4 are satisfied. (E,

r).]

Assume the hypotheses

Then, in a neighborhood of

r) = (0, 0) there exists a unique, once continuously

differentiable function y(~', r) = [x(E, r),

U(E,

rr),W(E, r)l

T

satisfying: VL(x, u, w,

E)

=

u.g.(x, 1 1

E)

= r,

i = 1,

h.(x,

E)

= w.r, 7

j = 1, .a.

3

*

with y(0, 0) = [x

,

u

0,

*

,

* T w ]

.

..., m, r

(2.4.6)

and

Pr

Furthermore, for any

(E,

r) near

(0, 0) with r > 0, X(E, r) is a locally unique unconstrained local minimizing point of W(x, for each i = 1,

..., m,

E,

and V:W(x,

r) with gi[x(E, r), € 1 > 0 E,

r) is positive definite

at x = X(E, r). The existence of higher-order derivatives of Y(E) depends on the degree of (continuous) differentiability of the problem functions.

This follows directly from an application of the

classical implicit function theorem to the first-order necessary conditions for a solution of P 3 ( € ) .

An analogous result

holds for Y(E, r), the solution of (2.4.6).

In fact, it

51

2.4 Implicit Function Theorem Results

f o l l o w s e a s i l y t h a t under t h e a p p r o p r i a t e c o n d i t i o n s n o t o n l y do h i g h e r - o r d e r

d e r i v a t i v e s of

Y(E,

r ) e x i s t , but these deriva-

t i v e s converge t o t h e corresponding d e r i v a t i v e s of Y ( E ) .

This

r e s u l t i s s t a t e d n e x t and l a t e r subsumed i n C o r o l l a r y 6.4.1. L e t f , g , and h have c o n t i n u o u s d e r i v a -

Theorem 2 . 4 . 1 0 .

t i v e s o f a l l o r d e r s up t o p

+

1.

Assume t h a t the conditions

Then, i n a neighborhood o f

of Theorem 2 4 . 4 a r e s a t i s f i e d .

r ) = ( 0 , 0 ) , t h e r e e x i s t s a unique f u n c t i o n

(E,

r)

Y(E,

[x

=

E,

r) ,

U(E,

r),

W(E,

Y(E,

r)

E

Cp,

r ) l Ts a t i s f y i n g ( 2 . 4 . 6 1 ,

with

r)

Y(Er

+

Y(E)r

and t h e j - t h p a r t i a l d e r i v a t i v e o f y w i t h r e s p e c t t o

at

E

(E,

r ) c o n v e r g e s t o t h e j - t h p a r t i a l d e r i v a t i v e of y a t

r

0 for

+

(E,

r ) n e a r ( 0 , 0 ) , where j = 1,

...

r

E

as

p.

w a s s u b s e q u e n t l y g i v e n by Buys

An a n a l o g o f Theorem 2 . 4 . 9

and Gonin (1977) and Armacost and F i a c c o ( 1 9 7 7 ) , u s i n g an augmented L a g r a n g i a n p r e v i o u s l y u t i l i z e d by Buys ( 1 9 7 2 ) . gives “exact” s e n s i t i v i t y r e s u l t s , i.e., w i t h t h o s e of Theorem 2 . 4 . 4 . a t e d w i t h problem P

3

(E)

r e s u l t s coinciding

The augmented L a g r a n g i a n a s s o c i -

i s defined a s

h

L(xi

Ur

Wr

= f(x,

E r

E)

C)

-

2

ieJ(E

ieK(E)

where J ( E )= { i l u cgi(x’

E)

< 01.

i

-

cgi(xr

It

E)

L 0 ) and K ( E ) = { i l ui

-

2 Basic Sensitivity and Stability Results

52

Under t h e a s s u m p t i o n s of Theorem 2 . 4 . 4 ,

Theorem 2 . 4 . 1 1 . there exists c

*

such t h a t f o r

n e a r 0 and c > c

*

,

there

f u n c t i o n y ( E , c) = [ x ( E , c ) , u ( E , c ) ,

c1

e x i s t s a unique

E

c ) I ' s~ a t i s f y i n g

W(E,

( i ) v ~ ( x ,u, w,

( i i ) u1 .g1 . (x, (iii) h . ( x , 3

with

= 0,

w e have t h a t

..., m, ..., p ,

j = 1,

and

F u r t h e r m o r e , f o r any

c) = Y ( E ) .

Y(E,

0,

=

i = 1,

= 0,

E)

E)

c)

E,

n e a r 0 and c > c*

E

c) is a l o c a l l y unique unconstrained l o c a l

X(E,

h

minimizing p o i n t o f L [ x ,

U(E,

c),

W(E,

c),

*

p o s i t i v e d e f i n i t e f o r [ x , u, w] n e a r [x

2^

cl and V L i s

E,

* * , u ,w I.

This

A

r e s u l t , r e a d i l y f o l l o w i n g from t h e d e f i n i t i o n o f L and Theorem 2.4.4,

is proved and f o r m u l a s f o r V E y ( € , c ) are developed i n

C h a p t e r 7. Armacost and F i a c c o (1975) also o b t a i n e d f o r m u l a s f o r t h e f i r s t - and second-order d e r i v a t i v e s o f t h e o p t i m a l v a l u e f u n c t i o n of t h e problem P 3 ( ~ ) . These f o l l o w from t h e n e x t r e s u l t , which i s a n e a s y consequence o f Theorem 2.4.4. Suppose t h e a s s u m p t i o n s of Theorem 2.4.4

Theor em 2 . 4 . 1 2 .

h o l d f o r P 3 ( ~ ) . Then, i n a neighborhood of

E

= 0, t h e optimal

value function f*(E) is t w i c e continuously d i f f e r e n t i a b l e a s a f u n c t i o n of

E,

and

(i) f*(E) = L [ x ( E ) ,

U(E)

,~

( € 1 ,€ 1

( i i ) VEf*(E) = V E L [ x ( ~ )U, ( E )

,~

( € 1 , €1, and

(iii) v E2 f * ( E ) = V ~ ~ V ~ L [ X ( u E ()E, ) , C o n s i d e r t h e problem minx f ( x )

s.t. gi(x) L

E

h.(x) =

E

3

~ ~

..., m, = 1, ..., p.

i = 1,

I +

j

~

I

w(E)

,

ElT].

53

2.4 Implicit Function Theorem R e h u h

The L a g r a n g i a n f o r P 2 ( ~ )i s g i v e n by m L ( x , u, w ,

E)

=

f(x)

2

+

i=l

-

wj [hj(x)

j=1

E .

7 +m

I.

The f o l l o w i n g r e s u l t f o l l o w s i m m e d i a t e l y from Theorem 2 . 4 . 1 2 . I t i s s t a t e d s e p a r a t e l y because of i t s importance.

Theorem 2 . 4 . 1 3 .

L e t f , g , and h b e t w i c e continuously

d i f f e r e n t i a b l e i n x i n a neighborhood of x a s s u m p t i o n s o f Theorem 2 . 4 . 4

*

.

Suppose t h e Then, i n a

hold f o r P 2 ( € ) .

neighborhood of E = 0,

(i)

VEf*(E) = [u(EIT,

Theorem 2 . 4 . 1 2

and 2.4.13

-w(E)

T

I

T

a r e p r o v e d i n C h a p t e r 3 (Theorem

3.4.1 and C o r o l l a r y 3.4.4).

Formulas f o r VEf

*

i n t e r m s of t h e

d e r i v a t i v e s o f t h e o r i g i n a l p r o b l e m f u n c t i o n s are d e v e l o p e d i n C h a p t e r 3 , S e c t i o n 3.4. A p p l i c a t i o n s o f t h e s e n s i t i v i t y r e s u l t s g i v e n i n Theorems 2.4.4,

2.4.9,

and 2.4.12

have b e e n implemented v i a a p e n a l t y

f u n c t i o n a l g o r i t h m and a r e r e p o r t e d subsequently.

Here, w e

b r i e f l y s u m m a r i z e some o f t h e i n i t i a l c o m p u t a t i o n a l e x p e r i e n c e . Armacost a n d F i a c c o (1974) i l l u s t r a t e d c o m p u t a t i o n a l a s p e c t s o f some o f t h e r e s u l t s o u t l i n e d i n t h i s s e c t i o n . Using t h e SUMT ( S e q u e n t i a l U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e ) V e r s i o n 4 computer c o d e d e v e l o p e d by Mylander, H o l m e s , a n d McCormick (1971) a n d a s u b r o u t i n e f o r s e n s i t i v i t y

analysis coded by Armacost and Mylander (1973) that implemented a procedure for calculating the solution parameter derivatives based on the results given in Theorems 2.4.4, 2.4.9, and 2.4.12, they obtained results that computationally corroborated the theory, e.g., the convergence of the first partial derivatives of the optimal solution and the optimal value function of several problems.

The resulting code was further developed and

refined by Armacost (1976a,b).

Subsequently, Armacost and

Fiacco (1978) used this computer program, which came to be known as SENSUMT, to analyze the behavior of the solution of an inventory problem relative to changes in several problem parameters.

This application is discussed in Chapter 8.

Fur-

ther development of SENSUMT and other applications will be discussed as the theory is developed. 2.5.

OPTIMAL VALUE AND SOLUTION BOUNDS The foregoing results are concerned mainly with the con-

tinuity or differentiability properties of the optimal value

*

function f

(E)

or a Karush-Kuhn-Tucker triple [x(E), u(E) ,

~ ( € 1 1 . Among the results we have given, only a few, e.g., Robinson's generalized implicit function theorem result (2.4.3) for the general system (2.4.21, the particular realization of this obtained earlier for the positive definite quadratic program by Daniel in Theorem 2.4.3, Robinson's bounds on a Karush-Kuhn-Tucker triple given in Theorem 2.4.8, and Stern and Topkis' Lipschitz condition on the optimal value function of P1 ( E ) given in Theorem 2.2.12 provide parametric solution bounds. All of these results involve valuable existence proofs,

providing incisive characterizations of solution properties

2 . 5 Optiinal Valuc

dnd

55

Solution Hound\

r e l a t i v e t o parametric perturbations.

N o n e t h e l e s s , much

remains t o b e done t o d e v e l o p computable t e c h n i q u e s f o r c a l c u l a t i n g u s e f u l n u m e r i c a l e s t i m a t e s o f bounds f o r g e n e r a l c l a s s e s of problems. The q u e s t i o n a r i s e s w h e t h e r p r a c t i c a l l y u s e f u l computable p a r a m e t r i c s o l u t i o n bounds c a n b e c a l c u l a t e d f o r g e n e r a l c l a s s e s of NLP p r o b l e m s .

P r a c t i c a l i t y demands t h a t t h e re-

q u i r e d e f f o r t b e f e a s i b l e and t h e p e r t u r b a t i o n s b e f i n i t e , while usefulness suggests a d d i t i o n a l l y t h a t a s u i t a b l e degree of a c c u r a c y b e a c h i e v e d .

W e find that generally applicable

bounds r e s u l t s m e e t i n g t h e p r e s c r i b e d p r o p e r t i e s a r e r a r e indeed.

I n f a c t , t h e r e a p p e a r t o b e two f u n d a m e n t a l d i r e c -

t i o n s , o n e d u e t o K a n t o r o v i c h ( 1 9 4 8 ) and o n e d u e t o Moore (1966, 1977).

Both i n v o l v e N e w t o n ' s method, t h e l a t t e r

i n c o r p o r a t i n g r e s u l t s from i n t e r v a l a r i t h m e t i c .

Most g e n e r a l

a p p r o a c h e s f o r o b t a i n i n g p a r a m e t r i c s o l u t i o n bounds and e r r o r bounds s e e m t o b e a s s o c i a t e d w i t h one o f t h e s e b a s i c r e s u l t s . Examples o f i n t r i g u i n g a n d i m p o r t a n t r e c e n t e r r o r bounds r e s u l t s o f t h e K a n t o r o v i c h t y p e may b e found i n t h e work o f M i e l ( 1 9 8 0 ) , a n d o f a m i x t u r e o f t h e s e t y p e s i n t h e work o f Mancini and McCormick ( 1 9 7 6 , 1 9 7 9 ) , and McCormick ( 1 9 8 0 ) . T h e o r e t i c a l a n d n u m e r i c a l c o m p a r i s o n s o f t h e two b a s i c t e c h n i q u e s f o r c o m p u t i n g e r r o r bounds h a v e b e e n r e p o r t e d by Rall (1979). A n o t h e r s t r a t e g y f o r o b t a i n i n g s o l u t i o n bounds i s t h e e x p l o i t a t i o n of s t r u c t u r e of p a r t i c u l a r c l a s s e s of f u n c t i o n s . F o r e x a m p l e , i f t h e g i v e n p r o b l e m i s a l i n e a r programming problem, t h e n under u s u a l nondegeneracy assumptions one c a n e s s e n t i a l l y c o n s t r u c t an e x a c t parametric s o l u t i o n f o r f i n i t e and r a t h e r g e n e r a l c l a s s e s o f p a r a m e t r i c p e r t u r b a t i o n s .

Of

56

2 Basic Sensitivity and Stability Results

c o u r s e , t h i s may e n t a i l an i n o r d i n a t e amount of e f f o r t a n d , i f t h i s i s p r o h i b i t i v e , q u e s t i o n s a r i s e a s t o whether s u i t a b l y a c c u r a t e p a r a m e t r i c bounds on a s o l u t i o n might be c a l c u l a t e d with t o l e r a b l e e f f o r t .

I n n o n l i n e a r programming an i m p o r t a n t

c o m p u t a t i o n a l l y e x p l o i t a b l e p r o p e r t y is s e p a r a b i l i t y o f t h e problem f u n c t i o n s , i . e . ,

t h e r e p r e s e n t a t i o n of a f u n c t i o n a s

t h e sum of f u n c t i o n s o f one v a r i a b l e .

S i n c e e a c h one-

d i m e n s i o n a l "component" o f a s e p a r a b l e f u n c t i o n can g e n e r a l l y b e s e p a r a t e l y e s t i m a t e d o r bounded by s i m p l e r f u n c t i o n s , t h i s l e a d s t o t h e p o s s i b i l i t y of e s t i m a t i n g o r bounding a m u l t i d i m e n s i o n a l s e p a r a b l e problem by a s i m p l e r problem.

Assuming

t h a t t h e one-dimensional e s t i m a t i o n e r r o r s o r e r r o r bounds a r e t r a c t a b l e , t h e n t h e s o l u t i o n and e r r o r bound i n f o r m a t i o n a s s o c i a t e d w i t h t h e a p p r o x i m a t i o n problem c a n p o s s i b l y b e r e l a t e d t o t h e s o l u t i o n - and e r r o r - b o u n d i n f o r m a t i o n o f t h e o r i g i n a l problem. The most s t r i k i n g and w e l l known a p p l i c a t i o n o f t h e e x p l o i t a t i o n of s e p a r a b i l i t y i s undoubtedly t h e e x t e n s i v e l y developed and w i d e l y u s e d branch and bound methods f o r nonconvex s e p a r a b l e programming p i o n e e r e d by F a l k and Soland ( 1 9 6 9 ) , based on g l o b a l l y s o l v i n g t h e g i v e n problem by g e n e r -

a t i n g a sequence of s e p a r a b l e convex u n d e r e s t i m a t i n g programs. C l o s e r t o t h e s p i r i t of s o l u t i o n e r r o r bounds, however, G e o f f r i o n (1977) n o t e d t h a t t h e e r r o r bounds i n v o l v e d i n f i t t i n g f u n c t i o n s ( o f one v a r i a b l e ) t o t h e r e s p e c t i v e compon e n t s of a g i v e n s e p a r a b l e problem c o u l d b e v e r y p r e c i s e l y r e l a t e d t o t h e d i s c r e p a n c y between t h e s o l u t i o n v a l u e o f t h e g i v e n problem and t h a t o f t h e a p p r o x i m a t i o n problem.

H e gave

s e v e r a l t h e o r e t i c a l r e s u l t s f o r bounding t h e d i f f e r e n c e o f t h e o p t i m a l v a l u e and c o n s t r u c t i v e c u r v e - f i t t i n g t e c h n i q u e s f o r

2.5 Optimal Value and Solution Bounds

57

reducing t h e r e s u l t i n g deviation.

More r e c e n t l y , Meyer ( 1 9 7 9 ,

1980) d e v e l o p e d t e c h n i q u e s f o r c a l c u l a t i n g error bounds on t h e o p t i m a l v a l u e of l a r g e convex s e p a r a b l e programs and h a s been i n v e s t i g a t i n g t h e e x p l o i t a t i o n of s e p a r a b i l i t y t o c a l c u l a t e

error bounds on s o l u t i o n p o i n t s .

Thakur ( 1 9 7 8 , 1 9 8 0 , 1 9 8 1 )

computes o p t i m a l v a l u e f u n c t i o n and s o l u t i o n p o i n t e r r o r - b o u n d i n f o r m a t i o n f o r convex ( h i g h l y n o n l i n e a r ) s e p a r a b l e p r o g r a m s , b a s e d on a n a n a l y s i s o f a c o r r e s p o n d i n g p i e c e w i s e l i n e a r a p p r o x i m a t i o n problem.

An i n t r i g u i n g u s e of t h i s i n f o r m a t i o n

i s t o a c c e l e r a t e c o n v e r g e n c e by r e d u c i n g t h e r e g i o n known t o c o n t a i n a s o l u t i o n i n a scheme b a s e d on s o l v i n g a s e q u e n c e o f p r o g r e s s i v e l y t i g h t e r l i n e a r a p p r o x i m a t i o n problems.

Again, i n

t h e c o n t e x t of s e p a r a b l e , b u t now s e p a r a b l e p a r a m e t r i c p r o g r a m s , Benson ( 1 9 8 0 ) h a s e x t e n d e d t h e u s u a l b r a n c h a n d bound a p p r o a c h t o a l l o w f o r t h e s i m u l t a n e o u s c a l c u l a t i o n of a s e t of s o l u t i o n s of a g i v e n s e p a r a b l e program c o r r e s p o n d i n g t o a f i n i t e number of r e s o u r c e l e v e l s ( r i g h t - h a n d s i d e s ) . D i n k e l and K o c h e n b e r g e r ( 1 9 7 7 , 1 9 7 8 ) and D i n k e l , K o c h e n b e r g e r , a n d Wong ( 1 9 7 8 , 1 9 8 2 ) h a v e c o n d u c t e d numerous experiments i n c a l c u l a t i n g t h e parameter d e r i v a t i v e s of a Karush-Kuhn-Tucker

triple [x(E)

,

u ( E ) , W ( E ) ] and e x t r a p o l a t i n g

f o r new s o l u t i o n s , i n a v a r i e t y o f a p p l i e d G P p r o b l e m s ( e s s e n t i a l l y u n d e r t h e a s s u m p t i o n s of Theorem 2 . 4 . 4 ) .

A distinctive

f e a t u r e o f t h e i r work i s t h e u s e o f a t e c h n i q u e g e n e r a l l y a t t r i b u t e d t o Davidenko ( 1 9 5 3 ) :

extrapolation over a small

parameter i n t e r v a l u s i n g f i r s t p a r a m e t e r - d e r i v a t i v e information f o l l o w e d by a r e e v a l u a t i o n o f f u n c t i o n s i n v o l v e d a n d a r e c a l c u l a t i o n o f d e r i v a t i v e s a t t h e new p o i n t , t i o n , etc.

f o l l o w e d by e x t r a p o l a -

They h a v e o b t a i n e d much g r e a t e r a c c u r a c y u s i n g t h i s

i n c r e m e n t a l a p p r o a c h t h a n i s o b t a i n e d by u s i n g one l i n e a r

58

2 Basic Sensitivity and Stability Results

extrapolation, the accuracy apparently being proportional to the number of increments

- though

obviously, there is a trade-

off with computational effort that dictates a compromise strategy.

They have also developed heuristics for estimating

the range over which extrapolation is valid. Our computational approach to the calculation of optimal value and Karush-Kuhn-Tucker triple parameter derivatives is discussed in detail in Chapters 3

-

8,

so we confine ourselves

here to a few remarks concerning a central topic of this section, the calculation of optimal solution bounds.

Our point

of view has been to start with the highly structured problem and to focus our attention explicitly on the calculation of parametric bounds on the optimal value function corresponding to finite parameter changes.

We have found the class of prob-

lems with convex (or concave) optimal value function to have many computationally exploitable properties, while being general enough to be of wide practical interest, and have thus been naturally led to the jointly convex parametric NLP P 3 ( c ) as a logical starting point on which to build a computable bounding methodology.

The convexity of the problem in x leads

to immediate connections with duality theory and the determination of dual parametric bounds.

The convexity of the optimal

value function allows for the immediate calculation of a global lower parametric linear bound whenever a solution and a subgradient is calculated at any specified parameter value, and an upper linear bound via linear interpolation between the optimal values whenever solutions for two parameter values have been calculated.

The joint convexity of the feasible

region also easily yields a feasible parametric vector, whenever solutions are calculated for two parameter values, by

59

2.5 Optimal Value and Solution Bounds

l i n e a r i n t e r p o l a t i o n between t h e g i v e n s o l u t i o n p o i n t s . t h e bounds on f

*

(E)

Also,

open t h e d o o r t o computable t e c h n i q u e s for

bounding a Karush-Kuhn-Tucker

triple [x(E)

,

U(E)

,

W(E)

1.

Finally, the p o s s i b i l i t y of estimating corresponding r e s u l t s f o r nonconvex p a r a m e t r i c programs i s r e a l i z e d f o r t h o s e programs t h a t c a n be e s t i m a t e d by j o i n t l y convex programs.

In

p a r t i c u l a r , r e s u l t s c a n be o b t a i n e d f o r j o i n t l y s e p a r a b l e nonconvex programs s i n c e , a s n o t e d e a r l i e r , an i m p o r t a n t approach f o r s o l v i n g s t a n d a r d ( i . e . , n o n p a r a m e t r i c ) nonconvex s e p a r a b l e problems i s b r a n c h and bound methodology t h a t i s based on s o l v i n g a s e q u e n c e of convex u n d e r e s t i m a t i n g problems. I t s u f f i c e s t o o b s e r v e t h a t t h e same t e c h n i q u e s can be used

j o i n t l y i n (x,

E)

t o g e n e r a t e p a r a m e t r i c j o i n t l y convex under-

e s t i m a t i n g problems o f a p a r a m e t r i c j o i n t l y s e p a r a b l e nonconvex program.

Further, s i n c e t h e s a m e i d e a s involving s e p a r a b i l i t y

c a n b e e x t e n d e d t o a s i g n i f i c a n t l y more g e n e r a l s t r u c t u r e c a l l e d " f a c t o r a b i l i t y " (McCormick, 1976), encompassing most f u n c t i o n s commonly e n c o u n t e r e d i n p r a c t i c e , t h e a 2 p l i c a b i l i t y o f r e s u l t s b a s e d on j o i n t l y convex programs t o nonconvex programs i s clear. The d e t a i l s of t h e i n d i c a t e d r e s u l t s and t h e i r a p p l i c a t i o n i n d e v e l o p i n g s o l u t i o n bounds a n d r e f i n e d p a r a m e t r i c s o l u t i o n estimates w i l l n o t b e p u r s u e d f u r t h e r h e r e , b u t a r e p r e s e n t e d i n C h a p t e r 9. W e close by n o t i n g t h a t t h e c l a s s o f problems t h a t have a n

o p t i m a l v a l u e f u n c t i o n w i t h c o n v e x i t y or c o n c a v i t y p r o p e r t i e s is c o n s i d e r a b l y l a r g e r t h a n m i g h t be imagined from a c u r s o r y

analysis.

R e v e r s e convex programs ( e . g . ,

P2(€) w i t h f concave,

t h e gi convex a n d t h e h . a f f i n e i n x ) may be e x p e c t e d t o have a 3

piecewise concave o p t i m a l v a l u e f u n c t i o n .

Also, large classes

60

2 Basic Sensitivity and Stability Results

of parametric posynomial GP problems have a convex optimal value function (all the more intriguing since signomial GP problems may be underestimated or overestimated by posynomial GP problems).

Finally, it is noted that GP duality theory can

be exploited to calculate global parametric bounds on the optimal value function. 2.6.

GENERAL RESULTS FROM RHS RESULTS

The general parametric problem P 3 ( ~ )and the rhs parametric problem P 2 ( ~ )are more closely associated than their formulations may suggest.

In fact, surprisingly, any general problem

of the form minX f(x,

E)

s.t. g(x,

E)

2 0, h(x,

E)

P3 ( € 1

= 0

may be formulated as an e q u i v a l e n t rhs problem minX f(x)

s.t. g(x) 2 E ~ h(x) , =

by simply redefining

E

E

2

P2 ( € 1

in P3(€) to be a variable and intro-

ducing a new parameter a such that E = a.

This results in the

problem min

(XI€)

f(x,

E)

s.t. g(x,

E)

2 0, h(x,

E)

= 0,

€ = a

P (0.1

which is clearly equivalent to P3(€) and of the form P2(~). The relevance and utility of this equivalence is its use in organizing more tightly and relating more precisely the body of results that hold for the various formulations.

It is

obvious that results for P 3 ( ~ )can be applied to P2(~). The reformulation P(a) of P3(€) implies that the converse of this fact is true:

results for P 2 ( ~ )obviously apply to P(a) and

can be applied to P3(~),when the assumptions on P3(") can be

61

2 6 General R e w h troni RHS Studkc\

shown t o imply t h e r e q u i r e d P2(f) a s s u m p t i o n s v i a P(a).

This

h a s many i n t e r e s t i n g a p p l i c a t i o n s i n i m m e d i a t e l y e x t e n d i n g c e r t a i n r e s u l t s f o r P2(€) to t h e i r n a t u r a l g e n e r a l i z a t i o n f o r P3(c).

F o r example, C Q 1 h o l d s f o r P 3 ( E ) a t x = I w i t h

w i t h r e s p e c t t o t h e v e c t o r 7 , i f and o n l y i f V x q ( S r , Vxh(St, F ) y = 0 , and t h e r o w s o f Vxh(F, pendent.

But t h i s means t h a t V

Also, d e f n i n g H E

(,”),

(X,

E

F)y

= E,

> 0,

y ) a r e l i n e a r l y inde-

E )g (-x , T I ( ; ) = Vxg(X, E ) y > 0 .

it follows t h a t

(x, E and t h e rows o f

a r e l i n e a r l y independent. problem P(a) a t ( x , the vector

(y, 0 ) .

E)

=

T h i s means t h a t C Q 1 h o l d s f o r t h e

-

(2, E ) w i t h a

=

T, w i t h r e s p e c t t o

Thus, C Q 1 i n P ( E ) i s i n h e r i t e d by a

r e a l i z a t i o n of P * ( E ) .

A s a n immediate a p p l i c a t i o n ,

t h a t t h e Gauvin-Tolle r e s u l t s , t h e c o n t i n u i t y o f f

*

it f o l l o w s (E)

and t h e

d i f f e r e n t i a l s t a b i l i t y bounds on t h e D i n i d e r i v a t i v e s o f P 2 ( € ) described i n S e c t i o n 2.3,

e a s i l y e x t e n d t o P 3 ( ~ )v i a P(a), i f

t h e n a t u r a l g e n e r a l i z a t i o n s o f t h e Gauvin-Tolle a s s u m p t i o n s a r e assumed t o h o l d f o r P3(€).

This p a r t i c u l a r application

was n o t e d by J a n i n ( p e r s o n a l communication, 1981) and R o c k a f e l l a r ( 1 9 8 2 a ) and i n d e p e n d e n t l y a p p l i e d by G o l l a n (1981a) t o p r o v e some g e n e r a l r e s u l t s . A n o t h e r i m p o r t a n t example i s b a s e d on w e l l known r e s u l t s and i n v o l v e s d e d u c i n g t h e c o n v e x i t y o f t h e o p t i m a l v a l u e funct i o n of t h e g e n e r a l problem P 3 ( € ) from t h e c o n v e x i t y o f t h e

62

2 Basic Sensitivity and Stahility Results

o p t i m a l v a l u e f u n c t i o n f o r P2(c).

A c l a s s i c a l r e s u l t is t h e

c o n v e x i t y o f t h e o p t i m a l v a l u e f u n c t i o n o f P2(€), i f t h i s problem i s convex i n x.

It follows t h a t t h e optimal value of

P(a) i s convex if t h i s p r o b l e m i s j o i n t l y convex i n ( x , E). But c l e a r l y , i f P 3 ( € ) i s convex i n ( x ,

E)

,

t h e n so i s P ( a ) .

Hence, t h e f o l l o w i n g r e s u l t , p r o v e d i n d e p e n d e n t l y o f t h e r e s u l t

f o r P2 ( E ) by Mangasarian a n d Rosen ( 1 9 6 4 ) , i s s e e n t o be i m m e d i a t e l y i m p l i e d by t h e P 2 ( ~ )r e s u l t : P

3

(E)

t h e optimal value of

i s convex i f t h i s problem i s j o i n t l y convex i n ( x ,

E).

W e make i m p o r t a n t u s e o f t h i s r e s u l t i n t h e b o u n d i n g p r o c e d u r e

d e s c r i b e d i n Chapter 9 , as i n d i c a t e d i n t h e l a s t s e c t i o n . The s i t u a t i o n i s n o t a l w a y s s o s t r a i g h t f o r w a r d , b u t c l e a r l y c a r e f u l examination of extensions t o P 3 ( c ) v i a P(a) i s w a r r a n t e d whenever a r e s u l t f o r P2(c) h a s b e e n o b t a i n e d .

Ob-

v i o u s l y , i f a r e s u l t f o r P 3 ( ~ )c a n be p r o v e d d i r e c t l y w i t h l i t t l e more e f f o r t t h e n t h a t r e q u i r e d f o r t h e c o r r e s p o n d i n g

P 2 ( € ) r e s u l t , t h i s would p r o b a b l y be p r e f e r a b l e .

However,

c o n s i d e r a b l e t h e o r e t i c a l s i m p l i f i c a t i o n s and a more c o n s t r u c t i v e p r o o f m i g h t make i t d e s i r a b l e t o d e v e l o p a r e s u l t f o r

.

P2(E) and e x t e n d t o P 3 ( € ) t h r o u g h P ( a )

I t would b e i n t e r -

e s t i n g t o e x p l o r e e x i s t i n g r e s u l t s from t h i s p o i n t of view. 2.7.

SUMMARY

T h i s c h a p t e r p r o v i d e s a c o n c i s e s u r v e y o f a number o f b a s i c s e n s i t i v i t y and s t a b i l i t y r e s u l t s f o r g e n e r a l classes o f n o n l i n e a r p a r a m e t r i c problems.

I t s u r e l y does n o t c o n t a i n a l l

t h e i m p o r t a n t f i n d i n g s , b u t h o p e f u l l y g i v e s a good i d e a o f s e v e r a l mainstream d i r e c t i o n s of r e s e a r c h t h a t have evolved and a r e s t i l l b e i n g d e v e l o p e d .

In the past several years, a s

63

2.7 Summary

the references will attest, developments have accelerated and the level of sophistication of results has been appreciably elevated. We shall endeavor to anticipate some of the future research directions in the final chapter.

Otherwise, except for the

hounds results in Chapter 9, the rest of this book is devoted to a rather detailed treatment of a fundamental segment of the methodology that has been developed rather thoroughly in the past few years.

This body of results is perhaps the most

highly structured for the general problem of interest.

It is

based on one of the strongest collections of assumptions that might be invoked, while still remaining in the province of assumptions that are typically utilized to obtain the strongest consequences in other important mathematical programming applications, e.g.,

strong characterizations of optimality,

and convergence and rate of convergence of algorithms. The assumptions needed for these ideal results are the assumptions of Theorem 2.4.4:

second-order sufficiency, linear

independence, and strict complementary slackness conditions. These provide an abundance of theoretical sensitivity and stability results and sufficient structure to readily allow the development of simple formulas and solution-algorithmbased computational approximation techniques.

It is perhaps

to be expected that this collection of results would be one of the first to be most fully developed and applied.

The results

are of intrinsic interest and are useful, since practical problems meet the requisite conditions surprisingly often. They also provide an essential interface with more general

64

1 Basic Sensitivity and Stability Results

r e s u l t s and a v i t a l p e r s p e c t i v e and p o i n t of d e p a r t u r e , e s t a b l i s h i n g e s s e n t i a l l y t h e most t h a t c a n be e x p e c t e d under r a t h e r i d e a l circumstances.

Theory and Calculation of Solution Parameter Derivatives

This Page Intentionally Left Blank

Chcipter--3

Sensitivity Analysis under Second-Order Assumptions

3.1.

INTRODUCTION In this chapter we give the theoretical validation of the

result stated as Theorem 2 . 4 . 4 in Chapter 2, a basic characterization of the sensitivity of a local solution of a general mathematical programming problem to a small parametric variation of the problem functions.

This material is based largely

on Fiacco (1976) and Armacost and Fiacco (1975).

Although

existence results for higher-order derivatives are immediate, the analysis concentrates on the explicitly tractable firstorder (i.e., first partial derivative) sensitivity information. As noted in Chapter 2, the theorem is an extension of a result presented by Fiacco and McCormick (1968, Theorem 6). The distinguishing feature of the Fiacco-McCormick result was a demonstration of the relationship of the second-order sufficient optimality conditions (for a strict local solution) to the existence and behavior of first-order variations of a local solution and the associated Lagrange multipliers, when the problem functions are subject to parametric variations, and an explicit representation of the first partial derivatives of the local solution point and the associated Lagrange

67

68

3 Sensitivity Analysis under Second-Order Assumptions

multipliers with respect to the problem parameters. 3

-

Chapters

5 describe the extension of this theory to incorporate a

larger class of parametric problems, and Chapter 6 provides a synthesis of this sensitivity theory with classical penalty function theory. The basic sensitivity result is presented in Section 3.2 and applies to a large class of twice-differentiable parametric programs.

The partial parameter derivatives of a

Karush-Kuhn-Tucker triple are shown to exist and are explicitly obtained, under well known assumptions.

Existence of higher-

order partial parameter derivatives follows readily. small examples illustrate the results in Section 3.3.

Two The

existence and representation of the first and second partial derivatives of the optimal value function follow easily, and are given in Section 3.4. 3.2.

FIRST-ORDER SENSITIVITY ANALYSIS OF A SECOND-ORDER LOCAL SOLUTION Consider the problem of determining a local solution

X(E)

of

,..., P), k is a parameter vector in E . h.(x, 3

where x

E

En and

E

E)

= 0 (j = 1

P3(E)

The Lagrangian of P 3 ( ~ )is defined as

(3.2.1)

3.2 Analysis of a Second-Order Local Solution

T h r o u g h o u t t h e book, i f t h e r e are no i n e q u a l i t y cons t r a i n t s , s i m p l y s u p p r e s s r e f e r e n c e t o t h e f u n c t i o n s gi and t h e a s s o c i a t e d m u l t i p l i e r s ui,

and do l i k e w i s e f o r t h e h

1

and

i f t h e r e are no e q u a l i t i e s . Recall t h a t t h e gradient vector j a n d H e s s i a n m a t r i x o p e r a t o r s , V and V2, a r e t a k e n t h r o u g h o u t

w

with r e s p e c t t o x i f they a r e not subscripted. W e are i n t e r e s t e d i n analyzing t h e behavior of a l o c a l

s o l u t i o n x ( T ) o f P ( T ) when T i s s u b j e c t t o p e r t u r b a t i o n . 3

For

s i m p l i c i t y i n n o t a t i o n w e s h a l l assume t h a t F = 0 ( w i t h o u t l o s s C o n d i t i o n s w i l l be g i v e n t h a t g u a r a n t e e t h e

of g e n e r a l i t y ) .

existence of a l o c a l s o l u t i o n of x ( 0 ) f o r

E

X(E)

of P 3 ( € ) i n a neighborhood

i n a neighborhood of 0 , a l o n g w i t h t h e a s s o c i -

a t e d o p t i m a l Lagrange m u l t i p l i e r s

U(E)

and w ( f ) .

Under t h e s e

c o n d i t i o n s t h e components o f a l l t h e s e q u a n t i t i e s a r e shown t o b e u n i q u e l y d e f i n e d d i f f e r e n t i a b l e f u n c t i o n s of neighborhood of

E

f

in a

= 0.

W e s h a l l make c r i t i c a l u s e o f t h e s e c o n d - o r d e r

sufficient

c o n d i t i o n s f o r a s t r i c t l o c a l s o l u t i o n of a mathematical programming p r o b l e m .

T h e s e c o n d i t i o n s a r e now w e l l known,

a l t h o u g h t h e y have been s e r i o u s l y e x p l o i t e d o n l y i n t h e p a s t several years.

They were d e v e l o p e d a n d v e r i f i e d by F i a c c o and

McCormick ( 1 9 6 8 ) a n d o t h e r s .

For t h e problem P j ( 0 ) t h e

c o n d i t i o n s may b e s t a t e d a s f o l l o w s . Lemma 3 . 2 . I .

[Second-order s u f f i c i e n t c o n d i t i o n s f o r a

s t r i c t l o c a l m i n i m i z i n g p o i n t of p r o b l e m P (O).] 3

I f the

f u n c t i o n s d e f i n i n g problem P 3 ( 0 ) are t w i c e c o n t i n u o u s l y d i f f e r e n t i a b l e i n a neighborhood o f x

*

,

minimizing p o i n t o f problem P ( 0 ) hood o f x

*

3

then x

*

is a strict local

( i . e . , t h e r e i s a neighbor-

s u c h t h a t t h e r e d o e s n o t e x i s t any f e a s i b l e x # x

*

I0

3 Sensitivity Analysis under Second-Order Assilrnptions

*

where f ( x , 0 ) 5 f ( x v e c t o r s u*

,

Em a n d w*

E

0 ) ) i f t h e r e e x i s t (Lagrange m u l t i p l i e r ) E

Ep s u c h t h a t t h e f i r s t - o r d e r

Karush-

Kuhn-Tucker c o n d i t i o n s ( K a r u s h , 1939; Kuhn a n d T u c k e r , 1 9 5 1 ) hold, i.e., gi(X hj(x

*

ui g i ( x

* I

* *

0) 2 0,

, 0)

,

= 0,

0) = 0 ,

*

ui 2 0 ,

*

*

..., m , j = 1, ..., P , i 1, ..., m, i 1, ..., m, i = 1,

=

=

m

*

i=l

+

P

1

j =1

w*Vh.(x*, 0) = 0 3 3

and, f u r t h e r , i f

T 2 * * * z V L(x , u , w , C)z > 0 Vgi(x Vgi(x

*

*

f o r a l l z # 0 such t h a t

,

0)z 1. 0

f o r a l l i , where g i ( x

,

0)z

f o r a l l i , where ui

,

0)z = 0 ,

=

0

*

*

,

0) = 0.

> 0,

and

*

Vh.(x 3

j = 1,

..., p.

Han a n d Mangasarian ( 1 9 7 9 ) n o t e d t h a t t h e r e s t r i c t i o n s o n z are e q u i v a l e n t t o z

+

0, v f ( x * , 0)z = 0, v g i ( x * ,

' *

0)z 1.

o

if

g i ( x * , 0 ) = 0 , and Vh. ( x , 0) z = 0 f o r e v e r y j . 3 These c o n d i t i o n s a r e a p p l i c a b l e w h e t h e r o r n o t c o n s t r a i n t s a r e p r e s e n t , and w h e t h e r o r n o t t h e r e e x i s t s a v e c t o r z a s indicated.

I f t h e r e a r e no c o n s t r a i n t s , t h e a b o v e c o n d i t i o n s

7I

3 . 2 A n a l y w of a Second-Order Local Solution

a r e l o g i c a l l y v a l i d i f r e f e r e n c e t o t h e c o n s t r a i n t s i s supT h i s l e a d s t o t h e w e l l known s u f f i c i e n t c o n d i t i o n s

pressed. that x

*

be a n i s o l a t e d

(i.e.,

l o c a l l y u n i q u e ) l o c a l uncon-

*

s t r a i n e d minimizing p o i n t o f f ( x , 0 ) : Vf(x T 2

z V f

* (x ,

# 0.

0)z > 0 for all z

,

0 ) = 0 and

I f t h e r e a r e no z # 0 s a t i s -

fying the indicated relationships with the constraint gradients a n d t h e f i r s t - o r d e r Karush-Kuhn-Tucker

*

(x

,

u

*

,

*

conditions hold, then

w ) a g a i n l o g i c a l l y s a t i s f i e s t h e s e second-order

conditions.

A s a p o i n t of i n t e r e s t i t i s o b s e r v e d t h a t i n

s u c h a case t h e r e m u s t b e n l i n e a r l y i n d e p e n d e n t b i n d i n g constraint gradients a t x

*

.

For example, i f P 3 ( 0 ) is a l i n e a r

programming p r o b l e m a n d t h e s e c o n d - o r d e r

*

(x

,

u

*

,

*

w ),

2

then, since V L

3

conditions hold a t

0 , t h e r e c a n be n o z # 0

satisfying the given inequalities, i.e.,

x

*

must be a v e r t e x

o f t h e f e a s i b l e r e g i o n d e f i n e d by t h e c o n s t r a i n t s o f P 3 ( 0 ) . I t i s c a u t i o n e d t h a t t h e c o n c l u s i o n o f Lemma 3 . 2 . 1

be s t r e n g t h e n e d t o assert t h a t x

*

can not

i s a l o c a l l y unique and hence

i s o Z a t e d l o c a l minimum o f p r o b l e m P 3 ( 0 ) , a s e r r o n e o u s l y a s s e r t e d i n F i a c c o a n d McCormick ( 1 9 6 8 , Theorem 4 )

is proved t h a t x

*

i s a s t r i c t l o c a l minimum).

counterexample i n E l ,

P:

( w h e r e it

A simple

min x L / 2 s . t . h ( x ) = 0 , w h e r e h ( x ) =

x 6 s i n ( l / x ) i f x # 0 a n d h ( x ) = 0 i f x = 0 w a s p r o v i d e d by Robinson 3.2.1

(1980b).

I n t h i s e x a m p l e , t h e a s s u m p t i o n s o f Lemma

hold a t x = 0 and 0 is a s t r i c t local ( i n f a c t , g l o b a l )

minimum o f p r o b l e m P.

However, i t f o l l o w s t h a t xk = l / k r i s a

l o c a l s o l u t i o n ( i n f a c t , a n i s o l a t e d f e a s i b l e p o i n t and h e n c e a n i s o l a t e d l o c a l s o l u t i o n ! ) of p r o b l e m P f o r a n y i n t e g e r k # 0 , a n d e v e r y n e i g h b o r h o o d of x = 0 c o n t a i n s xk f o r Ikl s u f f i ciently large.

Hence, x

=

9 i s c e r t a i n l y n o t a n i s o l a t e d local

72

3 \cn\itir

(or global) minimum.

ity Andly\i\ under Second-Order A\surnption\

Robinson then shows that x

*

is an iso-

lated local minimum if the second-order sufficient conditions of Lemma 3 . 2 . 1 are strengthened by assuming these hold for all optimal Lagrange multipliers (u, w) associated with x^ and if it is also assumed that CQ1 holds at x

*

.

Note that this

extended form of the second-order sufficient conditions in Lemma 3 . 2 . 1 is n o t enough per se to imply that the local

*

minimum x

is isolated, since this stronger condition holds in

Robinson's example.

Hence, an additional regularity assumption

such as CQ1 is needed. If we add the assumptions of linear independence (CQ3) and strict complimentary slackness to the second-order strict

*

sufficiency conditions at (x

,

01,

and the appropriate differ-

entiability assumptions, we obtain the following result. [Basic Sensitivity Theorem (Fiacco, 1976).

Theorem 3 . 2 . 2 .

First-order sensitivity results for a second-order local

*

minimizing point x . I

If

(i) the functions defining P 3 ( E ) are twice continuously differentiable in x and if their gradients with respect to x and the constraints are once continuously differentiable in

*

in a neighborhood of (x

,

E

0),

(ii) the second-order sufficient conditions for a local minimum of P3(0) hold at x*, with associated Lagrange multipliers u

*

and w

*

,

(iii) the gradients Vgi(x

*

= 0) and Vh. ( x

3

,

*

,

0) (for i such that gi(x

*

,

0) (all j) are linearly independent (CQ3),

0)

73

3.2 Analysis of a Second-Order Local Solution

and

*

(iv) ui > 0 when gi(x

*

,

0) = 0 (i = 1,

...,m)

(i.e.,

strict complementary slackness holds), then (a) x

*

is a local i s o l a t e d minimizing point of

problem P3(0) and the associated Lagrange multipliers u (b) for

E

*

and w

*

are unique,

in a neighborhood of 0, there exists a

unique, once continuously differentiable vector function Y(E) = [x(E), ~ ( € 1 ~,( € 1 T 1 satisfying the second-order sufficient conditions for a local minimum of problem P(E) such * * * T * that y(0) = (x , u , w ) = y , and hence X(E) is a l o c a l l y u n i q u e local minimum of problem P3(c) with associated unique Lagrange multipliers U(E) and ~ ( € 1 , and (c) for

E

near 0, the set of binding inequalities

is unchanged, strict complementary slackness

holds, and the binding constraint gradients are linearly independent at Part (a) follows if (b) is true.

Proof.

~(€1. It is stated

separately because it is of intrinsic interest.

That x

*

is a

from assumption (ii), strict local minimum of ~ ~ ( follows 0 ) which also implies that VL(x of u* and w

*

*

,

u

*

* ,w ,

0) = 0.

The uniqueness

follows from this and assumption (iii).

The proof of (b) follows from a straightforward application of the implicit function theorem to the first-order necessary optimality conditions of P3(€), as follows.

A s s u m p t i o n ( i i ) i m p l i e s t h e s a t i s f a c t i o n of t h e K a r u s h Kuhn-Tucker

first-order

VL(x, u , w ,

uigi(x,

a t ( x , u, w,

E)

0

E)

=

E)

= 0,

(x

=

conditions

*

,

i = 1, . . . , m ,

u

*

* ,w ,

(3.2.2)

Assumption ( i )i m p l i e s

0).

i s once continuously

t h a t t h e system of equations (3.2.2)

d i f f e r e n t i a b l e i n a l l t h e arguments; so, i n p a r t i c u l a r , Jacobian m a t r i x of defined.

with respect t o (x, u, w) i s w e l l

(3.2.2)

( i i ) ,( i i i ) ,a n d ( i v )

I t follows t h a t assumptions

imply t h e e x i s t e n c e of t h e i n v e r s e o f t h i s m a t r i x a t ( x

w

*

,

0)

the

*

,

u

*

,

( a n i m m e d i a t e e x t e n s i o n o f Theorem 1 4 o f F i a c c o a n d

McCormick ( 1 9 6 8 ) , a n d now a w e l l known r e s u l t . ) The a s s u m p t i o n s o f t h e i m p l i c i t f u n c t i o n t h e o r e m (Theorem w i t h r e s p e c t to t h e equations (3.2.2)

2.4.1)

*

solution (x

,

u

*

*

,w ,

and t h e p a r t i c u l a r

0 ) are s a t i s f i e d and w e can c o n c l u d e

*

t h a t i n a neighborhood of

(x

,

u

*

,

w

*

),

for

E

i n a neighborhood

of 0 , t h e r e e x i s t s a unique once continuously d i f f e r e n t i a b l e f u n c t i o n [ x ( E ), u ( E ) ,

*

u ( O ) , w ( O ) ] = (x that for

E

,

near 0,

u

*

W(E)

,

X(E)

w

*

1 s a t i s f y i n g ( 3 . 2 . 2 ) w i t h [ x ( O ),

).

The s a t i s f a c t i o n o f

(3.2.2)

means

i s a f i r s t - o r d e r Karush-Kuhn-Tucker

p v i n t of p r o b l e m P 3 ( € ) , w i t h a s s o c i a t e d L a g r a n g e m u l t i p l i e r s U(E)

and

w(E).

To p r o v e

( c ), w e f i r s t n o t e t h a t t h e b i n d i n g c o n s t r a i n t

s e t a t x ( 0 ) r e m a i n s t h e same f o r

E

n e a r 0.

This i s seen

immediately f o r t h e e q u a l i t i e s h . [ x ( E ) , € 1 = 0, s i n c e X ( E ) 3 s a t i s f i e s ( 3 . 2 . 2 ) n e a r E = 0. For t h e i n e q u a l i t i e s , w e h a v e from ( 3 . 2 . 2 ) E

= 0.

t h a t u ~ ( E ) ~ = [ x ( E ) E, ] = 0 ( i = 1,

If gi[x(0),

..., m )

01 = 0 f o r some i , t h e n u i ( 0 )

> 0

near

75

3 . 2 Analysis of' a Second-Order Local Solution

( b y s t r i c t complementary s l a c k n e s s ) , h e n c e ui E

= 0 by c o n t i n u i t - y o f

> 0 near

and w e c o n c l u d e t h a t g . [ x ( f ) ,

U(E)

I f g i [ x ( 0 ) , 01 > 0 f o r s o m e i , t h e n g i [ x ( E ) ,

= 0. E

(E)

= 0 by c o n t i n u i t y .

E]

> 0 near

E]

Therefore, defining

B ( E ) :{ i l g . [ x ( E ) ,

E]

=

01,

w e have concluded t h a t B ( E ) = B ( 0 ) f o r

E

n e a r 0.

(The a r g u -

ment a l s o shows t h a t s t r i c t complementary s l a c k n e s s i s p r e served f o r

E

n e a r 0 , p r o v i n g t h e f i r s t p a r t of

W e now show t h a t t h e s e c o n d - o r d e r

conditions E

near 0.

any

E

I E ~<

h o l d a t [ x ( E ) , w ( E ) , w ( E ) ] f o r any

6 , it follows t h a t

> 0 f o r any v e c t o r

= 0 for a l l i

Z(E)

Then t h e r e m u s t e x i s t Vgi [ x ( E ~ ) E,

~

f o r a l l k.

E~

V L[x(E), u(E), [x(E),

E]Z(E) = 0 for a l l j. 3 Suppose t h e a s s e r t i o n i s f a l s e .

E

2

... .

T 2

# 0 s u c h t h a t Vg.

> 0 and zk

zk] = 0 f o r a l l i

f o r a l l j , and

Z(E)

B ( 0 ) and Vh. [ x ( E ) ,

E

T h i s may b e p r o v e d a s f o l l o w s .

k = 1, 2 ,

sufficient optimality

W e must show t h a t t h e r e e x i s t s 6 > 0 s u c h t h a t f o r

such t h a t

w(E)]z(E) E]Z(E)

(Lemma 3 . 2 . 1 )

(c).)

# 0 such t h a t

B(0)

,

E~

Oh. [ x ( E ~ ) , 3

E ~ z kI

V L [ x ( E ~ ) ,u ( E ~ ) , w ( E ~ ) , ~

~

W i t h o u t l o s s o f g e n e r a l i t y , assume

S e l e c t a c o n v e r g e n t s u b s e q u e n c e of { z

t h e subsequence { z

k

Taking l i m i t s a s k

1

0,

+

k

= 0

5 01 f o r2 k

112

11

= 1

1, relabel

f o r c o n v e n i e n c e and c a l l t h e l i m i t Z.

and r e c a l l i n g a s s u m p t i o n ( i ) y i e l d s 2 * * * t h e c o n c l u s i o n t h a t ZT V L ( x , u , w , 0 ) Z L 0 f o r some F s u c h + +m

t h a t 11Tl1 = 1, Vgi(x*,

0)Z = 0 for a l l i

E

*

B ( 0 ) and V h . ( x 3

,

But t h i s i s a c o n t r a d i c t i o n of a s s u m p t i o n

0)Z = 0 f o r a l l j .

( i i ) and t h e p r o o f o f t h e a s s e r t i o n i s c o m p l e t e . Since it w a s established t h a t s o l v e s (3.2.2)

for

E

[x(E), u(E), w(E)]

near 0 , it follows t h a t

X(E)

uniquely

is a locally

~

76

3 Sensitivity Analysis under Second-Order Assumptions

u n i q u e l o c a l minimum of P ( E ) w i t h a s s o c i a t e d u n i q u e L a g r a n g e multipliers

U(E)

and

W(E)

,

completing t h e proof o f p a r t ( b ) .

The p r e s e r v a t i o n of s t r i c t complementary s l a c k n e s s w a s p r o v e d above.

+

the (say) r

The p r e s e r v a t i o n o f t h e l i n e a r i n d e p e n d e n c e o f p binding c o n s t r a i n t g r a d i e n t s a t

X(E)

n e a r 0 f o l l o w s d i r e c t l y from t h e f a c t t h a t a n ( r

+

for E

p ) by

( r + p ) J a c o b i a n o f t h e s y s t e m o f e q u a t i o n s d e f i n e d by t h e c o n s t r a i n t s t h a t a r e b i n d i n g a t x ( 0 ) must be n o n s i n g u l a r , a l o n g w i t h t h e assumed c o n t i n u i t y o f t h e f i r s t d e r i v a t i v e s . The p r o o f i s c o m p l e t e . Robinson ( 1 9 7 4 ) p r o v e d a n a n a l o g o u s t h e o r e m (Theorem 2.4.7, C h a p t e r 2 ) u n d e r weaker a s s u m p t i o n s ( s p e c i f i c a l l y , a s s u m p t i o n ( i ) i s r e p l a c e d by t h e a s s u m p t i o n t h a t t h e s e c o n d p a r t i a l

d e r i v a t i v e s o f t h e problem f u n c t i o n s w i t h r e s p e c t t o x are j o i n t l y c o n t i n u o u s i n (x, c o n t i n u i t y of

[x(E)

,

U(E)

,

W(E)

bounds o n t h e v a r i a t i o n of changes i n

E

and d e m o n s t r a t e d t h e r e s u l t i n g

E ) ) ,

] near

[x(E)

,

E

U(E)

H e also o b t a i n s

= 0.

, w(E)]for

small

a n d u s e s t h e s e t o d e t e r m i n e con-

(Theorem 2.4.8)

vergence r a t e s f o r a l a r g e class of algorithms. Theorem 3.2.2

reduces to t h e following statement i f t h e r e

a r e nu c o n s t r a i n t s p r e s e n t or b i n d i n g i n P 3 ( 0 ) . C o r o l l a r y 3.2.3.

[ S e n s i t i v i t y a n a l y s i s f o r t h e uncon-

s t r a i n e d problem: minx f (x,

E)

.I

I f f (x,

o u s l y d i f f e r e n t i a b l e i n x and V x f ( x , differentiable i n

E,

E)

is t w i c e c o n t i n u -

E)

i s once c o n t i n u o u s l y

i n a neighborhood of

*

(x

,

01,

and if t h e

s e c o n d - T r d e r s u f f i c i e n t c o n d i t i o n s f o r a n u n c o n s t r a i n e d local minimum of f ( x , 0 ) h o l d a t x

*

then, f o r

E

near 0, there e x i s t s

a unique once c o n t i n u o u s l y d i f f e r e n t i a b l e v e c t o r f u n c t i o n s a t i s f y i n g t h e second-order s u f f i c i e n t c o n d i t i o n s for an

X(E)

77

3.2 Analysih of a Second-Order Local Solution

u n c o n s t r a i n e d l o c a l minimum o f f ( x , hence,

E)

such t h a t x ( 0 ) = x

*

and

i s a l o c a l l y u n i q u e u n c o n s t r a i n e d l o c a l minimum o f

X(E)

f(xr €1 * With (x, u , w ) = [ x ( E ) , u ( E ) , w ( E ) ] , ( 3 . 2 . 2 ) i s i d e n t i c a l l y satisfied for

to

E

n e a r 0 and c a n b e d i f f e r e n t i a t e d w i t h r e s p e c t

t o y i e l d e x p l i c i t expressions f o r the f i r s t p a r t i a l

E

d e r i v a t i v e s of t h i s v e c t o r f u n c t i o n . It follows t h a t

M ( E ) V ~ Y ( E ) = N(E),

(3.2.3)

where -

v2L

I

T

-Vg1,

... -vgmT

91

UlV91

Ohl, T

... VhTP

0

0

M E

0

UmVgm

gm

I

r

0

m

t h e J a c o b i a n m a t r i x o f ( 3 . 2 . 2 ) w i t h r e s p e c t t o y = (x, u , w ) ’ , e v a l u a t e d a t [ Y ( E ) , E ] and N

f

[-v2

L ~ -ulvEgl, , T

EX

...

r

-umVEgm, T -VEhl, T

..., -V

hTIT E P

t h e n e g a t i v e of t h e J a c o b i a n m a t r i x o f ( 3 . 2 . 2 ) w i t h r e s p e c t t o E,

evaluated a t

[Y(E),

€1.

Since M i s nonsingular f o r

E

near

0, it follows t h a t

vEy(E) = M(E)-~N(E)

(3.2.4)

78

3 Sensitivity Analysis under Second-Order Assumptions

where the quantities are evaluated as in (3.2.3) and neighborhood of 0. v,y(o)

In particular, we have that (3.2.5)

(M*)-~N*,

=

where y(0) = [ x ( O ) , u(O), w 0)IT and N

*

is in a

E

= N(0).

S

*

(x

,

u

*

,w

* T )

,

M

*

= M(0)

Also, since the quantities involved in (3.2.4)

are all continuous in their arguments (from the assumptions of the theorem and the consequences of the implicit function theorem), (3.2.5)

is also the limit of

3.2.4)

as

0, i.e.

E +

(3.2.6

lim VEy(~)= VEy(0). E+O

It is important to note that VEy(~), a first-order estimate of the variation of an isolated local solution

X(E)

of problem P 3 ( € ) and the associated unique Lagrange multipliers U(E) and w(E), can b e c a Z c u Z a t e d from (3.2.4) once [ x ( E ) , u(E), w(E)] has been determined.

In particular, VEy(0), and hence a

first-order Taylor's series approximation of in a neighborhood of u

*

,w

*

)

is known.

E =

[x(E),

u(E), W(E)

*

0 is available from (3.2.5) once (x

,

This is summarized in the following

statement. CoroZZary 3 . 2 . 4 .

w(E)] near

= 0.1

E

[First-order estimation of [x(E), u(E), Under the assumptions of Theorem 3.2.2,

a

first-order approximation of [ x ( E ) , u(E), w(E)] in a neighborhood of

E

= 0 is given by

(3.2.7)

*

*

, w* )

*

where (x

,

and

and N ( E ) are defined as in (3.2.31,

M(E)

means that

u

= [x(O), u(O), ~ ( 0 1 1 , M

@ ( ~ ) / l l ~ l0l as +

E

+

0.

= M(O),

and

N

@(El

*

= N(O), =

O(llEll)

3 . 2 t\naly\i\ 01

:i

79

Second-Ordcr I.ocal Soltilitin

i s a n immediate

The f o l l o w i n g c o r o l l a r y o f Theorem 3 . 2 . 2 i m p l i c a t i o n o f t h e i m p l i c i t f u n c t i o n theorem.

[Exist2nce of higher-order derivatives.

Corollary 3.2.5.

I f t h e c o n d i t i o n s o f Theorem 3 . 2 . 2

hold, with t h e respective

assumed o r d e r s o f d i f f e r e n t i a b i l i t y b e i n g p a s s u m e d , w i t h p 1. 1, t h e n i n a n e i g h b o r h o o d of

E

:[x(E),

Y(E)

= 0.

I

-

1 m o r e than t h a t

u(E),

w ( E ) I ~E

Cp

I f t h e problem f u n c t i o n s a r e

a n a l y t i c i n (x, c ) i n a n e i g h b o r h o o d o f a n a l y t i c i n a n e i g h b o r h o o d of

E

*

(x

,

then

O),

Y(E)

is

= 0.

The e x i s t e n c e a n d c o n t i n u i t y c f t h e h i g h e r - o r d e r

Proof.

p a r t i a l d e r i v a t i v e s and a n a l y t i c i t y r e s u l t from t h e i m p l i c i t f u n c t i o n theorems,

Theorem 2 . 4 . 1

system of e q u a t i o n s (3.2.2) If f(x,

applied to the

a s i n t h e p r o o f o f Theorem 3 . 2 . 2 .

i s n o t c o n s t r a i n e d , and t h e c o n d i t i o n s o f

E)

C o r o l l a r y 3.2.3 E

and 2.4.2,

2 * hold, then V f ( x

,

0)

-1

e x i s t s and hence n e a r

= 0 t h e r e e x i s t s a once continuously d i f f e r e n t i a b l e f u n c t i o n

t h a t l o c a l l y u n i q u e l y s a . t i s f i e s Vxf [ x ( E ) ,

X(E)

f e r e n t i a t i o n by VXf 2

VEX

+

E

v:xf

and t h e r e f o r e n e a r

v

x

=

E]

= 0.

Dif-

yields = 0, E

= o

-v X2 f - l v 2E X f

where t h e q u a n t i t i e s are e v a l u a t e d a t x ( E ) .

The a p z l o g y of

80

3 Sensitivity Analysis under Second-Order Assumptions

EXAMPLES

3.3.

The a p p l i c a t i o n o f t h e p r e v i o u s t h e o r e t i c a l r e s u l t s f o r e s t i m a t i n g s e n s i t i v i t y i n f o r m a t i o n may be i l l u s t r a t e d by t h e f o l l o w i n g s i m p l e examples. E x a m p Z e 3 . 3.1 min X x

s.t. x

(x e E ~ ) .

E

The s o l u t i o n o f t h e problem i s g i v e n u n i q u e l y by X C E ) f o r any

E.

The h y p o t h e s e s o f Theorem 3.2.2

v e r i f i e d f o r t h i s problem.

=

E,

may b e r e a d i l y

(Note t h a t t h e second-order

suf-

f i c i e n t optimality conditions a r e s a t i s f i e d i n a l o g i c a l sense, s i n c e t h e r e a r e no nonzero v e c t o r s o r t h o g o n a l t o t h e b i n d i n g constraint gradients.) L(x, u,

x

E)

-

u(x

-

E)

Tucker c o n d i t i o n s y i e l d dx ( € 1 dU'E) 7 = 1 , - dE

i s g i v e n by

The Lagrangian (3.2.1)

and t h e f i r s t - o r d e r Karush-Kuhn-

U(E)

= 0,

= 1.

We t h u s f i n d

for a l l

E.

E x a m p l e 3. 3.2 min where

E~

X1

+

E1X2

> 0 and

S.t.

# 0 (x

E~

>

X2

E

2 2

E 2 X 1, - X I

2

2 0

The problem i s d e p i c t e d i n

E ).

F i g . 3.3.1. The f i r s t o r d e r Karush-Kuhn-Tucker

c o n d i t i o n s imply t h a t

t h e s o l u t i o n and t h e a s s o c i a t e d Lagrange m u l t i p l i e r s a r e u n i q u e l y g i v e n by 2

) 1 x ~ ( E ) = - 1 / 2 ~ ~ x~ 2~( ~ , =

/

2 2 4

E

=

and u ~ ( E= ) E ~ ,

~ ~ ( =€ 01, where

~

~

~

~

81

3.3 Examples

Fig.

3.3.1.

D e p i c t i o n of ExampZe,-%. 3 . 2 .

It is easily verified that the second-order conditions hold for [x(E)

,

U(E)

]

as given, and hence it is observed that all

the hypotheses of Theorem 3.2.2 E

are satisfied for any value of

as indicated. It follows that 1

2 2 / 2

3 ~~ ~ / E ~ ~ ~E ~~

and

These matrices,.whose coefficients give the desired firstorder sensitivity information, can also be calculated directly from (3.2.4).

82

3 Sensitivity Analysis under Second-Order Assumption$

The n e x t s e c t i o n d e v e l o p s f o r m u l a s f o r t h e d e r i v a t i v e s o f t h e "optimal value function," f

*

(E)

P 3 ( ~ a) n d P 2 ( € ) a l o n g t h e Karush-Kuhn-Tucker tX(E) r

3.4.

~ ( € r1

W(E)

I

€ 1 , of problems

= f[x(E),

-

point trajectory

FIRST- AND SECOND-ORDER PARAMETER DERIVATIVES O F THE OPTIMAL VALUE FUNCTION Because of i m p o r t a n t c o n n e c t i o n s w i t h Lagrange m u l t i p l i e r s

and d u a l i t y t h e o r y , f i r s t - o r d e r

changes i n t h e c p t i m a l v a l u e

f u n c t i o n have t r a d i t i o n a l l y been a n a l y z e d w i t h r e s p e c t t o v a r i a t i o n s i n t h e " r i g h t hand s i d e " of t h e c o n s t r a i n t s . (1972) a l s o d e r i v e s second-order

Buys

changes i n c o n n e c t i o n w i t h a n

a n a l y s i s of t h e b e h a v i o r o f t h e o p t i m a l v a l u e o f a n a s s o c i a t e d A more c o m p l e t e t r e a t m e n t was

augmented L a g r a n g i a n f u n c t i o n .

g i v e n s u b s e q u e n t l y by A r m a c o s t and F i a c c o ( 1 9 7 6 ) . a s s u m p t i o n s of Theorem 3.2.2, extended these f i r s t - o r d e r

Under t h e

Armacost a n d F i a c c o ( 1 9 7 5 )

r e s u l t s a n d a l s o d e v e l o p e d second-

o r d e r r e s u l t s f o r a g e n e r a l class of p a r a m e t r i c v a r i a t i o n s . The o t h e r r e f e r e n c e d r e s u l t s a r e o b t a i n e d as s p e c i a l cases. Theorem 3.4.1

a n d C o r o l l a r y 3.4.4

Theorems 2 . 4 . 1 2 Let Y(E)

a n d 2.4.13.

= [x(E)

t r i p l e , where

were c i t e d i n C h a p t e r 2 as

X(E)

,

U(E)

,

W(E)

I T b e a Karush-Kuhn-Tucker

s o l v e s p r o b l e m P 3 ( ~ )f o r

E

n e a r 0.

Then

t h e ( l o c a l ) "optimal value function" i s defined a s f*(E)

f[X(E)r

€1,

(3.4.1)

and t h e " o p t i m a l v a l u e L a g r a n g i a n " i s d e f i n e d a s L*(€) = L [ x ( E ) , u ( E ) ,

w(E)

, €1.

(3.4.2)

83

3.4 Derivatives of the Optimal Value Function

Theorem 3 . 4 . I .

[ F i r s t - and s e c o n d - o r d e r

I f t h e c o n d i t i o n s o f Theorem

optimal value function of P ( E ) ] . 3 3.2.2

changes i n t h e

h o l d f o r p r o b l e m P 3 ( € ) , and i f t h e problem f u n c t i o n s a r e

t w i c e continuously d i f f e r e n t i a b l e i n (x, i n a neighborhood of (a)

(b)

f

*

(E)

= L

*

E)

*

near ( x

,

0) then,

= 0,

E

(3.4.3)

(E);

m

2

-

V E f * ( E ) = VEL = VEf

ui(c)VEgi

i=l P

(3.4.4)

= VEf

-

T

U(E)

VEg

+

W(E)

T

VEh;

and h e n c e a l s o 2 *

= V2 XELVEX(E)

-

m

1

(3.4.5)

VECJiTVEUi(E)

i=l

+

VEh;VEwj(E)

+

VEL. 2

j=1

Proof.

R e c a l l t h a t i n a neighborhood o f

u ~ ( E ) ~ ~ [ x ( EE ]) , E

0 , i = 1,

slackness holds, hj[x(E), [x(E),

f

*

u(E),

w ( E ) ]E ~cl.

( € 1 :f [X(E), €1

E]

..., m,

E

=

0,

s t r i c t complementary

: 0, j = 1,

..., p ,

and Y ( E ) =

I t follows immediately t h a t

84

3 Fcn\iu\ ity

AndI)\i\

under Second-Order A\\umption\

yielding conclusion (a). Furthermore, we can differentiate (3.4.6) to obtain

*

VEf

(E)

=

*

€1

VEf[X(E),

= VEL

(E)

= V€L[X(E), U(E), W(E), E l =

(3.4.7)

+ VULVEU(€) + VWLVEW(€) + VEL,

VXLVEX(E)

where L is evaluated as in (3.4.6). Since the Karush-Kuhn-Tucker conditions hold at y ( ~ ) ,it follows that V L = 0. X

Complementary slackness implies ui(0) or

0, i = I,

..., m.

gi[x(0), 0

=

continuity

and differentiability then imply one of two con(i) gi[x(0), 01 > 0, implying

sequences, respectively: gi[x(E),

V €u. (E) 1

E]

> 0 for

Strict complementary slackness,

near 0, implying u ~ ( E ): 0, implying

E

0; or (ii) ui(0) > 0, implying u ~ ( E )> 0 for

0, implying gi[x(E), E]

5

0.

VULVEu(E) = (-gl[x(E), €1, since h.[x(E), E] 3

,..., hP[x(E),

E]

*

(3.4.7) that VEf

5

0 for

=

0.

Also,

near 0, VWLVEw(~)= (hl[x(E),

E])V~W(E)= 0. (E)

near

From this it follows that

..., -grn[x(E), EI)VEu(~)=

E

E

VEL for

E

We therefore conclude from near 0, proving conclusion

(b). Differentiation with respect to

E

of the result obtained

in (b) gives

v E2 f* ( E )

+ V&LVEU(€) = VXELVEX(E) 2

+ VWELVEW(E) 2 + VEL. 2

Calculation of the derivatives yields conclusion (c).

To be

perfectly clear about what is involved in calculating this

85

3.4 Derivatives of the Optimal Value Function

Hessian, we w r i t e (3.4.5)

i n t e r m s o f t h e o r i g i n a l problem

W e have

functions.

I

+

m

1

-

)V:f

1

1 P

+

ui(E)VZgi

i=l

\

'I

wj(E)V2h.(

j=l

m VEgiVEui(E) T

+

P

1

VEhTVEwj( € 1

(3.4.8)

1

j=1

The f o l l o w i n g two c o r o l l a r i e s a r e i m m e d i a t e , f o r two important r e a l i z a t i o n s of problem P 3 ( € ) . [ F i r s t - and s e c o n d - o r d e r

Coro 1 l a r y 3 . 4 . 2 ,

changes i n t h e

optimal value function of P 3 ( ~ ) w , i t h c o n s t r a i n t s independent of E.]

I f t h e c o n d i t i o n s o f Theorem 3 . 4 . 1

h o l d and t h e con-

s t r a i n t s o f P 3 ( ~ )a r e f u n c t i o n s o n l y o f x , t h e n i n a n e i g h b o r hood o f

(a)

E

= 0,

f

*

*

= L (E);

(E)

(b)

V E f * ( E ) = VEf;

(c)

2 * VEf ( € 1 =

and 2

VXftX(E),

€IBEX(€)

or

where VEx i s o b t a i n e d from ( 3 . 2 . 4 1 , N = [-VEXf

,

Corollary 3 . 4 . 3 .

El

with

[ F i r s t - and second-order changes i n t h e

unconstrained optimal value function.] C o r o l l a r y 3.2.3

2

+ VEf[X(E),

are s a t i s f i e d and f ( x ,

ously d i f f e r e n t i a b l e i n (x,

E)

I f t h e assumptions of E)

n e a r (x*,

is t w i c e continu-

o),

then i n a

86

3 Sensitivity Analysis under Second-Order Assumptions

neighborhood of

*

(a) VEf ( € 1

= 0,

E

=

VEf,

=

-VxEf 2 Vxf2 1 VEXf 2

+ VZfI [x(E) r

~

l

Another useful expression for VZf* is obtained by recalling that Vf(x(E) ,

E)

2 = 0 implies that Vxf VEx

+ V2xf

= 0, so that

Using this fact, it is easy to see that

v:f*

T

where

(3.4.9)

A formu a analogous to

3.4.9)

for the general problem,

with the Lagrangian L replacing f, will be obtained in Section 4.2 after manipulations of (3.2.3) that considerably simplify the analysis. Equation (3.4.4) immediately yields previously established results when certain problem structures are considered.

One

realization of (3.4.4) gives the well known "Lagrange multiplier sensitivity result" and then (3.4.5) also establishes relations for the Hessian of the optimal value function, with

87

3.4 Derivatives of the Optimal Value Function

respect to the rhs perturbations of the constraints.

For this

case, problem P 3 ( € 1 reduces to min

X

f (x)

~ . t .gi(x) 1. he 3

(XI

=

€j+,,,r

i = 1,

...

j = 1,

..., p.

r

P2 ( € 1

m,

Results for this problem will be developed in some detail in Chapter 5.

However, two important results are noted here

since they are immediate corollaries of Theorem 3.4.1. Coro Z Zary 3 . 4 . 4 .

[Optimal value function derivatives for

rhs perturbations.]

If

(i) the functions defining P 2 ( ~ )in a neighborhood of E

=

0 are twice continuously differentiable in x, in a neigh-

*

borhood of x

,

and

(ii) conditions (ii) in a neighborhood of

Proof.

i = 1,

E

E)

(iv) of Theorem 3.2.2 hold, then,

= 0,

We let f (x,

..., m, h3.(x,

-

=

E)

,

= f (x)

h. 3 (x)

-

gijx,

E)

=

j+m’ j = 1,

E.

gi (x)

-

E ~ ,

..., p, and apply

Because the functions defining

the results of Theorem 3.4.1.

P 2 ( ~ )are twice continuously differentiable in x, in a neigh-

*

borhood of x

,

the functions defining P 2 ( ~ )are twice continu-

ously differentiable in (x,

E)

in a neighborhood of (x*, 0 ) .

This, together with the satisfaction of conditions (ii)

-

(iv)

88

3 Sensitivity Analysis under Second-Order Assumptions

of Theorem 3.2.2,

satisfies the conditions of Theorem 3.4.1,

and hence the conclusions of Theorem 3.4.1

apply to P 2 ( E )

.

Let ek denote the m + p vector with a 1 in the k-th component and 0 ' s elsewhere. Theorem 3.4.1,

v E f*(E)

Then, recalling conclusion (b) of

we have that for

=

VEL = VEf

-

E

near 0,

m

2

uivEgi

i=l

and applying this to problem P 2 ( ~ )yields m

P

and hence VEf* ( E l T

for c near 0,

.= JE ); ( : [

as asserted in conclusion (a). Conclusion (b) results immediately from the differentiability of [u(E),

W(E)

I (conclusion (b) of Theorem 3.2.2) ,

differentiating the result of (a) with respect to

E.

A second, more general, corollary extends a result

established by Fiacco and McCormick (1968, Theorem 6 ) for the problem minx

f(x)

+ eoa0(x)

s.t. gi(x) + Eibi(x) 1. 0,

i = 1,

. . . I

m,

89

3.4 Derivatives of the Optimal Value Function

C o r o l l a r y 3.4.5.

[Optimal value derivatives for perturba-

tions that are linear in the parameters.]

If

(i) the functions of problem P L ; ( E ) are twice continu-

*

ously differentiable in x, in a neighborhood of (x (ii) conditions (ii)

,

O), and

- (iv) of Theorem 3.2.2 hold, then,

and

Proof.

+

We let f (x,

~ . b(x), . i = 1, 1 1

j = 1, ...,p,

E)

=

f (x)

..., m, h.(x, 7

E)

+ Eoao(x), gi (x, E ) = h. (x)

3

= g . (x)

+ ~~+~c~(x),

and apply conclusion (b) of Theorem 3.4.1,

having verified as in Corollary 3.4.4 that the conditions of Theorem 3.2.2

are satisfied.

In particular, with

*

denoting

90

3 Sensitivity Analysis under Second-Order Assumptions

evaluation at VEf*(0)

=

E

= 0, we have that

* *

(a*,, -Ulblr

* *

. * - I

* *

-Umbmr ~ 1 ~ - 1 . . r , W*C*)r P P

the result obtained by Fiacco and McCormick.

Differentiation

of (a) yields (b). Note (from Theorem 3.4.1) that the values of the optimal value function and its gradient can be calculated once the Karush-Kuhn-Tucker triple [x( E ) mined.

, u(E) ,

w ( E ) ] has been deter-

However, for the general problem, the value of the

Hessian matrix of the optimal value function requires the determination of both the triple and its first derivatives. Applications and examples of these results for the optimal value function and its derivatives are deferred to Section 4.3 (Chapter 4), where formulas are developed for the calculation of V Y(E) from (3.2.4).

These result from the various con-

ditions that can hold, under the assumptions of Theorem -1 3.2.2, leading to corresponding expressions for M(E) as a function of block components of this matrix.

The development

of these calculations follows from results originally presented in Armacost and Fiacco (1975).

However, the results

involving the connection between the optimal value Hessian and the second-order optimality conditions of the given parametric problem, viewing (x, for the first time.

E)

as the variable, are presented here

Chapter 4

Computational Aspects of Sensitivity Calculations: The General Problem

4.1.

INTRODUCTION In this chapter, we examine the numerical aspects of

calculating the parameter derivatives of the Karush-KuhnTucker triple for the general problem P(E), in particular seeking computational efficiencies.

Aside from their use in

calculating the optimal value function Hessian, they are of considerable importance in other applications, e.g., in characterizing the stability of the solution subject to perturbation and in providing a first-order estimate of KarushKuhn-Tucker triples of problems involving different values of the parameters, once one such triple has been determined. Formulas for the Karush-Kuhn-Tucker parameter-derivatives are given in Section 4 . 2 .

A small NLP example and a general

parametric LP example illustrate the results in Section 4.3. FORMULAS FOR THE PARAMETER FIRST DERIVATIVES OF A KARUSH-KUHN-TUCKER TRIPLE

4.2.

As noted in Section 3.2 (Chapter 3), when u(E),

W(E)

IT

Y(E)

=

is available, VEy(~)can be calculated.

[x(E), We

briefly recapitulate and then derive expressions for various

91

92

4 Computational Aspects of Sensitivity Calculations

cases.

R e c a l l t h a t t h e Karush-Kuhn-Tucker s a t i s f i e d by

n e c e s s a r y c o n d i t i o n s (3.2.2)

s.t. g(x,

min X f ( x , E)

2 0, h ( x ,

E)

first-order for

Y(E) E)

= 0

are ~ , L [ Y ( E ) , € 1 :V X f [ x ( E ) ,

m

1

-

€1

ui(E)VXgi[x(E),

€1

i=l

P

+

1

w.(E)V

j=1

3

0,

i = 1,

..., m,

hj[x(E), €1 = 0,

j = 1,

..., p ,

u ~ ( E ) ~ ~ [ x ( E E) ], =

along with

h . [ x ( E ) , € 1 = 0, X I

u ~ ( E )2

0 and g i [ x ( E ) , € 1 2 0 , i = 1,

(4.2.1)

..., m.

Before p r o c e e d i n g w i t h an a n a l y s i s o f ( 4 . 2 . 1 ) , that for

E

n e a r 0 , Theorem 3.2.2

assumed t h r o u g h o u t ) a s s u r e s t h a t

we note

(whose c o n d i t i o n s a r e b e i n g Y(E)

E

mentary s l a c k n e s s w i l l c o n t i n u e t o hold.

C1 and s t r i c t compleIn particular, t h i s

means t h a t nonbinding c o n s t r a i n t s remain nonbinding n e a r and

u ~ ( E )>

0 near

E

s o l u t i o n s o f (4.2.1)

= 0 i f u . (0) > 0. 1

near

E =

E

= 0

T h e r e f o r e , i n examining

0 , under t h e g i v e n a s s u m p t i o n s ,

w e can ( i )d e l e t e nonbinding c o n s t r a i n t i n f o r m a t i o n and ( i i ) d i v i d e o u t t h e nonzero these appear, i.e., gi = 0.

u ~ ( E ) from

t h o s e e q u a t i o n s i n which

w e c a n f o r s u c h i r e p l a c e uigi

= 0 by

I n t h e s u b s e q u e n t a n a l y s i s w e do i n f a c t d e a l w i t h

and e l i m i n a t e nonbinding c o n s t r a i n t i n f o r m a t i o n immediately. However, we choose n o t t o t a k e a d v a n t a g e o f b i n d i n g c o n s t r a i n t information, i.e.,

of t h e f a c t t h a t u i ( E ) g i [ x ( E ) ,

be r e p l a c e d by g i [ x ( E ) ,

E]

= 0 i f ui(0)

> 0.

E]

= 0 can

Our p u r p o s e i s

t o p r o v i d e comparisons v i a a p a r a l l e l t r e a t m e n t o f t h e c a l c u l a t i o n s with those t h a t t y p i c a l l y a r i s e i n "algorithmic"

93

4.2 Derivatives of a Karush-Kuhn-Tucker Triple

implementations, i.e., when a solution algorithm provides a solution

7

of a perturbation of (4.2.1) or when an estimate of

Y(E) solving (4.2.1) near

E

= 0 is given.

In such computa-

tions, the identification of the set of binding constraints at any given iteration may be at best tenuous and determined only in some limiting sense.

It is desirable to have each step of

the "nonalgorithmic" calculations available for comparison. An example of parallel algorithmic calculations of this information is given in terms of the penalty function computations in Chapter 6, Section 6.5. analysis of (4.2.1)

.

As in (3.2.3), let N(E)

-V hT, € 1

.IT

We return to a direct

=

[-v:xLT,

..., -u.v g., T 1

E

. . . I

l

and let

Since the system (4.2.1) is identically satisfied for 0, it can be differentiated with respect to

M(E)V~Y(E)=

"€1.

Under the conditions of Theorem 3.2.2,

E

E

near

to obtain (4.2.2)

M(E)

has an inverse;

thus vEy(E) = M(E)-~N(E); is an n

M(E)

n

+

m

+ p

x

(4.2.3)

+ m + p square matrix and VEy(€) and N(E) are

k matrices.

94

4 Computational Aspects of Sensitivity Calculations

Clearly, any method of solving ( 4 . 2 . 2 ) ,

a system of linear

equations, is satisfactory and M ( E ) need not be inverted as in -1 (4.2.3). However, the work involved in calculating M ( E ) can be significantly reduced, as will become evident. throughout that the conditions of Theorem 3.2.2

Assume

hold, and

suppose, without loss of generality, that the first r inequality constraints are binding.

For simplicity, we now proLet

ceed to eliminate nonbinding constraint information.

m

be defined as follows:

Then, rearranging the rows and columns corresponding to the last m

-

r (nonbinding) inequality constraints, it follows

that

M=[:

Let Y ( E )

=

[x(E), Ti(€),

w(€)IT and G ( E )

..., -V hT3 ' . IT, where i = T [ul(€),..., ur ( € 1 1 , and S

T -uiVEgi,

U(E)

=

E

gi[x(0), 01 > 0, i = r

1,

=

1,.

.., m,

continuity imply that u.(E) = 0, i

*

..., r,

= (gl,

+

I-v:XLT, .., j = 1,

..., 4,) . T

..., p, Since

complementary slackness and =

r

+

1,

..., m

for

E

near 0.

Hence, the corresponding components of V Y(E) are zero, i.e.,

95

4.2 Derivatives of a Karush-Kuhn-Tucker Triple

VEui = 0, i = r

+

1,

..., m.

It is thus assumed that non-

binding constraint information is suppressed in the Lagrangian or its derivatives with respect to x or It follows that ( 4 . 2 . 2 )

(Y(E), E).

E,

when evaluated at

may be reduced to solving

the system

-

M ( E ) V ~ ~ ( E=) E ( E ) .

-

Let, G since

5

E

diag(g.) and

(4.2.4) E

diag (ui), i = 1,

..., r.

Note that,

= 0,

0 0

0

Thus

(4.2.5)

-- 1

where U

=

diag(l/ui) for i = 1,

-

.

- ,r.

Thus far we have assumed no special structure on the problem.

However, in order to make further progress in calcu2 -1 , several special cases: (1) VxL lating B ( E ) - ~ consider

exists; (2) ViL = 0; ( 3 ) there are n linearly independent constraint gradients; and ( 4 ) r

+ p

< n.

Let

96

4 Computational Aspect\ 01 Scnvtivity Calculation\

an (r

+

p)

x

n matrix.

Let

k ( ~ ) -denote l

the left matrix on

3.

the right-hand side of (4.2.5); thus

P-["i"

(4.2.6)

Now suppose that (4.2.7)

I.

Our task is to determine the Aij for the various cases. eral general properties of the A.. are noted. 13

-1

Sev-

Clearly since

must be symmetric, All and A22 must be symmetric and T Aal = A12. Also, it is easy to show that All is positive semidefinite, under the given assumptions. This and zTAllz > 0 if Allz # 0 follow from the fact that All = AllV 2L All, M

PAll

=

0 and the second-order sufficiency condition.

It also

follows immediatly from our general Case 4 result below. We T 2 also note that A22 = -A12 V L A12 (Section 5.5), hence A22 is negative semidefinite if V 2L is positive semidefinite. Case 1.

V;L-'

Exists

It is easily shown that

- VXL 2 -1PT IPVXL 2 -1PTI-' , -

A12

=

A21

A22

=

-[PVxL 2 -1PTI-'

-'

Note that [PVxL P'1-l

. exists by our assumptions.

(4.2.8)

91

4.2 Derivatives of a Karush-Kuhn-Tucker Triple

Case 2 .

2

V L = 0 X

T h e r e a r e two p o s s i b l e s i t u a t i o n s :

there are r

+

p < n or

r + p = n l i n e a r l y independent binding c o n s t r a i n t gradients. I f t h e r e a r e f e w e r t h a n n , a s s u m p t i o n ( i i )o f Theorem 3.2.2

f i ( ~ ) -dIo e s

v i o l a t e d and i t i s e a s i l y s e e n t h a t

is

not exist.

For e x a m p l e , t h i s c o r r e s p o n d s t o t h e s i t u a t i o n c h a r a c t e r i z i n g a d e g e n e r a t e s o l u t i o n i n l i n e a r programming.

y (

W e s h a l l n o t pursue t h i s

~ may ) n o t even b e continuous.

p o s s i b i l i t y f u r t h e r here.

I n t h i s case,

When t h e r e are n l i n e a r l y independ-

e n t b i n d i n g c o n s t r a i n t g r a d i e n t s w e have a s p e c i a l i n s t a n c e o f Case 3, which i s d e v e l o p e d n e x t . Case 3 .

There a r e n LinearZy Independent Binding C o n s t r a i n t Gradients

The J a c o b i a n of t h e n c o n s t r a i n t s w i t h r e s p e c t t o t h e n -1 v a r a i a b l e s i s nonvanishing, i.e., P e x i s t s . Hence, All

= 0,

A12

= A l:

= P

-1

Note t h a t i f V:L-’

,

(4.2.9)

exists,

(4.2.9)

i s c o m p u t a b l e from ( 4 . 2 . 8 ) ;

however, h e r e t h e e x i s t e n c e of V2L-l X

i s n o t assumed.

Also,

t h e r e m a i n i n g C a s e 2 p o s s i b i l i t y m e n t i o n e d above g i v e s All

= A22 = 0

a n d A12

= AT1 = P-’.

N o t e also t h a t t h e

s t a n d a r d l i n e a r programming problem n o n d e g e n e r a t e s o l u t i o n c a s e f a l l s i n t o t h i s l a t t e r category, w i t h n l i n e a r l y independent b i n d i n g c o n s t r a i n t g r a d i e n t s and V L: Case 4 .

r

+

= 0.

p < n

T h i s r e p r e s e n t s t h e more g e n e r a l s i t u a t i o n a n d i s t r e a t e d i n d e t a i l here. matrix.

I n (4.2.4),

E(E) i s a n n + r + p s q u a r e

By a s s u m p t i o n (iii)o f Theorem 3.2.2,

t h e Vxgi,

98

4 Computational Aspects of Sensitivity Calculations

..., r and

...,

the V h. j = 1, p are linearly independx 3' By assumption (ii) of Theorem 3.2.2 the second-order

i = 1, ent.

sufficient conditions for a local minimum are satisfied, and by assumption (iv), strict complementary slackness holds. Thus, the Hessian of the Lagrangian is positive definite with respect to those nonzero vectors orthogonal to the binding constraint gradients, and since r + p vectors is nonempty. z

T VxLz 2 > 0,

n, the set of such

Hence

for all z # 0 such that m

'I'

(4.2.10)

PZ = [-VxqT, VxhT) z = 0

and at least one such z exists.

Under the assumption of

linear independence of the binding constraint gradients, P has rank r

+ p, and without

l o s s of generality, it can be assumed

that the submatrix involving the first r nonsingular. =

p columns of P is

Therefore, we partition P as follows:

[pa, pbl I

where Pa is an (r an (r

+

+ p)

x

+

p)

x

(r

+

p) nonsingular matrix and Pb is

q matrix, where q

=

n

- r

-

p.

This partition

and (4.2.10) allow us to express the first r z in terms of the remaining q components. where z E Er and zb E E q . a tion given in (4.2.10) yields +

and hence za = -P,lPbzb.

+

p components of T Let z = (za, zb) ,

Then the orthogonality condi-

99

4.2 Derivatives of a Karush-Kuhn-Tucker Triple

Then

s a t i s f i e s (4.2.10). T

=

Letting

(pij1

-T

= pa

1..

we can f u r t h e r conclude t h a t

+:.[

-P;lPb

.;[-.;,TI

I

' 0.

(4.2.11)

Thus, t h e m a t r i x

Dxx

T 2 = T VxL T I

where T =

is p o s i t i v e d e f i n i t e and hence has an inverse. l e a d s us t o a r e p r e s e n t a t i o n of M(E) A

Recalling t h a t $ ( e ) - l right-hand s i d e of

(4.2.5)

-1

.

This f a c t

d e n o t e s t h e l e f t m a t r i x on t h e I

and g i v e n t h a t

we can w r i t e

A

M =

(4.2.12)

I00

4 Computational Aspects of Sensitivity Calculations

Using the partitions defined above, the result is All = T Dii TT,

(4.2.13) -1 A22

=

-A.]21 :[L2Vx

We have now developed analytical expressions for

G ( E ) -1

for all of the specified cases, and now turn to the calculation of VET(€), using the block components Aij of

[

i-’.

Defining N = -VzxLT, VETT, -VEhTIT and recalling that N = [-VExL 2 T , -(EVEg) T, -VEh , it follows that A

0

-u--1

0

0

0 N,

I,

(4.2.14)

1

-A1lV,xL 2 E)

+

=

-A21VExL 2 +

-

A12[-::”,1

““3

I

-VEh

J

.

(4.2.15)

101

4.2 Derivatives of a Karush-Kuhn-Tucker Triple

Two aspects of Eqs. (4.2.14) and (4.2.15) may be noted. First, the diagonal matrix involving the Lagrange multipliers (4.2.5) cancels out (as already anticipated) ,

in B(E)-'

resulting in a computational savings, since we need only concern ourselves with

in solving (4.2.15).

Second, the

portion of (4.2.15) corresponding to the partial derivatives of the Lagrange multipliers with respect to the parameters is given in conglomerate form, i.e., we have not obtained separate expressions for the quantities VEU(~)and V€W(E) at this level of generality, although V E X(E) may be calculated separately. Since we would generally be interested in estimating either changes in X(E) or changes in the Karush-Kuhn-Tucker triple [x(E), u(E), w(E)], this poses no problem.

Also, there is no

difficulty in applying (4.2.15) intact to calculate the 2 * * Hessian VEf (E) of the op.tima1 value function f ( € 1 ,

since it

follows easily from (3.4.5) that = -fiTV

V:f* where

fi

E

7

+

1^ 2 V2 L = -N M N + VEL E

is defined as in (4.2.14) and

VET is

(4.2.16) calculated as in

(4.2.15). Equation (4.2.14) holds for the general problem whenever (4.2.4) is well defined, in particular when the conditions of Theorem 3.2.2 hold.

For the (exhaustive) cases treated in

this section, V EY ( E ) may then be calculated from (4.2.15) by first evaluating the Aij as given in (4.2.81, (4.2.91, or (4.2.131, depending on which conditions apply.

102

4 Computational Aspects of Sensitivity Calculations

Defining D

(x, E ) L

(VxL, V E L ) , we observe t h a t V 2 f *

=

(E)

may

be expressed d i r e c t l y i n t e r m s of t h e j o i n t H e s s i a n

of t h e L a q r a n q i a n ( 3 . 2 . 1 )

of problem P 3 ( € ) .

T h e r e s u l t may

be d e r i v e d f r o m t h e p r e v i o u s r e l a t i o n s , b u t w e s i m p l y s t a t e it

here a n d t h e n v e r i f y it.

v;f*

The f o r m u l a i s

:I.

v x =

[ V E XT I

I,,.&[2

(4.2.17)

To c h e c k i t s v a l i d i t y , we e x p a n d t h e e x p r e s s i o n and u s e t h e known f a c t s t o show t h a t t h e r e s u l t agrees w i t h ( 3 . 4 . 5 ) . Expanding (4.2.17)

v;f*

=

yields

V E X TVXLVEX 2

From ( 4 . 2 . 6 )

+

V;€LvEX

and (4.2.14)

+

vEXTv:XL

+

VEL. 2

(4.2.18)

we r e c a l l t h a t

which yields

(4.2.19)

4.2 Derivativeb of a Karush-Kuhn-Tucker Triple

103

Multiplying the first equation by VExT , using the second equation, and rearranging gives

Using this in (4.2.18) and rearranging gives V:f*

=

VXELVEx 2 - (VEgT, -VEhT]

which corresponds precisely with (nontrivial) components of (3.4.5). The formula (4.2.17) explicitly provides the precise relationship between the Hessian of the optimal value function and the joint (variable-parameter) Hessian of the Lagrangian of the general parametric program P 3 ( ~ ) ,under the given assumptions.

Some well known results are corroborated im-

mediately and some new enlightening relationships may be inferred from this expression. If VEx is defined and the Lagrangian joint Hessian is

defined and positive semidefinite along

[Y(E),

E]

on some

domain, then it follows immediately from (4.2.17) that the optimal value Hessian V 2 f"

(6)

of problem P3(€) is defined and

positive semidefinite over that domain.

This is consistent

with the well known fact (Mangasarian and Rosen (1964)) that

*

if P3 ( E ) is a jointly convex program, then f

(E)

is convex.

I04

4 Computational Aspects of Sensitivity Calculations

Of c o u r s e ,

(4.2.17)

t e l l s u s more.

I t i s n e c e s s a r y and s u f f i T 2

c i e n t f o r V:f*

t o be p o s i t i v e s e m i d e f i n i t e , t h a t z V

f o r a l l z such t h a t z

T

= y

T

(VEX

T

,

11.

LZ 0 (x, E ) This extends t h e noted

r e s u l t , under t h e g i v e n a s s u m p t i o n s , and e x p l i c i t l y shows why t h e o p t i m a l v a l u e o f a l a r g e r ( t h a n j o i n t l y convex) c l a s s o f problems i s convex.

I t would be i n t e r e s t i n g t o d e v e l o p t h e

i m p l i c a t i o n s o f t h i s r e s u l t and, i n p a r t i c u l a r , t o o b t a i n analogs of (4.2.17)

under weaker a s s u m p t i o n s .

Rephrasing t h e l a s t o b s e r v a t i o n l e a d s t o a n o t h e r v e r y i n t r i g u i n g c h a r a c t e r i z a t i o n of what i m p l i e s t h e c o n v e x i t y of f * ( E ) .

Recall t h a t

T [ x ( E ) , €1

which w e deduce t h a t V x g E x Defining PE =

(-VETT,

+ VET =

0 and VxhVEx

+

VEh = 0 .

VEhTIT and r e c a l l i n g t h a t P =

V hTIT w e c o n c l u d e t h a t PVEx X

: 0 and h [ x ( E ) , € 1 = 0 , from

+

[-VxqT,

PE = 0 , o r e q u i v a l e n t l y t h a t

(4.2.20)

T h i s r e l a t i o n s h i p i s o f c o u r s e e q u i v a l e n t t o t h e second equat i o n of (4.2.19)

,

and w e have chosen t o r e d e r i v e i t t o r e c a l l

i t s o r i g i n f o r o u r p r e s e n t purpose. 0 (>O)

T 2

Suppose t h a t z V

2.

f o r e v e r y v e c t o r z(z # 0 ) such t h a t ( P , P E ) z = 0.

t h e n c o n c l u d e i n p a r t i c u l a r from (4.2.17) 2 * VEf i s p o s i t i v e s e m i d e f i n i t e ( d e f i n i t e ) .

and ( 4 . 2 . 2 0 )

We

that

(The d e f i n i t e p a r t

of t h e s t a t e m e n t f o l l o w s a s w e l l , n o t i n g t h a t no column of

(

VEXT,

11'

is the zero vector.)

But r e c a l l i n g t h e second-order

s u f f i c i e n t c o n d i t i o n s (Lemma 3.2.1)

f o r a s t r i c t l o c a l minimum,

and n o t i n g t h a t t h e second-order n e c e s s a r y c o n d i t i o n s ( w i t h a p p r o p r i a t e r e g u l a r i t y ) a r e t h e same w i t h t h e s t r i c t i n e q u a l i t y

sTV 2L s > 0 r e p l a c e d by i n e q u a l i t y , w e o b s e r v e t h e f o l l o w i n g

4.2 Derivatives of a Karush-Kuhn-Tucker

fact:

For given

E

Triple

105

such that the assumptions of Theorem 3 . 2 . 2

hold, the Hessian V:f*

is positive definite (semidefinite) if

the second-order part of the second-order sufficient (necessary) conditions for a local minimum of P: min (XrE) f (xr E ) s.t. g(x, E ) 2 0, h(x, E ) = 0 hold, relative to (x, E ) viewed jointly as the variables, at (x,

E)

= [x(E),

€1.

This

establishes a direct connection between the convexity of the

*

optimal value f

(E)

of P 3 ( ~ )and second-order sufficient

(necessary) conditions for P 3 ( € ) , where (x,

E)

is viewed as a

variable. This relationship should lead to additional insights regarding the optimal value Hessian. is suggested. tion over (x,

An immediate application

Suppose the given problem P requires minimizaE)

jointly, and we attempt to solve the problem

by a decomposition scheme based on fixing a minimum xo to update

E

=

=

E =

solving for

x ( E ~ )over x, using the accumulated information E ~ etc. ,

(An example application of this type This might be viewed as a scheme

is given in Section 5.6.)

for solving P based on solving a sequence of parametric pro, = 0 , 1, grams ( P ( E ~ ) ) k

(51,

... .

Suppose the solution of P is

E) and that the given scheme successfully generates a - - ( X r ur Wr E ) with

minimizing sequence (xk r

ukr wkr E ~ ) (uk, wk) the associated optimal Lagrange multipliers. +

suppose the assumptions of Theorem 3 . 2 . 2

Finally,

hold for problem P ( E )

for large k, and the second-order conditions for a strict local minimum of problem P hold at (F, Ti, W, F ) . 2 * large k , the Hessian VEf

( E ~ )of

Then, for

the optimal value of P ( E ~ )

(with nonbinding constraint information deleted) will be positive definite, a fact that can be exploited; e.g., justifying applications of Newton's method to update E ~ and ,

106

4 Computational Aspects of Sensitivity Calculations

i n t r o d u c i n g t h e p o s s i b i l i t y of c a l c u l a t i n g bounds on t h e o p t i ma1 v a l u e f

*

of P ( E ) near

(E)

Chapter 9 1 .

T ( e . g . , see S e c t i o n 2.5 and

=

E

Many more a p p l i c a t i o n s o f t h i s c h a r a c t e r i z a t i o n

a r e anticipated. 4.3.

APPLICATIONS AND EXAMPLES An i n d i c a t e d a p p l i c a t i o n o f t h e s e n s i t i v i t y a n a l y s i s

r e s u l t s o b t a i n e d h e r e i s a c a l c u l a t i o n o f f i r s t - and secondorder estimates of t h e optimal value function, using a T a y l o r ' s series e x p a n s i o n . P

3

(E)

If

X(E)

i s a s o l u t i o n of problem

s a t i s f y i n g t h e c o n d i t i o n s of Theorem 3 . 2 . 2

(Chapter 3 ) ,

t h e n a f i r s t - o r d e r e s t i m a t e of t h e o p t i m a l v a l u e f u n c t i o n , f [ x ( ~ ) €, 1 ,

f* ( E)

f*(E)

+

f*(O)

f*(O)

*

where VEf

o f VEf

VEf*(0)E,

+

1 VEf*(0)E+ 3

E

T 2 * VEf ( O ) E ,

(4.3.2) and (3.4.51,

A s an i l l u s t r a t i v e example of t h e c a l c u l a t i o n

2 * and VEf ( E ) ,

(E)

(4.3.1)

2 * and VEf ( E ) a r e d e f i n e d by ( 3 . 4 . 4 )

(E)

respectively.

*

i n a neighborhood of 0 , i s g i v e n by

E

e s t i m a t e i s g i v e n by

and a second-order f*(E)

for

c o n s i d e r t h e f o l l o w i n g convex ( i n x )

problem, which was d i s c u s s e d i n Armacost and F i a c c o ( 1 9 7 4 ) .

Example 4. 3. 1 min

X

f(x,

s.t. gl(x, for

-

E~

> 0.

ulgl(x,

E)

L ( x , u,

E)

= x1

+

~~x~

E)

= El

2

-

2 x1

-

2 x2

L

0

The Lagrangian i s d e f i n e d a s L ( x , u, and i s g i v e n h e r e by E)

E

x1

+

E2X2

-

ul(

E;

-

2 x1

-

2 x2).

E)

= f(x,

E)

107

4.3 Applications and Examples

By application of the Karush-Kuhn-Tucker conditions the solution may be found algebraically to be -&

(1

+

1

€;y2

-€lE2

(1

= U1(E) =

U(E)

+

f*(d = -El(l

(1

+

1/2

€3

+

(4.3.3)

2E1 E2 2

The surface depicting the optimal value function is difficult to portray graphically in three dimensions.

However,

isovalue contours of the function in parameter space are shown in Fig. 4.3.1.

Since we have the optimal value function

analytically, we can determine its gradient and Hessian directly, giving

vEf*

(€1 =

(af*

af* (€)lac2) (4.3.4)

and -2

0 2 *

VEf

(E)

(1

=

-‘2 (1

+

€;y2

-I

+ -El

(4.3.5)

108

3 Computational Aspects of Senbitivity Calculations

=2

2

1

0

-1

-2

Fig.

4.3.1.

I s o v a l u e c o n t o u r s of

f*fE)

=

-E~(I

+

1/2 E:)

To i l l u s t r a t e t h e a p p l i c a t i o n o f t h e g e n e r a l e q u a t i o n s

d e r i v e d i n Theorem 3.4.1,

w e s h a l l u s e them t o d e r i v e t h e

g r a d i e n t and t h e H e s s i a n of t h e o p t i m a l v a l u e f u n c t i o n f o r

and hence

which i s p r e c i s e l y (4.3.4)

a t t h e s o l u t i o n p o i n t (4.3.3).

.

109

4.3 Applications and Examples

From (3.4.5), VEf 2 * (E)

t h e H e s s i a n of f

+

= V2 XELVEX(E)

evaluated a t [ x ( E ) , u ( E ) ] , W e obtain

v 2f * ( € 1

[ o0

=

E

,] 0

VEX(€)

VEL 2

-

*

(E)

i s g i v e n by

vEgTvEU1(E)

where,

from a b o v e , V L = (-2u1c1, E

;] ";.I -

+ [-2u;(E)

and s i n c e from t h e above s o l u t i o n for [ x ( E ) , u

late that

v EX ( E )

= 1

L

(1 +

E

y

+ €$I2

(1

w e have t h a t 0 2 * V E f (E) =

-E,

0 -E-

..

.

.;y2 2

'1

+

-1 (1 +

E$/

2

x2).

VEU1(E)

(E)]

1

we calcu-

4 Computational Abpects of Sensitivity Calculations

I10 which c o r r e s p o n d s w i t h (4.3.5), differentiating the solution f

*

t h e r e s u l t o b t a i n e d by t w i c e (E)

obtained analytically.

Of c o u r s e , w e c a n n o t g e n e r a l l y e v a l u a t e t h e r e q u i r e d V X(E) E

and V

E

U(E)

a n a l y t i c a l l y a n d must t u r n t o some n u m e r i c a l

technique f o r s o l v i n g t h e system (4.2.3 ) ,

last section, f o r these quantities.

t h e s u b j e c t of t h e

The example s e r v e s f u r -

t h e r t o i l l u s t r a t e t h e usefulness of t h e r e s u l t s obtained there.

2:j

F o r example, w e f i n d t h a t

v 2L X

= ";I

= 2UlI,

and s i n c e u ~ ( E ) > 0 f o r a l l

E

t h e r e s u l t s given i n (4.2.8)

u n d e r c o n s i d e r a t i o n , C a s e 1 and apply.

i n v e r s e i s r e q u i r e d i s g i v e n by ( 4 . 2 . 6 )

h=[

,"

V2L

o]l 2 x 2 ) f o r t h e example.

W e wish t o d e t e r -

such t h a t

and t h e n c a l c u l a t e t h e V € X ( E ) S i n c e V$,-'

as

PT

where P = Vxg = (2x1, mine t h e A i j

h

The m a t r i x M ( E ) whose

and V € U ( E )

from ( 4 . 2 . 1 5 ) .

= ( 1 / 2 u 1 ( ~ ) ) 1 ,w e f i n d from ( 4 . 2 . 8 )

that

4.3 Applications and Examples

A12

= AT

1

21

A22 -

1

=-I 2ul2 ( x f

-2 ( x f

+

1

+

x;

x;).

We now proceed to evaluate ( 4 . 2 . 1 5 ) .

vXL

= (1

+

2U1X1.

+

E2

Since

2u1x2)

it f o l l o w s that 2 EX

0 =

[o

0 1 1 1

and since g l ( x r

E)

=

2

E~

-

2

x1

-

2 x2 ' we have that VEgl = ( 2 ~ ~ ~ 0 ) .

We now have all of the information necessary to calculate V€Y(E) from ( 4 . 2 . 1 5 ) ,

and obtain

r =I

1

-1 2(x;

+ x;)

0 x2)[o

0 11

+

2(xf

-1

+

XC)

(2c1r

0)

2 Since x1

+ x22

=

E

2

~

,

x1x2

7 2 9 1

2 -xl 2 2UlEl

VEY(E) =

-x2

7 2E1

Therefore, we may conclude that

VEX(€) =

12

J

,

-x:

VEU(E)

=

(-2 ->). 2E1

7 2UlE1

It may readily be verified (with the understanding that u ~ ( E ) )that this corresponds exactly with the result obtained above by explicit differentiation. All

x1

=

x l ( ~ )and u1

=

the required calculations can be made for any admissible value of E, once

X(E)

and

U(E)

are known.

As another example, consider a parametric linear programn ming problem in E .

I13

4.3 Applications and Examples

Example 4 . 3 . 2 minx

f(x,

E)

= C

E)

= gi(x)

s.t. g i ( x ,

~

+~

E

-

*X. * ~

+

c

E x n n n

1. 0 ,

E~

i = 1,

..., m,

LP

(E)

where m 2 n , t h e ci are c o n s t a n t s , t h e gi are l i n e a r , a n d t h e c o n d i t i o n s o f Theorem 3.2.2

hold.

W e now c a l c u l a t e V € X ( E )

and

using t h e formulas developed i n t h e l a s t s e c t i o n .

V€U(E),

The L a g r a n g i a n f o r problem L P ( E ) i s L(x, u,

=

E)

c

T

(E)X

-

m

1

uitgi(x)

-

Eil,

i=l

where c

T

(E)

...,

= ( C ~ E ~ ,

2

c ~ E ~ ) S . i n c e VxL =

0, C a s e 2 a p p l i e s .

The s a t i s f a c t i o n o f t h e c o n d i t i o n s of Theorem 3.2.2

requires

t h a t t h e r e be n l i n e a r l y i n d e p e n d e n t b i n d i n g c o n s t r a i n t g r a d i -

A(€)-'

e n t s , a n d h e n c e w e o b t a i n t h e components o f

from

(4.2.9).

L e t u s assume t h a t t h e f i r s t n c o n s t r a i n t s a r e

binding.

T h i s i s o n e o f s e v e r a l p o s s i b l e cases.

V a r i a n t s of

t h i s s i t u a t i o n i n v o l v i n g o t h e r sets o f b i n d i n g c o n s t r a i n t s c a n a l s o b e h a n d l e d i n a s t r a i g h t f o r w a r d manner, though t h e c a l c u -

l a t i o n s a r e somewhat more t e d i o u s . Consistent with our p r i o r notation, g ( x ,

€1,

..., g n ( x ,

E)I

Equation (4.2.9) A 2 2 = -P-TViLP-l,

fi-l

= [p;T

T

uiVxgi,

f o r i = 1,

a n d P = [-V XQ1 = 1-V x g 1'

g i v e s All

"1.

...,

-Vxg:lT. -1 = P , and

T = A21

= 0 , A12

= [g,(x,

E)

and hence

To u s e ( 4 . 2 . 1 5 )

-

T

,

w e r e q u i r e V 2 L and EX

VET.

S i n c e V X L = cT

w e f i n d t h a t VZxL = ( C , 01, where C = d i a g ( c i )

..., n.

Since g . (x,

E)

=

gi(x)

-

E

~

,i t

follows

I14 that

4 Computarional Ahpecth of Senbitivity Calculations

VET =

W e c a n now c a l c u l a t e

(-1, 0).

= [p-?

-P-TC ,

...,

E)

and o b t a i n

11

where G ( E ) = [ u ~ ( E, ) gi(xr

(4.2.15)

+

1. 0 for i = n

u ~ ( E ])

T

.

Since the constraints

l , . . . , m are n o t b i n d i n g , i t f o l l o w s

t h a t u1 . ( E ) = 0 and V E u . l ( E ) = 0 f o r i = n

+

1,

..., m.

This

, V€U(E) I . c o n s t i t u t e s t h e c o m p l e t e s o l u t i o n for [V€X(E) W e can a l s o e a s i l y c a l c u l a t e t h e g r a d i e n t and t h e Hessian

of t h e o p t i m a l v a l u e f u n c t i o n s , using (3.4.4) obtaining

v

f*(E)

T

= VEL =

+

(x (E)C, 0)

= (XT(E)C, 0) = (XT(E)C

+

+

(iiT(E)

TiT(€),

T

u (€1

,

0)

0)

and

y

-1

=

=[

-cP-l

,

0)

0

-

P-TC, 0

1

(P-TC, 0)

"1.

and ( 3 . 4 . 5 ) ,

115

4.3 Applications and Examples

This example illustrates how we can easily calculate sensitivity results for many classes of parametric problems, using the general relationships developed. The next chapter continues the analysis that has been conducted here for the general parametric nonlinear programming problem P 3 ( E ) , pursuing ramifications for important realizations of P3(€), where the parameters appear only in the righthand sides of the constraints. on Armacost and Fiacco (1976).

This material is based largely

Chapter 5

Computational Aspects: RHS Perturbations

5.1.

INTRODUCTION It is clear from the previous results and is also a long-

established fact that optimal Lagrange multipliers are associated with a measure of the sensitivity of the optimal value of the objective function with respect to rhs changes in the constraints of a mathematical programming problem.

Corol-

lary 3.4.4 of Chapter 3 gives a precise result under the assumptions of Theorem 3.2.2 of that chapter.

Recent work in

sensitivity and stability analysis for nonlinear programming, as indicated in Chapter 2 , provides incisive characterizations under very general assumptions.

Given the assumptions of

Theorem 3.2.2, Theorem 3.4.1 gives closed form expressions for the gradient VEf* and Hessian VZf* of the optimal value of P3W.

As a matter of historical interest, this chapter first sketches the development of the interpretation of Lagrange multipliers as sensitivity estimates in optimization problems subject to rhs perturbations.

Following this brief chrono-

logical perspective, additional results are developed that are based on the general treatment presented in the last two

I I6

I17

5.2 Use and Initial Interpretation of Lagrange Multipliers

chapters.

The reader not wishing to make this historical

digression may proceed directly to Section 5.4, where the development of technical results is resumed. The brief recapitulation of developments in the use and interpretations of Lagrange multipliers in linear and nonlinear programming problems is given in Section 5.2.

In Section 5.3,

several developments of the analytical interpretation of Lagrange multipliers as sensitivity estimates not covered previously are quickly reviewed.

In Section 5.4, the given

theoretical results that hold for general parametric nonlinear programming problems are specialized and easily yield the Lagrange multiplier sensitivity result and the second-order change in the optimal value function for problem P2(~). Formulas for calculating the first derivatives of a Karush-KuhnTucker triple and the Hessian of the optimal value function are developed in Section 5 . 5 .

In Section 5.6, several appli-

cations of first- and second-order sensitivity information of

*

the optimal value function f

(E)

of problem P 2 ( ~ )are indi-

* (E)

cated, including the second-order estimation of f values of

E

for

near the starting value, and the use of sensitivity

information in developing a cyclical (decomposition) scheme for solving a "standard" NLP problem. 5.2.

THE USE AND INITIAL INTERPRETATION OF LAGRANGE MULTIPLIERS

A classical problem that has been examined by numerous mathematicians is the determination of the maxima or minima of a function f(x) of several variables required to satisfy one or more equations, h.(x) = 0, j = l,..., 7

p.

One approach to

solving this problem was proposed by Lagrange (1881) and

I18

5 Computational Aspects: RHS Perturbations

involved a function he introduced, now known as the Lagrangian. For the indicated problem, the Lagrangian is defined as L(x, w) = f(x)

+

P

2

wjhj(x).

(5.2.1)

j=1

The constants w

j

were often called undetermined multipliers,

and are now generally known as Lagrange multipliers.

The

"Lagrange Multiplier Rule" consists of solving the system of equations VxL(x, w) = 0, h .( x ) = 0, j = 1, 7

..., p, for x and w.

One of the resulting solutions will sometimes yield a solution of the given problem.

Lagrange did not provide a detailed

explanation of when or why. Incredibly, not until an extensive work published by Hancock (1917), do we find a rigorous unified treatment of necessary and sufficient conditions, based on Lagrange's approach, that characterizes relative extrema subject to subsidiary equalities.

Caratheodory (1935) likewise established

necessary and sufficient conditions involving Lagrange multipliers for a "minimum in the small," conditions less cumbersome in their development than Hancock's.

Both authors

established the important second-order sufficient condition for an isolated local minimum, i.e., that at a local solution the Hessian of the Lagrangian be positive definite with respect to all nonzero vectors orthogonal to the gradients of the constraints. Hancock also developed a test involving determinants to determine if the sufficient condition is satisfied.

Mann

(1943) developed a procedure that was simpler than that given by Hancock to test the satisfaction of the second-order sufficient condition, again based on evaluating certain determinants.

I19

5.2 Use and Initial Interpretation of Lagrange Multipliers

Curiously, many current texts in analysis and advanced calculus still replicate the classical approach and examine only equality side conditions in extremal problems. order conditions are often not developed.

Second-

In addition, most

of these classical mathematical treatments do not appear to offer an i n t e r p r e t a t i o n of the Lagrange multipliers.

Even

most books in nonlinear programming have not until very recently developed or applied the second-order conditions. A primary use and subsequent development of the "sensitivity" interpretation of Lagrange multipliers and associated terminology appears to derive from an analysis of economic models.

Allen's (1938) text for economists used Lagrange

multipliers to solve various models, though it did not offer a specific interpretation of them.

Hicks (1939), while not di-

rectly employing Lagrange multipliers in his mathematical treatment, makes frequent reference to the corresponding concepts, the "marginal utility of money" (attributed to Marshall (1920)) and "shadow prices." Samuelson (1947) gave several interpretations of Lagrange multipliers in an economic setting and developed rigorous approaches to different economic structures using analytical methods, including an analysis of mathematical models involving extrema subject to equality constraints (with all functions twice continuously differentiable).

The solutions were ob-

tained by Lagrange's method of undetermined multipliers.

For

a cost minimization problem in the theory of the firm, the Lagrange multiplier is clearly identified as a marginal cost, in the theory of consumer behavior as the marginal utility of money, and in welfare economics as a price.

In a section

dealing with transformations of independent variables Samuelson

I20

5 Computational Aspects: RHS Perturbations

showed a n a l y t i c a l l y t h a t t h e p a r t i a l d e r i v a t i v e o f t h e o p t i m a l v a l u e f u n c t i o n w i t h r e s p e c t t o t h e r h s p a r a m e t e r o f an e q u a l i t y c o n s t r a i n t i s t h e Lagrange m u l t i p l i e r a s s o c i a t e d w i t h t h a t constraint. Following t h i s i n i t i a l r i g o r o u s i n t e r p r e t a t i o n o f Lagrange m u l t i p l i e r s , s u b s e q u e n t developments f o l l o w e d a l o n g t h e l i n e s of t h e newly emerging t h e o r y o f l i n e a r and n o n l i n e a r programming.

Fundamental r e s u l t s i n l i n e a r programming i n 1947-1948

were p r e s e n t e d t o s e v e r a l m e e t i n g s o f t h e Econometric S o c i e t y i n 1948 and a p p e a r e d i n s e v e r a l p a p e r s p u b l i s h e d under t h e e d i t o r s h i p o f Koopmans ( 1 9 5 1 a ) .

Possibly t h e f i r s t proof o f a

d u a l i t y theorem f o r l i n e a r programming by Gale, Kuhn, and Tucker ( 1 9 5 1 ) , i s c o n t a i n e d t h e r e i n .

A r e l a t i o n s h i p between

d u a l v a r i a b l e s and Lagrange m u l t i p l i e r s i s c u r i o u s l y n o t n o t e d . I n t h e same volume, Koopmans (1951b) f o r m u l a t e d a l i n e a r p r o d u c t i o n model i n t h e t e r m i n o l o g y o f " a c t i v i t y a n a l y s i s .

"

I n t h i s c o n t e x t , t h e v e c t o r s normal t o t h e " a c h i e v a b l e cone" a t an " e f f i c i e n t p o i n t " a r e i d e n t i f i e d a s "shadow p r i c e s . " The e x i s t e n c e o f p r i c e s i s shown t o be n e c e s s a r y and s u f f i c i e n t f o r t h e a c t i v i t y v e c t o r t o l e a d t o an e f f i c i e n t p o i n t .

The

p r i c e v e c t o r i s shown t o d e f i n e t h e " m a r g i n a l r a t e o f subs t i t u t i o n " and i s u s a b l e under a regime o f d e c e n t r a l i z e d decisions. Goldman and Tucker ( 1 9 5 6 ) , i n a volume e d i t e d by Kuhn and Tucker ( 1 9 5 6 ) , p r e s e n t e d a t h e o r e t i c a l development o f l i n e a r programming, and began by i n t r o d u c i n g d u a l v a r i a b l e s a s p r i c e s i n t h e c o n t e x t o f an economic problem.

I n t h a t paper they

developed t h e e x p l i c i t r e l a t i o n s h i p between d u a l i t y and Lagrange m u l t i p l i e r s f o r l i n e a r programs.

Another p a p e r i n

t h a t volume by M i l l s (1956) examined t h e m a r g i n a l v a l u e o f

121

5.2 Use and Initial Interpretation of Lagrange Multipliers

matrix games (and simultaneously the marginal value of linear programs due to the equivalence between matrix games and linear programs) when a particular kind of perturbation was allowed.

It was shown that with this perturbation Lagrange

multipliers correspond to dual variables.

Under certain

specified conditions the marginal value with respect to a constraint is the corresponding dual variable and is also the "generalized" Lagrange multiplier.

It was also pointed out

that this interpretation was easily extended to nonlinear programming.

In their preface, Kuhn and Tucker (1956) identified

this examination of marginal values as a sensitivity analysis. Charnes and Cooper

1961) indicated that a sensitivity inter-

pretation should be extended to "dual evaluators,'' their term for dual variables. [e.g., Dantzig (196

In subsequent works on linear programming ),

Gale (1960)], liberal economic inter-

pretations are made and the relationship to Lagrange multipliers is often explicit.

Dantzig makes extensive use of the

term "sensitivity analysis" when describing what is also called "post-optimality analysis" for linear programs. As noted above, the development of Lagrange's method was

essentially limited to the problem where the constraints are equations.

Necessary conditions that allowed inequality con-

straints were obtained by Karush (1939) using the calculus of variations, though this result in his unpublished master's thesis became widely known only several years ago.

Fritz John

(1948) established general necessary conditions for the solution of an optimization problem involving i n e q u a l i t y side conditions, using a Lagrangian approach.

However, John's re-

sult allowed for the nonexistence of finite multipliers.

At a

1950 symposium Kuhn and Tucker (1951) presented their paper on

122

5 Computational Aspects: RHS Perturbations

nonlinear programming that extended the Lagrange Multiplier Rule to guarantee finite multipliers for the inequality constrained problem, and gave convexity conditions such that a mathematical programming problem would be equivalent to the problem of finding a saddle point of the Lagrangian.

This

extension involved a now well-known "constraint qualification," requiring once differentiability of the problem functions. Slater (1950) obtained analogous results for convex programs without requiring differentiability, his constraint qualification being an interiority assumption characterizing the feasible region.

Arrow and Hurwicz (1956) extended the Kuhn-

Tucker result to nonlinear programs that did not satisfy the convexity property. Subsequently, considerable effort was made to develop and relate various types of constraint qualifications.

For

example, Uzawa (1958) proved the Kuhn-Tucker theorem under different qualifications on the constraints.

Mangasarian and

Fromovitz (1967) developed another necessary optimality condition involving a constraint qualification that accommodates both inequality and equality constraints.

Additional impor-

tant constraint qualifications were developed by Abadie, Cottle, Gould, and, most recently, by Rockafellar.

The inter-

ested reader may begin by consulting Section 2 . 3 and one of the sources given in the next paragraph. Hestenes (1966) and Fiacco and McCormick (1968) presented a comprehensive examination of necessary and sufficient optimality conditions and associated constraint qualifications. In addition, the books by Canon, Cullum, and Polak (1970) and Mangasarian (1969) give detailed treatments of optimality conditions and constraint qualifications and, along with

123

5.3 Early Sensitivity Interpretations of Lagrange Multipliers

Bazaraa e t aZ.

( 1 9 7 2 ) a n d o t h e r s , h a v e reexamined a n d f u r t h e r

d e v e l o p e d t h e i n t e r r e l a t i o n s h i p s among v a r i o u s c o n s t r a i n t qualifications. 5.3.

EXAMPLES O F EARLY SENSITIVITY INTERPRETATIONS OF LAGRANGE MULTIPLIERS A s w i t h t h e g e n e r a l d e v e l o p m e n t a n d u e o f Lagra ge m u l t i -

p l i e r s , a n a l y t i c a l i n t e r p r e t a t i o n s o f t h e Lagrange m u l t i p l i e r s

w e r e i n i t i a l l y made f o r p r o b l e m s w i t h o n l y e q u a l i t y constraints.

One o f t h e f i r s t a f t e r Samuelson ( 1 9 4 7 ) a p p e a r s t o

be d u e t o P h i p p s ( 1 9 5 2 ) , who c o n s i d e r e d a problem e q u i v a l e n t

t o P 2 ( € ) w i t h m = 0 and p

The f u n c t i o n s a r e assumed t o

n.

be twice c o n t i n u o u s l y d i f f e r e n t i a b l e , and t h e m a t r i x o f f i r s t p a r t i a l d e r i v a t i v e s of t h e h have r a n k p a t a s o l u t i o n .

w i t h r e s p e c t t o x i s assumed t o j Phipps developed t h e u s u al f i r s t -

o r d e r necessary conditions involving t h e s t a t i o n a r i t y of t h e Lagrangian, i . e . , V,L(X,

=

0

h.(x) =

E

W,

E)

3

~

,

j = 1,

+

a set of n + p equations i n n

wj,

j = 1,

L(x, w,

E)

..., p )

= f (x)

p unknowns ( x i ,

and p parameters

(E

-

E.

+ ZPZl w j [ h . ( x ) 3

(5.3.1)

..., p ,

3

j r

I.

j = 1,

i = 1,

..., n;

..., p ) ,

where

H e used t h e suf-

f i c i e n t c o n d i t i o n of Hancock (1917) w i t h t h e t e s t d e v e l o p e d by Mann ( 1 9 4 3 ) .

Along w i t h t h e assumed l i n e a r i n d e p e n d e n c e o f t h e

g r a d i e n t s o f t h e c o n s t r a i n t s , t h i s a s s u r e s t h a t t h e Jacobian of (5.3.1) (5.3.1).

w i t h r e s p e c t t o (x, w) i s n o n s i n g u l a r a t a s o l u t i o n o f Phipps concluded t h a t t h e c o o r d i n a t e s of each sta-

t i o n a r y p o i n t and t h e a s s o c i a t e d o p t i m a l m u l t i p l i e r s , as w e l l

a s t h e " s t a t i o n a r y v a l u e " o f f , are a l l w e l l d e f i n e d f u n c t i o n s

I24

5 Computational Aspects: RHS Perturbations

of t h e p a r a m e t e r s .

Note t h a t u n d e r t h e assumed c o n d i t i o n s

t h i s r e s u l t follows r e a d i l y s i n c e t h e i m p l i c i t f u n c t i o n t h e o r e m g u a r a n t e e s t h e e x i s t e n c e of u n i q u e , o n c e c o n t i n u o u s l y d i f f e r e n t i a b l e f u n c t i o n s x i ( € ) , ~ ~ ( € 1i ,= 1,

..., n ,

j = 1,

..., p.

P h i p p s o f f e r e d a n i n t e r p r e t a t i o n of t h e Lagrange m u l t i p l i e r s a n d e x h i b i t e d t h e now w e l l known Lagrange m u l t i p l i e r s e n s i t i v i t y r e s u l t , i.e.,

af

*

/a€

*

= -wjr

j = 1,

..., p.

Wilde (1962) u s e d d i f f e r e n t i a l c a l c u l u s t o d e r i v e t h i s r e s u l t , a l o n g w i t h c o n d i t i o n s f o r a l o c a l minimum o f a g e n e r a l c l a s s of p r o b l e m s i n v o l v i n g i n e q u a l i t y c o n s t r a i n t s a n d o n c e continuously d i f f e r e n t i a b l e functions.

"Slack" v a r i a b l e s a r e

i n t r o d u c e d t o c o n v e r t t h e i n e q u a l i t i e s t o e q u a l i t i e s a n d it i s assumed t h a t t h e r e i s a n o n v a n i s h i n g J a c o b i a n o f t h e r e s u l t i n g s y s t e m w i t h r e s p e c t t o s u b s e t s o f t h e s l a c k and " d e c i s i o n " variables.

Using t h i s s t r u c t u r e a n d t h e i m p l i c i t f u n c t i o n

theorem t o i d e n t i f y f u n c t i o n a l d e p e n d e n c e between s u b s e t s o f variables, the optimal value function i s d i f f e r e n t i a t e d with r e s p e c t t o t h e "independent" v a r i a b l e s and t h e r e s u l t i n g part i a l d e r i v a t i v e s are t e r m e d " s e n s i t i v i t i e s . "

e q u i v a l e n t t o t h e Karush-Kuhn-Tucker

Conditions

necessary c o n d i t i o n s are

developed and, i n p a r t i c u l a r , a s u b s e t of t h e s e n s i t i v i t i e s i s shown t o c o r r e s p o n d t o o p t i m a l L a g r a n g e m u l t i p l i e r s .

In a

s u b s e q u e n t p a p e r ( 1 9 6 5 ) Wilde n o t e s t h a t a l t h o u g h t h i s p r o c e d u r e i s s t r a i g h f o r w a r d f o r l i n e a r c o n s t r a i n t s and s i m p l e obj e c t i v e f u n c t i o n s , i t becomes i m p r a c t i c a l i n more r e a l i s t i c situations. E v e r e t t ( 1 9 6 3 ) p r e s e n t e d a somewhat h e u r i s t i c method o f Lagrange m u l t i p l i e r s t o s o l v e c e r t a i n c l a s s e s ( p a r t i c u l a r l y t h o s e i n v o l v i n g c e r t a i n t y p e s o f s e p a r a b l e f u n c t i o n s ) o f nonl i n e a r programming p r o b l e m s a n d showed t h a t t h e L a g r a n g e

I25

5.4 Supporting Theory

multipliers can sometimes be used to bound changes in the optimal value.

The optimal multipliers are identified as the

derivatives of the optimum "payoff" with respect to each resource. Boot (1963) presented a thorough examination of sensitivity analysis for convex quadratic programming and analytically computed the many partial derivatives involved.

He developed

formulas that can be used to show that the partial derivative of the optimal value function, with respect to the components of the right-hand side (parameters) of the constraints, are optimal Lagrange multipliers. Contemporary developments are reported in Chapter 2 and are further documented in the works cited and in the bibliography. 5.4.

SUPPORTING THEORY

As noted in Section 3.4 (Chapter 3), the results for the general problem immediately yield the well known Lagrange multiplier sensitivity result and the not so well known second-order changes in the optimal value function for the problem P2(&).

Since this problem is the main concern of this

chapter, we restate the resulting specialization of Theorem 3.2.2 and Theorem 3.4.1 in full.

Conclusions (a), (b) and (d)

follow immediately from Theorem 3.2.2, conclusion (c) from Theorem 3.4.1, and conclusions (e) and (f) from Corollary 3.4.4.

Recall that the problem is

min X f(x)

s.t. gi(x) 1. E ~ , i = 1,

..., m,

I26

5 Computational Aspects: RHS Perturbations

with associated Lagrangian L(X, u, w, € 1 = f(x)

f

-

ui(gi(x)

-

Ei)

i=l

2 P

+

w.(h.(x) 3

j=l Theorem 5 . 4 . 1 .

3

(5.4.1)

E~+~).

[First-order changes in a Karush-Kuhn-

Tucker triple and first- and second-order changes in the optimal value function for the problem with rhs perturbations.]

If

(i) the functions defining P2(€) are twice continuously

*

differentiable in x in a neighborhood of x , (ii) the second-order sufficient condition for a local

*

minimum of P2(0) hold at x pliers u

*

and w

*

with associated Lagrange multi-

,

(iii) the gradients Vxgi(x

*

)

(for i such that gi(x

*

)

= 0)

and V h.(x*) (all j) are linearly independent, and X I * * (iv) ui > 0 when gi(x ) = 0, i = 1, m,

...,

then (a) x

*

is a local isolated minimizing point of P2(0) and

the associated Lagrange multipliers u* and w* are unique, (b) for

E

in a neighborhood of 0 there exists a unique,

once continuously differentiable vector function Y(E) [ x (E) , u ( E )

, w ( E)]

=

satisfying the second-order sufficient

conditions for a local minimum of problem P2(€) such that

*

y ( 0 ) = (x

,

u

*

,

and hence X(E) is a locally unique mini-

mum of P 2 ( ~ )with associated Lagrange multipliers

U(E)

and

W(E) I (c) for

E

in a neighborhood of 0, (5.4.2)

127

5.5 Karush-Kuhn-Tucker Triple and Optimal Value Function

(d) strict complementarity and linear independence of the binding constraint gradients hold at X(E) for (e)

for

E

E

near 0,

in a neighborhood of 0 the gradient of the

optimal value function is (5.4.3) (f)

for

E

in a neighborhood of 0, the Hessian of the

optimal value function is (5.4.4)

Equations (5.4.3) and (5.4.4) give us the means, at least

*

in theory, to calculate the gradient VEf

2 *

and the Hessian VEf

of the optimal value function for problem P*(E).

In practice,

estimates of the values of the Lagrange multipliers are

*

usually available and we can thus estimate VEf

.

However, we

may not generally have available a functional relationship for the Lagrange multipliers in terms of the parameters and therefore may not be able to calculate the gradient of the multipliers (the Hessian of the optimal value function) directly. However, the general procedure for obtaining the derivatives of a Karush-Kuhn-Tucker triple of P3(€) given in Chapter 4 is readily specialized to problem P 2 ( ~ )in the next section. 5.5.

FORMULAS FOR THE PARAMETER FIRST DERIVATIVES OF A KARUSH-KUHN-TUCKER TRIPLE AND SECOND DERIVATIVES OF THE OPTIMAL VALUE FUNCTION For the general parametric problem minx f(x,

E)

s.t. gi(x,

E)

2 0,

i = 1,

..., m,

128

5 Computational Aspects: RHS Perturbations

it was shown i n S e c t i o n 4 . 2 t h a t t h e n o n t r i v i a l p a r a m e t e r d e r i v a t i v e s o f a Karush-Kuhn-Tucker W(E)

triple

Y(E)

:[x(E)

,

U(E)

,

I T a r e g i v e n by

where t h e A.. a r e t h e b l o c k components o f t h e i n v e r s e o f 13

(4.2.6)

the matrix

L d e n o t e s t h e Lagrangian

(3.2.1)

o f P 3 ( ~ ) ,and t h e s u p e r b a r

d e n o t e s r e f e r e n c e t o t h e s e t o f i n d i c e s i = 1,

..., r

associ-

a t e d with t h e i n e q u a l i t y c o n s t r a i n t s t h a t a r e binding a t t h e given solution

x(E).

obtained f o r t h e Aij,

Furthermore, e x p l i c i t formulas w e r e i n t e r m s o f t h e f i r s t and second p a r t i a l

d e r i v a t i v e s o f t h e problem f u n c t i o n s w i t h r e s p e c t t o x. These r e s u l t s h o l d i n a neighborhood of

E

= 0 for the

g e n e r a l problem P 3 ( ~ ) ,and s i m p l i f y c o n s i d e r a b l y f o r problem P2(€).

I n p a r t i c u l a r , w e o b t a i n g r e a t l y s i m p l i f i e d expres-

s i o n s f o r t h e Hessian of t h e optimal value f u n c t i o n s f o r t h e various cases specified.

I g n o r i n g t h e nonbinding i n e q u a l i t y

5.5 Karush-Kuhn-Tucker Triple and Optimal Value Function

I29

constraints, the Lagrangian for problem P 2 ( ~ )is L(x, u, w, € 1 = f(x)

-

2 r

-

ui(gi(x)

Ei)

i=l

j=1

and hence

2

r VXL

=

P

VXf

j=1

i=l which is not

n explicit function of

the zero matrix.

w.V h.(x), 1 x 1 2 Therefore, VEXL is

E.

In addition, with gi

5

gi(x)

-

E i'

v,gi

-

..., 0, -1, 0, ..., 0), where the -1 is in the i-th column, hj (XI - E ~ + V~ h, i = 1, ..., r. Similarly, with h (0, ..., 0, -1, 0, ..., 0), where the -1 is in the ( j + r)-th column, j = 1, ..., p. Note that there are r + p parameters, (0,

=

the right-hand side of the binding constraints.

Therefore,

and (4.2.15) reduces to

€1 =

(5.5.1)

I

L

[Y( € 1

,E l

It is interesting to note that V€X(E) involves only A12 2 "

and VEf

(E)

(recalling (5.4.4)) involves only A22 (although

these may be interrelated, depending on the characteristics of the particular problem).

130

5 Computational Aspects: RHS Perturbations

Suppressing the rows and columns corresponding to the (zero) partial derivatives associated with the nonbinding inequality constraints, we now calculate the Hessian of the optimal value function with respect to the right-hand side of the binding constraints. dimension r

ip .

Let H(E) denote this Hessian of

From (5.4.4)

11

[T 11.

Therefore, from ( 5 . 5 . 1 ) , H(E)

=

-[:

it follows that

A22(E)

(5.5.2)

In particular, the following result is an immediate impliAll functions are evaluated at [x(E), u(E),

cation of ( 5 . 5 . 2 ) .

w(E), E] unless otherwise indicated.

For emphasis we

explicitly denote the functional dependence of the matrix AZ2 on the parameter vector

E

as A22 ( E )

.

[Positive (semi)definiteness of the

Corollary 5.5.1.

Hessian V2f* of the optimal value function.] E

tions of Theorem 5 . 4 . 1 neighborhood of

E

=

If the assump-

hold for problem P 2 ( ~ ) ,then f o r

in a

E

0,

(a) A 2 2 ( ~ )is negative semidefinite if and only if V 2 f* ( E ) E

is positive semidefinite, and (b) A 2 2 ( ~ )is negative definite if and only if H(E) is positive definite. (c) If A22(0) is negative definite and if a l l the constraints are binding at x(0) positive definite.

=

*

x

,

2 *

then V E f

(E)

= H(E) is

131

5.5 Karush-Kuhn-Tucker Triple and Optimal Value Function

Thus, i n a convex n e i g h b o r h o o d o f

E

= 0,

f

*

(E)

i s convex

if and o n l y i f A 2 2 ( ~ ) i s n e g a t i v e s e m i d e f i n i t e , and f

s t r i c t l y convex f u n c t i o n o f t h o s e components o f

E

* (E)

is a

associated

w i t h t h e c o n s t r a i n t s t h a t a r e b i n d i n g a t x ( 0 ) = x* i f A 2 2 ( 0 )

is negative d e f i n i t e . The e x p l i c i t H e s s i a n s and V € X ( E )

a r e d i s p l a y e d below f o r

t h e t h r e e ( a n d o n l y p o s s i b l e ) cases t r e a t e d p r e v i o u s l y .

Note

t h a t from ( 5 . 5 . 1 ) ,

Case 1 .

V:L-'

Exists

From ( 4 . 2 . 8 )

i t f o l l o w s t h a t A12

: hence (5.5.1)

= VxL 2 -1PTI PVxL 2 -1PT I - 1

and (5.5.2)

and

yield

and

Case 2 .

There a r e n L i n e a r l y I n d e p e n d e n t Binding C o n s t r a i n t Gradients

From ( 4 . 2 . 9 ) , (5.5.1)

A12

and (5.5.2)

v EX ( E )

= P-1 a n d A2 2 = -P-TV:LP-l;

hence

yield

= p-l[-ol

:]I

[Y( € 1 , E l

and (5.5.4)

132

S Computational Aspects: RHS Perturbations

Note that if P 2 ( € ) is a linear programming problem, then V€X(E) is constant and H(E) = 0, and if P2(€) is a quadratic programing problem, then VEx(€) is constant and H(E) is constant. Case 3 .

P

+

p

n

From (4.2.13),

where -1 T

All = TDxxT

and hence (5.5.1

and (5.5.2) yield

and

(5.5.5)

where -1

T = [palpb]

and

Dxx

=

TTVxLT. 2

An equivalent expression for the Hessian of the optimal value function in Case 1 was obtained by Buys (1972) by utilizing an augmented Lagrangian function. The following corollary follows easily and summarizes some of the implications of (5.5.2)

-

(5.5.5).

133

S . 5 Karush-Kuhn-Tucker Triple and Optimal Value Function

C o r o l l a r y 5.5.2.

2 [Relationship between AZ2 ( E ) and VxL.l

If the assumptions of Theorem 5.4.1 hold for problem P 2 ( ~ ) , then for any

E

in a neighborhood of

(a) Case 1 .

E =

0:

2

If VxL is positive definite, then AZ2 is

negative definite. (b) Case 2 .

2

The matrix VxL is positive definite (semi-

definite) if and only if A22 is negative definite (semidefinite). (c) Case 3 .

If V 2L is positive semidefinite, then A22 is X

negative semidefinite. Proof.

Conclusions (a) and (b) are immediate.

Conclusion

(c) is less obvious but may be readily proved as follows.

It

suffices to show that the matrix

[

B = L:v

-

V ~ L -1 T DT~ V2~ T LJ

is positive semidefinite.

2 Since V XL is symmetric and positive

semidefinite, it can be factored into the product of its 2 T "square root. " Let C be the square root: thus, V XL = C C and B is rewritten as B

=

CT(I

-

-1 TCT IC. CTDxxT

T Calling the matrix in brackets D, it follows that D D

=

D2 = D;

therefore D and hence also B are positive semidefinite, and it follows that AZ2(€) is negative semidefinite. In particular, the next result is an immediate implication of Corollaries 5.5.1 and 5.5.2

and subsumes (under the stipu-

*

lated conditions) the well known fact that f

(E)

problem P 2 ( ~ )is a convex programming problem.

is convex if

5 Computational Aspects: RHS Perturbations

134 C o r o l Z a r y 5.5.3.

[Local convexity of the optimal value

function f*(E) of problem P2(~).1 If the assumptions of Theorem 5.4.1 hold for problem P 2 ( ~ ) ,then for neighborhood of

E

E

in a

= 0:

L (a) H(E) is positive definite if VXL[x(O)

, ~ ( 0 ,) W(0) r

is positive definite, if Case 1 or Case 2 holds, and 2 * 2 (b) VEf ( E ) is positive semidefinite if VXL[x(€), W(E) ,

E]

01

U(E)r

is positive semidefinite.

Hence, in particular, if the conditions of Theorem 5.4.1 hold for any

E

in a convex set S, then f*(E) will be twice

differentiable in S and, if P 2 ( ~ )is a convex program, then

*

f

(E)

will be convex in

S.

We note that an alternative derivation of the Hessian H(E) is provided by the general expression (4.2.17) that we verified in Section 4.2, where we recall that

It follows immediately for P2(~) that

so that

135

5.5 Karush-Kuhn-Tucker Triple and Optimal Value Function

We first observe that this expression for H ( E ) is consistent with the formula given in (5.5.2).

To check this,

note from (5.5.1) that (5.5.6) yields

T 2 Consistency with (5.5.2) requires that A22 = -A12VxLA12. expressions (4.2.6) and ( 4 . 2 . 7 )

The

,

M =

imply, in particular, that 2 VXLAl2

+

PTA22 = 0, PAl2 = I.

Multiplying the first equation by A'i2 and using the second equation gives T 2 0 = A12VxLA12

T T + A12P A22

= A

T 2 12V xLAl2 + A22r

which yields the desired result. The results we have derived earlier in this section for H ( E ) can, of course, be readily derived from (5.5.6) as well.

In particular, the convexity of the optimal value function for the convex problem P 2 ( ~ )follows immediately from (5.5.6). Many other implications follow immediately from the various formulas that have been derived.

For example, if P2(€) is

a quadratic programming problem, then it is easy to see from the given results that Y(E) is locally affine and f* locally quadratic, under the given assumptions, and in fact closedform explicit expressions for these quantities may readily be derived.

Other structured problem classes, e.g., linear,

136

5 Computational Aspects: RHS Perturbations

geometric, and separable programs, may also be readily addressed to derive respective realizations of the sensitivity formulas from the general formulas obtained in this and the previous chapter. EXAMPLES AND APPLICATIONS

5.6.

First-order sensitivity information in the form of Lagrange multipliers for problem P2(€) is often used to estimate the value of the objective function when the constraints are changed slightly.

An

immediate application of the prior

sensitivity results for problem P2(€) is the incorporation of second-order sensitivity information to provide a better estimate of the value of the objective function when the constraints are perturbed.

The value of the objective function

in a neighborhood of

0 may be approximated by means of a

E =

truncated Taylor series including terms of the second degree, i.e. , ftX(E)I

A

f[x(O)l + Vxf[x(0)l tX(E)

-

X(0)l (5.6.1)

x

[X(E)

-

X(0)l.

Since we can measure the change in f directly as a function of E,

*

using the optimal value function f

(E)

3

f[x(~)] we obtain (5.6.2)

Recall from (5.4.3) that in a neighborhood of VEf*(E) =

[U(E)

TI

and from (5.4.4) that

-W(E)

TI ,

E

= 0

137

5.6 Examples and Applications

Therefore, for problem P 2 ( ~ )in a neighborhood of

E

=

0 we can

write

+

+ ET [ : : ]

f[X(O)l

f*(E)

(112)E ~ [ vEu(o)]~. -VEW(0)

(5.6.3)

The derivatives of the multipliers can be calculated using (5.5.1) and the Case 1

-

3 formulas for A22, once u(0) and w(0)

are known. Note that we can also estimate [x(E), u(E), ~ ( € 1 1 by a first-order Taylor'.~series at about

E =

0, since we can cal-

culate VEx(0), VEu(0), and VEw(0) once [x(O), u(O), w(0)I is known. We illustrate the usefulness of this second-order estimate, using computational results obtained from a well known test problem, the Shell Dual (Colville, 1968). E x a m p l e 5. 6.1

[The Shell Dual:

first- and second-order estimates of

.

f* ( E ) ] The Shell Primal is the minimization of a cubic objective function subject to linear constraints,

2 n

minX

ejxj

+

n

n

xicijxj

i=l j=1

j=l

2

2 2 n

aijxj 2 Ei,

i = 1, 2,

xj,O,

j=l,2

j=1

1

djxi,

j=l

n

s.t.

+

...,m,

,..., n,

for specified values of the coefficients, ej, cij, dj, aij, and rhs levels

E ~ .

138

5 Computational Aspects: RHS Perturbations

A dual of this problem, with cubic objective function and

quadratic constraints, may be written

n

m

n

n

m

1

s.t.

ajiuj = ei + 2

j=l

+

cjixj

2 3dixi,

j=l

i = l , 2

,..., n,

xi)O,

i=l, 2

,..., n,

ui)O,

j=l,2

,..., m.

In our example n = 5 and m = 10. shown in Table 5.6.1.

The problem data are

As is obvious from the notation used to

designate the problems, the right-hand sides of the nontrivial primal constraints ci (i parameters.

=

1,

..., 10) are treated here as

The values E O and c1 that are assigned to

E

in

this analysis are given in Table 5.6.2, along with A € = E ~ E-0. Table 5.6.1. i\j ej

' ij

a

i j

S h e l l P r o b l e m Data 4

5

- 36

-18

-12

-20 39 - 6 -31 32

-10 - 6 70 - 6 -10

32 -31 - 6 39 -20

-10 32 -10 -20 30

4

8

10

6

2

-16

2 - 2

- 2 2

1 0.4 0 - 4 1 0 - 1

0

- 2 - 9 0

0 0 2 0 - 2 - 4 - 3 3

2

3

-15

-27

2 3 4

5

30 -20 -10 32 -10

1

1

2 3 4

1

-

0

3.5

5

0 0

10

- 1 - 1 1 1

6 7 8 9

2

0

- 1

1

- 1

1

2

0 - 1

-

2.8

- 2 4

0 - 1 - 1 5

1

1

139

5.6 Examples and Applications

Table 5.6.2.

A l t e r n a t e Primal Right-Hand S i d e s

i

0 Ei

1

Ei

1

2

-40

-2

-40

-2

0

0

3

7

8

9

10

-1.5

-40

-60

3.5

1

-5

-1.25

-40

-60

3.5

1

1

0.25

0

0

0

0

4

5

6

-0.35

-4

-6

-0.3

-4

0

0.05

Denote the primal objective function by f(x).

The optimal are there-

value functions of the primal problem at EO and fore denoted by f" ( E0 ) and f* (E1) , respectively.

For purposes of experimentation it was decided to solve the dual, rather than the simpler primal. exercise was then to solve the dual at

E

* the (common primal-dual) optimal value f

The object of this =

E

0

,

thus yielding

o ( E ) , and thence to

obtain a first- and second-order estimate of f

*

(E

1 *

pare these results to the actual solution value f primal problem at

E

=

E

1

.

It follows that the variables u

) (E

and com1

of the

)

...,

(j = 1, 10) in the j dual, at a Karush-Kuhn-Tucker point, are the (primal) Lagrange

multipliers associated with the nontrivial primal constraints. Thus, since Theorem 5.4.1 was found to be applicable, corresponds to the gradient V E f *

U(E

0

o ) , and the partial deriva-

(E

tives of U(E0 ) with respect to the parameters give the 2 * components of the Hessian VEf

function f*

o

(E ) .

0

(E )

of the optimal value

The Hessian was obtained directly from an

estimate of these derivatives.

I40

5 Computational Aspects: RHS Perturbations

The solutions of the dual problems at E O and E~ were estimated by the SENSUMT program, applying the algorithm based on the logarithmic-quadratic penalty function W(x, will be described in the next chapter.

The first derivatives

of the u components of the solution point the dual at

E',

using (6.8.3).

with respect to

E,

r) that

E,

[X(E

0

)

,

U(E

0

)

I of

were then estimated directly

Estimates of all the quantities required

for the Taylor's series approximations of f* ( E 1) were then available. The x component of the solution obtained for the dual 0 T problem at E' was X ( E ) = (0.44, 0.532, 0.595, 0.453, 0.169) ,

yielding f*

o)

(E

=

-43.729.

The u component was given by

0

0 = 4.11, U ~ ( E ) = 2.299, U ~ ( E ) = 10.28, and the re-

0

U~(E )

maining components equaling zero.

(This corresponds to the

fact that the third, fifth, and sixth nontrivial primal constraints are binding at

X(E

0

The calculated value of

) .)

as desired. That portion H(E0 ) 2 * ( E 0) = VEu(€0) inof the optimal value function Hessian VEf "

VEf

0

(E

)

coincided with

U(E

0

volving only the parameters

)

,

E3,

[

E

~

and ,

E

~

corresponding , to

the right-hand sides of the binding primal constraints, was estimated to be au3/aE3

au6/aE3 Let E~

=

au3/aE5

au6/aE5

au3/aE6

au6/aE6

(0.05, 1.0, 0.25), i.e.,

the nonzero components of AE.

5.210

-0.725

4.382

-0.725

0.702

-0.884

4.382

-0.884

5.984

the vector composed of

1

.

141

5.6 Examples and Applications

A f i r s t - o r d e r estimate o f f f * ( E 1 ) = f * ( Eo ) + V E f 1

*

(E

*

(E

1

)

i s g i v e n by

o )Eb

+ (0.05, 1 . 0 , 0 . 2 5 ) ( 4 . 1 1 , 2 . 2 9 9 ,

= -43.729

10.28)

T

= -38.655,

a n d a s e c o n d - o r d e r estimate by

= -38.655

+

0.337

= -38.318.

1 * 1 By s o l v i n g D ( E ) i t w a s c a l c u l a t e d t h a t f ( E ) = -38.330.

In

t h i s example t h e a b s o l u t e p e r c e n t a g e error f o r t h e f i r s t - o r d e r

estimate i s 0 . 8 5 % , w h i l e for t h e s e c o n d - o r d e r e s t i m a t e t h e p e r c e n t a g e error i s 0.03%. A s noted, H(E

0

1.

the duality relation was exploited t o calculate

S u b s e q u e n t l y , Armacost

(1976a) s o l v e d t h e S h e l l Primal

d i r e c t l y , and t h e components o f t h e H e s s i a n o f t h e o p t i m a l v a l u e f u n c t i o n o b t a i n e d by means o f

(6.8.2)

a n d (6.8.31,

a g r e e d c l o s e l y w i t h t h e estimates o b t a i n e d above by means o f t h e S h e l l Dual. I n a d d r e s s i n g problem P2 ( E )

,

a t t e n t i o n was f o c u s e d on an

a n a l y s i s of t h e s e n s i t i v i t y of a s o l u t i o n corresponding t o a p a r t i c u l a r v a l u e E O of

E.

But q u i t e obviously,

i f t h e re-

quired conditions hold with respect t o a d i f f e r e n t value E,

E

of

t h e n s i m i l a r s e n s i t i v i t y i n f o r m a t i o n c o u l d be o b t a i n e d a t

.

1

E~

T h i s l e a d s t o a n e x t e n s i o n of t h e t y p e o f a p p l i c a t i o n

j u s t d i s c u s s e d , where values.

E

may b e a l l o w e d t o t a k e on a number o f

142

5 Computational Aspects: RHS Perturbations

C o n s i d e r t h e problem of f i n d i n g a s o l u t i o n t o

m in (XI€)

f (XI

s . t . g 1. ( x )

with

E

E

~

i = 1,

..., m ,

A s s u m e t h a t f o r any

i n a set S C Em+'.

d i t i o n s o f Theorem 5 . 4 . 1

,

Q

E

i n S t h e con-

apply.

One approach t o s o l v i n g problem Q i s t o t r e a t x and

E

w i t h o u t d i s t i n c t i o n and s o l v e t h e r e s u l t i n g problem i n n

+ m +

p variables.

phase approach.

Another p o s s i b i l i t y i s t o u s e a two-

F o r a g i v e n v a l u e of

E

0

= EO i n S, f i n d X ( E )

t h a t solves Q, along with t h e associated optimal m u l t i p l i e r s U(E U(E

0

)

0),

and W(E

W(E

0)

0

).

0 Then, u s i n g d e r i v a t i v e i n f o r m a t i o n a t [ X ( E 1,

I , determine

E~

t h a t d e c r e a s e s f [ x ( E0 ) I

s u b j e c t t o c1 b e i n g r e s t r i c t e d t o S.

w ( E') ] and c o n t i n u e .

1 Solve f o r [x(E )

W e can i n c o r p o r a t e second-order

f*

(E

o)

,

U(E

1

),

optimal

v a l u e s e n s i t i v i t y i n f o r m a t i o n i n t o t h i s approach. (Given an 0 0 U ( E ) , W ( E ) I , w e can a l s o o b t a i n a f i r s t estimate of [x(E'), 1 1 o r d e r e s t i m a t e of [ X ( E ) , u ( E ~ ) , w ( E ) ] and b e g i n t h e second i t e r a t i o n from t h i s a p p r o x i m a t i o n . ) I m p o r t a n t a p p l i c a t i o n s r e q u i r i n g t h e s o l u t i o n o f problems h a v i n g t h e same g e n e r a l s t r u c t u r e a s problem Q can be found. For example, G e o f f r i o n ( 1 9 7 0 ) and Silverman ( 1 9 7 2 a , b ) enc o u n t e r s u c h problems i n an approach t o s o l v i n g a c l a s s of

143

5.6 Examples and Applications

decomposable n o n l i n e a r programming problems h a v i n g t h e form k

k

s.t.

1

gRi(xR) 2 bit

i = 1,

xR

R = 1,

R=l

Xi,

E

..., m, ..., k.

The g e n e r a l a p p r o a c h , c a l l e d r e s o u r c e - d i r e c t i v e by G e o f f r i o n a n d r e s o u r c e a l l o c a t i o n by S i l v e r m a n , i s t o d e t e r m i n e t h e m-component v e c t o r s

E~

=

( E % ~ ,

...,

E

~

s u~ c h ) t h a t t h e

o p t i m a l s o l u t i o n s of t h e subproblems min

XR

fR(xR)

s . t . gLi(XR) 2 xi

where R = 1,

t-

E

L i

i = 1,

..., m,

XR'

..., k ,

ar

r e s p e c t i v e o p t i m a l compon

s o l u t i o n of t h e o r i g i n a l problem.

ts o f t h e

This usually involves t h e

s o l u t i o n of a "master" p r o b l e m k

MP

k

s.t.

*

1

E~~

L bi,

i = 1,

R=l

..., m ,

where f R ( E R ) i s d e f i n e d a s t h e o p t i m a l v a l u e o f t h e subproblem S P L ( ~ R ) . The f u n c t i o n s a n d c o n s t r a i n t s i n v o l v e d are s u c h t h a t t h e s u b p r o b l e m s a r e convex programs.

Both a u t h o r s solve t h e

master problem by u t i l i z i n g some s o r t o f a d i r e c t i o n - f i n d i n g procedure i n v o l v i n g d i r e c t i o n a l d e r i v a t i v e s o r s u b g r ad ien ts of

*

f L , and hence of t h e subproblem o p t i m a l v a l u e f u n c t i o n s . C l e a r l y S P Q ( ~ R ) ,a c c o u n t i n g f o r t h e c o n s t r a i n t s of MP, g i v e s

r i s e t o a problem h a v i n g t h e g e n e r a l form Q.

5 Computational Aspects: RHS Perturbations

144

I t i s p o s s i b l e t h a t t h e second-order

r e s u l t s developed

above can be used t o d e v e l o p a s e c o n d - o r d e r d i r e c t i o n - f i n d i n g p r o c e d u r e t o s o l v e decomposable n o n l i n e a r programs when t h e necessary hypotheses a r e s a t i s f i e d . The f o l l o w i n g s m a l l example i l l u s t r a t e s t h e two i n d i c a t e d approaches t o o b t a i n i n g a s o l u t i o n of a problem of t h e t y p e Q.

Ezarnple 5 . 6 . 2 [ D i r e c t s o l u t i o n v e r s u s s o l u t i o n by d e c o m p o s i t i o n u s i n g

sensitivity results.] min

(X,

f(x,

€1

E)

= x2

E)

= x2

-

(xl

E)

= x2

-

( x l + 1)

s.t. gl(x, g2(x, where

( E ~ , E ~ E)

C o n s i d e r t h e problem

{

S =

1

E:

-

2

E~

-

-

2

-

El

2 0,

2

-

E2

1. 0,

1)

E;

Q

1. 0 ) .

W e s o l v e t h i s f i r s t a s a problem i n f o u r v a r i a b l e s .

The

Lagrangian i s d e f i n e d a s L ( x , u,

E)

-

= x2

-

u1(x2

(x,

-

1) E;).

The f i r s t - o r d e r n e c e s s a r y o p t i m a l i t y c o n d i t i o n s t h a t ( x ,

E)

(x, + 1)2

s o l v e Q r e q u i r e t h a t t h e r e e x i s t ul, b e f e a s i b l e and s a t i s f y 2U1(Xl

-

1)

+

2u2(x1 1

-

u1

+

1) = 0 ,

-

u2 = 0 ,

u1

+

2U3E1

= 0,

u

+

2U3E2

= 0,

2

-

-

-

u2(x2

-

-

E2)

u3(1

-

€1 2

u2 1. 0 s u c h t h a t ( x ,

E)

145

5.6 Examples and Applications

u1(x2

-

(x,

112

-

El)

u2(x2

-

(x, + 1)2

-

E

-

E;)

-

u3(1

-

E:

2

= 0,

)=

0,

(5.6.4)

= 0.

The r e s u l t u1 = u2 = 0 i s i m p o s s i b l e , w h i l e u1 = 1 and u2 = 0

i s a l s o excluded s i n c e t h i s l e a d s t o g 2 ( x I u

1

= 0 and u2 = 1 l e a d s t o g l ( x ,

< 0.

Similarly,

Therefore, t h e only

< 0.

E)

E)

remaining p o s s i b i l i t y i s t h a t u1 > 0 , u2 > 0 , and u3 > 0. may be v e r i f i e d t h a t t h e s y s t e m ( 5 . 6 . 4 )

and t h e f e a s i b i l i t y

r e q u i r e m e n t s a r e u n i q u e l y s a t i s f i e d by x1 = 0 , x2 = 1 E~

= E~

= -fi/2,

u1 = u2 = 1 / 2 ,

It

and u3 = f i / 4 .

-

fi/2,

T h i s is t h e

g l o b a l minimum b e c a u s e of t h e c o n v e x i t y of problem Q i n (x,

E)

.

The o p t i m a l s o l u t i o n v a l u e i s g i v e n by f ( x ,

E)

=

1

-

n/2. W e now view problem Q a s a problem i n t h e two v a r i a b l e s x1

and x 2 , p a r a m e t e r i z e d on t h e r i g h t - h a n d straints.

s i d e of t h e con-

To emphasize t h e d i s t i n c t i o n i n t r e a t m e n t , w e

r e l a b e l t h e problem

As noted,

Q(E).

t h i s corresponds t o t h e

g e n e r a l s t r u c t u r e o f a subproblem i n v o l v e d i n t h e i n d i c a t e d approach t o s o l v i n g c e r t a i n decomposable n o n l i n e a r programs. The L a g r a n g i a n i s now d e f i n e d a s L ( x , u,

E)

= x2

-

-

u1(x2

U2(X2

-

-

(xl

2

(xl + 1)

112

-

-

E 2 b

El)

and t h e f i r s t - o r d e r n e c e s s a r y o p t i m a l i t y c o n d i t i o n s t h a t solve Q ( E ) r e q u i r e t h a t , f o r given

E

i n S, there e x i s t

X(E)

146

5 Computational Aspects: RHS Perturbations

u2 ( E ) 1. 0 s u c h t h a t

u ~ ( E ) ,

-

2u1(x1

1)

+

u1(x2

-

(xl

u2(x2

-

(xl

2u ( x 2 1

= 0

+

1) = 0 ,

-

u2 = 0,

1

-

-

1)2

-

El)

+

1)2

-

E

=

u2 = 0.

I t i s impossible f o r u

u1

u1

1

(5.6.5) = 0,

)=

2

and u2 = 1 i m p l i e s t h a t

< 0 for a l l

E

be f e a s i b l e and s a t i s f y

X(E)

i n S, i.e.,

X(E)

0.

I t may be v e r i f i e d t h a t

X(E)

= (-1, E ~ and )

i s not feasible.

u1 = 0 and u2 = 1 i m p l i e s t h a t g 2 [ x ( E ) , The o n l y r e m a i n i n g p o s s i b i l i t y i s t h a t for a l l

E

i n S.

E]

g l [ x ( ~ ) IE ]

Similarly,

< 0 for a l l

u ~ ( E )

E

i n S.

> 0 and ~ ~ ( >€ 01

I t f o l l o w s t h a t t h e unique s o l u t i o n of

(5.6.5)

c o m p a t i b l e w i t h t h e f e a s i b i l i t y r e q u i r e m e n t s i s g i v e n by =

X1(E)

€1

-

€2

,

+

-

E;

+ xl I x ( E )

U1(E)

=

2

U2(E)

=

l - X1I -

~

--

E

4 + E 1 - E

- 4

-

8

-

that

E

i s i n S.

X(E)

El

8

X(E)

where

+ ~8~~ +E 8~~~ +

+

2

16) = f*(E),

I

E2 I

The c o n v e x i t y o f Q ( E ) i n x f o r a l l

i s t h e unique g l o b a l s o l u t i o n of

Note t h a t f o r any

E

E

assures

Q(E).

i n S t h e s e c o n d - o r d e r s u f f i c i e n t con-

d i t i o n s a r e s a t i s f i e d , t h e b i n d i n g c o n s t r a i n t g r a d i e n t s Vxgl = (2

-

2x, 1) and Vxg2 = (-2

-

2x, 1) a r e l i n e a r l y i n d e p e n d e n t ,

and s t r i c t complementary s l a c k n e s s h o l d s .

S i n c e t h e problem

f u n c t i o n s a r e t w i c e c o n t i n u o u s l y d i f f e r e n t i a b l e , Theorem 5.4.1 applies.

147

5.6 Examples and Applications

S i n c e w e h a v e t h e s o l u t i o n e x p l i c i t l y i n terms o f t h e parameters, w e can c a l c u l a t e t h e p a r t i a l d e r i v a t i v e s d i r e c t l y , obtaining V x E

l

(E)

= (1/4,

-1/4), E2

-

r

VEu2 ( E ) = (-1/8, N o t e t h a t VE€*

VEf

*

E

8

+

4, = VEf*(E),

1/8).

= u ( E ) ~a ,s

W e may a l s o c a l c u l a t e

expected.

by t h e c h a i n r u l e

VEf

*

-1/4 (E)

= vxfs

E2

-

El

+

4

8

-

(El

-

E2

+ 4

2 *

N o t e a l s o t h a t VEf

(E)

E2

r

8 =

-

El

+

4)

8

V E u ( ~ ) a, s e x p e c t e d .

F o r p u r p o s e s o f i l l u s t r a t i o n and comparison w e s h a l l der i v e t h e same r e s u l t s u s i n g t h e g e n e r a l t h e o r y o f o b t a i n i n g t h e g r a d i e n t of t h e Karush-Kuhn-Tucker

t r i p l e presented i n

S e c t i o n 5.5. D i f f e r e n t i a t i n g (5.6.5) t h e J a c o b i a n m a t r i x M.

with r e s p e c t t o (x, u ) , we obtain

F o r c o n v e n i e n c e w e do n o t i n d i c a t e t h e

f u n c t i o n a l dependence on E a n d p e r f o r m t h e s u b s t i t u t i o n a t t h e

148

5 Computational Aspects: RHS Perturbations

end o f t h e c a l c u l a t i o n s . 2Ul

+

I t follows t h a t

1)

2 ( x 1 + 1)

-1

0

u1

l)Ul

-

2(x1

0

2u2

x2

-

u2-

-

(x,

-1

u2

E1

0

x2

W e n o t e from t h e s e c o n d e q u a t i o n i n (5.6.5)

0

(x,

t h a t u1

+ 1)2 - E 2 +

u 2 = 1.

w e can w r i t e 0 0 -U

0

" 0 ];.

1

-u2

where 0

-

2(x1

0

-1

-1

0

-1

0

Since t h e r e a r e t w o l i n e a r l y independent binding c o n s t r a i n t g r a d i e n t s , Case 2 a p p l i e s , and w e o b t a i n 1 -_ 4 1 -z(xl

I f we let

I-

u2 > 0 and t h e l a s t t w o e q u a t i o n s

Using a l s o t h e f a c t t h a t ul, i n (5.6.51,

-

+ 1)

1 4

+(X1

-

--18

1 8

1 8

--1 8

5.6 Examples and Applications

I49

then

-

+

1) - 2 ( x 1

1)

1

Hence,

1

0

0

1

o r (5.5.1)

(4.2.14)

yields -

-a -2(x1 1 -

+ 1)

T1( X 1

VEY(E)

1

1 4

=

evaluated a t x = x (E)

VEY(E)

+

-

El

€2

8

=

4

-

+

El

-8

1

-1 8

-

-

1 -_ 4

4

8

-8

Thus,

-1

4

.

1)

1

1 -

-

-

E2

8

1 -_ 8 These r e s u l t s c o r r e s p o n d e x a c t l y w i t h t h o s e o b t a i n e d by d i r e c t differentiation. The g r a d i e n t and H e s s i a n , o b t a i n e d d i r e c t l y o r a p p r o x i m a t e l y , can b e u s e d t o d e v e l o p a s e c o n d - o r d e r a p p r o x i m a t i o n f o r F [ x ( E ) ] a s i n (5.6.2).

To s o l v e problem Q ( E ) w e would

t h e n s e e k t o minimize f

s u b j e c t t o t h e c o n s t r a i n t s on

*

(E)

E.

I n g e n e r a l , w e would p r o c e e d i t e r a t i v e l y , as i n d i c a t e d p r e viously.

I t i s o b v i o u s t h a t f o r t h e problem Q ( E ) t r e a t e d

above, t h e second-order approximation of f f

*

(E)

i s quadratic.

*

(E)

is exact, since

Consider t h e approximation a t

E

= 0.

150

5 Computational A\pect\. RHS Perturbation\

Then

and it f o l l o w s t h a t f

*

= f

(E)

*

0 ) + u ( O ) T E + ZE 1 TVEU(0)E

+

- mlE1

c2

2

-

~

E

+

~8~~

+

E

8~~~ + 1 6 1 .

To c o m p l e t e t h e s o l u t i o n o f problem Q ( E )

, we

now r e q u i r e

t h e m i n i m i z a t i o n of f * ( E ) i n p a r a m e t e r s p a c e , where 1 E;

S p e c i f i c a l l y , w e must d e t e r m i n e

2 0.

E~

and

E~

-

E~

1

-

t h a t solve

t h e problem,

s.t. 1

-

E2

1

E

-

> 0.

E2

2 -

The L a q r a n q i a n i s g i v e n by L(E, A)

=

-(161 -

E~2

+

-

A1(l

E~2

E l2

-

~

-

E

+ ~8c1 + E

B E 2~

16)

+

EZ),

and t h e f i r s t - o r d e r n e c e s s a r y c o n d i t i o n s t h a t q u i r e t h e e x i s t e n c e o f A 1. 0 s u c h t h a t

E

E

s o l v e E re-

b e f e a s i b l e and

satisfy El

-

E2

+

4

8 E 2 - E

8

A1(l

1

+ 4

-

E;

+

AIEl

= 0,

+

A1E2

= 0,

-

EZ)

= 0.

(5.6.6)

I5 I

5.6 Examples and Application5

By symmetry, w e must h a v e

E~

= E ~ ,from

f o l l o w s t h a t t h e u n i q u e s o l u t i o n of g i v e n by

E

= (-fl/2,

-fi/2)

which it r e a d i l y

(5.6.6)

and X = f l / 2 ,

(for X

0) i s

with f*(E) = 1

-

S i n c e problem E i s convex, t h i s g i v e s t h e unique g l o b a l

fi/2.

s o l u t i o n o f p r o b l e m E, a n d o u r c o m p o s i t e s o l u t i o n i s i d e n t i c a l t o t h a t o b t a i n e d by s o l v i n g problem Q ( E ) a s a problem i n f o u r variables. I t s h o u l d b e n o t e d t h a t a t y p i c a l c h a r a c t e r i s t i c of decom-

p o s a b l e p r o b l e m s n o t p r e s e n t i n t h i s example i s t h a t t h e cons t r a i n t s on t h e r h s p a r a m e t e r s o f e a c h subproblem u s u a l l y i n v o l v e t h e r h s p a r a m e t e r s from m o r e t h a n o n e subproblem. Thus, i n a d d i t i o n t o t h e g i v e n c o n s t r a i n t , o n e must a l s o i n t r o d u c e a d d i t i o n a l c o n s t r a i n t s on t h e v a l u e s of t h e right-hand s i d e s which i n s u r e t h a t t h e p a r a m e t e r v a l u e s o b t a i n e d i n t h e s o l u t i o n o f a g i v e n subproblem a r e c o m p a t i b l e w i t h t h e requirements of a l l "coupled" subproblems.

This complication

w a s n o t i n t r o d u c e d i n t h e i l l u s t r a t i v e problem Q. i.e.,

I f it w e r e ,

i f t h e p a r a m e t e r c o n s t r a i n t s d e f i n i n g S i n c l u d e d param-

e t e r s from o t h e r s u b p r o b l e m s , t h e " c o n t i n g e n t " subproblem f e a s i b i l i t y c o n s t r a i n t s would h a v e r e q u i r e d e x p l i c i t and j o i n t consideration.

O b v i o u s l y , t h i s d i f f i c u l t y would have t o b e

s u r m o u n t e d by any a l g o r i t h m i c a p p r o a c h .

Nonetheless, t h e

a v a i l a b i l i t y of second-order s e n s i t i v i t y i n f o r m a t i o n should s t i l l o f f e r a d i s t i n c t a d v a n t a g e i n making m o r e e f f e c t i v e adj u s t m e n t s of t h e p a r a m e t e r s . Numerous a d d i t i o n a l a p p l i c a t i o n s o f t h e u s e o f s e n s i t i v i t y i n f o r m a t i o n w i l l u n d o u b t e d l y c o n t i n u e t o b e found.

F o r ex-

ample, numerous t h e o r e t i c a l r e s u l t s i n d u a l i t y t h e o r y and s t a b i l i t y a n a l y s i s have been concerned w i t h t h e f i r s t - o r d e r

152

5 Computational Aspects: RHS Perturbations

*

behavior of the optimal value function f

(E)

of P 2 ( E ) .

Compu-

tational implementations and, quite possibly, additional theoretical insights, might well be further enhanced by incorporating the full power of second-order sensitivity information. A

computational procedure (and the computer program

SENSUMT) for estimating the first- and second-order sensitivity

of the optimal value function and the first-order sensitivity of a Karush-Kuhn-Tucker triple has evolved, through several stages of refinement.

It is based on a well known class of

penalty function algorithms for solving the usual nonlinear programming problem and provides a model for demonstrating how standard NLP algorithms can effectively estimate sensitivity information in the normal course of solving a problem.

This

algorithmic approach is developed fully in the next chapter.

111 Algorithmic Approximations

This Page Intentionally Left Blank

Chapter 6

Estimates of Sensitivity Information Using Penalty Functions

6.1

INTRODUCTION

I n Chapters 3

-

5 a first-order

a n a l y s i s is given of t h e

v a r i a t i o n of a l o c a l i s o l a t e d s o l u t i o n

X(E)

of a general

p r o b l e m P 3 ( ~ ) ,a l o n g w i t h t h e a s s o c i a t e d Lagrange m u l t i p l i e r s [u(E),

w(E)],

function.

and a s e c o n d - o r d e r a n a l y s i s of t h e o p t i m a l v a l u e

*

As i n d i c a t e d , o n c e ( x

,

u

*

,w

*

= [x(O), u(O), w(0)l

i s a v a i l a b l e , a l l t h e p a r a m e t e r d e r i v a t i v e s c a n ( i n t h e o r y ) be calculated. Although t h e s e r e s u l t s a r e o f c o n s i d e r a b l e i n t e r e s t , i n p r a c t i c e i t i s d e s i r a b l e t o e x p l o i t t h e f a c t t h a t it i s poss i b l e t o e s t i m a t e s e n s i t i v i t y i n f o r m a t i o n from i n f o r m a t i o n g e n e r a t e d by a s o l u t i o n a l g o r i t h m a s a s o l u t i o n i s a p p r o a c h e d . I t seems a p p a r e n t t h a t any s o l u t i o n a l g o r i t h m must g e n e r a t e ,

o r a t l e a s t manipulate,

an abundance o f i n f o r m a t i o n t h a t i s

r e l e v a n t t o s e n s i t i v i t y and s t a b i l i t y a n a l y s i s . I t f o l l o w s t h a t t h e c l a s s o f a l g o r i t h m s b a s e d on t w i c e -

d i f f e r e n t i a b l e p e n a l t y f u n c t i o n s can r e a d i l y be adapted t o p r o v i d e e s t i m a t e s of t h e s e n s i t i v i t y information developed i n Chapters 3

-

5 , without additional assumptions.

t h e b a s i s f o r t h i s r e s u l t i n Theorem 6 . 2 . 1

We establish

i n the next section,

156

6 Sensitivity Information Using Penalty Functions

formulating an appropriate penalty function for the problem P 3 ( ~ )and absorbing the problem parameter directly into the penalty function.

It follows that the assumptions of Theorem

of Chapter 3 guarantee the existence of a trajectory of

3.2.2

unconstrained local minima of the penalty function converging

*

to x

.

For changes in

E

near

E

=

0, perturbations of this

trajectory are shown to relate closely to perturbations of x(E).

The material in this chapter is based on Fiacco (1976)

and Armacost and Fiacco (1975, 1976). Though we shall confine our analysis to a class of penalty function algorithms in this chapter, it is reiterated that any valid solution algorithm utilizes and generates analogous information that can be manipulated to provide corresponding sensitivity estimates.

This fact will be further corroborated

in the next chapter. 6.2.

APPROXIMATION OF SENSITIVITY INFORMATION USING THE LOGARITHMIC-QUADRATIC MIXED BARRIER-PENALTY FUNCTION METHOD For convenience and specificity the results will be given

in terms of the logarithmic barrier function combined with a quadratic penalty term to handle the equality constraints. For the problem P3 ( E ) , minx f(x,

E)

s.t. gi (x,

E)

2 0,

i. = 1,

h.(x,

E)

=

0,

j = 1,

3

..., m, P

P3 ( € 1

157

6.2 Logarithmic-Quadratic Mixed Barrier-Penalty Function Method

t h i s f u n c t i o n i s d e f i n e d as

2 P

+ (1/2r)

(6.2.1) 2

hj

(Xr

E ) r

j=1

where r i s a p a r a m e t e r d e f i n e d on t h e p o s i t i v e r e a l s .

The

a n a l y s i s c a n b e c a r r i e d o u t i n a s i m i l a r f a s h i o n f o r any twice-differentiable penalty function.

Numerous i n t e r e s t i n g

p r o p e r t i e s i n v o l v i n g s u c h p e n a l t y f u n c t i o n s have been documented and are w e l l known.

F o r o u r p r e s e n t p u r p o s e s , w e need

t h e f o l l o w i n g r e s u l t s , which are e x t e n s i o n s o f r e s u l t s o b t a i n e d by F i a c c o a n d McCormick (1968, Theorems 6 a n d 1 7 ) . W e r e c a l l t h a t t h e p e n a l t y f u n c t i o n a l g o r i t h m b a s e d on

(6.2.1)

c o n s i s t s e s s e n t i a l l y of approximating a local s o l u t i o n

of p r o b l e m P 3 ( ~ )by a n u n c o n s t r a i n e d l o c a l minimum W(x,

E,

X(E,

r ) of

r ) f o r r > 0 and s m a l l t h a t s t r i c t l y s a t i s f i e s t h e

i n e q u a l i t i e s of P 3 ( c ) .

Under t h e p r e s e n t a s s u m p t i o n s , t h e

v a l i d i t y o f t h e a l g o r i t h m i s e s t a b l i s h e d a s an immediate c o r o l l a r y o f t h e n e x t theorem. To a v o i d v a r i o u s t r i v i a l e x c e p t i o n s , i t i s assumed i n t h e f o l l o w i n g t h a t a t l e a s t one c o n s t r a i n t i s p r e s e n t i n P 3 ( c ) . Theorem 6.2.1.

[Approximation of f i r s t - o r d e r

r e s u l t s a n d d e t e r m i n a t i o n of estimates from W(x, 19761.

I f t h e a s s u m p t i o n s o f Theorem 3.2.2

neighborhood about

(E,

sensitivity E,

r ) (Fiacco,

hold, then i n a

r ) = ( 0 , 0) t h e r e e x i s t s a u n i q u e o n c e

158

6 Sensitivity Information Using Penalty Functions

r) , u ( E , r) ,

c o n t i n u o u s l y d i f f e r e n t i a b l e v e c t o r f u n c t i o n [x (E, r ) ] satisfying

W(E,

u, w,

E)

= 0

uigi(xr

E)

= r,

VL(X,

i = 1,

..., m , *

w i t h [ x ( O , O), u ( 0 , O), w ( 0 , O ) ] = ( x f o r any

r ) n e a r ( 0 , 0 ) and r > 0 ,

(E,

,

(6.2.2)

u

*

,w

*

1 , and such t h a t

r) is a locally

X(E,

u n i q u e u n c o n s t r a i n e d l o c a l m i n i m i z i n g p o i n t o f W(x, E, r ) , 2 w i t h g i [ x ( E r r ) , E ] > 0 ( i = 1, m ) a n d V W [ X ( E , r ) , E, rl

...,

positive definite. [x(E, r ) ,

The e x i s t e n c e o f

Proof.

U(E,

r),

W(E,

r ) l as

d e s c r i b e d f o l l o w s f r o m t h e i m p l i c i t f u n c t i o n t h e o r e m (Theorem u s i n g t h e same a r g u m e n t a s i s u s e d i n Theorem 3.2.2

2.4.1)

prove p a r t ( b ) , n o t i n g t h a t t h e J a c o b i a n m a t r i c e s of

The f a c t t h a t g i [ x ( E , gi(x

,

gi[x(E,

(6.2.2)

c o i n c i d e when r = 0 .

and ( 3 . 2 . 2 )

*

to

r),

E]

> 0 f o r a l l i such t h a t

0 ) > 0 f o l l o w s f o r a l l (E, r ) n e a r ( 0 , 0) s i n c e

r ) , E]

+

gi(x*,

0) a s

(E,

0 ) = 0 , t h e f a c t t h a t u ~ ( E r, )

t h a t gi(x*,

For i such

r) * (0, 0). +

> 0 (the latter

u;

i n e q u a l i t y d e r i v i n g from t h e a s s u m p t i o n o f s t r i c t complementary s l a c k n e s s ) a s (E, r ) * ( 0 , 0 ) i m p l i e s f r o m ( 6 . 2 . 2 ) gi[x(E, r ) , €1 = r/ui(E, Thus, g i [ x ( E ,

( 0 , 0).

r > 0 and

(E,

r ) > 0 f o r r > 0 and r),

E]

vw

..., m )

r) near provided t h a t

r ) i s s u f f i c i e n t l y close t o (0, 0 ) .

By t h e d e f i n i t i o n s ( 6 . 2 . 1 ) (3.2.1)

> 0 ( i = 1,

(E,

that

o f t h e p e n a l t y f u n c t i o n W and

o f t h e L a g r a n g i a n L, w e h a v e t h a t = Vf

-

m

1

i=l

P

(r/gi)Vgi

+

1

j=l

(hj/r)Vh j

(6.2.3)

6.2

159

ogarithmic-Quadratic Mixed Barrier-Penalty Function Method

and VL = Vf

m

1

-

2 P

+

uiVgi

(6.2.4)

wjVhj.

j=l

i=l

C o n s e q u e n t l y , p r o v i d e d o n l y t h a t gi # 0 ( a l l i ) and r # 0 ,

SO

t h a t W i s w e l l d e f i n e d , a n y s o l u t i o n (x, u , w ) o f t h e s y s t e m (6.2.2)

y i e l d s VW = VL = 0.

(x, u , w ) = [ x ( E , r ) ,

I n particular, t h i s is true for

r),

U(E,

W(E,

r )I

,

and w e c a n c o n c l u d e

( u s i n g also t h e f a c t proved i n t h e p r e v i o u s paragraph) t h a t VW[x(E, r ) , f o r r > 0 and first-order

r ] :V L [ X ( E , r ) ,

E,

(E,

r),

U(E,

W(E,

r ) , €1 = 0

r) s u f f i c i e n t l y close t o (0, 0).

necessary condition t h a t

s t r a i n e d m i n i m i z i n g p o i n t o f W(x,

E,

X(E,

Thus, t h e

r ) b e a n uncon-

r) is s a t i s f i e d f o r

(E,

r)

as indicated. I n t h e r e m a i n d e r o f t h e p r o o f assume, f o r c o n v e n i e n c e of n o t a t i o n , t h a t a l l f u n c t i o n s are e v a l u a t e d a t U(E,

r),

W(E,

r ) ] unless otherwise specified.

d e f i n i t e n e s s of VLW f o r

[x(E,

r),

The p o s i t i v e

n e a r 0 and r > 0 and s m a l l can be

E

shown a s f o l l o w s . D i f f e r e n t i a t i n g (6.2.3)

and ( 6 . 2 . 4 )

w i t h r e s p e c t t o x and

comparing y i e l d s m

P

i=l

and p r e -

j=1

a n d p o s t m u l t i p l y i n g by any v e c t o r

z T v 2wz = z

v

LZ

+

1 m

(r/g:)(vgiz)

i=l

2 P

(Vhjz) j=1

+ (l/r)

Z

gives

2 (6.2.5)

.

2,

I60

6 Sensitivity Information Using Penalty Functions

The s e c o n d - o r d e r

,

s u f f i c i e n t c o n d i t i o n s (Lemma 3.2.1)

t o g e t h e r w i t h s t r i c t complementary s l a c k n e s s , i m p l y t h a t

* , u* , w* , 0 ) z * Vgi(x , 0 ) z

T 2

z V L(x

> 0

f o r z = 0 such t h a t

= 0

for all i e B

*

(6.2.6)

- { i l gi(x*, 0) = Vh.(x*, 0 ) z = 0 , 7

j = 1,

01,

..., p .

C o n s i d e r a n y s e q u e n c e C E ~ , r k } w i t h rk > 0 a n d ( c k , r k )

+

T

2 W e s h a l l show t h a t z VW z > 0 f o r k l a r g e , w h e r e {zk} (0, 0 ) . k k k 2 i s a n a r b i t r a r y s e q u e n c e o f u n i t v e c t o r s i n En and V Wk d e n o t e s t h e Hessian of W e v a l u a t e d a t ( f k , r k ) . gi*

-z

gi(x

*

,

0),

*

*

h . ( , ~ 01, 3

h. 3

L

*

For convenience, l e t 5 L(x

*

,

u

*

,

w

*

,

0).

S e l e c t a c o n v e r g e n t s u b s e q u e n c e of { z k } , r e l a b e l it {zk} f o r c o n v e n i e n c e , and c a l l t h e l i m i t ( u n i t v e c t o r ) ?.

If

VgzZ # 0 f o r some i e B* o r i f Vh*Z # 0 f o r some j , t h e n t a k i n g 3 l i m i t s i n (6.2.5) w i t h z = z k and ( E , r ) = ( f k , r k ) and rec a l l i n g t h e a s s u m p t i o n of s t r i c t c o m p l e m e n t a r y s l a c k n e s s , T 2 *I f Vg.2 = y i e l d s t h e c o n c l u s i o n t h a t zkV Wkzk + +m a s k -+ +m. 1 0 for a l l i

E

B

*

*-

a n d Vh.z = 0 f o r j = 1, . . . , p , 7

then we again

t a k e l i m i t s o v e r t h e a p p r o p r i a t e s u b s e q u e n c e i n (6.2.5) t o conclude t h a t

T 2 T 2 -T 2 *l i m i n f k zkV W z > l i m i n f zkV Lkzk = z V L z > 0 , k k b e c a u s e of

(6.2.6)

.

T 2 T h i s shows t h a t z V Wz > 0 f o r a l l z # 0 , p r o v i d i n g r > 0 and

(f,

r) i s close t o (0, 0)

f o r a l l such

(E,

r).

,

i.e.,

VLW i s p o s i t i v e d e f i n i t e

This a l s o implies t h a t

u n i q u e l o c a l m i n i m i z i n g p o i n t o f W(x,

E,

X(E,

r) i s a

r ) f o r any

and r > 0 and s m a l l , c o m p l e t i n g t h e p r o o f .

E

near 0

161

6.2 Logarithmic-Quadratic Mixed Barrier-Penalty Function Method

) =~ [ x ( E , r ) ,

With ( x , u , w

t i o n of

(6.2.2)

M ( E f

r),

W(E,

r)lT E

are i d e n t i c a l l y s a t i s f i e d f o r

the equations (6.2.2) a neighborhood o f

U(E,

(0, 0 ) .

r) i n

(E,

E

yields

r)r

N ( E r

t h e p r e c i s e a n a l o g y of

r),

Using t h e c h a i n r u l e , d i f f e r e n t i a -

with respect t o

r ) V E Y ( E r r) =

Y(E,

(3.2.3).

(6.2.7) The q u a n t i t i e s i n t h i s

e q u a t i o n c o r r e s p o n d t e r m by t e r m w i t h t h o s e a p p e a r i n g i n (3.2.3)

,

except t h a t the p

x

p zero matrix ( t h e last p rows

and columns) t h a t a p p e a r e d i n t h e J a c o b i a n m a t r i x M ( E ) i s now r e p l a c e d i n t h e new J a c o b i a n m a t r i x M ( E , r ) by a p m a t r i x w i t h e a c h d i a g o n a l e l e m e n t e q u a l t o -r. t o (3.2.4)

and (3.2.5)

,

x

p diagonal

Corresponding

r e s p e c t i v e l y , w e have t h a t (6.2.8)

-1

*

(6.2.9)

N r

where M

*

and N

*

a r e a s d e f i n e d i n (3.2 5 ) .

S i n c e t h e s y s t e m s of e q u a t i o n s ( 3 . 2 . 2 )

and ( 6 . 2 . 2 )

when r = 0 , t h e c o n c l u s i o n s o f Theorems 3.2.2 that

0) = Y(E)

Y(E,

imply

Y(E,

0 ) = V E y ( ~ f) o r

E

sufficiently

Hence, t h e f o l l o w i n g c o r o l l a r y i s i m m e d i a t e .

c l o s e t o 0.

Corollary 6 . 2 . 2 .

[Convergence o f p e n a l t y f u n c t i o n s o l u -

t i o n s and s e n s i t i v i t y e s t i m a t e s . ] Theorem 6 . 2 . 1

r)

and 6 . 2 . 1

( r e t a i n i n g t h e n o t a t i o n w e have i n t r o d u c e d ,

f o r convenience) and V

Y(Er

coincide

hold, then f o r

I f t h e assumptions of

F near 0,

0) = y(F)

+

(6.2.10)

and VEy ( E ,

r)

+

VEy ( T , 0 ) = VEy (5)

(6.2.11)

162 as

6 Sensitivity Information Using Penalty Functions E

(0,

-+

T and r

+

0, with

In particular, for

0).

r ) c o n f i n e d to a neighborhood of

(Er

(E,

r)

(0,

-+

01,

and (6.2.13) Based on t h e s e r e s u l t s , i t i s a p p a r e n t t h a t w e c a n e s t i -

mate

Y(E)

= [x(E),

desired for

E

u(E),

w ( € ) I T a n d V E y ( ~ as ) c l o s e l y as

n e a r 0 , by Y ( E ,

r) = [x(E,

r),

U(E,

r),

T

r)]

W(E,

and V E y ( € , r ) , r e s p e c t i v e l y , p r o v i d i n g r i s s u f f i c i e n t l y close t o 0.

Having o b t a i n e d a s o l u t i o n Y ( E ,

s u f f i c i e n t l y c l o s e t o ( 0 , O),

r ) of (6.2.2)

for

(E,

r)

V E y ( € , r ) c a n b e c a l c u l a t e d from

(6.2.8). Though, i n p r i n c i p l e , any t e c h n i q u e c o u l d be u t i l i z e d t h a t y i e l d s a s o l u t i o n of

(6.2.2)

near

r ) = (0,

(E,

-

obviously formulated t h e system (6.2.2)

01, w e h a v e

which s h o u l d p r o p e r l y

be viewed a s a p e r t u r b a t i o n of t h e f i r s t - o r d e r

n e c e s s a r y con-

d i t i o n s f o r a local minimizing p o i n t of problem P 3 ( € ) t h a t i t i s s a t i s f i e d by [ x ( E ,

r),

U(E,

r),

W(E,

-

such

r ) ] for r > 0

i f a n d o n l y i f t h e l o g a r i t h m i c - q u a d r a t i c mixed b a r r i e r - p e n a l t y f u n c t i o n W(x, pliers

U(E,

E,

r ) i s minimized by

r ) and

W(E,

X(E,

r ) (with the multi-

r ) a p p r o p r i a t e l y d e f i n e d , as s p e c i f i e d ) .

Thus, t h e i n d i c a t e d a l g o r i t h m i s t h e u s u a l p e n a l t y f u n c t i o n procedure based on determining, minimizing p o i n t

rk 4 0 , k = 1, 2 ,

X(E,

for fixed

r k ) o f W(x,

..., f o r

E,

a unconstrained

E,

r k ) f o r rk > 0 a n d

x such t h a t g i ( x r

an a p p r o p r i a t e n e i g h b o r h o o d o f x ( 0 , 0 ) = x

*

E)

.

> 0 ( a l l i) i n

I63

6.2 Logarithmic-Quadratic Mixed Barrier-Penalty Function Method

Having d e t e r m i n e d

X(E,

r k ) , a n estimate of

l a r g e , t h e Lagrange m u l t i p l i e r s from t h e r e l a t i o n s

u ~ ( E r , k) =

[u ( E )

U(E,

rk) =

W.(E,

3

It follows t h a t the vector

3

r k ) ,W ( E ,

rk) = [ x ( E , r k ) , ( 6 . 2 . 2 ) and Y ( E , r k ) * [ x ( E ) ,

c a n be e s t i m a t e d

rk), €1,

rk/gi[x(E,

h . [ x ( E , r k ) , € ] / r k ( a l l i and j ) . Y(E,

, w (E) I

when k i s

X(E)

u(E),

r ) I i s a s o l u t i o n of k

'I'

w ( E ) ] as k *

rk) with respect t o

m.

The f i r s t

c a n t h e n be

p a r t i a l d e r i v a t i v e s of

Y(E,

o b t a i n e d from ( 6 . 2 . 8 1 ,

and would c o n s t i t u t e a n estimate o f

E

V € Y ( E ) ( a c c o r d i n g t o ( 6 . 2 . 1 1 ) ) , p r o v i d e d t h a t k i s l a r g e and

E

i s n e a r 0. An a l t e r n a t i v e t o u t i l i z i n g (6.2.8)

to calculate

V E y ( ~ r, ) i s a v a i l a b l e , u s i n g t h e f a c t t h a t t h e H e s s i a n of W(x,

E,

r ) is n o n s i n g u l a r a t a minimizing p o i n t

X(E,

r)

.

This

w i l l be s e e n t o have t h e advantage of i n v o l v i n g a s m a l l e r

-

matrix inverse t h e (n

+

m

i n (6.2.8)

t h a t of t h e n

+

+

x

n Hessian of W r a t h e r th an

+

p)

-

and r e q u i r e s o n l y i n f o r m a t i o n t h a t i s r e a d i l y

x

(n

m

p ) Jacobian matrix M ( E , r ) appearing

a v a i l a b l e from t h e W f u n c t i o n a t a minimizing p o i n t .

In t h i s

approach, t h e p e n a l t y f u n c t i o n i s minimized t o o b t a i n

X(E,

which d e te r min es V € X ( E , from W [ X ( E , r ) ,

E,

r),

r ) i n terms o f q u a n t i t i e s a v a i l a b l e

r ] and i t s H e s s i a n .

The m u l t i p l i e r s and

t h e i r d e r i v a t i v e s c a n t h e n be c a l c u l a t e d t o s a t i s f y ( 6 . 2 . 2 1 , w h i c h d e t e r m i n e s t h e s e q u a n t i t i e s a s f u n c t i o n s of x, The m a t r i x V € X ( E ,

E,

and r.

r) i s o b t a i n e d from t h e f o l l o w i n g r e s u l t .

To a v o i d any a m b i g u i t y , d e n o t e by X ( E , r ) t h e t r a j e c t o r y o f l o c a l minima o f W(x, Theorem 6.2.1.

X(E, r)

E,

r) whose e x i s t e n c e i s p r o v e d by

W e showed t h a t when r > 0 X(E,

r)

and

VEX(€, r)

V€X(E,

r)

(6.2.14)

I64

6 Sensitivity Information Using Penally Functions

for

The f o l l o w i n g c o r o l l a r y i s a d i r e c t

r) near (0, 0 ) .

(E,

consequence o f Theorems 3.2.2 f o r e s t i m a t i n g changes i n

-

by c o r r e s p o n d i n g c h a n g e s i n

[Approximation of

CoroZZary 6 . 2 . 3 .

VEX(^,

r) a n d

6.2.1 X(E,

X(E)

and p r o v i d e s a b a s i s

r).

X ( E r

X(E,

and 6 . 2 . 1

for

r)

by

Under t h e a s s u m p t i o n s o f Theorem

x ( T ) and V € K ( E , r)

(E,

and V € X ( E )

r ) n e a r ( 0 , 0 ) and r > 0 , i t f o l l o w s t h a t

(E,

+

VEX

r).]

X(E)

2

-

-+

r) = -V W [ X ( E , r ) ,

VEx(F) as E,

E

+

F and r

rl -1 V 2E X W [ X ( ~ , r),

0 , and

+

rl

E,

(6.2.15) S i n c e t h e system o f e q u a t i o n s (3.2.2)

Proof.

and (6.2.2)

c o i n c i d e when r = 0 , it f o l l o w s from t h e c o n c l u s i o n s o f Theorems 3 . 2 . 2

X(E,

r)

t h a t , f o r r > 0 and

and 6 . 2 . 1

(E,

r ) n e a r (0, 01,

x ( E , 0) = x ( F )

-+

(6.2.16)

and V€X(E,

as

E -+

r)

VEx(E, 0 ) = VEx(EI)

-+

h and r

-+

0 (where c o n v e r g e n c e i s component by compoFrom t h e f a c t t h a t VW[X(E, r ) ,

nent, i n a l l cases).

rl = 0

E,

a n d K ( E , r ) i s o n c e c o n t i n u o u s l y d i f f e r e n t i a b l e f o r any 0 a n d r > 0 and s m a l l , v~w[x(E,

r),

E,

it follows t h a t

E,

r1

= 0,

2 a n d s i n c e w e showed t h a t ' V W i s p o s i t i v e d e f i n i t e f o r

(E,

n e a r ( 0 , 0 ) and r > 0 , w e o b t a i n t h e e x p r e s s i o n g i v e n i n f o r VEST(€,

near

~ I V ~ X ( E r, )

+ V ; ~ W [ X ( Er, ) ,

(6.2.15)

E

r).

r)

165

6.2 Loearithmic-Quadratic Mixed Barrier-Penalty Function Method

T h i s r e s u l t gives t h e b a s i s f o r a p p r o x i m a t i n g u ( T ) a n d

w ( F ) as w e l l .

With

-u ~ ( E r, )

r ) near ( 0 , 0 ) and r > 0 , d e f i n i n g

(E,

r), €1,

r/gi[X(E,

i = 1,

...

m,

(6.2.17)

. . . I

p r

(6.2.18)

r

and

W.

3

h . [ S t ( E , r ) , ~ l / r , j = 1, 3

r)

(E,

since VL[P(E, r ) ,

-

U(E,

= VW[T(E, r ) ,

it f o l l o w s t h a t s o l u t i o n of

Y(E,

r),

-

rl = 0

E,

[X(E, r ) ,

r)

(6.2.2).

r ) , €1

W(Er

-

U(E,

T h e r e f o r e , T h e o r e m s 3.2.2

imply that

-

u ~ ( E ,r )

+

W.

+

iii(Er 0 )

= ui(E),

as

E

-+

(E,

r)

E and r

ij

-+

( T , 0) =

j

...

r

m,

(6.2.19)

3

t h e s-th component o f

(6.2.17) E

j = 1,

...

p,

(6.2.20)

r

and ( 6 . 2 . 1 8 )

w i t h respect t o

yields

r)/aEs

= ( - r / g i )2 ( v' g i [ a S i l ( E r

+

(3,

0.

D i f f e r e n t i a t i o n of

aiii(E,

W.

a n d 6.2.1

i = 1,

and 1

r ) , W ( E , r ) l T is a

i = 1,

agi/aEs),

...

r)/aEsr

s = 1,

r

aFn(E,

r)/aes1 T

..., m; ..., k,

(6.2.21)

and

aw.7 ( E ,

= (l/r)(vhj[aF,(E,

+

ah./ass) 3

r

j = 1,

s

..., axn(€,

r)/acsr

= 1,

...

r

p;

..., k,

r)/aEsl

T

(6.2.22)

I66

6 Sensitivity Information Using Penalty Functions

where t h e f u n c t i o n s are a l l e v a l u a t e d a t [ Y ( E , r ) , € 1 .

The

p r e v i o u s r e s u l t s a l s o i m p l y t h a t t h e matrices whose components a r e g i v e n r e s p e c t i v e l y by (6.2.21)

and (6.2.22)

converge

component-wise as f o l l o w s :

v ~ U ( E ,r )

.+

(6.2.23)

V E u ( F r 0 ) = VEu(F)

(6.2.24) as

E

7 and r

+

0.

+

C o n s e q u e n t l y , w e c a n estimate Y ( E ) a n d V E y ( € ) by and V € Y ( E ,

r > 0.

r) f o r

(E,

F o r any s u c h

Y(E, r)

r ) s u f f i c i e n t l y close t o ( 0 , 0 ) a n d (E,

r ) , t h e s e q u a n t i t i e s can a l l be c a l -

c u l a t e d o n c e a l o c a l u n c o n s t r a i n e d m i n i m i z i n g p o i n t X ( E , r ) of W(x,

E,

r ) h a s been determined ( i n t h e r e g i o n such t h a t

gi[x(E, r ) , containing

E]

> 0 ( i = 1,

..., m ) ) i n

a s u i t a b l e open s e t

0).

X(E,

Returning t o t h e problem P 3 ( 0 ) , it f o l l o w s t h a t t h e usual p e n a l t y f u n c t i o n approach u t i l i z i n g W(x, 0 , r ) t o f i n d a l o c a l s o l u t i o n o f t h i s problem can be used t o approximate y ( 0 ) =

*

(x

,

u

*

w * ) a~n d V E y ( 0 ) .

A minimizing sequence

{x(O, r k ) }

of

{W(x, 0 , r k ) Ic o n v e r g i n g t o x* i s g u a r a n t e e d by Theorem 6 . 2 . 1 f o r rk > 0 and small. The p o i n t F ( 0 , r k ) may b e c o n s i d e r e d a n

estimate of x

*

.

The q u a n t i t i e s i n v o l v e d i n t h e r i g h t - h a n d

s i d e of ( 6 . 2 . 1 5 ) c a n be e v a l u a t e d a t E = 0 , r = r k r o n c e x ( 0 , r ) has been determined, y i e l d i n g t h e e s t i m a t e

k

v Ex ( 0 ) a t [?(Or

A

v E~

( 0 ,r k ) =

r k ) , O r rkl.

2 -v 2w-1 vEXw

(6.2.25)

167

6.2 Logarithmic-Quadratic Mixed Barrier-Penalty Function Method

The a s s o c i a t e d L a g r a n g e m u l t i p l i e r s f i r s t partial derivatives a t U(O) W(O)

v EU ( O )

f

E

[ u ( E ) , w ( E ) I and t h e i r

= 0 c a n t h e n be e s t i m a t e d from

U(O, r k ) ,

(6.2.26)

~ ( 0 ,r k ) ,

(6.2.27)

A vEii(O,

rk),

(6.2.28)

(6.2.29) where t h e r e s p e c t i v e q u a n t i t i e s o n t h e r i g h t - h a n d o b t a i n e d from t h e r e l a t i o n s ( 6 . 2 . 1 7 ) , (6.2.22)

evaluated a t

E

6.2.18),

s i d e are

6.2.211,

and

= 0 , r = r k'

It i s important t o note,

from t h e p o i n t o f v i e w o f compu-

t h a t an estimate o f t h e H e s s i a n 2 matrix of t h e penalty function V W t h a t i s involved i n calcut a t i o n a l implementation,

l a t i n g t h e estimate VEW(O,

r k ) o f V E x ( 0 ) i n (6.2.251,

wi22

a l r e a d y b e a v a i l a b Z e i f o n e of many v a r i a n t s o f Newton's method

i s u s e d t o compute t h e u n c o n s t r a i n e d m i n i m i z i n g p o i n t X ( 0 , r k ) o f W(x, 0 , r k ) . Thus, u t i l i z i n g any o f t h e s e w e l l known proc e d u r e s f o r t h e u n c o n s t r a i n e d m i n i m i z a t i o n s , much o f t h e i n f o r m a t i o n r e q u i r e d t o c a l c u l a t e t h e s e n s i t i v i t y i n f o r m a t i o n by t h i s t e c h n i q u e w i l l h a v e a l r e a d y been g e n e r a t e d i n implementing t h e usual penalty function algorithm. V'

W appearing i n (6.2.25)

EX

(Of c o u r s e , t h e o t h e r t e r m

i s a l s o known o n c e t h e problem i s

s p e c i f i e d and n e e d o n l y b e e v a l u a t e d a t [ W ( O ,

rk), 0 , r k l . )

The a n a l o g o u s o b s e r v a t i o n h o l d s f o r t h e c a l c u l a t i o n o f t h e f i r s t p a r t i a l d e r i v a t i v e s o f t h e problem f u n c t i o n s a t [Z(O,

r k ) , 01 w i t h r e s p e c t t o

E.

L e t y(0, r k )

f [ z ( O , r k ) , 01.

Using t h e c h a i n r u l e , w e o b t a i n z

v E f ( 0 , r k ) = VXfVEX + V E f ,

(6.2.30)

168

6 Sensitivity Information Using Penalty Functions

where t h e f u n c t i o n s are e v a l u a t e d a t ( x ,

r ) = lTI(0, r k ) , 0 ,

E,

The vector of p a r t i a l s V f w i l l b e known o n c e t h e prob-

rkl.

E

l e m i s s p e c i f i e d a n d need o n l y be e v a l u a t e d .

The g r a d i e n t Vxf

w i l l normally have a l r e a d y been c a l c u l a t e d , i n a p p l y i n g t h e

usual penalty function algorithm.

Thus, t h e work r e q u i r e d t o

o b t a i n t h e estimate

v Ef*(O)

VXfVETt + VEf,

where a l l f u n c t i o n s o n t h e r i g h t - h a n d s i d e a r e e v a l u a t e d as i n (6.2.30)

,

and h

t h e d e r i v a t i v e s o f which a p p e a r i n ( 6 . 2 . 2 1 ) a n d

j'

is c o n s i d e r a b l y reduced.

The same a p p l i e s t o t h e gi

(6.2.22).

EXAMPLES OF ESTIMATES O F SOLUTION POINT AND LAGRANGE MULTIPLIER PARAMETER DERIVATIVES

6.3.

W e use t h e examples o f S e c t i o n 3 . 3

(Chapter 3) to i l l u s -

t r a t e the application of these algorithmic results. reader is r e f e r r e d t o Section 3 . 3 f o r t h e values of VEY

(E)

The and

Y(E)

.

E x a m p l e 3. 3 . 1

s.t. x

min X x

x

E,

E

E

1

The l o g a r i t h m i c b a r r i e r f u n c t i o n f o r t h i s p r o b l e m i s W(x,

E,

r)

x

any r > 0 by X(E,

r)

+

E

-

r ln(x

X(E,

=

X(E)

r) =

-

+ r.

E

as r

E),

-+

u n i q u e l y minimized f o r x > Hence, f o r any v a l u e o f

0 and V € X ( E ,

U(E,

r ) :r / g [ x ( E , r ) ] , w i t h g [ x ( E , r )

have t h a t a l l r.

U(E,

r)

=

1=

U(E)

and V € U ( E

E

r)

r)

illus-

since

Also,

X(E,

and

E,

r) = 1 = V€X(E),

t r a t i n g t h e c o n c l u s i o n s of C o r o l l a r y 6 2 . 3 .

E

-

E

= r, we

= 0 = V U(E)

for

I69

6.3 Solution Point and Laprange Multiplier Parameter Derivatives

F o r t h i s example, t h e s o l u t i o n d i f f e r e n t i a b l e f o r any v a l u e o f

E,

X(E)

o f P 3 ( ~ )i s u n i q u e and

and i t i s n o t e d t h a t t h e

c o r r e s p o n d i n g estimates u s i n g t h e p e n a l t y f u n c t i o n a r e v a l i d f o r any

T h i s m i g h t h a v e been a n t i c i p a t e d by Theorems 3 . 2 . 2

E.

and 6 . 2 . 1 and C o r o l l a r y 6 . 2 . 3 ,

are s a t i s f i e d f o r any

s i n c e t h e r e q u i r e d assumptions

E.

Examvle 3.3.2

min

x x1

+

€lX2 g2(x,

where

> 0 and

E~

E~

E

- -xl

0,

# 0.

To estimate t h e s o l u t i o n and s e n s i t i v i t i e s , w e u s e t h e

logarithmic b a r r i e r function (6.2.1), W(x,

E,

As u s u a l ,

r)

+

x

E1x2

-

r 1n(x2

which f o r o u r example is

-

E2x1) 2 2

-

r ln(-xl).

r > 0 and t h e f u n c t i o n i s minimized o v e r t h e set o f

p o i n t s s a t i s f y i n g t h e c o n s t r a i n t s w i t h s t r i c t i n e q u a l i t y as

r * 0.

From t h e r e q u i r e m e n t o f s t a t i o n a r i t y , VW(x,

w e f i n d t h a t W(x,

E,

r)

E,

r ) i s minimized u n i q u e l y ( o v e r t h e i n d i -

c a t e d r e g i o n ) by

4

f o r any r > 0 .

r) =

4

As e x p e c t e d from t h e t h e o r y ,

€;XI

x ~ ( E ,

r)

+ +

~

2

= 0,

- 1 / 2 ~= ~~ ~~ ~( and € 1 x ~ ( E r, )

2 2

* 1 / 4 ~ ~= 6~ ~ ( as € 1 r * 0.

170

6 Sensitivity Information Using Penalty Functions

The Lagrange m u l t i p l i e r s a s s o c i a t e d w i t h

X(E,

r ) are given

and

Calculating f i r s t derivatives with respect t o

E

-1/2

a x 1 ( € , r ) - -16~~~:(1 + 8 ~ ~ ~ : r )r + 4 ~ ;( 1 + 8 2 4 aEl 1 6 ~ ~

we obtain 2

1/2

~

~

~

~

~

r

)

~

and ax1(€, r ) 3E.2

-1/2

-

- 3 21 ~2 ~ +~ 8 ~ ~ ( ~1 ~ ; rr )+

ax1(€,

r)/aE2

+ ~

/

+

1

E

2 4

1

I t f o l l o w s t h a t a x 1 ( € , ‘)/as1

~

6

I+ 1

~

(1+E 8

~

~

/ 2 22 =~ a x~l ( E~) / a E ~l

E3 ~= E ax1(E)/aE2, ~

as r

+

~

2 )1/2]

~

~

~

~

and

0.

One c a n a l s o v e r i f y t h a t

and ax2(€, r) aE2

-

ax1(€, r )

--2~~

aE2

Taking l i m i t s y i e l d s a x 2 ( € , ax2(€,

r)/aE2

Thus, V € X ( E ,

+

r)

/

r)/aEl

+

2 3 2 ~= ~a x ~ 2 ( E~ ) / a E 2 as r

-

1

+

V€X(E)

as r

c o n c l u d e d from t h e t h e o r y .

+

3 2

- 1 / 2 ~= ~a x~2 ( E ~ ) / a E 2 and -t

0 , as d e s i r e d .

0 , component by component, a s

~

~

171

6.4 Extensions

F i n a l l y , from t h e p r e v i o u s c a l c u l a t i o n o f

r ) , w e have

U(E,

that

v E u ( ~ r, ) =

as r

+

[ ]

-+

1

0

0

0

= VEU(E)

0 , and t h e f i r s t - o r d e r r e s u l t s a r e complete.

Note t h a t , a s i n t h e p r e v i o u s example, t h e r e s u l t s a r e v a l i d f o r a s e t of v a l u e s o f

In t h i s case t h e conditions

E.

i m p l y i n g e x i s t e n c e and convergence o f t h e q u a n t i t i e s o b t a i n e d above a r e s a t i s f i e d a s l o n g as

E~

0 and

E

~

# E 0.

~

Thus, f o r

b o t h examples t h e r e s u l t s go w e l l beyond t h e c a l c u l a t i o n and e s t i m a t i o n of s e n s i t i v i t y information a t a p o i n t ( f o r a given value of

E)

,

and e s s e n t i a l l y p r o v i d e a " p a r a m e t r i c a n a l y s i s . "

The i n d i c a t e d f u n c t i o n s e x i s t f o r a l a r g e s e t o f v a l u e s o f

E

and t h e f u n c t i o n s depending on t h e p e n a l t y p a r a m e t e r r converge point-wise i n E t o t h e i r indicated l i m i t s a s r

+

0.

EXTENSIONS

6.4.

These examples s u g g e s t t h a t t h e t h e o r e t i c a l r e s u l t s o b t a i n e d h e r e can be e x t e n d e d , under a p p r o p r i a t e c o n d i t i o n s , t o allow f o r a parametric analysis (i.e., ysis i n the large).

a s e n s i t i v i t y anal-

Such would a p p e a r t o f o l l o w , n o t i n g t h a t

t h e p r o o f s of t h e main r e s u l t s h i n g e p r i m a r i l y on t h e i m p l i c i t f u n c t i o n theorem a p p l i e d t o t h e Karush-Kuhn-Tucker

c o n d i t i o n s of

problem P3(0) and on an a p p r o p r i a t e p e r t u r b a t i o n o f t h o s e conditions. o f any

E

The r e s p e c t i v e r e s u l t s a r e v a l i d i n a neighborhood or

(E,

r ) f o r which t h e c o n d i t i o n s ( s a t i s f y i n g t h e

h y p o t h e s e s o f t h e i m p l i c i t f u n c t i o n theorem) a r e assumed t o

172

6 Senqitivity Informalion Using Penalty Functions

hold.

I t i s c l e a r t h a t w e need o n l y i n v o k e t h e a n a l o g o u s

conditions f o r every

or

E

r ) i n a p p r o p r i a t e sets i n which

(E,

t h e s e parameters are allowed t o vary, t o e s t a b l i s h t h e v a l i d i t y o f t h e c o n c l u s i o n s i n t h e s e sets. The f o l l o w i n g c o r o l l a r y o f Theorem 6 . 2 . 1 i s a l s o , a n a l o g o u s l y , an immediate consequence o f t h e same e x t e n s i o n s o f t h e c l a s s i c a l i m p l i c i t f u n c t i o n theorem and t h e f a c t t h a t t h e s y s t e m s of e q u a t i o n s ( 3 . 2 . 2 )

and ( 6 . 2 . 2 )

c o i n c i d e when r = 0 .

The f i r s t p a r t i s a n e x t e n s i o n o f Theorem 2 . 4 . 1 0

and p r e c i s e l y

p a r a l l e l s C o r o l l a r y 3.2.5. [ E x i s t e n c e and convergence o f h i g h e r -

Coro l l a r y 6 . 4 . I.

r ) .]

order d e r i v a t i v e s of Y ( E , l a r y 3.2.5

hold, then

y ( ~ r, ) : [ X ( E ,

r),

i n a neighborhood o f

(E,

r),

U(E,

w(E,

r) = (0, 0 ) .

t i o n s a r e a n a l y t i c i n (x,

r ) l T E CP I f t h e problem func-

i n a neighborhood of

E)

r ) i s a n a l y t i c i n a neighborhood of

then Y ( E ,

Furthermore,

Y(E,

y with respect t o

r) E

at

(E,

...,

Proof.

(E,

(x*, O ) ,

r)

=

(0, 0 ) .

and t h e j - t h p a r t i a l d e r i v a t i v e s of

+ Y(E)

r ) converge t o t h e c o r r e s p o n d i n g

j-th partial derivatives a t j = 1,

I f t h e a s s u m p t i o n s of Corol-

E

as r

+

0, for

E

n e a r 0 , where

p. The e x i s t e n c e and c o n t i n u i t y o f t h e h i g h e r - o r d e r

p a r t i a l d e r i v a t i v e s and a n a l y t i c i t y r e s u l t from t h e i m p l i c i t f u n c t i o n theorems,

Theorems 2 . 4 . 1

system of e q u a t i o n s ( 6 . 2 . 2 )

and 2 . 4 . 2 ,

applied t o the

as i n t h e p r o o f o f Theorem 6 . 2 . 1 .

From t h i s and t h e f a c t t h a t ( 3 . 2 . 2 ) and ( 6 . 2 . 2 )

r = 0 , w e conclude t h a t Y(Er

r)

+.

Y(E,

0) = Y ( E )

c o i n c i d e when

173

6.5 Perturbed Karush-Kuhn-Tucker System

and t h e p a r t i a l d e r i v a t i v e s a t

(E,

r ) also converge t o t h e

corresponding p a r t i a l d e r i v a t i v e s a t hood o f

E

=

as r

E

+

0, i n a neighbor-

0.

These r e s u l t s a r e n o t p u r s u e d f u r t h e r i n t h i s book. could have a p p l i c a t i o n , e.g.,

i n providing a b a s i s f o r esti-

mating t h e para m e t r i c v e c t o r f u n c t i o n component o f mate E~(E).

X(E)

They

by d e v e l o p i n g e a c h

X(E)

a s a power series i n

E,

y i e l d i n g t h e esti-

This i s p r e c i s e l y analogous t o t h e e x t r a p o l a t i o n

t h e o r y developed f o r e s t i m a t i n g t h e c o u r s e of a minimizing t r a j e c t o r y i n t e r m s of t h e involved parameter i n p en alty f u n c t i o n methodology ( F i a c c o and McCormick, 1 9 6 8 ) . X(E,

r ) c o u l d b e d e v e l o p e d i n a power series

Similarly,

r) i n

;(E,

(E,

F o r r n e a r 0 , G ( E , r ) may b e an a d e q u a t e e s t i m a t e o f G ( E ) . e s t i m a t e G(E,

r). To

r ) , one could u t i l i z e t h e p e n a l t y f u n c t i o n

method b a s e d on W(x, s a t i s f y i n g (6.2.2)

E,

r ) t o o b t a i n s e v e r a l v a l u e s of

X(E,

r)

for r > 0 and f i x e d , corresponding to

s e v e r a l v a l u e s o f E i n a s u i t a b l e domain. 6.5.

SENSITIVITY CALCULATIONS BASED ON THE PERTURBED KARUSH-KUHN-TUCKER SYSTEM W e now t u r n b r i e f l y t o t h e p r o b l e m o f o b t a i n i n g a d i r e c t

s o l u t i o n o f t h e system V E y ( ~ r, ) =

M(E,

r)-'N(E,

(6.2.8)

r),

which r e s u l t s from d i f f e r e n t i a t i o n w i t h r e s p e c t t o p e r t u r b e d Karush-Kuhn-Tucker

s y s t e m (6.2.2)

at Y(E,

E

of t h e

r ) , and

show how t h i s r e l a t e s t o t h e s o l u t i o n o b t a i n e d i n S e c t i o n 6 . 4 by way o f t h e p e n a l t y f u n c t i o n minimum

X(E,

r).

Again, assume

174

6 Sensitivity Information Using Penalty Functions

t h a t t h e c o q d i t i o n s of Theorem 3.2.2

6.2.1

guarantees t h a t f o r

(E,

hold.

As shown, Theorem

r ) n e a r ( 0 , O),

Y(E,

r ) i s i n C1

and s a t i s f i e s t h e f o l l o w i n g s y s t e m : VXL(X,

u, w,

uigi(x, h . (x, 3

E)

= 0

E)

=

r,

i = 1,

..., m ,

E)

=

w.r,

j = 1,

..., p .

3

L e t U = d i a g ( u i ) and G = d i a g ( g . ) . 1

(6.2.2)

Then, t h e J a c o b i a n of

(6.2.2) t a k e n w i t h r e s p e c t t o ( x , u, w ) i s g i v e n by

M(E,

r)

=

uivxgi

gi

0

0

-r

L

(6.5.1)

Then, d e f i n i n g T

-vx9

r)

-U-lG

0

we obtain

M(E,

r) E

[

0

6

-U

0

0

1

vtl

(6.5.2)

-rI

(6.5.3)

I75

6.5 Perturbed Karush-Kuhn-Tucker System

A s b e f o r e , w e s h a l l see t h a t w e n e e d o n l y c a l c u l a t e

G(E,

r)

gi[x(E,

1,

.

-1

r ) , € 1 > 0 , and u i ( € , r )

[r :I,

..., m ) . Q E

R e c a l l t h a t Theorem 6 . 2 . 1

provides t h a t r > 0,

r ) , € 1 (i = -1 rU and d e f i n e

= r/gi[x(E,

T h e r e f o r e , w e can w r i t e G = where U2 = UU.

Then (6.5.4) e x i s t s and

(6.5.5)

where P = (-Vxg T , VxhTjT.

Letting

+

i t i s e a s i l y shown t h a t i f (V:L All

= (V:L

A12

=

T

+

-1

(l/r)PTQP)

= (l/r)AllP

T

(l/r)PTQP)-’

exists,

,

Q,

(6.5.6)

2 A 2 2 = - ( l / r ) Q+ ( l / r )QPAllPTQ. f o r r > 0 and (E, r ) n e a r

To show t h e e x i s t e n c e o f All

(0, 0) w e now make u s e of t h e p e n a l t y f u n c t i o n W(x, t h e r e s u l t s o f Theorem 6.2.1. Lagrangian L ( x , u, w ,

E)

E,

r ) and

Recalling t h e d e f i n i t i o n of t h e

(3.2.1)

and t h e p e n a l t y f u n c t i o n

I76 W(x,

6 Sensitivity Information Using Penalty Functions E,

r ) (6.2.11,

it may e a s i l y be v e r i f i e d by d i r e c t

calculations that, a t

2

X(E,

r),

m

V W = VX 2L + X 2

(r/g:)VxgTVxgi

+ (l/r)

i=1

using (6.2.17) VxW 2 = VxL 2

P

VxhTVxhj

j=l

and h e n c e

+

(l/r)PTQP,

where P and Q a r e d e f i n e d above. 2

t h a t V W is p o s i t i v e d e f i n i t e f o r X

and h e n c e All

2

= V;W-'

I t f o l l o w s f r o m Theorem 6 . 2 . 1 (E,

r ) n e a r ( 0 , 0 ) and r > 0

e x i s t s , as d e s i r e d .

F o l l o w i n g t h e same p r o c e d u r e as b e f o r e f o r M ( E ) ,

letting

so t h a t

w e have (6.5.7)

177

6.5 Perturbed Karush-Kuhn-Tucker System

L e t N ( E , r) :

(-VExL 2 T , -(UVEg)T , -VEh

Jacobian of t h e system (6.2.2) VEY(Er

b e t h e n e g a t i v e of t h e

taken with respect t o

E.

Then

r ) = M ( E , = ) - I N ( €r ,)

A

and h e n c e , w i t h N ( E ,

r)

= I-VExL 2 T

2 -A21VExL + A 2 2

E q u a t i o n (6.5.8)

,

VEgT, -VEhTIT, w e o b t a i n

(-;I:)

i s a d i r e c t analog of

(4.2.15).

S i n c e w e h a v e g i v e n o s t e n s i b l y t w o methods f o r c a l c u l a t i n g t h e u n i q u e p a r t i a l d e r i v a t i v e s V E y ( € , r ) , o n e by (6.5.8) a n d a s e c o n d by ( 6 . 2 . 1 5 ) ,

(6.2.211,

and ( 6 . 2 . 2 2 )

,

it i s

n a t u r a l t o i n q u i r e which p r o c e d u r e m i g h t be p r e f e r a b l e a n d how t h e c a l c u l a t i o n s are r e l a t e d .

I t t u r n s o u t t h a t t h e pro-

c e d u r e s are e s s e n t i a l l y e q u i v a l e n t :

w e n e e d o n l y compute All

t o solve ( 6 . 5 . 6 ) ,

t h e o t h e r components A i j

of

A(E,

S i n c e All

and h e n c e (6.5.81,

r)-’ b e i n g a f u n c t i o n o f All = V;W-’,

and known q u a n t i t i e s .

t h i s i s seen to be proportionate to t h e

e f f o r t n e e d e d t o s o l v e t h e s y s t e m g i v e n by (6.2.151, and (6.2.22).

(6.2.211,

I n f a c t , t h e two p r o c e d u r e s c a n be p l a c e d i n 2 p r e c i s e c o r r e s p o n d e n c e by e x p r e s s i n g VxW a n d VxW a s f u n c t i o n s

178

6 Sensitivity Information Using Penalty Functions

2 of VxL and VxL,

respectively, a t

X(E,

r).

I n particular, note

that

m

V E2X W =

v 2E XL +

D

1

(r/gf)VxgTVEgi i=l

+ (l/r)

$

VxhTVEhj

j=1

using previously defined notation.

From ( 6 . 2 . 1 5 )

and t h e

above r e s u l t s , w e h a v e t h a t V ~ X ( E ,

r)

2 -1 2

= -V XW

=-A

VEXW

V 2E XL +

11

and t h e r e f o r e V€X(E,

r ) = -A

V2 L

11

EX

+

A12( -VEh vE9)

where A12 i s g i v e n by ( 6 . 5 6 ) .

This r e s u l t coincides with

t h a t o b t a i n e d i n t h e upper p a r t of

(6.5.8).

Using t h e e x p r e s -

s i o n f o r VEx(€, r ) a n d t h e f o r m u l a s g i v e n f o r V E u ( € , r ) a n d VEw(c, r ) i n (6.2.21)

a n d (6.2.22), w e a l s o o b t a i n , t e r m by

t e r m , t h e bottom p a r t of

(6.5.8)

,

a f t e r regrouping t e r m s i n a

s t r a i g h t f o r w a r d manner. Thus , t h e p r o c e d u r e s b a s e d o n ( 6 . 5 . 8 )

(6.2.21), a n d ( 6 . 2 . 2 2 ) ,

, or

(6.2.15)

,

a r e s e e n t o be v i r t u a l l y i d e n t i c a l ,

t h e only d i f f e r e n c e being whether t h e d a t a i s o r g an ized i n

179

6.6 Optimal Value Function Sensitivity Estimates

terms o f t h e L a g r a n g i a n o r t h e p e n a l t y f u n c t i o n , r e s p e c t i v e l y . N o t e t h a t any t e c h n i q u e f o r o b t a i n i n g s o l u t i o n s o f

c o u l d be u s e d .

However, s i n c e t h e p e r t u r b e d Karush-Kuhn-

Tucker system (6.2.2) first-order

(6.2.2)

is simply another r e p r e s e n t a t i o n of t h e

n e c e s s a r y c o n d i t i o n s t h a t hold a t an unconstrained

minimum o f t h e l o g a r i t h m i c - q u a d r a t i c l o s s f u n c t i o n W(x, (6.2.1),

t h e o r g a n i z a t i o n o f d a t a b a s e d o n (6.2.151,

and (6.2.22)

r)

(6.2.211,

would p r o b a b l y b e more e f f i c i e n t if a n a l g o r i t h m

based on W w e r e used t o o b t a i n t h e s o l u t i o n o f 6.6.

E,

(6.2.2).

OPTIMAL VALUE FUNCTION SENSITIVITY ESTIMATES C o n s i d e r t h e c l a s s o f a l g o r i t h m s b a s e d on t w i c e c o n t i n u -

o u s l y d i f f e r e n t i a b l e p e n a l t y f u n c t i o n s and, as b e f o r e , f o r s p e c i f i c i t y , c o n s i d e r t h e l o g a r i t h m i c - q u a d r a t i c loss p e n a l t y function (6.2.1). hold. X(E,

Let Y(E,

A s s u m e t h a t t h e c o n d i t i o n s o f Theorem 3.2.2

r ) be d e f i n e d as i n Theorem 6.2.1,

where

r ) i s a l o c a l l y u n i q u e m i n i m i z i n g p o i n t o f W(x,

E,

r).

Define t h e " p e n a l t y f u n c t i o n o p t i m a l value" a s w * ( E , r) z w[X(E, r ) , Since l i m y r+O

lim W

r+O+

*

E,

E,

r) =

Y(E),

E,

rl.

(6.6.1)

it r e a d i l y f o l l o w s t h a t

r ) = l i m W[X(E, r ) , r+O+ = L[x(E),

U ( E ) ,

E,

W(E),

r]

EI

= f[X(E), E l = f*(E)I

t h e o p t i m a l v a l u e f u n c t i o n o f p r o b l e m Pj ( E ) .

W e m i g h t con-

I80

6 Sensitivity Information Using Penalty Functions

2

2 *

l i m V E W ( ~ , r ) = VEf ( E ) . r+O+ R e c a l l t h a t VXW[x(€, r ) , VEW

*

= VXWVEX

= VEf

+

VEW =

-

These r e s u l t s do i n f a c t f o l l o w , E,

rl

v Ew

(r/gi)VEgi

-

5

2

+

(hj/r)VEhj

j=1

i=l = VEf

Thus

= 0.

uiVEgi

f

+

wjVEhj

j=1

i=l = VEL,

2 " from which i t a l s o f o l l o w s t h a t V E W ( E ) = V E [ V E L T ] .

r e l a t i o n s c h a r a c t e r i z e t h e behavior of W

*

(E,

These

r ) , essentially

t h e optimal value function of t h e associated penalty function problem : min

X

W(x,

h.(x, 3

E,

E)

r)

= 0,

,..., p ,

j = 1

and p r e c i s e l y p a r a l l e l t h o s e g i v e n i n Theorem 3.4.1

f o r f*CE).

As a c o r o l l a r y , w e immediately o b t a i n a v e r i f i c a t i o n o f t h e c o n j e c t u r e s t a t e d above. W e have t h u s proved t h e f o l l o w i n g r e s u l t s .

[ F i r s t - and second-order

Theorem 6.6.1.

penalty function optimal value W o f Theorem 3.2.2 hood o f (a)

E

*

(E,

r).]

changes i n t h e

I f the conditions

h o l d f o r problem P 3 ( ~ ) ,t h e n i n a neighbor-

= 0,

*

VEW ( E ,

r)

= V€L[X(E,

r),

U(E,

r),

W(E,

r), €1, (6.6.2)

181

6.6 Optimal Value Function Sensitivity Estimates

(6.6.3)

i=l

Using these results and recalling the assumed twice continuous differentiability of the functions in (x,

*

neighborhood of (x

,

(r),

E)

in a

the following corollary follows

immediately. [Convergence of the penalty function

Corollary 6.6.2.

optimal value, gradient and Hessian to the respective quantities associated with the optimal value function.]

Given the

conditions of Theorem 3.2.2 of Chapter 3, then in a neighborhood of

E

= 0,

(6.6.4)

(b)

lim VEW*(e, r) = VEL[x(~),U(E) r+O+ * = f (€1,

, w(E),

€1 (6.6.5)

and hence also

We can therefore estimate the value of the gradient and Hessian of the optimal value function of problem P3(€) by evaluating the corresponding quantities obtained from the penalty function at a minimizing point near (0, 0) and r

.+

0.

X(E,

r), for

(E,

r)

182

6 Senhitivity Information Using Penalty Functions

EXAMPLE OF ESTIMATES OF OPTIMAL VALUE AND FIRST- AND SECONDPARAMETER DERIVATIVES

6.7.

The second example of S e c t i o n 3 . 3 w i l l be used t o i l l u s t r a t e t h e a p p l i c a t i o n s of t h e s e r e s u l t s . Example 3.3.2

m i n x x1 i-€ l X 2 s.t. gl(x, g2(x,

2 2

E)

= -E2X1

E)

= -xl

+ x2

2 0,

> 0 -

where

E~

> 0 and

E~

# 0.

The s o l u t i o n p o i n t was found t o be

X(E)

I.

=

U(E)

=

[b']

The o p t i m a l v a l u e f u n c t i o n i s f

*

(E)

=

-1

2' 4E1E2

Hence

and

2 * VEf ( € 1 =

-

1 2E1E2

32

"1 2E1E2

T h i s example was used i n S e c t i o n 6 . 3 t o i l l u s t r a t e t h e u s e o f p e n a l t y f u n c t i o n methods t o o b t a i n e s t i m a t e s o f t h e p a r t i a l d e r i v a t i v e s of t h e s o l u t i o n p o i n t and t h e Lagrange m u l t i p l i e r s .

183

6.7 Optimal Value and First- and Second-Parameter Derivatives

W e u s e it i n a s i m i l a r way h e r e t o i l l u s t r a t e t h e u s e o f

p e n a l t y f u n c t i o n methods t o o b t a i n estimates o f t h e o p t i m a l v a l u e f u n c t i o n a n d f i r s t - and s e c o n d - o r d e r c h a n g e s w i t h r e s p e c t to parametric perturbations.

The l o g a r i t h m i c - q u a d r a t i c l o s s p e n a l t y f u n c t i o n

~ ( x E, , r ) = x1

+

- r

E1x2

In(x2

-

2 2 E2x1

)-

r ln(-xl).

The g r a d i e n t o f W w i t h r e s p e c t t o x i s T

2

v Xw =

-

X2-E:Xf

S i n c e V XW = 0 a t a m i n i m i z i n g p o i n t

X(E,

r ) , w e can use

r )~ =

E

t h i s f a c t to calculate x ~ ( E ,

- l+,/l +

8 2 4E1E:2

r) =

Clearly

X(E,

2 r

r ) = [ x ~ ( E r, ) ,

~ and

~

x*(E,

x ~ ( E , r ) l T+ X ( E )

2~ 2 x (E, r) 2 1

as r

+

-.€1

+ r

0, and,

from t h e t h e o r y o f p e n a l t y f u n c t i o n s , w e know a l s o t h a t W

*

(E,

r) z W[X(E, r ) ,

From Theorem 6.6.1

E,

1-1

+ x ~ ( E )

and C o r o l l a r y 6 . 6 . 2 ,

n e a r ( 0 , 0 ) a n d r > 0 a n estimate o f V E f VEW

*

(E,

r) = V L

I

[x(E,r) ,u(E,r)

,EI

+

E ~ x ~ ( E=)

f

*

(E).

w e know t h a t f o r

*

(E)

i s g i v e n by

(E,

r)

I84

6 Sensitivity Information Using Penalty Functions

where t h e L a g r a n g i a n f o r o u r problem i s d e f i n e d as L(x, u,

+

x1

E)

E1X2

-

-

u1(x2

E;x;)

+

u2x1.

This y i e l d s VEW

*

( E r

r) =

(X2(Er

r ) , 0)

w e have t h a t

From ( 6 . 2 . 2 1 ,

-

r)(O,

U1(Er

u ~ ( E ,r) =

r/gl[x(E,

r ) , € 1 and

from t h e s o l u t i o n f o r x 2 ( ~ r, ) w e f i n d t h a t g1 E x 2 Consequently,

r/El.

* vEw

As r + O

* vEw

(E,

U ~ ( E ,r) = E

r)).

- 2 E 2 X 1 (2E ,

-

2 2 ~

=X

~

by c a l c u l a t i n g V E2 W"

(E,

r)

E

a, n d h e n c e

~

r) =

r

( E r

r)

+

a s e x p e c t e d from C o r o l l a r y 6.6.2. S i m i l a r l y , w e c a n e s t i m a t e VEf 2 " for

(E,

(E)

r ) n e a r ( 0 , 0) a n d r > 0 , by Theorem 6 . 6 . 1 a n d obtaining

Corollary 6.6.2,

where u1 = u1

(E

,

r ) and x1

[VEx(E, r ) , V € U ( E , l a t e d from ( 6 . 5 . 8 ) .

r ) l of

= x1 ( E [x(E,

,

r).

r),

U(E,

The p a r t i a l d e r i v a t i v e s

r ) ] c a n be c a l c u -

A l l r e q u i r e d q u a n t i t i e s c a n be computed

once t h e u n c o n s t r a i n e d m i n i m i z i n g p o i n t

X(E,

r ) i s known.

I85

6.8 Sensitivity Approximations for RHS Perturbations

W e do n o t c a r r y o u t t h e c a l c u l a t i o n s ,

2 * c o r o l l a r y t h a t VEW

(E,

r)

b u t we know f r o m t h e

Vgf*(E) a s r

-f

+

0.

P u r s u i n g t h e a l g o r i t h m i c r e s u l t s p a r a l l e l i n g t h o s e obt a i n e d f o r t h e r h s p e r t u r b a t i o n problem P 2 ( € ) i n Chapter 5, w e n e x t c o n s i d e r t h e a p p l i c a t i o n o f t h e f o r e g o i n g p e n a l t y funct i o n r e s u l t s t o t h i s i m p o r t a n t r e a l i z a t i o n of t h e g e n e r a l problem P j ( € 1 . 6.8.

SENSITIVITY APPROXIMATIONS FOR RHS PERTURBATIONS The p r o b l e m o f i n t e r e s t i s min

where Em + p ) ,

=

E

s.t. g ( x ) 2

f(x)

X

(E

1

,

E

2

1,

E~

=

E

~

( E ~ ,

h, ( x ) =

...,

E

m

E

2

,

P2 ( E l

and c 2 =

...

r

a n d f o r which t h e g e n e r a l r e s u l t s g i v e n i n S e c t i o n 6.5

simplify considerably. from ( 6 . 5 . 8 )

I n p a r t i c u l a r , s i n c e V 2 L = 0 , w e have EX

that

VEy(E, r ) =

where t h e A i j

are g i v e n i n ( 6 . 5 . 6 ) .

p r e c i s e a n a l o g y of All

2

= V X W [ x ( ~ r, ) ,

(5.5.1). E,

r1-l.

T h i s i s of c o u r s e t h e

I t i s shown i n S e c t i o n 6.5 t h a t

Thus,

from ( 6 . 5 . 6 1 ,

(6.8.1)

I86

6 Sensitivity Information Using Penalty Functions

where w e r e c a l l t h a t P =

[-:I]

and

where U = d i a g ( u . ) .

Q

=

[

U2

: ] I

Hence, (6.8.2)

Also, p r e c i s e l y analogous t o (5.5.2)

,

( 6 6.3)

E E,r

"I

yields

r)

where

I-$ + Q

=

7 1 QPVxW-'PTQ][ r

:].

W e c a n a l s o c a l c u l a t e t h e s e d e r i v a t i v e s i n t e r m s of V ~ X ( E ,

r ) , r e c a l l i n g t h a t ui = r / g i

[vIw(E,I:] V

U(Er

=

-$

:]

+

and G = d i a g ( g i ) , o b t a i n i n g

$ QPVEX(Er

I87

6.X Sensitivity Approxiinalions for RHS Perturbation\

V€W(E,

r ) = r1[ ( O ,

-I)

+

VXhVEx(E, r ) l .

(6.8.4)

W e may u s e t h e p e n a l t y f u n c t i o n m i n i m i z i n g p o i n t d i r e c t l y

a s i n Section 6.2

t o o b t a i n t h e same r e s u l t .

This approach,

which w e now i n d i c a t e , i s t h e o n e t h a t was implemented comput a t i o n a l l y v i a SENSUMT by Armacost and Mylande? Armacost ( 1 9 7 6 a ) .

I t a l s o y i e l d s (6.8.2)

-

( 1 9 7 3 ) a n d by

( 6 . 8 . 4 ) and i n -

volves manipulating information t h a t i s r e a d i l y a v a i l a b l e a t t h e s o l u t i o n p o i n t of t h e penalty function.

F o r problem P 2 ( E ) ,

t h e logarithmic-quadratic loss p e n a l t y function i s

i=l

j=l

and t h e g r a d i e n t t a k e n w i t h r e s p e c t t o x is

j=1

Taking p a r t i a l d e r i v a t i v e s w i t h r e s p e c t t o t h e r h s parameters E,

w e have a t

a (vxw) -aEi

X(E,

r)

rVxgi [ x ( E , r ) l [gi[x(E, r ) l

-

ci1 2,

i = 1,

...

r

m,

I88

6 Scn\iti\ity Inlorination Ubing Pcnalty Functlclns

and

a (VxW) a€. 3 +m

--

-

-Vxh.

[ X ( E r

r )1

,

r

..., p .

j = 1,

S i n c e VxW i s i d e n t i c a l l y z e r o i n a n e i g h b o r h o o d o f

r) =

(E,

( 0 , O), r > 0 , w e c a n t a k e t h e t o t a l d e r i v a t i v e o f VxW = 0

with respect t o

E

t o obtain

U s i n g t h e above r e l a t i o n s h i p s , w e o b t a i n

[gi[x(E, r ) l

which i s e x a c t l y ( 6 . 8 . 2 ) . i f i # k.

-

ci1

... ...

2

= 1 i f

D e f i n e 6ik

W e know t h a t

a[gi[x(Er r ) l

-

E ~ ] / =~ -cSik, E ~

i = 1,

k = 1,

a [ h 7. [ x ( E , r ) l - E ~ + ~ I / ~ =E ,- &m,k'

Vxh. T

-

E ~ =I VXqi[x(E, r ) l , i

VX[hj[x(€, r ) ]

-

E ~ += ~V ]h . [ X ( E r r ) l ,

u ~ ( E r, ) = r / [ g i [ x ( E ,

X

..., m ,

..., m

r)l - ~ i l r i

w.(E, r ) = [ h j [ x ( E , r ) ] 7

-

=

+ p;

..., p , ..., m

1,

...

~ ~ + ~ l j/ =r ,1,

r

+

p;

...

= 1,

m;

r

...

j = 1,

I

..]

i = k a n d €iik = 0

j = 1,

r)]

r )1

r

k = 1,

VX[qi[x(E,

[X(Er

r

m;

...

r

P.

p;

I89

6.X Sensitivity Approximations lor RHS Pcrturbationh

Using a l l of t h e s e r e l a t i o n s h i p s , w e o b t a i n

aw.

(E,

r)

(x(E,

aEk i = 1,

..., m ,

r))aX r ,r a E(kE

j = 1,

..., p t

-

6j+m,k

1

'

k = 1, . . . r

m + P.

Using t h e n o t a t i o n i n t r o d u c e d a b o v e , w e o b t a i n V E u ( ~ r, ) = r G - 2 [(I, 0 )

-

VX9VEX(Er

r)]

and

which a r e e x a c t l y ( 6 . 8 . 3 ) a n d ( 6 . 8 . 4 ) . The matrices d e t e r m i n e d by ( 6 . 8 . 3 ) a n d t h e n e g a t i v e o f ( 6 . 8 . 4 ) y i e l d a n a p p r o x i m a t i o n o f t h e H e s s i a n of t h e o p t i m a l

value function f right-hand

*

(E)

w i t h r e s p e c t t o t h e components of t h e

s i d e of t h e c o n s t r a i n t s f o r p r o b l e m P 2 ( ~ ) . T h i s

f o l l o w s d i r e c t l y from c o n c l u s i o n ( f ) o f Theorem 5 . 4 . 1 a n d Corollary 6 . 2 . 2 .

T h i s r e s u l t may a l s o b e s e e n by r e c a l l i n g 2 *

from C o r o l l a r y 6 . 6 . 2 t h a t VEW*(€, r ) + V E f * ( c ) and V E W ( E , r) 2 * + Since it i s readily calculated t h a t + V f (E) a s r 0 E -+

*

VEW ( E ,

and

r) =

.

I90

6 Sensitivity Informalion Using Penalty Functions

we conclude t h a t lim

r+O+

and

lim

r+O+

vE‘

-vEw

E, E,

r)

=

r)

2 *

Vcf

(E)

=

VEU(E) [-vEw

(E)

as state 6.9.

RECAPITULATION In Chapters 2

-

5 , t h e o r e t i c a l and c o m p u t a t i o n a l a s p e c t s

of s e n s i t i v i t y a n a l y s i s have been d e v e l o p e d i n p a r a l l e l .

The

t h e o r e t i c a l r e s u l t s have been shown t o be e a s i l y i m p l e m e n t a b l e computationally v i a a penalty function technique.

In the

example above, t h e e s t i m a t e s by way of a p e n a l t y f u n c t i o n were d e r i v e d a n a l y t i c a l l y ; however, i n p r a c t i c e , w e o b t a i n n u m e r i c a l e s t i m a t e s f o r a d e c r e a s i n g s e q u e n c e o f p o s i t i v e v a l u e s of r. I t r e q u i r e s o n l y a modest e f f o r t t o modify t h e u s u a l p e n a l t y

f u n c t i o n a l g o r i t h m t o o b t a i n t h e r e s u l t s d e v e l o p e d above. The r e s u l t s o f C h a p t e r s 3

-

5 are a l g o r i t h m - i n d e p e n d e n t .

They r e q u i r e o n l y t h e c o n d i t i o n s o f Theorem 3 . 2 . 2 . once a Karush-Kuhn-Tucker

Clearly,

t r i p l e i s known, o b t a i n i n g t h e

gradient of t h e optimal value function is a t r i v i a l matter c o m p u t a t i o n a l l y , r e q u i r i n g o n l y t h e e v a l u a t i o n of t h e p a r t i a l d e r i v a t i v e o f t h e Lagrangian w i t h r e s p e c t t o t h e parameters. I f t h e g r a d i e n t o f t h e Karush-Kuhn-Tucker

t r i p l e i s known,

t h e n t h e c a l c u l a t i o n of t h e H e s s i a n o f t h e o p t i m a l v a l u e funct i o n i s a l s o a r e l a t i v e l y s i m p l e matter u s i n g ( 3 . 4 . 5 ) . The c e n t r a l d i f f i c u l t y a r i s e s i n o b t a i n i n g t h e g r a d i e n t o f t h e Karush-Kuhn-Tucker

t r i p l e f o r t h e g e n e r a l case.

A general

191

6.9 Recapitulation

e x p r e s s i o n was g i v e n i n ( 4 . 2 . 1 5 )

i n t e r m s of t h e inverse of

~ ( E IT .h r e e d i f f e r e n t sets o f e q u a t i o n s ( 4 . 2 . 8 ) , (4.2.13),

lem.

(4.2.91,

or

a r i s e , d e p e n d i n g o n t h e c h a r a c t e r i s t i c s o f t h e probe x i s t s o r when t h e r e a r e n l i n e a r l y independ-

When V;L-’

ent constraints,

(4.2.8)

and ( 4 . 2 . 9 )

may b e implemented t o

p e r m i t t h e c a l c u l a t i o n o f V E y ( € ) w i t h modest e f f o r t .

However,

when n e i t h e r o f t h e s e c o n d i t i o n s a p p l i e s , t h e e f f o r t i n v o l v e d

t o c o n s t r u c t t h e matrices r e q u i r e d i n ( 4 . 2 . 1 3 )

i s much g r e a t e r .

N o n e t h e l e s s , t h e p r o c e d u r e d e s c r i b e d by t h e e q u a t i o n s i n (4.2.13)

i s much more e f f i c i e n t t h a n o n e s i m p l y i n v o l v i n g t h e

inversion of

G(E)

t h a t does not e x p l o i t t h e r e l a t i o n s

stipulated i n the optimality conditions. W e h a v e shown t h a t p e n a l t y f u n c t i o n methods e a s i l y p r o v i d e

the desired estimates.

The g r a d i e n t o f t h e p e n a l t y f u n c t i o n

i s t h e estimate o f t h e g r a d i e n t o f t h e o p t i m a l v a l u e f u n c t i o n . S i m i l a r l y , t h e H e s s i a n o f t h e p e n a l t y f u n c t i o n i s a n estimate of t h e H e s s i a n o f t h e o p t i m a l v a l u e f u n c t i o n a n d i s e a s i l y c a l c u l a t e d when a n estimate o f t h e g r a d i e n t o f t h e KarushKuhn-Tucker known. M(E,

t r i p l e taken with respect t o t h e parameters i s

A t h e o r y f o r o b t a i n i n g VEy(€, r ) w a s developed using

r ) , by way of s t r a i g h t f o r w a r d m a n i p u l a t i o n o f ( 6 . 2 . 8 ) .

However, a more e f f i c i e n t a l t e r n a t i v e a p p r o a c h u s i n g E q . (6.2.15)

is a v a i l a b l e , i f t h e given penalty function algorithm

i s being u t i l i z e d . the solution of

Armacost a n d Mylander ( 1 9 7 3 ) implemented

(6.2.15)

t o obtain V€X(E,

r ) , u s i n g t h e SUMT

V e r s i o n 4 computer c o d e w i t h t h e l o g a r i t h m i c - q u a d r a t i c l o s s penalty function. (6.2.21)

I t i s t h e n a t r i v i a l m a t t e r t o implement

and 6.2.22)

t o o b t a i n V E u ( € , r ) and V € W ( E ,

r ) , thus

o b t a i n i n g a n estimate o f t h e g r a d i e n t o f t h e Karush-Kuhn-Tucker

192

6 Sensitivity Information Using Penalty Functions

t r i p l e , a n d a l l o w i n g f o r e a s y c a l c u l a t i o n o f a n estimate of t h e Hessian of t h e o p t i m a l v a l u e f u n c t io n . S e v e r a l a p p l i c a t i o n s of t h e s e r e s u l t s have been mentioned. W e r e c a p i t u l a t e h e r e and d i s c u s s s e v e r a l o t h e r p o t e n t i a l ap-

plications.

The estimate o f t h e s e n s i t i v i t y o f t h e Karush-

Kuhn-Tucker t r i p l e g i v e s a measure o f t h e s t a b i l i t y o f t h e solution point.

I n a d d i t i o n , t h i s i n f o r m a t i o n c a n be u s e d t o

g i v e a f i r s t - o r d e r esti.mate o f Karush-Kuhn-Tucker

t r i p l e s of

problems i n v o l v i n g d i f f e r e n t v a l u e s o f t h e parameters, once one s u c h t r i p l e h a s b e e n d e t e r m i n e d . s i t i v i t y of t h e Karush-Kuhn-Tucker

An estimate o f t h e s e n -

t r i p l e i s necessary t o

o b t a i n t h e Hessian of t h e o p t i m a l v a l u e f u n ctio n . The estimates o f t h e s e n s i t i v i t y o f t h e o p t i m a l v a l u e f u n c t i o n c a n b e u s e d t o o b t a i n f i r s t - and s e c o n d - o r d e r e s t i mates o f t h i s f u n c t i o n , u s i n g a t r u n c a t e d T a y l o r ' s s e r i e s . This s e n s i t i v i t y information f o r t h e optimal value function and t h e Karush-Kuhn-Tucker

t r i p l e may b e u s e d i n a c y c l i c

p r o c e d u r e f o r s o l v i n g P 3 ( € ) , where t h e p a r a m e t e r s c o r r e s p o n d t o a s u b s e t o f t h e variables.

F o r problem P 2 ( € ) , where t h e

parameters are on t h e right-hand

s i d e of t h e c o n s t r a i n t s , such

a procedure u s i n g t h e f i r s t - and second-order

sensitivity

i n f o r m a t i o n may have a p p l i c a t i o n i n s o l v i n g p r o b l e m s by r h s d e c o m p o s i t i o n , a well-known

technique.

idea i s given i n Chapter 5, Section 5.6. p a r a m e t e r i z e d on t h e r i g h t - h a n d

An example o f t h i s Problems

s i d e o f t h e c o n s t r a i n t s are

i n t i m a t e l y i n v o l v e d i n much of d u a l i t y t h e o r y and t h i s s e n s i t i v i t y i n f o r m a t i o n may h a v e a p p l i c a t i o n i n a l g o r i t h m s s o l v i n g P ( E ) using various duals.

193

6.9 Recapitulation

While w e h a v e c o n c e n t r a t e d on p a r a m e t r i c s e n s i t i v i t y a n a l y s i s and d e v e l o p e d n e i g h b o r h o o d r e s u l t s , many of t h e s e r e s u l t s e x t e n d t o p a r a m e t r i c n o n l i n e a r programming, where t h e param-

eters are p e r m i t t e d t o range i n a p r e s c r i b e d set.

This w i l l

u n d o u b t e d l y be a s u b j e c t of c o n t i n u i n g f u t u r e i n v e s t i g a t i o n s .

Chapter 7

Calculation of Sensitivity Information Using Other Algorithms

7.1.

INTRODUCTION

I n t h e p r e v i o u s c h a p t e r i t i s shown t h a t a l l t h e s e n s i t i v i t y i n f o r m a t i o n t h a t was developed i n P a r t I1 o f t h i s book, namely t h e f i r s t and second p a r a m e t e r d e r i v a t i v e s o f t h e o p t i m a l v a l u e f u n c t i o n and t h e f i r s t p a r a m e t e r d e r i v a t i v e s of a Karush-Kuhn-Tucker

t r i p l e , can be e s t i m a t e d u s i n g informa-

t i o n t h a t i s a v a i l a b l e i n t h e u s u a l a p p l i c a t i o n of a s o l u t i o n algorithm.

The a n a l y s i s t h e r e c o n c e n t r a t e s on an i m p o r t a n t

c l a s s o f a l g o r i t h m s b a s e d on c l a s s i c a l p e n a l t y f u n c t i o n s and

i s s p e c i a l i z e d t o p e r h a p s t h e most p o p u l a r and most h e a v i l y u t i l i z e d r e p r e s e n t a t i v e of t h i s c l a s s , t h e logarithmicq u a d r a t i c mixed b a r r i e r - p e n a l t y f u n c t i o n ( 6 . 2 . 1 ) .

However, i t

was n o t e d t h a t a l g o r i t h m s g e n e r a l l y use and g e n e r a t e informat i o n t h a t i s r e l e v a n t t o t h e c a l c u l a t i o n of s e n s i t i v i t y estimates

.

W e support t h i s f a c t i n t h i s chapter, indicating s p e c i f i c

and e x p l o i t a b l e c o n n e c t i o n s w i t h t y p i c a l l y a v a i l a b l e a l g o r i t h m i c i n f o r m a t i o n and b r i e f l y p u r s u i n g t h e s e f o r s e v e r a l i m p o r t a n t c l a s s e s of a l g o r i t h m s , i n c l u d i n g Newton, p r o j e c t e d - g r a d i e n t , reduced-gradient,

and augmented Lagrangian methods.

I94

Before

195

7.1 Introduction

doing so, w e d i g r e s s b r i e f l y t o s u p p o r t t h e i d e a t h a t s o l u t i o n a l g o r i t h m c a l c u l a t i o n s may o f t e n be d i r e c t l y r e l a t e d t o s e n s i t i v i t y and s t a b i l i t y estimates. I t i s a p p a r e n t t h a t t h e c o n n e c t i o n between s o l u t i o n algo-

r i t h m c a l c u l a t i o n s and s e n s i t i v i t y c a l c u l a t i o n s may b e cons i d e r a b l y d e e p e r t h a n t h e f a c t t h a t t h e s e c a l c u l a t i o n s simply r e q u i r e common i n f o r m a t i o n .

Solution algorithms t y p i c a l l y

s o l v e o r e s t i m a t e a s o l u t i o n o f a sequence o f subproblems o f t h e g i v e n problem whose s o l u t i o n s "converge" i n some s e n s e t o t h e g i v e n problem.

The subproblems may o f t e n be viewed a s

p e r t u r b a t i o n s o f t h e o r i g i n a l problem, a s o l u t i o n o f which would t h e n presumably be a l i m i t of s o l u t i o n s o f an a s s o c i a t e d sequence o f t h e p e r t u r b e d problems.

Thus, once t h e l a t t e r

sequence o f problems h a s been i d e n t i f i e d , a p r e d i c t i o n of a s o l u t i o n t o t h e n e x t problem, g i v e n a s o l u t i o n t o t h e problem a t hand, might be p r o v i d e d by a p e r t u r b a t i o n a l a n a l y s i s o f t h e problem a t hand and e x t r a p o l a t i o n .

Thus, i t seems p l a u s i b l e

t h a t such an a l g o r i t h m might be r a t h e r c o m p l e t e l y c h a r a c t e r i z e d i n t e r m s of e s t i m a t e s d e r i v e d from a n a p p r o p r i a t e s e n s i t i v i t y o r s t a b i l i t y a n a l y s i s of t h e g i v e n problem. I n f a c t , i m p o r t a n t f a m i l i e s o f a l g o r i t h m s a r e known t o d e f i n e s o l u t i o n t r a j e c t o r i e s o r s e q u e n c e s o f an a s s o c i a t e d f a m i l y o f subproblems t h a t have been a n a l y z e d v i a s e n s i t i v i t y and s t a b i l i t y t e c h n i q u e s .

Such r e s u l t s have been r e p o r t e d by

Meyer ( 1 9 7 0 ) , F i a c c o and McCormick (19681, and Robinson ( 1 9 7 4 ) , i n t h e f i r s t i n s t a n c e t o a n a l y z e and p r o v e convergence, i n t h e second t o c h a r a c t e r i z e and a c c e l e r a t e convergence, and i n t h e t h i r d t o e s t a b l i s h t h e r a t e o f convergence. Section 6.2,

F o r example, i n

w e n o t e d t h a t t h e p e n a l t y f u n c t i o n method u t i l i z e d

196

7 Calculation of Sensitivity Using Other Algorithms

i n Chapter 6 could be l o c a l l y completely c h a r a c t e r i z e d i n

t e r m s o f s o l u t i o n s of a p e r t u r b a t i o n of t h e Karush-Kuhn-Tucker c o n d i t i o n s , t h u s immediately p l a c i n g t h e p e n a l t y f u n c t i o n minimization i n correspondence w i t h a s o l u t i o n of a p er tu r b at i o n of t h e g i v e n problem.

This correspondence w a s f i r s t

r e p o r t e d by F i a c c o and McCormick ( 1 9 6 8 ) . Thus, t h e f a c t t h a t s e n s i t i v i t y a n d s o l u t i o n a l g o r i t h m a n a l y s i s may c o n c e r n t h e same p r o b l e m - p e r t u r b a t i o n

solution

t r a j e c t o r y h a s been o b s e r v e d a n d e x p l o i t e d h e r e and e l s e w h e r e . I n c o n c l u s i o n , w e n o t e s t i l l a n o t h e r i n t e r e s t i n g correspondence. Suppose f r C 2 and w e w i s h t o s o l v e Vf ( x ) = 0 , g i v e n such t h a t Vf(F) =

F #

2

0 and s u p p o s e V f ( F ) - l e x i s t s .

x

I t is

n o t d i f f i c u l t t o show t h e f o l l o w i n g e q u i v a l e n c e between s e n s i S t a r t i n g a t K, t h e

t i v i t y a n a l y s i s a n d Newton's method:

Newton estimate ( i . e . , t h e f i r s t i t e r a t e ) o f a s o l u t i o n o f (Taylor's series)

V f ( x ) = 0 i s t h e same as t h e f i r s t - o r d e r e s t i m a t e o f x ( 0 ) a t x ( S ) = X, where s o l u t i o n o f V ~ [ X ( E )= ] E n e a r

E

X(E)

= 7.

E

C1

is t h e unique

Thus, any i t e r a t i o n o f

Newton's a l g o r i t h m i s t h e s a m e a s a n e x t r a p o l a t i o n b a s e d on a p a r a m e t r i c s e n s i t i v i t y a n a l y s i s , a n d h e n c e Newton's a l g o r i t h m may be c h a r a c t e r i z e d i n t h i s s e t t i n g . 7.2.

CONNECTIONS BETWEEN ALGORITHMIC AND SENSITIVITY CALCULATIONS The m o s t o b v i o u s and c o m p e l l i n g way t o see t h e c o n n e c t i o n

between t h e v a l u e s a n d t h e p a r a m e t e r d e r i v a t i v e s o f t h e o p t i -

m a l v a l u e f u n c t i o n o r a Karush-Kuhn-Tucker

t r i p l e and algo-

r i t h m i c information i s through t h e d e r i v a t i o n o f t h e s e quantities.

Though v a r i o u s s u f f i c i e n t c o n d i t i o n s h a v e b e e n

i n v o k e d t o p r o v i d e e x i s t e n c e p r o o f s and t h e d e t e r m i n a t i o n of

197

7.2 Algorithmic and Sensitivity Calculations

e x p l i c i t f o r m u l a s and c o n t i n u i t y p r o p e r t i e s , n o t e t h a t c r u c i a l s e n s i t i v i t y measurements a r e b a s e d o n t h e s a t i s f a c t i o n o f necessary c o n d i t i o n s c h a r a c t e r i z i n g o p t i m a l i t y .

This leads t o

s e v e r a l r e l e v a n t connections. Assuming d i f f e r e n t i a b i l i t y o f t h e problem f u n c t i o n s and any o f a v a r i e t y o f w e l l known a n d t y p i c a l l y i n v o k e d c o n s t r a i n t q u a l i f i c a t i o n s (see S e c t i o n 2 . 3 ) , any v a l i d a l g o r i t h m a p p l i e d

*

to P2(0) w i l l , i f successful, generally yield a t r i p l e (x

u

*

,w

*

)

,

s a t i s f y i n g t h e f i r s t - o r d e r necessary conditions ( 3 . 2 . 2 ) I t i s w e l l known a n d h a s b e e n amply d e m o n s t r a t e d

for P2(0).

h e r e i n i n Chapter 5 and elsewhere t h a t t h e o p t i m a l m u l t i -

*

p l i e r s (u

,w

*

)

are i n t i m a t e l y r e l a t e d t o t h e r a t e o f change

of t h e o p t i m a l v a l u e f u n c t i o n w i t h r e s p e c t t o c o n s t r a i n t perturbations.

F o r example, u n d e r t h e a s s u m p t i o n s o f C o r o l l a r y * T * T 3 . 4 . 4 o r Theorem 5.4.1, w e know t h a t V E f * ( 0 ) = [ ( u ) , (-w 1 1.

F o r t h e g e n e r a l problem P 3 ( 0 ) a n d u n d e r t h e a s s u m p t i o n s of Theorem 3 . 2 . 2 ,

*

VEf (0) would o f c o u r s e b e g i v e n by ( 3 . 4 . 4 1 ,

*

which c a n be c a l c u l a t e d o n c e ( x

,

u

*

,w

*

)

i s known.

i s t h a t t h i s s e n s i t i v i t y i n f o r m a t i o n , namely u

*

and w

The p o i n t

*

,

is not

m e r e l y a v a i l a b l e b u t i s o f t e n e x p l i c i t l y required t o d e t e r m i n e whether a c a n d i d a t e s o l u t i o n has been o b t a i n e d .

Thus, t h e

c o n n e c t i o n between a l g o r i t h m i c c a l c u l a t i o n s a n d s e n s i t i v i t y analysis is inevitable. T u r n i n g o u r a t t e n t i o n t o t h e p a r a m e t e r d e r i v a t i v e s VEy ( € 1 o f a Karush-Kuhn-Tucker

t r i p l e [ x ( E ),

U(E)

, w ( E ) ]of

P3(€)

We

recall t h a t M ( E ) V ~ Y ( E )= N(E)

where M a n d -N

(3.2.3)

are, r e s p e c t i v e l y , t h e Jacobians of t h e first-

o r d e r n e c e s s a r y c o n d i t i o n s ( 3 . 2 . 2 ) o f P 3 ( ~ ,) w i t h r e s p e c t t o

198

7 Calculation of Sensitivity Using Other Algorithms

( x , u, w ) and

E.

S i n c e M and N i n v o l v e o n l y t h e v a l u e s o r

p a r t i a l d e r i v a t i v e s o f t h e o r i g i n a l problem f u n c t i o n s and t h e multipliers,

i t i s a p p a r e n t t h a t M ( 0 ) and N ( 0 ) a r e a l s o com-

*

p l e t e l y d e t e r m i n e d , once ( x

* * ,u ,w )

i s known.

I n view o f

( 3 . 2 . 3 ) and o u r a s s u m p t i o n s , t h i s i n f o r m a t i o n c o m p l e t e l y de-

t e r m i n e s V E y ( 0 ) . T h e r e f o r e , any v a l i d a l g o r i t h m a p p l i e d t o P 3 ( 0 ) c a n p r o v i d e an e s t i m a t e of

*

(x

,

u

*

,

w

*

),

i f successful,

and t h i s can b e used i n ( 3 . 2 . 3 ) t o c a l c u l a t e an estimate o f 2 * V E y ( ~ a) t E = 0 . S i n c e t h e c a l c u l a t i o n of VEf (0) ( 3 . 4 . 5 ) c a n a l s o be made once VEy(0) i s a v a i l a b l e , t h i s means t h a t any p r o c e d u r e t h a t can e s t i m a t e a Karush-Kuhn-Tucker u

*

,

w

*

),

i.e.,

*

t r i p l e (x

,

s a t i s f y t h e f i r s t - o r d e r necessary conditions

( 3 . 2 . 2 ) f o r a l o c a l s o l u t i o n , provides s u f f i c i e n t information

t o estimate a l l t h e s o l u t i o n parameter-derivative t h a t w e have p r e s e n t e d .

calculations

Summarizing, t h e s o l u t i o n s e n s i t i v i t y

c a l c u l a t i o n s a r e b a s e d on t h e s o l u t i o n i n f o r m a t i o n t h a t a s o l u t i o n algorithm i s t y p i c a l l y g e n e r a l l y endeavoring t o estimate, under t h e g i v e n assumptions. A c t u a l l y , t h e c o n n e c t i o n i s c o n s i d e r a b l y d e e p e r and more e x p l i c i t , as may r e a d i l y b e e s t a b l i s h e d .

The f i r s t - o r d e r

necessary c o n d i t i o n s ( 3 . 2 . 2 ) f o r a l o c a l s o l u t i o n of P ( 0 ) a r e system o f e q u a t i o n s i n y = ( x , u , w )

T

.

A w e l l known t e c h n i q u e

f o r s o l v i n g a system o f equations, p a r t i c u l a r l y a t t r a c t i v e because of e x c e l l e n t convergence p r o p e r t i e s

(under t h e given

c o n d i t i o n s and when a good e s t i m a t e o f a s o l u t i o n i s a v a i l a b l e ) ,

i s Newton's method.

L e t t i n g F = (VL, u 1g 1,

..., umgml h l , ...,

AsT I it f o l l o w s t h a t ( 3 . 2 . 2 ) i s e q u i v a l e n t t o F = 0 . hP) * suming t h a t yk n e a r a s o l u t i o n y i s a v a i l a b l e , t h e n e x t

I99

7.2 Algorithmic and Sensitivity Calculations

v e c t o r yk+l d e t e r m i n e d by Newton's method i s Yk+l = Yk

-

(7.2.1)

MilFk

where t h e s u b s c r i p t d e n o t e s e v a l u a t i o n a t y = y k ,

E

= 0 , and

t h e matrix M is t h e usual Jacobian of F with respect t o (x, u, w)

,

t h e same m a t r i x a p p e a r i n g i n t h e e q u a t i o n ( 3 . 2 . 3 ) t h a t

d e t e r m i n e s V E y ( ~ ) . R e c a l l i n g t h a t M ( 0 ) i s n o n s i n g u l a r under o u r a s s u m p t i o n s (Theorem 3.2.21,

w e a r e t a c i t l y assuming t h a t

yk is c l o s e enough t o y* t o a s s u r e t h a t systems o f e q u a t i o n s ,

q1e x i s t s .

( 3 . 2 . 3 ) and ( 7 . 2 . 1 ) ,

Both

may b e w r i t t e n i n

t h e form

Mz = b

(7.2.2)

where z i s t h e v a r i a b l e .

C o n s e q u e n t l y , any t h e o r e t i c a l re-

s u l t s o r numerical techniques f o r analyzing o r s o l v i n g (7.2.2)

w i l l be a p p l i c a b l e t o s o l v i n g ( 3 . 2 . 3 ) ,

and v i c e - v e r s a .

In

p a r t i c u l a r , it i s e v i d e n t t h a t an a n a l y s i s o f t h e s t r u c t u r e o f M and o f e f f i c i e n t t e c h n i q u e s f o r s o l v i n g (7.2.2)

becomes a

common c o n c e r n o f any a l g o r i t h m i c p r o c e d u r e s b a s e d on i m i t a t i n g (7.2.1)

and of any n u m e r i c a l t e c h n i q u e s f o r c a l c u l a t i n g

t h e s o l u t i o n p a r a m e t e r d e r i v a t i v e s V E y ( ~ from ) (3.2.3).

For

example, t h e d e c o m p o s i t i o n s o f M g i v e n i n C h a p t e r s 4 and 5 a r e r e l e v a n t t o any a l g o r i t h m b a s e d on ( 7 . 2 . 1 ) . s o l u t i o n a l g o r i t h m b a s e d on (7.2.1) solve (7.2.2)

C o n v e r s e l y , any

w i l l a l r e a d y be geared t o

and hence w i l l be a b l e t o s o l v e (3.2.31,

and

most i m p o r t a n t l y , t h e same m a t r i x would be i n v o l v e d i n making a given e s t i m a t e .

This f a c t i s obviously e x p l o i t a b l e , both

t h e o r e t i c a l l y and c o m p u t a t i o n a l l y .

F o r example, it g r e a t l y

e a s e s t h e c o d i n g , s t o r a g e , and d a t a p r o c e s s i n g r e q u i r e m e n t s needed t o add t h e s e n s i t i v i t y a n a l y s i s c a p a b i l i t y t o any

e x i s t i n g NLP code b a s e d on v a r i a n t s o f ( 7 . 2 . 1 ) .

200

7 Calculation of Sensitivity Using Other Algorithms

The i d e a o f u s i n g ( 7 . 2 . 1 )

a s a b a s i s f o r an a l g o r i t h m , o r

a s a " c l o s i n g t e c h n i q u e , " i n t h e f i n a l s t a g e s o f a n y NLP a l g o r i t h m i s a t t r a c t i v e and p o p u l a r .

E f f i c i e n t NLP a l g o r i t h m s

must u t i l i z e second-order i n f o r m a t i o n and must u l t i m a t e l y comp e t e w i t h t h e performance o f t h e i t e r a t i v e p r o c e s s ( 7 . 2 . 1 ) . I n d e e d , t h e second-order a l g o r i t h m s o f t e n a p p r o x i m a t e t h i s p r o c e s s a s a s o l u t i o n i s approached.

S t r o n g and e x p l o i t a b l e

c o n n e c t i o n s t h u s e x i s t between t h e s e n s i t i v i t y a n a l y s i s c a l c u l a t i o n s and t h e most e f f e c t i v e NLP a l g o r i t h m s . Thus, i t would a p p e a r t h a t e s t i m a t e s o f t h e m a t r i x M and t h e p r o c e s s d e f i n e d by ( 7 . 2 . 1 )

a r e i n e v i t a b l y involved i n t h e

c a l c u l a t i o n s of second-order algorithms, a t l e a s t n e a r a solution.

C l e a r l y , e s t i m a t e s o f a Karush-Kuhn-Tucker

triple are

e s s e n t i a l and s u f f i c e t o p r o v i d e e s t i m a t e s o f t h e c o n s t i t u e n t s of M near a s o l u t i o n of t h e necessary conditions ( 3 . 2 . 2 ) . Algorithms d i f f e r i n how t h e d a t a t h e y m a n i p u l a t e a r e o r g a n i z e d and how t h e e s t i m a t e s o f t h e b i n d i n g c o n s t r a i n t s and t h e Lagrange m u l t i p l i e r s a r e made.

This l e a d s t o d i f f e r e n t repre-

s e n t a t i o n s o f t h e m a t r i x M and i t s i n v e r s e , f o r d i f f e r e n t algorithms.

W e have a l r e a d y s e e n one s u c h r e p r e s e n t a t i o n ,

f a c t , given i n (6.5.6) mixed b a r r i e r - p e n a l t y

in

f o r t4-l from t h e l o g a r i t h m i c - q u a d r a t i c function algorithm.

I n the next

s e c t i o n , w e i n v e s t i g a t e t h e s t r u c t u r e o f M-1 f o r a g e n e r a l a l g o r i t h m i c s t r a t e g y t h a t i n c l u d e s a s r e a l i z a t i o n s two a d d i t i o n a l important c l a s s e s of algorithms, projected-gradient reduced-gradient algorithms.

and

20 I

7.3 Inverse of the Jacobian of the Karush-Kuhn-Tucker System

ALGORITHMIC CALCULATIONS O F THE INVERSE O F THE JACOBIAN OF THE KARUSH-KUHN-TUCKER SYSTEM

7.3.

I n C h a p t e r 4, f o r m u l a s f o r t h e b l o c k components o f t h e i n v e r s e o f t h e J a c o b i a n M o f t h e f i r s t - o r d e r n e c e s s a r y condit i o n s (3.2.2)

f o r a s o l u t i o n of t h e g e n e r a l problem P ( E ) w e r e

o b t a i n e d , i n terms o f t h e f u n c t i o n s and d e r i v a t i v e s o f t h e f u n c t i o n s d e f i n i n g t h e problem.

T h i s was done i n t h e c o u r s e

of developing expressions f o r t h e s o l u t i o n V € Y ( E )

of t h e

system M(E)V~Y(E) = N(E)

(3.2.3)

t h a t r e s u l t e d from t h e d i f f e r e n t i a t i o n of t o t h e parameter

E

(3.2.2) w i t h r e s p e c t

a t t h e Karush-Kuhn-Tucker

t r i p l e ~ ( € 1 . In

C h a p t e r 4 i t was n o t e d t h a t t h e e l i m i n a t i o n o f nonbinding cons t r a i n t i n f o r m a t i o n and an a p p r o p r i a t e f a c t o r i z a t i o n l e d t o a r e d u c t i o n (4.2.4)

of

(3.2.31,

so t h a t i t was a c t u a l l y o n l y

n e c e s s a r y t o c a l c u l a t e t h e b l o c k components A i j of a reduced m a t r i x

h ( ~d)e r i v e d

of t h e inverse

from M ( E ) and g i v e n i n (4.2.6).

I n t h i s s e c t i o n w e give expressions f o r t h e Aij

in terms

of a l g o r i t h m i c c a l c u l a t i o n s a s s o c i a t e d with procedures based on t h e s t r a t e g y o f m a i n t a i n i n g t h e s a t i s f a c t i o n , up t o a f i r s t o r d e r e s t i m a t e , o f t h e c o n s t r a i n t s t h a t a r e c u r r e n t l y deemed binding.

W e f i r s t g i v e g e n e r a l r e s u l t s , f o l l o w e d by s p e c i a l i -

z a t i o n s t o r e d u c e d - g r a d i e n t and p r o j e c t e d - g r a d i e n t c a l c u l a tions.

W e f o l l o w t h e development g i v e n p r e v i o u s l y by t h e

author (Fiacco, 1980a).

W e c o n f i n e o u r a t t e n t i o n t o t h e Case

4 s i t u a t i o n of C h a p t e r 4, S e c t i o n 4.2, where t h e number of b i n d i n g i n e q u a l i t y c o n s t r a i n t s r and e q u a l i t y c o n s t r a i n t s p i s

less t h a n t h e number of v a r i a b l e s n.

202

7 Calculation of Sensitivity Using Other Algorithms

T h i s i s t h e l e a s t s t r u c t u r e d , and hence m o s t g e n e r a l , s i t u a t i o n t h a t c a n be e n c o u n t e r e d u n d e r t h e g i v e n a s s u m p t i o n s . Many r e p r e s e n t a t i o n s of d a t a are o r g a n i z e d .

fi-'

a r e p o s s i b l e , d e p e n d i n g on how t h e

However, a g e n e r a l r e p r e s e n t a t i o n t h a t i s

t a i l o r e d t o t h e a s s u m p t i o n s w e are making h e r e w a s o b t a i n e d by McCormick ( 1 9 7 7 ) and w i l l s e r v e o u r p u r p o s e e x t r e m e l y a p t l y . W e f o l l o w h i s development h e r e r a t h e r c l o s e l y .

F i r s t , n o t e t h a t t h e second-order s u f f i c i e n t c o n d i t i o n s and s t r i c t complementary s l a c k n e s s , a s s u m p t i o n s ( i i )and ( i v )

*

o f Theorem 3.2.2,

imply t h a t , a t ( x

S i s any n by

VxhTIT, a n ( r

[n

-

u

*

,w

2 zTVxLz > 0 for all z

must have (4.2.10) : where P = [-Vx7jT,

,

+

*

1, with

we

E = 0,

# 0 s u c h t h a t Pz

p ) by n m a t r i x .

= 0,

Hence, i f

(r + p ) ] matrix t h a t generates t h e n u l l

s p a c e o f P , t h e n z = Sv f o r some v i n E n - ( r + P )

implies t h a t

Pz = 0 ( s i n c e PS = 0) and a l s o t h a t z # 0 i f v # 0 ( s i n c e w e

are t a c i t l y assuming t h a t S h a s f u l l column r a n k [ n Thus, zTV2Lz = v TSTV 2 LSv > 0 , p r o v i d i n g t h a t v # 0. X

T 2 c l u d e t h a t D = S V LS i s a p o s i t i v e d e f i n i t e [n

[n

X

-

(r

+

p ) ] matrix.

-

Further, s i n c e P has rank r

-

(r

+

( r + PI]).

W e con-

+ p ) ] by p , by t h e

l i n e a r i n d e p e n d e n c e a s s u m p t i o n , a s s u m p t i o n (iii) o f Theorem 3.2.2,

an n by ( r

rank ( r

+

+

p ) pseudo-inverse

p ) m a t r i x P'

s a t i s f y i n g PP'P

of P e x i s t s , i.e., = P.

a

These c o n s t r u c t s

and o b s e r v a t i o n s l e a d t o a g e n e r a l r e p r e s e n t a t i o n of t h e b l o c k components

of

2-l.

I t i s assumed t h a t P# and S a r e g e n e r a t e d by some m a t r i x

t e c h n i q u e which r e l a t e s t h e s e q u a n t i t i e s by t h e e x p r e s s i o n I

-

P'P

=

s i n c e PP'

SW, where W i s some [ n = I

-

and Vxf(x*, O ) T = -P

(r T

+

p ) ] by n m a t r i x .

* ( u , w * ) a~ t o p t i m a l i t y

Also,

1.3 Inverse of the Jacobian of the Karush-Kuhn-Tucker System

203

it may be o f i n t e r e s t t o n o t e t h a t

This m o t i v a t e s t h e widely used e s t i m a t i o n

- (P#)TVxfTf o r

the

Lagrange m u l t i p l i e r s i n a l g o r i t h m s i n v o l v i n g t h e s e c o n s t r u c t s . of

The r e s u l t , i n t h e form o f t h e b l o c k components A i j

h-’,

f o l l o w s r e a d i l y a n d i s g i v e n by -1 T

,

All

= SD

S

A12

= Afl

= [I

A 2 2 = -A

V 2 LP # 21 x

-

(7.3.1)

A~~v:LJP#,

.

T h e r e a r e many good t e c h n i q u e s c u r r e n t l y a v a i l a b l e f o r c a l c u l a t i n g S a n d P # , m o t i v a t e d by v a r i o u s n u m e r i c a l e f f i c i e n c y , s t a b i l i t y and a l g o r i t h m i c c o n s i d e r a t i o n s .

W e mention

two h e r e f o r c o m p l e t e n e s s and b e c a u s e t h e y a r e p r e c i s e l y t a i l o r e d t o t h e c a l c u l a t i o n s a s s o c i a t e d w i t h two i m p o r t a n t f a m i l i e s o f m a t h e m a t i c a l programming a l g o r i t h m s , reducedg r a d i e n t and p r o j e c t e d - g r a d i e n t

type algorithms.

The f i r s t t e c h n i q u e i s a s s o c i a t e d w i t h t h e r e d u c e d - g r a d i e n t

o r v a r i a b l e - r e d u c t i o n t y p e a l g o r i t h m s f o r n o n l i n e a r programming, a n d i s a c r u c i a l p a r t o f t h e s i m p l e x method f o r l i n e a r programming.

I t i s b a s e d on t h e s i m p l e o b s e r v a t i o n t h a t t h e

l i n e a r i n d e p e n d e n c e a s s u m p t i o n i m p l i e s t h e e x i s t e n c e o f an

(r

+

p ) by ( r

+

p) nonsingular submatrix P

for simplicity t h a t the f i r s t r

+

a

o f P.

p columns o f P are l i n e a r l y

i n d e p e n d e n t , w e c a n p a r t i t i o n P a s P = [Pa, Pb] an ( r

+

p ) by [n

-

(r

+

Assuming

p ) ] matrix.

decomposition o f t h e v a r i a b l e x = ( x

, where

Pb i s

This induces a n a t u r a l

x ~ ) and ~ , since

204

7 Calculation of Sensitivity Using Other Algorithms

* g(x ,

*

0 ) = 0 and h ( x

,

0 ) = 0 , a l l o w s a p p l i c a t i o n of t h e i m -

p l i c i t f u n c t i o n theorem t o conclude t h a t t h e r e e x i s t s a t w i c e

,

d i f f e r e n t i a b l e v e c t o r f u n c t i o n xa ( x b ) s u c h t h a t g [ x a (x,)

= x

*

.

xb,

The xa and xb may be t h o u g h t o f as " d e p e n d e n t " a n d

"independent" v a r i a b l e s , r e s p e c t i v e l y .

Once t h e b i n d i n g con-

s t r a i n t s a r e i d e n t i f i e d it s u f f i c e s t o m i n i m i z e f [ x a ( x b ) , xb, 01 o v e r xb u s i n g a n y a p p r o p r i a t e u n c o n s t r a i n e d method,

*

d e t e r m i n e xb, a n d h e n c e x

*

.

to

The i n d i c a t e d a l g o r i t h m s a c t u a l l y

i n v o k e t h e l i n e a r i n d p e n d e n c e a s s u m p t i o n f o r aZZ f e a s i b l e boundary p o i n t s , a n d h e n c e a t any g i v e n i t e r a t i o n c a n e i t h e r reduce f ( x ) wit h o u t e n c o u n t e r i n g c o n s t r a i n t s or w i l l be i n a s i t u a t i o n completely analogous t o t h e one d e s c r i b e d a t t h e o u t s e t , a n d c a n p r o c e e d t o minimize f o v e r t h e c u r r e n t l y i n d e p e n d e n t v a r i a b l e s i n t h e s p a c e of c u r r e n t l y b i n d i n g constraints. R e t u r n i n g t o t h e d e t e r m i n a t i o n o f S and P# f o r t h i s t y p e o f a l g o r i t h m , w e observe t h a t S : IS:, PS = [Pa, P b l [ s , l = 0, -1

so Sa = -Pa PbSb a n d h e n c e

S i m i l a r l y , s i n c e PP'P

= PI defining

must s a t i s f y

205

7 . 3 Inverse of the Jacobian of the Karush-Kuhn-Tucker System

gives the result p# =

[

p2 - pIp2)]

=

[

+

[-'rPb]

p2'

I n t e r m s o f t h e q u a n t i t i e s d e f i n e d , t h e b l o c k components of

Aij

fi-'

a r e t h e r e f o r e g i v e n by ( 7 . 3 . 1 ) , where

S = TSb, P # =

[;'I

+

TP2,

and

a n n by [ n any t n [n

-

-

-

(r

( r + p ) ] m a t r i x w i t h r a n k [n

-

( r + p ) ] , Sb i s

+ p ) ] s q u a r e n o n s i n g u l a r m a t r i x , and P2 i s any

( r + p ) ] by ( r

+

p) matrix.

For t h e projected-gradient type algorithms t h e g r a d i e n t s

of b i n d i n g c o n s t r a i n t s a r e a g a i n assumed l i n e a r l y i n d e p e n d e n t a t f e a s i b l e boundary p o i n t s , w i t h t h e d a t a b e i n g o r g a n i z e d m a i n l y t o accommodate a p r o j e c t i o n m a t r i x o f t h e form, PR = I

-

PT(PPT)-'P,

used t o p r o j e c t a given d i r e c t i o n v e c t o r i n t o

t h e l i n e a r s u b s p a c e a s s o c i a t e d w i t h t h e c u r r e n t l y b i n d i n g constraints.

Here, t h e rows o f P would r e p r e s e n t t h e g r a d i e n t s

o f t h e c o n s t r a i n t s c u r r e n t l y deemed t o r e m a i n b i n d i n g i n t h e n e x t i t e r a t i o n , and PPT i s a n ( r

+

p ) by ( r

+

p ) m a t r i x , non-

s i n g u l a r under t h e l i n e a r independence assumption. t h a t P# = PT(PPT)-'

We find

s a t i s f i e s t h e r e q u i r e m e n t s f o r a pseudo-

i n v e r s e o f P , a n d it f o l l o w s e a s i l y t h a t a s u i t a b l e c h o i c e f o r S i s any m a t r i x SR formed by s e l e c t i n g a n y [n

-

(r

+ p)l

l i n e a r l y i n d e p e n d e n t columns of PR. The b l o c k components of 1 M f o r g r a d i e n t - p r o j e c t i o n type c a l c u l a t i o n s are t h e r e f o r e T -1 t h e same as ( 7 . 3 . 1 ) , w i t h S = SR and P# = PT(PP ) A-

.

206 7.4.

7 Calculation of Sensitivity Using Other Algorithms

SENSITIVITY RESULTS FOR AUGMENTED LAGRANGIANS R e c e n t l y , t h e augmented L a g r a n g i a n a p p r o a c h h a s b e e n viewed

more and more i n t h e s p i r i t o f a Quasi-Newton method r e l a t e d

t o i t e r a t i o n s of t h e g e n e r a l form ( 7 . 2 . 1 ) .

Thus, w e may an-

t i c i p a t e a c l o s e c o n n e c t i o n between t h e a l g o r i t h m i c c a l c u l a t i o n s and t h e s e n s i t i v i t y c a l c u l a t i o n s ( 3 . 2 . 3 1 , i n t h e l a s t two s e c t i o n s .

as i n d i c a t e d

A t h e o r e t i c a l connection i s r e a d i l y

e s t a b l i s h e d w i t h o u r p r i o r r e s u l t s , w i t h t h e augmented Lagrangian p l a y i n g t h e role of t h e u s u a l Lagrangian.

W e con-

clude t h i s chapter with s e n s i t i v i t y r e s u l t s f o r a typical augmented L a g r a n g i a n f u n c t i o n .

The Karush-Kuhn-Tucker

param-

e t e r d e r i v a t i v e s t u r n o u t t o be e x a c t f o r t h i s c l a s s o f a l g o r i t h m s , i f t h e o p t i m a l Lagrange m u l t i p l i e r s a r e known. T h i s d e v e l o p m e n t i s b a s e d l a r g e l y o n t h a t g i v e n by A r m a c o s t and Fiacco (1977). P e n a l t y f u n c t i o n a l g o r i t h m s , s u c h as t h e l o g a r i t h m i c b a r r i e r q u a d r a t i c - l o s s p e n a l t y f u n c t i o n s t u d i e d i n Chapter 6, b e l o n g t o a g e n e r a l c l a s s o f a l g o r i t h m s whereby t h e c o n s t r a i n e d p r o b l e m i s t r a n s f o r m e d i n t o a s e q u e n c e o f u n c o n s t r a i n e d prob-

l e m s by means o f a n a u x i l i a r y f u n c t i o n .

M o t i v a t e d i n p a r t by

an e f f o r t e i t h e r t o " r e g u l a r i z e " or "convexify" t h e u s u a l L a g r a n g i a n o r t o overcome t h e now w e l l u n d e r s t o o d and w e l l documented i l l - c o n d i t i o n i n g t y p i c a l l y a s s o c i a t e d w i t h t h e c l a s s i c a l p e n a l t y f u n c t i o n p r o c e d u r e s a s a s o l u t i o n i s app r o a c h e d , o t h e r t y p e s o f a u x i l i a r y f u n c t i o n s h a v e more r e c e n t l y received considerable attention.

These i n c l u d e " e x a c t p e n a l t y

f u n c t i o n s " and " g e n e r a l i z e d Lagrangians"

[ A r r o w , Gould, a n d

Howe (1973) ] a n d "augmented L a g r a n g i a n s " [ H e s t e n e s ( 1 9 6 9 ) a n d Buys ( 1 9 7 2 ) I .

207

7.4 Sensitivity Results for Augmented Lagrangians

G e n e r a l i z e d L a g r a n g i a n s g e n e r a l l y r e f e r t o more g e n e r a l forms o f t h e u s u a l Lagrangian a s s o c i a t e d w i t h problem P: minx f ( x ) s . t . g ( x ) 1. 0 , h ( x ) = 0 .

T y p i c a l l y , f , gi,

a r e r e p l a c e d by f u n c t i o n s o f t h e s e q u a n t i t i e s .

or w

h j , ui,

j

The o b j e c t i s

t o s t r u c t u r e t h e new f u n c t i o n so t h a t it behaves " l i k e " a Lagrangian (e.g.,

i n characterizing optimality) but acquires

c e r t a i n des irable p r o p e r t i e s (e.g.,

convexity o r strict

c o n v e x i t y ) t h a t t h e u s u a l Lagrangian d o e s n o t p o s s e s s .

A

p o p u l a r e x t e n s i o n i n t r o d u c e s g e n e r a l i z e d Lagrange m u l t i p l i e r s t h a t may a l s o be f u n c t i o n s of x.

Augmented Lagrangians a r e

u s u a l l y formed by a d d i n g a p e n a l t y term t o t h e u s u a l L a g r a n g i a n , though t h i s c l a s s can e a s i l y be e n l a r g e d by u s i n g t h e g e n e r a l i z e d L a g r a n g i a n s d e s c r i b e d e a r l i e r i n t h e paragraph. The m o t i v a t i o n was a p p a r e n t l y f i r s t " c o n v e x i f i c a t i o n , " t h e n a way o f overcoming i l l - c o n d i t i o n i n g and a l s o a way o f developi n g an " e x a c t " method, a s d e s c r i b e d i n t h e n e x t p a r a g r a p h .

In

t h i s c o n t e x t , t h e t e r m "Method o f M u l t i p l i e r s ' ' r e f e r s t o a p a r t i c u l a r a p p r o a c h u s i n g a p a r t i c u l a r form o f an augmented Lagrangian.

The t e r m i s due t o Hestenes ( 1 9 6 9 ) , who proposed

an a l g o r i t h m b a s e d on s e q u e n t i a l l y improving t h e e s t i m a t e s of t h e Lagrange m u l t i p l i e r s .

S i m i l a r i d e a s w e r e a l s o proposed by

Powell ( 1 9 6 9 ) and o t h e r s a t a b o u t t h e same t i m e . A p e n a l t y f u n c t i o n would be c o n s i d e r e d e x a c t i f , f o r p a r t i c u l a r v a l u e s o f c e r t a i n p a r a m e t e r s , an u n c o n s t r a i n e d ( l o c a l ) minimum o f t h e f u n c t i o n ( l o c a l l y ) s o l v e s t h e g i v e n programming problem.

S i n c e t h e " o p t i m a l v a l u e s " of t h e s e

p a r a m e t e r s a r e g e n e r a l l y unknown, t h e y must be e s t i m a t e d , and s i n c e t h e s e p a r a m e t e r s o f t e n c o r r e s p o n d t o Lagrange m u l t i p l i e r s , t h e r e s u l t i s t h a t augmented Lagrangian and e x a c t penalty function algorithms a r e q u i t e s i m i l a r i n s p i r i t .

208

7 Calculation of Sensitivity Using Other Algorithms

I t i s much e a s i e r t o d e a l w i t h e q u a l i t y c o n s t r a i n t s i n

t h e s e methods.

When c o n s i d e r i n g i n e q u a l i t y c o n s t r a i n t s ,

p e n a l t y f u n c t i o n s may b e i n t r o d u c e d f o r which t h e same o r d e r of d i f f e r e n t i a b i l i t y i s n o t i n h e r i t e d from t h e problem functions.

Also,

s e r i o u s c o m p l i c a t i o n s arise i n c o n s t r u c t i n g

e f f e c t i v e s t r a t e g i e s a s s o c i a t e d w i t h t h e i d e n t i f i c a t i o n of t h e binding inequality constraints. Arrow e t a l .

(1973) developed a s a d d l e - p o i n t t h e o r y f o r an

"extended" L a g r a n g i a n , d e f i n i n g m u l t i p l i e r f u n c t i o n s w i t h certain limiting properties.

R e a l i z a t i o n s of t h e i r extended

Lagrangian y i e l d many o f t h e augmented L a g r a n g i a n s and e x a c t p e n a l t y f u n c t i o n s found i n t h e l i t e r a t u r e .

As i n d i c a t e d , a p r i m a r y m o t i v a t i o n f o r t h e s t u d y of augmented L a g r a n g i a n s and e x a c t p e n a l t y f u n c t i o n s h a s been t o overcome t h e problems o f i l l - c o n d i t i o n i n g a s s o c i a t e d w i t h ordinary penalty functions.

Because of t h i s , c o n s i d e r a b l e

e f f o r t h a s been made t o i n v e s t i g a t e t h e c o m p u t a t i o n a l a s p e c t s of t h e s e a l g o r i t h m s [ e . g . ,

Bertsekas (1975), Rockafellar

( 1 9 7 3 ) , and Rupp ( 1 9 7 6 ) J .

Buys ( 1 9 7 2 ) p r o v i d e d p e r h a p s t h e

e a r l i e s t b e s t d e t a i l e d a n a l y s i s o f a d u a l a p p r o a c h , u s i n g an augmented Lagrangian f i r s t i n t r o d u c e d by R o c k a f e l l a r ( 1 9 7 1 ) . Buys and Gonin ( 1 9 7 7 ) and Armacost and F i a c c o (1977) used t h e same approach t o o b t a i n s e n s i t i v i t y a n a l y s i s r e s u l t s i n t e r m s o f t h i s augmented Lagrangian f o r m u l a t i o n .

S i n c e o u r i n t e n t is

t o address t h e s e n s i t i v i t y r e s u l t s , f u r t h e r descriptions of the computational approaches a r e n o t pursued. For s i m p l i c i t y , t h e s p e c i f i c augmented Lagrangian used by Buys and Gonin (1977) w i l l be u t i l i z e d h e r e .

I t i s noted,

however, t h a t o u r approach i s d i r e c t l y a p p l i c a b l e t o t h e

209

7.4 Sensitivity Results for Augmented Lagrangians

g e n e r a l form o f t h e e x t e n d e d Laqrangian g i v e n by Arrow e t a l . , t h u s encompassing a l a r g e f a m i l y o f augmented L a g r a n g i a n s and exact penalty functions.

W e a d d r e s s problem P 3 ( ~ ) .

R a t h e r t h a n u s e t h e d u a l approach o f Buys, t h e s e n s i t i v i t y r e s u l t s t h a t o b t a i n u s i n g augmented L a q r a n g i a n s f o l l o w d i r e c t l y by c o n s i d e r i n g t h e e q u a t i o n s t h a t a r e s a t i s f i e d a t a solution point.

The key p o i n t i n t h e f o l l o w i n g development i s

t h a t t h e g r a d i e n t o f t h e augmented Lagrangian i s e q u a l t o t h e g r a d i e n t o f t h e o r d i n a r y Laqranqian n e a r a s o l u t i o n p o i n t , when t h e problem p a r a m e t e r

E

i s perturbed.

L e t c > 0 be a c o n s t a n t , J

i = 1,

...,m }

-

and K E { i l ui

-

{ i l ui cgi(x,

E)

cqi(x,

E)

2 0,

< 0 , i = 1,

As u s u a l , w e suppose t h e a s s u m p t i o n s o f Theorem 3.2.2 problem P 3 ( ~ ) . L e t x

*

{ i l q i ( x * , 0 ) = 0 , i = 1, ui

-

-

c g i ( x * , 0 ) = u;

hold f o r

be a l o c a l minimum o f problem P 3 ( 0 )

w i t h a s s o c i a t e d Laqrange m u l t i p l i e r s u

*

..., m}.

..., m).

*

-

*

..., m

*

.

Then, f o r i

> 0 and f o r i = 1,

qi ( x * , 0 ) > 0 and hence u;

and w

L e t B (0) =

B*(O),

E

f B*(O),

cgi ( x * , 0) = - c g i ( x

*

,

0) < 0.

Thus, a t t h e s o l u t i o n p o i n t , J i s d e f i n e d w i t h s t r i c t i n e q u a l i t y and c o r r e s p o n d s t o B * ( O ) , and K c o n t a i n s a l l i such t h a t qi(x*,

0) > 0 .

The augmented Lagranqian f o r problem Pj ( E ) Section 2.4,

,

as

in

is defined a s

+

7 P

(wj

+

(1/2)ch.(x, E ) ) h . ( x , 3

j=l

-

(1/2c)

2

2

Ui'

7

E)

(7.4.1)

210

7 Calculation of Sensitivity Using Other Algorithms

The g r a d i e n t o f @ t a k e n w i t h r e s p e c t t o x i s h

VxL(x, u, w ,

c) =

E,

v XL

+ ieJ

P

2

chjVxhj.

*

*

+

(7.4.2)

j=l At

(x, u, w ,

*

c) = (x

E,

, u , w , 0,

c ) , s i n c e g.(x*, 0) = 0, 1

i e J , and h . ( x * , 0) = 0 €or a l l j , i t f o l l o w s t h a t

I

A

*

VXL(X

,

u

*

*

,w ,

0 , c) = VXL(X

*

,

u

*

* ,w ,

(7.4.3)

0).

The augmented L a g r a n g i a n i s t w i c e c o n t i n u o u s l y d i f f e r e n t i a b l e

-

i n x, e x c e p t a t p o i n t s where ui t i o n ( i v ) of Theorem 3 . 2 . 2 , and hence

*

near (x

,

2 is u

*

,

ui

*

cgi(x, cgi(x

*

E)

,

= 0.

By assump-

0) # 0 f o r a l l i,

t w i c e c o n t i n u o u s l y d i f f e r e n t i a b l e f o r (x, u, 0).

Differentiating (7.4.2)

E)

with respect to x

yields 2*

VXL(X,

u, w,

E,

2

c) = VXL + ieJ

2 P

+

ieJ 2

ch.V h 1 x 1

j=l

+

(7.4.4)

cVxhTVxhj. j=1

Again, a t (x, 2 A *

VXL(X

Ur

w,

, u* , w * ,

*

E,

c ) = (x

* * ,u ,w ,

0 , c) ,

*T

0, c ) =

*

ieJ

+

2 P

c

j=l

*T * Vxhj Vxhj,

(7.4.5)

21 1

7.4 Sensitivity Results for Augmented Lagrangians

*

where t h e e x p o n e n t

at

* * * (x , u , w , *

J = B

(0) = 11,

denotes evaluation of t h e given functions W i t h o u t loss o f g e n e r a l i t y , assume t h a t

0).

..., r ) .

Then, r e c a l l i n g t h a t

it follows t h a t 26 2 VXL = VXL

T

+

CP P.

(7.4.6)

T 2 * S i n c e z VxL z > 0 f o r a l l z # 0 s u c h t h a t Pz = 0 , by a s s u m p t i o n s ( i i )a n d ( i v ) o f Theorem 3.2.2,

then f o r a l l c

26 s u f f i c i e n t l y l a r g e i t f o l l o w s e a s i l y t h a t VxL i s p o s i t i v e

*

d e f i n i t e n e a r (x

c

*

,

u

> 0 such t h a t f o r

*

* , w , 0 , c ) . Thus, t h e r e i s a number * 2 n * * * c > c , VxL(x , u , w , 0 , c ) i s p o s i t i v e *

d e f i n i t e (and hence n o n s i n g u l a r ) .

Assume t h a t c > c

and a s

b e f o r e , t h a t t h e f i r s t r i n e q u a l i t y c o n s t r a i n t s are assumed b i n d i n g , t h e s u p e r b a r i n d i c a t i n g e v a l u a t i o n a t i = l , . . . , r. For convenience w e s h a l l u s e a b a r underscore t o denote evaluation a t i = r

+

l , . . . , m.

The main r e s u l t c a n b e s t a t e d as

follows. [ S e n s i t i v i t y r e s u l t s u s i n g a n augmented

Theorem 7 . 4 , l .

Lagrangian f o r problem P ( ~ 1 . 1 I f t h e a s s u m p t i o n s o f Theorem 3.2.2 then f o r

E

n e a r 0 and c > c

*

h o l d f o r p r o b l e m P3(€),

t h e r e e x i s t s a u n i q u e , o n c e con-

tinuously d i f f e r e n t i a b l e vector function y a ( € , c) =

-u ( E ,

c), n

w(E,

VXL(X,

c),

u, w,

g(E, E,

uigi(x, h.(x, 3 uigi(x,

[x(E,

c),

c)lT satisfying

c ) = 0, E)

= 0,

i = 1,

E)

= 0,

j = 1

E)

= 0,

..., r ,

,..., P , i = r + 1, ..., m,

(7.4.7) (7.4.8 (7.4.9 (7.4.10

212

7 Calculation of Sensitivity Using Other Algorithms

with

c),

[x(E,

c),

U(E,

( t h e Karush-Kuhn-Tucker f o r any

[x(E),

u(E),

t r i p l e of Theorem 3.2.2)

n e a r 0 and c > c

E

c ) l T=

W(E,

*

,

A

E,

c ] and VxL[x, u, w,

*

n e a r (x

*

,u ,w

*

u),

(7.4.7)

-VxgT

VxhT

G

0

0

Wr

gr

= (X

C)

E,

V?:* :M

u

r

r

w

*

*

I

0

, is

..., r

..., m.

and

G

=

E v a l u a t e d a t (x, U,

2 , 0, c ) , Ma becomes

-Vx?J*T Vxh*T

=

taken

G

d i a g ( u i ) , i = r + 1,

* - *

7.4.10)

system (3.2.2)

= d i a g ( g i ) , U = d i a g ( u i ) , i = 1,

1=

-

-Vx9

-

diag(gi),

c) ,

W(E,

t h e p r e c i s e analogy of t h e

J a c o b i a n M ( E ) o f t h e Karush-Kuhn-Tucker

where

c),

).

w i t h r e s p e c t t o (x, U, w ,

V?:

U(E,

cl i s p o s i t i v e d e f i n i t e f o r (x, u , w )

E,

The J a c o b i a n m a t r i x o f

Proof.

and s u c h t h a t

c ) i s a l o c a l l y u n i q u e un-

X(E,

c o n s t r a i n e d l o c a l m i n i m i z i n g p o i n t of L[x, 2"

w ( E ) ]: ~Y ( E )

0

0

0

0

0

0

-Vxg (7.4.11)

0

G

Under t h e g i v e n a s s u m p t i o n s i t f o l l o w s t h a t MZ-'

e x i s t s and

h e n c e , by t h e i m p l i c i t f u n c t i o n theorem, t h e r e e x i s t s a u n i q u e , once c o n t i n u o u s l y d i f f e r e n t i a b l e v e c t o r f u n c t i o n y [X(Er

c ) r

-

U(Er

c ) r

W(E,

c),

U(Erc)lT

(E,

s a t i s f y i n g (7.4.7)

c) =

-

n e a r 0 a n d c > c*. 2A AS i n d i c a t e d above, V x ~i s p o s i t i v e d e f i n i t e a t , a n d h e n c e

(7.4.10) near ,

x

for

*

,

u

E

*

* ,w ,

0, c ) .

I t follows t h a t

A

u n i q u e l o c a l minimum of L[x, n e a r 0 and c > c

*

.

U(E,

c),

W(E,

X(E,

c) is a l o c a l l y

c ) , E, cl f o r

E

213

7.4 Sensitivity Results for Augmented Lagrangians

Observing t h a t near

E

= 0,

Vxt

V L at [x(E, c), U(E, X

a comparison o f (7.4.7)

Kuhn-Tucker s y s t e m ( 3 . 2 . 2 1 ,

u(E),

w ( E ) ]for

E

(7.4.10)

c),

[x(E,

h

c),

W(E,

c)l f o r

E

c)]

and t h e Karush-

U(E,

c),

W(E,

c)]

n e a r 0 , and t h e proof i s complete.

I t may a l s o b e n o t e d i n p a s s i n g t h a t L U(E,

W(E,

and t h e u n i q u e n e s s o f t h e s o l u -

t i o n s of both systems, implies t h a t [x(E),

-

c),

n e a r 0.

f a t [x(E, c),

Along w i t h t h e c o n c l u s i o n s

given i n t h e l a s t p a r t of t h e p r o o f , t h i s immediately i m p l i e s t h e r e s u l t s t h a t p r e c i s e l y p a r a l l e l t h o s e g i v e n i n Theorem f o r t h e o p t i m a l v a l u e f u n c t i o n and i t s f i r s t and s e c o n d

3.4.1

A

d e r i v a t i v e s , where t h e augmented L a g r a n g i a n L r e p l a c e s t h e u s ua l Lagrangian L i n t h e given e x p r e s sio n s . S i n c e t h e system (7.4.7)

-

(7.4.10)

i s once continuously

d i f f e r e n t i a b l e and i d e n t i c a l l y e q u a l t o z e r o f o r

c > c

*

,

Ma(€,

i t c a n be d i f f e r e n t i a t e d w i t h r e s p e c t t o c)VEya(~c , ) = Na(€,

where M a ( € ,

v ~a y

and

(E,

c) is defined f o r

c)

=

c) E

n e a r 0 and c > c

E

n e a r 0 and

E

to y i e l d (7.4.12)

*

,

214

7 Calculation of Sensitivity Using Other Algorithms

N o t i c e f i r s t t h a t (7.4.12) since

G

L(E,

c) E 0 f o r i

-N a ( € ,

y i e l d s ~ V € U ( E , c ) = 0 , and

> 0 i t f o l l o w s t h a t V E ~ ( € , c) = 0 , as expected,

{

B*(O).

c ) b e t h e p o r t i o n s of

inequality constraints.

Let

since

Z a ( € , c ) , VETa(€, c ) , a n d exclu d in g t h e nonbinding

(7.4.12)

Then w i t h

and l e t t i n g

it follows t h a t

Letting (7.4.13) w e have t h a t

The e l e m e n t s C i j

Recall t h e computations f o r t h e g e n e r a l

given i n Chapter 4. p r o b l e m when V$-' i d e n t i c a l , w i t h; : V Cll

= VXL 2

C12

=

c21 T

yI =

are i m m e d i a t e l y d e t e r m i n e d u s i n g a r e s u l t

e x i s t s (4.2.8). 2 r e p l a c i n g VxL,

-

The s i t u a t i o n h e r e i s and w e o b t a i n

P'( P v y P T j 1 P v ; i - l ) ,

v y P T ( P v x 2A L - lpT)-'

r

(7.4.14)

215

7.4 Sensitivity Results for Augmented Lagrangians

Following t h e development i n Chapter 4 , S e c t i o n 4.2 p a r t i c u l a r , u s i n g (4.2.1511,

[in

t h e estimate o f t h e f i r s t - o r d e r

s e n s i t i v i t y o f t h e n o n t r i v i a l components o f a Karush-KuhnT u c k e r t r i p l e f o r problem P 3 ( ~ )i s g i v e n by

-VEh

.

(7.4.15)

-VEh c ) :Y ( E ) n e a r

r

V E y ( ~ f) o r

E

(E, c) ~a The s t r u c t u r e of t h e o r i g i n a l p r o b l e m

n e a r 0.

E

= 0 , w e must h a v e V y

a n d t h e augmented L a g r a n g i a n c a n b e u s e d t o d e m o n s t r a t e exp l i c i t l y t h a t V E y a ( ~ ,c ) : V E y ( ~ ) . R e c a l l t h a t f o r t h e g i v e n p r o b l e m P 3 ( ~,)

where

a n d t h e components A i j (4.2.13)

,

a r e d e f i n e d by ( 4 . 2 . 8 ) ,

d e p e n d i n g on which c o n d i t i o n s a p p l y .

D i f f e r e n t i a t i n g (7.4.2) with r e s p e c t t o

v 2E XAL

(4.2.9),

r

= VEXL 2

+

+

c

1

i=l

j=1

yields

r

9iv:x9i

h.V 3 2E Xh 3.

c

E

+

+

c

c

1

j=1

1

i=l

Vx9TVEgi

VxhTVEhj.

or

!

7 Calculation of Sensitivity Using Other Algorithms

216

T h e s e and t h e f o l l o w i n g e q u a t i o n s are e v a l u a t e d a t y a ( € , c ) for

E

n e a r 0 , and hence, qi[x(E,

hj[x(E, c ) ,

= 0 , j = 1,

E]

..., p.

c),

E]

= 0,

i

E

*

J = B (0) a n d

This y i e l d s

v2

VEXL 2 " =

L

-

CpT[

EX

vE'],

-VEh

w i t h P d e f i n e d as b e f o r e .

An e x p l i c i t e x p r e s s i o n f o r VEya(E, c )

c a n now b e d e r i v e d , which s i m p l i f i e s (7.4.15). 5 1v2E X2 = c l l v ~ x L

-

ccllpT[-;:;],

a n d s i n c e (7.4.14) i m p l i e s t h a t CllPT

= 0,

i t follows t h a t

cllv;x2

= cllv;xL.

Similarly, 2 "

-VEh a n d s i n c e (7.4.14) i m p l i e s t h a t CZIPT = I ,

we obtain

Since

217

7.4 Sensitivity Results for Augmented Lagranpians

The f i r s t - o r d e r s e n s i t i v i t y o f t h e Karush-Kuhn-Tucker

triple

may t h e n be w r i t t e n a s

=

for

E

[ i . &

c1-1 + [O0

ol]i,(E1

C I

(7.4.16 1

n e a r 0.

S i n c e i t was p r e v i o u s l y concluded t h a t V E f T ( ~ ) i V ~a 7 for

E

n e a r 0 and c > c

*

,

(E,

c)

i t must b e t r u e t h a t

(7.4.17) To show e x p l i c i t l y t h a t t h i s r e l a t i o n s h i p h o l d s , n o t e t h a t (definition)

( u s i n g (7.4.6) 1

P

= (I

+

0

cPT

(0

0

I)[

V2L P

PT 0 1

218

7 Calculation of Sensitivity Using Other Algorithms

P r e m u l t i p l y i n g t h e l a s t e q u a t i o n by k a ( c , c ) - l a n d p o s t m u l t i p l y i n g by f i ( ~ 1 - l y i e l d s 7.5.

(7.4.17).

CONCLUSIONS AND EXTENSIONS Under t h e g i v e n c o n d i t i o n s , t h e e q u a t i o n s ( 7 . 4 . 1 4 )

and

( 7 . 4 . 1 5 ) o b t a i n e d u s i n g t h e augmented L a g r a n g i a n may b e viewed

as a n a l t e r n a t i v e c o m p u t a t i o n a l d e v i c e f o r c a l c u l a t i n g t h e f i r s t - o r d e r s e n s i t i v i t y o f a Karush-Kuhn-Tucker problem P 3 ( f ) , e n c o m p a s s i n g ( 4 . 2 . 1 5 ) equations (4.2.8),

(4.2.9),

v a r i o u s cases s p e c i f i e d .

triple for

together with the

and (4.2.13)

t h a t apply i n t h e

T h a t i s , u s i n g t h e augmented

L a g r a n g i a n i n s t e a d of t h e u s u a l L a g r a n g i a n , o n l y Case 1 ( 4 . 2 . 8 ) can a r i s e , and, hence, o n l y t h e one set o f e q u a t i o n s ( 7 . 4 . 1 4 ) f o r t h e i n v e r s e o f t h e J a c o b i a n i s needed u n d e r t h e g i v e n conditions.

Note t h a t b o t h a p p r o a c h e s r e q u i r e knowledge o f

t h e b i n d i n g i n e q u a l i t y c o n s t r a i n t i n d i c e s and i n f a c t r e q u i r e

t h e d e t e r m i n a t i o n o f a Karush-Kuhn-Tucker (7.4.14)

definite.

triple.

In addition,

2

r e q u i r e s a v a l u e o f c f o r which V X L i s p o s i t i v e Using e i t h e r t h e u s u a l L a g r a n g i a n o r t h e augmented

Lagrangian, t h e i n d i c a t e d i n f o r m a t i o n per mits an e x a c t calcul a t i o n of t h e f i r s t - o r d e r

s e n s i t i v i t y o f t h e Karush-Kuhn-

Tucker t r i p l e . The r e q u i r e m e n t s f o r e x a c t s e n s i t i v i t y i n f o r m a t i o n are r a t h e r s e v e r e , and i n e f f e c t r e s u l t i n p o s t o p t i m a l i t y s e n s i t i v i t y analysis calculations (i.e., a solution is required b e f o r e s e n s i t i v i t y c a l c u l a t i o n s c a n b e made).

I f inexact

s e n s i t i v i t y information i s considered, t h e n one obvious p o s s ib i l i t y i s t o u s e estimates of t h e l o c a l s o l u t i o n a n d i t s a s s o c i a t e d o p t i m a l Lagrange m u l t i p l i e r s i n t h e g i v e n e q u a t i o n s .

219

7.5 Conclusions and Extensions

The augmented Lagrangian approach i s a l r e a d y " a l g o r i t h m i c , i n v o l v i n g f i r s t e s t i m a t i n g t h e c o n s t a n t c and t h e o p t i m a l Lagrange m u l t i p l i e r s

U(E,

c) ,

h

t h e n minimizing L o v e r x.

c ) f o r problem P 3 ( € ) , and

W(E,

Of c o u r s e , any v a l i d u n c o n s t r a i n e d

m i n i m i z a t i o n a l g o r i t h m c o u l d be used, a s c o u l d any a p p r o p r i a t e procedure f o r e s t i m a t i n g t h e optimal m u l t i p l i e r s .

i t may be o b s e r v e d t h a t i f

r e g a r d , from Theorem 7.4.1 W(E)

clo se t o 0, then [ u ( E , c ) , E

u(E),

a r e t h e ( l o c a l l y ) unique o p t i m a l Lagrange m u l t i p l i e r s f o r

problem P 3 ( ~ )a s s o c i a t e d w i t h

as

In t h i s

+

and i f

X(E)

W(E,

c)l =

E

[u(E),

is s u f f i c i e n t l y

*

~ ( € 1 1* ( u , w

0 and hence t h e l o c a l l y unique l o c a l minimum

?[x,

u(E),

X(E)

+

w(E),

x(0) = x

*

(with res pect t o

c ] i s g i v e n by G ( E , c ) =

.

E)

X(E,

c) =

*

)

;(E,

c ) of

X(E)

and

Clearly, the f i r s t p a r t i a l derivatives of

[x(E,

c),

U(E,

c),

W(E,

c ) l a l s o con-

v e r g e (component by component) t o t h e f i r s t p a r t i a l d e r i v a t i v e s

*

o f [ x ( O ) , u ( O ) , w ( O ) ] = (x

*

, u ,w

*

).

A c t u a l l y , any p r o c e d u r e

t h a t d e t e r m i n e s t h e o p t i m a l m u l t i p l i e r s of problem P 3 ( 0 ) as E +

0 can be used.

U n c o n s t r a i n e d m i n i m i z a t i o n o f t h e augmented

Lagrangian ( i n t h e a p p r o p r i a t e neighborhood) w i l l t h e n y i e l d an e s t i m a t e o f t h e l o c a l s o l u t i o n x ( 0 ) o f problem P 3 ( 0 ) , and t h e equations given i n (7.4.14)

and (7.4.15)

can be used t o

c a l c u l a t e t h e c o r r e s p o n d i n g estimates o f t h e f i r s t p a r t i a l d e r i v a t i v e s o f t h e Karush-Kuhn-Tucker

t r i p l e [x(O) , u ( 0 ) r W ( 0 ) l

o f problem P 3 (0). An a l t e r n a t i v e t o t h e above p r o c e d u r e s f o r e s t i m a t i n g t h e d e s i r e d s e n s i t i v i t y i n f o r m a t i o n i s t h e p e n a l t y f u n c t i o n proc e d u r e developed i n C h a p t e r 6 .

Analogous t o t h e augmented

Lagrangian r e s u l t , t h e p e n a l t y f u n c t i o n Hessian was shown t o b e p o s i t i v e d e f i n i t e n e a r a s o l u t i o n , so t h a t one set o f

220

7 Calculation of Sensitivity Using Other Algorithms

e q u a t i o n s f ( 6 . 5 . 6 ) and (6.5.8) (7.4.1511

, analogous

to (7.4.14)

and

suffices to calculate the p a r t i a l derivatives of the

Karush-Kuhn-Tucker

triple.

With t h e p e n a l t y f u n c t i o n , t h e r e

i s n o n e e d t o make a p r i o r c a l c u l a t i o n o f t h e o p t i m a l L a g r a n g e m u l t i p l i e r s o r o f any o t h e r i n f o r m a t i o n , e x c e p t f o r t h e un( h a v i n g p r e s e t a s c a l a r param-

c o n s t r a i n e d minimizing p o i n t s

e t e r ) n o r m a l l y r e q u i r e d by t h e a l g o r i t h m , t o estimate t h e T h u s , t h e r e q u i r e d e f f o r t i s com-

t r i p l e and i t s d e r i v a t i v e s .

p a r a t i v e l y m o d e s t , w i t h r e s p e c t t o t h e L a g r a n g i a n a n d augmented L a g r a n g i a n c a l c u l a t i o n s , and makes t h i s p r o c e d u r e a p p e a l i n g a s a preoptimazity s e n s i t i v i t y a n a l y s i s e s t i m a t i o n technique. T h i s a p p e a l i s somewhat o f f s e t by t h e t y p i c a l i l l - c o n d i t i o n i n g t h a t c h a r a c t e r i z e s t h e p e n a l t y f u n c t i o n Hessian n ear a local s o l u t i o n o f t h e g i v e n problem.

Thus, a l l t h e i n d i c a t e d

approaches i n v o l v e compensating f a c t o r s , each o f f e r i n g advantages and disadvantages. A s a f i n a l o b s e r v a t i o n , t h e e x p r e s s i o n d e v e l o p e d above f o r

V E y a ( ~ ,c ) i n ( 7 . 4 . 1 4 )

and (7.4.15)

c a n be p l a c e d i n p r e c i s e

correspondence w i t h t h e s e n s i t i v i t y e x pr es s io n o b t a i n e d by Buys and Gonin ( 1 9 7 7 ) .

Our method of p r o o f i s s i m p l e r ,

however, u t i l i z i n g as shown a r e s u l t p r e v i o u s l y o b t a i n e d f o r t h e usual Lagrangian.

F u r t h e r , t h e r e l a t i o n s h i p between t h e

L a g r a n g i a n a n d augmented L a g r a n g i a n c a l c u l a t i o n s i s demons t r a t e d e x p l i c i t l y a n d allows a p p l i c a t i o n o f a l l p r i o r s e n s i t i v i t y r e s u l t s i n v o l v i n g t h e usual Lagrangian. Although a s p e c i f i c augmented L a g r a n g i a n f u n c t i o n was u s e d above t o o b t a i n t h e s e n s i t i v i t y r e s u l t s , t h e a n a l y s i s a n d a n a l o g o u s r e s u l t s o b t a i n f o r a more g e n e r a l f u n c t i o n , s u c h a s t h e Arrow e t aZ.

(1973) " e x t e n d e d " L a g r a n g i a n , which encom-

p a s s e s m o s t of t h e p o p u l a r f u n c t i o n s o f t h e augmented t y p e .

221

7.5 Conclusions and Extensions

I t seems c l e a r t h a t any v a l i d a l g o r i t h m w i l l y i e l d s e n s i t i v i t y

r e s u l t s a n a l o g o u s t o t h o s e d e v e l o p e d h e r e t h a t may be u t i l i z e d t o p r o v i d e e s t i m a t e s of t h e s e n s i t i v i t y of t h e o p t i m a l v a l u e f u n c t i o n and t h e Karush-Kuhn-Tucker

triple.

This Page Intentionally Left Blank

IV Applications and Future Research

This Page Intentionally Left Blank

Chapter 8

An Example of Computational Implementations: A Multi-Item Continuous Review Inventory Model

8.1.

INTRODUCTION

This c h a p t e r i s d i r e c t e d t o t h e p r a c t i c a l uses of s e n s i t i v i t y and s t a b i l i t y i n f o r m a t i o n .

Sensitivity analysis

r e s u l t s o b t a i n e d by Armacost and F i a c c o (1978) a r e r e p o r t e d , f o r a m u l t i - i t e m i n v e n t o r y model developed by Schrady and Choe ( 1 9 7 1 ) f o r t h e U.S.

Navy.

The computer program SENSUMT, i m -

p l e m e n t i n g t h e p e n a l t y f u n c t i o n method and s e n s i t i v i t y e s t i mates d e s c r i b e d i n C h a p t e r 6 , was used t o s o l v e t h e problem and a p p r o x i m a t e t h e s e n s i t i v i t i e s .

The example a n a l y z e d was

t h e one t r e a t e d by Schrady and Choe, though r e a d i l y e x t e n d e d t o a l a r g e - s c a l e model.

These r e s u l t s p r o v i d e d one of t h e

f i r s t v e r i f i c a t i o n s o f t h e f e a s i b i l i t y and p o t e n t i a l v a l u e o f a d e t a i l e d automated s e n s i t i v i t y a n a l y s i s c a p a b i l i t y f o r NLP i n practical situations.

They i l l u s t r a t e t h e numerous r i c h

i n t e r p r e t a t i o n s and i n s i g h t s t h a t can be d e r i v e d from t h i s i n f o r m a t i o n , a l s o i n d i c a t i n g t h e c a u t i o n t h a t must be t a k e n i n i n t e r p r e t i n g t h e r e s u l t s and making v a l i d i n f e r e n c e s . W e commence o u r d i s c u s s i o n w i t h an i s s u e t h a t w i l l be

r e c o g n i z e d a s v i t a l by anyone who h a s s e r i o u s l y pursued t h e generation of r e l e v a n t s e n s i t i v i t y information:

22s

indiscriminate

226

8 Multi-Item Continuous Review Inventory Model

p r o l i f e r a t i o n of d a t a .

An o b v i o u s f i l t e r i s t h e e f f e c t o f t h e

g i v e n p e r t u r b a t i o n s on t h e o p t i m a l v a l u e f u n c t i o n .

Algorith-

mic c a l c u l a t i o n s can i n e v i t a b l y be e x p l o i t e d t o e s t i m a t e t h e g r e a t e s t e f f e c t s and hence t h e most i n f l u e n t i a l p e r t u r b a t i o n s . W e i n d i c a t e how t h i s can be done, i n e x p e n s i v e l y and a s a

n a t u r a l byproduct o f a l g o r i t h m i c c a l c u l a t i o n s , u s i n g t h e c l a s s i c a l p e n a l t y f u n c t i o n methods o f t h e t y p e u t i l i z e d i n C h a p t e r 6.

Extension of t h e i d e a t o o t h e r f a m i l i e s of algorithms i s

q u i t e evident. 8.2.

SCREENING OF SENSITIVITY INFORMATION For problems i n v o l v i n g a l a r g e number of p a r a m e t e r s , a

very l a r g e number o f p a r t i a l d e r i v a t i v e s may be c a l c u l a t e d i f one p r o c e e d s i n d i s c r i m i n a t e l y .

This i s not only t i m e -

consuming, b u t may a l s o be q u i t e burdensome t o a u s e r who must evaluate t h e o v e r a l l significance of t h e r e s u l t s .

One impor-

t a n t measure o f t h e l a t t e r i s t h e e f f e c t o f a p e r t u r b a t i o n on the s o l u t i o n v a l u e , i.e.,

t h e optimal value function.

I t is

q u i t e p o s s i b l e and i s o f t e n o b s e r v e d i n p r a c t i c e t h a t t h e o p t i mal o b j e c t i v e f u n c t i o n v a l u e i s much more s e n s i t i v e t o v e r y few o f t h e many p a r a m e t e r s p r e s e n t .

With t h i s i n mind, a s i m p l e

technique f o r estimating t h e f i r s t - o r d e r s e n s i t i v i t y of t h e o p t i m a l v a l u e f u n c t i o n i s i n c o r p o r a t e d i n t h e SENSUMT computer program t o p r o v i d e an o p t i o n f o r p r e l i m i n a r y s c r e e n i n g o f t h e parameters i n order t o e l i m i n a t e f u r t h e r c a l c u l a t i o n s involving p e r t u r b a t i o n s o f p a r a m e t e r s deemed t o have r e l a t i v e l y l i t t l e e f f e c t on t h e o p t i m a l v a l u e f u n c t i o n .

A u s e r may i n t r o d u c e

h i s own c r i t e r i a o f s i g n i f i c a n c e i n t h i s d e t e r m i n a t i o n . t h e f o r m u l a s d e v e l o p e d by F i a c c o C1976), a s e c o n d o p t i o n

Using

221

8.2 Screening of Sensitivity Information

i s i n c l u d e d which p e r m i t s t h e c a l c u l a t i o n o f t h e s e n s i t i v i t y

estimates f o r t h e L a g r a n g e m u l t i p l i e r s .

This screening device

and t h e i n d i c a t e d o p t i o n s are d i s c u s s e d n e x t . C o r o l l a r y 6.6.2

provides a j u s t i f i c a t i o n f o r estimating

f * ( ~ )V , E f * ( E ) , and VEf 2 * ( E ) by W*(E, r ) , VEW*(€, r ) , a n d 2 *

VEW

(E,

r), r e s p e c t i v e l y , when r i s p o s i t i v e and s m a l l enough.

Since C o r o l l a r y 6.2.2 f[x(E, r ) ,

E]

=

f * (E)

,

and c o n t i n u i t y imply t h a t l i m r*O+ a n o t h e r estimate o f t h e o p t i m a l v a l u e

f u n c t i o n f * ( E ) i s p r o v i d e d by

r)

?(E,

E f[x(E,

r),

E]

when

r > 0 and s m a l l , a s noted i n t h e f i n a l p a r t of S e c t i o n 6.2. As i n ( 6 . 2 . 3 0 )

,

d i r e c t application of t h e chain r u l e f o r dif-

ferentiation yields, for x =

r),

X(E,

N

VEf(E, r) = Vxf(x, E ) V ~ X ( E , r )

+ VEf(x,

(8.2.1)

E).

Under t h e g i v e n a s s u m p t i o n s c o n t i n u i t y a l s o a s s u r e s t h a t

V E T ( ~ r, ) * VEW

(E,

-f

V E f * ( c ) as r * 0

+.

r ) a r e estimates of V E f

*

Thus, b o t h (E)

VET(€,

r ) and

for r sufficiently small.

I t s h o u l d be n o t e d t h a t t h e s e estimates are f u n c t i o n a l l y

related, since m

N

From t h i s e x p r e s s i o n i t i s c l e a r t h a t V E f ( E , r ) i s t h e b e t t e r

estimate o f V E f * ( E ) , t h e r e m a i n i n g terms i n VEW*(€, r ) s i m p l y constituting "noise" t h a t i s eliminated a s r by u s i n g t h e e x p r e s s i o n ( 6 . 6 . 2 ) 6.6.1,

f o r VEW

*

(E,

-+

0

+

.

However,

r) g i v e n i n Theorem

V E W * ( ~ ,r ) c a n b e e v a l u a t e d w i t h o u t n e c e s s i t a t i n g t h e

c a l c u l a t i o n of V € X ( E ,

r ) , which i s r e q u i r e d t o compute ( 8 . 2 . 1 ) .

228

8 Multi-Item Continuous Review Inventory Model

Thus, t h e c r u d e r b u t c o m p u t a t i o n a l l y much c h e a p e r e s t i m a t e of V f E

*

( € 1 g i v e n by V E W

*

(E,

r) i n (6.6.2)

(noting (6.6.5)) has

been i n t r o d u c e d a s a n o p t i o n i n t h e computer program a s t h e indicated preliminary screening device t o identify c r u c i a l R e s t r i c t i o n of subsequent c a l c u l a t i o n s t o t h e s e

parameters.

p a r a m e t e r s , and o t h e r c a l c u l a t i o n s , s u c h a s t h e s h a r p e r e s t i -

m a t e o f VEf

*

are p r o v i d e d a s a d d i t i o n a l

g i v e n by ( 8 . 2 . 1 ) ,

(E)

options. I n summary, t h e b a s i s f o r t h e e s t i m a t i o n p r o c e d u r e u t i l i z e d h e r e i s t h e m i n i m i z a t i o n o f t h e p e n a l t y f u n c t i o n W(x,

E,

r)

T h i s y i e l d s a p o i n t K ( E , r ) t h a t is viewed

g i v e n by ( 6 . 2 . 1 ) .

a s a n e s t i m a t e o f a ( l o c a l ) s o l u t i o n x* o f t h e g i v e n p r o b l e m when r > 0 i s s m a l l .

The a s s o c i a t e d o p t i m a l Lagrange m u l t i -

p l i e r s u* a n d w* a r e t h e n e s t i m a t e d by U(E, r ) a n d W ( E , using t h e r e l a t i o n s h i p s given i n (6.2.17)

and (6.2.18)

r ) by

,

respectively. I f d e s i r e d , a l l of t h e f i r s t p a r t i a l d e r i v a t i v e s of =

[x(E, r ) ,

U(E,

r),

W(E,

r ) l Tw i t h r e s p e c t t o

E,

and ( 6 . 2 . 2 2 ) .

i s c a l c u l a t e d , t h e n VEf

*

(E)

r)

an estimate

of V E y ( ~ )may , b e o b t a i n e d by f i r s t s o l v i n g ( 6 . 2 . 1 5 ) applying ( 6 . 2 . 2 1 )

Y(E,

and t h e n

I f the f u l l matrix V € X ( E ,

i s e s t i m a t e d by ( 8 . 2 . 1 ) .

r)

However,

i f i t i s d e s i r e d t o e l i m i n a t e c a l c u l a t i o n s i n v o l v i n g param-

*

e t e r s h a v i n g less e f f e c t on l o c a l c h a n g e s of f ( € 1 , s c r e e n i n g d e v i c e d e s c r i b e d above i s u s e d . e s t i m a t i o n o f VEf

*

(E)

by V E W * ( ~ ,r ) .

the

This e n t a i l s i n i t i a l

Components o f

s p o n d i n g t o t h e r e s p e c t i v e components o f VEf

*

(E)

E

corre-

t h a t are deemed

i n c o n s e q u e n t i a l may t h e n b e d e l e t e d and would n o t e n t e r i n t o any s u b s e q u e n t c a l c u l a t i o n s .

*

that (x

,

u

*

,w

*

),

f

*

(E),

I n p a r t i c u l a r , it i s emphasized

and VEf

*

(E)

may b e e s t i m a t e d by

229

8 . 3 Example of Sensitivity Calculations by SENSUMT Y(E,

r ) , f [ x ( E , r ) , €1, and DEW

c a l c u l a t i n g a n y components o f V

*

(E,

r ) , respectively, without

Y(E,

r ) , once

X(E,

r ) i s known.

T h e s e , and o t h e r c a l c u l a t i o n s t h a t have b e e n i n c l u d e d i n t h e computer c o d e s i m p l e m e n t i n g t h e p r o c e d u r e , are i l l u s t r a t e d i n t h e n e x t example a n d d i s c u s s e d i n t h e i n v e n t o r y example a n d i n t h e f i n a l s e c t i o n of C h a p t e r 9. 8.3.

EXAMPLE SENSITIVITY CALCULATIONS BY SENSUMT B r i e f l y , t h e conduct o f a s e n s i t i v i t y a n a l y s i s i s

c o n t r o l l e d by f o u r o p t i o n s :

no s e n s i t i v i t y a n a l y s i s , a s e n s i -

t i v i t y a n a l y s i s a t t h e f i n a l "subproblem," i . e . ,

minimization

o f t h e p e n a l t y f u n c t i o n , a s e n s i t i v i t y a n a l y s i s a t e a c h subproblem a l o n g t h e p e n a l t y f u n c t i o n minimizing t r a j e c t o r y , o r a s e n s i t i v i t y a n a l y s i s a t t h e f i n a l subproblem, f o r a range of d i f f e r e n c i n g increments. A s a n i l l u s t r a t i o n o f t h e k i n d o f i n f o r m a t i o n t h a t c a n be

g e n e r a t e d , c o n s i d e r t h e f o l l o w i n g s i m p l e convex ( i n x ) p a r a m e t r i c n o n l i n e a r programming problem. E x a m p l e 8.3.1 min

X

f(x,

s.t. g l ( x , f o r E~ > 0 a n d

E)

= x

1

+

-

E2X2

x 1 - x 22 - > 0

E)

=

E~

not restricted.

El

W e f i r s t g i v e t h e s o l u t i o n i n c l o s e d form.

t h e Karush-Kuhn-Tucker

Application of

conditions y i e l d s t h e following:

8 Multi-Item Continuous Review Inventory Model

230 U1(E)

= (1 +

€22) 1/

2

/2E1.

and gl[X(E),

=

El

0.

I t may b e r e a d i l y v e r i f i e d t h a t t h e a s s u m p t i o n s o f Theorem

are s a t i s f i e d .

3.2.2

f*(E)

= -E1(1

+

From t h e e x p r e s s i o n f o r

€;)Ii2.

1..)J

Taking p a r t i a l d e r i v a t i v e s w i t h r e s p e c t t o

v EX ( E )

€:y2

(1 +

1

=

(1

+

= (-(1

X(E)

+

(1

+

and

E~

E

=

(

-(1

+

€2

.;y2,

-E1E2/(1

1/2 E;)

w, e o b t a i n

€3

+ €$I2),

and

v E U*(E) l

~

1 2 4 E2/(2El(l

Suppose w e are i n t e r e s t e d i n t h e v a l u e s

+

E;)

E~

1/2 = 2,

)). E~

= 1.

E v a l u a t i o n of t h e above e x p r e s s i o n s y i e l d s

[-"I,

f* = - 2 0 ,

VEf* =

x* =

VEX

-fi

u;

= n/4,

* --

(-0, -a), [-W2

12/21r

-a/2 - n / 2

VEu: = ( - n / 8 ,

n / 8 ) .

F o r i l l u s t r a t i v e p u r p o s e s , n u m e r i c a l r e s u l t s o b t a i n e d by t h e computer program are i n c l u d e d i n T a b l e 8 . 3 . 1 f o r t h e o p t i m a l v a l u e f u n c t i o n and Lagrange m u l t i p l i e r s e n s i t i v i t y . The subproblem k c o r r e s p o n d s t o m i n i m i z a t i o n o f t h e p e n a l t y

T a b l e 8.3.1.

S e n s i t i v i t y E s t i m a t e s f o r Example 8.3.1, where Lagrangian

E

= (2, 1 )

Chain R u l e

Subproblem

f

af/acl

w a E 2

w a c 1

af/ac2

U1

aUl/aEl

aUl/aE 2

1

-1.9999

-1.9999

-0.9999

-1.3333

-1.3333

0.4999

-0.3333

0.1666

2

-2.5393

-1.5440

-1.2947

-1.4087

-1.4088

0.3860

-0.2200

0.1761

3

-2. 7765

-1.4439

-1.3833

-2.4139

-1.4139

0.3609

-0.1844

0.1767

4

-2.8128

-2.4243

-1.4064

-1.4142

-1.4142

0.3560

-0.1790

0.1768

5

-2.8245

-1.4127

-1.4123

-1.4142

-1.4142

0.3531

-0.1768

0.1768

6

-2.8274

-1.4137

-1.4137

-1.4142

-1.4142

0.3532

-0.1767

0.1768

Ana Zy t i c a l

-2.8282

-1.4142

-1.4142

-1.4142

-1.4142

0.3537

-0.1769

0.1768

232

8 Multi-Item Continuous Review Inventory Model

f u n c t i o n W(x,

E,

r k ) , where rk > rk+l > 0.

The v a l u e s o f t h e

f i r s t - o r d e r o p t i m a l v a l u e f u n c t i o n s e n s i t i v i t y computed b o t h by t h e c h a i n r u l e (8.2.1) a n d by t a k i n g p a r t i a l d e r i v a t i v e s o f t h e Lagrangian w i t h r e s p e c t t o t h e parameters p r e s e n t e d f o r comparison.

(6.6.2) a r e

As i n d i c a t e d p r e v i o u s l y a n d a s

e x p e c t e d , t h e c h a i n r u l e estimates are s h a r p e r . t h e estimates o b t a i n e d by ( 6 . 6 . 2 ) tions.

Nonetheless,

g i v e v e r y good approxima-

The v a l u e s o f f are g i v e n by f [ x ( E , r ) , € 1 , w h i l e t h e

v a l u e s o f u1 a n d VEul w e r e o b t a i n e d by a p p l y i n g ( 6 . 2 . 1 7 ) (6.2.21).

The components o f

X(E)

and

and V E x ( € )w e r e c a l c u l a t e d

also b u t are n o t l i s t e d . The r e s u l t s a r e t y p i c a l .

Computational ex p er ien ce i n d i -

cates t h a t t h e s e n s i t i v i t y estimates w i l l c o n v e r g e s a t i s f a c t o r i l y whenever t h e g i v e n a l g o r i t h m s u c c e e d s i n d e t e r m i n i n g a good estimate o f a Karush-Kuhn-Tucker 8 4.

A MULTI-ITEM

point.

INVENTORY MODEL

T r a d i t i o n a l l y , i n v e n t o r y models h a v e b e e n f o r m u l a t e d t o minimize some f u n c t i o n of t h e o r d e r i n g , h o l d i n g , a n d s h o r t a g e

( o r b a c k o r d e r ) costs s u b j e c t t o v a r i o u s c o n s t r a i n t s .

Schrady

a n d Choe (1971) f o r m u l a t e d a n i n v e n t o r y model t h a t a p p e a r s t o h a v e much g r e a t e r r e l e v a n c e f o r a n i n v e n t o r y s y s t e m i n a noncommercial e n v i r o n m e n t , s u c h a s i n s t i t u t i o n a l o r m i l i t a r y . The c o s t s u s e d i n t h e t r a d i t i o n a l models may be q u i t e a r t i f i -

c i a l a n d t h e r e a l o b j e c t i v e o f t h e s y s t e m i s o f t e n maximizat i o n o f a m e a s u r e o f r e a d i n e s s o r service, h e r e assumed t o be equivalent t o minimization of stockouts.

In addition, t h e

s t o c k p o i n t s of s u c h s u p p l y s y s t e m s are i n e v i t a b l y c o n s t r a i n e d by i n v e s t m e n t a n d reorder work load l i m i t a t i o n s .

233

8.4 Multi-item Inventory Model

Schrady and Choe's m u l t i - i t e m i n v e n t o r y system assumes t h e s e c o n s t r a i n t s a l o n g w i t h t h e s p e c i f i c o b j e c t i v e o f minim i z i n g t h e t o t a l time-weighted

shortages.

The d e c i s i o n v a r i -

a b l e s a r e t a k e n t o b e t h e " r e o r d e r q u a n t i t i e s " and t h e " r e o r d e r p o i n t s , " r e s p e c t i v e l y , how much t o o r d e r and when t o o r d e r each i t e m i n t h e inventory.

A t h r e e - i t e m example problem was

s o l v e d by Schrady and Choe ( 1 9 7 1 ) u s i n g t h e SUMT computer code (Mylander e t a l . ,

1971).

S u b s e q u e n t l y , McCormick (1972)

showed how t h e s p e c i a l s t r u c t u r e o f t h i s i n v e n t o r y model can be used t o f a c i l i t a t e t h e u s e o f t h e SUMT code t o s o l v e v e r y l a r g e i n v e n t o r y problems.

H e a l s o e x t e n d e d t h e model t o i n -

c l u d e c o n s t r a i n t s on s t o r a g e volume and t h e p r o b a b i l i t y o f depletion of c r i t i c a l i t e m s . The model and example p r e s e n t e d h e r e a r e t h e o r i g i n a l o n e s due t o Schrady and Choe.

The computer program d e s c r i b e d i n

t h e p r e c e d i n g s e c t i o n was used t o s o l v e t h e example and c a l c u l a t e t h e p a r t i a l d e r i v a t i v e s of various q u a n t i t i e s of i n t e r e s t , w i t h r e s p e c t t o e a c h p a r a m e t e r i n v o l v e d i n d e f i n i n g t h e model. The a n a l y s i s can be a p p l i e d t o t h e e x t e n d e d model w i t h o u t difficulty. D e t a i l e d development o f t h e model i s n o t g i v e n h e r e .

The

i n t e r e s t e d r e a d e r i s r e f e r r e d t o t h e Schrady-Choe and McCormick papers.

Here, w e g i v e a summary t r e a t m e n t o f t h e v a r i o u s con-

d i t i o n s and r e l a t i o n s h i p s upon which t h e model is based.

We

then t a b u l a t e t h e r e s u l t s obtained i n solving the r e s u l t i n g n o n l i n e a r programming problem and a p p l y i n g t h e s e n s i t i v i t y a n a l y s i s methodology.

A number o f o b s e r v a t i o n s and i n t e r p r e -

t a t i o n s a r e o f f e r e d t o i l l u s t r a t e t h e many u s e s t o which t h e s e n s i t i v i t y i n f o r m a t i o n might be a p p l i e d .

234

8 Multi-Item Continuous Review Inventory Model

I t i s assumed t h a t t h e amount o f e a c h i t e m i n i n v e n t o r y i s

a l w a y s known, t h a t a l l demand t h a t o c c u r s when t h e on-hand s t o c k i s zero i s b a c k - o r d e r e d ,

and t h a t t h e demand t h a t o c c u r s

d u r i n g t h e t i m e between t h e p l a c e m e n t o f a n o r d e r and i t s r e c e i p t by t h e s t o c k p o i n t ( i . e . , t h e " l e a d t i m e demand") i s n o r m a l l y d i s t r i b u t e d w i t h known mean For t h e i - t h i t e m ,

c

i

ui

and v a r i a n c e u

1'

let

= i t e m u n i t cost ( i n d o l l a r s ) ,

Xi

= mean demand p e r u n i t t i m e ( i n u n i t s ) ,

ri

=

Qi

= reorder quantity,

reorder point,

$ ( x ) = t h e N o r m a l ( 0 , 1) d e n s i t y f u n c t i o n , @ ( z )=

/z

@ ( x ) d x= t h e N o r m a l

(0,

1) complementary

cumulative d i s t r i b u t i o n function. I n a d d i t i o n , l e t K1 b e t h e i n v e s t m e n t l i m i t i n d o l l a r s , K2 be t h e number o f o r d e r s p e r u n i t o f t i m e t h a t c o n s t i t u t e s reo r d e r w o r k l o a d l i m i t , a n d N b e t h e t o t a l number o f i t e m s i n the inventory. I t c a n b e shown t h a t t h e e x p e c t e d t i m e - w e i g h t e d s h o r t a g e

o f i t e m i a t any p o i n t i n t i m e i s g i v e n by B ~ ( Q ~r i, ) = L [ B (r.1 . Qi 1 1

where

-

B ~ ( + Q r~i ) l

235

8.4 Multi-Item Inventory Model

+

The e x p e c t e d on-hand i n v e n t o r y o f i t e m i i s g i v e n by ri

-

pi

+

Qi/2

r i ) a n d t h e e x p e c t e d number of o r d e r s p l a c e d p e r

Bi(Qi,

u n i t t i m e f o r i t e m i i s A./Q 1

i'

U s i n g t h e above e x p r e s s i o n s a n d a s s u m p t i o n s , S c h r a d y and Choe ( 1 9 7 1 ) i n d i c a t e t h a t m e a n i n g f u l a p p r o x i m a t i o n s o f t h e g i v e n q u a n t i t i e s a r e o b t a i n e d even when t h e s e c o n d t e r m i s d r o p p e d from t h e e x p r e s s i o n f o r t h e e x p e c t e d s h o r t a g e s , and when t h e l a s t t e r m i s d r o p p e d from t h e e x p r e s s i o n f o r e x p e c t e d on-hand i n v e n t o r y .

The g i v e n a s s u m p t i o n s and s i m p l i f i c a t i o n s

t h e n l e a d r e a d i l y t o t h e f o l l o w i n g n o n l i n e a r programming p r o b l e m (which S c h r a d y and Choe ( 1 9 7 1 ) p r o v e d c o n v e x ) : N

N

i=1

w i t h ri u n r e s t r i c t e d i n s i g n , Qi 2 0 , i = l r . . . , N , (Q1r

- - I

QN)

T

,

r = (rl,

..., r N ) , a n d g1 T

Q

=

and g 2 r e p r e s e n t i n g

t h e i n v e s t m e n t and w o r k l o a d c o n s t r a i n t s , r e s p e c t i v e l y . The p r o b l e m d a t a f o r t h e Schrady-Choe

t h r e e - i t e m example

a n d t h e i n i t i a l s t a r t i n g p o i n t f o r t h e SENSUMT program a r e shown i n T a b l e 8.4.1.

As i n d i c a t e d i n t h e t a b l e , t h e l e a d t i m e

demands a n d s t a n d a r d d e v i a t i o n s , t h e i t e m u n i t c o s t s and mean demands, a n d t h e i n v e s t m e n t a n d w o r k l o a d l i m i t s are a l l t r e a t e d a s parameters i n conducting t h e s e n s i t i v i t y a n a l y s i s .

Our c a l -

c u l a t i o n s i n d i c a t e d t h a t t h e a s s u m p t i o n s o f Theorem 3.2.2 were s a t i s f i e d f o r (SC) a t t h e g i v e n s o l u t i o n , f o r t h e g i v e n d a t a .

236

8 Multi-Item Continuous Review Inventory Model

Table 8.4.1.

I n v e n t o r y P r o b l e m Data Item i

Quantity

1

2

3

(mean o f l e a d t i m e demand)

100

200

300

(S.D. o f l e a d t i m e demand)

100

100

200

1

10

20

I , 000

I, 5 0 0

Parameters

'i U

i

ci

( i t e m u n i t c o s t , $)

'i

(mean demand/ unit time)

Kl

(investment l i m i t ) ( r e o r d e r work load limit )

K2

2,000

$8,000 15 r e o r d e r s / u n i t time

Var i a b 1 e s 0

(amount o r d e r e d )

600

270

300

0

(reorder level)

200

260

400

Qi P

i

T a b l e 8 . 4 . 2 g i v e s t h e computer s o l u t i o n and T a b l e 8 . 4 . 3 the f i n a l e s t i m a t e of t h e f i r s t p a r t i a l d e r i v a t i v e s of t h e optimal value function Z

eters.

*

w i t h r e s p e c t t o t h e problem param-

R e l a t i v e t o t h e c r i t e r i o n used i n t h e computer program,

t h e T a b l e 8 . 4 . 3 r e s u l t s i n d i c a t e t h a t t h e o p t i m a l v a l u e funct i o n i s s e n s i t i v e t o p a r a m e t e r s KZ, c l , a2, c 2 , a 3 , and c3. Many i n f e r e n c e s a r e p o s s i b l e .

For example, t h e f a c t t h a t t h e

s o l u t i o n i s p a r t i c u l a r l y s e n s i t i v e t o t h e v a l u e s of t h e s t a n d a r d d e v i a t i o n s o f t h e l e a d t i m e demand of i t e m s 2 and 3 might i n d i c a t e t h a t , s i n c e t h e s e p a r a m e t e r s w e r e o b t a i n e d by sampling, a d d i t i o n a l s a m p l i n g o f t h e s e l e a d t i m e demands may v e r y w e l l be w a r r a n t e d t o r e d u c e t h e a s s o c i a t e d s t a n d a r d deviations.

237

8.4 Multi-Item Inventory Model

T a b l e 8.4.2.

S o l u t i o n and Lagrange M u l t i p l i e r s

Item i Quantity

Variables

* Qi * ri

1

2

3

533

246

285

253

277

437

Lagrange m u l t i p l i e r s

* *

0.0052

u3

0.6230

u2 Value

z Table 8.4.3

*

12.987

a l s o suggests t h a t t h e optimal s o l u t i o n value

i s v e r y s e n s i t i v e t o a l l of t h e i t e m c o s t s .

If the structure

of problem ( S C ) i s examined, t h i s r e s u l t may a t f i r s t a p p e a r

c o n t r a d i c t o r y s i n c e t h e ci a p p e a r o n l y i n t h e i n v e s t m e n t c o n s t r a i n t and t h e o p t i m a l v a l u e f u n c t i o n , a c c o r d i n g t o T a b l e 8.4.3.

O p t i m a l Value F u n c t i o n Derivatives Item i

P a r t i a 1s

az

*

1

2

3

lapi

-0.0000

az*/aoi

0.0119

0. 0897a

0.172ga

az

2.1713a

I. 034.Sa

1.445Za

0.0012

0.0025

0.0022

*

laci

a z* / a xi * az /aKl * az /aK2

-0.0003

-0.0008

-0.0052

-0. 6230a

*

I/?

*

aDeemed " s i g n i f i c a n t " b y c r i t e r i o n , [ A l Z > 0.001 f o r a u n i t change i n t h e g i v e n parameter, where A I Z i s t h e e s t i mated f i r s t - o r d e r change in Z*. T h i s criterion #as s e l e c t e d Criteria appropriate a r b i t r a r i l y f o r i l l u s t r a t i v e purposes. t o t h e p a r t i c u l a r a p p Z i c a t i o n can be s e l e c t e d b y a u s e r .

238

8 Multi-Item Continuous Review Inventory Model

Table 8.4.3,

l i m i t K1.

i s apparently not very s e n s i t i v e t o t h e investment

The p r o b l e m i s o n e o f p r e c i s e i n t e r p r e t a t i o n .

p a r t i a l d e r i v a t i v e s measure r a t e o f change. t h e i n v e s t m e n t c o n s t r a i n t g l ( Q , r ) a t (Q

*

,

*

The

But i n s p e c t i o n o f

r ) reveals t h a t

t h e change i n a n i t e m cost ci by any amount A c i

h a s t h e same

e f f e c t on t h e c o n s t r a i n t as a c h a n g e i n t h e i n v e s t m e n t l i m i t K1 of -(r:

+

Q;/2

-

Since t h e quantity i n parentheses

pi)dci.

may be v e r i f i e d f r o m T a b l e s 8 . 4 . 1

and 8.4.2

t o b e much g r e a t e r

t h a n 1 f o r a l l i , i t f o l l o w s t h a t t h e e f f e c t o f c h a n g i n g any

c

by any i n c r e m e n t 6 w i l l b e much g r e a t e r o n t h e c o n s t r a i n t i * (and hence, on t h e o p t i m a l v a l u e Z , s i n c e t h e c o n s t r a i n t i s

b i n d i n g ) t h a n t h e e f f e c t o f c h a n g i n g K1 by t h e same amount 6 . This implies t h a t

I aZ*/aci I

>

I aZ*/aK1l

f a c t , i t c a n be shown h e r e t h a t aZ*/aci

*

a 2 /aK,,

f o r each i and, i n = -(r:

+

Q;/2

-

pi)

x

so t h a t t h e r e l a t i o n s h i p s i n d i c a t e d a r e i n d e e d

precisely verified. The above o b s e r v a t i o n s m i g h t a l s o s u g g e s t t h a t some care must b e t a k e n i n i n t e r p r e t i n g t h e r e s u l t s .

Changes i n t h e

parameter a s s o c i a t e d w i t h t h e l a r g e s t ( i n a b s o l u t e value) p a r t i a l d e r i v a t i v e w i l l g i v e t h e g r e a t e s t l o c a l change i n t h e o p t i m a l v a l u e o f t h e o b j e c t i v e f u n c t i o n , compared t o a c h a n g e o f t h e same m a g n i t u d e i n any o t h e r p a r a m e t e r t a k e n i n d i v i d u a l l y . This f o l l o w s because t h e o b j e c t i v e f u n c t i o n and/or s o m e o f t h e c o n s t r a i n t s (as above) are m o s t s i g n i f i c a n t l y a f f e c t e d by t h i s p a r a m e t e r change a t t h e c u r r e n t s o l u t i o n , i n t e r m s o f r a t e o f change.

G e n e r a l r u l e s have n o t b e e n g i v e n f o r s e l e c t i o n o f

optimal l o c a l changes i n t h e parameters, i.e.,

f o r determining

t h e o p t i m a l m a g n i t u d e and c o m b i n a t i o n o f s u c h c h a n g e s .

W e do

n o t pursue t h i s "marginal a n a l y s i s " d e t e r m i n a t i o n , though it s h o u l d be n o t e d t h a t t h e g r e a t e s t l o c a l r a t e of decrease i n

239

8.4 Multi-Item Inventory Model

t h e optimal value f u n c t i o n i s along t h e d i r e c t i o n of t h e negative of t h e g r a d i e n t of t h i s f u n c t i o n i n parameter space

( i . e . , a l o n g t h e v e c t o r composed o f t h e n e g a t i v e o f t h e compon e n t s of t h e p a r t i a l d e r i v a t i v e s w i t h r e s p e c t t o t h e v a r i o u s A u s e r would n o n e t h e l e s s have t o d e t e r m i n e t h e

parameters).

f e a s i b i l i t y o f t h i s d i r e c t i o n o f change and, i f f e a s i b l e , t h e o p t i m a l move a l o n g t h i s v e c t o r , t a k i n g i n t o a c c o u n t o t h e r f a c t o r s s u c h a s t h e r e l a t i v e " c o s t e f f e c t i v e n e s s " o f any s c h e d u l e o f changes i n any model p a r a m e t e r . R e f e r r i n g back t o T a b l e 8.4.2,

*

w e n o t e t h a t t h e Lagrange

*

m u l t i p l i e r u2 i s much g r e a t e r t h a n ul.

Recalling the "sensi-

t i v i t y " i n t e r p r e t a t i o n o f Lagrange m u l t i p l i e r s , which h o l d s under t h e p r e s e n t c o n d i t i o n s , i t f o l l o w s t h a t u:

*

*

u2 = - 3 2 / a K 2 .

= -32

*

/aK1and

This conclusion i s consistent with t h e r e s u l t

o b t a i n e d i n T a b l e 8.4.3,

and i t means t h a t t h e workload con-

s t r a i n t g2 i s by f a r t h e more e f f e c t i v e i n d e t e r m i n i n g t h e minimum number of e x p e c t e d time-weighted r e n t v a l u e of t h e p a r a m e t e r s , e . g . , have a g r e a t e r e f f e c t on r e d u c i n g Z K1.

*

s h o r t a g e s a t t h e cur-

a s m a l l i n c r e a s e i n K2 w i l l than a small increase i n

N o n e t h e l e s s , a u s e r must a g a i n s i m u l t a n e o u s l y c o n s i d e r

t h e c o m p a r a t i v e c o s t s i n v o l v e d i n making f i n i t e changes, i n c o n j u n c t i o n w i t h t h e i r e x p e c t e d e f f e c t s , t o a r r i v e a t an o p t i mal m a r g i n a l improvement b a s e d on t h i s f i r s t - o r d e r i n f o r m a t i o n . The s e n s i t i v i t y i n f o r m a t i o n i s v a l u a b l e , b u t r e q u i r e s some care i n exploiting. T a b l e 8.4.4

gives t h e e s t i m a t e s of t h e f i r s t d e r i v a t i v e s

o f t h e o p t i m a l r e o r d e r q u a n t i t i e s Qi

and r e o r d e r p o i n t s ri w i t h

r e s p e c t t o e a c h o f t h e problem p a r a m e t e r s .

This i s extremely

d e t a i l e d i n f o r m a t i o n t h a t g i v e s an i n d i c a t i o n o f how t h e comp o n e n t s o f t h e s o l u t i o n v e c t o r i t s e l f w i l l change a s t h e

240

8 Multi-Item Continuous Review Inventory Model

T a b l e 8.4.4.

Solution Point Sensitivity Item

Par t i aIs

i

2

1 -47.3187

-18. 7610

3

-14.9065

5.2265

6.1961

9.9671

,208.8688

15.3140

14.3755

-31. 7918

-10.3425

-20.0020

-0.8469

0.2271

-0.1084

-0.1273

1.0337

-0.4919

8.1908

-4.9719

3.8522

-2.6783

-7.2676

-7.1087

-1.2374

-0.4033

0.5843

-0.261 1

-0. 3 8 3 9

0.0446

-0. 1 6 7 0

0.4442

-0.4251

-3.1702

-12.1523

-2.3072

v a r i o u s p a r a m e t e r s change.

I n p a r t i c u l a r , t h i s information

can b e used t o o b t a i n a f i r s t - o r d e r e s t i m a t e o f t h e s o l u t i o n v e c t o r o f a problem i n v o l v i n g d i f f e r e n t p a r a m e t e r v a l u e s , once a s o l u t i o n f o r a g i v e n s e t o f p a r a m e t e r s h a s been o b t a i n e d . The p a r t i a l d e r i v a t i v e s o f t h e Lagrange m u l t i p l i e r s w i t h r e s p e c t t o t h e parameters a r e given i n Table 8 . 4 . 5 .

Again,

t h e s e can be used t o o b t a i n f i r s t - o r d e r e s t i m a t e ' s o f t h e Lagrange m u l t i p l i e r s o f a problem w i t h d i f f e r e n t p a r a m e t e r values.

I n p a r t i c u l a r , t h e r e l a t i v e e f f e c t s of t h e c o n s t r a i n t s

on t h e o p t i m a l v a l u e o f problems i n v o l v i n g d i f f e r e n t p a r a m e t e r v a l u e s can b e e s t i m a t e d .

F u r t h e r m o r e , it f o l l o w s t h a t t h e

p a r t i a l d e r i v a t i v e s o f t h e o p t i m a l m u l t i p l i e r s u1 and u2 w i t h r e s p e c t t o K1 and K2, r e s p e c t i v e l y , y i e l d t h e s e c o n d p a r t i a l

241

8.4 Multi-Item Inventory Model

L . M.

T a b l e 8.4.5. Derivatives

Lagrange M u l t i p l i e r S e n s i t i v i t y :

Optimal

Constraint i

Par t i a 1s

Investment

*

aUi/aK2

*

aui/ac3

Workload

-0.0002

-0,1382

0.0006

0.1635

0.0000

0.0006

0.0002

0.0489

0.0000

0.0020

0.0003

0.0323

d e r i v a t i v e s of t h e o p t i m a l v a l u e f u n c t i o n w i t h r e s p e c t t o t h e p a r a m e t e r s K1

and K 2 u n d e r t h e p r e s e n t c o n d i t i o n s .

c a n be u s e d t o pro-

kind of information given i n Table 8 . 4 . 5 vide a second-order

Thus, t h e

estimate of t h e o p t i m a l v a l u e f u n c t i o n Z

*

f o r d i f f e r e n t v a l u e s of t h e s e parameters. T o i l l u s t r a t e and t e s t t h e a p p l i c a t i o n o f t h e t y p e o f

information provided here, t h e f i r s t p a r t i a l d e r i v a t i v e s with r e s p e c t t o c1 o f t h e o p t i m a l v a l u e f u n c t i o n Z components Q

i

*

,

the solution

and ri, and t h e Lagrange m u l t i p l i e r s u1 and u2

w e r e u s e d t o g i v e a f i r s t - o r d e r estimate o f t h e c o r r e s p o n d i n g s o l u t i o n v a l u e s a s s o c i a t e d w i t h t h e problem i n which t h e g i v e n v a l u e of c1 w a s i n c r e a s e d by 1 d o l l a r .

These estimates w e r e

compared w i t h t h e r e s p e c t i v e v a l u e s of t h e s o l u t i o n o b t a i n e d by a c t u a l l y s o l v i n g t h e p e r t u r b e d problem. summarized i n T a b l e 8.4.6.

The r e s u l t s are

Though t h e p e r t u r b a t i o n i s l a r g e

( t h e p a r a m e t e r b e i n g i n c r e a s e d by 100% o f i t s c u r r e n t v a l u e ) , t h e estimates are s e e n t o b e e x t r e m e l y a c c u r a t e , w i t h t h e e x c e p t i o n of t h e e s t i m a t e d r e o r d e r q u a n t i t y Q1.

Many u s e s

242

8 Multi-Item Continuous Review Inventory Model

T a b l e 8.4.6. F i r s t - O r d e r E s t i m a t e s f o r a U n i t I n c r e a s e of Parameter E = e l , where F = 1 and ? = 2 Quantity F(e)

E s t i m at e a

F(e)

Actual

X Abs. e r r o r b

15.159

14.996

1.08

324

412

21.36

221

229

3.49

261

257

1.56

267

268

0.37

299

297

0. 67

417

420

0.71

0,0058

0.0057

1.75

0. 7865

0. 7 6 7 1

1.94

c o u l d b e made o f t h e e s t i m a t e d s o l u t i o n ; e . g . ,

s h o u l d it be

d e s i r a b l e t o s o l v e t h e p e r t u r b e d p r o b l e m a c c u r a t e l y , i t would g e n e r a l l y be computationally extremely advantageous t o use t h e e s t i m a t e d s o l u t i o n as a s t a r t i n g p o i n t . The c o m p l e t e e x p l o i t a t i o n o f t h i s s e n s i t i v i t y a n a l y s i s i n f o r m a t i o n now a v a i l a b l e w i l l depend l a r g e l y o n u s e r i n t e r e s t and i n g e n u i t y .

8.5 Additional Computational Experience with Applications

8.5.

243

A D D I T I O N A L COMPUTATIONAL EXPERIENCE W I T H APPLICATIONS

A s i n d i c a t e d i n C h a p t e r 5 , S e c t i o n 5.6,

t h e SENSUMT

program was used t o s o l v e and g e n e r a t e s e n s i t i v i t y i n f o r m a t i o n f o r an a n a l y s i s o f t h e S h e l l Dual problem (Armacost and F i a c c o , 1 9 7 6 1 , following i n i t i a l s u c c e s s f u l experiments with a v a r i e t y

o f s m a l l t e s t problems, i n c l u d i n g a d e t e r m i n i s t i c e q u i v a l e n t o f a c a t t l e f e e d problem (Armacost and F i a c c o , 1 9 7 4 ) .

Our

s u c c e s s w i t h t h e s e problems m o t i v a t e d u s t o e s t a b l i s h t h e p r a c t i c a l a p p l i c a b i l i t y o f NLP s e n s i t i v i t y c a l c u l a t i o n s i n g e n e r a l , and t h e g i v e n a l g o r i t h m i c approach i n p a r t i c u l a r , by a p p l y i n g o u r methods t o w e l l known models f o r m u l a t e d by o t h e r s and p u b l i s h e d i n t h e open l i t e r a t u r e .

The f i r s t s u c h e x p e r i -

ment was t h e a p p l i c a t i o n o f SENSUMT t o t h e Schrady-Choe i n v e n t o r y model (Armacost and F i a c c o , 1 9 7 8 ) ; r e s u l t s o f t h a t analysis a r e reported i n t h i s chapter.

Two a d d i t i o n a l models

have been t h e s u b j e c t o f numerous a d d i t i o n a l s e n s i t i v i t y s t u d i e s u s i n g SENSUMT:

a w e l l known G P model o f a stream-

w a t e r p o l l u t i o n abatement system, and a n o n l i n e a r s t r u c t u r a l d e s i g n model.

W e b r i e f l y i n d i c a t e t h e s t u d i e s t h a t have been

conducted w i t h t h e s e models. For a s t r e a m w a t e r p o l l u t i o n GP model, whose p r o t o t y p e was f o r m u l a t e d by Charnes and Gemmell ( 1 9 6 4 ) and e x t e n d e d and a n a l y z e d by Ecker (1975) u s i n g an upper Hudson R i v e r d a t a b a s e , F i a c c o and Ghaemi (1979a) f i r s t o b t a i n e d t h e o p t i m a l w a s t e t r e a t m e n t l e v e l s minimizing t h e a n n u a l c o s t .

(Ecker's r e s u l t s

w e r e obtained using a GP algorithm, while t h e l a t t e r study makes u s e o f t h e p e n a l t y f u n c t i o n a l g o r i t h m a p p l i e d t o t h e u s u a l convex e q u i v a l e n t o f t h e g i v e n GP problem.)

I t i s shown

t h a t t h e p o l i c y o f f i x e d d i s s o l v e d oxygen r e q u i r e m e n t y i e l d s

244

8 Multi-Item Continuous Review Inventory Model

t h e minimum a n n u a l w a s t e t r e a t m e n t c o s t , r e l a t i v e t o two o t h e r environmental c o n t r o l p o l i c i e s .

Using SENSUMT, F i a c c o and

Ghaemi (197933) s u b s e q u e n t l y a n a l y z e d t h e s e n s i t i v i t y o f t h e o p t i m a l waste t r e a t m e n t c o s t t o c h a n g e s i n t h e 6 7 p a r a m e t e r s i n v o l v e d i n t h e f o r m u l a t i o n o f t h e model.

This a n a l y s i s i s

c o n d u c t e d f o r t h e f i x e d d i s s o l v e d oxygen r e q u i r e m e n t p o l i c y . I t i s shown t h a t t h e maximum a l l o w a b l e d i s s o l v e d oxygen de-

f i c i t i s u n e q u i v o c a l l y t h e p a r a m e t e r t o which t h e o p t i m a l

w a s t e t r e a t m e n t c o s t i s most s e n s i t i v e .

A l l of t h e many

p a r a m e t e r s i n v o l v e d i n t h e model are a n a l y z e d a n d , r e l a t i v e t o s t i p u l a t e d a s s u m p t i o n s , are shown t o have from s i g n i f i c a n t t o n e g l i g i b l e i m p a c t on t h e w a s t e t r e a t m e n t c o s t . The SENSUMT code w a s u t i l i z e d by F i a c c o and Ghaemi (1982) t o c o n d u c t a s e n s i t i v i t y a n a l y s i s o f a n o n l i n e a r programming problem a s s o c i a t e d w i t h t h e o p t i m i z a t i o n o f t h e s t r u c t u r a l d e s i g n o f a v e r t i c a l l y c o r r u g a t e d t r a n s v e r s e b u l k h e a d of a n o i l tanker.

Major a s s u m p t i o n s and r e s u l t s a s s o c i a t e d w i t h t h e

b a s i c s e n s i t i v i t y a n a l y s i s a p p r o a c h are v e r i f i e d f o r t h e subj e c t problem.

F i r s t - o r d e r s e n s i t i v i t y information i s used t o

estimate measures a s s o c i a t e d w i t h v a r i o u s p e r t u r b a t i o n s of t h e g i v e n problem d a t a .

D e t a i l s o f s t a b i l i t y and convergence

c h a r a c t e r i s t i c s o f t h e s o l u t i o n p o i n t s and t h e i r p a r t i a l der i v a t i v e s w i t h r e s p e c t t o t h e problem p a r a m e t e r s a r e a n a l y z e d . Using t h e r e s u l t s o f t h e p r e v i o u s a n a l y s i s o f t h e c o r r u g a t e d b u l k h e a d model, a c l o s e d form o f t h e o p t i m a l s o l u t i o n o f t h i s model i s d e r i v e d by F i a c c o and Ghaemi (1981a) as a funct i o n of t h e many d e s i g n p a r a m e t e r s .

The a n a l y t i c s o l u t i o n i s

shown t o b e v a l i d o v e r l a r g e c h a n g e s i n t h e p a r a m e t e r s .

This

d e m o n s t r a t e s t h e p o s s i b i l i t y o f making v e r y a c c u r a t e ( i n t h i s i n s t a n c e , e x a c t ) l o c a l estimates o f a markedly n o n l i n e a r

245

8.5 Additional Computational Experience with Applications

p a r a m e t r i c s o l u t i o n , once one s o l u t i o n i s found, even though t h e o r i g i n a l problem s t a t e m e n t might a p p e a r t o make t h i s prohibitive.

I t i s shown t h a t t h e l o c a l p a r a m e t r i c s o l u t i o n

o b t a i n e d c a n be e x t e n d e d t o i n c l u d e l a r g e changes i n t h e p a r a m e t e r s , p a r t i c u l a r l y when t h e p a r a m e t r i c p e r t u r b a t i o n s a r e r e s t r i c t e d and h i g h l y s t r u c t u r e d , a s i s t y p i c a l i n p r a c t i c a l applications.

A s i d e from t h e p r a c t i c a l v a l u e o f s u c h a r e s u l t ,

i t i s r e p o r t e d b e c a u s e t h i s problem i s w e l l p u b l i c i z e d and has been u t i l i z e d f o r some t i m e a s a t e s t problem.

Knowledge o f

t h e p r e c i s e s o l u t i o n f o r a range o f parameter values provides i d e a l information f o r f u r t h e r studies.

Chapter 9

Computable Optimal Value Bounds and Solution Vector Estimates for Parametric NLP Programs

9.1.

INTRODUCTION

In Chapter 2 , Section 2.5,

w e gave a b r i e f p e r s p e c t i v e on

s e v e r a l a p p r o a c h e s t o t h e t h e o r e t i c a l and c o m p u t a t i o n a l c a l c u l a t i o n o f o p t i m a l v a l u e and s o l u t i o n bounds.

W e briefly

summarized t h e d i r e c t i o n w e have t a k e n i n s e e k i n g computable p r o c e d u r e s f o r f i n i t e p a r a m e t e r changes, b a s e d e s s e n t i a l l y on e x p l o i t i n g t h e r i c h s t r u c t u r e o f t h e j o i n t l y convex p a r a m e t r i c program.

T h i s i d e a was developed i n F i a c c o (1981) and i s

presented i n t h i s chapter. Summarizing, a s i m p l e t e c h n i q u e i s p r o p o s e d f o r c a l c u l a t i n g p i e c e w i s e - l i n e a r c o n t i n u o u s g l o b a l upper and lower p a r a -

metric bounds on t h e o p t i m a l v a l u e of n o n l i n e a r p a r a m e t r i c programs t h a t have a convex o r concave o p t i m a l v a l u e f u n c t i o n . This provides a procedure f o r c a l c u l a t i n g parametric optimal v a l u e bounds f o r g e n e r a l nonconvex p a r a m e t r i c programs, whene v e r convex o r concave u n d e r e s t i m a t i n g o r o v e r e s t i m a t i n g problems can be c o n s t r u c t e d .

For t h e j o i n t l y convex program,

t h i s approach l e a d s immediately t o t h e c o n s t r u c t i o n o f a p a r a m e t r i c f e a s i b l e v e c t o r y i e l d i n g a computable and g e n e r a l l y s h a r p e r n o n l i n e a r o p t i m a l v a l u e upper p a r a m e t r i c bound.

246

247

9.2 Piecewise Linear Upper and Lower Optimal Value Bounds

Connections and e x t e n s i o n s o f w e l l known d u a l i t y r e s u l t s a r e developed t h a t l e a d t o c o n s t r u c t i v e i n t e r p r e t a t i o n s o f t h e bounds r e s u l t s and g e n e r a l l y s h a r p e r n o n l i n e a r p a r a m e t r i c lower An approach i s proposed f o r t h e c a l c u l a t i o n o f s o l u -

bounds.

t i o n p o i n t p a r a m e t r i c bounds, b a s e d on o u r a b i l i t y t o c a l c u l a t e t h e o p t i m a l v a l u e f u n c t i o n bounds. COMPUTABLE PIECEWISE LINEAR UPPER AND LOWER OPTIMAL VALUE BOUNDS

9.2.

The a u t h o r r e c e n t l y p r o p o s e d a s i m p l e p r o c e d u r e f o r o b t a i n i n g computable g l o b a l upper and lower p a r a m e t r i c bounds on t h e o p t i m a l v a l u e o f g e n e r a l n o n l i n e a r p a r a m e t r i c programs f o r which o v e r e s t i m a t i n g and u n d e r e s t i m a t i n g problems can be c o n s t r u c t e d t h a t have e i t h e r a convex o r concave o p t i m a l v a l u e function.

The c l a s s o f programming problems t h a t can be s o

bounded t u r n s o u t t o be e x t r e m e l y l a r g e , and hence t h e potent i a l applications are vast.

The i d e a i s b a s e d on p r o p e r t i e s

t h a t a r e e l e m e n t a r y and w e l l known b u t a p p a r e n t l y have n o t been p r e v i o u s l y e x p l o i t e d o r s y s t e m a t i c a l l y developed f o r t h e i n d i c a t e d purpose. The bounding p r o c e d u r e may p e r h a p s b e most e a s i l y conveyed g e o m e t r i c a l l y , f o r problems whose o p t i m a l v a l u e f u n c t i o n i s convex. min

C o n s i d e r t h e g e n e r a l problem X

f(x,

E)

s.t. g ( x ,

E)

2 0, h(x,

I f P 3 ( € ) i s j o i n t l y convex, i . e . , and

E,

E)

= 0.

i f f i s j o i n t l y convex i n x

t h e components of g a r e j o i n t l y concave i n x and

t h o s e o f h j o i n t l y a f f i n e i n x and

P3 ( € 1

E,

E,

and

t h e n i t h a s been known

f o r some t i m e (Mangasarian and Rosen, 1 9 6 4 ) t h a t t h e o p t i m a l

248

9 Optimal Value Bounds and Solution Vector Estimates

/ Y =e2

?

y = f*(c)

--L/

--/

Fig.

C onv e x o p t i m a l v a l u e f u n c t i o n and b o u n d s .

9.2.2.

*

v a l u e f u n c t i o n f ( € 1 o f P 3 ( E ) i s a convex f u n c t i o n of

Under

E.

t h e s e circumstances a t y p i c a l graph o f t h e optimal value f f r e l a t i v e t o a g i v e n component E~

i s shown i n F i g . 9 . 2 . 1 . Suppose w e have e v a l u a t e d f values of

E,

*

(E)

of

with

E

E

j

*

of

fixed, j # i,

and i t s s l o p e a t two

t h o s e f o r which component

E~

( f o r s i m p l i c i t y ) t h e o t h e r components o f

= 0 and E

Fit and

a r e f i x e d a t 0.

Then, under g e n e r a l c i r c u m s t a n c e s , t h e r e e x i s t t h e s u p p o r t s y = a , ( € ) and y = ! L 2 ( ~ ) o f ( t h e e p i g r a p h o f ) f * ( ~ a) t and

E~

= Ti,

r e s p e c t i v e l y , above which f

l i n e y = ! L 3 ( ~ ) through LO, f

*

(E)

must l i e , so t h a t f

*

f * (0)I (E)

,

[Ti,

*

(E)

Ei.

= 0

must l i e , and t h e

f * (Fi) I

,

below which

must l i e i n t h e c r o s s - h a t c h e d

t r i a n g u l a r a r e a i n d i c a t e d i n F i g . 9 . 2 . 1 when 0 and

E~

E~

v a r i e s between

I f f * ( ~ i) s d i f f e r e n t i a b l e , t h e n c o n v e x i t y o f f * ( ~ )

249

9.3 Parametric Solution Vector and Sharper Convex Upper Bound

implies t h a t f*(E) 2 f*(T)

+

V-f*(T) t

(E

-

T ) , and i t f o l l o w s t h a t

1

f

*

* af (0) + aEi

(0)

i'

n o t d i f f e r e n t i a b l e , t h e n any s u b g r a d i e n t of f any v e c t o r v s u c h t h a t f * ( E ) 2 f * ( E ) +

through [ T I f * (T) I. derivative of f

*

(E)

- E) ,

Vt ( E

would s u f f i c e t o y i e l d a g l o b a l l i n e a r lower bound on f

*

*

(E)

I n p a r t i c u l a r , w e can use t h e d i r e c t i o n a l

a t any p o i n t i n t h e i n t e r i o r o f i t s domain,

s i n c e t h i s i s known t o e x i s t ( R o c k a f e l l a r , 1 9 7 0 b ) . s i m i l a r c o n s t r u c t s apply i f f

*

(E)

i s concave.)

(Obviously,

Thus, i n t h e

i n t e r i o r o f i t s domain, e a s i l y computable upper and lower bounds on f

*

f o r any f i n i t e changes i n

E

can be p r o v i d e d by

computing two s o l u t i o n s o f P 3 ( E ) c o r r e s p o n d i n g t o two v a l u e s of

E

and t h e a s s o c i a t e d s u p p o r t s o f t h e e p i g r a p h o f f

t h e s e l e c t e d values of t h e parameters.

*

(E)

at

A s n o t e d , most of t h e

r e q u i r e d i n f o r m a t i o n i s p r o v i d e d by most s t a n d a r d NLP a l g o r i t h m s when a s o l u t i o n h a s been d e t e r m i n e d f o r t h e g i v e n parameter v a l u e s and t h e a s s o c i a t e d o p t i m a l Lagrange m u l t i pliers exist. 9.3.

ESTIMATES OF A PARAMETRIC SOLUTION VECTOR AND A SHARPER CONVEX UPPER BOUND

An e x t r e m e l y i m p o r t a n t b y p r o d u c t o f t h i s bounds c a l c u l a t i o n i s i t s immediate a p p l i c a b i l i t y t o r n u l t i p a r a m e t r i c p e r t u r -

bation and t h e a v a i l a b i l i t y o f an a s s o c i a t e d f e a s i b l e parametric vector.

Denote by

parameter v e c t o r

E

e E

E~

k

.

and

E,

two d i s t i n c t v a l u e s o f t h e

Suppose F ( E ~i )s f e a s i b l e t o problem

P 3 ( ~ 1 ) and x ( E ~ ) s o l v e s P 3 ( ~ l ) l w h i l e X ( E , ) i s f e a s i b l e t o

250

9 Optimal Value Bounds and Solution Vector Estimates

P3(~2)w r ith

+

Define € ( a )

*

3 (E) is jointlyconvex. x ( a ) = a ? i ( ~ , ) + (1 - ~ ) X ( E ~ ) ,

its solution.

x ( E ~ )

-

(1

Assume P

N

+ (1 - a ) x ( s l ) , where

and x ( a ) =

0 5 a 5 1.

I t can

e a s i l y be shown t h a t ? ( a ) (hence a l s o x * ( a ) ) i s i n R [ E ( ~ ) ] f o r a 5 1 where R ( E ) i s t h e f e a s i b l e r e g i o n o f P 3 ( ~ ) . ( I n

any 0

t h e t e r m i n o l o g y o f p o i n t t o s e t maps (see S e c t i o n 2 . 2 )

,

means t h e g r a p h G o f R i s a convex s u b s e t o f Ek

i.e.,

G =

{(E,

x)

IE

E

E

k

,x *

E

En,

x

R ( E ) ) i s a convex s e t . )

this

Furthermore,

i t f o l l o w s t h a t f [ x ( a ), € ( a ) ] i s a convex f u n c t i o n o f a t h a t l i e s above t h e (unknown) o p t i m a l v a l u e f

*

[ € ( a ) ]and on o r be-

*

l o w t h e known l i n e a r upper bound y = R 3 [ ~ ( a ) ]o f f [ ~ ( a ) ] f,o r 0 5 a 5 1, w i t h f [ x * ( O ) , ~ ( 0 1 1= f*(E1) and f [ x * ( l ) , ~ ( 1 1 1= f

*

(E2).

These f a c t s a r e a c t u a l l y i n v o l v e d i n d e m o n s t r a t i n g t h e convexity of f

*

and i t s domain (Mangasarian and Rosen, 1 9 6 4 ) S i n c e P 3 ( ~ )i s j o i n t l y

and a r e r e a d i l y shown a s f o l l o w s . convex, i t f o l l o w s t h a t g [ ? ( a ) , € ( a ) ] :g [ a X ( E 2 )

+

(1

>

ag[51(E2),

>

0.

-

E21

~ ) X ( E ~ ) , + (1

+

(1

-

a)g[X(E,)

r

Ell

The f i r s t i n e q u a l i t y (meaning t h a t t h e i n e q u a l i t y h o l d s f o r e a c h component o f g ) f o l l o w s from j o i n t c o n c a v i t y o f e a c h gir and t h e second from t h e f e a s i b i l i t y o f X ( E ~ f) o r P 3 ( ~ 1 ) and f o r P 3 ( ~ 2 ) . Since t h e equality c o n s t r a i n t s a r e j o i n t l y a f f i n e i f P3(€) i s j o i n t l y convex, t h e a n a l o g o u s r e l a t i o n s h o l d f o r h [ z ( a ), equalities.

E

( a )]

Hence,

, with

z(a) E

t h e i n e q u a l i t i e s r e p l a c e d by

R [ E ( a )1.

25 I

9.3 Parametric Solution Vector and Sharper Convex Upper Bound

If f(x,

E)

i s j o i n t l y convex, t h e n

*

t h e f i r s t i n e q u a l i t y f o l l o w i n g from t h e f a c t t h a t x ( a ) E R [ ~ ( a ) l which , f o l l o w s from t h e p r e c e d i n g r e s u l t , and t h e second from t h e c o n v e x i t y assumption. (x,

E)

and t h e f a c t t h a t ( x * ,

€ ( a ) ] i s convex i n a .

*

E)

The c o n v e x i t y o f f i n

i s a f f i n e i n a imply f [ x * ( a ) ,

Thus, t h e s t a t e d r e s u l t s h o l d .

The d e p i c t i o n o f f [ € ( a ) ] and i t s bounds a r e shown i n F i g . 9.3.1.

For s i m p l i c i t y , w e d e n o t e L i [ e ( a ) ] by a i ( a ) .

f i g u r e i s comparable t o Fig.

*

i - t h component o f x ( a )

I

E

9.2.1.

This

W e have a l s o d e p i c t e d t h e

R [ E ( ~ ) ]i n t h e f i g u r e .

I

1

*

Note t h a t

a

B o u n d s o n t h e o p t i m a l 3 a l u e f [€(a)] and t h e F i g . 9.3.1. i - t h component of t h e f e a s i b l e v e c t o r x ( a ) , when t h e problem i s s o l v e d for a = 0, 1. A l l s o l i d l i n e s r e p r e s e n t computable quantities.

252

9 Optimal Value Bounds and Solution Vector Estimates

t h e a s s u m p t i o n t h a t P J ( ~ )i s j o i n t l y convex c a n be r e l a x e d . I t s u f f i c e s t h a t P [ ~ ( a ) be l j o i n t l y convex i n x and a t o o b t a i n

t h e s e r e s u l t s and t h i s d e p i c t i o n f o r

E

=

E

(a).

T h i s remarkably r e g u l a r b e h a v i o r i s e x p l o i t a b l e i n many

*

Since x ( a ) is t h e r e s u l t of l i n e a r i n t e r p o l a t i o n

ways.

between two g i v e n s o l u t i o n s , i f a d d i t i o n a l s o l u t i o n s are obt a i n e d c o r r e s p o n d i n g t o i n t e r m e d i a t e v a l u e s o f a , 0 < a < 1, then it is apparent t h a t a piecewise-linear

continuous f e a s i b l e

p a r a m e t r i c v e c t o r s ( a ) can be o b t a i n e d by l i n e a r i n t e r p o l a t i o n between c o n t i g u o u s s o l u t i o n s .

Further, t h e function f

*

[E

( a )I

w i l l l i e between p i e c e w i s e - l i n e a r c o n t i n u o u s u p p e r and l o w e r p a r a m e t r i c bounds t h a t c a n be r e a d i l y computed, and t h a t w i l l become t i g h t e r w i t h e a c h new i n t e r m e d i a t e s o l u t i o n t h a t i s computed, e . g . ,

a t a v a l u e o f a where t h e d e v i a t i o n between

t h e upper and l o w e r bounds i s g r e a t e s t .

*

Hence, i f a p r e s c r i b e d

e r r o r t o l e r a n c e i n e s t i m a t i n g f [ € ( a ) ]f o r 0 5 a 5 1 i s g i v e n , t h e n w e need o n l y s o l v e P 3 [ (~a )1 f o r s e v e r a l v a l u e s o f a i n

[O, 1 1 , u n t i l t h e bounds on f * [ E ( a )1 do n o t d e v i a t e by more than t h e given t o l e r a n c e . I t f o l l o w s from what w a s shown t h a t t h e o b j e c t i v e f u n c t i o n

f(x,

E)

a l o n g [ ; ( a ) , € ( a ) ] i s a piecewise-convex

continuous

f u n c t i o n , does n o t underestimate t h e a c t u a l optimal value

*

f u n c t i o n f [ € ( a ) ,] and h a s i t s g r a p h on o r below t h e known piecewise-linear

upper bound, o v e r t h e s p e c i f i e d r a n g e o f a .

Thus, w i t h e v e r y s o l u t i o n c o r r e s p o n d i n g t o an i n t e r m e d i a t e v a l u e o f a w e may e a s i l y c a l c u l a t e , i n a d d i t i o n t o b e t t e r piecewise-linear

c o n t i n u o u s upper and lower bounds, a more

a c c u r a t e p i e c e w i s e - l i n e a r c o n t i n u o u s f e a s i b l e estimate o f t h e p a r a m e t r i c s o l u t i o n v e c t o r a l o n g w i t h a more a c c u r a t e piecewise-convex

continuous overestimate of t h e optimal value

253

9.3 Parametric Solution Vector and Sharper Convex Upper Bound

II

\

*

F i g . 9.3.2. B o u n d s on t h e o p t i m a l _ v a l u e f [ ~ ( a ) a] n d t h e - t h c o m p o n e n t of t h e f e a s i b l e v e c t o r x ( a l , when t h e p r o b l e m s s o l v e d f o r a = 0, a, 1 . A l l s o l i d l i n e s r e p r e s e n t comput a b 1 e quan t i t i e s

.

function.

W e i l l u s t r a t e t h e p a r a m e t r i c o p t i m a l v a l u e bounds

and a s s o c i a t e d f e a s i b l e v e c t o r s o l u t i o n e s t i m a t e a v a i l a b l e a f t e r s o l u t i o n s have been o b t a i n e d f o r t h r e e v a l u e of a , i n F i g . 9.3.2. T h i s p r o v i d e s a g e n e r a l approach f o r g e n e r a t i n g o p t i m a l v a l u e bounds and an a s s o c i a t e d p a r a m e t r i c s o l u t i o n v e c t o r e s t i m a t e f o r g e n e r a l n o n l i n e a r p a r a m e t r i c problems.

Most o f

t h e s e c o n c e p t s a r e w e l l known and s i m p l e , t h e e s s e n t i a l n o v e l t y b e i n g t h e i r e x p l o i t a t i o n , a d a p t a t i o n , and t a i l o r i n g t o

254

9 Optimal Value Bounds and Solution Vector Estimates

t h e p u r p o s e a t hand.

The c a l c u l a t i o n s i n v o l v e t h e manipula-

t i o n of i n f o r m a t i o n g e n e r a l l y a l r e a d y p r o v i d e d by s t a n d a r d so lution algorithms. There a r e , however, numerous i m p o r t a n t t h e o r e t i c a l and a l g o r i t h m i c r e f i n e m e n t s t h a t must b e made t o s h a p e t h e s e t e c h n i q u e s i n t o a c o h e r e n t , i n c i s i v e and e f f i c i e n t methodology. T h e i r development, v a l i d a t i o n , and c o m p u t a t i o n a l i m p l e m e n t a t i o n a r e b e i n g pursued.

But i t a l r e a d y a p p e a r s p l a u s i b l e t h a t an

abundance of s e n s i t i v i t y and s t a b i l i t y i n f o r m a t i o n can be c a l c u l a t e d f o r any s o l v a b l e NLP problem, w i t h r e l a t i v e l y l i t t l e a d d i t i o n a l computational e f f o r t .

The k i n d o f i n f o r m a t i o n t h a t

can be c a l c u l a t e d b e g i n s t o be somewhat comparable i n s c o p e and d e t a i l t o t h e e s s e n t i a l p o s t - o p t i m a l i t y p a r a m e t r i c s e n s i t i v i t y and range a n a l y s i s i n f o r m a t i o n t h a t i s h e a v i l y u t i l i z e d i n l i n e a r programming.

I n f a c t , it seems v e r y l i k e l y t h a t NLP

s t a b i l i t y r e s u l t s w i l l b r i n g new t e c h n i q u e s f o r e s t i m a t i n g t h e e f f e c t s o f l a r g e p e r t u r b a t i o n s i n l i n e a r programming a s w e l l . Many r e s e a r c h d i r e c t i o n s r e q u i r e e x p l o r a t i o n i n d e v e l o p i n g t h e proposed bounds t e c h n i q u e s .

W e mention a few:

(1) s y s -

t e m a t i c a c c u m u l a t i o n o f c o m p u t a t i o n a l e x p e r i e n c e w i t h moderates i z e d j o i n t l y convex programs, a l l o w i n g f o r t h e s i m u l t a n e o u s perturbation of s e v e r a l parameters;

( 2 ) development o f t h e

o p t i m a l v a l u e bounding p r o c e d u r e f o r s e p a r a b l e and f a c t o r a b l e programs, based on known p r o c e d u r e s f o r f o r m u l a t i n g a j o i n t l y convex u n d e r e s t i m a t i n g problem f o r t h e g i v e n p a r a m e t r i c nonconvex problem, a p p l y i n g t h e bounding t e c h n i q u e t o t h e convex problem, and b r a n c h and bound and o t h e r t e c h n i q u e s f o r i m proving accuracy;

( 3 ) a d e e p e r a n a l y s i s o f c o n d i t i o n s on t h e

problem f u n c t i o n s t h a t a r e s u f f i c i e n t f o r t h e c o n v e x i t y o r

255

9.4 Connections between Optimal Value Bounds and Duality

concavity of the optimal value function f o r s p e c i f i e d , s i m p l e , e. g.

, vector)

(often

p e r t u r b a t i o n s of problem p a r a m e t e r s ,

a l l o w i n g f o r t h e a p p l i c a b i l i t y of t h e o p t i m a l v a l u e bounding p r o c e d u r e f o r nonconvex problems w i t h o u t r e q u i r i n g t h e cons t r u c t i o n o f convex u n d e r e s t i m a t i n g o r o v e r e s t i m a t i n g problems; ( 4 ) c o n t i n u e d d e m o n s t r a t i o n s o f p r a c t i c a l a p p l i c a b i l i t y ; and

(5) t h e f u r t h e r development o f c o m p u t a t i o n a l t e c h n i q u e s t h a t e x p l o i t s t a n d a r d NLP a l g o r i t h m c a l c u l a t i o n s analogous t o t h o s e i n t r o d u c e d by t h e a u t h o r f o r p e n a l t y f u n c t i o n a l g o r i t h m s , t h u s , h o p e f u l l y , s t i m u l a t i n g t h e r o u t i n e and w i d e s p r e a d use of s e n s i t i v i t y and s t a b i l i t y i n f o r m a t i o n f o r a l l s t a n d a r d NLP a l g o r i t h m s j u s t a s i n l i n e a r programming. CONNECTIONS BETWEEN OPTIMAL VALUE BOUNDS AND DUALITY

9.4.

The f o r e g o i n g o p t i m a l v a l u e bounds r e s u l t s have many p o i n t s of contact with d u a l i t y theory.

I n f a c t , t h e bounding pro-

c e d u r e and i t s c o m p u t a b i l i t y w e r e i n i t i a l l y s u g g e s t e d t o t h e a u t h o r by t h e f o l l o w i n g w e l l known c o n n e c t i o n s w i t h o p t i m a l i t y Suppose t h e problem i s a r h s

c o n d i t i o n s and d u a l i t y r e s u l t s . p e r t u r b a t i o n problem o f t h e form min X f ( x )

s . t . g ( x ) 1.

P1

E,

An immediate c o n n e c t i o n e a s i l y f o l l o w s :

i f t h e Karush-Kuhn-

Tucker c o n d i t i o n s h o l d a t [ x ( E ) , u ( E ) ] and i f unique and d i f f e r e n t i a b l e n e a r E =

0.

E =

[x(E),u(E)I

0 , t h e n V,f*(E) =

is

u ( E ) n~e a r

Suppose a l s o t h a t P 1 ( ~ ) i s convex i n x and f o r

s i m p l i c i t y assume t h a t o n l y one p a r a m e t e r E

(€1

1. 0 , t h a t a s o l u t i o n

X(E)

E~

i s perturbed, t h a t

of P 1 ( ~ ) e x i s t s ( f o r a l l

E

~

)

256

9 Optimal Value Bounds and Solution Vector Estimates

s a t i s f y i n g t h e Karush-Kuhn-Tucker

c o n d i t i o n s , and t h a t g [ x ( f ) ]

*

Then i t f o l l o w s t h a t t h e o p t i m a l v a l u e f

= E.

and n o n d e c r e a s i n g a s a f u n c t i o n of

E

,as

i s convex

i n Fig. 9.2.1.

At

= Ti and f = 0, j # i, i j and y = Q 2 ( ~ )a s d e p i c t e d i n F i g .

t h e s o l u t i o n s x ( 0 ) and x ( Y ) , where t h e lower bounds y = Q 1 ( € )

~

(f)

f

a p p l y , and have n o n n e g a t i v e s l o p e s c o r r e s p o n d i n g t o 1 2 n o n n e g a t i v e o p t i m a l Lagrange m u l t i p l i e r s , s a y ui and u i r * 1 respectively. C l e a r l y , w e o b t a i n a , ( € ) = f (0) + u i ~ i a n d 9.2.1

Q2(f)

= f

*

(F) +

u

2

~

(

-E T~i ) .

I t i s c l e a r from F i g .

9.2.1

that

z

any s u c h t a n g e n t t o t h e g r a p h o f f " ( E ) w i l l i n t e r s e c t t h e y a x i s a t o r below t h e o p t i m a l v a l u e f

*

(E)

at

E

= 0.

I n f a c t , i t may be i n f e r r e d from t h e a s s u m p t i o n s and t h e figure that m

y

-

2

UiEi

= f

*

(El

-

(9.4.1)

m i=l

= L t X ( E ) r u ( T ) r 01,

where x ( 0 ) s o l v e s P1 ( 0 ) w i t h a s s o c i a t e d o p t i m a l Lagrange 1 m u l t i p l i e r u ( 0 ) s u c h t h a t u i ( 0 ) = u i r and x ( E ) s o l v e s P1(F) w i t h a s s o c i a t e d o p t i m a l Lagrange m u l t i p l i e r u(.) u.

(z)

P1(0).

2 = uir

such t h a t

L d e n o t i n g t h e u s u a l Lagrangian a s s o c i a t e d w i t h

Hence, t h e 111 and R 2 y - i n t e r c e p t s are L [ x ( O ) , u ( o ) , 0 1

and L[x(EI) , u ( 5 ) , 0 1 , r e s p e c t i v e l y .

I t i s o b v i o u s from t h e

f i g u r e and t h e assumptions t h a t f * ( O ) = L [ x ( O ) ,

U(0)r

01.

257

9.4 Connections between Optimal Value Bounds and Duality

Dual p r o b l e m s i n v o l v e d t h e d e t e r m i n a t i o n o f t h e s l o p e (Lagrange m u l t i p l i e r ) of a s u p p o r t i n g l i n e ( o r p l a n e , i n general) l i k e y =

a,(€)

s u c h t h a t t h e minimal L a g r a n g i a n v a l u e

o v e r x ( i . e . , t h e y i n t e r c e p t ) i s maximized.

For example, t h e

Wolfe d u a l (Wolfe, 1 9 6 1 ) o f t h e convex program minx f ( x )

s. t. g ( x ) 1. 0

(PI

i s g i v e n by

max

(x,u)

s . t . VXL(X, u ) = 0 , u 2 0 ,

L(x, u)

where L(x, u ) = f ( x )

-

(D) It is

Cu.g. ( x ) , t h e u s u a l Lagrangian. 1 1

e a s i l y shown t h a t (9.4.2)

w h e r e R i s t h e f e a s i b l e r e g i o n of p r o b l e m P a n d t o t h e f e a s i b l e s e t F$, o f D.

(G, G)

belongs

Under a p p r o p r i a t e r e g u l a r i t y

conditions it a l s o follows t h a t min f ( x ) = max L(y, u ) , xeR ( Y , u ) '% a n d t h e s e c o n d i t i o n s e s s e n t i a l l y summarize t h e d u a l i t y relationship. To e s t a b l i s h a c o n n e c t i o n between t h i s d u a l i t y c o n c e p t and

t h e bounds d i s c u s s e d e a r l i e r a n d d e p i c t e d i n F i g . 9 . 2 . 1 ,

con-

s i d e r t h e Wolfe d u a l o f P 1 ( € ) , g i v e n by

s.t. VXL(X, u,

E)

-

= 0, u 2 0 ,

E ] and VxL = Vxf

-

For s i m p l i c i t y , s u p p o s e P 1 ( ~ ) h a s a s o l u t i o n f o r any

E.

where L ( x , u ,

E)

= f (x)

T u [g(x)

-

D1

i m p o r t a n t p r o p e r t i e s o f D 1 ( ~ ) s h o u l d be n o t e d : s t r a i n t s of D 1 ( € )

do n o t depend o n

E

(€1

T

u Vxg.

Two

( i )t h e con-

a n d ( i i )t h e d u a l i t y

258

9 Optimal Value Bounds and Solution Vector Estimates

r e s u l t s j u s t g i v e n imply t h a t t h e o p t i m a l v a l u e f

*

( € 1 of P 1 ( ~ )

i s bounded below by t h e o b j e c t i v e f u n c t i o n of D 1 ( E ) a t any i.e.,

f e a s i b l e p o i n t of D l ( 0 ) , f * ( E ) 1. L ( 2 ,

6,

E

= f(2) = GTE

f o r every

(2, 6)

-

GT[g(2)

+ L ( 2 , 6,

E

= 0,

El

(9.4.3)

0)

t h a t s a t i s f i e s VxL = 0 and u 1. 0.

e r a l i z e s t h e d u a l i t y r e s u l t (9.4.2) for

-

T h i s gen-

j u s t g i v e n (which a p p l i e s

e s s e n t i a l l y ) and p r o v i d e s immediate c o n n e c t i o n s and

e x t e n s i o n s o f t h e bounds r e s u l t s g i v e n e a r l i e r . Feasible p o i n t s of D1(€)

Suppose t h e problem P 1 ( ~ ) i s s o l v e d

perturbations of P1(0). a t T by x ( T ) .

may be found by s o l v i n g

Then, under a p p r o p r i a t e r e g u l a r i t y c o n d i t i o n s

t h e r e e x i s t s u ( T ) 1. 0 s u c h t h a t VXL[x(E), u ( T ) ,

-

E]

= 0 ; hence

T

[ x ( T ) , u ( E ) I i s d u a l f e a s i b l e , and f u r t h e r , u ( S ) [ g [ x ( ? ) I W e n o t e i n p a s s i n g t h a t i f t h e s e Karush-Kuhn-Tucker

= 0.

-

d i t i o n s hold, then f * (E) = L [ x ( Z ) , u ( s ) , € 1 ,

- El con-

and hence t h e

above r e l a t i o n s imply t h a t [ x ( T ) , u ( T ) 3 s o l v e s t h e d u a l D1 (T) of P1(E).

R e t u r n i n g t o t h e bounds r e l a t i o n s h i p , w e o b s e r v e

t h a t [ x ( T ) , u ( T ) ] i s a f e a s i b l e point of the dual D 1 ( ~ ) of P1(€)

f o r any

E.

Thus, from (9.4.3) w e c o n c l u d e t h a t

f * ( E ) 2 L[X(E), u ( + ) ,

El

= f[x(T)I T = u(T) E

-

+

u(T)T(S

-

E)

L [ x ( T ) , U ( E ) , 01.

(9.4.4)

As o b s e r v e d , under a p p r o p r i a t e r e g u l a r i t y a s s u m p t i o n s t h i s bound i s m e t when

strict for a l l

E

E

= 7 , with t h e i n e q u a l i t y generally being

# T i n t h e domain o f f

*

(E).

I t i s c l e a r t h a t i f P 1 ( € ) s a t i s f i e s t h e c o n d i t i o n s imposed

a t t h e b e g i n n i n g o f t h i s s e c t i o n t h e n t h e lower bound on f t h a t w e have j u s t o b t a i n e d i n (9.4.4)

*

(E)

i s t h e same a s ! L z ( ~ ) i n

259

9.5 Nonlinear Dual Lower Bounds

t h e bound y =

i n F i g . 9.2.1

t h a t w e previously calculated

The bound (9.4.4) i s more g e n e r a l i n a l l o w i n g t h e

i n (9.4.1).

p e r t u r b a t i o n o f t h e e n t i r e parameter v e c t o r

E,

one component a s i n (9.4.1) and F i g . 9 . 2 . 1 . c o r r e s p o n d e n c e w i t h t h e bound y =

a,(€)

r a t h e r than only

Of c o u r s e , t h e

i n F i g . 9.2.1

(9.4.1) is a l s o e s t a b l i s h e d a s above, r e p l a c i n g

F

and

by 0.

Sum-

m a r i z i n g , i t f o l l o w s t h a t a l i n e a r lower bound on t h e o p t i m a l value of f*(E) of P 1 ( ~ ) a t 7, supporting t h e set of (y,

E

(E)

-

E)

such

,

c o r r e s p o n d s t o t h e Lagrangian Wolfe d u a l v a l u e

, where

x is e v a l u a t e d a t a s o l u t i o n x ( F ) o f P1 ( € 1 f o r

t h a t y 1. f o f P1

*

(E)

= E , w i t h u e q u a l t o t h e a s s o c i a t e d o p t i m a l Lagrange m u l t i -

Assuming t h a t [ x ( T ) , u ( F ) I s a t i s f i e s t h e Karush-

p l i e r u(E).

Kuhn-Tucker c o n d i t i o n s , i t f o l l o w s t h a t f* ( E ) = L [ x ( E ) , u ( s ) ,

E]

and [ x ( F ) , u ( F ) ] i s a f e a s i b l e p o i n t of D 1 ( € ) *

for a l l

E

in

t h e domain o f f " ( € 1 . 9.5.

NONLINEAR DUAL LOWER BOUNDS

This d u a l i t y i n t e r p r e t a t i o n leads t o f u r t h e r extensions of t h e bounds r e s u l t s .

Observe a g a i n t h a t t h e two e s s e n t i a l

properties of t h e dual D1(€) bounds on f

*

(E)

t h a t l e a d t o computable lower

a r e t h e d u a l lower bound (9.4.3) and t h e l a c k

o f dependence of t h e c o n s t r a i n t s o f D 1 ( € )

on

E.

This permits

t h e c a l c u l a t i o n o f t h e p a r a m e t r i c lower bound (9.4.3) whenever

any feasible point

( 2 , 6) of

D1(€)

h a s been computed.

t h e s e c o n d i t i o n s and r e s u l t s a l s o h o l d f o r t h e d u a l t h e more g e n e r a l p a r a m e t r i c problem min

X

where ? =

f(x)

s . t . g ( x ) 1. F ( E ) ,

T(E)is

an m-component v e c t o r f u n c t i o n .

But

s l ( ~of)

260

9 Optimal Value Bounds and Solution Vector Estimates

Assuming

pl(~) is

ma x

L(xr

(Xru)

Feasible points

ur

convex i n x , t h e Wolfe d u a l i s

-

8) s . t . V X ~ ( x U, ,

(x, u) o f

El(€)

E)

-

~ ~ ( € 1

u 2 0.

= 0,

a g a i n do n o t depend on

E

and a

bound o f t h e form (9.4.3) may be c a l c u l a t e d a n a l o g o u s l y , y i e l d i n g f o r any d u a l f e a s i b l e p o i n t

(z, 5 )

El(€)

of

the

inequality f*(E) 2 L G , 5, 7 ) = f ( z )

-

= -Tu g(E) i n t e r m s of t h e c o n s t i t u e n t s of

GT[9(Z)

+

-

LG,

G,

F1(€)

and

?(E)

1

0)

(9.5.1)

C1 ( € 1 .

In particu-

l a r , by analogy t o t h e p r e v i o u s t r e a t m e n t w i t h r e s p e c t t o P1(€) and D 1 ( ~ )t h a t g i v e s (9.4.4), w e c o n c l u d e from (9.5.1) t h a t f * ( E ) 2 L [ x ( ' i ) , U(E) ,

-

= U ( a T g E )

El

+ L[X(E)r u(E),

-

(9.5.2)

OIr

where f * and L now r e f e r t o t h e o b j e c t i v e f u n c t i o n s o f D1, r e s p e c t i v e l y , and x ( T ) now s o l v e s

o p t i m a l Lagrange m u l t i p l i e r u ( F ) .

f[x(E)I

-

I f t h e Karush-Kuhn-Tucker

e q u a l i t y when

E

= F.

g e n e r a l i z e (9.4.3) for a l l

E

- a ( € ) ] and ,

u ( S ) [:(+)

i n t h e domain o f f

o f a l l components o f

(9.5.2)

E,

*

becomes

o f c o u r s e (9.5.2) h o l d s w i t h

The bounds g i v e n by (9.5.1)

and (9.4.4).

and

1 (a) w i t h a s s o c i a t e d

conditions hold, then t h e right-hand s i d e of T

p1

and (9.5.2)

They a r e g l o b a l ( i . e . , v a l i d allow f o r the perturbation

(E)),

and do n o t r e q u i r e any c o n v e x i t y re-

s t r i c t i o n s on ? ( € ) . I t i s n o t e d , however, t h a t i f t h e components o f

F ( E )a r e

convex o r concave, t h e n t h e (more g e n e r a 1 ) l o w e r bound (9.5.1)

i s a c c o r d i n g l y convex o r concave. Lagrangian ( d u a l ) lower bound on f

*

I n t h e former c a s e , t h e (E)

w i l l obviously coincide

261

9.5 Nonlinear Dual Lower Bounds

w i t h o r b e s h a r p e r t h a n t h e bound d e r i v e d from i t s l i n e a r i z a t i o n (as i n Fig. 9.2.1) f

*

a t any v a l u e o f

E

( € 1 , o v e r a n y s u b s e t o f t h e domain o f f

*

i n t h e domain o f (E).

In the latter

case, l i n e a r i n t e r p o l a t i o n between t w o p a r a m e t e r v a l u e s would y i e l d a l i n e a r lower bound on ( 9 . 5 . 1 )

-

and h e n c e o n f

*

(E)

.

I t s h o u l d be n o t e d t h a t t h e d u a l i t y r e s u l t s g i v e n f o r

P1(€) e x t e n d i n a n o b v i o u s manner t o t h e more g e n e r a l inequality-equality

s . t . g ( x ) 1. q

rnin X f ( x )

- (€1

c o n s t r a i n e d r h s convex program of t h e form

( ~, )h ( x )

=

G(E),

P2

-

where h ( x ) i s a f f i n e i n x , r e c a l l i n g t h a t t h e Wolfe d u a l o f P 2 ( ~ )i s

where L now d e n o t e s t h e L a g r a n g i a n L ( x , U,

of

W,

7)

F 2 ( € )and 7

=

d o n o t depend on

f(x)

-

T

u [g(x)

[ G ( E ), h ( E ) 1. ,.s

E,

-

T(E)I

+ wT [ h ( x )

-

c(E)1

A g a i n , t h e c o n s t r a i n t s of

and a bound o f t h e form ( 9 . 5 . 2 )

Ej2(€)

applies.

The f a c t t h a t n o c o n v e x i t y ( o r c o n c a v i t y ) r e s t r i c t i o n s a r e needed on

-

;(E)

or

X(E)

t o c a l c u l a t e bounds may g r e a t l y f a c i l i -

t a t e t h e c a l c u l a t i o n of u s e f u l bounds f o r p r o g r a m s o f t h e form

P1(€) o r

F 2 ( ~ t)h a t

are n o t convex i n x.

In particular,

it

s u f f i c e s t o o b t a i n a convex u n d e r e s t i m a t i n g problem i n x ,

i.e.

, without

altering

F(E)o r

%(E).

The d u a l i t y r e s u l t s j u s t

g i v e n c a n be a p p l i e d t o t h i s p r o b l e m , y i e l d i n g a lower bound of t h e form ( 9 . 5 . 1 )

o n i t s o b j e c t i v e f u n c t i o n v a l u e , hence on

262

9 Optimal Value Bounds and Solution Vector Estimates

t h e o p t i m a l v a l u e of t h e o r i g i n a l problem, whenever a d u a l f e a s i b l e p o i n t of t h e u n d e r e s t i m a t i n g problem h a s been calculated. I t i s s t r e s s e d t h a t it i s n o t necessary t o solve t h e primal

problem i; ( E ) o r 1

P 2 ( & )f o r

some v a l u e o f

i n t e r e s t t o o b t a i n a lower bound on f

*

i n t h e domain of

E

It suffices t o

(E).

o b t a i n any d u a l - f e a s i b l e p o i n t t o e v a l u a t e t h e p a r a m e t r i c lower bound g i v e n i n ( 9 . 5 . 1 ) .

T h i s f a c t i s e x p l o i t a b l e by any

NLP a l g o r i t h m t h a t c a l c u l a t e s a sequence o f d u a l f e a s i b l e p o i n t s i n approaching a s o l u t i o n .

I n p a r t i c u l a r , t h e penalty

f u n c t i o n c a l c u l a t i o n s t h a t w e have h e a v i l y u t i l i z e d f o r t h e a l g o r i t h m i c e s t i m a t i o n o f s e n s i t i v i t y a n a l y s i s ( F i a c c o , 1976) a l s o p r o v i d e an e x c e l l e n t example o f t h i s b a s i c i d e a o f an i t e r a t i v e a l g o r i t h m i c a p p r o x i m a t i o n o f o p t i m a l v a l u e bounds. For example, i f a sequence o f d u a l - f e a s i b l e p o i n t s I [ x k ( F ) , uk (F)] 1 i s g e n e r a t e d f o r

El (T)when

k

=

1, 2 ,

s o l u t i o n algorithm i n t h e process of s o l v i n g

... by

t h e given

F1(E),a s

is

c h a r a c t e r i s t i c a l l y t r u e o f p e n a l t y f u n c t i o n s ( F i a c c o and McCormick, 1968; see a l s o Theorem 6 . 2 . 1

i n t h i s book), then w e

c o u l d c o n c l u d e from ( 9 . 5 . 1 ) t h a t f o r e a c h k f * ( d 2 Uk(T)tiJ(E)+ L[Xk(E),

Uk(S),

o v e r t h e domain o f f * ( ~ ) p, r o v i d e d t h a t

01

F1(&) i s

convex i n x.

(The i n e q u a l i t y a p p l i e s even i f F i s r e p l a c e d by Fk, i . e . , even i f F changes f o r e a c h k . )

[x, (El, uk ( E ) 11 con-

Suppose

v e r g e s t o [ x (F), u ( F ) ] , where x ( F ) s o l v e s

Pl ( F )

with associated

d u a l m u l t i p l i e r u ( F ) , a s g e n e r a l l y happens f o r p e n a l t y f u n c t i o n d u a l - f e a s i b l e c o n v e r g e n t subsequences.

Then o f c o u r s e t h e

above bound a p p r o a c h e s f * ( ~ ) u ( E ) ~ ~ ~+ ( LE[ )x ( F ) , u ( T ) , 0 1 if

263

9.5 Nonlinear Dual Lower Bounds

t h e problem f u n c t i o n s a r e c o n t i n u o u s . s a t i s f i e s t h e Karush-Kuhn-Tucker

[ x ( F ) , u(F)1

If

c o n d i t i o n s , t h e n t h e bound

i s s a t i s f i e d e x a c t l y a t F. The d u a l problem and a r e s u l t i n g lower bound can a l s o be g i v e n f o r t h e g e n e r a l p a r a m e t r i c program P 3 ( ~ )t h a t i s convex i n x.

The problem and i t s d u a l a r e

minx f ( x ,

s.t. g ( x ,

E)

2 0, h(x,

E)

P3 (€1

= 0

E)

and max

(X?U?W)

L ( x , u, w ,

E)

s . t . V X L ( X , u, w ,

E)

= 0,

u 1. 0 , where now L ( x , u , w ,

E)

= f (x,

E)

-

u g(x,

E)

+ wTh ( x ,

D3(c)

As

E).

b e f o r e , t h e o p t i m a l v a l u e of P 3 ( ~ )is bounded below by any d u a l - f e a s i b l e v a l u e o f L ( x , u, w , f * ( E ) 1. ~

[ f i ( E ) ,

i.e.

€1,

~ ( € 1 ,G ( E ) ,

(2, G, G)

f o r any p o i n t

E),

satisfying the dual constraints.

d i f f e r e n c e now i s t h a t t h e d u a l - f e a s i b l e p o i n t e r a l l y depends on

E

gen-

and may n o t be computable, even o v e r a

s m a l l s u b s e t o f t h e domain o f noted i n Section 9.2, can be c a l c u l a t e d i f f L1

(2, 6, G)

The

E

of i n t e r e s t .

Nonetheless, a s

p i e c e w i s e - l i n e a r upper and lower bounds

*

(E)

i s convex.

The l i n e a r s u p p o r t s

and L 2 f o r P 3 ( ~ )were based s o l e l y on t h i s c o n v e x i t y

assumption.

A d u a l i t y i n t e r p r e t a t i o n c a n a g a i n be g i v e n ,

however, when P 3 ( € ) i s j o i n t l y convex.

Motivation f o r t h i s i s

t h e i d e a t h a t w e may b e a b l e t o c h a r a c t e r i z e any l i n e a r lower bound s u c h as g i v e n i n F i g . 9 . 2 . 1

a s t h e graph of t h e optimal

v a l u e a s s o c i a t e d w i t h some p a r a m e t r i c l i n e a r program. convexity of f

*

(E)

The

and i t s r e l a t i o n s h i p t o t h e Lagrangian

s u p p o r t t h i s i d e a and i n f a c t immediately y i e l d an e q u i v a l e n t

264

9 Optimal Value Bounds and Solution Vector Estimates

L a g r a n g i a n bound.

Suppose [ x ( E ) ,

Karush-Kuhn-Tucker

c o n d i t i o n s and f

u(E),

*

(6)

w(E)]satisfies the

is d i f f e r e n t i a b l e .

These c o n d i t i o n s a n d t h e f o l l o w i n g c o n s e q u e n c e s h o l d u n d e r t h e f * ( E ) = L [ x ( E ) , u ( E ) , ~ ( € 1 ,€ 1

a s s u m p t i o n s o f Theorem 3.2.2: a n d VEf

*

= V € L [ X ( E ) ,u ( E ) ,

(E)

it f o l l o w s t h a t f * ( E ) 2 f * ( F ) domain o f f

*

(E).

w(E),

+

€1.

Since f

V E f * ( T )( E

-

*

i s convex,

(E)

T),f o r

E,

F i n the

The g i v e n r e l a t i o n s h i p s i m p l y t h a t t h i s

) L[x(E), u(E), convexity condition i s equivalent to f * ( ~ = El

W(E),

(E

- T).

+

2 L [ x ( T ) , U(E), w m , FI

VEL[X(T),

U(F),

w ( T ) , TI

The g i v e n l o w e r bound o n f * ( E ) is p r e c i s e l y a n a l o g o u s

t o t h e l i n e a r lower bounds d e r i v e d e a r l i e r and s u g g e s t s t h e a p p r o p r i a t e l i n e a r program a n d d u a l i t y i n t e r p r e t a t i o n . C o n s i d e r t h e l i n e a r program d e r i v e d from t h e j o i n t l y convex program P3 ( E ) by l i n e a r i z i n g t h e o b j e c t i v e f u n c t i o n and b i n d i n g c o n s t r a i n t s a b o u t [ x ( F ) , F ] , t h u s y i e l d i n g ( w i t h nonbinding constraints deleted)

-

x(31

E l (E

-

F)

s . t . g [ x ( T ) , TI + V x g [ x ( E ) , F l [ x

-

x(S)I

(E

-

F ) 1. 0 ,

x ( F ) , TI [ x

-

x(F)I

-

F ) = 0.

min X f [ x ( F ) , F ]

+

VXf[x(F), F l [ x

+

VEf [ x ( F ) ,

-

x ( F ) , FI

X(F),

FI

(E

LP~(E)

S i n c e P 3 ( E ) i s j o i n t l y convex, i t f o l l o w s t h a t L P 3 ( ~ )i s a n u n d e r e s t i m a t i n g problem f o r P3

-* f

(E)

(E),

i.e.,

i s t h e optimal value of LP3(€).

-* f

(E) 5 f *

(€1,

The Wolfe d u a l of

where

265

9.6 Extensions

L P 3 ( ~ )i s g i v e n by

max

(x,u,w)

f +

- r)

V E R E

+

Vx9(x

-

-

+ wT[li +

VXii(X

-

F) +

s . t . vxF

- X) +

VXF(X

-

uT[q

hTV

7

X

t-

x)

+ VET(€ - 3 1 VEFi(E

=

3 1

T w VXE = 0 , u 1. 0 ,

where t h e s u p e r b a r d e n o t e s e v a l u a t i o n a t x =

x

-

D L P (~€ 1

X,

E

= F , and

~ ( 7 ) .T h i s r e d u c e s t o max

(u,w)

L

+ PEL(€ - -a

s.t. V,f;

= 0 , u 2 0.

Thus, w e deduce from t h e d u a l i t y r e l a t i o n s h i p t h a t f*(E) where

z*=

T * ( E ) 1. E.*+ V C E * ( E

-

F),

L [ x ( F ), u ( E ) , w ( E ) , F ]

.

This corresponds p r e c i s e l y

t o t h e bound d e r i v e d j u s t p r e v i o u s l y , r e s u l t i n g simply from t h e c o n v e x i t y of f

*

(E).

Summarizing, f o r t h e g e n e r a l j o i n t l y convex program P 3 ( ~ a) l i n e a r lower bound on f t h e l i n e a r i z a t i o n of P j

*

(E)

(E)

with optimal m u l t i p l i e r s

i s p r o v i d e d by t h e d u a l v a l u e of a t some (F, F ), where 51 s o l v e s P 3 (5)

(u, w).

EXTENSIONS

9.6.

A p a r t from d u a l i t y r e s u l t s and t h e e x p l o i t a t i o n of conv e x i t y , w e n o t e t h a t o t h e r problem s t r u c t u r e s may a l l o w t h e c a l c u l a t i o n of bounds.

F o r example, suppose t h e problem i s

s . t . x e R ( E ) , where R i s a convex p o i n t k t o - s e t map ( i . e . , zl, z2 i n t h e graph G = { ( E , x ) I E E E ,

P ( E ) : minx f ( x ,

x

E

0 5

E)

R ( E ) ) of R i m p l i e s t h a t a z l c1

+

(1

-

a ) z Z e G for every

5 1) a s it i s i n t h e j o i n t l y convex program, b u t f ( x ,

h a s no p a r t i c u l a r c o n v e x i t y s t r u c t u r e .

E)

W e can s t i l l c a l c u l a t e

266

9 Optimal Value Bounds and Solution Vector Estimates

a p a r a m e t r i c u p p e r bound on f when

E~

and

*

+

[ ~ ( a ) f]o r € ( a ) =

(1

a r e i n a convex s u b s e t of t h e domain of f

E~

and 0 5 a 5 1, once X ( E ~ E) R ( E ~ )and

*

-

a)E1,

(E)

c R ( E ~ )a r e a v a i l -

'~E(E,)

a b l e , s i n c e t h e c o n v e x i t y o f t h e map R i m p l i e s , a s o b s e r v e d i n S e c t i o n 9.3,

t h a t z ( a ) = ~ T I ( E ~+) (1

f o r 0 5 a 5 1.

I t follows t h a t f

*

-

~ ) Z ( E E~ R) [ ~ ( a ) l

[ ~ ( a ) 5] f [ Z ( a ) , € ( a ) ] f o r

a l l 0 5 a 5 1. Another c l a s s o f problems, one n o t r e q u i r i n g c o n v e x i t y i n x, f o r which b o t h upper and lower bounds on f

*

(E

can be c a l -

c u l a t e d , a r e problems o f t h e form minx f ( x ,

E)

s.t. x

E

where R does n o t depend on

R

If f(x,

E.

t h e n i t i s w e l l known t h a t f

*

(E)

E)

i s concave i n

i s concave.

E,

This allows f o r

t h e c a l c u l a t i o n o f p i e c e w i s e - l i n e a r upper and lower g l o b a l bounds, u s i n g l i n e a r s u p p o r t s t o p r o v i d e upper bounds and l i n e a r i n t e r p o l a t i o n t o p r o v i d e a lower bound on f

*

(E),

analo-

gous t o t h e bounding p r o c e d u r e t h a t was d i s c u s s e d a t t h e b e g i n n i n g o f t h i s s e c t i o n f o r t h e convex o p t i m a l v a l u e function. The t e c h n i q u e s d e s c r i b e d h e r e p r o v i d e an approach f o r o b t a i n i n g computable o p t i m a l v a l u e p a r a m e t r i c bounds and an associated parametric f e a s i b l e vector f o r l a r g e c l a s s e s of p a r a m e t r i c n o n l i n e a r programs.

A b a s i s f o r o b t a i n i n g upper

and lower bounds on t h e o p t i m a l v a l u e i s t h e c o n v e x i t y o r conc a v i t y of t h e o p t i m a l v a l u e f u n c t i o n o f t h e problem o f i n t e r e s t , o r o f an a u x i l i a r y problem t h a t u n d e r e s t i m a t e s or o v e r e s t i m a t e s t h e o p t i m a l v a l u e o f t h e g i v e n problem.

The d e r i v a t i o n o f an

a s s o c i a t e d f e a s i b l e p a r a m e t r i c v e c t o r f u n c t i o n i s an immediate consequence o f t h e c o n v e x i t y o f t h e p o i n t t o s e t map t h a t def i n e s t h e f e a s i b l e region R ( E ) a s a function of the parameter€.

261

9.6 Extensions

The bounding approach h a s d i r e c t c o n n e c t i o n s w i t h d u a l i t y t h e o r y , a s n o t e d , l e a d i n g t o f u r t h e r r e s u l t s f o r lower bounds. Namely, t h e c o n v e x i t y of t h e g i v e n problem i n t h e v a r i a b l e s becomes e x p l o i t a b l e , g i v i n g a d d i t i o n a l and sometimes s h a r p e r bounds f o r problems i n v o l v i n g more g e n e r a l r i g h t - h a n d - s i d e perturbations.

The a p p l i c a t i o n of t h e s e t e c h n i q u e s t o non-

convex programs t o o b t a i n g l o b a l p a r a m e t r i c lower bounds v i a convex u n d e r e s t i m a t i n g problems i s immediate and l e a d s t o t h e i d e a o f e x p l o i t i n g nonconvex s o l u t i o n methodologies based on convex u n d e r e s t i m a t i n g problems, e . g . , for calculating global solutions. developed.

branch and bound methods

T h i s approach remains t o be

Other d i r e c t i o n s of r e s e a r c h t h a t s h o u l d be c a r e -

f u l l y e x p l o r e d are t h e i d e n t i f i c a t i o n of a d d i t i o n a l c l a s s e s of problems t h a t have convex o r concave o p t i m a l v a l u e f u n c t i o n s , t h e f u r t h e r e x p l o i t a t i o n o f problem s t r u c t u r e , such a s s e p a r a b i l i t y , t o c o n s t r u c t r e l a t e d c o m p u t a t i o n a l l y t r a c t a b l e programs w i t h convex o r concave o p t i m a l v a l u e f u n c t i o n s , t h e u s e o f o t h e r convex and nonconvex d u a l i t y r e s u l t s Ce.g.,

Rockafellar, 1970,

1971, 1974) and t h e development o f bounds on t h e p a r a m e t r i c s o l u t i o n v e c t o r and t h e a s s o c i a t e d o p t i m a l m u l t i p l i e r s .

Re-

s u l t s b a s e d on weaker c o n d i t i o n s i m p l y i n g any e x p l o i t a b l e p r o p e r t y o f t h e o p t i m a l v a l u e f u n c t i o n , e.g., pseudo-convexity,

quasi- o r

would b e e x t r e m e l y u s e f u l t o accommodate

l a r g e r classes o f problems b u t remain t o be developed. The r e a d e r i n t e r e s t e d i n i n t e r p r e t a t i o n s a s s o c i a t e d w i t h Fig. 9.2.1

i s a l s o r e f e r r e d t o Lasdon (1970) and Bazaraa and

S h e t t y (1979).

268

9 Optimal Value Bounds and Solution Vector Estimates

BOUNDS ON A SOLUTION P O I N T

9.7.

W e t u r n b r i e f l y t o t h e problem o f c o n s t r u c t i n g computable

bounds on a s o l u t i o n p o i n t min X f ( x ,

s.t. g ( x ,

E)

o f t h e g e n e r a l problem

X(E) E)

2 0, h(x,

Suppose P 3 ( ~ )i s j o i n t l y convex, x ( E ~ )s o l v e s P 3 ( ~ 2 ) .

R e c a l l i n g t h e n o t a t i o n and r e s u l t s o f

*

0 5 a 2 1, where € ( a ) =

-

P3(E)

s o l v e s P 3 ( ~ 1 ) , and

i t was concluded t h a t x ( a )

S e c t i o n 9.3,

(1

x ( E ~ )

= 0.

E)

+

aE2

(1

-

E

R [ E ( ~ ) f]o r any

*

a)E1 and x ( a ) =

+

a ) x ( ~ ~ )F .u r t h e r m o r e , w e found t h a t m a x [ ( l l ( a ) , ! L 2 ( a ) l 5 Thus, g i v e n any v a l u e o f a

f * [ ~ ( a ) 5] f [ x * ( a ) , € ( a ) ] .

E

*

[O, 11,

w e a r e a b l e t o c a l c u l a t e a f e a s i b l e p o i n t x ( a ) o f P 3 [ € ( a )1 and upper and lower bounds on t h e o p t i m a l v a l u e f m[ E ( a )I o f P 3 [ ~ ( a ) ] once , s o l u t i o n s o f P 3 [ € ( 0 ) ] and P 3 [ c ( l ) l a r e a v a i l .We a d d r e s s t h e problem o f e s t i m a t i n g t h e d i s t a n c e o f

able.

x* ( a ) t o F* ( a ) , where F* ( a ) is a s o l u t i o n o f P 3 [ E ( a )1 f o r some i n ( 0 , 1).

fixed value of a =

The f o l l o w i n g a p p r o a c h h a s been s u g g e s t e d by F i a c c o and Kyparisis (Fiacco, 1981).

A s s u m e t h e problem f u n c t i o n s a r e

t w i c e continuously d i f f e r e n t i a b l e .

Let

z

*

=

-* -

x ( a ) and

Then t h e c o n v e x i t y o f f i m p l i e s t h a t f o r any v e c t o r z

E

= E(Z). En,

the

first-order inequality

AT holds.

F f(z*

+ z, F)

*

* , -E )

f(y

2 Vxf(y*, F ) z

A second-order i n e q u a l i t y a l s o holds.

conditions hold a t z

Karush-Kuhn-Tucker and w

-

, Taylor's

L* F L ( Z * = f

+

2 ,

u

+

with multipliers u

*

* , w * , -E ) T 2

*

Assuming t h e

series y i e l d s

(z*, 8 ) + ( 1 / 2 ) 2 VxL(rI, u

where q = z

*

(9.7.1)

Bz f o r some B

E

*

* ,w ,az, *

[0, 1 1 .

If z

+ z

is a feasible

269

9.7 Bounds on a Solution Point

p o i n t of P 3 ( F ) , t h e n f ( z

*

+

z,

E)

2. L

*

a n d h e n c e w e may con-

clude t h a t

AT

E f(z*

-

+ z, E )

f ( z * , 5)

*

T 2

L ( 1 / 2 ) 2 VxL(rl, u , w

llZll

W e are s e e k i n g a bound on

-

x*(E)

that i f z = Z

observed, and (9.7.1)

z*,

* , -€12. IIx* ( E l

t h e n z*

and (9.7.2)

(9.7.2)

+Z

- z*(a)11.

Note

= x * ( E ) E R ( F ) , as

apply.

F u r t h e r , w e can

c a l c u l a t e a n o v e r e s t i m a t i o n of t h e l e f t - h a n d s i d e o f t h e s e i n e q u a l i t i e s f r o m o u r p r e v i o u s o p t i m a l v a l u e bounds c a l c u l a tions, obtaining i n particular t h a t B(Z) E f [ x * ( a ) ,

-

€1

-

max[tl(iji), f i 2 ( i j i ) 1

1. f [ X * ( E : ) , 51 f(z

*

+ z,

5)

-

-

f(z

W e now h a v e from (9.7.11,

Vxf(z*,

*

, 5). (9.7.21,

a n d (9.7.3)

that

5 B(a),

(9.7.4)

* * -T 2 ( 1 / 2 ) 2 V x L ( q , u , w , F)? 5 B ( a ) .

(9.7.5)

I t r e m a i n s t o u n d e r e s t i m a t e t h e l e f t - h a n d sides of

(9.7.4)

and (9.7.5) p1

(9.7.3)

f [ F * ( E i ) , FI

i n terms o f 11211.

A n a t u r a l approach is t o seek

> 0 and p2 > 0 such t h a t

Vxf(z

* , E)E

1.

lJ1ll~ll

(9.7.6)

and (9.7.7)

270

9 Optimal Value Bounds and Solution Vector Estimates

Having c a l c u l a t e d p1 a n d p z r w e c o u l d t h e n c o n c l u d e from (9.7.4)

(9.7.7)

a n d ( 9 . 7 . 6 ) t h a t ))1)1 5 B(Z)/ul 2 t h a t 115711 5 B ( E ) / v 2 .

However, n o t i n g t h a t ( 9 . 7 . 6 )

i s n o t r e p r e s e n t e d i n (9.7.7)

v2

0 so

gram V'L

combining ( 9 . 7 . 1 )

= 0 and

(9.7.7)

and (9.7.2)

and

d o e s n o t a c c o u n t f o r second-

o r d e r e f f e c t s and t h a t t h e f i r s t - o r d e r (9.7.6)

and from ( 9 . 7 . 5 )

information yielding (e.g.,

for a linear pro

y i e l d s no bound)

,

suggests

t o y i e l d a bound i n c o r p o r a t i n g

b o t h f i r s t - a n d s e c o n d - o r d e r i n f o r m a t i o n a t t h e o u t s e t and a t h e r e f o r e h o p e f u l l y more c o m p r e h e n s i v e a n d s h a r p e r bound. n o t e f i r s t t h a t simply adding (9.7.1)

and ( 9 . 7 . 2 )

t o a bound o f t h e t y p e g i v e n i n ( 9 . 7 . 6 )

We

does n o t l e a d

o r (9.7.71,

because

of t h e l a c k o f homogeneity w i t h r e s p e c t t o z o f t h e t e r m s involving

However, a s i m p l e m o d i f i c a t i o n d o e s l e a d t o t h i s

2.

t y p e of bound.

N o t e t h a t t h e right-hand

+

n o n n e g a t i v e i f z*

z

E

RE),

t h e r e f o r e s q u a r e b o t h s i d e s of (9.7.2),

a

+

since z

*

(9.7.1)

s i d e of

is

(9.7.1)

s o l v e s P ~ ( E ) . W e may and a d d t h e r e s u l t t o

w e i g h t e d r e s p e c t i v e l y by a 1. 0 and B 2 0 , where

= 1, f o r a d d i t i o n a l f l e x i b i l i t y , o b t a i n i n g f o r a l l

s u c h t h a t z*

+

z

E

z

R(E),

As b e f o r e , t h i s e x p r e s s i o n a n d ( 9 . 7 . 3 )

imply t h a t

-

I f w e c a n f i n d any v ( a , 8 ) > 0 s u c h t h a t

(9.7.8)

271

9.7 Bounds on a Solution Point

t h e n we can conclude t h a t (9.7.9) The p a r a m e t e r s a a n d B h a v e b e e n i n t r o d u c e d f o r a d d i t i o n a l f l e x i b i l i t y and g e n e r a l i t y , i n c o r p o r a t i n g t h e bounds b a s e d on (9.7.6)

and ( 9 . 7 . 7 )

a s s p e c i a l cases o f ( 9 . 7 . 9 )

and a l l o w i n g

t h e p o s s i b i l i t y o f i m p r o v i n g on t h e f o r m e r bounds.

Ideally,

w e would l i k e t o f i x a a n d B s u c h t h a t t h e l a r g e s t v a l u e o f

B ) w i l l r e s u l t , a n d t h e t h e o r e t i c a l i m p l i c a t i o n s of t h i s

p(ct,

o p t i m a l s e l e c t i o n might be i n t e r e s t i n g t o e x p l o r e .

I n prac-

t i c e , a p o s s i b l e s t r a t e g y m i g h t be f i r s t t o check t h e bounds o b t a i n e d f r o m ( a , B ) = (1, O), and B = 0 t h e n p = p l ,

( 0 , 1). Obviously, i f a = 1

and i f a = 0 and B = 1, t h e n p = u2.

Suppose w e a l l o w a t m o s t o n e a t t e m p t t o improve t h e bound.

If

e i t h e r p l o r p 2 i s 0 , b u t n o t b o t h , w e c o u l d a c c e p t t h e non= p2 = 0 , w e

z e r o v a l u e f o r p t o c a l c u l a t e t h e bound.

I f pl

might t r y "unbiased" w e i g h t s , a = 8 = 1 / 2 .

Finally, i f

u1 > 0

a n d u2 > 0 , w e m i g h t s e l e c t a a n d B p r o p o r t i o n a t e t o p1 a n d p z , r e s p e c t i v e l y , e.g., p2/(pl

+

p2).

w e may t a k e a = p l / ( p l

+

p2)

and B =

The v a r i o u s p o s s i b i l i t i e s a n d t h e i r i m p l i c a t i o n s

w i l l be e x p l o r e d . Having s e l e c t e d v a l u e s f o r a a n d B, w e a r e l e d t o t h e s t u d y o f a p r o b l e m o f t h e form

o v e r a n a p p r o p r i a t e set c o n t a i n i n g ( z

*

* * , u , w , z).

The cal-

c u l a t i o n of p l e a d s t o an a n a l y s i s of v a r i o u s n o n l i n e a r

212

9 Optimal Value Bounds and Solution Vector Estimates

s.t.

-

iil

2

Or

i

V h.(x, E)z =

O r

j = 1,

Vxgi(x,

E)Z

X I

1lZ1l2

E

gi(xr F) = . . . I

01,

p,

= 1,

w i t h p o s s i b l y some a d d i t i o n a l r e s t r i c t i o n s .

As n o t e d , t h e

development o f bounds r e s u l t s u s i n g t h i s a p p r o a c h i s c u r r e n t l y being i n v e s t i g a t e d . Two i n t r i g u i n g f a c t s a s s o c i a t e d w i t h t h e P ( a , 8 ) bound are worth mentioning.

One i s a c o n n e c t i o n between t h e p o s i t i v i t y

o f t h e o b j e c t i v e f u n c t i o n o f P ( a , B ) a t ?I f o r

c1

> 0,

8 > 0,

w i t h ( a n e q u i v a l e n t form o f ) t h e s e c o n d - o r d e r s u f f i c i e n t con-

X b e a s t r i c t l o c a l minimum o f P 3 ( T ) (see comment

ditions that

f o l l o w i n g Lemma 3 . 2 . 1 ) .

The s e c o n d i s t h e s i m i l a r i t y of P(c1,

6 ) w i t h a q u a d r a t i c a u x i l i a r y p r o b l e m o f t h e form minz ~

~

f F (E )+x~( 1 ~ / 2 ) z~T v ; ~ ( x o , u o r wor

V h.(xo, E ) z = 0, X

I

j = 1,

E)Z

..., p

t h a t arises i n c o n n e c t i o n w i t h t h e r e c u r s i v e programming methods o f Wilson ( 1 9 6 3 ) , L e v i t i n a n d Polyak ( 1 9 6 6 1 , Robinson ( 1 9 7 2 ) , and o t h e r contemporary v e r s i o n s , f o r s o l v i n g P 3 ( T ) w i t h

( x o r u o r wo) g i v e n a n d k = 1, 2 ,

... .

These i n t e r e s t i n g c l o s e

correspondences w i t h important r e s u l t s m e r i t f u r t h e r i n v e s t i g a t i o n b u t w i l l n o t be pursued here.

273

9 . 8 Further Extensions and Applications

9.8.

FURTHER EXTENSIONS

AND A P P L I C A T I O N S

If

t h e g i v e n problem P 3 ( € ) d o e s n o t have a convex o p t i m a l

f u n c t i o n , t h e n w e c a n s t i l l g e n e r a t e upper o r lower bounds on f

*

i f w e have a t o u r d i s p o s a l a j o i n t l y convex u n d e r e s t i m a t i n g

o r o v e r e s t i m a t i n g problem of P 3 ( € ) , i . e . ,

a problem whose

o p t i m a l v a l u e must u n d e r e s t i m a t e o r o v e r e s t i m a t e f parameter values of i n t e r e s t .

*

for the

For s t a n d a r d ( n o n p a r a m e t r i c ) NLP

programs t h e r e e x i s t s a v a r i e t y o f t e c h n i q u e s f o r g e n e r a t i n g convex u n d e r e s t i m a t i n g problems i f t h e f u n c t i o n s o f t h e g i v e n nonconvex problem a r e s e p a r a b l e ( F a l k and Soland, 1 9 6 9 ) o r f a c t o r a b l e (McCormick, 1 9 7 6 ) , t h e l a t t e r c l a s s i n c l u d i n q a l l t h e f u n c t i o n s w e have e n c o u n t e r e d i n p r a c t i c e . F o r p a r a m e t r i c programs i t i s s i m p l y a m a t t e r o f a p p l y i n g t h e s e t e c h n i q u e s jointly o v e r b o t h t h e d e c i s i o n v a r i a b l e x and t h e parameter

E.

For a l l s u c h p r o c e d u r e s , t h e c a l c u l a t i o n o f

bounds o f t h e form 111 o r !La f o r t h e j o i n t l y convex undere s t i m a t i n g problem w i l l p r o v i d e p i e c e w i s e - l i n e a r lower bounds on t h e o p t i m a l v a l u e o f t h e g i v e n nonconvex problem.

This

t e c h n i q u e seems e s p e c i a l l y e x p l o i t a b l e by branch and bound methods t h a t a r e based on s o l v i n g a sequence o f convex undere s t i m a t i n g problems. The methodology f o r g e n e r a t i n g convex o v e r e s t i m a t i n g problems i s less d e v e l o p e d , though some r e s u l t s e x i s t .

In-

t e r i o r p e n a l t y ( b a r r i e r ) f u n c t i o n methods ( F i a c c o and McCormick, 1968) a r e based on d e v i s i n g an u n c o n s t r a i n e d o v e r e s t i m a t i n g problem t h a t i s convex i f t h e problem i s convex and l o c a l l y convex o t h e r w i s e .

Meyer (1970) developed a l g o r i t h m s f o r re-

v e r s e convex c o n s t r a i n t s b a s e d on c o n s t r u c t i n g a sequence o f

214

9 Optimal Value Bounds and Solution Vector Estimates

l i n e a r o v e r e s t i m a t i n g problems.

A v r i e l , Dembo, and Passy

(1975) have d e v e l o p e d an a l g o r i t h m f o r s o l v i n g a g e n e r a l i z e d g e o m e t r i c programming problem, u t i l i z i n g a sequence o f posynomial ( h e n c e , e q u i v a l e n t t o convex) o v e r e s t i m a t i n g problems i n c o n j u n c t i o n w i t h an " i n t e r i o r " c u t t i n g p l a n e method.

Ghaemi

(1980) h a s d e r i v e d f o r m u l a s f o r c a l c u l a t i n g a convex o v e r e s t i m a t i n g problem of a f a c t o r a b l e program.

I f a j o i n t l y con-

vex o v e r e s t i m a t i n g problem c a n be c o n s t r u c t e d , t h e n t h e c a l c u l a t i o n of a bound of t h e form i 3 f o r t h i s problem w i l l p r o v i d e a l i n e a r p a r a m e t r i c upper bound on t h e o p t i m a l v a l u e o f t h e o r i g i n a l problem o v e r t h e r a n g e of p a r a m e t e r v a l u e s c o n s i d e r e d . Analogous bounds can o b v i o u s l y be c o n s t r u c t e d from programs whose o p t i m a l v a l u e i s concave, and some p r e l i m i n a r y r e s u l t s a r e a l s o r e p o r t e d by Ghaemi ( 1 9 8 0 ) . The p r o c e d u r e f o r c a l c u l a t i n g p i e c e w i s e - l i n e a r upper and lower bounds f o r t h e convex o p t i m a l v a l u e f u n c t i o n was f i r s t implemented by F i a c c o and Ghaemi and r e p o r t e d by Ghaemi (1980) i n h i s d o c t o r a l d i s s e r t a t i o n .

Ghaemi added t h e bounds

calculation c a p a b i l i t y t o the penalty-function s e n s i t i v i t y a n a l y s i s computer program SENSUMT and used t h i s program s u c c e s s f u l l y t o o b t a i n bounds f o r s e v e r a l s m a l l convex and nonconvex problems.

Ghaemi a p p l i e d t h e program t o c a l c u l a t e

bounds on t h e o p t i m a l v a l u e o f a convex e q u i v a l e n t of a g e o m e t r i c programming model of a stream-water p o l l u t i o n abatement system r e l a t i v e t o t h e most i m p o r t a n t p a r a m e t e r , a s d e t e r m i n e d by t h e Fiacco-Ghaemi s e n s i t i v i t y s t u d y (1979b) (see S e c t i o n 8.5 o f t h i s b o o k ) .

F i a c c o and Ghaemi (1980) a l s o p r e -

p a r e d a d e t a i l e d u s e r s ' manual f o r SENSUMT t h a t i n c l u d e s t h e

275

9.8 Further Extensions and Applications

o p t i m a l v a l u e bounds c a l c u l a t i o n a s a u s e r o p t i o n , a s w e l l a s a comprehensive s e n s i t i v i t y a n a l y s i s c a p a b i l i t y . The i n d i c a t e d w a t e r p o l l u t i o n bounds c a l c u l a t i o n i n v o l v e s t h e p e r t u r b a t i o n of a s i n g l e r h s parameter, t h e allowable oxygen d e f i c i t l e v e l i n t h e f i n a l r e a c h o f t h e s t r e a m , t h a t proved unequivocably t o b e t h e s i n g l e most i n f l u e n t i a l param-

e t e r i n t h e p r i o r s e n s i t i v i t y s t u d y by F i a c c o and Ghaemi (197933).

S u b s e q u e n t l y , F i a c c o and K y p a r i s i s (1981a) u t i l i z e d

SENSUMT t o c a l c u l a t e bounds on t h e o p t i m a l v a l u e c o s t f u n c t i o n

when t h e 30 most i n f l u e n t i a l p a r a m e t e r s w e r e p e r t u r b e d simultaneously.

I n t h i s a p p l i c a t i o n , t h e optimal value function

( t h e a n n u a l w a s t e t r e a t m e n t c o s t ) i s n o t convex i n a f u l l neighborhood o f t h e b a s e v a l u e o f t h e p a r a m e t e r v e c t o r . e v e r , it was p o s s i b l e t o show t h a t t h e r e s t r i c t i o n o f f

*

How(E)

to

t h e s u b s e t of parameters involved i n t h e d e s i r e d p e r t u r b a t i o n

i s convex.

I n a c o n t i n u a t i o n o f t h i s s t u d y , F i a c c o and

K y p a r i s i s (1981b) implemented f o r t h e f i r s t t i m e t h e r e f i n e ment p r o c e d u r e d e s c r i b e d i n S e c t i o n 9.3,

o b t a i n i n g much s h a r p e r

o p t i m a l v a l u e bounds and a p i e c e w i s e - l i n e a r solution estimate, a s w e l l .

Also,

f e a s i b l e parametric

s o l u t i o n v e c t o r bounds were

c a l c u l a t e d , a g a i n an i n i t i a l i m p l e m e n t a t i o n , u s i n g some of t h e r e s u l t s o f S e c t i o n 9.7. I n a n o t h e r s t u d y , which i n v o l v e d t h e convex e q u i v a l e n t of a g e o m e t r i c programming model o f a power system e n e r g y model d e v e l o p e d by Ecker and Wiebking (1976) t o f i n d t h e t u r b i n e e x h a u s t a n n u l u s and c o n d e n s e r s y s t e m d e s i g n t h a t minimizes t o t a l a n n u a l f i x e d p l u s o p e r a t i n g c o s t , F i a c c o and Ghaemi (1981b) u s e d SENSUMT t o o b t a i n p a r a m e t e r d e r i v a t i v e s and bounds on t h e o p t i m a l v a l u e f u n c t i o n f o r a v a r i e t y o f s i n g l e

216

9 Optimal Value Bounds and Solution Vector Estimates

o b j e c t i v e f u n c t i o n and c o n s t r a i n t p a r a m e t e r changes.

A

novelty of t h i s a n a l y s i s i s t h e use of t h e concavity r e s u l t g i v e n i n t h e second p a r a g r a p h o f S e c t i o n 9 . 6 and an e x p l o i t a t i o n o f problem s t r u c t u r e t o c a l c u l a t e a nonlinear lower bound on t h e o p t i m a l v a l u e f u n c t i o n . The s e r i o u s i m p l e m e n t a t i o n o f t h e bounds p r o c e d u r e f o r s e p a r a b l e and f a c t o r a b l e nonconvex programs, implemented f o r some examples by Ghaemi ( 1 9 8 0 ) , remains t o be developed.

Chapter 10

Future Research and Applications

10.1.

RECAPITULATION AND OTHER

RESEARCH DIRECTIONS

C h a p t e r 2 p r e s e n t s a number o f b a s i c c o n t r i b u t i o n s t o t h e t h e o r y o f s e n s i t i v i t y and s t a b i l i t y a n a l y s i s f o r g e n e r a l c l a s s e s o f n o n l i n e a r programming problems.

Hopefully, i t

c a p t u r e s a f l a v o r o f t h e main t h r u s t and r a n g e of developments f o r t h e s t a t i c nonconvex d e t e r m i n i s t i c problem t h a t r e l a t e s t h e behavior of t h e optimal value o r s o l u t i o n set t o perturbat i o n s o f problem p a r a m e t e r s .

That b r i e f a c c o u n t n e c e s s a r i l y

o m i t s many i m p o r t a n t r e s u l t s t h a t weave t h e f a b r i c more t i g h t l y and t h a t e x t e n d t h e framework of t h e t h e o r y t o more general spaces.

Also, r e s u l t s t h a t s i g n i f i c a n t l y e x p l o i t

s p e c i a l problem s t r u c t u r e have been d e t a i l e d o n l y f o r param-

e t e r d e r i v a t i v e s under r a t h e r i d e a l second-order c o n d i t i o n s (Chapters 3

-

8 ) and f o r p a r a m e t r i c bounds f o r j o i n t l y convex

programs ( C h a p t e r 9 ) .

I n t h i s s e c t i o n w e i n d i c a t e important

r e s e a r c h d i r e c t i o n s t h a t have been touched upon v e r y l i g h t l y o r not a t a l l . Numerous i m p o r t a n t r e s u l t s r e l e v a n t t o c o n v e x i t y and s t a b i l i t y a r e known f o r convex programs and t h e i r d u a l s , o t h e r t h a n t h o s e n o t e d i n C h a p t e r 2 , under v a r i o u s smoothness and 211

278

10 Future Research and Applications

nonsmoothness a s s u m p t i o n s .

S e e , f o r example, R o c k a f e l l a r

(1970, 1 9 8 2 a , b ) G o l ' s t e i n

( 1 9 7 2 ) , o r Hogan ( 1 9 7 3 ~ ) . L i n e a r

and q u a d r a t i c p a r a m e t r i c programming c h a r a c t e r i z a t i o n s h a v e been r a t h e r thoroughly developed:

see, e . g . ,

the references

c i t e d a t t h e b e g i n n i n g o f C h a p t e r 2 , a s w e l l a s Bereanu ( 1 9 7 6 ) , M a r t i n (19751, and Robinson ( 1 9 7 3 a , 1975, 1 9 7 7 ) f o r c o n t i n u i t y and s t a b i l i t y c h a r a c t e r i z a t i o n s .

P a r a m e t r i c r a n g e and p o s t -

o p t i m a l i t y s e n s i t i v i t y a n a l y s i s t e c h n i q u e s f o r l i n e a r programming h a v e b e e n known a n d r o u t i n e l y implemented f o r s o m e t i m e ( D a n t z i g , 1963; B r a d l e y , Hax, and M a g n a n t i , 1 9 7 7 ) .

A

recent

novel development a p p e a r s t o be a " s e n s i t i v i t y t o l e r a n c e " a p p r o a c h by Wendell ( 1 9 8 1 a , b ) t h a t a l l o w s f o r t h e s i m u l t a n e o u s and i n d p e n d e n t p e r t u r b a t i o n o f s e l e c t e d p a r a m e t e r s a n d t h e comp u t a t i o n o f a maximum t o l e r a n c e r a n g e w i t h i n which p a r a m e t r i c v a r i a t i o n does n o t a l t e r a given o p t i m al b a s i s .

Parametric

g e o m e t r i c programs h a v e a s t r o n g s t r u c t u r e t h a t c a n r e a d i l y b e e x p l o i t e d f o r s e n s i t i v i t y a n d g l o b a l bound i n f o r m a t i o n , a s n o t e d by D u f f i n e t a l .

(19671, B e i g h t l e r a n d P h i l l i p s (19761,

Dembo ( 1 9 8 2 ) , and i n some o f o u r r e c e n t work ( e . g . , applications discussed i n Section 9.8).

see t h e

The numerous computa-

t i o n a l i m p l e m e n t a t i o n s by D i n k e l a n d Kochenberger (1977, 1 9 7 8 ) a n d D i n k e l , K o c h e n b e r g e r , and Wong (1978, 1 9 8 2 ) m e n t i o n e d i n S e c t i o n 2.5,

using an incremental predictor-corrector

approach

t o e x t r a p o l a t e s e n s i t i v i t y i n f o r m a t i o n , are r e l e v a n t i n t h i s context. I n s e e k i n g t o e x p l o i t s t r u c t u r e i n d e v i s i n g computable methods f o r c a l c u l a t i n g m e a s u r e s o f s t a b i l i t y , it i s clear t h a t q u a n t i t i e s s u c h as bounds t h a t c a n be d e r i v e d i n t e r m s o f f u n c t i o n s o f a s i n g l e v a r i a b l e are o f s p e c i a l i n t e r e s t .

This

i m m e d i a t e l y b r i n g s t o mind s e p a r a b l e p r o g r a m s , e x t e n s i v e l y

219

10. I Recapitulation

e x p l o i t e d by F a l k a n d S o l a n d ( 1 9 6 9 ) a n d s u b s e q u e n t l y by many o t h e r s t o d e v i s e g l o b a l s o l u t i o n s t r a t e g i e s b a s e d o n branchand-bound and convex u n d e r e s t i m a t i o n .

There h a s been sur-

p r i s i n g l y l i t t l e e x p l o i t a t i o n of t h i s class o f problems i n calculating s e n s i t i v i t y o r s t a b i l i t y information, except f o r t h e r e l a t e d p r e l i m i n a r y s t u d y of G e o f f r i o n ( 1 9 7 7 ) , t h e r e c e n t u s e o f error bound i n f o r m a t i o n by Meyer ( 1 9 7 9 , 1 9 8 0 ) a n d Thakur (1978, 1 9 8 0 , 1 9 8 1 ) f o r convex p r o g r a m s , as b r i e f l y n o t e d i n S e c t i o n 2.5.

The e x t e n s i o n by Benson (1980) o f a

l a r g e c l a s s o f b r a n c h a n d bound methods t o p a r a m e t r i c nonconvex s e p a r a b l e programming w i t h r h s p e r t u r b a t i o n s i s a n i n t e r e s t i n g development. I t s h o u l d be e m p h a s i z e d t h a t i m p o r t a n t g e n e r a l c o n t e m p o r a r y

treatments of parametric s t a b i l i t y r e s u l t s obtained recently, e.g.,

t h e many i n c i s i v e r e s u l t s o b t a i n e d i n t h e s t u d y of gen-

e r a l i z e d e q u a t i o n s by Robinson ( 1 9 7 3

-

1 9 8 0 ) and o t h e r s , and

deep d i f f e r e n t i a l s t a b i l i t y r e s u l t s of R o c k a f e l l a r (1982a,b) have been mentioned o n l y i n p a s s i n g i n Chapter 2.

These re-

s u l t s are f a r - r e a c h i n g a n d w i l l u n d o u b t e d l y i n s p i r e and i n f l u e n c e many f u t u r e d e v e l o p m e n t s . W e h a v e n o t i n c l u d e d any r e s u l t s t a i l o r e d e s p e c i a l l y t o

o p t i m a l c o n t r o l t h e o r y , e x c e p t t o m e n t i o n t h e p a p e r s o f Maurer ( 1 9 7 7 a , b ) i n S e c t i o n 2.3. infinite-dimensional

The r e s u l t s g i v e n i n C h a p t e r 2 f o r

s p a c e s a r e g e n e r a l l y a p p l i c a b l e , of

course.

Some r e c e n t r e s u l t s h a v e b e e n r e p o r t e d by G o l l a n

(1979).

W e h a v e a l s o n o t a d d r e s s e d zero-one o r i n t e g e r pro-

gramming s e n s i t i v i t y a n a l y s i s , t h o u g h many r e s u l t s have been developed, e . g. ,

by Roodman (1972) , S h a p i r o ( 1 9 7 7 ) , M a r s t e n

and Morin ( 1 9 7 7 ) , H o l m a n d K l e i n (1978, 19821, Wolsey (19811, a n d Graves a n d S h a p i r o ( 1 9 8 2 ) .

280

10 Future Research and Applications

I t i s r e l e v a n t t o mention t h e " d e c i s i o n v a r i a b l e s e n s i t i v -

i t y a n a l y s i s " t e c h n i q u e s d e v e l o p e d by McKeown ( 1 9 8 0 ) .

The

motivation i s t o determine t h e s e n s i t i v i t y o f t h e optimal v a l u e t o changes i n t h e components o f a s o l u t i o n v e c t o r . leads t o a "curvature analysis,

'I

i.e.,

This

an a n a l y s i s o f t h e

e i g e n v a l u e s and e i g e n v e c t o r s o f t h e Hessian o f t h e o b j e c t i v e f u n c t i o n , i n t h e u n c o n s t r a i n e d problem.

Up t o s e c o n d - o r d e r

i n f o r m a t i o n , t h e o p t i m a l v a l u e f u n c t i o n i s l o c a l l y most s e n s i t i v e t o a change i n t h e d i r e c t i o n o f t h e e i g e n v e c t o r a s s o c i a t e d w i t h an e i g e n v a l u e whose a b s o l u t e v a l u e i s l a r g e r t h a n t h a t o f any o t h e r e i g e n v a l u e , and l e a s t s e n s i t i v e t o a change i n t h e d i r e c t i o n o f an e i g e n v e c t o r a s s o c i a t e d w i t h an e i g e n value with l e a s t absolute value.

For t h e c o n s t r a i n e d problem,

t h e r e l e v a n t Hessian i s t h e r e s t r i c t i o n of t h e H e s s i a n o f t h e Lagrangian t o t h e c o n s t r a i n t t a n g e n t s u b s p a c e a t a s o l u t i o n . McKeown h a s d e v e l o p e d d e t a i l e d n u m e r i c a l p r o c e d u r e s t h a t i m plement t h i s a n a l y s i s , i n a computer program whose s u c c e s s f u l use h a s been r e p o r t e d i n p r a c t i c a l a p p l i c a t i o n s ( e . g . ,

Archetti,

B a l l a b i o , and V e r c e l l i s , 1 9 8 0 ) . W e s h o u l d a l s o mention t h a t an e n t i r e body o f t h e o r y ,

somewhat more g e n e r a l b u t t o a g r e a t e x t e n t a n a l o g o u s t o t h e t h e o r y i n v o l v i n g p a r a m e t r i c v a r i a t i o n , h a s d e v e l o p e d more o r

less i n p a r a l l e l w i t h t h e p a r a m e t r i c t h e o r y .

This involves

t h e s t u d y of t h e e f f e c t s o f g e n e r a l p e r t u r b a t i o n s o f t h e prob-

l e m f u n c t i o n s on s o l u t i o n b e h a v i o r . For example, c o n s i d e r Pk: k k k k minx f ( x ) s . t . g ( x ) 2 0 , h ( x ) = 0 , where t h e f u n c t i o n s f , g k l and hk converge i n some s p e c i f i e d s e n s e t o f , g , h , respectively, as k

+

=.

Questions concerning t h e r e l a t i o n s h i p

281

10.2 Future Research Directions and Applications

o f s o l u t i o n s o f Pk t o s o l u t i o n s o f P:

minx f ( x ) s.t. g ( x ) 2

0 , h ( x ) = 0 , are of i n t e r e s t and o b v i o u s l y r e l a t e t o s e n s i t i v -

i t y and s t a b i l i t y q u e s t i o n s .

Noteworthy background r e a d i n g s

i n t h i s a r e a are t h e p a p e r s by Wijsman (1966) a n d Mosco ( 1 9 6 9 ) . The p a p e r s by D a n t z i g e t aZ. quite relevant.

(1967) a n d F i a c c o (1974) are a l s o

Numerous r e f e r e n c e s c o u l d be g i v e n h e r e , i n

a d d i t i o n t o those already provided f o r t h e parametric perturb a t i o n s , which a r e c e r t a i n l y r e l e v a n t ( n o t i n g t h a t t h e problem P ( E ) may be a n a l y s e d a t E = Z by c o n s i d e r i n g p r o b l e m s o f t h e k f o r m Pk, where f ( x ) = f ( x , E ~ ) ,e t c . , w h e r e E~ * E a s k * -1. The i n t e r e s t e d r e a d e r i s a l s o r e f e r r e d t o t h e r e c e n t work o f

Wets ( 1 9 7 7 ) a n d S a l i n e t t i a n d W e t s (1979) t h a t g i v e s a number of i n t e r e s t i n g r e s u l t s i n v o l v i n g s e q u e n c e s of convex sets and t h e i r a p p l i c a t i o n t o convex s t o c h a s t i c p r o g r a m i n g a s w e l l as many r e f e r e n c e s t o o t h e r work i n t h i s area.

This application

r e m i n d s u s t h a t t h e g e n e r a l area of s t o c h a s t i c programming h a s n o t b e e n a d d r e s s e d i n t h i s book e i t h e r , a l t h o u g h t h e i n e v i t a b l e presence o f u n c e r t a i n t y , e.g.,

p a r a m e t e r s t h a t are random

v a r i a b l e s , would o b v i o u s l y s u g g e s t t h a t p e r t u r b a t i o n a n a l y s i s r e s u l t s c h a r a c t e r i z i n g s o l u t i o n s e n s i t i v i t y o r s t a b i l i t y would c l e a r l y be a p p l i c a b l e .

E x p l i c i t c o n n e c t i o n s h a v e a l r e a d y been

made, as s u g g e s t e d by S a l i n e t t i a n d W e t s (19791, Bereanu (1976) and o t h e r s . 10.2.

FUTURE RESEARCH DIRECTIONS

AND APPLICATIONS

W e e x p e c t t o see a n e v e r - i n c r e a s i n g e m p h a s i s on computa-

b i l i t y a n d p r a c t i c a l a p p l i c a b i l i t y i n t h e f u t u r e development of s e n s i t i v i t y and s t a b i l i t y a n a l y s i s methodology i n mathe-

matical p r o g r a m i n g .

I t i s i n t h e s p i r i t of o u r t i m e s a n d i s

282

10 Future Research and Applications

a l s o p a r t o f a n a t u r a l m a t u r a t i o n p r o c e s s o f an a p p l i e d science.

As p r a c t i t i o n e r s become more aware o f what a d d i t i o n a l

v a l u a b l e s o l u t i o n i n f o r m a t i o n can r e a d i l y be g e n e r a t e d by a g i v e n s o l u t i o n a l g o r i t h m t o r e l a t e s o l u t i o n changes t o d a t a changes, t h e y w i l l want t h i s i n f o r m a t i o n .

W e i n d i c a t e some

r e s e a r c h d i r e c t i o n s and a r e a s o f a p p l i c a t i o n s t h a t have cont r i b u t e d o r t h a t w e e x p e c t w i l l c o n t r i b u t e t o t h i s development. The i n d i c a t e d r e s u l t s i n s e p a r a b l e programming mentioned i n t h e l a s t s e c t i o n a l s o s u g g e s t an approach, b r i e f l y ment i o n e d i n S e c t i o n s 2 . 5 and 9 . 8 ,

f o r generating s t a b i l i t y

i n f o r m a t i o n f o r nonconvex s e p a r a b l e o r f a c t o r a b l e programs. E r r o r bounds a r e f r e q u e n t l y o b t a i n e d by g e n e r a t i n g s i m p l e r f u n c t i o n s t h a t bound t h e g i v e n problem f u n c t i o n s , e . g . ,

convex

e n v e l o p e s of t h e c o n s t r a i n t f u n c t i o n s , a s i n t h e s e p a r a b l e nonconvex programming branch-and-bound

approach o f F a l k and

Soland ( 1 9 6 9 ) o r convex u n d e r e s t i m a t i n g o r concave o v e r e s t i mating f u n c t i o n s , a s i n t h e nonconvex f a c t o r a b l e programming approach o f McCormick ( 1 9 7 6 ) .

But once a p r o c e d u r e f o r gener-

a t i n g s i m p l e r "bounding problems" i s a t hand, i t can g e n e r a l l y be a p p l i e d t o a p e r t u r b a t i o n o f t h e o r i g i n a l problem t o o b t a i n "simple" (e.g.,

convex) bounding problems.

I f t h e perturba-

t i o n a n a l y s i s o f t h e s i m p l e r a s s o c i a t e d problems i s t r a c t a b l e , t h e n once t h e r e l a t i o n s h i p between t h e o p t i m a l s o l u t i o n o f t h e p e r t u r b e d bounding problems and t h e o r i g i n a l p e r t u r b e d problem

i s u n d e r s t o o d , w e have a p o s s i b l e p r o c e d u r e f o r g e n e r a t i n g bounds on t h e p e r t u r b e d s o l u t i o n o f t h e o r i g i n a l problem.

As n o t e d i n C h a p t e r 9 , t h e bounds approach e x p l o i t i n g t h e p r o p e r t i e s o f t h e j o i n t l y convex program i s p a r t i c u l a r l y w e l l s u i t e d t o t h e g e n e r a t i o n o f p a r a m e t r i c bounds f o r p a r a m e t r i c nonconvex s e p a r a b l e programs.

Once a j o i n t l y convex s e p a r a b l e

283

10.2 Future Research Directions and Applications

u n d e r e s t i m a t i n g p r o b l e m i s a t hand, t h e n w e c a n c a l c u l a t e p a r a m e t r i c o p t i m a l v a l u e l o w e r bounds f o r t h i s problem.

Ob-

v i o u s l y , t h e s e w i l l c o n s t i t u t e lower bounds f o r t h e o p t i m a l v a l u e f u n c t i o n o f t h e g i v e n nonconvex program. d e v e l o p e d branch-and-bound

The w e l l

methodology f o r c a l c u l a t i n g t h e

u n d e r e s t i m a t i n g p r o b l e m c a n o b v i o u s l y b e exploited.

Prelimi-

n a r y c a l c u l a t i o n s u s i n g t h i s i d e a h a v e b e e n made by Ghaemi (1980).

T h i s bounds c a l c u l a t i o n c o u l d r e a d i l y be i n t e r f a c e d

w i t h t h e a p p r o a c h p r o p o s e d by Benson (1980) f o r s i m u l t a n e o u s l y c a l c u l a t i n g t h e s o l u t i o n s of a g i v e n problem c o r r e s p o n d i n g t o s e v e r a l s p e c i f i e d resource ( r h s ) l e v e l s i n one p as s o f a b r a n c h - and-bound a l g o r i t h m

.

Bounds r e s u l t s i n t e r m s o f o r i g i n a l p r o b l e m d a t a , s u c h as t h o s e i n d i c a t e d i n C h a p t e r 2 by D a n i e l (1973) f o r q u a d r a t i c programming a n d , a t a h i g h e r l e v e l o f g e n e r a l i t y , by Robinson ( 1 9 7 6 a ) , are o f c o n s i d e r a b l e i n t e r e s t b u t remain t o b e comput a t i o n a l l y implemented.

A study of t h e connection with sensi-

t i v i t y a n d s t a b i l i t y t h e o r y of p r o c e d u r e s f o r g e n e r a t i n g s o l u t i o n error bounds, e . g . ,

t e c h n i q u e s u t i l i z i n g t h e u s e of

i n t e r v a l a r i t h m e t i c c o n c e p t s s u c h as t h o s e p r o p o s e d by Robinson (1973b) , Mancini and McCormick (1979) , and McCormick (1980) , s h o u l d be a s u b j e c t of f r u i t f u l r e s e a r c h . C o m p u t a t i o n a l t e c h n i q u e s f o r o p t i m a l v a l u e and KarushKuhn-Tucker t r i p l e s e n s i t i v i t y a n d s t a b i l i t y bounds m i g h t be e x p e c t e d t o d e v e l o p from t h e i n c i s i v e r e s u l t s d e v e l o p e d i n t h e f o l l o w i n g areas:

( i ) g e n e r a l i z e d e q u a t i o n s (Robinson, 1 9 7 6 a , b ,

1979, 1 9 8 0 a ) ; ( i i ) d i f f e r e n t i a l s t a b i l i t y a n d s u b d i f f e r e n t i a l a n a l y s i s o f t h e o p t i m a l v a l u e f u n c t i o n (Gollan, 1981a,b; R o c k a f e l l a r , 1 9 8 2 a , b ) ; (iii) d e f o r m a t i o n , c o n t i n u a t i o n a n d p a r a m e t r i c imbedding (Mine, Fukushima, a n d Ryang, 1 9 7 7 , 1978;

284

10 Future Research and Applications

K o j i m a and Hirabayashi, 1981); ( i v ) higher-order

( t h a n two)

o p t i m a l i t y c o n d i t i o n s a p p l i e d t o s t a b i l i t y a n a l y s i s and para-

metric programming ( G o l l a n , 1 9 8 1 ~ ) ;( v ) g e n e r i c c o n d i t i o n s f o r o p t i m a l i t y a n d s t a b i l i t y ( S p i n g a r n and R o c k a f e l l a r , 1 9 7 9 ) ; ( v i ) c o n n e c t i o n s between nonconvex d u a l i t y r e s u l t s a n d s t a b i l i t y ( B a l d e r , 1977; D o l e c k i a n d Kurcyusz, 1 9 7 8 ) ; ( v i i ) d e t e r m i n a t i o n of r e g i o n s o f s t a b i l i t y (Zlobec, Gardner, and Ben-Israel,

1982);

( v i i i ) r e c e n t a d v a n c e s i n p a r a m e t r i c l i n e a r a n d n o n l i n e a r programming ( G a l , 1979; NoiiEka e t a Z . , 1974; Lommatzsch, 1979; Bank e t a l . ,

1982; Brosowski, 1 9 8 2 ) ; a n d ( i x ) t h e a p p l i c a t i o n

of c o n t e m p o r a r y t e c h n i q u e s i n nonsmooth o p t i m i z a t i o n methodo l o g y t o s e n s i t i v i t y , s t a b i l i t y , a n d p a r a m e t r i c programming methodology

(Gauvin, 1 9 7 8 ) .

Some a p p l i c a t i o n s o f s e n s i t i v i t y and s t a b i l i t y a n a l y s i s i n m a t h e m a t i c a l programming a r e r a t h e r o b v i o u s , e . g . ,

estimation

of s o l u t i o n s o f p e r t u r b e d p r o b l e m s , g i v e n a s o l u t i o n o f a p r o b l e m w i t h g i v e n p a r a m e t e r v a l u e s , a n d d e t e r m i n a t i o n of parameters t o which t h e o p t i m a l v a l u e or s o l u t i o n p o i n t i s most s e n s i t i v e .

Numerous a p p l i c a t i o n s h a v e b e e n i n d i c a t e d

t h r o u g h o u t t h i s book. here:

W e c o l l e c t a number o f t h e s e a n d o t h e r s

( i )o p t i m a l i t y c o n d i t i o n s a n d a l g o r i t h m d e r i v a t i o n a n d

a c c e l e r a t i o n ( F i a c c o a n d McCormick, 1968; F i a c c o , 1 9 7 6 ) ; (ii) c o n v e r g e n c e o f a l g o r i t h m s (Meyer, 1 9 7 0 ) ; (iii)rate o f conv e r g e n c e o f a l g o r i t h m s (Robinson, 1 9 7 4 ) ; ( i v ) d e c o m p o s i t i o n ( G e o f f r i o n , 1970; Lasdon, 1970, Gauvin, 1 9 7 8 ) ; ( v ) i m p l i c i t l y defined-function

m i n i m i z a t i o n (Hogan, 1973a; d e S i l v a , 1 9 7 8 ) ;

( v i ) model v a l i d a t i o n a n d c o s t b e n e f i t a n a l y s i s ( A r c h e t t i e t aZ., 1980; F i a c c o a n d Ghaemi, 197933, 1 9 8 2 ) ; a n d homotopy c o n t i n u a t i o n methods

( R h e i n b o l d t , 1969; Garcia a n d Gould, 1978; Garcia

and Z a n g w i l l , 1 9 7 9 ; A l l g o w e r a n d Georg, 1 9 8 0 ) .

Conversely,

285

10.3 Conclusions

a l l t h e areas m e n t i o n e d w i l l u n d o u b t e d l y c o n t r i b u t e t o t h e development o f m a t h e m a t i c a l programming s e n s i t i v i t y a n d s t a b i l i t y methodology. 10.3.

CONCLUSIONS

The f u t u r e s h o u l d see a c o n c e r t e d e f f o r t t o w a r d s t h e f u r t h e r u n i f i c a t i o n of t h e powerful c o l l e c t i o n o f s e n s i t i v i t y a n d s t a b i l i t y r e s u l t s t h a t are s c a t t e r e d t h r o u g h o u t t h e l i t e r ature.

U n i f i c a t i o n i n e v i t a b l y f o l l o w s t h e development o f a

c r i t i c a l mass of r e s u l t s t h a t are i n c i s i v e enough a n d s u f f i c i e n t l y c o h e s i v e t o p r o v i d e a s i g n i f i c a n t body o f t h e o r y and a methodology.

The m a t u r a t i o n o f t h e s t a t e of t h e a r t i s s i g -

n a l l e d by t h e emergence o f books t h a t u l t i m a t e l y p r o v i d e t h e u n i f i c a t i o n of t h e s c i e n c e .

Only a few books h a v e b e e n w r i t t e n

i n t h i s g e n e r a l s u b j e c t a r e a i n "continuous" mathematical programming.

The f i r s t , by D i n k e l b a c h (19691, t r e a t s p a r a -

metric s e n s i t i v i t y a n a l y s i s i n l i n e a r p r o g r a m i n g .

Books i n

p a r a m e t r i c l i n e a r programming h a v e a l s o b e e n w r i t t e n , by Gal (1979) ; NoiiEka e t aZ.

( 1 9 7 4 ) ; Lommatzsch ( e d i t o r , 1 9 7 9 ) :

Kausmann, Lommatzsch, a n d NoiiEka ( 1 9 7 6 ) ; a n d Lorenzen ( 1 9 7 4 ) . Books i n p a r a m e t r i c n o n l i n e a r programming h a v e o n l y r e c e n t l y b e e n c o m p l e t e d , by Bank e t aZ.

( 1 9 8 2 ) a n d Brosowski (19821, t h e

f o r m e r e m p h a s i z i n g l i n e a r a n d q u a d r a t i c p a r a m e t r i c programming, the latter semi-infinite

( l i n e a r ) p a r a m e t r i c programming.

This

book is a p p a r e n t l y t h e f i r s t d e v o t e d e n t i r e l y t o g e n e r a l cont i n u o u s m a t h e m a t i c a l programming s e n s i t i v i t y a n d s t a b i l i t y analysis results. Of c o u r s e , s e n s i t i v i t y a n d s t a b i l i t y r e s u l t s are d i r e c t l y

a p p l i c a b l e a n d i n d e e d are t h e b u i l d i n g b l o c k s f o r c o n s t r u c t i n g and c h a r a c t e r i z i n g p a r a m e t r i c s o l u t i o n s .

U n i f i c a t i o n of

286

10 Future Research and Applications

s e n s i t i v i t y and s t a b i l i t y methodology w i l l a c c e l e r a t e t h e u n d e r s t a n d i n g o f b a s i c t h e o r y , s t i m u l a t e a l g o r i t h m s and s o f t w a r e developments, and h a s t e n w i d e s p r e a d , r o u t i n e implementations i n a l l a r e a s of a p p l i c a t i o n . S e n s i t i v i t y and s t a b i l i t y a n a l y s i s r e s u l t s are r e a d y f o r e x t e n s i v e computational implementation.

Experimental r e s u l t s

have u n e q u i v o c a l l y d e m o n s t r a t e d t h e p r a c t i c a l a p p l i c a b i l i t y o f v a r i o u s c o m p u t a t i o n a l schemes t h a t can g e n e r a t e a w e a l t h o f i n f o r m a t i o n t h a t s h o u l d be e x t r e m e l y v a l u a b l e t o u s e r s .

As

n o t e d i n t h i s book, e x t e n s i v e c o m p u t a t i o n s have been s u c c e s s f u l l y performed w i t h t h e SENSUMT computer program t h a t i n t e r f a c e s subprograms c a l c u l a t i n g s e n s i t i v i t y i n f o r m a t i o n w i t h t h e

w e l l known SUMT program.

The p r o c e d u r e u t i l i z e s i n f o r m a t i o n

g e n e r a t e d by a s o l u t i o n a l g o r i t h m t o c a l c u l a t e s e n s i t i v i t y and s t a b i l i t y bound i n f o r m a t i o n a s a s o l u t i o n i s approached.

This

i d e a i s n a t u r a l and e f f e c t i v e and i s b a s e d on t h e t h e o r e t i c a l l y d e m o n s t r a b l e f a c t t h a t s o l u t i o n and s e n s i t i v i t y and s t a b i l i t y i n f o r m a t i o n r e q u i r e e s s e n t i a l l y t h e same a s s u m p t i o n s and a n a l ogous c a l c u l a t i o n s .

The p a r t i c u l a r a l g o r i t h m u t i l i z e d t o

o b t a i n t h e c o m p u t a t i o n a l r e s u l t s r e p o r t e d i n t h i s book i s based on a c l a s s i c a l p e n a l t y f u n c t i o n .

However, t h e approach i s

c l e a r l y a p p l i c a b l e t o any v a l i d s o l u t i o n a l g o r i t h m and s h o u l d y i e l d an e f f i c i e n t p r o c e d u r e f o r a d d i n g a s e n s i t i v i t y and s t a b i l i t y a n a l y s i s c a p a b i l i t y t o any n o n l i n e a r programming code, once t h e f o r m u l a s a p p r o p r i a t e t o t h e g e n e r a l a l g o r i t h m i c m a n i p u l a t i o n s and d a t a o r g a n i z a t i o n a r e developed. W e have p r o v i d e d f i r s t - and second-order

formulas €or projected-gradient,

sensitivity

reduced-gradient,

and

augmented-Lagrangian c a l c u l a t i o n s , and have shown t h e i n t i m a t e and n a t u r a l c o n n e c t i o n between s e n s i t i v i t y c a l c u l a t i o n s and

287

10.3 Conclusions

c a l c u l a t i o n s b a s e d on Newton's method.

Thus, t h e e x p l o i t a t i o n

of s o l u t i o n algorithm c a l c u l a t i o n s t o e s t i m a t e s e n s i t i v i t y i n f o r m a t i o n a s a s o l u t i o n i s approached h a s been demonstrated f o r l a r g e c l a s s e s o f t h e most e f f e c t i v e a l g o r i t h m s .

Aside

from t h e e x p l o i t a t i o n o f a l g o r i t h m i c c a l c u l a t i o n s , w e have g i v e n s e n s i t i v i t y and s t a b i l i t y bound f o r m u l a s d i r e c t l y i n

t e r m s o f t h e g i v e n problem f u n c t i o n s . Computational e x p e r i e n c e d e m o n s t r a t e s t h e p r a c t i c a l f e a s i b i l i t y and a p p l i c a b i l i t y o f g e n e r a t i n g r a t h e r i n t r i c a t e s e n s i t i v i t y and s t a b i l i t y i n f o r m a t i o n f o r n o n l i n e a r programming problems o f p r a c t i c a l i n t e r e s t .

Indeed, given t h e c l o s e

c o n n e c t i o n between s o l u t i o n a l g o r i t h m and s e n s i t i v i t y and s t a b i l i t y c a l c u l a t i o n s , it s e e m s a p p a r e n t t h a t i f an a l g o r i t h m can d e t e r m i n e a s o l u t i o n , t h e n a l g o r i t h m i c m a n i p u l a t i o n s can be u t i l i z e d t o y i e l d an abundance o f s e n s i t i v i t y and s t a b i l i t y i n f o r m a t i o n a s t h e s o l u t i o n i s approached.

Practitioners

s h o u l d know t h a t t h e r e i s no t h e o r e t i c a l impediment t o t h e c o m p u t a t i o n a l i m p l e m e n t a t i o n and r o u t i n e e x p l o i t a t i o n of t h i s fact. A s a f i n a l comment, it s h o u l d be s t r e s s e d t h a t s e n s i t i v i t y

and s t a b i l i t y methodology i s n o t merely a c o l l e c t i o n of t e c h niques f o r generating i n s i g h t s concerning t h e s t a t u s of a s o l u t i o n r e l a t i v e t o d a t a changes.

I t i s a p o i n t of view, an

a p p r o a c h , from which a l l a s p e c t s o f m a t h e m a t i c a l programming can and undoubtedly w i l l be r e d e v e l o p e d and e x t e n d e d .

This Page Intentionally Left Blank

Appendix I

Notation, Conventions, and Symbols

1.

INTRODUCTION Definitions of many key concepts are given in the text as

they are first introduced, or appropriate references are suggested.

Here, for convenience, we summarize frequently used

conventions.

We have drawn freely from other sources, especi-

ally from the lucid summary of standard conventions given by Mangasarian (1969, Chap. 1 and App. A-D).

Several references

that have been consulted or that may be recommended are Berge (1963) for topology and point-to-set maps; Fiacco and McCormick (1968) for first- and second-order optimality conditions, convex programming, and Wolfe duality and penalty functions; Hestenes (1966) for results in the calculus of variations and optimal control; Hestenes (1975) and Luenberger (1969) for functional analysis results that are applicable to optimization; Luenberger (1973) for basic results in linear and nonlinear programming and for an introduction to nonlinear programming algorithms; Mangasarian (1969) for first-order optimality conditions, constraint qualifications, and convexity notions; Ortega and Rheinboldt (1970) for general results in functional analysis, linear algebra, and iterative processes;

290

Appendix I

a n d R o c k a f e l l a r (1970) f o r convex a n a l y s i s .

The books by

L u e n b e r g e r a n d O r t e g a and R h e i n b o l d t a r e l i s t e d i n t h e s e l e c t e d b i b l i o g r a p h y and t h e o t h e r s i n t h e r e f e r e n c e s . SETS, VECTORS, AND SPACES

2.

S e t s a n d s p a c e s are u s u a l l y d e n o t e d by c a p i t a l A r a b i c

l e t t e r s , v e c t o r s a n d c o n s t a n t s by s m a l l A r a b i c l e t t e r s , a n d t h e empty s e t by fl.

The i n t e r s e c t i o n o f t h e t w o sets M a n d N

i s t h e set o f e l e m e n t s b e l o n g i n g t o b o t h M a n d N a n d i s d e n o t e d The u n i o n o f M a n d N i s t h e s e t o f e l e m e n t s be-

by M C7 N.

l o n g i n g t o e i t h e r M o r N a n d i s d e n o t e d by M U N .

I f M is a

s u b s e t o f N , w e w r i t e M C N o r N 3 M.

T h i s means t h a t e a c h

e l e m e n t of M i s a l s o a n e l e m e n t o f N .

I f x i s an element of M,

t h i s i s d e n o t e d by x x f M.

E

and i f x i s n o t a n e l e m e n t o f M, by

M,

W e sometimes w r i t e x , y

E

M, f o r x

M and y

E

E

M.

In

g e n e r a l , a s l a s h across a symbol d e n o t e s i t s n e g a t i o n . I f M i s t h e set o f x

E

N having an a d d i t i o n a l property,

s a y f ( x ) = 0 , where f i s a r e a l - v a l u e d f u n c t i o n , t h e n w e w r i t e M = {x

E

N l f ( x ) = 0) or M = { x l f ( x ) = 0) i f x

from t h e c o n t e x t .

E

N is understood

I f t h e e l e m e n t s of M c a n b e l i s t e d , e . g . ,

i f M i s t h e s e t o f i n t e g e r s from 1 t h r o u g h m , w e may w r i t e M = 11, 2 ,

..., m).

I f M C N and N C M w e w r i t e M = N, and i f

t h e r e a r e no e l e m e n t s i n M w e w r i t e M = fl. M

#

Thus, M # N o r

means M a n d N are n o t t h e same s e t s , o r t h a t M i s n o t

empty, r e s p e c t i v e l y . The ( C a r t e s i a n ) p r o d u c t o f two sets M a n d N i s d e n o t e d by M x N and i s d e f i n e d a s t h e s e t o f o r d e r e d p a i r s

by M

x

N = {(x, y ) ( x

E

M,

y

E

N).

(x, y) given

291

Appendix I

The symbol En r e p r e s e n t s n - d i m e n s i o n a l E u c l i d e a n s p a c e , whose b a s i c p r o p e r t i e s are assumed known. x = (xl,

..., x n I T I a n

Thus, x e En means

o r d e r e d n - t u p l e of r e a l numbers x l ,

...,

xn, d e f i n i n g a n n-component column "vector," t h e s u p e r s c r i p t T The v e c t o r x e En i s a l s o c a l l e d a

denoting transposition.

F r e q u e n t l y u s e d p r o p e r t i e s of En w i l l b e 1 q u i c k l y reviewed. I f x , y e En a n d a E , then x + y = (x 1 " p o i n t " i n En.

yl,

..., xn

+ ynIT and a x

t h i s i s aM = I x l x = az, z

z

E NI,

..., ax,).T

= (axl, E

MI a n d M

+

+

Consistent with

N = {xlx = y

+ z, y

e M,

i f M, N c E". T

The " s c a l a r " o r " i n n e r " p r o d u c t x y of two v e c t o r s T

+

x , y E En i s d e f i n e d by x y = xlyl

8 . .

+

xnyn.

The E u c l i d e a n

norm llxll of x E En is d e f i n e d a s llxll = (xTx)l'*. t a n c e between x

En a n d y e En i s g i v e n by

E

11 x -

The d i s y ( I , which

d e f i n e s a metric on En. A

" n e i g h b o r h o o d " of a p o i n t xo i n a g i v e n s p a c e i s d e n o t e d

by N ( x o ) a n d i s t a k e n t o mean a n o p e n s u b s e t o f t h e s p a c e c o n t a i n i n g xo.

I n En,

a n e i g h b o r h o o d of xo E En may be t a k e n

t o b e a n "open b a l l " o f r a d i u s A ( c e n t e r e d ) a t xo g i v e n by B A ( x O )= { x E En

I

IIx

-

xoII < A 1 f o r some r e a l number A > 0.

W e may t h e n d e f i n e x t o b e a n " i n t e r i o r p o i n t " of M C En i f

some n e i g h b o r h o o d o f x i s c o n t a i n e d i n M.

I f e v e r y p o i n t of M

i s a n i n t e r i o r p o i n t of M, t h e n M i s s a i d t o be open.

A point

x i s s a i d t o b e a p o i n t of c l o s u r e o f a s e t M i f e v e r y n e i g h borhood of x c o n t a i n s a t l e a s t o n e e l e m e n t of M. need n o t i t s e l f be a n e l e m e n t of M . )

(Note that x

The s e t M i s s a i d t o b e

c l o s e d i f e v e r y p o i n t o f c l o s u r e o f M i s i n M.

The s e t of

p o i n t s of c l o s u r e o f M i s c a l l e d t h e c l o s u r e of M a n d i s den o t e d by

E.

The s e t o f i n t e r i o r p o i n t s o f M i s c a l l e d t h e

i n t e r i o r o f M a n d i s d e n o t e d by M

0

.

A boundary p o i n t o f M i s

292

Appendix I

any p o i n t x s u c h t h a t e v e r y n e i g h b o r h o o d o f x c o n t a i n s a t l e a s t one e l e m e n t of M a n d o n e e l e m e n t n o t i n M.

A set M i s c a l l e d

bounded i f t h e r e e x i s t s a r e a l number c s u c h t h a t IIxII 5 c f o r each x

M.

E

a n d bounded.

The s e t M

c

En i s c a l l e d compact i f i t i s c l o s e d

An i m p o r t a n t p r o p e r t y o f a compact s e t M i s t h a t

e v e r y s e q u e n c e o f p o i n t s i n M h a s a l i m i t p o i n t i n M. Although xi may b e u s e d t o d e s i g n a t e t h e i - t h component o f t h e v e c t o r x, t h i s n o t a t i o n w i l l be more f r e q u e n t l y u s e d t o denote a p a r t i c u l a r v e c t o r i n t h e given space.

w i l l be clear from t h e c o n t e x t . t h e n {x,) o f {x,)

d e n o t e s t h e "Sequence" x l ,

k-*m

- -X I [

o r xk

x2,

+

-

...,

En f o r k = 1, 2 ,

... .

A "subsequence"

... .

The 1 , where j = 1, 2 , j E" c o n v e r g e s t o a # * l i m i t " x i f and o n l y i f

= 0 , which means t h a t , g i v e n 6 > 0 ,

t h e r e e x i s t s SF

XI\ < 6 f o r a l l k 1. F.

W e a l s o w r i t e l i m xk = kTI t o e x p r e s s t h i s c o n v e r g e n c e . A p o i n t 2 E En i s a

such t h a t ( ( x k

x

E

w i l l b e d e n o t e d by {x,

sequence {xk) c

l i m IIxk

I f xk

The meaning

" l i m i t p o i n t " o f {x,)

i f and o n l y i f t h e r e e x i s t s a subsequence

s u c h t h a t xk + 2 . I f {xk} c o n v e r g e s t o X, it i s j j c l e a r t h a t a n y c o n v e r g e n t s u b s e q u e n c e Ix, 1 must a l s o c o n v e r g e {xk 1 o f {x,)

t o TI, i.e.,

x is

j t h e u n i q u e l i m i t p o i n t o f { x k l i f and o n l y i f

K is t h e l i m i t of {xk). The f o l l o w i n g c o n s t r u c t s p r o v i d e a d d i t i o n a l v e r y i m p o r t a n t a n d f r e q u e n t l y u s e d l i m i t i n g p r o p e r t i e s o f a s e q u e n c e {x,)

real numbers t h a t n e e d n o t b e c o n v e r g e n t .

of

They e s s e n t i a l l y

d e f i n e t h e smallest a n d l a r g e s t l i m i t p o i n t s o f t h e s e q u e n c e . L e t Sk = { x i ( i 1. k ) and zk = i n f Sk, f o r k = 1, 2 ,

Sk+l C Sk, w e must h a v e z

... .

~ 1.+ zk~ f o r e a c h k, i.e.,

n o n d e c r e a s i n g s e q u e n c e o f r e a l numbers.

(2,)

Hence, if {x,)

bounded, it f o l l o w s t h a t zk * 8, a number c a l l e d t h e

Since

is a

is

293

Appendix I

lim xk

"limit inferior" of {xk}, and denoted by is not bounded below, we write superior" of {x,) If

lim k-tw

X,

=

k+m

lim xk = Z or xk

,.

k

k+m

=

-m.

is denoted and defined by

+ J xk = Z,

k-+m

lim x

k-tm

-+

= S.

If {x,)

The "limit k+m

xk = -=(-xk).

then {xk} converges to S, i.e.

k+m

,

S, in terms of our previous notation.

also write "lim inf" for

lim and

We

"lim sup" for E.

The notation lim $(x) = $(T)or $(x)

+

$ ( S ) as x

+

Z is

X-+Z

frequently used, where x and S may be vectors or numbers and $ is a real-valued vector function.

The symbol x

+

H means that

x converges to Z in any manner, i.e., we may take x = xk where {x,}

S

is any sequence (in the set to which x may be

restricted). -+

+

For each such {xk),

$L

(x) + $ ( S ) means that @(xk)

$ (9).

It is convenient to have a convention €or limiting an operation to elements that are a subset of a given set of elements.

The practice we follow is common and is illustrated

by the following examples:

if B C N = 11,

..., n},

then

CieB zi means sum those elements zi over those i belonging to the subset B of integers 1, n; denotes the

...,

UtEN(~-r(t)

union of those sets r(t) over those values of t belonging to

N < F ) C TI where N(F) may, for example, represent a neighborhood of

F

in some space TI or T may be an index set and we may

have r(t) =

r t'

We close this section with one additional very important concept, "linear independence" of several vectors. combination" of xl,

..., xm

E

"linear

En is a vector y E En such that Am E E1 If y = 0 implies that

...,

y = Cmi=l X.X. for some hi, 1 1 each Xi = 0, then the vectors xl, linearly independent.

A

.

..., xm

Otherwise, i.e.

,

are said to be

if Cy=l Xixi = 0 and

294

Appendix I

some A

..., xm a r e

i # 0, t h e n xl,

dependent."

s a i d t o be " l i n e a r l y

I t f o l l o w s t h a t any m v e c t o r s i n En a r e l i n e a r l y

dependent i f m > n. 3.

FUNCTIONS AND MAPPINGS "Maps" ( o r mappings) and o p e r a t o r s a r e g e n e r a l l y d e n o t e d

by c a p i t a l A r a b i c o r Greek l e t t e r s , w h i l e s m a l l A r a b i c o r Greek l e t t e r s a r e u s u a l l y used f o r r e a l - o r v e c t o r - v a l u e d f u n c t i o n s . A "mapping"

r

from t h e s e t X t o t h e s e t Y i s a correspond-

r

e n c e a s s o c i a t i n g a s u b s e t ( c a l l e d t h e "image" o r " v a l u e " o f a t x) r ( x ) of Y with every x

E

A " f u n c t i o n " $ from a s e t X

X.

i n t o a set Y i s a correspondence t h a t a s s o c i a t e s with every element x

E

X a s i n g l e element y

E

Y.

W e w r i t e 4:

X

-+

Y to

d e s i g n a t e t h i s c o r r e s p o n d e n c e and @ ( x )= y t o d e n o t e t h e f a c t t h a t t h e image ( o r v a l u e ) $ ( x ) o f $ a t x i s t h e s i n g l e e l e m e n t y.

A scalar-valued o r real-valued

from a s e t X i n t o E

.

1

function $ is a function

An m-dimensional v e c t o r f u n c t i o n $ i s a

f u n c t i o n from X i n t o Em.

The components $ l ( x ) ,

$ ( x ) a r e r e a l - v a l u e d f u n c t i o n s on X.

..., $,(x)

of

I f e a c h component of

$(x) h a s a g i v e n p r o p e r t y , t h e n u s u a l l y $ ( x ) i s s a i d t o have t h a t property.

The n o t a t i o n $

0 , $ > 0, $ = 0, $ # 0 means

t h a t e a c h component o f 4 i s n o n n e g a t i v e , o r p o s i t i v e , o r 0 , o r t h a t some component i s n o t 0 , r e s p e c t i v e l y . A

real-valued

f u n c t i o n $ d e f i n e d on an open s e t M C E

s a i d t o be c o n t i n u o u s a t TI

E M

quence {xk} C M such t h a t xk

-+

i f $(xk)

x.

-+

$(?)

n

is

f o r e a c h se-

The f u n c t i o n $ i s s a i d t o

be c o n t i n u o u s on M i f i t i s c o n t i n u o u s a t e a c h p o i n t o f M. C o n t i n u i t y p r o p e r t i e s o f f u n c t i o n s and maps a r e f u r t h e r d i s c u s s e d i n C h a p t e r 2 , pp. 11-14.

295

Appendix I

Consistent with the above, an m-component vector-valued function @ = (@l,

.'I'

..., @m)

is said to be continuous if each

component function @i (i = 1,

..., m)

is continuous.

If each

component of @ is continuous on an open subset of the space containing the set M we write

@

C(M), or simply

E

@

E

C if

restriction to M is understood or if M is the entire space. If the component functions have continuous partial derivatives of order k, this is designated analogously by @ k @ E C .

The infimum of a real-valued function is a number f3 such that @(x) 1. 6 for all x

E

Ck (M) or

defined on a set M

@ E

M (i.e., @ is

bounded from below on M by 8 ) and further, given any 6 > 0, there exists x @(x)

= f3

E

M such that @(x) < B

or f3 = infx @(x) s.t. x

ject to." x

E

If there exists T

E

E

+

6.

We write infxeM

M, where "s.t." means "sub-

M such that @(F) 5 @(XI forevery

M, then @(F) is called the "minimum value" or "global

minimum value" or "optimal value" of @ on M I and I is called a "global minimum'' of

@(F) = minX @(x)

s

@

on M.

t. x

e

We write @(F) = minxeM @(x) or If

M.

is a global minimum of @ on

M n B X (I)for some X > 0, then @(TI) is called a "local minimum value" or (locally optimal value and Z a "local minimum" or "local solution. 'I Supremum and maximum are defined analogously to infimum Specifically, supxeM @ (x) = -infxeM[-@(x) I and

and minimum.

-minxeM [-@(x)], so that it suffices to study either minimization or maximization. An important result is

maxxeM @(x)

=

that a lower (upper) semicontinuous function (see definition on p. 12) @ defined on a nonempty compact set M C En is bounded from below (above) and there exists x global minimum (maximum) of

@

on M.

e

M such that W is a

296

Appendix I A real-valued

f u n c t i o n $ d e f i n e d on a n open s e t M C En i s

s a i d t o be " d i f f e r e n t i a b l e " a t P

+ 8 ( X , x ) llxll f o r a l l x

E En

E

M if

such t h a t

$(P + x )

X + x

a v e c t o r ( w i t h f i n i t e components) and B ( f ,

$(P)+ v(P)'x

M, where v ( X ) i s

E

x)

=

-P

0 as x

-P

0.

The f u n c t i o n 4 i s s a i d t o be d i f f e r e n t i a b l e on M i f i t i s The f u n c t i o n $ i s s a i d t o

d i f f e r e n t i a b l e a t e a c h p o i n t i n M. have a p a r t i a l d e r i v a t i v e a t

x with

ponent xi o f x i f t h e q u o t i e n t t o a f i n i t e number a s 6

+.

r e s p e c t t o t h e i - t h com-

[$(P+ 6 ei)

0 , where ei

E

-

$(P)]/6c o n v e r g e s

En i s a v e c t o r h a v i n g

one i n t h e i - t h component and z e r o i n e a c h r e m a i n i n g component. When i t e x i s t s and i s f i n i t e , t h e l i m i t o f t h e g i v e n q u o t i e n t

i s c a l l e d t h e " p a r t i a l d e r i v a t i v e " o f $ w i t h r e s p e c t t o xi a t

Z and i s d e n o t e d by a $ ( P ) / 3 x i . V $ (X) = 1 3 4 (P)/ax,,

..., a $ ( P ) / a x n ]

(row) v e c t o r

i s c a l l e d the "gradient"

I f $ i s d i f f e r e n t i a b l e a t R, t h e n V $ ( f I )

o f $ a t Z.

w e may t a k e v ( E ) = V $ ( f I )

@(K+ x).

The n-component

e x i s t s and

i n t h e e x p r e s s i o n g i v e n above f o r

Another i m p o r t a n t f a c t i s t h a t i f V$(W) e x i s t s and

V $ i s c o n t i n u o u s a t R, t h e n C$ i s d i f f e r e n t i a b l e a t X. I f 4 = ($1,

...,

an m-component v e c t o r f u n c t i o n

d e f i n e d on a n open s e t M C En,

t h e n $ i s s a i d t o be d i f f e r -

M i f e a c h o f i t s components $i i s d i f f e r e n -

entiable a t W

E

t i a b l e a t P.

A s before,

$ i s s a i d t o be d i f f e r e n t i a b l e on M

i f it i s d i f f e r e n t i a b l e a t e v e r y p o i n t o f PI.

The v e c t o r func-

t i o n $ i s s a i d t o have p a r t i a l d e r i v a t i v e s a t W w i t h r e s p e c t t o t h e components xl,

..., xn

of x

E

E n i f e a c h component Qi

h a s p a r t i a l d e r i v a t i v e s a t R w i t h r e s p e c t t o xl,

..., xn.

The

Appendix I

297

n o t a t i o n V$(H) t h e n r e p r e s e n t s

an m

x

n m a t r i x c a l l e d t h e J a c o b i a n o f $ a t K.

The r e a l - v a l u e d f u n c t i o n $ d e f i n e d on an open set M

c En

$(X + x )

i s s a i d t o be " t w i c e d i f f e r e n t i a b l e " a t X E M i f

=

~ $ ( ~ r +) x( 1 / 2 ) x T v 2 $ ( z ) x + y ~ x)llx112, , f o r a l l x E E" 2 such t h a t + x E M, where V is an n x n matrix (with

+

x

$(x)

f i n i t e e l e m e n t s ) and y ( Z , x )

+

0 as x

i s c a l l e d t h e H e s s i a n o f $ a t Z.

+

0.

The m a t r i x V2$(F)

I t f o l l o w s t h a t i f V$ is

d i f f e r e n t i a b l e a t E o r i f V$ h a s c o n t i n u o u s ( f i r s t p a r t i a l ) d e r i v a t i v e s a t K, t h e n $ i s t w i c e d i f f e r e n t i a b l e a t K and

1

a 2 $(rr)/ax,ax, /axlaxn a2$ (TI)

- 9 .

a 2 $(rr)/axnaxl a2$

( a/axnaxn

so t h a t t h e i j - t h e l e m e n t of V 2 $(K) i s g i v e n by f o r i , j = 1, of $ a t I.

..., n ,

a 2$(K)/ax.axi

7 c a l l e d t h e "second p a r t i a l d e r i v a t i v e s "

2

~f V $ i s c o n t i n u o u s a t

X (i.e., i f t h e elements

of V 2 $ a r e c o n t i n u o u s ) , t h e n V2$ (K) i s symmetric. p a r t i a l derivatives of

+

The k-th

a t H a r e defined analogously.

298

Appendix I

d e f i n e d , w e may g i v e t h e well-known

With V $ a n d V$ '

"mean-

v a l u e " a n d T a y l o r ' s s e c o n d - o r d e r t h e o r e m s , e a c h of which w e use f r e q u e n t l y .

The "mean-value theorem,

'I

which w e a l s o r e f e r

to a s a f i r s t - o r d e r Taylor's expansion, states t h a t i f $ i s a r e a l - v a l u e d d i f f e r e n t i a b l e f u n c t i o n o n a n open convex s e t

M C En, t h e n $(XI = $(Y) + V$[Q(X, y , 6 ) 1 ( x f o r any g i v e n x , y

E

M,

-

Y)

where q ( x , y , 6 ) = y

some r e a l number 6 s u c h t h a t 0 < 6 < 1.

+

6(x

-

y) for

I f , additionally, $

i s t w i c e d i f f e r e n t i a b l e o n M, t h e n T a y l o r ' s t h e o r e m a s s u r e s t h e v a l i d i t y of t h e " s e c o n d - o r d e r T a y l o r ' s e x p a n s i o n , " $ ( x ) = $(Y) + V$(Y) ( x + 1/2(x

where ~ ( x y, ,

-

B) = y

Y)

+

-

Y)

T 2

v $[n

B(x

-

(x,

Yr

B) 1 (x

-

Y)

I

y ) f o r s o m e r e a l number B s u c h

t h a t 0 < 6 < 1. 4.

MATRICES

Matrices, o t h e r t h a n t h e H e s s i a n V 2 f of a r e a l - v a l u e d f u n c t i o n f or t h e J a c o b i a n V$ of a v e c t o r - v a l u e d

f u n c t i o n $,

w i l l g e n e r a l l y b e d e n o t e d by c a p i t a l A r a b i c l e t t e r s .

An m

x

where a i j ,

p m a t r i x A i s d e f i n e d as

..., m, j = 1, ..., p , (ail, ..., a ) i s c a l l e d iP

i = 1,

The r o w vector

t h e column v e c t o r ( a l j , o f A.

..., am j ) T i s

a r e real numbers. t h e i - t h r o w o f A and

c a l l e d t h e j - t h column

The " t r a n s p o s e " o f A i s d e n o t e d by AT a n d i s t h e p

x

m

299

Appendix I

m a t r i x d e f i n e d as

The m a t r i x A i s s a i d t o b e "symmetric" i f A = A

T

.

A submatrix

of A i s a m a t r i x w i t h rows a n d columns e x t r a c t e d , r e s p e c t i v e l y , from rows a n d columns o f A. The r a n k o f a m a t r i x A i s t h e l a r g e s t number o f l i n e a r l y i n d e p e n d e n t r o w s ( c a l l e d t h e r o w r a n k ) or columns ( c a l l e d t h e column r a n k , which a l w a y s e q u a l s t h e r o w r a n k ) .

An

m

x

m

m a t r i x A i s s a i d t o be " n o n s i n g u l a r " i f it h a s r a n k m , i . e . , i f it h a s "full r a n k . "

m

x

I n t h i s i n s t a n c e , t h e r e e x i s t s a unique

m m a t r i x c a l l e d t h e i n v e r s e o f A a n d d e n o t e d by A - l

t h a t A A-l m a t r i x"

1 =

= A-lA

[:

- 0 .

= I, where I d e n o t e s t h e m x

:I,

such

m "identity

where t h e z e r o e n t r i e s mean t h a t a l l t h e "of f - d i a g o n a l " eleW e sometimes w r i t e A - ~f o r ( A - ~ ) ~ .

ments are zero. I f A is an m

x

p m a t r i x and B i s a p

x

q m a t r i x d e f i n e d as

follows,

where ai,

b.

3

E

Ep,

i = 1,

..., m

a n d j = 1,

..., q,

then t h e

300

Appendix I

product AB, A times B, exists and is defined as

It follows that (AB)T = BTA T If A and B are m

x

.

p matrices, then the sum A

+

B

exists and is defined as

A + B =

I

where A is defined as above and B analogously. product aA for any a

E

Also, the

E1 is the matrix whose ij-th element is

aaijm

An

T z Az

x

m matrix A is said to be positive semidefinite if

0 for all z

E

Em and positive definite if zTAz > 0 for

Em such that z # 0.

Similarly, an m x m matrix A is said to be negative semidefinite if zTAz 5 0 €or all z E Em and negative definite if zTAz < 0 for all z c Em such that all z

z # 0.

E

If A is positive or negative definite, then it follows

that A is nonsingular and A-l is, respectively, positive or negative definite. A diagonal m

x

m matrix with elements zl,

denoted by diag(zi), i = 1,

..., zm

..., m, and defined to be

is

301

Appendix I

The i n v e r s e [ d i a g ( z i ) ] - l e x i s t s i f a n d o n l y i f zf'

..., m, d i a g ( zf') .

a n d it i s e a s i l y s e e n t h a t [ d i a g ( z i ) 1 - l =

i = 1,

t h e norm of t h e m a t r i x A i s d e f i n e d a s IlAll =

If x e En,

I

supx{IIAxII

exists for

llxll = 1 ) .

This definition extends t o general

normed s p a c e s . 5.

PARAMETRIC NLP The r e s u l t s i n t h e book a r e m a i n l y c o n c e r n e d w i t h r e a l i z a -

t i o n s or g e n e r a l i z a t i o n s of a p a r a m e t r i c n o n l i n e a r p r o g r a m i n g (NLP) problem o f t h e f o r m minx f ( x , where x e En, h: En

x

E E

* Ep.

Ek

R ( E ) = {x

s.t.

E)

E

E

g(x,

k , f : En

E

x

E)

h(x,

2 0,

k * E 1, g : En

x

E)

P3(€)

= 0

Ek -+ Em, and

The f e a s i b l e r e g i o n o f P 3 ( € ) i s d e n o t e d by

Enlg(x,

0, h(x,

E)

E)

= 0 ) and d e f i n e s a p o i n t -

to-set map ( o r mapping) R from Ek t o s u b s e t s o f En.

The

o p t i m a l v a l u e o f P J ( ~ )i s d e f i n e d a s

f

*

(E)

=

infx{f (x,

E)

Ix

R(E)1

E

{+-

# Jl

if

R(E)

if

R(E) =

Jl

*

w i t h e f f e c t v e domain o f € a n d domain o f R d e f i n e d t o be k D = { E E E R ( E ) # Jl} a n d t h e s e t o f ( g l o b a l ) s o l u t i o n s by S(E) = {x

E

R ( ~ ) l f ( x E, ) = f * ( E ) } ,

a l s o defining a point-to-set A "local s o l u t i o n "

by

X(E)

map S f r o m Ek t o s u b s e t s of En.

o r " l o c a l minimum" o f P 3 ( € ) w i l l b e d e n o t e d

a n d many r e s u l t s w i l l a l s o c o n c e r n t h e l o c a l o p t i m a l

v a l u e f u n c t i o n f * ( ~ )= f [ x ( E ) ,

€1.

The ( u s u a l ) L a g r a n g i a n

a s s o c i a t e d w i t h problem P 3 ( € ) i s d e f i n e d t o b e L(x,

U, W, E )

= f(x,

E)

-

T

u g(x,

E)

T + w h ( x , €1,

302

Appendix I

where u

Em and w

E

Ep are called Lagrange multipliers and

E

the superscript T denotes transpose, so that we have L: En Em

x

Ep

Ek

x

-+

El.

A local solution

X(E)

x

along with so-called

optimal multipliers (that satisfy the well-known Karush-KuhnTucker first-order necessary conditions) define what will be called a Karush-Kuhn-Tucker triple, y ( E ) = [x(E), U(E), w ( € 1 I T

.

Important realizations of P3(€) are: (i) f and the components of g and h are affine functions of x, (ii) f is a quadratic function of x [i.e., f(x, - ) = xTAx + bTx and the matrix A and vector b do not depend on XI, and the components of g and h are affine functions of x; (iii) f is convex in x, the components of g are concave functions of x, and the components of h are affine functions of x.

In the first instance

P3 is called a linear program, in the second instance a quadratic program, and in the third a convex program. Important specializations of problem P 3 ( ~ )will be considered, e.g.,

the so-called right-hand-side (rhs) perturbation

problem minx f(x) where

E~

E

' E g(x) 2 ,

s.t.

Em, c2

E

Epr

E = (E

h(x) =

E

2

,

1, E ~ ) and ~ , the rhs inequality-

constrained perturbation problem minx f(x) where

E

E

Em.

P1(E)

Also, we shall give many results for generaliza-

tions of P 3 ( ~,) e.g. minx f(x,

g(x) 2 E,

s.t.

E)

,

for

s.t.

x

E

R(E)

where R(E) is not necessarily defined in terms of functional inequalities or equalities, results for more general spaces, and results encompassing the possibility of an abstract set

303

Appendix 1

inclusion x tioned.

E

M, i n a d d i t i o n t o t h e v a r i o u s c o n s t r a i n t s men-

However, t h e g e n e r a l framework w i l l be a s d e s c r i b e d

f o r t h e p r o t o t y p e model P 3 ( ~ ) . Our main c o n c e r n s w i l l be t h e c o n t i n u i t y p r o p e r t i e s o f R and S and c o n t i n u i t y and d i f f e r e n t i a b i l i t y p r o p e r t i e s o f f

*

and y , a l o n g w i t h bounds on t h e s e q u a n t i t i e s , r e l a t i v e t o changes i n t h e problem p a r a m e t e r

E.

W e s h a l l a l s o be i n t e r e s t e d

i n c o n v e x i t y p r o p e r t i e s o f R, S , and f*. W e u s e t h e term bounds r a t h e r l o o s e l y t o d e s c r i b e any s e t

c o n t a i n i n g any numbers or p o i n t s a s s o c i a t e d w i t h s o l u t i o n i n formation of i n t e r e s t , e.g.,

upper and lower bounds on t h e

o p t i m a l v a l u e f u n c t i o n on a g i v e n s e t o f p a r a m e t e r s , a n upper bound on t h e d i s t a n c e between s o l u t i o n p o i n t s f o r d i f f e r e n t parameter values, etc. t h e problem p a r a m e t e r s .

are.

Bounds may o r may n o t b e a f u n c t i o n of " P a r a m e t r i c bounds" are t h o s e t h a t

" E r r o r bounds" a r e bounds on t h e d i s c r e p a n c y between a

c a l c u l a t e d s o l u t i o n v a l u e and a n a c t u a l s o l u t i o n v a l u e . F i n a l l y , " s e n s i t i v i t y " i n f o r m a t i o n h e r e g e n e r a l l y refers t o any measure a s s o c i a t e d w i t h a s o l u t i o n change c o r r e s p o n d i n g t o an a r b i t r a r i l y s m a l l change i n t h e problem p a r a m e t e r s , w h i l e " s t a b i l i t y " i n f o r m a t i o n i s any such measure c o r r e s p o n d i n g t o a f i n i t e change.

W e s h a l l o f t e n mean o p t i m a l v a l u e o r Karush-

Kuhn-Tucker t r i p l e p a r a m e t e r d e r i v a t i v e s by s e n s i t i v i t y i n f o r m a t i o n , when t h e s e q u a n t i t i e s e x i s t , and w i l l o f t e n be r e f e r r i n g t o o p t i m a l v a l u e o r KKT-triple bounds, r e l a t i v e t o f i n i t e p a r a m e t e r changes, i n t h e c o n t e x t o f s t a b i l i t y information.

304 6.

Appendix I

SYMBOLS For c o n v e n i e n c e w e l i s t t h e p r i n c i p a l symbols a n d d e f i n i -

t i o n s t h a t a r e a d o p t e d , f o l l o w e d by t h e number o f t h e p a g e on which t h e symbol o r c o n c e p t i s f i r s t d e f i n e d o r d e s c r i b e d .

In

cases where t h e symbol i s n o t d i s c u s s e d i n t h i s a p p e n d i x , t h e t e x t p a g e o n which it f i r s t a p p e a r s i s g i v e n .

I n t h o s e few

i n s t a n c e s where a p a g e number i s n o t g i v e n , t h e g i v e n c o n c e p t

is n o t u s e d i n t h i s book b u t i s w e l l known a n d i s i n c l u d e d f o r completeness. S e c t i o n s 2-5,

The symbols are g r o u p e d u n d e r t h e t i t l e s of where t h e r e s p e c t i v e n o t i o n or r e l a t e d c o n s t r u c t s

are d e s c r i b e d . SETS, VECTORS, AND SPACES

PI X E

t h e empty set, 2 9 0 M

x i s a n e l e m e n t of M,

290

x is n o t a n e l e m e n t of M,

290

M

t h e c l o s u r e o f t h e s e t M,

291

MO

t h e i n t e r i o r of t h e s e t M,

M U N

t h e u n i o n of t h e s e t s M a n d N ,

M n N

t h e i n t e r s e c t i o n of M a n d N , 290

x i

-

"ieIMi

nieI'i M C N

291 290

t h e u n i o n a n d i n t e r s e c t i o n of t h e sets Mi, r e s p e c t i v e l y , where i i s a n e l e m e n t of t h e set I,

293

M i s a subset of N, i . e . ,

i s a l s o i n N , 290

e a c h e l e m e n t of M

Appendix I

305 t h e e l e m e n t s o f M a r e t h e same a s t h o s e of N ,

M = N

290

t h e e l e m e n t s o f M a r e n o t t h e same a s t h o s e o f

N, 2 9 0 t h e s e t of x f o r which t h e s t a t e m e n t f o l l o w i n g t h e v e r t i c a l bar is t r u e , 290

M x N

t h e C a r t e s i a n p r o d u c t o f t h e s e t s M and N ,

[a, b l

= {X E E la

(a, b)

= {x

290

1

x 5 b}, t h e c l o s e d r e a l i n t e r v a l from a t o b , 1 6 E

E 1( a < x < b}, t h e open real i n t e r v a l

from a t o b , 1 7 ( a , bl

= ( x e E1 ( a < x 5 b ) ,

38

[a, b ) En

n-dimensional E u c l i d e a n s p a c e , 2 9 1

T

a s a s u p e r s c r i p t , denotes t r a n s p o s i t i o n , 291

X

= (xl,

...

= xlyl

+

, x n l T i f x e E", vector, 291

T

X Y

* a *

o f x,' y

-

x y t h e scalar or i n n e r p r o d u c t n n'

En,

291

norm of t h e v e c t o r x; i f x e En, t h e E u c l i d e a n norm i s g i v e n by )I x1I = (xTx)1'2, 291

llxll

IIX

E

+

an n-component

YII

d i s t a n c e between x En,

291

E

En and y e En,

a metric on

306

Appendix I

= {x

BA(xo)

I

IIx

-

xoII < A ) ,

an open b a l l o f r a d i u s

X > 0 a t xo, 2 9 1

. ..,

c1X

= (axl,

aM

= {xlx = a z ,

X + Y

=

where a z

E

MI, where a

..., xn

( x l + yl, y E En, 2 9 1

+

E1 and x

E

E

E

1

,

E

En, 2 9 1

291

y n I T , where x

En and

E

M + N

a sequence o f v e c t o r s xk

lxk

2,

{x, j

I

..., 2 9 2

E

E

n

, where

k = 1,

a subsequence o f { x 1 , where j = 1, 2 , k

l i m xk = 51

k-+m

t h e l i m i t of t h e sequence {x,}

l i m IIxk

k-

l i r n xk

- -X I [

=

..., 2 9 2

i s Z, i - e . ,

0 , a l s o w r i t t e n xk

+

I, 2 9 2

t h e l i m i t i n f e r i o r of t h e sequence {x,) o f r e a l numbers, a l s o w r i t t e n l i m i n f x k , 293

G

k+m

-

l i m xk

t h e l i m i t s u p e r i o r of t h e sequence { x k l o f r e a l

k-

numbers, a l s o w r i t t e n l i m s u p xk, 293 k-

l i m @(XI

t h e l i m i t o f t h e r e a l - v a l u e d v e c t o r f u n c t i o n $I a s x c o n v e r g e s t o xo i n any manner, ( i . e . , f o r

X'X0

e a c h sequence {xn} such t h a t xn

+

xo, t h e

g i v e n l i m i t i n g q u a n t i t y i s d e f i n e d and u n i q u e ) , 293

l i m inf x'xO

@

(x)

t h e l i m i t i n f e r i o r of t h e real-valued function @ ( x )a s x c o n v e r g e s t o xo i n any manner, 1 4

l i m s u p @ ( x ) t h e l i m i t s u p e r i o r o f @ ( x )a s x c o n v e r g e s t o x+xo x0, 1 4

Appendix I

l i m Rn n+-

301 t h e i n n e r l i m i t of a sequence o f sets (R,},

15

t h e o u t e r l i m i t o f a sequence o f s e t s {Rn), 1 5

l i m Rn n-

t h e l i m i t of a sequence o f sets {R,},

{ z i , i e 1)

t h e s e t of e l e m e n t s zi s u c h t h a t t h e i n d e x i

w r i t t e n Rn

-+

also

R, 1 6

b e l o n g s t o some s e t I, 2 4

1

i eI

5

zi

AiXi

i=l

t h e sum o f t h e e l e m e n t s zi s u c h t h a t i e I , 293

=

A x + 11 xl, 293

+ Amxm, a l i n e a r combination o f

..., xm

e En,

where A l l

..., Am

e E

1

,

FUNCTIONS A N D MAPPINGS

$ 2 0

o r d e r i n g r e l a t i o n s , meaning r e s p e c t i v e l y t h a t

$'O $ = O

e a c h component of t h e r e a l - v a l u e d v e c t o r f u n c t i o n $ i s nonnegative, p o s i t i v e , o r zero,

$ # O

o r t h a t some component i s n o t z e r o , 2 9 4

r : ~ + x

g e n e r a l symbol used f o r a p o i n t - t o - s e t

map, a

correspondence t h a t a s s o c i a t e s with every e l e m e n t t of T a s u b s e t o f X I 2 9 4 v a l u e (image) of t h e map = {t e Tlr(t)

(Note:

r

at t

E

T, 2 9 4

# 01, c a l l e d t h e domain o f T , 1 5

T h i s i s a l s o used f o r t h e domain of

p a r t i c u l a r maps, e.g., point-to-set

map R, 301

t h e f e a s i b l e region

308

Appendix I

-

G

"t€T"xer graph of

t ) ( t , x ) = { ( t , x ) E T x Xlx E r ( t ) ) , 1 2 (also used f o r t h e graph of

r

o t h e r maps =

r-l

(ti

e.g.,

t h e map R,

U t E A r ( t ) , t h e image o f r on

250 A C D,

called t h e

range of

r

i f A = D, t h e domain o f

r

= {x E x l x

E

r ( t )1 ,

at t

r

t h e inverse of

E

T,

39

a@ ( x ) / a x i

t h e p a r t i a l d e r i v a t i v e o f t h e r e a l - v a l u e d funct i o n @ w i t h r e s p e c t t o xi a t t h e p o i n t X, 2 9 6

a L @ (Z) /axiax

j

t h e second p a r t i a l d e r i v a t i v e o f t h e r e a l - v a l u e d f u n c t i o n @ a t 51 w i t h r e s p e c t t o xi, =

[af (X)/axl,

..., a f ( Z ) / a x n i , t h e

(TI) T ,

..., V@m(51)T]T,t h e

m

j r

297

g r a d i e n t of

f u n c t i o n f a t T? E En,

t h e real-valued = [Val

x

296

n Jacobian at m a t r i x o f t h e m-component vector f u n c t i o n E

En,

V @ is

297

a(@l,

.

...

(Note: r

x

+

A c l a s s i c a l symbol f o r

@m)/a(xlr

...

r

x n ) r 36)

t h e Hessian of t h e real-v alu ed f u n c t i o n @ a t

x

E

En,

an n

x

n m a t r i x whose i j - t h e l e m e n t i s

t h e s e t of f u n c t i o n s t h a t a r e d e f i n e d a n d cont i n u o u s on a n o p e n s u b s e t ( o f t h e g i v e n s p a c e ) t h a t c o n t a i n s t h e s e t M,

295

t h e s et o f f u n c t i o n s t h a t are k t i m e s c o n t i n u o u s l y d i f f e r e n t i a b l e on a n open s u b s e t t h a t c o n t a i n s M,

295

309

Appendix I

infxeM$(x)

t h e infimum o f t h e r e a l - v a l u e d s e t M, 295

minx,,$(x)

t h e minimum ( v a l u e ) o f t h e r e a l - v a l u e d $ on t h e

f u n c t i o n $ on t h e

function

s e t M. 295

o(llhll)

s m a l l o of l l h l l , r e p r e s e n t i n g a r e a l - v a l u e d f u n c t i o n $ ( h ) s u c h t h a t $ ( h ) / l l h l I * 0 as h + 0, 78

lcl

a b s o l u t e v a l u e of t h e number c, 2 1

0 * o+

t h e l i m i t a s 0 c o n v e r g e s t o 0 a l o n g any s e q u e n c e

of p o s i t i v e r e a l numbers, 25 = lim[$(t

B+O+ .~

+ 02)

-

$ ( t ) ] / B , t h e (one-sided)

d i r e c t i o n a l d e r i v a t i v e of t h e r e a l - v a l u e d

func-

t i o n $ a t t h e p o i n t t i n t h e d i r e c t i o n z , 25 - $ 1 ( x ) I $ 2 ( Z ) = $ 1[ $ 2 (z)], it b e i n g u n d e r s t o o d

t h a t a l l functions preceding the vertical bar i n t h e g i v e n e x p r e s s i o n are e v a l u a t e d a t t h e s p e c i f i e d argument u n l e s s o t h e r w i s e s t a t e d , 4 4

M A TRICES

A

,

an m

p m a t r i x , 298

x

AT

t h e t r a n s p o s e of t h e m m a t r i x , 299 I

the i d e n t i t y matrix, 299

x

p m a t r i x A, a p

x

m

310

Appendix I

=I

AB

i

r

the product of the m x p matrix amlT and B = [blr . . . I m, where air b. E Ep, i = 1, 3 j = 1, q, 3 0 0

A = [a,,

...,

...,

...,

bqlr

A + B

the sum of the m x p matrices A and B , whose ij-th elements are, respectively, hij and bij, 300

i

the product of the matrix A by a E E1 , a matrix whose ij-th element is aaij, where the ij-th element of A is aij, 300

aA

diag ( zi)

a diagonal m x m matrix whose i-th diagonal m, 300 element is zi, i = 1,

...,

C = [AIB]

c

A-

=

121

the matrix C is equal to the partitioned matrix given, where the vertical bar denotes the partition and A and B are matrices, also written , [ A B ] , 7 7 same as above, where the horizontal bar denotes the partition, also written without the horizontal bar, 77 inverse matrix, 2 9 9

31 I

Appendix I

A

-T

t h e transpose of t h e inverse of t h e matrix A,

i.e.,

= supx{IIAxII Illxll =

IlAll

A,

= (AT)-’,

-2

-1 -1 A

= A

= AB,

A * B

299

1 1 , t h e norm of t h e m a t r i x

301

= AA, where A is m x m,

A2

A

= (A-lIT

A-T

-

(A2)-l,

175

where A i s m

X

m, 175

t h e p r o d u c t of t h e m a t r i c e s A and B, t h i s

symbol b e i n g u s e d t o d i s t i n g u i s h [A B] = [ A I B ] from [AB] = [A

a m b i g u i t y, 1 29

B ] where t h e r e may be

PARAMETRIC NLP E

=

(El,

. . . I

a k-component parameter, f : En

X

Ek

-+

column v e c t o r i n E

k , t h e problem

301

E1

t h e problem o b j e c t i v e f u n c t i o n , 301 g: En

X

Ek

-f

Em

t h e problem i n e q u a l i t y - c o n s t r a i n t f u n c t i o n g = (g,, h: E n x Ek

-+

..., g,IT.

vector

301

Ep

t h e problem e q u a l i t y - c o n s t r a i n t h = (hl, P3(€): minx f ( x ,

..., hp) T ,

vector function,

301

s.t. g(x, E) 0 h(x, E) = 0 t h e g e n e r a l p a r a m e t r i c NLP problem: f o r a given v a l u e o f E , f i n d an x t h a t minimizes f ( x , E ) s u b j e c t t o g i ( x , E ) 1. 0 ( i = 1, m ) and h . ( x , E) = 0 ( j = 1, p ) , 301 E)

3

...,

...,

312

Appendix I

1 2 s.t. g(x) E , h(x) = E an important special case of P 3 ( € ) called a right-hand-side (rhs) (perturbation) problem, where E~ e Em, E~ E Ep, and E = ( E ~ ,E ~ ) ~302 ,

P~(E):minx f(x)

P1(~):minX f(x) s.t. g(x) L E an important special case of P 3 ( ~ ) ,302 E En(g(x, E ) 2 0 , h(x, E ) = 01, 301 (also used for a general constraint set)

R(E)

= Ex

R

point-to-set map whose value is R ( E ) , 301

f* ( € 1

f*

optimal value function, whose value is given by * ( N o t e : The same f ( E ) as defined, 301. symbol will be used for a l o c a l optimal value function in many results. The meaning will be clear from the context)

S

point-to-set map whose value is

S(E),

301

X(E) = [X1(E), . . . I Xn(E)l T a local solution of P3(~),302

the local optimal value function, when exists, 301 L(x, u, w, € 1 = f(x, € 1 - u g(x, E ) + wTh(x, Lagrangian of P 3 ( ~ ) ,where u

X(E)

the usual Em, w E Ep,

E), E

301

an optimal Lagrange (inequality constraint) multiplier vector associated with ~ ( € 1 , 3 0 2

313

Appendix I

a n o p t i m a l Lagrange ( e q u a l i t y c o n s t r a i n t ) m u l t i p l i e r a s s o c i a t e d with x ( E ) , 302

W(E)

= [ x ( E ), U ( E )

, W ( E ) IT,

a Karush-Kuhn-Tucker

t r i p l e , 302 =

L [ Y ( E ) , €1, t h e o p t i m a l v a l u e L a g r a n g i a n ,

t h e d i r e c t i o n a l d e r i v a t i v e of f z a t the point

D:f*

(E)

E,

i n the direction

26

t h e d i r e c t i o n a l second d e r i v a t i v e of f direction z a t the point

V.

*

82

E,

*

i n the

44

t h e g r a d i e n t operator, subscripted i f necessary f o r c l a r i t y , to denote t h e v a r i a b l e s of t h e partial differentiation.

The g r a d i e n t o f a

s c a l a r f u n c t i o n i s d e f i n e d h e r e t o be a row vector.

I f n o t s u b s c r i p t e d , it i s assumed

t h a t d i f f e r e n t i a t i o n is with respect t o t h e f i r s t argument, 296

t h e g r a d i e n t of t h e optimal value function, 4 4

Appendix I

2

VX~(x,U r W,

E)

Vx[VxL(x, u, w, €IT], the Hessian (matrix) of L(x, u, w, E) with respect to x, an n x n 2 matrix whose ij-th element is a L(x, u, w, E)/ ax.axi = a/ax.[aL(x, U, W, ~ ) / a x ~ Ifor , i, 7 3 j = 1, n. If not subscripted, it is assumed that differentiation is with respect to the first argument, 43

=

...,

= VE[VEf* (E) T1,

VZf* € 1

respect to VL (x u, w, (i = 1,

E)

= 0

..., m;

E,

uigi(xr

j = 1,

...

r

the k

x

k Hessian of f(x, E) with

52 E)

P)

= 0

h . (x, 3

E)

=

0

the Karush-Kuhn-Tucker conditions associated with a local solution x of problem P3(€), 7 4

M

the Jacobian matrix of the Karush-Kuhn-Tucker (first-order) optimality conditions of P3(~), with respect to (x, u, w), 77

315

Appendix I

=

[:

a L./ a x1

1

JI

aL/axn) :

(X,U,W r

T

2

VxEL(xr ur wr (Note:

=

[:;lL

E)

E)

For simplicity, we write

v VEL ixL]r

102

N

-V hT]',

the negative of the Jacobian matrix E P of the Karush-Kuhn-Tucker equations for P 3 ( ~ ) , with respect to E, 77 M(E)V~Y(E)= N ( E ) basic first-order sensitivity equation, 77

-

g(Xr E)

= [gl(xl

...,

E), gr(x, E)lT, the vector of binding inequality constraints, 94

316

Appendix I

P

A t h e r e d u c e d form o f t h e J a c o b i a n o f t h e Karush-Kuhn-Tucker

equations with respect t o

( x , u, w ) , w i t h nonbinding c o n s t r a i n t informat i o n d e l e t e d , and r e a r r a n g e d f o r symmetry, 9 6

A- 1 t h e b l o c k component r e p r e s e n t a t i o n o f t h e inverse of

i, 9 6

317

Appendix I

2 T = I-VExL I

fi

T

VETTI -VEhT] ,

t h e r e d u c e d form o f

t h e n e g a t i v e o f t h e J a c o b i a n o f t h e KarushKuhn-Tucker e q u a t i o n s , w i t h r e s p e c t t o

=

100

A ( E ) - ~ ( E ), g e n e r a l e x p r e s s i o n f o r t h e f i r s t p a r t i a l d e r i v a t i v e s o f a Karush-Kuhn-Tucker t r i p l e , r e d u c e d t o s u p p r e s s n o n b i n d i n g cons t r a i n t information, 100

21

EX

-VEh

f o r m u l a f o r t h e d e r i v a t i v e s o f a r e d u c e d KKT t r i p l e of problem P 3 ( ~ )i n terms of t h e b l o c k components of v:f

*

=

iTVEy+

i-',

2 AT"VEL = -N M

100

'i +

2 VEL, formula f o r t h e

o p t i m a l v a l u e H e s s i a n i n terms o f

i-', 1 0 1

formula f o r t h e o p t i m a l v a l u e Hessian as a f u n c t i o n of V

2

(XI € 1 L,

102

318

Appendix I

formula for the derivatives of a (reduced) KKT triple of problem P 2 ( ~ , ) 129

(reduced) Hessian of the optimal value of P2 ( € 1 r 130 TV 2LVEx, (reduced) Hessian of the optimal X 2 value of P2(€) in terms of VxL, 134

= VEx

VL x, u, w, (i = 1,

M(E,

E)

=

0

u.g. (x, 1 1

E)

=

r

hj(xr

E)

= w.r

I j = 1, p) the necessary conditions associated with a local minimizing point of the barrier-penalty function W(x, E , r), and a perturbation of the Karush-Kuhn-Tucker conditions associated with a local solution of problem P3(€), 158

..., m;

...,

r)VEy(c, r) = N ( E r r) first-order sensitivity equation derived from the perturbed Karush-Kuhn-Tucker equations and associated with W(x, E , r), 161

i=l

j=1

the logarithmic-quadratic mixed barrierpenalty function for P3(€), 50

319

Appendix I

W

*

a l o c a l minimizing p o i n t o f W(x,

r)

X(Er

(E,

r)

E,

r ) , 50

r ) , E, r ] , t h e p e n a l t y f u n c t i o n o p t i m a l v a l u e , 179

= W[X(E,

VEX(€, r)

=

r),

-v~w[?E(E,

E,

r1 -1v E2 X w [ y ( E , r ) ,

E r

r1,

a formula f o r t h e f i r s t d e r i v a t i v e s o f a l o c a l minimum o f W(x,

E,

r ) , 164

%I

SM

the point-to-set

sM

*

t h e mapping SM when SM i s a s i n g l e t o n , 1 6

sM

S6

map d e f i n e d by S M ( g ) , 1 6

E R ( E ) I f ( x ) 5 f * ( E ) + 6 , 6 1. 01, t h e s e t o f 6-optimal s o l u t i o n s o f problem P 1 ( € ) , 20

(€1

= {x

t h e s e t o f o p t i m a l Lagrange m u l t i p l i e r s a s s o c i -

K(x, 0)

a t e d with a solution point x, 27

s-t.

P ( a ) : min ( x , E ) f ( x r € 1 h(x,

E)

= 0,

E

g(x,

E)

L 0,

= a

a r h s p e r t u r b a t i o n problem e q u i v a l e n t t o t h e g e n e r a l p a r a m e t r i c problem P 3 ( ~ ) ,6 0

EXAMPLES OF THE CHAIN RULE FOR D I F F E R E N T I A T I O N

Assuming t h e i n v o l v e d q u a n t i t i e s and d e r i v a t i v e s e x i s t :

320

Appendix I

A B BRE V I A T I 0 NS

GP

g e o m e t r i c programming, 37

KKT

Karush-Kuhn-Tucker,

lsc

lower s e m i c o n t i n u o u s ( u s e d f o r r e a l - v a l u e d f u n c t i o n s only), 1 4

NLP

n o n l i n e a r programming, 3 0 1

rhs

right-hand s i d e , 302

s.t.

s u b j e c t t o , 295

USC

upper semicontinuous (used f o r r e a l - v a l u e d f u n c t i o n s only), 12

303

Appendix II

Lemmas, Theorems, Corollaries, Definitions, and Examples

Example 2 . 1 . 1

[ f * ( E ) and ~ ( € 1 1 , 9

Example 2 . 1 . 2

[ f * ( E ) a n d x ( E ) ] , 10

Definition 2.2.1

[lower a n d u p p e r s e m i c o n t i n u o u s p o i n t - t o - s e t map], 1 2

D e f i n i t i o n 2.2.2

[open a n d closed p o i n t - t o - s e t

Theorem 2 . 2 . 1

[ f * u s c (1sc) f o r ~ ( € 1 : min f ( x , € 1 s . t .

Theorem 2.2.2

x

E

R(E).

[f*

E

C and S ( E ) u p p e r s e m i c o n t i n u o u s f o r

P(E).

Theorem 2.2.3

B e r g e (19631.1,

E

R(E)

(1967). I ,

.

E

{x

E

(1967) . I ,

D a n t z i g , Folkman, and S h a p i r o 16

M l g ( x ) 2 0).

Dantzig e t aZ.

16

[Solution set continuous f o r P ( g ) .

e t aZ. Theorem 2 . 2 . 6

14

[ S o l u t i o n s e t c l o s e d f o r P ( g ) : min f ( x ) s . t . x

Theorem 2.2.5

Berge (19631.1, 14

[ x ( E ) i s c o n t i n u o u s f o r P ( E ) : min f ( x ) s . t . x

Theorem 2 . 2 . 4

map], 1 2

[S

Dantzig

(1967).1, 1 7

i s c o n t i n u o u s and convex v a l u e d f o r P ( E ) :

min f ( x ,

E)

s.t. x

E

R(E) r l X,

E

E

T.

Robinson a n d Day (19741.1, 1 8 Theorem 2.2.7

[ E q u i v a l e n t c o n d i t i o n s f o r u p p e r a n d lower s e m i c o n t i n u i t y of R ( E ) , f o r P 1 ( € ) . Evans a n d Gould ( 1 9 7 0 ) . ] , 18

32 I

322

Appendix I1

*

Theorem 2.2.8

[Conditions for f lsc or USC, for P1(&). Evans and Gould (1970).I , 19

Theorem 2.2.9

[Conditions for f , R ( E ) continuous, for P1(&). Evans and Gould (1970). I , 19

Theorem 2.2.10

[f continuous for P(E): min f(x) s.t. g(x) E , x E X. Hogan (1973c).], 20

Theorem 2.2.11

[f continuous for P(E): min f(x, E ) s.t. g(x, E ) 1. 0, x E X. Hogan (1973c).l, 20

Definition 2.2.3

[Lipschitz continuous function], 21

Definition 2.2.4

[Uniformly linearly continuous point-to-set

*

*

*.

map1

21

Theorem 2.2.12

[f”( E ) Lipschitz for P1(~). Stern and Topkis (1976).], 21

Theorem 2.2.13

[Continuity of &-optimal solutions of P1(&). Stern and Topkis (1976). I , 22

Definition 2.3.1

[Pseudoconvex function. I , 24

CQ1

[Mangasarian-Fromovitz Constraint Qualification], 23

CQ2

[Slater Constraint Qualification], 24

CQ3

[Linear Independence], 24

CQ4

[Cottle Constraint Qualification], 24

CQ 5

[Kuhn-Tucker Constraint Qualification] , 24

Definition 2.3.2

[Directional derivative], 25

Theorem 2.3.1

[Directional derivative DZf of f ( E ) for P(E): min f(x, E ) s.t. x E R. Danskin (1966, 19671.1, 25

Theorem 2.3.2

[Existence of DZf*(E) for P(E): min f(Xr E) s.t. g(x, E ) 0 , x E X, P(E) convex in x. Hogan (197323).I , 26

Theorem 2.3.3

[f (E) E C with CQ1 for P2(~). Gauvin and Tolle (1977).], 28

*

*

*

323

Appendix 11

Theorem 2.3.4

[Optimal v a l u e D i n i d e r i v a t i v e lower bound for P3(€).

Gauvin and Dubeau (1979) and

F i a c c o ( 1 9 8 0 ~ )I ., 2 9 C o r o l l a r y 2.3.5

[Weaker optimal v a l u e D i n i d e r i v a t i v e l o w e r Gauvin and Dubeau ( 1 9 7 9 )

bound f o r P 3 ( € ) .

a n d F i a c c o ( 1 9 8 0 ~. )I , Theorem 2 . 3 . 6

[ O p t i m a l v a l u e Din1 d e r i v a t i v e u p p e r bound for P3(€).

Gauvin a n d Dubeau (1979) and

F i a c c o ( 1 9 8 0 ~. )I C o r o l l a r y 2.3.7

29

,

30

[DZf* f o r P 3 ( € ) w i t h CQ3.

Gauvin and Dubeau

(1979) a n d F i a c c o ( 1 9 8 0 ~. )I ,

C o r o l l a r y 2.3.8

*

[DZf

f o r P 3 ( € ) convex i n x.

30 F i a c c o and

H u t z l e r (1979a) and F i a c c o (1980b) . I , Theorem 2.3.9

[ C o n d i t i o n s s u c h t h a t Pi(€) i s c a l m .

31

Clarke

(19761.1, 35 Theorem 2 . 4 . 1

[ I m p l i c i t F u n c t i o n Theorem f o r Ck F u n c t i o n . Bochner a n d M a r t i n ( 1 9 4 8 ) . I ,

Theorem 2 . 4 . 2

[ I m p l i c i t F u n c t i o n Theorem f o r A n a l y t i c Function.

Theorem 2.4.3

Bochner a n d M a r t i n ( 1 9 4 8 ) . I

,

36

[ S o l u t i o n bound f o r a p o s i t i v e d e f i n i t e q u a d r a t i c program.

Theorem 2 . 4 . 4

36

Daniel (1973). I

[ B a s i c S e n s i t i v i t y Theorem:

, 41

Sufficient

c o n d i t i o n s f o r a l o c a l l y u n i q u e C1 KKT t r i p l e of P 3 ( € ) . Theorem 2.4.5

Fiacco (1976). I

,

42

[ S u f f i c i e n t c o n d i t i o n s for local uniqueness, c o n t i n u i t y and d i r e c t i o n a l d i f f e r e n t i a b i l i t y o f a KKT t r i p l e and d i f f e r e n t i a b i l -

*

for P 3 ( € ) , without strict complementary s l a c k n e s s . J i t t o r n t r u m i t y of f

(E)

(1978, 1 9 8 1 ) . 1 , 43 Definition 2.4.1

[ C y r t o h e d r o n of class C k (19771.1,

D e f i n i t i o n 2.4.2

.

Spingarn

46

[Strong second-order

conditions for P i ( € ) .

Spingarn (1977). I , 47

324 Theorem 2 . 4 . 6

Appendix 11

[ S u f f i c i e n t c o n d i t i o n s f o r a l o c a l l y unique 1 C K K T t r i p l e of P i ( € ) . S p i n g a r n (1977) . I ,

Theorem 2 . 4 . 7

48

[Local u n i q u e n e s s a n d c o n t i n u i t y o f a KKT

t r i p l e f o r P3 ( E ) . Theorem 2.4.8

Robinson ( 1 9 7 4 ) . 1 ,

[Bounds o n a KKT t r i p l e .

48

Robinson ( 1 9 7 4 ) . ] ,

49

Theorem 2 . 4 . 9

[ L o c a l l y u n i q u e C1-KKT

t r i p l e associated

with a l o c a l l y unique unconstrained l o c a l minimum o f t h e b a r r i e r - p e n a l t y f u n c t i o n W(x, Theorem 2.4.10

E,

Fiacco (1976) . I ,

r).

50

[ L o c a l l y u n i q u e a p p r o x i m a t i o n of a CP KKT t r i p l e for C P + l problem f u n c t i o n s , a s s o c i a t e d w i t h P 3 ( € ) a n d W(x,

E,

r).

Fiacco

(19761.1, 51 Theorem 2.4.11

[Locally unique

c1

KKT t r i p l e of P ( € 1 u s i n g

a n augmented L a g r a n g i a n .

3 Buys a n d Gonin

( 1 9 7 7 ) a n d A r m a c o s t a n d F i a c c o (1977) . I , 52

Theorem 2 . 4 . 1 2

[Formulas f o r t h e o p t i m a l v a l u e f P 3 ( € ) , its gradient V f

*

E

*

(E)

of

2 * a n d H e s s i a n VEf

Armacost a n d F i a c c o ( 1 9 7 5 ) . 1, 52 Theorem 2.4.13

[Lagrange m u l t i p l i e r s e n s i t i v i t y r e s u l t s ,

for P2 ( E ) 53 Lemma 3 . 2 . 1

.

.

A r m a c o s t and F i a c c o (1975) . I ,

[Second-order s u f f i c i e n t c o n d i t i o n s f o r a

s t r i c t l o c a l m i n i m i z i n g p o i n t of p r o b l e m P3 ( 0 ) . F i a c c o a n d McCormick ( i 9 6 8 ) . I , 6 9 Theorem 3.2.2

[ B a s i c S e n s i t i v i t y Theorem, a s l i g h t l y

s t r o n g e r v e r s i o n of Theorem 2 . 4 . 4 .

Easy

e x t e n s i o n of main r e s u l t i n F i a c c o (1976) . I ,

C o r o l l a r y 3.2.3

[Sensitivity analysis for the unconstrained problem:

(1976). I ,

C o r o l l a r y 3.2.4

72-76 minx f ( x ,

E).

[ F i r s t - o r d e r e s t i m a t i o n of near

E =

Based on F i a c c o

76-77 0.

[ x ( E ) , ~ ( € 1 ,w ( E ) 1

F i a c c o (1976) . I ,

78

Appendix I1

325

Corollary 3.2.5

[Existence of higher-order KKT-triple derivatives. Fiacco (1976). 1 , 79

Example 3.3.1

[One variable, calculation of solution derivative] , 80

Example 3.3.2

[Two variables, calculation of solution derivatives], 8 0

Theorem 3.4.1

[First- and second-order changes in the optimal value function of P3(~): Equivalent to Theorem 2.4.12. Armacost and Fiacco (1975).I , 83-85

Corollary 3.4.2

[First- and second-order changes in the optimal value function of P3(~), with constraints independent of E , following easily from Armacost and Fiacco (19751.1, 85

Corollary 3.4.3

[First- and second-order changes in the unconstrained optimal value function. New result, following easily from Armacost and Fiacco (1975).I , 85-86

Corollary 3.4.4

[Optimal value function derivatives for rhs perturbations. Equivalent to Theorem 2.4.13. Armacost and Fiacco (19751.1, 87-88

Corollary 3.4.5

[Optimal value derivatives for perturbations that are linear in the parameters. Armacost and Fiacco (1975).], 89-90 2 *1, 106-112 [Calculation of VEf* and VEf

Example 4.3.1 Example 4.3.2

[Calculation of VET, VEf*, and Vzf* for a parametric LP], 113-114

Theorem 5.4.1

[First-order changes in a KKT triple and first- and second-order changes in the optimal value function for the problem with rhs perturbations. Fiacco (1976) and Armacost and Fiacco (1975).], 126-127

3 26

Appendix 11

Corollary 5.5.1

[Positive (semi) definiteness of the Hessian V,2 f * of the optimal value function. Armacost and Fiacco (1976).I

,

130

Corollary 5.5.2

: L . [Relationship between A22 ( E ) and V Armacost and Fiacco (1976).I , 133

Corollary 5.5.3

[Local convexity of the optimal value func* tion f ( E ) of problem P2(~). Armacost and Fiacco (1976). I , 134

Example 5.6.1

[The Shell Dual: first- and second-order * estimates of f ( E ) ] , 137-141

Example 5.6.2

[Direct solution vs solution by decomposition using sensitivity results], 144-151

Theorem 6.2.1

[Approximation of first-order sensitivity results and determination of estimates from W(x, E, r). A slightly stronger version of Theorem 2.4.9. Based on Fiacco (1976).I , 157-160

Corollary 6.2.2

[Convergence of penalty function solutions and sensitivity estimates. Fiacco (1976). I , 161-162

Corollary 6.2.3

[Approximation of X(E) and VEx(c) by Z(E, r) and V E y ( ~ ,r). Fiacco (1976). I , 164

Example 3.3.1 continued

,

[Penalty function sensitivity estimates], 168

Example 3.3.2 continued

,

[Penalty function sensitivity estimates], 169

Corollary 6.4.1

[Existence and convergence of higher-order derivatives of y ( E , r) , an extension of Theorem 2.4.10. Fiacco (1976).I , 172-173

Theorem 6.6.1

[First- and second-order changes in the * penalty function optimal value W ( E , r). Armacost and Fiacco (1975).I

Corollary 6.6.2

,

180

[Convergence of the penalty function optimal value, gradient, and Hessian to the respective quantities associated with the optimal value function. Armacost and Fiacco (1975). 1 , 181

327

Appendix II

Example 3.3.2, continued

[Penalty function estimates of optimal value

Theorem 7.4.1

[Sensitivity results using an augmented Sharpened Lagrangian for problem P ( E ) version of Theorem 2.4.11. Armacost and Fiacco (1977). J , 211-213

Example 8.3.1

[Illustration of sensitivity calculations by SENSUMT], 229

and its derivatives], 182-185

.

This Page Intentionally Left Blank

References

ALLEN, R. G. D. (1938). M a t h e m a t i c a l A n a l y s i s f o r E c o n o m i s t s . Macmillan, New York. ALLGOWER, E. , AND GEORG, K. (isso). "Simplicia1 and Continuation Methods for Approximating Fixed Points and Solutions to Systems of Equations." SIAM R e v . 22, 28-85. ARCHETTI, F. , BALLABIO, D. , and VERCELLIS, C. (1980). "CostBenefit Analysis of Insulation of Buildings Via Nonlinear Optimization." In N u m e r i c a l O p t i m i z a t i o n of Dynamic S y s t e m s (L. C. W. Dixon and G. P. Szegs, eds.), NorthHolland, Amsterdam, pp. 363-375. ARMACOST, R. L. (1976a). "Computational Experience with Optimal Value Function and Lagrange Multiplier Sensitivity in NLP." Technical Paper T-335, Institute for Management Science and Engineering, George Washington Univ., Washington, D.C. ARMACOST, R. L. (1976b). "Sensitivity Analysis in Parametric Nonlinear Programming." Ph.D. dissertation, School of Engineering and Applied Science, George Washington Univ., Washington, D.C. ARMACOST, R. L., and FIACCO, A. V. (1974). "Computational Experience in Sensitivity Analysis for Nonlinear Programming," Math. Programming 6 (3), 301-326. ARMACOST, R. L., and FIACCO, A. V. (1975). "Second-Order Parametric Sensitivity Analysis in NLP and Estimates by Penalty Function Methods." Technical Paper T-324, Institute for Management Science and Engineering, George Washington Univ., Washington, D.C. ARMACOST, R. L., and FIACCO, A. V. (1976). "NLP Sensitivity Analysis for RHS Perturbations: A Brief Survey and Second Order Extensions." Technical Paper T-334, Institute for Management Science and Engineering, George Washington Univ., Washington, D.C.

329

330

References

ARMACOST, R. L . , and FIACCO, A. V. ( 1 9 7 7 ) . "Exact Sensitivity A n a l y s i s U s i n g Augmented Lagrangians." T e c h n i c a l Paper T - 3 4 9 , I n s t i t u t e f o r M a n a g e m e n t Science and E n g i n e e r i n g , G e o r g e Washington U n i v . , Washington, D.C. ARMACOST, R. L . , and F I A C C O , A. V. ( 1 9 7 8 ) . "Sensitivity A n a l y s i s f o r P a r a m e t r i c N o n l i n e a r Programming U s i n g P e n a l t y Methods." I n C o m p u t e r s and MathematicaZ Programming, N a t i o n a l B u r e a u of S t a n d a r d s Special P u b l i c a t i o n 5 0 2 , 261-269. R. L . , and MYLANDER, W. C. ( 1 9 7 3 ) . "A G u i d e t o t h e V e r s i o n 4 C o m p u t e r Subroutine for I m p l e m e n t i n g Sensitivity A n a l y s i s i n N o n l i n e a r Programming." Technical Paper T - 2 8 7 , I n s t i t u t e f o r M a n a g e m e n t S c i e n c e and E n g i n e e r i n g , G e o r g e Washington U n i v . , Washington, D.C.

ARMACOST, SUMT

-

"A G e n e r a l ARROW, K. J . , GOULD, F. J., and HOWE, S. M. ( 1 9 7 3 ) . Saddle P o i n t R e s u l t f o r C o n s t r a i n e d O p t i m i z a t i o n . " Math. Programming 5 ( 2 ) , 2 2 5 - 2 3 4 . K. J . , a n d HURWICZ, L. ( 1 9 5 6 ) . " R e d u c t i o n of C o n s t r a i n e d M a x i m a t o Saddle P o i n t P r o b l e m s . " I n Proceedings o f t h e T h i r d B e r k e l e y Symposium o n MathematicaZ S t a t i s t i c s and P r o b a b i l i t y (J. N e y m a n , e d . ) , U n i v . C a l i f o r n i a ,

ARROW,

Berkeley,

Calif.

"Constraint ARROW, K. J . , HURWICZ, L., and UZAWA, H. ( 1 9 6 1 ) . Q u a l i f i c a t i o n s i n Maximization Problems." Naval R e s . L o g i s t i c s Q . 8 ( 2 ) , 175-190. AUSLENDER, A. ( 1 9 7 9 ) . " D i f f e r e n t i a b l e Stability i n Nonconvex and N o n d i f f e r e n t i a b l e P r o g r a m m i n g . " I n MathematicaZ Programming S t u d i e s , V o l . 1 0 ( P . H u a r d , e d . ) , N o r t h - H o l l a n d , A m s t e r d a m , pp. 2 9 - 4 1 . AVRIEL, M., DEMBO, R . , and PASSY, U. ( 1 9 7 5 ) . G e n e r a l i z e d G e o m e t r i c P r o g r a m s . I' I n t . J . Eng. 9 , 1 4 9 - 1 6 8 .

" S o l u t i o n of NurnericaZ Meth.

BALDER, E. J . ( 1 9 7 7 ) . " A n E x t e n s i o n of D u a l i t y - S t a b i l i t y R e l a t i o n s to Nonconvex O p t i m i z a t i o n Problems. SIAM J . ControZ O p t i m i z . 1 5 ( 2 ) , 3 2 9 - 3 4 3 . BANK, B . , GUDDAT, J . , KLATTE, D., KUMMER, B., and TAMMER, K. NichtZineare Parametrishe Optimierung. (1982). S e m i n a r b e r i c h t N r . 31, H u m b o l d t U n i v e r s i t y , B e r l i n , S e k t i o n M a t h e m a t i k , 1981. T o be p u b l i s h e d a s N o n l i n e a r Parametric Optimization, Akademie-Verlag, B e r l i n . BAZARAA, M. S . , GOODE, J. J . , and SHETTY, C. M. ( 1 9 7 2 ) . " C o n s t r a i n t Q u a l i f i c a t i o n s R e v i s i t e d . 'I Management S c i . 2 8 ( l o ) , 567-573. BAZARAA, M. S . , and SHETTY, C. M. ming: Theory and A l g o r i t h m s .

(1979).

N o n l i n e a r ProgramWiley, New Y o r k .

BEIGHTLER, C. S., and P H I L L I P S , D. T . ( 1 9 7 6 ) . m e t r i c Programming. Wiley, New Y o r k .

A p p l i e d Geo-

331

References

BENSON, H. P. ( 1 9 8 0 ) . " A l g o r i t h m s f o r P a r a m e t r i c Nonconvex Programming." D i s c u s s i o n P a p e r #20, C e n t e r f o r Econometric a n d D e c i s i o n S c i e n c e , Univ. F l o r i d a . (1976). "The C o n t i n u i t y o f t h e Optimum i n P a r a metric Programming a n d A p p l i c a t i o n s t o S t o c h a s t i c Programming." J . O p t i m i z . Theory A p p l . 1 8 ( 3 1 , 319-333.

BEREANU, B.

BERGE, C. ( 1 9 6 3 ) . T o p o l o g i c a l Spaces M a c m i l l a n , New York.

( t r a n s . E. M.

Patterson).

.

BERTSEKAS, D . P. ( 1 9 7 5 ) "Combined P r i m a l - D u a l a n d P e n a l t y SIAM J . C o n t r o l Methods f o r C o n s t r a i n e d M i n i m i z a t i o n . 1 3 ( 3 ) , 521-544. BIGELOW, J. H . , a n d SHAPIRO, N . 2. ( 1 9 7 4 ) . "Implicit Function Theorems f o r M a t h e m a t i c a l Programming a n d f o r S y s t e m s o f Inequalities." Math. Programming 6 ( 2 1 , 141-156. BOCHNER, S . , a n d MARTIN, W. ( 1 9 4 8 ) . S e v e r a l Complex V a r i a b l e s . P r i n c e t o n Univ. P r e s s , P r i n c e t o n , N. J.

BOOT, J. C. G. ( 1 9 6 3 ) . "On S e n s i t i v i t y A n a l y s i s i n Convex Q u a d r a t i c Programming P r o b l e m s . " O p e r a t i o n s R e s . I 1 (51, 7 71- 786.

BRADLEY, S. P . ,

HAX,

A.

C.,

a n d MAGNANTI, T. L. ( 1 9 7 7 ) . Addison-Wesley, R e a d i n g ,

A p p l i e d Mathematical Programming.

Mass.

.

BROSOWSKI , B. ( 1 9 8 2 ) P a r a m e t r i c Semi- I n f i n
CANON, M.

C. ( 1 9 3 5 ) . C a l c u l u s of V a r i a t i o n s a n d P a r t i a l D i f f e r e n t i a l E q u a t i o n s of t h e F i r s t Order. B e r l i n .

CARATHEODORY,

T r a n s l a t e d ( 1 9 6 5 ) , Holden-Day,

San F r a n c i s c o .

"The I m p l i c i t F u n c t i o n Theorem i n FuncCESARI, L. ( 1 9 6 6 ) . t i o n a l Analysis." Duke Math. J . 33, 417-440. a n d COOPER, W. W. ( 1 9 6 1 ) . Management Models and I n d u s t r i a 1 A p p Z i ca t i o n s of L i n e a r Programming W iley,

CHARNES, A . ,

New York.

.

332

References

CHARNES, A,, and GEMMELL, R. E. (1964). "A Method of Solution of Some Nonlinear Problems in Abatement of Stream Pollution." Systems Research Memorandum No. 103, The Technological Institute, Northwestern Univ., Evanston, 111. CLARKE, F. H. (1975). "Generalized Gradients and Applications." T r a n s . A m . Math. S o c . 2 0 5 , 247-262. CLARKE, F. H. (1976). "A New Approach to Lagrange Multipliers." Math. O p e r a t i o n s R e s . 1(2), 165-174. COLVILLE, A. R. (1968). "A Comparative Study of Nonlinear Programming Codes. Technical Report 320-2949, IBM New York Scientific Center, New York. DANIEL, J. W. (1973). "Stability of the Solution of Definite Quadratic Programs." Math. Programming 5 (11, 41-53. DANSKIN, J. M. (1966). "The Theory of Max-Min with Applications." SIAM J . A p p Z . Math. 1 4 (41, 651-665. DANSKIN, J. M. (1967). T h e T h e o r y of Max-Min. Verlag, New York.

Springer-

DANTZIG, G. B. (1963). L i n e a r Programming a n d E x t e n s i o n s . Princeton Univ. Press, Princeton, N. J. DANTZIG, G. B., FOLKMAN, J., and SHAPIRO, N. 2 . (1967). "On the Continuity of the Minimum Set of a Continuous Function." J . Math. A n a l . A p p Z . 17(3), 519-548. DAVIDENKO, D. (1953). "On a New Method of Numerically Integrating a System of Nonlinear Equations." D o k l . A k a d . Nauk. S S S R 8 8 , 601-604 (in Russian). DEMBO, R. S. (1982). "Sensitivity Analysis in Geometric Programming." J . O p t i m i z . T h e o r y A p p l . 37(1), 1-21. DEM'YANOV, V. F., and PEVNYI, A. B. (1972). "First and Second Marginal Values of Mathematical Programming Problems." S O V . Math. D o k l . 13 (6), 1502-1506. DEM'YANOV, V. F., and RUBINOV, A. M. (1968). "Minimization of Functionals in Normed Spaces." SIAM J . C o n t r o l 6 (1), 73-88. de SILVA, A. H. (1978). "Sensitivity Formulas for Nonlinear Factorable Programming and their Application to the Solution of an Implicitly Defined Optimization Model of U. S. Crude Oil Production." Ph.D. dissertation, School of Engineering and Applied Science, George Washington Univ., Washington, D.C. DINKEL, J. J., and KOCHENBERGER, G. A. (1977). "On Sensitivity Analysis in Geometric Programming." O p e r a t i o n s R e s . 2 5 (1), 155-163.

333

References

DINKEL, J. J., and KOCHENBERGER, G. A. (1978). "An Implementation of Some Implicit Function Theorems with Applications to Sensitivity Analysis." Math. Programming 15 (3), 261-267. DINKEL, J. J., KOCHENBERGER, G. A., and WONG, S. N. (1978). "Sensitivity Analysis Procedures for Geometric Programs: Computational Aspects." ACM T r a n s . Math. S o f t w a r e 4 (11, 1-14. DINKEL, J. J., KOCHENBERGER, G. A., and WONG, S. N. (1982). "Parametric Analysis in Geometric Programming: An Incremental Approach." In M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s , Vol. 73, Marcel Dekker, New York, 93-109. DINKELBACH, W. (1969). S e n s i t i v i t y A n a l y s i s and P a r a m e t r i c Programming. Springer-Verlag, Berlin (in German). DOLECKI, S., and KURCYUSZ, S. (1978). "On @-Convexity in Extremal Problems." SIAM J . C o n t r o l O p t i m i z . 16(2), 277-300. DUFFIN, R. J., PETERSON, E. L., and ZENER, C. (1967). m e t r i c Programming. Wiley, New York.

Geo-

ECKER, J. G. (1975). "A Geometric Programming Model for Optimal A'llocation of Stream Dissolved Oxygen." Management S c i . 21(6), 658-668. ECKER, J. G., and WIEBKING, R. D. (1976). "Optimal Selection of Steam Turbine Exhaust Annulus and Condenser Sizes by Geometric Programming." Eng. O p t i m i z . 2 , 173-181. EVANS, J. P., and GOULD, F. J. (1970). "Stability in Nonlinear Programming. O p e r a t i o n s R e s . 18 (11, 107-118. EVERETT, H. (1963). "Generalized Lagrange Multiplier Method for Solving Problems of Optimum Allocation of Resources.' O p e r a t i o n s R e s . 1 3 (31, 399-417. FALK, J. E., and SOLAND, R. M. (1969). "An Algorithm for Separable Nonconvex Programming Problems." Management S c i . 15(9), 550-569. FENCHEL, W. (1949). Math. I , 73-77.

"On Conjugate Convex Functions. 'I

Can. J .

FIACCO, A. V. (1974). "Convergence Properties of Local Solutions of Sequences of Mathematical Programming Problems in General Spaces." J . O p t i m i z . T h e o r y A p p l . 13(1), 1-12. FIACCO, A. V. (1976). "Sensitivity Analysis for Nonlinear Math. Programming Programming Using Penalty Methods. 10 (31, 287-311.

334

References

FIACCO, A. V. ( 1 9 8 0 a ) . " N o n l i n e a r Programming S e n s i t i v i t y A n a l y s i s R e s u l t s Using S t r o n g Second O r d e r A s s u m p t i o n s . " I n Numerical O p t i m i z a t i o n of Dynamic S y s t e m s (L. C . W. Dixon and G. P . S z e g 6 , e d s . ) , North-Holland, Amsterdam, pp. 327-348. FIACCO, A. V. ( 1 9 8 0 b ) . " C o n t i n u i t y o f t h e O p t i m a l V a l u e F u n c t i o n u n d e r t h e Mangasarian-Fromovitz C o n s t r a i n t Q u a l i f i c a t i o n . " T e c h n i c a l P a p e r T-432 , I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C. FIACCO, A . V. ( 1 9 8 0 ~ ) . " O p t i m a l V a l u e C o n t i n u i t y and D i f f e r e n t i a l S t a b i l i t y Bounds u n d e r t h e Mangasarian-Fromovitz Constraint Qualification." T e c h n i c a l P a p e r T-435, I n s t i t u t e f o r Management S c i e n c e and E n g i n e e r i n g , George Washington Univ., Washington, D.C. FIACCO, A. V. ( 1 9 8 1 ) . "Computable O p t i m a l V a l u e Bounds a n d S o l u t i o n V e c t o r E s t i m a t e s f o r G e n e r a l Parametric NLP Programs. 'I T e c h n i c a l P a p e r T-451, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D. C.

*

FIACCO, A. V., and GHAEMI, A. ( 1 9 7 9 a ) . "Optimal T r e a t m e n t L e v e l s o f a S t r e a m P o l l u t i o n Abatement System u n d e r T h r e e E n v i r o n m e n t a l C o n t r o l P o l i c i e s , P a r t I: S o l u t i o n and A n a l y s i s o f Convex E q u i v a l e n t s o f E c k e r ' s GP Models Using SUMT." T e c h n i c a l P a p e r T-387 , I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C.

*

V . , and GHAEMI, A. (197913). "Optimal T r e a t m e n t Levels o f a S t r e a m P o l l u t i o n Abatement System u n d e r T h r e e E n v i r o n m e n t a l C o n t r o l P o l i c i e s , P a r t 11: P r e l i m i n a r y S e n s i t i v i t y A n a l y s i s o f a Convex E q u i v a l e n t of t h e F i x e d D i s s o l v e d Oxygen Requirement P o l i c y GP Model Using SENSUMT." T e c h n i c a l P a p e r T-405, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C.

FIACCO, A.

FIACCO, A. V., a n d GHAEMI, A. ( 1 9 8 0 ) . "A User's Manual f o r SENSUMT: A P e n a l t y F u n c t i o n Computer Program f o r S o l u t i o n , S e n s i t i v i t y A n a l y s i s a n d O p t i m a l V a l u e Bound C a l c u l a t i o n i n P a r a m e t r i c N o n l i n e a r Programs." T e c h n i c a l P a p e r T-434, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C. FIACCO, A. V . , a n d GHAEMI, A. ( 1 9 8 1 a ) . A C l o s e d Form Local S o l u t i o n o f a N o n l i n e a r S t r u c t u r a l D e s i g n Problem i n T e r m s o f Design Parameters." T e c h n i c a l P a p e r T -4 4 9 , I n s t i t u t e f o r Management S c i e n c e and E n g i n e e r i n g , George Washington Univ., Washington, D.C.

*

See S e l e c t e d B i b l i o g r a p h y for p u b l i c a t i o n s by t h e same a u t h o r s based o n t h i s t e c h n i c a l r e p o r t .

335

References

FIACCO, A. V . , a n d GHAEMI, A. ( 1 9 8 1 b ) . " S e n s i t i v i t y a n d P a r a m e t r i c Bound A n a l y s i s of O p t i m a l Steam T u r b i n e E x h a u s t Annulus and Condenser S i z e s . " T e c h n i c a l P a p e r T-437, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C. FIACCO, A. V . , and GHAEMI, A. ( 1 9 8 2 ) . " S e n s i t i v i t y Analysis o f a N o n l i n e a r S t r u c t u r a l D e s i g n Problem. I' Computers O p e r a t i o n s Res. 9 ( 1 ) , 29-55.

*

"Extensions of t h e FIACCO, A. V., and HUTZLER, W. P. ( 1 9 7 9 a ) . Gauvin-Tolle Optimal Value D i f f e r e n t i a l S t a b i l i t y R e s u l t s t o G e n e r a l M a t h e m a t i c a l Programs." T e c h n i c a l P a p e r T-393, I n s t i t u t e f o r Management S c i e n c e and E n g i n e e r i n g , George Washington Univ., Washington, D.C.

*

FIACCO, A. V., a n d HUTZLER, W. P. ( 1 9 7 9 b ) . "Basic R e s u l t s i n t h e Development o f S e n s i t i v i t y a n d S t a b i l i t y A n a l y s i s i n C o n s t r a i n e d M a t h e m a t i c a l Programming." Technical Paper T-407, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washinton Univ., Washington, D.C. FIACCO, A. V., a n d KYPARISIS, J . ( 1 9 8 1 a ) . "Computable Param e t r i c Bounds f o r S i m u l t a n e o u s L a r g e P e r t u r b a t i o n s o f T h i r t y P a r a m e t e r s i n a Water P o l l u t i o n Abatement GP Model, P a r t I: O p t i m a l V a l u e Bounds." T e c h n i c a l P a p e r T-453, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C. FIACCO, A.

V.,

and KYPARISIS, J .

(1981b).

"Computable Para-

metric Bounds f o r S i m u l t a n e o u s L a r g e P e r t u r b a t i o n s o f T h i r t y P a r a m e t e r s i n a Water P o l l u t i o n Abatement GP Model,

R e f i n e m e n t o f O p t i m a l V a l u e Bounds a n d ParaT e c h n i c a l P a p e r T-461, I n s t i t u t e f o r Management S c i e n c e a n d E n g i n e e r i n g , George Washington Univ., Washington, D.C. P a r t 11:

m e t r i c S o l u t i o n Bounds."

FIACCO, A.

V., a n d McCORMICK, G. P. ( 1 9 6 8 ) . N o n l i n e a r Programming: S e q u e n t i a Z U n c o n s t r a i n e d M i n i m i z a t i o n Techn i q u e s . W i l e y , New York.

FONTANIE, G. ( 1 9 8 0 ) . "Locally Lipschitz Functions d i f f e r e n t i a b l e Programming." Master's t h e s i s , R e p o r t 80-3, C u r r i c u l u m i n O p e r a t i o n s R e s e a r c h A n a l y s i s , U n i v e r s i t y o f North C a r o l i n a , Chapel

a n d NonTechnical and Systems H i l l , N.C.

GAL, T.

( 1 9 7 9 ) . P o s t o p t i m a l A n a l y s i s , P a r a m e t r i c Programming, and R e l a t e d T o p i c s . M c G r a w - H i l l , New York.

GALE, D. ( 1 9 6 0 ) . The Theory of L i n e a r Economic Models. M c G r a w - H i l l , New York. GALE, D. ( 1 9 6 7 ) . "A G e o m e t r i c D u a l i t y Theorem w i t h Economic Applications." Rev. Econ. S t u d . 3 4 , 19-24.

*

See S e l e c t e d B i b l i o g r a p h y f o r p u b l i c a t i o n s by t h e same a u t h o r s based on t h e t e c h n i c a Z r e p o r t s .

336

References

GALE, D . , KUHN, H. W . , a n d TUCKER, A. W. ( 1 9 5 1 ) . " L i n e a r In A c t i v i t y AnaZysis Programming a n d t h e Theory o f G a m e s . " of P r o d u c t i o n and A Z Z o c a t i o n (T. C. Koopmans, e d . ) , W i l e y , New York. GARCIA, C.

Paths.

B . , a n d GOULD, F. J. ( 1 9 7 8 ) . "A Theorem o n Homotopy Math. O p e r a t i o n s R e s . 3 ( 4 ) , 282-289.

B . , and ZANGWILL, W. I . ( 1 9 7 9 ) . " D e t e r m i n i n g A l l S o l u t i o n s t o C e r t a i n Systems o f N o n l i n e a r E q u a t i o n s . " Math. O p e r a t i o n s R e s . 4 ( 1 ) , 1 - 1 4 .

GARCIA, C.

GAUVIN, J. ( 1 9 7 7 ) . "A N e c e s s a r y and S u f f i c i e n t R e g u l a r i t y C o n d i t i o n t o Have Bounded M u l t i p l i e r s i n Nonconvex Programming." Math. Programming 1 2 , 136-138. J. ( 1 9 7 8 ) . "The Method o f Parametric Decomposition i n M a t h e m a t i c a l Programming: The Nonconvex C a s e . " I n Nons m o o t h O p t i m i z a t i o n (C. Lemarechal and R. M i f f l i n , e d s . ) , Pergamon, O x f o r d , pp. 131-149.

GAWIN,

J. (1979). "The G e n e r a l i z e d G r a d i e n t of t h e M a r g i n a l Math. O p e r a t i o n s F u n c t i o n i n M a t h e m a t i c a l Programming." Res. 4 ( 4 ) , 458-463.

GAUVIN,

GAUVIN,

J . , a n d DUBEAU,

F.

(1979).

"Differential Properties

of t h e M a r g i n a l F u n c t i o n i n M a t h e m a t i c a l Programming."

To be p u b l i s h e d P r e p r i n t , Ecole Polytechnique d e Montreal. i n MathematicaZ Programming S t u d i e s (M. G u i g n a r d , e d . ) , N o r t h - H o l l a n d , Amsterdam.

a n d TOLLE, J . W. ( 1 9 7 7 ) . " D i f f e r e n t i a l S t a b i l i t y SIAM J . C o n t r o l O p t i m i z . 1 5 ( 2 1 , i n N o n l i n e a r Programming." 294-311.

GAUVIN, J . ,

.

" P r i m a l R e s o u r c e D i r e c t i v e Approaches GEOFFRION, A. M. (1970) Operaf o r O p t i m i z i n g N o n l i n e a r Decomposable S y s t e m s . " t i o n s R e s . 1 8 ( 3 ) , 375-403. GEOFFRION, A. M. ( 1 9 7 1 ) . " D u a l i t y i n N o n l i n e a r Programming: A S i m p l i f i e d A p p l i c a t i o n O r i e n t e d Development. 'I SIAM Rev. 1 3 ( 1 ) , 1-37.

.

GEOFFRION, A. M. (1977) "Objective Function Approximation i n Math. Programming 1 3 , 23-37. M a t h e m a t i c a l Programming." G H A E M I , A. (1980). "Computable S t a b i l i t y A n a l y s i s T e c h n i q u e s f o r N o n l i n e a r Programming: S e n s i t i v i t i e s , Optimal Value Bounds, a n d A p p l i c a t i o n s . " Ph.D. d i s s e r t a t i o n , S c h o o l o f E n g i n e e r i n g and A p p l i e d S c i e n c e , George Washington Univ., Washington, D.C. GOLDMAN, A . J . , a n d TUCKER, A. W. ( 1 9 5 6 ) . "Theory o f L i n e a r Programming." I n L i n e a r I n e q u a Z i t i e s and R e l a t e d S y s t e m s (J. W. Kuhn a n d A. W. T u c k e r , e d s . ) , A n n a l s o f M a t h e m a t i c s , P r i n c e t o n Univ. P r e s s , P r i n c e t o n , N. J. S t u d y N o . 38.

337

References

GOLLAN, B. (1979). "Sensitivity Results in Optimization with Applications to Optimal Control Problems.'' In D i f f e r e n t i a l Games and C o n t r o l T h e o r y , Vol. 3 (P. T. Liu and E. Roxin, eds. ) , Marcel Dekker, New York. GOLLAN, B. (1981a). "A General Perturbation Theory for Abstract Optimization Problems." J . O p t . T h e o r y A p p l . 3 5 (3), 417-441. GOLLAN, B. (1981b). "On the Marginal Function in Nonlinear Programming." Preprint No. 69, Mathematisches Institut der Universitst am Hubland, Wiirzburg. GOLLAN, B. (1981~). "Higher Order Necessary Conditions for an Abstract Optimization Problem." M a t h e m a t i c a l Programming S t u d i e s , Vol. 14 (H. Konig, B. Korte, and K. Ritter, eds.), North-Holland, Amsterdam, 69-76. GOL'STEIN, E. G. (1972). T h e o r y of Conver Programming. Translations of Mathematical Monographs, Vol. 36, American Mathematical Society, Providence, R.I. GRAVES, S. C., and SHAPIRO, J. F. (1982). "Problem Formulations and Numerical Analysis in Integer Programming and Combinatorial Optimization." In M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s , Vol. 73, Marcel Dekker, New York, 131-148. GREENBERG, H. J., and PIERSKALLA, W. P. (1972). "Extensions of the Evans-Gould Stability Theorems for Mathematical Programs." O p e r a t i o n s R e s . 2 0 (1), 143-153. HAGER, W. W. (1979). "Lipschitz Continuity for Constrained Processes." SIAM J . C o n t r o l O p t i m i z . 17 (3), 321-338. HAN, S.-P. , and MANGASARIAN, 0. L. (1979). "Exact Penalty Math. Programming Functions in Nonlinear Programming. 17(3), 251-269. HANCOCK, H. (1917). Boston.

T h e o r y o f Maxima and Minima.

Ginn,

HESTENES, M. R. (1966). C a l c u l u s of V a r i a t i o n s and O p t i m a l C o n t r o l T h e o r y . Wiley, New York. HESTENES, M. R. (1969). "Multiplier and Gradient Methods." J. O p t i m i z . T h e o r y A p p Z . 4 , 303-320. HESTENES, M. R. (1975). O p t i m i z a t i o n T h e o r y : D i m e n s i o n a l Case. Wiley, New York. HICKS, J. R. (1939). London.

V a l u e and C a p i t a l .

The F i n i t e

Oxford Univ. Press,

HILDEBRANDT, T., and GRAVES, L. (1927). "Implicit Functions and Their Differentials in General Analysis." T r a n s . Am. Math. S O C . 29(1), 127-153.

338

References

HIRIART-URRUTY, J.-B. (1979). "Refinements of Necessary Optimality Conditions in Nondifferentiable Programming I." A p p l . Math. O p t i m i z . 5, 63-82. HOGAN, W. W. (1973a). "Applications of a General Convergence Theory for Outer Approximation Algorithms.'' Math. Programming 5 (2), 151-168.

.

HOGAN, W. W. (197323) "Directional Derivatives for Extremal Value Functions with Applications to the Completely Convex Case." O p e r a t i o n s R e s . 2 1 (1), 188-209. HOGAN, W. W. (1973~). "The Continuity of the Perturbation Function of a Convex Program." O p e r a t i o n s R e s . 2 1 (1), 351-352.

.

"Point-to-Set Maps in Mathematical HOGAN, W. W. (1973d) Programming." SIAM R e v . 15 (3), 591-603. HOLM, S. , and KLEIN, D. (1978). "Discrete Right Hand Side Eur. J . Parametrization for Linear Integer Programs. O p e r a t i o n s R e s . 2 , 50-53. HOLM, S . , and KLEIN, D. (1982). "Three Methods for Postoptimal Analysis in ILP Based Duality Theory." Preprint, Odense University, Odense. IOFFE, A. D., and TIKHOMIROV, V. R. (1980). T h e o r y of E x t r e m a l North-Holland, Amsterdam (Russian original). Problems. JITTORNTRUM, K. (1978). "Sequential Algorithms in Nonlinear Programming." Ph.D. dissertation, Australian National Univ., Canberra.

.

"Solution Point Differentiability JITTORNTRUM, K. (1981) without Strict Complementarity in Nonlinear Programming." To be published in M a t h e m a t i c a l Programming S t u d i e s (A. V. Fiacco, ed.), North-Holland, Amsterdam. JOHN, F. (1948). "Extremum Problems with Inequalities as Side Conditions." In S t u d i e s a n d E s s a y s , C o u r a n t A n n i v e r s a r y Volume (K. 0 . Friedrichs, 0 . E. Neugebauer, and J. J. Stoker, eds.), Wiley, New York. KANTOROVICH, L. (1948). "On Newton's Method for Functional Equations." D o k l . A k a d . Nauk. SSSR 5 9 , 1237-1240 (in Russian). KARUSH, W. (1939). "Minima of Functions of Several Variables with Inequalities as Side Conditions." M.S. thesis, Dept. of Mathematics, Univ. Chicago, Chicago. KAUSMANN, U., LOMMATZSCH, K., and NOiIEKA, F. (1976). L i n e a r Parametric Optimization. Akademie-Verlag, Berlin (in German).

339

References

(1980). "Strongly S t a b l e S t a t i o n a r y Solutions i n N o n l i n e a r Programs." I n A n a l y s i s and C o m p u t a t i o n o f F i x e d P o i n t s (S. M. R o b i n s o n , e d . ) , Academic P r e s s , New York, pp. 93-138.

K O J I M A , M.

, a n d HIRABAYASHI, R. ( 1 9 8 1 ) . " C o n t i n u o u s D e f o r m a t i o n o f N o n l i n e a r Programs." T e c h n i c a l Report B-101, Dept. o f I n f o r m a t i o n S c i e n c e s , Tokyo I n s t i t u t e o f Techn o l o g y , Tokyo.

K O J I M A , M.

KOOPMANS, T. C. ( e d . ) ( 1 9 5 1 a ) . A c t i v i t y A n a l y s i s of Product i o n and A l l o c a t i o n . W i l e y , New York. " A n a l y s i s of P r o d u c t i o n as a n KOOPMANS, T. C. ( 1 9 5 1 b ) . E f f i c i e n t C o m b i n a t i o n of A c t i v i t i e s . " I n A c t i v i t y A n a l y s i s of P r o d u c t i o n and A l l o c a t i o n (T. C. Koopmans, e d . ) , W i l e y , New York. KUHN, H. W. , and TUCKER, A. W. ( 1 9 5 1 ) . " N o n l i n e a r Programming." I n P r o c e e d i n g s of t h e 2nd B e r k e l e y Symposium on Mathemat i c a l S t a t i s t i c s and P r o b a b i l i t y (J. Neyman, e d . ) , Univ. C a l i f o r n i a P r e s s , B e r k e l e y , C a l i f . , pp. 481-493. H. W . , a n d TUCKER, A. W. ( e d s . ) ( 1 9 5 6 ) . L i n e a r I n e q u a l i t i e s and R e l a t e d S y s t e m s , A n n a l s of M a t h e m a t i c s , S t u d y N o . 38. P r i n c e t o n Univ. P r e s s , P r i n c e t o n , N. J .

KUHN,

LAGRANGE,

J. L.

(1881).

O e u v r e s de L a g r a n g e .

Paris.

LASDON, L. S . ( 1 9 7 0 ) . O p t i m i z a t i o n T h e o r y f o r Large S y s t e m s . MacMillan, New York. F. , a n d MAURER, H. ( 1 9 8 0 ) . "Differential Stability i n A p p l . Math. I n f i n i t e - D i m e n s i o n a l N o n l i n e a r Programs." O p t i m i z . 6 , 139-152.

LEMPIO,

S. ( 1 9 7 6 ) . " D i f f e r e n t i a b i l i t y w i t h Respect t o a P a r a m e t e r o f t h e O p t i m a l V a l u e i n Parametric P r o b l e m s o f M a t h e m a t i c a l Programming." C y b e r n e t i c s 1 2 (1), 46-64.

LEVITIN, E.

L E V I T I N , E. S . , a n d POLYAK, B. T. ( 1 9 6 6 ) . "Constrained USSR Computat. Math. and Math. M i n i m i z a t i o n Methods." P h y s . 6 ( 5 ) , 1-50.

LOMMATZSCH, K . ( e d . ) ( 1 9 7 9 ) . A p p l i c a t i o n s of L i n e a r Param e t r i c O p t i m i z a t i o n . Akademie-Verlag, B e r l i n ( i n German). LORENZEN,

G.

cations.

( 1 9 7 4 ) . P a r a m e t r i c O p t i m i z a t i o n and Some A p p l i R. O l d e n b u r g V e r l a g , Miinchen-Vienna ( i n German).

L . J . , a n d McCORMICK, G. P . ( 1 9 7 6 ) . "Bounding G l o b a l Math. O p e r a t i o n s Res. I ( 1 ) , 50-53. Minima."

MANCINI,

M A N C I N I , L. J . , a n d McCORMICK, G. P. ( 1 9 7 9 ) . "Bounding G l o b a l Minima w i t h I n t e r v a l A r i t h m e t i c . " Operations Res. 2 7 ( 4 ) , 743-754.

340

References

MANGASARIAN, 0. L. ( 1 9 6 9 ) . H i l l , New York.

N o n l i n e a r Programming.

McGraw-

"The F r i t z J o h n MANGASARIAN, 0. L . , a n d FROMOVITZ, S . ( 1 9 6 7 ) . Necessary O p t i m a l i t y Conditions i n t h e Presence o f E q u a l i t y J . Math. A n a l . A p p l . 1 7 , and I n e q u a l i t y C o n s t r a i n t s . " 37-47. "Inequalities MANGASARIAN, 0. L . , a n d ROSEN, J . B. ( 1 9 6 4 ) . for S t o c h a s t i c N o n l i n e a r Programming P r o b l e m s . 'I O p e r a t i o n s Res. 1 2 (1), 143-154. MANN, H. B. ( 1 9 4 3 ) . " Q u a d r a t i c Forms w i t h L i n e a r C o n s t r a i n t s . " A m . Math. Mon. 5 0 , 430-433.

MARSHALL, A. ( 1 9 2 0 ) . P r i n c i p l e s of Economics, 8 t h Ed. M a c m i l l a n , London. MARSTEN, R. E., a n d MORIN, T. L. ( 1 9 7 7 ) . " P a r a m e t r i c I n t e g e r Programming: The R i g h t Hand S i d e C a s e . " In Studies i n I n t e g e r Programming ( P . Hammer e t a l . , e d s . ) , E l s e v i e r N o r t h - H o l l a n d , N e w York. MARTIN, D. H. ( 1 9 7 5 ) . "On t h e C o n t i n u i t y o f t h e Maximum i n J . O p t i m i z . Theory A p p l . P a r a m e t r i c L i n e a r Programming." 1 7 ( 3 ) , 205-210.

H. ( 1 9 7 7 a ) . "A S e n s i t i v i t y R e s u l t f o r I n f i n i t e NonMathel i n e a r Programming Problems; P a r t I: Theory." m a t i s c h e s I n s t i t u t der U n i v e r s i t s t Wfirzburg, Wiirzburg.

MAURER,

H. ( 1 9 7 7 b ) . "A S e n s i t i v i t y R e s u l t f o r I n f i n i t e Nonl i n e a r Programming P r o b l e m s ; P a r t 11: A p p l i c a t i o n s t o O p t i m a l C o n t r o l Problems." M a t h e m a t i s c h e s I n s t i t u t der U n i v e r s i t s t Wiirzburg, Wiirzburg.

MAURER,

McCORMICK, G. P . ( 1 9 7 2 ) . " C o m p u t a t i o n a l A s p e c t s o f N o n l i n e a r Programming S o l u t i o n s t o L a r g e S c a l e I n v e n t o r y Problems. T e c h n i c a l Memorandum TM-63488, Program i n L o g i s t i c s , George Washington Univ., Washington, D.C. McCORMICK, G. P. ( 1 9 7 6 ) . " C o m p u t a b i l i t y o f Global S o l u t i o n s P a r t I-Convex Undert o F a c t o r a b l e Nonconvex Programs: Math. Programming 1 0 ( 2 ) , 147-175. e s t i m a t i n g Problems." McCORMICK, G. P . ( 1 9 7 7 ) . "Future D i r e c t i o n s of Mathematical Programming." NATO-AS1 L e c t u r e Notes, E n g i n e e r i n g P l a s t i c i t y by M a t h e m a t i c a l Programming, Univ. Waterloo, Waterloo, Ontario. McCORMICK, G.

" L o c a t i n g a n I s o l a t e d G l o b a l Mini'' Math. Opera5 ( 3 ) , 435-443.

P.

(1980).

m i z e r of a C o n s t r a i n e d Nonconvex Program. t i o n s Res.

McKEOWN, J . J . ( 1 9 8 0 ) . "An Approach t o S e n s i t i v i t y A n a l y s i s . ' ' I n Numerical O p t i m i z a t i o n of Dynamic S y s t e m s (L. C . W. Dixon a n d G. P. S z e g z , e d s . ) , North-Holland, Amsterdam, pp. 349-362.

34 I

References

MEYER, R. R. ( 1 9 7 0 ) . "The V a l i d i t y of a F a m i l y o f O p t i m i z a t i o n Methods." SIAM J . C o n t r o l 8 (1), 41-54. MEYER, R. R. ( 1 9 7 9 ) . "Two-Segment Management Sci. 2 5 , 385-395.

S e p a r a b l e Programming."

.

MEYER, R. R. ( 1 9 8 0 ) " C o m p u t a t i o n a l A s p e c t s o f Two-Segment S e p a r a b l e Programming." Computer S c i e n c e s T e c h n i c a l R e p o r t No. 382, Computer S c i e n c e s D e p t . , Univ. W i s c o n s i n , Madison, W i s . MIEL, G. ( 1 9 8 0 ) . "An Updated V e r s i o n o f t h e K a n t o r o v i c h Theorem f o r N e w t o n ' s Method. " MRC T e c h n i c a l Summary R e p o r t N o . 2125, M a t h e m a t i c s R e s . C e n t e r , Univ. W i s c o n s i n , Madison, W i s .

MILLS, H. D. ( 1 9 5 6 ) . " M a r g i n a l V a l u e s o f Matrix G a m e s a n d L i n e a r Programs. I n L i n e a r I n e q u a Z i t i e s and R e l a t e d S y s t e m s (H. W. Kuhn a n d A. W. T u c k e r , e d s . ) , P r i n c e t o n Univ. P r e s s , P r i n c e t o n , N . J. MINE,

H.

H.,

FUKUSHIMA, M.,

a n d RYANG, Y .

J.

(1977).

"Para-

J.

(1978).

"Para-

metric N o n l i n e a r Programming f o r L i n e a r C o n s t r a i n t s . " Kyung-Pan Ron-Chong 2 ,

MINE,

H.

H.,

FUKUSHIMA, M.,

135-147.

a n d RYANG,

Y.

metric N o n l i n e a r Programming f o r G e n e r a l Cases a n d I t s A p p l i c a t i o n t o Some P r o b l e m s . ' Mem. Fac. Eng. Kyoto U n i v . XL, P a r t 3, 198-211.

MOORE, R. E . ( 1 9 6 6 ) . I n t e r v a l A n a Z y s i s . Englewood C l i f f s , N. J.

Prentice-Hall,

MOORE, R. E. ( 1 9 7 7 ) . "A T e s t f o r E x i s t e n c e of S o l u t i o n s t o N o n l i n e a r Systems." SIAM J . Num. A n a l . 1 4 , 611-615. MOSCO, U. ( 1 9 6 9 ) . " C o n v e r g e n c e o f Convex S e t s - a n d of S o l u Adv. Math 3 ( 4 1 , t i o n s of Variational Inequalities." 510-585. MYLANDER, W. C., HOLMES, R. L., and McCORMICK, G. P. ( 1 9 7 1 ) . "A G u i d e t o SUMT-Version 4 . " RAC-P-63, R e s e a r c h A n a l y s i s C o r p o r a t i o n , McLean, Va. NOZIEKA, F . , GUDDAT, J . , HOLLATZ, H., a n d BANK, B. ( 1 9 7 4 ) . T h e o r y of L i n e a r P a r a m e t r i c Optirniza t i o n . Akademie-Verlag, B e r l i n ( i n German). "A Review of C o n s t r a i n t Q u a l i f i c a t i o n s PETERSON, D. W. ( 1 9 7 3 ) . i n F i n i t e Dimensional Spaces." SIAM R e v . 1 5 , 639-654.

PHIPPS, C . G. ( 1 9 5 2 ) . "Maxima a n d Minima u n d e r R e s t r a i n t . " A m . Math. Mon. 5 9 , 230-235. POWELL, M. J. D. ( 1 9 6 9 ) . "A Method f o r N o n l i n e a r C o n s t r a i n t s i n M i n i m i z a t i o n P r o b l e m s . I' I n O p t i m i z a t i o n (R. F l e t c h e r , ed.) , A c a d e m i c P r e s s , N e w York.

342

References

RALL, L. B. ( 1 9 7 9 ) . "A Comparison o f t h e E x i s t e n c e Theorems of K a n t o r o v i c h and Moore. MRC T e c h n i c a l Summary R e p o r t No. 1 9 4 4 , M a t h e m a t i c s R e s . C e n t e r , Univ. W i s c o n s i n , Madison , W i s

.

W. C . ( 1 9 6 9 ) . " L o c a l Mapping R e l a t i o n s and G l o b a l T r a n s . A m . Math. SOC. 1 3 8 , I m p l i c i t F u n c t i o n Theorems." 1 8 3-19 8 .

RHEINBOLDT,

ROBINSON, S. M. ( 1 9 7 2 ) . "A Q u a d r a t i c a l l y C o n v e r g e n t A l g o r i t h m f o r G e n e r a l N o n l i n e a r Programming Problems. " Math. Programming 3 ( 2 ) , 145-156. ROBINSON, S. M. ( 1 9 7 3 a ) . "Bounds f o r E r r o r i n t h e S o l u t i o n Linear Algebra, A p p l . S e t o f a P e r t u r b e d L i n e a r Program." 6 , 69-81. ROBINSON, S. M. ( 1 9 7 3 b ) . l i n e a r Programming."

.

"Computable E r r o r Bounds f o r NonMath. Programming 5 ( 2 ) , 235-242.

" P e r t u r b e d Kuhn-Tucker P o i n t s and ROBINSON, S . M. ( 1 9 7 4 ) R a t e s o f Convergence f o r a C l a s s o f N o n l i n e a r Programming Algorithms.'' Math. Programming 7 (1), 1-16. " S t a b i l i t y Theory f o r Systems o f ROBINSON, S. M. ( 1 9 7 5 ) . I n e q u a l i t i e s , P a r t I : L i n e a r Systems." SIAM J . Num. A n a l . 1 2 , 754-769. ROBINSON, S. M. ( 1 9 7 6 a ) . " R e g u l a r i t y and S t a b i l i t y f o r Convex Multivalued Functions." Math. O p e r a t i o n s Res. l ( 2 ) , 130-143. ROBINSON, S. M. ( 1 9 7 6 b ) . " S t a b i l i t y Theory f o r S y s t e m s o f I n e q u a l i t i e s , P a r t 11: D i f f e r e n t i a b l e N o n l i n e a r Systems." S I A M J . Num. A n a l . 1 3 , 497-513.

"A C h a r a c t e r i z a t i o n o f S t a b i l i t y i n O p e r a t i o n s Res. 2 5 ( 3 ) , 435-447.

ROBINSON, S. M. ( 1 9 7 7 ) . L i n e a r Programming."

"Generalized Equations and T h e i r ROBINSON, S. M. ( 1 9 7 9 ) . S o l u t i o n s , P a r t I: B a s i c Theory." I n Mathematical Programming S t u d i e s , Vol. 1 0 (P. Huard, e d . ) , N o r t h - H o l l a n d , Amsterdam, pp. 128-141. ROBINSON, S. M. Equations."

.

(1980a) "Strongly Regular Generalized Math. O p e r a t i o n s Res. 5 (1), 43-62.

ROBINSON, S . M. ( 1 9 8 0 b ) . " G e n e r a l i z e d E q u a t i o n s and T h e i r S o l u t i o n s , P a r t 11: A p p l i c a t i o n s t o N o n l i n e a r Programming." MRC T e c h n i c a l Summary R e p o r t #2048, Mathematics R e s e a r c h C e n t e r , U n i v e r s i t y of W i s c o n s i n , Madison, W i s . ROBINSON, S. M . , a n d DAY, R. ( 1 9 7 4 ) . "A S u f f i c i e n t C o n d i t i o n f o r C o n t i n u i t y of O p t i m a l S e t s i n M a t h e m a t i c a l ProgramJ . Math. A n a l . A p p l . 4 5 (1), 506-511. ming." " D u a l i t y and S t a b i l i t y i n Extremum ROCKAFELLAR, R. T. ( 1 9 6 7 ) . Problems I n v o l v i n g Convex F u n c t i o n s . " P a c i f i c J . Math. 2 1 (1), 167-187.

343

References

ROCKAFELLAR, R. T. (1970). Convex A n a l y s i s . Press, Princeton, N. J.

Princeton Univ.

ROCKAFELLAR, R. T. (1971). "New Applications of Duality in In P r o c e e d i n g s of t h e 4 t h ConferNonlinear Programming. e n c e o n P r o b a b i l i t y (B. Bereanu e t a l . , eds.), Brasov. ROCKAFELLAR, R. T. (1973). "The Multiplier Method of Hestenes and Powell Applied to Convex Programming. 'I J . O p t i m i z . Theory A p p l . 1 2 (6), 555-562. ROCKAFELLAR, R. T. (1974). "Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming. I' SIAM J. C o n t r o l 1 2 (21, 268-285. ROCKAFELLAR, R. T. (1982a). "Lagrange Multipliers and Subderivatives of Optimal Value Functions in Nonlinear Programming." In Mathematical Programming S t u d i e s , Vol. 17 (D. C. Sorensen and R. J.-B. Wets, ed.), North-Holland, Amsterdam, pp. 28-66.

.

"Directional Differentiability of ROCKAFELLAR, R. T. (1982b) the Optimal Value Function in a Nonlinear Programming Problem." To be published in Mathematical Programming S t u d i e s (A. V. Fiacco, ed.), North-Holland, Amsterdam. ROODMAN, G . M. (1972). "Postoptimality Analysis in Zero-One Programming by Implicit Enumeration. 'I Naval Res. L o g i s t i c s Q. 19, 435-447. RUPP, R. D. (1976). "Quadratic Multiplier Method Convergence." J . O p t i m i z . Theory A p p l . 20(1), 1-12. SALINETTI, G., and WETS, R. J.-B. (1979). "On the Convergence of Sequences of Convex Sets in Finite Dimensions." SIAM Rev. 21(1), 18-33. SAMUELSON, P. A. (1947). F o u n d a t i o n s of Economic A n a l y s i s . Harvard Univ. Press, Cambridge, Mass. SCHRADY, D. A., and CHOE, U. C. (1971). "Models for MultiItem Continuous Review Inventory Policies Subject to Constraints." Naval Res. L o g i s t i c s Q . 1 8 , 541-563. SHAPIRO, J. F. (1977). "Sensitivity Analysis in Integer Programming." Ann. D i s c r e t e Math. I , 467-477.

SILVERMAN, G. J. (1972a). "Primal Decomposition of Mathematical Programs by Resource Allocation: I - Basic Theory and a Direction-Finding Procedure." O p e r a t i o n s Res. 20, 58-74. SILVERMAN, G. J. (197213). "Primal Decomposition of Mathematical Programs by Resource Allocation: I1 - Computational Algorithm with an Application to the Modular Design Problem." O p e r a t i o n s Res. 20, 75-93.

344

References

SLATER, M. ( 1 9 5 0 ) . " L a g r a n g e M u l t i p l i e r s R e v i s i t e d : A ConC o w l e s Commission t r i b u t i o n t o N o n l i n e a r Programming." D i s c u s s i o n P a p e r , Math. 403 (November), Yale U n i v . , N e w Haven, Conn. "Generic Conditions f o r Optimality i n SPINGARN, J . ( 1 9 7 7 ) . C o n s t r a i n e d M i n i m i z a t i o n Problems." Ph.D. d i s s e r t a t i o n , Dept. M a t h e m a t i c s , Univ. W a s h i n g t o n , S e a t t l e , Wash. SPINGARN, J . E . , a n d ROCKAFELLAR, R. T. ( 1 9 7 9 ) . "The G e n e r i c N a t u r e o f O p t i m a l i t y C o n d i t i o n s i n N o n l i n e a r Programming." Math. O p e r a t i o n s R e s . 4 ( 4 ) , 425-430. STERN, M. H., a n d TOPKIS, D. M. i n N o n l i n e a r Programming."

(1976). " R a t e s o f S t a b i l i t y O p e r a t i o n s R e s . 2 4 ( 3 ) , 462-476.

L. S . ( 1 9 7 8 ) . " E r r o r A n a l y s i s f o r Convex S e p a r a b l e Programs: The P i e c e w i s e L i n e a r A p p r o x i m a t i o n a n d t h e Bounds o n t h e O p t i m a l O b j e c t i v e V a l u e . " SIAM J . A p p Z . Math. 3 4 , 704-714.

THAKUR,

L. S. ( 1 9 8 0 ) . " E r r o r A n a l y s i s f o r Convex S e p a r a b l e Programs: Bounds o n t h e O p t i m a l S o l u t i o n a n d t h e D u a l O p t i m a l Solution." J . Math. A n a l . A p p Z . 7 5 , 486-494.

THAKUR,

THAKUR, L. S. ( 1 9 8 1 ) . " S o l v i n g H i g h l y N o n l i n e a r Convex S e p a r a b l e Programs Using S u c c e s s i v e Approximation." p r i n t , S h i p p e n s b u r g S t a t e C o l l . , S h i p p e n s b u r g , Pa.

Pre-

H. ( 1 9 5 8 ) . "The Kuhn-Tucker Theorem i n Concave Programming." I n S t u d i e s i n L i n e a r and N o n l i n e a r Programming ( K . J . Arrow, L. Hurwicz, a n d H. Uzawa, e d s . ) , S t a n f o r d Univ. P r e s s , S t a n f o r d , C a l i f .

UZAWA,

WENDELL, R. E. ( 1 9 8 1 a ) . "A New P e r s p e c t i v e o n S e n s i t i v i t y A T o l e r a n c e Approach. " A n a l y s i s i n L i n e a r Programming: Working P a p e r WP-448, G r a d u a t e S c h o o l o f B u s i n e s s , Univ. P i t t s b u r g h , P i t t s b u r g h , Pa. WENDELL, R.

E. (1981b). "A N e w P e r s p e c t i v e o n S e n s i t i v i t y The T o l e r a n c e Approach w i t h a p r i o r i Parametric Analysis: Information." Working P a p e r WP-454, G r a d u a t e S c h o o l O f B u s i n e s s , Univ. P i t t s b u r g h , P i t t s b u r g h , Pa.

WETS, R. J.-B. (1977). "On t h e C o n v e r g e n c e o f Random Convex S e t s . " I n C o n v e x A n a l y s i s and I t s A p p l i c a t i o n s . SpringerV e r l a g , New York, 191-206. W I E R Z B I C K I , A. P . , a n d KURCYUSZ, S. ( 1 9 7 7 ) . " P r o j e c t i o n on a Cone, P e n a l t y F u n c t i o n a l s a n d D u a l i t y T h e o r y f o r P r o b l e m s w i t h I n e q u a l i t y C o n s t r a i n t s i n H i l b e r t Space." SIAM J . C o n t r o l O p t i m i z . 1 5 (1) 25-56.

,

" C o n v e r g e n c e o f S e q u e n c e s of Convex S e t s , WIJSMAN, R. ( 1 9 6 6 ) . Trans. A m . Math. S O C . 1 2 3 , Cones a n d F u n c t i o n s , 11." 32-45.

References

345

WILDE, D. J. (1962). "Differential Calculus in Nonlinear Programming. O p e r a t i o n s R e s . 1 0 , 764-773. WILDE, D. J. (1965). "Jacobians in Constrained Nonlinear Optimization." O p e r a t i o n s Res. 13, 848-856. WILSON, R. B. (1963). "A Simplicia1 Algorithm for Concave Programming." Ph.D. dissertation, Graduate School of Business Administration, Harvard Univ., Cambridge, Mass. WOLFE, P. (1961). "A Duality Theorem for Nonlinear Programming." &. A p p l . Math. 1 9 ( 3 1 , 239-244. WOLSEY, L. A. (1981). "Integer Programming Duality: Price Functions and Sensitivity Analysis." Math. Programming 20 ( 2 ) , 173-195. ZLOBEC, S., GARDNER, R., and BEN-ISRAEL, A. (1982). "Regions of Stability for Arbitrarily Perturbed Convex Programs." In MathematicaZ Programming w i t h Data P e r t u r b a t i o n s I (A. V. Fiacco,ed.), L e c t u r e N o t e s i n Pure and A p p l i e d Mathem a t i c s , Vol. 73, Marcel Dekker, New York, pp. 69-89.

Selected Bibliography of Works Not Cited

ALT, W. "Lipschitzian Perturbations of Infinite Optimization Problems." To be published in Mathematical Programming with Data Perturbations 11 (A. V. Fiacco, ed.), Lecture Notes in Pure and Applied Mathematics Series, Marcel Dekker, New York. ARMACOST, R. L., and FIACCO, A. V. (1979). 'Sensitivity Analysis of a Well-Behaved Kuhn-Tucker Triple." In Survey of Mathematical Programming, Proceedings of the IX Symposium o n Mathematical Programming, Budapest, 1976, VOl. 1, (A. Prdkopa, ed.), North-Holland, Amsterdam, pp. 121134. AUBIN, J.-P. (1979). Economic Theory.

MathematicaZ Methods of Game and North-Holland, Amsterdam.

AUBIN, J.-P. (1980). "Further Properties of Lagrange MultiAppl. Math. Optimiz. pliers in Nonsmooth Optimization. 6(1), 79-90. AUBIN, J.-P., and CLARKE, F. H. (1979). "Shadow Prices and Duality for a Class of Optimal Control Problems." SIAM J . Control Optimiz. 1 7 ( 5 ) , 567-586. AUSLENDER, A. "Theorem of Constant Rank f o r Lipschitzian Maps and Applications in Optimization Theory." To be published in Mathematical Programming with Data Perturbations 11 (A. V. Fiacco, ed.), Lecture Notes in Pure and Applied Mathematics Series, Marcel Dekker, New York. BANK, B. (1980). "Stability Analysis in Pure and MixedInteger Linear Programming." In Lecture Notes in Control and Information Sciences, Vol. 23 (K. Iracki, K. Malanowski, and S. Walukiewicz, eds.) , Springer-Verlag, BerlinHeidelberg-New York, pp. 149-153. BERESNEV, V. V. (1973). "Necessary Conditions for an Extremum in Maximin Problems." Cybernetics 2, 302-312.

346

Selected Bibliography of Works Not Cited

347

BERESNEV, V. V. (1976). "Minimization of Solution Functions C y b e r n e t i c s 1 2 (5), for a Parametric Minimization Problem 755-765. BERESNEV, V. V., and PSCHENICHNYI, B. N. 1974). "The Differential Properties of Minimum Functions." USSR Computat. Math. Math. P h y s . 14(3) , 101-113. BITRAN, G. R., and MAGNANTI, T. L. (1976). "Duality and Sensitivity Analysis for Fractional Programs." O p e r a t i o n s R e s . 2 4 ( 4 ) , 675-699. BLAIR, C. E. , and JEROSLOW, R. G. (1977). "The Value Function of a Mixed Integer Program: I." D i s c r e t e Math. 1 9 , 121138. BLAIR, C. E. , and JEROSLOW, R. G. (1979). "The Value Function of a Mixed Integer Program: 11." D i s c r e t e Math. 25, 7-19. BLAIR, C. E., and JEROSLOW, R. G. (1982). "The Value Function of an Integer Program." Math. Programming 23(3), 237-273. BOHM, V. (1975). "On the Continuity of the Optimal Policy Set for Linear Programs." S I A M J . A p p l . Math. 28, 303-306. BORWEIN, J. M. (1981). "Convex Relations in Analysis and Optimization." In G e n e r a l i z e d C o n c a v i t y i n O p t i m i z a t i o n and Economics ( S . Schaible and W. T. Ziemba, eds.) , Academic Press, New York, pp. 335-377. BORWEIN, J. M. (1982). "Stability and Regular Points of Inequality Systems." Research Report 82-2, Department of Mathematics, Carnegie-Mellon University, Pittsburgh. BROSOWSKI, B. "Parametric Semi-Infinite Linear Programming.'' To be published in M a t h e m a t i c a l Programming S t u d i e s (A. V. Fiacco, ed.) , North-Holland, Amsterdam. BROSOWSKI, B. "On the Continuity of the Optimum Set in Parametric Semi-Infinite Programming." To be published in M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s 11 (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Marcel Dekker, New York. BROSOWSKI, B. "A Refinement of an Optimality Criterion and Its Application to Parametric Programming." J . O p t i m i z . T h e o r y A p p l . , (to be published). BROSOWSKI, B., and SCHNATZ, K. "Parametric Optimization, Differentiable Parameter Functions." Meth. O p e r a t i o n s R e s . 37, 99-118. DANIEL, J. W. (1973). "On Perturbations in Systems of Linear Inequalities. I' S I A M J . Num. A n a l . 10 (2), 299-307. DARKHOVSKII, B. S., and LEVITIN, E. S. (1976). "Differential Properties of the Value of a Finite-Dimensional Parametric Minimax Problem I.n C y b e r n e t i c s 12 (4), 553-566.

348

Selected Bibliography of Works Not Cited

DARKHOVSKII, B. S., and LEVITIN, E. S. (1976). "Differential Properties of the Value of a Finite-Dimensional Parametric Minimax Problem 11." C y b e r n e t i c s 1 2 (5), 771-781. DEMBO, R. S. (1980). "The Sensitivity of Optimal Engineering Designs using Geometric Programming. 'I Eng. O p t i m i z . 5 , 27-40. DEM'YANOV, V. F. (1973). "Second-Order Directional Derivatives of a Function of the Maximum.'' C y b e r n e t i c s 9 (5), 797-800. DE SILVA, A. H., and McCORMICK, G. P. (1978). "Sensitivity Analysis in Nonlinear Programming Using Factorable Symbolic Input." Technical Paper T-365, Institute for Management Science and Engineering, The George Washington University, Washington, D.C. DIEWERT, W. E. "Sensitivity Analysis in Economics." To be published in M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s 11 (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Marcel Dekker, New York. DINKEL, H., and KOCHENBERGER, G. (1979). "Constrained Entropy Models: Solvability and Sensitivity." Management S c i . 25 (6), 555-564. DOLECKI, S. (1978). "Semicontinuity in Constrained Optimization. Part 1.1: Metric Spaces." C o n t r o l and C y b e r n e t i c s 7(2), 5-16. DOLECKI, S. (1978). "Semicontinuity in Constrained Optimization. Part 1.2: Normed Spaces." C o n t r o l and C y b e r n e t i c s 7(3) , 17-26. DOLECKI, S. (1978). tion. Part 11: 7(4), 51-68.

"Semicontinuity in Constrained OptimizaOptimization." C o n t r o l and C y b e r n e t i c s

EKELAND, I. (1974). "On the Variational Principle." A n a l . A p p l . 47, 324-353.

J . Math.

EKELAND, I. , and LEBOURG, G. (1976). "Generic FrgchetDifferentiability and Perturbed Optimization Problems in Banach-Spaces." T r a n s . Amer. Math. S O C . 2 2 4 ( 2 1 , 193-216. EKELAND, I., and TEMAM, R. (1976). Convex A n a l y s i s and V a r i a t i o n a l Problems. North-Holland, Amsterdam-Oxford. Introduction t o t h e T h e o r y of L i n e a r and Convex Programming. Nauka, MOSCOW

EREMIN, I. I., and ASTAFIEV, N. N. (1976). (in Russian).

ESSER, H. (1972). "uber die Stetigkeit des Extremalwertes Nichtkonvexer Optimierungsaufgaben mit einer Anwendung auf die Diskretisierung von Kontrollproblemen." Z A M M 52, 535-542.

Selected Bibliography of Works Not Cited

349

FALK, J. E., and FIACCO, A. V. (eds.) (1982). M a t h e m a t i c a l Programming w i t h P a r a m e t e r s and M u l t i - L e v e l C o n s t r a i n t s . Special Issue of Computers and O p e r a t i o n s R e s e a r c h , Vol. 9, No. 1, Pergamon, New York. FIACCO, A. V. (ed.) (1982). M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I , L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Vol. 73. Marcel Dekker, New York. FIACCO, A. V. (ed.). M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I I . L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s . Marcel Dekker, New York (to be published). FIACCO, A. V. "Optimal Value Continuity and Differential Stability Bounds under the Mangasarian-Fromovitz Constraint Qualification." To be published in M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s 11 (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Marcel Dekker, New York. FIACCO, A. V., and GHAEMI, A. (1982). "Sensitivity Analysis of a Nonlinear Water Pollution Control Model Using an Upper Hudson River Data Base." O p e r a t i o n s R e s . 30 (11, 1-28. FIACCO, A. V., and GHAEMI, A. (1982). "Preliminary Sensitivity Analysis of a Stream Pollution Abatement System." In M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Vol. 73, Marcel Dekker, New York, pp. 111-130. FIACCO, A. V., GHAEMI, A., and KYPARISIS, J. A p p l i c a t i o n s of S e n s i t i v i t y and P a r a m e t r i c A n a l y s i s i n NLP (in preparation). FIACCO, A. V., and HUTZLER, W. P. (1982). "Basic Results in the Development of Sensitivity and Stability Analysis in Nonlinear Programming." In M a t h e m a t i c a l -Programming w i t h P a r a m e t e r s and M u l t i - L e v e l C o n s t r a i n t s (J. E. Falk and A. V. Fiacco, eds.), Special Issue of Computers and O p e r a t i o n s R e s e a r c h , Vol. 9, No. 1, Pergamon, New York, pp. 9-28. FIACCO, A. V., and HUTZLER, W. P. (1982). "Optimal Value Differential Stability Results for General Inequality Constrained Differentiable Mathematical Programs." In M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Vol. 73, Marcel Dekker, New York, pp. 29-43. FUJIWARA, 0. (1981). "Morse Programs: A Topological Approach to Smooth Constrained Optimization." MRC Technical Summary Report No. 2169, Mathematics Research Center, University of Wisconsin-Madison, Madison, Wis. FUJIWARA, 0. (1981). "Morse Programs: A Topological Approach to Smooth Constrained Optimization 11." MRC Technical Summary Report No. 2176, Mathematics Research Center, University of Wisconsin-Madison, Madison, Wis.

350

Selected Bibliography of Works Not Cited

GAL, T. (1980). "A 'Historiogramme' of Parametric Programming." J . O p e r a t i o n a l R e s . S o c . 31 , 449-451. GASS, S. I., and SAATY, T. L. (1955). "The Parametric Objective Function 2." O p e r a t i o n s R e s . 3, 395-401. GASS, S. I., and SAATY, T. L. (1955). "The Computational Algorithm for the Parametric Objective Function. 'I Naval R e s . Log. 2, 39-45. GAUVIN, J. (1980). "Shadow Prices in Nonconvex Mathematical Programming." Math. Programming 1 9 (3), 300-312. GEOFFRION, A. M. (1966). "Strictly Concave Parametric Programming, Part I: Basic Theory." Management S c i . l3(3) , 244-253. GEOFFRION, A. M. (1967). "Strictly Concave Parametric Programming, Part 11: Additional Theory and Computational Considerations." Management S c i . 13 (5), 359-370. GEOFFRION, A. M. , and NAUSS, R. (1977). "Parametric and Postoptimality Analysis in Integer Linear Programming." Management S c i . 23(5), 453-466. GOCHET, W., and SMEERS, Y. (1979). "A Branch and Bound Method Operations Res. for Reversed Geometric Programming. 2 7 ( 5 ) , 982-996. D u a l i t y T h e o r y i n M a t h e m a t i c a l Programming and I t s A p p l i c a t i o n s . Nauka, Moscow (in Russian).

GOL'STEIN, E. G. (1971).

GOULD, F. J. (1969). "Extensions of Lagrange Multipliers in Nonlinear Programming." SIAM J . A p p l . Math. 1 7 , 1280-1297. GREENBERG, H. J. , and PIERSKALLA, W. P. (1975). "Stability Theorems for Infinitely Constrained Mathematical Programs.'' J . O p t i m i z . T h e o r y A p p l . 16 (5/6) , 409-428. GUDDAT, J. (1976). "Stability in Convex Quadratic Programming." Math. O p e r a t i o n s f o r s c h . S t a t i s t . O p t i m i z . 7 (2), 223-245. GUDDAT, J. , and KLATTE, D. (1979). "Qualitative Stability in Nonlinear Optimization." In S u r v e y of M a t h e m a t i c a l Programming, P r o c e e d i n g s of t h e I X Symposium o n M a t h e m a t i c a l Programming, B u d a p e s t , 1 9 7 6 , V o l . 1, (A. Prgkopa, ed.),

North-Holland, Amsterdam, pp. 217-229.

HALKIN, H. (1974). "Implicit Functions and Optimization Problems without Continuous Differentiability of the Data." SIAM J . C o n t r o l 1 2 ( 2 ) , 229-236. HANSEN, E. R., and SENGUPTA, S. (1980). "Global Optimization Using Interval Analysis." In I n t e r v a l M a t h e m a t i c s 1 9 8 0 (K. Nickel, ed.), Academic Press, New York.

351

Selected Bibliography of Works Not Cited

HAUG, E. J., and KOMKOV, V. (1977). "Sensitivity Analysis in Distributed-Parameter Mechanical System Optimization." J . O p t i m i z . T h e o r y A p p l . 2 3 ( 3 ) , 445-464.

.

HIRIART-URRUTY, J.-B. (1978) "Gradients Ggngralisgs de Fonctions Marginales." SIAM J . C o n t r o l O p t i m i z . 1 6 (2), 301-316. HIRIART-URRUTY, J.-B. "Approximating a Second-Order Directional Derivative for Nonsmooth Convex Functions." SIAM J . C o n t r o l O p t i m i z . (to be published). HIRIART-URRUTY, J.-B. "Limiting Behavior of the Approximate First-Order and Second-Order Directional Derivatives for a Convex Function." To be published in N o n l i n e a r A n a l y s i s : T h e o r y , Methods and A p p l i c a t i o n s . HOFFMAN, A. J. (1952). "On Approximate Solutions of Systems of Linear Inequalities." J . R e s . N u t . Bur. S t a n d . 4 9 , 263-265. P o i n t - t o - S e t Maps and M a t h e m a t i c a l Programming, M a t h e m a t i c a l Programming S t u d i e s , Vol. 10,

HUARD, P. (ed.) (1979).

North-Holland, Amsterdam.

IRWIN, C. L., and YANG, C. W. "Iteration and Sensitivity for a Nonlinear Spatial Equilibrium Problem." To be published in M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s 11 (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Marcel Dekker, New York. JANIN, R. (1977). "Sensitivity for Nonconvex Optimization Problems. In Convex A n a l y s i s and I t s A p p l i c a t i o n s , L e c t u r e N o t e s i n Economics and M a t h e m a t i c a l S y s t e m s , VOl. 144, (A. Auslender, ed.), Springer-Verlag, New York, pp. 115-119. JANIN, R. (1977). "On Sensitivity in an Optimal Control Problem." J . Math. A n a l . A p p l . 6 0 (3), 631-657. JITTORNTRUM, K. (1978).

Optimiz. Theory A p p l .

"An

Implicit Function Theorem."

2 5 (4), 575-577.

J.

JOLY, J. L., and LAURENT, P. J. (1971). "Stability and Duality in Convex Minimization Problems." Rev. F r a n c o i s e d ' l n f o r m a t i q u e e t de R e c h e r c h e O p e r a t i o n e l l e 5 (R-2), 3-42. JONGEN, H. TH., JONKER, P., and TWILT, F. (1980). "On Deformation in Optimization." Meth. O p e r a t i o n s R e s . 3 7 , 171-184. JONGEN, H. TH., and TWILT, F. (1979). "On Decomposition arid Structural Stability in Nonconvex Optimization." In Num. Methoden b e i G r a p h e n t h e o r e t i s c h e n und K o m b i n a t o r i s c h e n P r o b l e m e n , Band 2 , ISNM 46, Birkhauser Verlag, Basel,

pp. 161-183.

KIRSCH, A. (1980). "Continuous Perturbations of Infinite Optimization Problems." J . O p t i m i z . T h e o r y A p p l . 3 2 (2)I 171-182.

352

Selected Bibliography of Works Not Cited

KLATTE, D. (1979). "On the Lower Semicontinuity of Optimal Sets in Convex Parametric Optimization." In M a t h e m a t i c a l Programming S t u d i e s , Vol. 10 (P. Huard, ed.), NorthHolland, Amsterdam, pp. 104-109. KLATTE, D. (1979). "Uber Lokale Stabilitztsmengen in der Parametrischen Quadratischen Optimierung," Math. Operat i o n s f o r s c h . S t a t i s t . O p t i m i z . I0 (4), 511-521. KOJIMA, M. (1978). "A Complementary Pivoting Approach to Parametric Programming." Math. O p e r a t i o n s R e s . 4 (41, 464-477. KORNBLUTH, J. S. H. (1975). "Duality, Indifference and Sensitivity Analysis in Multiobjective Linear Programming. " O p e r a t i o n s R e s . &. 2 5 , 599-614. KRABS, W. (1973). "Stabilitat und Stetigkeit bei Nichtlinearer Optimierung." In O p e r a t i o n s R e s e a r c h V e r f a h r e n X V I I , Teil 2, Verlag Anton Hain, Meisenheim am Glan, pp. 207-228. KRABS, W. (1977). "Stetige Abanderung der Daten bei Nichtlinearer Optimierung und Ihre Konsequenzen." In O p e r a t i o n s R e s e a r c h V e r f a h r e n X X V , Teil 1, Verlag Anton Hain, Meisenheim am Glan, pp. 93-113. KUMMER, B. (1977). "Global Stability of Optimization Problems." Math. O p e r a t i o n s f o r s c h . S t a t i s t . O p t i m i z . 8 (31, 367-383.. KUMMER, B. (1979). "A Note on the Continuity of the Solution Set of Special Dual Optimization Problems." In Mathem a t i c a l Programming S t u d i e s , Vol. 10 (P. Huard, ed.), North-Holland, Amsterdam, pp. 110-114. KYPARISIS, J. (1982). "Optimal Value Bounds for Posynomial Geometric Programs." Technical Paper T-464, Institute for Management Science and Engineering, The George Washington University, Washington, D.C. LAURENT, P.-J. (1972). Hermann, Paris.

Approximation e t Optimisation,

LEVITIN, E. S. (1974). "On Differential Properties of the Optimal Value of Parametric Problems in Mathematical Programming." S o u . Math. D o k l . 1 5 (21, 603-608. LEVITIN, E. S. (1975). "On the Local Perturbation Theory of a Problem of Mathematical Programming in a Banach Space." S o u . Math. D o k l . 16 (51, 1354-1358. LEVITIN, E. S . (1975). "On the Perturbation Theory of Nonsmooth Extremal Problems with Constraints." So#. Math. D o k l . 16 (5), 13894393. LUENBERGER, D. G. (1969). O p t i m i z a t i o n by V e c t o r Space M e t h o d s . Wiley, New York.

353

Selected Bibliography of Works Not Cited

LUENBERGER, D. G. (1973). I n t r o d u c t i o n to L i n e a r and Nonl i n e a r Programming. Addison-Wesley, Reading, Mass. MANGASARIAN, 0. L., and MEYER, R. R. (1979). "Nonlinear Perturbations of Linear Programs. SIAM J . Con t r o l O p t i m i z . 1 7 (61, 745-752. MANNE, G. B. (1953). "Notes on Parametric Linear Programming." Rand Co., Report No. B-468. MAURER, H. (1979). "Differential Stability in Optimal Control Problems." A p p l . Math. O p t i m i z . 5 (41, 283-295. MAURER, H. (1982). "First Order Sensitivity of the Optimal Value Function in Mathematical Programming and Optimal Control." In M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s I (A. V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Vol. 73, Marcel Dekker, New York, pp. 51-67. McCORMICK, G. P. N o n l i n e a r Programming, Wiley, New York (to be published). NACCHACHE, P. H. (1979). "Stability in Multicriteria Optimization." J . Math. A n a l . A p p l . 6 8 (21, 441-453. NAUSS, R. M. (1979). P a r a m e t r i c I n t e g e r Programming. Missouri Press, Columbus, Missouri.

Univ.

NGUYEN, V. H., STRODIOT, J.-J., and MIFFLIN, R. (1980). "On Conditions to Have Bounded Multipliers in Locally Lipschitz Programming." Math. Programming 18 (1), 100-106. S e n s i t i v i t y Analysis i n Discrete Linear O p t i m i z a t i o n P r o b l e m s , L e c t u r e N o t e s i n O p e r a t i o n s Res e a r c h and M a t h e m a t i c a l S y s t e m s , Vol. 30. Springer-Verlag,

NOLTEMEIER, H. (1970).

Berlin, Heidelberg (in German).

I t e r a t i v e SOZUt i o n of N o n l i n e a r E q u a t i o n s i n S e v e r a l V a r i a b l e s .

ORTEGA, J. M., and RHEINBOLDT, W. C. (1970). Academic Press, New York.

PENOT, J.-P. (1980). nOn Regularity Conditions in Mathematical Programming." Preprint, Universite de Paul Pau, France. PENOT, J.-P. (1981). "On the Existence of Lagrange Multipliers in Nonlinear Programming in Banach Spaces." In L e c t u r e N o t e s i n C o n t r o l and I n f o r m a t i o n S c i e n c e , Vol. 30 (A. Auslender, W. Oettli, and J. Stoer, eds.), Springer-Verlag, Berlin. PENOT, J.-P. (1982). "Differentiability of Relations and Differential Stability of Perturbed Optimization Problems." Preprint, Universite de Pau, Paul France. PETERSON, D. W. (1974). "On Sensitivity in Optimal Control Problems." J. O p t i r n i z . T h e o r y A p p l . 1 3 (11, 56-73.

354

Selected Bibliography of Works Not Cited

PETERSON, E.' L. (1976). 1 8 (1), 1-51.

"Geometric Programming."

S I A M Rev.

PETERSON, E. L. (1977). The Duality between Suboptimization and Parameter Deletion, with Application to Parametric Programming and Decomposition Theory in Geometric Programming." Math. O p e r a t i o n s Res. 2 (4), 311-319. PHILLIPS, D. (1974). "Sensitivity Analysis in Generalized Polynomial Programming." A I E E T r a n s . 6, 114-119. POMEROL, J.-CH. "The Lagrange Multiplier Set and the Generalized Gradient Set of the Marginal Function of a Differentiable Program in a Banach Space." J . O p t i m i z . Theory A p p l . (to be published). RADKE, M. A. (1975). "Sensitivity Analysis in Discrete Optimization." PhD dissertation: Working Paper No. 240, Western Management Science Institute, UCLA.

.

RIJCKAERT, M. J. (1974) "Sensitivity Analysis in Geometric Programming. In Mathematical Programs f o r A c t i v i t y A n a l y s i s (P. van Moeseke, ed.), North-Holland, Amsterdam, pp. 61-71. ROBINSON, S . M. (1976). "An Implicit-Function Theorem for Generalized Variational Inequalities." MRC Technical Summary Report No. 1672, University of Wisconsin-Madison, Madison, Wisconsin. ROBINSON, S . M. (1981). "Some Continuity Properties of Polyhedral Multifunctions." In Mathematical Programming S t u d i e s , Vol. 14, ( H . B. Kznig, B. Korte, and K. Ritter, eds.), North-Holland, Amsterdam, pp. 206-214. ROCKAFELLAR, R. T. (1974). SIAM, Philadelphia.

C o n j u g a t e D u a l i t y and O p t i m i z a t i o n .

ROCKAFELLAR, R. T. (1981). The Theory of S u b g r a d i e n t s and I t s Heldermann A p p l i c a t i o n s : Convex a n d Nonconvex F u n c t i o n s . Verlag, Berlin.

.

"Proximal Subgradients, Marginal ROCKAFELLAR, R. T. (1981) Values and Augmented Lagrangians in Nonconvex Optimization." Math. O p e r a t i o n s Res. 6 ( 3 ) , 424-436. ROCKAFELLAR, R. T. "Directional Differentiability of the Optimal Value Function in a Nonlinear Programming To be published in Mathematical Programming Problem. S t u d i e s (A. V. Fiacco, ed.), North-Holland, Amsterdam. ROCKAFELLAR, R. T. "Marginal Values and Second-Order Necessary Conditions for Optimality." Math. Programming (to be published). ROLEWICZ, S . (1981). "On Conditions Warranting G2-Subdifferentiability." In M a t h e m a t i c a l Programming S t u d i e s , VOl. 14 (H. B. Kgnig, B. Korte, and K. Ritter, eds.), NorthHolland, Amsterdam, pp. 215-224.

Selected Bibliography of Works Not Cited

355

SAATY, T. L., and GASS, S. I. (1954). "The Parametric Objective Function 1." O p e r a t i o n s R e s . 2 , 316-319. SCHAIBLE, S. (1975). "Marginalwerte in der Quotientenprogrammierung." 2 . B e t r i e b s w i r t s c h a f t 4 5 , 649-658. SCHAIBLE, S. (1978). A n a l y s i s and A p p l i c a t i o n s o f F r a c t i o n a l Programs. Verlag Anton Hain, Meisenheim am Glan (in German). SERTEL, M. R. (1975). "The Fundamental Continuity Theory of Optimization on a Compact Space I." J. Optirniz. T h e o r y Appl.

16 (5/6), 549-558.

SHETTY, C. M. (1959). "Sensitivity Analysis in Linear Programming." J . I n d . Eng. 1 0 , 379-386. SINGER, I. (1981). "Duality Theory for Perturbed Convex Optimization." J. Math. A n a l . A p p l . 8 1 ( 2 ) , 437-452. SOTSKOV, A. I. (1969). "Differentiability of a Functional Programming Problem in an Infinite Dimensional Space." C y b e r n e t i c s 5 (3), 334-342. SPINGARN, J. E. (1980). "Fixed and Variable Constraints in Sensitivity Analysis." SIAM J. C o n t r o l O p t i r n i z . 1 8 (3), 297-310.

.

TAMMER, K. (1974) "Die Abhgngigkeit eines Quadratischen Optimierungsproblems von einem Parameter in der Zielfunktion. 'I Math. O p e r a t i o n s f o r s c h . S t a t i s t . O p t i m i z . 5 (7/8), 573-590.

TANINO, T. and SAWARAGI, Y. (1980). "Stability of Nondominated Solutions in Multicriteria Decision-Making." J. O p t i r n i z . T h e o r y A p p l . 30 (21, 229-253. TUY, H. (1977). "Stability Property of a System of Inequalities." Math. O p e r a t i o n s f o r s c h . S t a t i s t . Optirniz. 8 , 27-39. WIERZBICKI, A. (1977). Models and S e n s i t i v i t y o f C o n t r o l S y s t e m s . Scientific and Technical Publishers, Warsaw (in Polish). WIERZBICKI, A., and DONTCHEV, A. (1974). "Basic Relations in Performance Sensitivity Analysis of Optimal Control Systems." C o n t r o l and C y b e r n e t i c s 3 (3/4), 119-134. WILDE, D. T., and BEIGHTLER, C. (1967). Foundations of O p t i m i z a t i o n . Prentice-Hall, Englewood Cliffs, N. J. WILLIAMS, A. C. (1963). "Marginal Values in Linear Programming." S I A M J . A p p l . Math. 1 1 , 82-94. WOLKOWICZ, H. (1980). "Shadow Prices for an Unstable Convex Program." U t i l i t a s Mathernatica 1 8 , 119-139.

356

Selected Bibliography of Works Not Cited

WOLKOWICZ, H. "Optimality Conditions and Shadow Prices." To be published in M a t h e m a t i c a l Programming w i t h Data P e r t u r b a t i o n s 11 ( A . V. Fiacco, ed.), L e c t u r e N o t e s i n Pure and A p p l i e d M a t h e m a t i c s S e r i e s , Marcel Dekker, New York. "Characterizing an Optimal Input in Perturbed ZLOBEC, S . Convex Programming." Math. Programming (to be published). "Perturbed Convex ZLOBEC, S . , and BEN-ISRAEL, A. ( 1 9 7 9 ) . Programs: Continuity of Optimal Solutions and Optimal Values." O p e r a t i o n s R e s e a r c h V e r f a h r e n 3 1 (11, 7 3 7 - 7 4 9 . ZOLEZZI, T. "On Stability Analysis in Mathematical Programming." To be published in M a t h e m a t i c a l Programming S t u d i e s ( A . V. Fiacco, ed.), North-Holland, Amsterdam. ZOWE, J., and KURCYUSZ, S . ( 1 9 7 9 ) . "Regularity and Stability for the Mathematical Programming Problem in Banach Spaces." A p p l . Math. O p t i m i z . 5 (1), 4 9 - 6 2 .

Author Index

Cesari, L., 37 Chames, A., 121, 243 Choe, U. C., 225, 232, 233, 235. 243 Clarke, F. H., 22, 34, 35 Colville, A. R., 137 Cooper, W. W., 120 Cottle, R. W., 24, 122 Cullurn, C. D., 122

A Abadie, J., 122 Allen, R. G. D., 119 Allgower, E., 284 Archetti, F., 280, 284 Amacost, R. L., 37, 51-54, 67. 82, 90, 106, 115, 141, 156, 187, 191, 206, 208, 225, 243 h o w , K. J., 25, 122, 206, 208. 209, 220 Auslender, A.. 29, 30, 37 Avriel, M., 274

D Daniel, J. W., 41, 54, 283 Danskin, J. M., 25, 26 Dantzig, G. B., 15-17, 19, 121, 278, 281 Davidenko, D., 57 Day, R., 15, 17, 18 Dernbo, R. S..274, 278 Dern’yanov, V. F., 26, 32 de Silva, A. H., 284 Dinkel, J. J., 57, 58, 278 Dinkelbach, W., 8, 285 Dolecki, S., 284 Dubeau, F., 22, 28-30, 37 Duffin, R. J., 37, 278

B Balder, E. J., 284 Ballabio, D.. 280, 284 Bank, B., 8, 284, 285 Bazaraa. M. S.. 25, 123, 267 Beightler, C. S., 278 Ben-Israel, A., 284 Benson, H. P., 57, 279, 283 Bereanu, B.,278, 281 Berge, C., 11-15, 19, 289 Bertsekas, D. P., 208 Bigelow. J. H., 37, 43 Bochner. S . . 36 Boot, J.,C. G., 8, 125 Bradley, S. P., 278 Brosowski, B., 8, 284, 285 Buys, J. D., 51, 82, 132, 206, 208, 209, 220

E Ecker, J.. G., 243, 275 Evans, J. P., 18, 19 Everett, H., 124

C

Canon, M. D., 122 Caratheodory, C.. I18

F Falk. J. E., 56. 273, 279, 282 Fenchel, W.. 34 357

358

Author Index

Fiacco, A. V., 8, 28-31, 37, 41, 42, 48, 50-54, 67, 69-74, 82, 88-90, 106, 115, 122, 156, 157, 173, 195, 196, 201, 206, 208, 225. 226, 243, 244, 246, 262, 268, 273-276, 281, 286, 289 Folkrnan, J.. 15-17, 19, 281 Fontanie, G.. 30 Fromovitz, S., 23, 122 Fukushima, M., 283

G Gal, T., 8, 284, 285 Gale, D., 33, 120. 121 Garcia, C. B., 284 Gardner, R.. 284 Gauvin. J., 22. 25. 28-30, 32, 37, 61, 284 Gemmell, R., E.. 243 Geoffnon, A. M.,33, 34, 56, 142. 143, 279, 284 Georg, K., 284 Ghaemi, A.. 243, 244, 274-276, 283, 284 Goldman, A. J.. 120 Gollan, B., 22, 32, 35, 61, 279, 283, 284 Gol'stein, E. G., 28, 31, 278 Gonin, R . , 51. 208, 220 Goode. J. J.. 25, 123 Could. F. J., 18. 19. 122, 206. 208, 209, 220, 284 Graves, L., 37 Graves, S. C.. 279 Greenberg. H. J., 19 Guddat. J.. 8, 284. 285

H Hager, W. W.. 22 Han. S.-P.. 70 Hancock, H.. 118. 123 Hax. A . C., 278 Hestenes. M. R., 30, 36, 37. 122. 206, 207, 289 Hicks. J. R., 119 Hildebrandt. T.. 37 Hirabayashi, R.. 284 Hiriart-Urmty, J.-B.. 35 Hogan, W. W.. 11-13. 20, 26, 27. 31, 278, 284 Hollatz, H., 8. 284, 285 Holm, S . . 279 Holmes, R. L.. 53, 233 Howe. S. M.. 206, 208, 209, 220

Hurwicz, L., 25, 122 Hutzler, W. P., 8, 29, 31 1

loffe, A. D., 49

J Janin, 61 Jittorntrum, K., 37, 43-45 John, F., 121

K Kantorovich, L., 55 Karush, W., 23, 70, 121 Kausmann, U., 285 Klatte. D., 8, 284, 285 Klein, D., 279 Kochenberger, G. A., 57, 58, 278 Kojima, M., 45, 284 Koopmans. T. C., 120 Kuhn, H. W., 23, 24, 70, 120. 121 Kummer, B.. 8, 284, 285 Kurcyusz, S., 49, 284 Kyparisis, J., 268, 275

L Lagrange, J. L., I 17. 1 18 Lasdon, L. S . , 267, 284 Lempio, F.. 31 Levitin, E.S.. 32. 272 Lornrnatzsch, K., 284, 285 Lorenzen, G., 285 Luenberger, D. G., 289, 290

M McCormick, G. P., 37, 41. 50, 53, 55, 59. 67. 69, 71, 74, 88-90, 122, 157, 173, 195, 196, 202, 233, 262, 273, 282-284, 286. 289 McKeown, J. J., 280 Magnanti, T. L.. 278 Mancini, L. J., 55, 283 Mangasarian, 0. L.. 23, 25, 62. 70. 103. 122, 247. 250, 289 Mann, H. B., 118. 123

Author Index

359

Marshall, A,, 119 Marsten, R. E., 279 Martin, D. H., 278 Martin, W., 36 Maurer, H., 31, 279 Meyer, R. R., 57, 195, 273, 279, 284 Miel, G., 55 Mills, H. D., 120 Mine, H. H., 283 Moore, R. E., 55 Morin, T. L., 279 Mosco, U., 281 Mylander, W. C., 53, 54, 187, 191, 233

S

Salinetti, G., 281 Samuelson, P. A,, 119, 123 Schrady. D. A., 225. 232, 233, 235, 243 Shapiro, J. F., 279 Shapiro, N. Z., 15-17, 19, 37, 43, 281 Shetty, C. M., 25, 123, 267 Silverman, G. J., 142, 143 Slater, M.,23, 24, 122 Soland, R. M., 56, 273, 279, 282 Spingarn, J. E., 37, 46-48, 284 Stem, M. h., 21, 22, 54

T

N Nozicka, F., 8, 284, 285

0 Ortega, J. M., 289, 290

Tammer, K., 8, 284. 285 Thakur, L. S.. 57, 279 Tikhomirov, V. R., 49 Tolle. J. W., 25, 28-30, 32, 61 Topkis, D. M.. 21, 22, 54 Tucker, A. W.. 23, 24, 70, 120, 121

P Passy, U., 274 Peterson, D. W.. 25 Peterson, E. L., 37, 278 Pevnyi, A. B., 32 Phillips, D. T., 278 Phipps, C. G., 123, 124 Pierskalla, W. P., 19 Polak, E., 122 Polyak, B. T.. 272 Powell, M. J. D., 207

R Rall, L. B., 55 Rheinboldt, W. C.. 37, 284. 289, 290 Robinson, S. M.,15, 17, 18, 25, 37-41, 43, 45, 48. 49. 54, 71, 72, 76. 195, 272, 78, 279. 283, 284 Rockafellar. R. T.. 22, 28, 30, 32-35, 61, 122, 208, 249. 267, 278, 279. 283, 284, 290 Roodman, G. M.,79 Rosen. J. B., 62, 103, 250 Rubinov. A. M., 26 Rupp. R. D., 208 Ryang. Y. J . , 283

U Uzawa, H.. 25. 122

v Vercellis, C., 280, 284

W Wendell, R. E., 278 Wets, R. J.-B., 281 Wiebking, R. D., 275 Wierzbicki, A. P., 49 Wijsman, R., 281 Wilde, D. J., 124 Wilson, R. B., 272 Wolfe, P.,257 Wolsey, L. A,. 279 Wong, S. N., 57, 58. 278

Z Zangwill, W. I . , 284 Zener, C., 37. 278 Zlobec, S., 284

This Page Intentionally Left Blank

Subject Index

A Affine function, 16 Algorithmic connections with sensitivity analysis, 196-200 augmented Lagrangian, 206-221 branch and bound, 56 through necessary conditions, 198-200 Newton's method, 198-200 penalty method, 50, 155-193 projected gradient, 205 reduced gradient method, 203-205 Applications of optimal value bounds, 273-276 power system energy GP model, 275, 276 water pollution GP model, 274, 275 Applications of sensitivity analysis, 225-245 decomposition, 142-151 inventory model, 232-242 parametric linear program, 113-1 15 power system energy GP model, 275, 276 Shell Dual, 137-141 structural design problem, 224-245 water pollution GP model, 243, 244 Augmented Lagrangian, 51, 206-221 differentiable KKT triple, 21 1-213 exact sensitivity, 52, 218-221 KKT derivatives, 21 1-218 KKT Jacobian inverse, 215 versus penalty function sensitivity calculations, 218-221 sensitivity formulas, 214, 215 sensitivity theorem, 52, 21 1-213 unconstrained minimum, 52

B Ball, 291 Block components of M-I augmented Lagrangian, 2 14 general algorithmic representation, 203 general direct representation, 96-100 penalty function, 175 projected gradient, 205 reduced gradient, 203-205 for the rhs problem, 131, 132 Bounds. 246-276, 303 KKT triple, 44, 48 optimal value, 54-60, 246-276 solution points, 44,48, 49, 54-60, 268-272 Branch and bound methods, 56, 254, 267, 279. 283 C

Calm program, 35 Compact set, 292 Computational experience, 243-245 Concave function, 20 strictly, 20 Connected set, 16, 46 Constraint qualifications Cottle (CQ4). 24 Kuhn-Tucker (CQS), 24 linear independence (CQ3). 24, 25, 30, 32, 42-45, 47, 71-73 Mangasarian-Fromovitz (CQI), 23-25, 28-31, 61, 72

36 1

362

Subject Index

Slater (CQ2). 24 Constraint set, 17 Control theory, 279 Convex function, 16, 24 strictly, 20 Convex overestimating problems, 60,273, 274 Convex point-to-set map, 266 Convex programs, 26-28, 3 1-33, 56-59, 133-136, 235. 247-265. 302 defined, 302 dual. 138, 255-265 Convex set, 16 Convex underestimating problems, 56. 57. 59, 60. 254, 267, 273. 279. 283 Cottle constraint qualification (CQ4), 24 Cyrtohedron, 46, 47

D Davidenko prediction-correction techniques, 57 Decision variable sensitivity analysis, 280 Decomposition using sensitivity results. 142- 152 Derivatives of a function Dini, 29, 30, 61 directional, 25-3 I , 34 directional second. 44 gradient. 69. 296 higher order partials. 51. 79, 172, 173, 297 Differential stability, see Optimal value function: Dini derivative bounds, directional derivative Dini derivatives, 29, 30 for general problem from rhs problem, 61 lower. 29 upper. 29 Directional derivative, 25-3 I , 34 Directional second derivatives, 44 Domain of point-to-set map, 15, 301 Dual problems, 138. 255-265

map, 14. 250. 265, 301. 302 continuous. 19 convex. 250. 266 convex-valued. 19 domain, 301 graph. 250. 265 lower semicontinuous, 19 uniformly linearly continuous, 2 I upper semicontinuous, 19 value. 18 First-order estimate of a KKT triple. 78 Function. 294 affine, 16 bounded below on a set. 295 concave, 20 continuous. 14. 294, 295 convex, 16. 20 differentiable. 296-298 global minimum value, 295 gradient, 296 Hessian, 297 infimum, 295 Lipschitz continuous, 21 local minimum value, 295 Isc, 14 minimum value, 295 optimal value, 295 piecewise linear, 10 psuedo-concave, 24 pseudo-convex, 24 quasi-convex, 16, 17, 24 real- or scalar-valued, 294 separable, 56, 57, 59 strictly concave, 20 strictly convex, 20 strictly quasi-convex, 17, 24 subgradient, 33. use, 14 vector-valued, 294-298 Future research directions and applications, 281 -287 G

E Effective domain off*(€), 18, 301 Energy GP model. 275, 276 Error bounds, 56, 57, 283 Euclidean space, 291 -298

Generalized equations, 38-40, 283 Geometric programs, 37, 57, 58, 274-276, 278 Global minimum, 295 Gradient, 69, 296 Graph of a mapping, 12, 250, 265

F Factorable programs, 59, 254, 274. 282 Feasible region, 18, 28, 301, 302

H Hessian. 297

Subject Index

363

Hessian of the optimal value function, 101-106

Higher order derivatives of a KKT triple, 79 of a penalty function KKT triple approximation, 50, 51, 172, 173

I Implicit function theorems, 36-39 Implicity defined function, 36, 37 Incremental approach, 57, 58 Indicator function, 39 Integer programming, 279 Interior point, 291 Interior of a set, 291 Interval arithmetic, 55, 283 Inventory model, 232-242 data, 236 first-order solution estimates and errors, 241, 242 Lagrange multiplier sensitivity, 241 optimal value function derivatives, 237 problem formulation, 235 sensitivities and interpretations, 237-242 solution and Lagrange multipliers, 237 solution point derivatives, 240 Isolated local minimum, 42, 43, 48-50, 71-73, 76, 77, 126, 158, 212

J Jacobian (matrix), 36, 297 Jacobian matrix of KKT necessary conditions, 77, 93-96 of perturbed KKT necessary conditions, 174 Jointly convex programs, see Parametric programs

K Karush-Kuhn-Tucker (KKT) conditions, 70, 74, 92 Karush-Kuhn-Tucker perturbed conditions, 50, 158, 174 Karush-Kuhn-Tucker system Jacobian, 77, 93-96 inverse in partitioned form, 96-100, 131, 132, 203, 214 Karush-Kuhn-Tucker system Jacobian inverse, algorithmic calculations, 173-179, 201-221 augmented Lagrangians, 206-22 1

general representation, 202 203 penalty function, 173-179 perturbed. 50, 158. 174 projected gradient, 205 reduced gradient, 203-205 Karush-Kuhn-Tucker triple, 302 bounds, 44, 48. 49, 59 connection with necessary conditions, 198-200 continuity, 42-45, 48, 73, 76 differentiability, 42, 50, 73. 126. 127, 21 1-213 directional differentiability, 44 first-order approximation, 78 first partial derivative formula, 100. 128, 217 higher order differentiability, 50, 79, 172, 173 penalty function estimates, 50, 51, 156-190 Karush-Kuhn-Tucker triple derivatives connection with the necessary conditions, 198-200 example calculations, 80, 81, 100-1 15, 136-152 formula for the rhs problem. 129 for the general problem, 77, 78, 93-100, 128, 215 with constraints not depending on the parameters, 85 penalty function estimates, 50, 51, 150-168, 173-179, 185-190 for the rhs problem, 126-136 Kuhn-Tucker constraint qualification (CQS), 24

L Lagrange multiplier rule, I18 Lagrange multipliers, 25. 27, 33, 117-125, 302 compactness condition, 25 as dual variables, 121 equivalence to subgradient off*, 33 Lipschitz continuity, 22 as optimal value function derivatives, 53, 87, 255-259 sensitivity interpretations, 123- 125 results, 53, 87-90, 124-126, 255-259 set, 25, 27 as shadow prices, 119, 120 unique, 25, 30 Lagrangian augmented, 51, 209

364

Subject Index

connections with duality, 257-267 generalized, 207-209 optimal value, 82 for the rhs problem. 120-129 usual. for the general parametric program, 68, 301 Lagrangian joint Hessian, 102-106 Limit inferior, 292, 293 point.292 of a sequence of points. 292, 293 of a sequence of sets, 15, 16 superior, 293 Linear combination, 293 Linear dependence, 294 Linear generalized equation, 40 Linear independence, 293 Linear independence constraint qualification (CQ3). 24. 25, 30, 32, 42-47, 72-76, I26 Linear programs, 113-1 15, 278, 302 LineariLation of P ~ ( E )264, , 265 of a vector function, 38 Lipschitz continuous function, 2 I Local minimum. see also Minimum and Solution point defined, 295 isolated, 42, 43. 48-50, 71-73, 120, 158, 212 isolated unconstrained, 50, 76, 77. 158, 212 strict, 69. 70 Local solution, see Local minimum, Minimum, and Solution point Local solution point extrapolation general problem, 78 unconstrained problem, 79 parameter derivatives. see KarushKuhn-Tucker triple derivatives Logarithmic-quadratic mixed barrier-penalty function, see Penalty function Lower semicontinuity of a function, 14

M Mangasarian-Fromovitz Constraint Qualification (CQI), 23-25, 28-31, 61, 72 Map or mapping, 294 Marginal function, see Optimal value function Matrix, 298-301

diagonal, 300 Hessian, 297 identity. 299 inverse, 299 Jacobian, 297 negative definite, 300 negative semidefinite, 300 nonsingular, 299 norm, 301 positive definite, 40, 50, 300 positive semidefinite, 39, 40, 300 product by a constant, 300 product of two, 300 pseudo-inverse, 202 rank, 299 sum, 300 symmetric, 40,299 transpose, 298, 299 Maximal monotone operator, 39 Mean-value theorem, 298 Metric, 291 Minimum, see also Local minimum and Solution point global, 295 isolated local, 42, 43, 48-50, 71-73, 126, 158 local, 295 strict local. 69 Monotone operator, 39

N Neighborhood, 291 Newton’s methods, 198-200 Nondegenerate function with respect to a set, 16 Nonparametric perturbations, 280-28 I Norm, 291 Normal program, 34, 35

0 Open ball, 291 Open set, 291 Optimal solution set S(a), see Solution set Optimal solution set map S. see Solution set point-to-set map Optimal value function, 18, 27, 35, 82, 126 bounds, 54-60, 247-267, 273-279 concavity, 58, 266 continuity, 15, 19, 20, 28 convexity, 33, 34, 56-62, 133-135, 247-255

Subject Index differentiability, 44, 52, 53 Dini derivative bounds, 29-32 directional derivative, 25-27, 30, 31, 249 effective domain, 18 extrapolation, 106, 136, 137 gradient, 44, 52, 53, 82-90, 179-190 Hessian, 52, 53, 82-90, 101-106. 130-136, 179-190 Lipschitz continuous. 21 local, 82 lower semicontinuity, 14. 19 marginal function, 18 penalty function estimates, 179-18 I second directional derivatives, 44 subgradient. 33 unconstrained. 76. 77 upper semicontinuity. 14. 19 Optimal value function derivatives application in calculating optimal value bounds. 249-259 example calculations. 106-1 15. 136-152 for the general problem, 44. 52, 82-90 with constraints not depending on the parameters, 85 Lagrange multipliers, 53, 87, 255-265 penalty function estimates, 167. 168. 179-190, 227-229 for perturbations linear in the parameters, 88-90 for the rhs problem. 53. 87. 126. 127 for the unconstrained problem. 85. 86 Optimal value function Hessian example calculations. 136- 141 penalty function estimate, 179-190 positive definite, 103-106. 130, 133-136 positive semidefinite. 103-106, 130. 133-136 for realizations of the general problem, 52. 53. 82-90. 101-106 for the rhs problem. 126- I36 Optimal value function parametric bounds. 54-60. 246-276 algorithmic calculations of dual bounds. 262. 273-276 computer implementations, 274-276 for convex or concave optimal value. 58-60. 247-255 convex upper bound, 249-255 dual linear lower bounds. 257-259, 264, 265 dual nonlinear lower bounds. 58, 259-265 extensions to nonconvex programs, 265-267. 273-276

365 LP and LP dual associated with linear lower bound, 264. 265 piecewise linear bounds. 247-255 for problems with special structure, 55-60, 265-267 refinement technique. 252. 253 used to calculate solution point bounds, 268-272 Optimal value Lagrangian. 82 Ordering relations, 294 Overestimating program. 60. 273, 274

P Parametric programs. see Programs Parametric solution vector estimate. 58. 59. 249-255. 265. 266, 275 Partial derivatives first. 296 second. 297 Penalty function. 50. 155-193 convergence of KKT triple estimates, 161, I62 dual-feasible points, 262 KKT triple estimates, 50. 156-168 derivatives, 50. 51. 156-168. 173-179. 185-190 example calculations of derivatives. 168-171 optimal value derivatives estimate, 167. 168. 179-190 for rhs perturbations. 185-190 unconstrained minimum 50, 157- 160 convergence, 161 -167 derivatives and convergence. 51, 161-167. 172. 173 Penalty function minimum derivative formula, 164 Penalty function optimal value, 179-190 Hessian. 179-190 Perturbed Karush-Kuhn-Tucker systems, 50. 158. 174 equivalence to penalty function conditions. 177-179 sensitivity formulas. 173-179 Point ball centered at, 291 boundary, 291, 292 of closure, 291 interior. 291 limit. 292, 293 neighborhood of. 291

366

Subject Index

sequence, 292 Point-to-set map, I I closed, 12, 13, 16 continuous, 12, 13 convex. 250, 266 convex valued, 17 domain, 15, 18 feasible region. 14. 250, 265. 301, 302 graph, 12, 250, 265 inverse, 39 lower semicontinuous, 12 open, 12, 13 solution set. 15-17, 301. 302 uniformly compact, 13. 29-31 uniformly linearly continuous, 21 upper semicontinuous, 12 value, 18 Prediction-correction technique, 57 Programs convex, 26-28. 31-33. 56-59, 133-136, 235. 247-265, 302 decomposable. 142-152 dual. 138. 255-265 factorable, 59, 254, 274, 282 general parametric. 301 -303 geometric. 37. 57. 58. 274-276. 278 infinite dimensional. 31. 32. 34. 35. 49 large-scale nonlinear, 142, 143 linear. 113-1 15. 302 in the parameters, P ( E ) , 88 linearization, 264. 265 overestimating. 60. 273. 274 quadratic. 40. 302 reverse-convex, 59 rhs equivalent to general parametric program. 60-62 rhs inequality-equality constrained, 116-152. 185-190, 302 separable. 56, 57, 59. 278, 279, 282. 283 for solution vector bounds calculation. 272 underestimating. 56. 57. 59, 60. 273. 279, 283 Projected gradient method. 205 Pseudo-concave function, 24 Pseudo-convex function. 24 Pseudo-inverse. 202

Q Quadratic program. 40. 302 Quasi-convex function. 16. 17. 24 strictly quasi-convex, 17. 24

R Rank of a matrix, 299 column rank, 299 row rank, 299 Reduced gradient method, 203-205 Reverse convex programs, 59 RHS problem KKT sensitivity formulas, 127-136 optimal value gradient. 127 optimal value Hessian, 130-141 sensitivity theorem, 126, 127 RHS-P3(c) equivalence, 60-62 for continuity off(€). 61 for convexity off(€), 61-62 for differential stability off(€), 61

S Scalar product of two vectors, 291 Screening of sensitivity information, 226-229 Second-order sufficient conditions, 69, 70 applied, 42, 43. 48, 49, 72-79 extended, 7 I , 72 strong for P?(E),143-145 for Pi(€).47, 48 Sensitivity information. meaning of, 303 SENSUMT. 54, 152. 187, 225-229. 233, 243-245. 274-276, 286 example calculations. 229-232 Separable program. 56, 57. 59. 278. 279. 282, 283 Sequence limit, 292. 293 limit inferior, 292, 293 limit point. 292 limit superior, 293 of points. 292. 293 of sets, 15, 16 subsequence, 292 Sets. 290. 291 Cartesian product, 290 closure, 291 compact, 292 connected, 16. 40 conventions, 290-294 convex, 16 empty, 290 equality. 290 interior. 18, 291 intersection, 290

Subject Index limits of a sequence, 5 , 16 neighborhood, 291 open ball, 291 product by a constant 29 I subset, 290 sum. 291 union, 290 Shell Dual, 138 Shell Primal, 137 Slater constraint qualification (CQ2). 24 Small o of a real valued function, 78 Solution point, 295, 301 continuity, 16, 17, 42. 45, 48, 76 derivatives, 79 differentiability, 42, SO, 72-77, 126, 211-213 directional differentiability, 44 first-order approximation, 78 Lipschitz continuity, 22 penalty function estimate, SO differentiability, 50. 51, 150-168. 172, 173 Solution point bounds, 44,48, 49, 54-60, 268-272 composite first- and second-order, 268-27 I computer implementation, 275 first-order, 268, 269 NLP program for calculating, 272 second-order, 268-27 I Solution point derivatives, see also Karush-Kuhn-Tucker triple derivatives penalty function estimates. 161-168 unconstrained problem. 79 Solution set, 301. 302 convex, 17 &optimal, 20-22 point-to-set map, 15, 301, 302 closed. 16 continuous, 17, 22 convex-valued, I7 upper semicontinuous, 14, 15 Stability of degree two. 28 Stability information, meaning of, 303 Stable program, 33 Stably set program. 34 Stochastic programming, 28 I Strict complementary slackness, 42-45. 47-49, 73. 126 Strict local minimum, 69. 70 Strictly convex (concave) function, 20

361 Strictly quasi-convex function, 17, 24 Strong second-order sufficient conditions for Pq(c).43-45 for P>(E),47, 48 Structural design model, 244, 245 Subdifferential operator, 39 Subgradient, 33 SUMT, 53, 191, 233, 286

T Taylor’s expansion first-order, 298 second-order, 298 Transposition of matrices, 298 symbol, T. 291 of vectors, 291

U Unconstrained problem sensitivity, 76. 79 Unconstrained solution point, see Solution point and Local minimum Underestimating program. 56, 57, 59, 60. 254, 267. 273, 279. 283 Uniformly linearly continuous point-to-set map, 21 Upper semicontinuity of a function, 14

V Vector conventions. 291 -295 distance between, 291 inner product, 291 norm, 291 ordering relations, 294 scalar product. 29 I sum. 291 Vector function. 294-298

W Water pollution GP model. 243. 244. 274, 275 Wolfe dual. 138. 255-265

This Page Intentionally Left Blank