Optimality and Stability in Mathematical Programming (Mathematical programming study)

IIr(t)ll ~< Ily(t)ll ~. Using (7.8) and (7.4) we see

Ilr(t)ll-< 2[[r~[I-< 4[IA-lll o(]ly (t)llk).

(7.9)

If T is the limit of the sequence {Tn}, then, letting n ~oo, relation (7.1) yields T = T - A - l h ( x + y(t) + r(t)),

or, in other words, h ( x + y(t) + r(t)) = O.

This together with (7.9) proves the assertion. It is obvious from the above proof that a continuous version of L e m m a 7.1 holds, where the sequences { y ( t i ) } i = l , 2 .... and {r(ti)}i:L2.... are replaced by functions y(t) and r(t), defined on some interval (0, to] with suitable positive to. We use this to prove: Proposition 7.2. L e t h be a m a p p i n g f r o m the B a n a c h s p a c e X to the B a n a c h space W which is twice F-differentiable in a neighborhood o f a given point x with h ( x ) = O. A s s u m e that h'(x) is su6ective. Then Th(x) = {d E X : h ' ( x ) d = 0},

Vh(x,d)={zEX:h'(x)z+89 In particular, h is regular at x.

(7.10) fordETh(x).

(7.11)

60

A. Ben-Tal and J. Zowe/ First and second order conditions

Proof. We only prove (7.11). The easier proof of (7.10) can be copied from this. Assume h'(x)d = 0 and z E Vh(x, d), i.e. there exists T > 0 and a curve r(t)

o(t2), h(x + t d + t 2 z + r ( t ) ) = O

for0~
A Taylor expansion gives (using h(x) = 0 and h'(x)d = O) h(x + td + t2z + r(t)) = = h'(x)(t2z + r(t)) + ~h"(x)(td +tEz + r(t), td + tEz + r(t)) + o(t2).

Since r ( t ) ~ o(t 2) the latter implies tE[h'(x)z + ~h"(x)(d, d)] + o(t 2) = 0,

0 < t ~< T.

Dividing by t 2 and letting t ~ 0 we get h'(x)z + ~h"(x)(d, d) = O.

This proves the inclusion C in (7.11). For the reverse inclusion assume h'(x)d = 0 and h ' ( x ) z + 8 9 Put y ( t ) : = t d + t E z for t E R . Then y ( t ) ~ 0 as t ~ 0 and

IIh(x + y(t))ll = lib(x) + h'(x)y(t) + 89

y(t)) + o(lly(t)ll2)l[ d)] + o(llY(t)ll2)ll

= IIt2[h'(x)z + 89

= o(lly(t)ll:). Hence L e m m a 7.1 (see remark following L e m m a 7.1) applies and with suitable r(t), h(x + y(t) + r(t)) = 0

where

IIr(t)ll-- o(lly(t)llb.

This shows z E Vh(x, d).

8. Necessary conditions under dillerentiability For the differentiable version of Theorem 2.1 we have to compute the support functionals appearing in Theorem 2.1. This is done in the following proposition.

Proposition 8.1. Let X, U, V and W be normed spaces and let f, g and h be twice F-differentiable. Let x, d E X be given. (i) Suppose ['(x )d <=0 and Q1(x, d) ~ O. Then [or Ii E A(Q~(x, d)), there is some u* E C § such that

li = u * . f'(x),

u*f'(x)d = O,

8"(1~ [ Qt(x, d)) = -~u*["(x)(d, d). (ii) Suppose g(x) ~ - K, g'(x)d ~ - cone(K + g(x)) and Qg(x, d) ~ O. Then for

A. Ben-Taf and J. Zowe/ First and second order conditions

61

lg E A(Qg(x, d)) there is some v* E K + such that v*g'(x)d = 0,

lg = v* 9g'(x),

~*(1~ I Q~(x, d)) = - 89

v*g(x) = 0,

d).

(iii) Suppose X and W are Banach spaces, h'(x) is onto, h ( x ) = 0 h'(x)d = O. Then for lh E Vh(X, d), there is some w* E W* such that

and

lh = W* " h ' ( x ) , ~*(lh [ Vh(X, d)) = - 89

d).

Proof. (i) The proof can be copied from (ii). (ii) Let lg E A(Qg(x, d)) be given. We put in L e m m a 3.5, A := g'(x)

and

S := - 89

d ) - int Kg(~.g'tx~d.

By Proposition 6.2(ii) and (6.4), lg E A ( A IS). Now condition (i) in L e m m a 3.5 is satisfied since, by assumption, Qg(x, d) # 0. We get from L e m m a 3.5 that with a suitable v* E A(S),

lg = v * . g'(x)

and

8*(v* ] S) = ~*(lg ] Qg(x, d)).

Taking into account that int Kg(x).g,~x)d= int K + {hg(x)+ i~g'(x)d: h,/~ 1> 0} (see L e m m a 5.4) we get for v* E A ( S ) ,

v* E K §

v*g(x) = O,

v*g'(x)d = 0

and, finally,

~*(v* I s ) = -~v*g"(x)(d, d). (iii) The assertion follows in a similar way from Proposition 7.2 and L e m m a 3.5 if one puts A := h'(x)

and

S := - 89

d).

We now get the following result:

Theorem 8.2. Let Xo be an optimal point for problem (P). Let X and W be Banach spaces and U and V normed. Suppose f, g and h are twice Fdifferentiable and the range R(h'(xo)) of h'(xo) is closed. Then for every d satisfying

['(xo)d <~O, g'(xo)d E - c o n e ( K + g(xo)),

h'(xo)d = 0,

(8.1)

there correspond continuous linear [unctionals u* E C +, v* E K § and w* ~ W*,

A. Ben-Tal and J. Zowe/ First and second order conditions

62

not all zero, such that u*f'(xo)d = O,

v*g(xo) = O,

v*g'(xo)d = O,

(8.2)

u * . Y ( x o ) + v * . g ' ( x o ) + w * . h ' ( x o ) = O,

(8.3)

(u* " ["(Xo) + v* " g"(Xo) + w* . h"(Xo))(d, d) >t O.

(8.4)

Proof. Suppose first, h'(xo) is not onto. Then R(h'(xd) and an arbitrary element in W not belonging to R(h'(xo)) can be separated, that is, there is some

if* E W*, v~* ~ 0, such that ~ * 9 h'(xo)x >t 0

for all x ~ X.

It follows that ~* 9 h'(xo) = 0. (8.2)-(8.4) hold with u* = 0, v* = 0 and w* = if* or w* = - a,*. Hence we may assume throughout the following that h'(xo) is onto. Now, suppose (8.1) holds with a given d. By Propositions 6.1 and 7.1, d satisfies (2.9) in Theorem 2.1. Propositions 7.2 and 6.2 together with (6.3) and (6.4) imply that Q1(x0, d), Q~(x0, d) and Vh(Xo, d) are convex; moreover, Vh(xo, d) is non-empty. If also Qs(x0, d) and Qg(x0, d) are non-empty, then f, g and h are d-regular and Theorem 2.1 applies. The assertion follows from Theorem 2.1 and Proposition 8.1. Now suppose Q~(x0, d ) = I~ (if Qi(x0, d ) = ~, then a similar argument applies). Then the non-empty convex sets {g'(xo)z + ~g"(Xo)(d, d): z E X}

and

- i n t Kg~xo),g'Cxo)d

can be separated and thus there exists some v* ~ 0 in V* such that for all z ~ X, v E K and )t, tx ~>O, v*(g'(xo)z + 89

d)) >- v * ( - k - hg(xo) - Ixg'(xo)d).

It follows that v* E K §

v*g(xo) = O,

v*g'(xo)d = 0

and, finally, v*g"(xo)(d, d) >~O. (8.3) and (8.4) follow if we put u* = 0 and w* = 0. We add a constraint qualification (compare Section 4, Remark 3): h'(x0) is onto and there exists z E X such that CQ(d)

g'(xo)z + g"(Xo)(d, d) E - int K~txo).g'<~o)d, h'(xo)z + h"(xo)(d, d) = O.

If CQ(d) holds for d satisfying (8.1), then Qs(x0, d ) O Vh(xo, d ) ~ O (Propositions 6.2(ii) and 7.2) and thus 11# 0 in Theorem 2.1 (see Remark 3 in Section 4); this shows u* # 0.

A. Ben-Tal and J. Zowe/ First and second order conditions

63

Corollary 8.3. If for the vector d in (8.1) condition CQ(d) is satisfied, then the corresponding u* in (8.2)-(8.4) is not the zero-functional.

9. The mathematical programming problem We specialize Theorem 2.1 to problem (MP). To do this denote by Dg,(x) the set of quasifeasible directions of the component gi of g (replace K by R+ in the definition); the same for Qg,(X, d) and D~(x). We call gi regular at x if Qg~(x, d) is non-empty and convex for all d E Dg,(X). Further recall the definition of I(x),

I(x) = {i E {1..... n}: gi(x) = 0}, and put for d E Dg(x) (compare (5.2)), J(x, d) = {i E I(x): d E D~(x)}. Theorem 9.1. Let Xo be an optimal point for problem (MP) and suppose f, h and

gi, i E I(xo), are regular at Xo. Then for every

d e D,(xo) n ( n Dg,(Xo)] n Th(xo) \iEI(xo)

/

(9.1)

there correspond continuous linear functionals If E A ( Q I ( x o , d ) ) ,

lh ~ A ( V h ( x o ,

d)),

(9.2)

lg, E A ( Qg,(Xo, d)) for i E J (Xo, d), not all zero, satisfying

If +

~

lg, + lh = 0,

(9.3)

i~J(x o, d)

~*(l~lQ~(xo,d))+ E

8*(Is, I Qg,(Xo, d))+ 8*(lh { Vh(Xo, d)) <~0. (9.4)

iEJ(xO,d)

Proof. Let d be given and suppose (9.1) holds. A look at the definition of Dg(xo) shows that

Dg(xo)= n Dg,(Xo). iEl(x O)

Hence, (2.9) is satisfied by d. Furthermore, one has

Qg(xo, d) = ~

iC=l(xo)

Qg,(x0,d)

and since, by assumption, Qgi(Xo, d) is convex and non-empty for i E I(x0), the set Qg(x0, d) is also convex. If Qg(x0, d) is non-empty, then f , g and h are

64

A. Ben-Tal and J. Z o w e / First and second order conditions

d-regular at x0 and Theorem 2.1 applies. From L e m m a 3.2 we get that lg in Theorem 2.1 can be written as a sum lg = ~

lg, where lg, ~ A(Qg,(Xo, d))

iEl(x O)

and that

~*(l~ I Q~(xo, d)) = E 8"(l~, I Qg,(x0, d)). iEI(xO)

(2.4) implies l~ = 0 for i ~ I(xo)\J(xo, d). (9.2)-(9.4) follows. Now suppose Qg(x0, d) = I~. Then the assertion follows from L e m m a 3.3 if we put l~ = 0 and lh = O. The above second order condition contains the famous first order condition by Dubobitskii and Milyutin [10, Theorem 6.1] as a special case. To see this, put d := 0 in Theorem 9.1. Obviously, (9.1) is satisfied. The second order sets in (9.2) become (see (2.3), (2.6) and (2.8)) A(Qi(xo, 0)) = - D~(xo) +,

A ( V h ( X o , 0)) = - Th(Xo) +,

A(Qg,(xo, O)) = - D<{x g~ o)x+ for i ~ J(x0,0) = I(xo) and (9.4) reduces to the trivial inequality 0 ~<0. We get:

Corollary 9.2. Let Xo be an optimal point for problem (MP) and suppose D~(x0), Th(xo) and D~(xo) for i ~ I(xo) are non-empty and convex. Then there are 1i ~ D~(xo) +, lh E Th(XO) + and lg, ~ D<~x g~ o)~§ for i E I(xo), not all zero, such that liq- ~

lgi-Flh=O.

i~t(x o)

The following differentiable version of Theorem 9.1 is easily proved by specializing Theorem 8.2.

Theorem 9.3. Let Xo be an optimal point for problem (MP). Let X and W be Banach spaces, suppose f, h and gi for i = 1, 2 ..... n, are twice F-differentiable and the range of h'(xo) is closed. Then for every d satisfying f'(xo)d <~O,

g~(xo)d<~0

for i ~ I(xo),

h'(xo)d = 0,

(9.5)

there correspond real numbers y0 t> 0 and Yi >10 for i E J(Xo, d) and a functional w* ~ W*, not all zero, such that Yof'(Xo) +

~

yg~(Xo)+ w* 9 h'(xo) = 0,

(9.6)

iEJ(xO,d)

(Y0f"(x0)+

: i~J(xo, d)

yg'[(Xo)+w*'h"(Xo))(d,d)>~O.

(9.7)

A. Ben-Tal and J. Zowe/ First and second order conditions

65

Remark 1. F o r dim W < ~ the range of h'(x0) is always closed and this assumption can be omitted. Theorem 9.3 reduces to a result by Ben-Tal [3]; see also Cox (in Hestenes [13, Chapter 6, Theorem 10.4]) where a similar statement is proved for finite dimensional X and for those d's only satisfying (9.5) with equality. Remark 2. Suppose under the assumptions of Theorem 9.3 that (9.5) holds for d. Then the corresponding y0 can be chosen as 1 if the following constraint qualification is met (see Corollary 8.3 and Proposition 5.3): h'(xo) is onto and there exists z E X for which

CQ(d)

g~(xo)z + g'[(Xo)(d, d) < 0

for i E J(xo, d),

h'(xo)z + h"(xo)(d, d) = O.

Remark 3. For d = 0 and W = R m, Theorem 9.3 reduces to the Fritz-John condition. The condition guaranteeing y0 = 1 in (9.6) becomes the MangasarianFr-mowitz constraint qualification (see e.g. Gauvin and Tolle [9]): h~(xo), j = 1 . . . . . m, are linearly independent and

there exists z E X such that

CQ(0) g~(xo)z < 0

for i E I(xo),

h)(xo)z = 0

for j = 1, 2 . . . . . m.

Remark 4. If there is ~i I> 0 for i E J(x0, d), (9.5)), then the set of from (9.6) (put yi: = 0

a d in (9.5) for which (9.6) holds with a multiplier Y0> 0, and ~* (e.g., whenever CQ(d) is met for at least one d in directions d satisfying (9.5) looks more familiar. One gets for i E I ( x o ) \ J ( x o , d)):

{d I f'(xo)d <~ O, g~(xo)d <<-0 for i E I(x0), h'(xo)d = 0} = =

d:g~(xo)d

OforiEI(x0) andfi=O

'

h'(xo)d=O

.

Remark 5. The following simple example shows that the multipliers in Theorem 9.3 depend on d (compare Remark 2 in Section 4). Let X = R 2 , / ( x ) : = x l2 - x 2 ,2 gl(x) = x l ( x 2 - 89 g2(x) = - xl(IEXl + x2) and g3(x) = - x 2 + x~. It is easily checked that x0 = (0, 0) is optimal (even a strict global minimizer). As f ' ( x o ) = g~(xo)= (0, 0) for i = 1, 2, 3 all d E R 2 satisfy (9.5). Suppose now (9.6) and (9.7) hold with fixed y0. . . . . y3 for all d. For d -- (~) we get from (9.7), 2 y 0 - yl - y2 - - 2y3 I> 0, and for d = (~, - 2yo >/O.

66

A. Ben-Tal and J. Zowel First and second order conditions

Together with y ~ > 0 this implies (Yo, Yl, Y2, Y3) ~ (0, O, O, 0).

y0 = yt = y2 = y 3 = 0

in contradiction to

The question arises under what conditions one can state a necessary condition where the same multipliers can be taken for all d. One such case will be when the multipliers in the Euler-Lagrange equation (9.6) are determined uniquely by (9.6). This is quaranteed by the condition CQ1

{(g'(xo)x + k, h'(xo)x): x E X, k E R" and ki = 0 for i E I(x0)} = R" • W.

This condition could be rephrased omitting k and using the active components of g only. But note that our formulation also makes sense for an arbitrary cone relation g ( x ) E - K (here k has to be taken from Kg(x0)N-Kg~x0)). The same applies to the next condition CQ2. For dim X < oo and h = (hi . . . . . hm) (i.e., W = Rm), CQ1 is equivalent to the well-known condition (see e.g. Fiacco and McCormick [8] or Luenberger [22]): The gradients Vgi(Xo) and Vhi(x0), i E I(xo) and j = 1. . . . . m, are linearly independent. Another type of regularity assumption is the following For every d satisfying (9.5) there is z E X and k ~ R" with ki = 0 for i E I(xo) such that CQ2

g'(xo)z + g"(Xo)(d, d) + k = O, h'(xo)z + h"(xo)(d, d) = O. For dim X < co and dim W < 0% CQ2 is implied by the so-called second order constraint qualification given by Fiacco and McCormick [8, p. 28]. We get the following result which for dim X < oo and dim W < oo can be found e.g. in [8, p. 25; 22; 13, p. 27].

Corollary 9.4. Consider problem (MP) with twice differentiable functions f, gi and h and Banach spaces X and W. Let Xo be optimal and suppose that either of the following conditions holds (i) CQ1, (ii) CQ2, the range of h'(xo) is closed and Y0 can be taken as 1 for all d satisfying (9.5). Then there are multipliers Yi for i E I(xo) and w* ~ W* such that f'(Xo)+ ~ Yigl(xo) + w*h'(xo) = O, iel(xo)'

(9.8)

(r,(x0)+ie;(x E o)y,gT(xo)+

(9.9)

A . B e n - T a l and J. Zowe/ First and s e c o n d order c o n d i t i o n s

67

for all d satisfying g~(xo)d = 0 for i E I(xo) and yi > 0, g~(xo)d <~0 for i E I(xo) and Yi = 0,

(9.10)

h'(xo)d = O.

Proof. (i) If CQ1 holds, then Y0 in (9.6) can be normalized to 1 for all d satisfying (9.5) (y0 = 0 leads to a contradiction to the onto condition and the fact that not all multipliers are zero in (9.6)). Once y0 is put equal to 1, CQ1 also implies that the yi and w* in (9.6) are uniquely determined. H e n c e (9.6) and (9.7) or (9.8) and (9.9), respectively, hold with fixed multipliers for all d satisfying (9.5). The rest follows from Remark 4 above. (ii) By assumption we can choose y0 = 1 in (9.6), (9.7) for all d satisfying (9.5). Now suppose (9.6), (9.7) hold for a fixed d with Yi, w* and for some other d with ~, if*; further let s and/~ be the element which corresponds to d via CQ2. All we have to show is that (9.7) also holds for d with y~ and w*. But this follows by some easy calculation:

f"(Xo) +

~_~ yig'I(x0) + w*h"(Xo))(d, d) = i •J(x o, d)

= l"(xo)(d, d ) -

y~(x0)~'- w*h'(xo)~.

~ i~J(x o, d)

= f"(Xo)(d, d) + f'(Xo)~. = f"(Xo)(d, d ) -

~

(by (9.6)) y~g~(x0)~- ~*h'(xo)~.

iEJ(x0, d)

= (f"(Xo) +

_ Yig"(x0) + ~'*h"(Xo))(d, d) i~J(x 0, d)

1>0.

I0. An example We study an example for Theorems 2.1 and 9.1. Consider the non-differentiable problem (compare [8, p. 24])

I)2+ x~,

minimize

f(x) = (xl-

subject to

g(x) = Ixt[--~

Ix~<0,

where k is a positive parameter. For what values of k can x0:= (0,0) x be optimal?

68

A. Ben-Tal and J. Zowe/First and second order conditions We calculate (see Propositions 6.1(i), 6.2(i) and e.g. (6.3)) Vf(x0)= (-20),

v2S(xo)= (~ 02),

Ds(xo) = {d: dTVf(xo) <~O} = { (dO2): d2 E R }, Q/(xo, d) = {z: - 2zl + d22 < 0} for d ~ D/(xo). The last relation shows that f is regular at x0. By Proposition 8.1(i),

f-12A.odTV2f(xo)d if y = XoVf(xo) for some )to/> O, 8*(y ] Q/(Xo, d)) = ~t ~

otherwise,

f-)tod~

too

if y : Xo(o2), Xo~>O, otherwise.

Further, by Proposition 6.1(ii),

Dg(xo) = {d: g'(x0; d) ~<0}

= {d:,im ,-'[,id,it~O+

L

r~

j

0}

and, by Corollary 6.3(ii),

D~(xo) = {d: g'(x0; d) <0} = 0; so Ds(xo) = D~(xo). Now for d E D~(xo), it follows from Proposition 6.2(ii) that Qg(xo, d) = {z: g"(xo, d; z) < 0} which (after some arithmetic) becomes 1

2

R}.

Hence g is d-regular at x0 if d2 ~ 0. Finally, by a direct computation,

8*(YlQg(x~

ly'ld2 otherwise.ifY=(~i)'YlER'

The Euler-Lagrange equation (2.11) is then (lO.1) and the Legendre inequality (2.12) is

-Xod +lly,ld -
(10.2)

A. Ben-Tal and L Zowe/ First and second order conditions

69

F r o m (10.1) it follows that lye[ = 2Ao, which, when substituted in (10.2), yields

H e n c e k I> 2, which answers the question. In fact, it can be shown that k I> 2 is also sufficient for the optimality of x0. Note that the first order condition, Corollary 4.1, does not provide any information since D~(xo)= Qg(xo, 0 ) = ~ (for any k) and therefore the E u l e r Lagrange equation (4.3) holds trivially.

11. Sufficient conditions We start with the differentiable case. L e t us call a point x0 of the feasible set F strict optimal for (P) if there is a neighborhood N of x0 such that for x E N fq F,

f ( x ) <=f(xo)

implies x = x0.

For finite-dimensional X we can p r o v e a counterpart of T h e o r e m 8.2. For U = •, V = R n and W = R m this reduces to a result b y Ben-Tal [2]; see also Hestenes [13, Chapter 4, T h e o r e m 7.4]. Theorem 11.1. Consider problem (P) with finite-dimensional X , closed convex cone C and twice F-differentiable functions f, g and h. L e t Xo be feasible for (P). Then Xo is strictly optimal if either of the following two conditions holds : (i) There is no solution d to the system

f'(xo)d <=0, g'(xo)d ~-Kg~xo),

h'(xo)d = 0 ,

d#0.

(11.1)

(ii) For every solutoin d of (11.1) there correspond multipliers u* E C +, v* E K + and w* E W * such that

u*f'(xo)d = O,

v*g'(xo)d = O,

v*g(xo) = 0,

(11.2)

u* . f'(xo) + v* 9g'(xo) + w* . h'(xo) = 0,

(11.3)

u *f"(Xo)(d, d) + v* g"(xo)(d, d) + w* h"(xo)(d, d) > 0.

(11.4)

Proof. The p r o o f is indirect. Suppose Xo is not strict optimal. Thetl there is a sequence {x,},=l ..... xn# x0 and xn ~ x 0 , such that for all n, f(x~) - f(xo) ~ - C

(11.5)

and g(x,) E - K, h(x,) = 0. F r o m the two last relations we get

g(xn) - g(Xo) E - Kg(xo), h ( x n ) - h ( x o ) = O.

(11.6)

70

A. Ben-Tal and J. Zowe/ First and second order conditions

Put z. : = x . - x 0 and t. :=llz.ll. Since the boundary of the unit ball in the finite-dimensional X is compact, the following limit exists (at least for a suitable subsequence): do := lim t ~ z . . n~cr

(11.5) and (11.6) imply (note that C and Kg(xo) are closed): f'(xo)do = lim t ~ l ( f ( x . ) - f(Xo)) E - C, n-~oo

g'(xo)do = lim t~'(g(x.) - g(xo)) E - Kg(x0), n--->o~

h'(xo)do = lim t ~ l ( h ( x . ) -

h(xo)) = O.

n--->oe

Hence do satisfies (11.1). Further do # 0. If (i) holds, then we reach a contradiction. Hence suppose (ii) holds, that is, (11.2)-(11.4) hold with suitable u~, v~ and w~. Now by definition of the derivative f ( x . ) - f(xo) = f'(xo)z. + ~f"(Xo)(Z., z.) + o(t~) and, because of (11.5),

f'(xo)z. + ~f"(xo)(Z., z.) + o(t~) e - C. Similarly, g'(xo)z. + 89

z.) + o(t2.) E - K~(xo),

h'(xo)z, + 89

z.) + o ( t b = O.

Multiplying these equations with u~, v~, w~ and summing up while using (11.3), we get (note that v~ E Kg(xo) since v~ E K + and v~Jg(xo) = 0): +

(u~f"(xo) + v~g"(xo) + w~h"(xo))(z~, z,) + o ( t b ~ O. Dividing by t 2 and letting n ~oo we get (u ~f"(xo) + v ~g"(xo) + w ~ h"(xo))(do, do) ~< 0 which contradicts (11.4). A simple modification of the above proof gives us a similar condition which for U = R and finite-dimensional V and W becomes [13, Chapter 4, Theorem 7.3]. The assumption dim X < ~ is essential for the above statement; see the following example.

Example 1. Let X be the real sequence 12 and I = R. Choose some s E X with

A. Ben-Tal and J. Zowe/ First and second order conditions

71

c o m p o n e n t s s, > 0 for all n and consider the free problem oo

minimize

oo

f (x ) = ~= six 2 - ~= x 4.

L e t x0 be the origin in 12. Then f'(x0) is the zero-functional on 12 and

f"(Xo)(d, d) = 2 ~ sid~ for all d E 12. H e n c e (11.2)-(11.4) hold with u* = 1 for all d E 12, d ~ 0. Nevertheless, x0 is not optimal. T a k e the sequence x n := 2N/J, 9I n for n = 1, 2 . . . . . where l" is the vector with all c o m p o n e n t s 0 except the n t h c o m p o n e n t which is 1. Then x ~ -->x0, but

f(x")=4s 2-16s 2<0

for a l l n .

For infinite-dimensional X the following sufficient second order condition can be proved; see Maurer and Z o w e [25]. For a further discussion of sufficient conditions see the recent p a p e r b y Ioffe [16] and a forthcoming article by L e m p i o and Z o w e [20].

Theorem 11.2. Consider problem (P) with U = •, with real Banach spaces X, V, W and twice F-differentiable functions. Let Xo be feasible for (P) and suppose

there is a map x--*d(x) of the feasible set F into the linear approximation {d E X : g'(xo)d E - Kg~xo),h'(xo)d = 0} of F at Xo such that

lid(x) - (x - x0)ll--= o(llx - x011)

f o r all f e a s i b l e x.

(11.7)

Suppose there are multipliers w * E W* and v * E K § with v*g(xo)=O and, further, positive reals [3, ~ such that f'(Xo) + v* . g'(Xo) + w * . h'(xo) = O,

(f"(Xo) + v*g"(Xo) + w*h"(Xo))(d, d) >1~lldll 2 for all d E {y ~ X ] g'(x0)y E - Kg~xo),h'(xo)y = 0, v*" g'(x0)y/> - [31lyl]}.

Then xo is strictly optimal. For conditions guaranteeing (11.7) see Maurer and Z o w e [25]. In [24] the relation of the a b o v e condition to well-known sufficient second order conditions in control theory is studied.

A. Ben-Tal and J. Zowe/ First and second order conditions

72

12. S u m m a r y of n e c e s s a r y a n d sufficient c o n d i t i o n s

N e c e s s a r y c o n d i t i o n s f o r the general p r o b l e m (P) x0 is a n o p t i m a l s o l u t i o n f o r (P) Statement

Source

Remark

Theorem 2.1

If CQ(d) holds, then It is not the zerofunctional (Remark 2 in Section 4).

Theorem 8.2

X, W Banach spaces, h'(xo)X closed.

Nondifferentiable case For every d ~ D1(x0) N Dg(xo) O Th(xo) for which f, g and h are d-regular there are continuous linear functionals lr ~EA(Q1(x0, d)),

lg E A(Qg(xo, d)),

Ih E A(Vh(xo, d)),

not all zero, such that l~+ Ig + lh = 0, 8"(1~ I (Qf(xo, d)) + ,~*(lg I Qg(xo, d)) + 8*(In I Vh(Xo, d)) <~O.

Differentiable case For every d satisfying

['(xo)d <=O, g'(xo)d ~ - cone(K + g(x0)), h'(xo)d = O, If CQ(d) holds, then u* is not the zero functional (Corollary 8.3).

there are continuous linear functionals u* E C § v* ~ K § w* E W*, not all zero, such that

u*['(xo)d = O, v*g(xo) = O, v*g'(xo)d = O, u*f'(xo) + v*g'(xo) + w*h'(xo) = O, (u*f"(xo) + v*g"(xo) + w*h"(xo))(d, d) >~O.

c o n e ( K + g(xo)) = {•(k + g(xo)) [ k E K , )~ >! 0},

Kg(xo) = cl c o n e ( K + g(xo)), Kg(xo), g'~xo~d= cl c o n e ( Kg(xo) + g '( xo)d ), CQ(d)

Qg(x0, d ) n Vh(Xo, d) ~ t~, ' h ' ( x 0 ) is o n t o a n d t h e r e e x i s t s z E X s u c h t h a t ,

CQ(d)

g'(xo)z + g"(Xo)(d, d) E - i n t Kg(xo), g'~xo)d

(differentiable version),

h'(xo)z + h"(Xo)(d, d) = O. N e c e s s a r y c o n d i t i o n s f o r the i n f i n i t e - d i m e n s i o n a l m a t h e m a t i c a l p r o g r a m m i n g problem (MP) x0 is a n o p t i m a l s o l u t i o n f o r ( M P )

A. Ben-Tal and J. Zowe/ F/rst and second order conditions Statement

73

Source

Remark

Theorem 9.1

For d = 0 one obtains the DubovitskiiMilyutin theorem (see Corollary 9.2).

Nondifferentiable case For every

d E Ds(xo)n ( i ~ o D~(xo)) n Th(x0), for which f, h and gi, i E I(x0), are d-regular, there are continuous linear functionals

lh E A(V,(xo, d))

I! E A(Qi(x0, d)), /g, E A(Qg~(x0, d))

for i E J(xo, d),

not all zero, such that

If+

~'~ l~ + lh = O, i E J ( x o, d)

8*(lllQ,(xo, d))+

~,

~*(lg, lQ,,(xo, d))

i E J ( x 0, d)

+ 8"(1, I Vh(XO, d)) <-0. Differentiable case For every d satisfying

y'(xo)d <~0, g~(xo)d <<-0 for i E I(xo), h'(xo)d = 0 (*) there are real numbers y0 t> 0, y~/> 0 for i E ](xo, d)

Theorem 9.3

and a functional w* E W*, not all zero, such that yof'(x0)+

ylg~(xo)+ w*h'(xo) =0,

~ i E J ( x o, d)

(yof"(x0)+

,ig'[(xo)+w*h"(xo))(d,d)~O.

~ iEJ(x O, d)

If either CQ1 or CQ2 + CQ(d) for all d satisfying (*) holds, then there are multipliers y~ for i E I(xo) and w* • W* such that

f'(xo)+ ~

yig~(x0)+ w*h'(xo)=O,

iEl(x o)

yig'[(xo)+w*h'(xo))(d,d)>~O

(f"(x0)+ ~ i E l ( x o)

for all d satisfying

g~(xo)d = 0 for i E I(xo) and yi > 0, g~(xo)d <~0 for i E I(xo) and yl = 0, h'(xo)d = O.

Corollary 9.4

X, W Banach spaces h'(xo)X closed.

74

A. Ben-Tal and J. Zowe/ First and second order conditions

I(x0)

: = {i:

gi(xo)

= 0},

J(xo, d) : = {i E I(xo): g~(xo)d = 0},

x ~ (gkxo(X0)X, h'(xo)x) m a p s X o n t o R card1(xo)• W, ( d i m X < oo, d i m W <

CQ1

oo: Vgi(xo) f o r i ~ I(xo) a n d Vhj(xo) f o r all J a r e l i n e a r l y i n d e p e n d e n t CQ2

F o r e v e r y d s a t i s f y i n g (*) t h e r e is z E X s u c h t h a t g~(xo)z + g'[(xo)(d, d) = 0

f o r i E I(xo),

( d i m X
h'(xo)z + h"(Xo)(d, d) = 0 the second order constraint

qualification by Fiacco and McCormick). Sufficient c o n d i t i o n s f o r p r o b l e m (P) u n d e r differentiability a s s u m p t i o n s X0 is a f e a s i b l e p o i n t f o r (P)

Statement

Source

Remark

Finite-dimensional X x0 is strictly optimal if either of the two conditions holds: (i) There is no solution to the system f'(xo)d < O, g'(xo)d E - Kg(xo), h'(xo)d = O, d e 0

Theorem 11.1 Theorem fails for infinite-dimensional X (see Example 1 in Section 11). (*)

(ii) For every solution d of (*) there are multipliers u* ~ C § v* E K § w* E W* such that u*.f'(xo)d = O, v*g'(xo)d = O, v*g(xo) = O, u*['(xo) + v*g'(xo) + w*h'(x0) = 0, (u*f"(xo) + v*g"(xo) + w*h"(xo))(d, d) > O. Infinite-dimensional X x0 is strictly optimal if the approximation property holds (see below) and if there are multipliers w* E W* and v* ~ K § with v*g(xo) = 0 and further real/3 > 0, ~ > 0 such that f'(xo) + v*g'(xo) + w*h'(xo) = O,

Theorem 11.2 U =R. X, V, W Banach spaces Constraint qualification (see Approximation property below).

q"(xo) + v*g"(xo) + w*h"(xo))(d, d) ~ 811dll~ for all d E {y ~ X: g'(xo)y E - Kg(x0),h'(x0)y = 0, v*g'(x0)y -/3llyll}.

Kg(xo) = cl c o n e ( K + g(x0)) A p p r o x i m a t i o n p r o p e r t y ( h o l d s f o r f i n i t e - d i m e n s i o n a l X ) : t h e r e is a m a p x -~ d ( x ) of the feasible

set into the linear approximation

{d ~ X :

g'(xo)d E - K g ( x 0 ) ,

A. Ben-Tal and J. Zowe/ First and second order conditions h'(xo)d = 0}, o f t h e f e a s i b l e s e t at Xo, s u c h t h a t lid(x) - (x - xo)ll = o(llx -

75

xoll) f o r

all

f e a s i b l e x.

Acknowledgment T h e a u t h o r s a r e i n d e b t e d to o n e o f t h e r e f e r e e s f o r his m a n y v a l u a b l e c o m m e n t s and helpful suggestions.

References [1] M.S. Bazaraa and C.M. Shetty, Foundations of optimization, Lecture notes in economics and mathematical systems 122 (Springer, Berlin, 1976). [2] A. Ben-Tal, "Second order and related extremality conditions in nonlinear programming", Journal of Optimization Theory and Applications 31 (1980) 143-165. [3] A. Ben-Tal, "Second order theory of extremum problems", in: A.V. Fiacco and K. Kortanek, eds., Extremal methods and system analysis (Springer, Berlin, 1980) pp. 336-356. [4] V.G. Boltyanskii, "The methods of tents in the theory of extremal problems", Russian Mathematical Surveys 30 0975) 1-54. [5] J.M. Borwein, "Optimization with respect to partial orderings', Ph.D. Thesis, Oxford University, Jesus College (1974). [6] M.J. Cox, "On necessary conditions for relative minima", American Journal of Mathematics 66 (1944) 170-198. [7] A. Dubovitskii and A.A. Milyutin, "Extremum problems in the presence of restrictions", USSR Computational Mathematics and Mathematical Physics 5 0965) 1-80. [8] A.V. Fiacco and G.P. McCormick, Nonlinear programming: Sequential unconstrained minimization techniques (Wiley, New York, 1968). [9] J. Gauvin and J.W. Tolle, "Differential stability in nonlinear programming", SIAM Journal on Control and Optimization 15 (1977) 295-311. [10] I.V. Girsanov, Lectures on mathematical theory of extremum problems, Lecture notes in economics and mathematical systems 67 (Springer, Berlin, 1972). Ill] M. Guignard, "Generalized Kuhn-Tucker conditions for mathematical programming problems in a Banach space", SIAM Journal on Control 7 (1969) 232-241. [12] A. Halkin, "A satisfactory treatment of equality and operator constraints in the DubovitskiiMityutin optimization formalism", Journal of Optimization Theory and Applications 6 (1970) 138-149. [13] M.R. Hestenes, Optimization theory--The finite-dimensional case (Wiley, New York, 1975). [14] K.H. Hoffmann and H.F. Kornstaedt, "Higher order necessary conditions in abstract mathematical programming", Journal of Optimization Theory and Applications 26 (1978) 533-569. [15l R.B. Holmes, A course on optimization and best approximation, Lecture notes in mathematics 257 (Springer, Berlin, 1972). [16] A.D. Ioffe, "Necessary and sufficient conditions for a local minimum. 3: Second order conditions and augmented duality", SIAM Journal on Control and Optimization 17 (1979) 266-288. [17] A.D. Ioffe and V.M. Tikhomirov, "Duality of convex functions and extremum problems", Russian Mathematical Surveys 23 (1968) 53-123. [18] M.G. Krein and M. Rutman, "Linear operators leaving invariant a cone in a Banach space", American Mathematical Society Translations, Translation No. 26 (1950) 1-128. [19] J.P. Laurent, Optimisation et approximation (Herman, Paris, 1972). [20] F. Lempio and J. Zowe, "Higher order optimality conditions", in: B. Korte, ed., Operations research (North-Holland, Amsterdam, 1981).

76

A. Ben-Tal and .1. Zowe/ First and second order conditions

[21] A. Linnemann, "Notwendige Olatim/ilitatsbedingungen h6herer Ordnung fiir abstrakte Optimierungsprobleme und Anwendungen", Diplomarbeit, Bremen (1979). [22] D.G. Luenberger, Introduction to linear and nonlinear programming (Addison-Wesley, Reading, MA, 1973). [23] L.A. Lyusternik and V.I. Sobolev, Elements of functional analysis (Nauka, Moscow, 1965). [24] H. Maurer, "First and second order sufficient optimality conditions in mathematical programming and optimal control", Preprint 38, Universit~it Miinster (1979). [25] H. Maurer and J. Zowe, "First and second order necessary and sufficient optimality conditions for infinite-dimensional programming problems", Mathematical Programming 16 (1979) 98-110. [26] J.J. Moreau, "Fonctionelles convexes", Lecture Notes, S6minaire Equations aux d6riv6es parti~lles, Coll~ge de France (1966). [27] M.Z. Nashed, "Differentiability and related properties of nonlinear operators: some aspects of the role of differentials in nonlinear functional analysis", in: L.B. Rall, ed., Nonlinear functional analysis and applications (Academic Press, New York, 1971). [28] L.W. Neustadt, Optimization, a theory of necessary conditions (Princeton University Press, Princeton, NJ, 1976). [29] L.L. Penissi, "An indirect sufficiency proof for the problem of Lagrange with differential inequalities as added side conditions", Transactions of American Mathematical Society 74 (1953) 177-198. [30] B.N. Psenichnyi, Necessary conditions for an extremum (Marcel Dekker, New York, 1971). [31] H.H. Schaefer, Topological vector spaces (MacMillan, New York, 1966). [32] P.P. Varaiya, "Nonlinear programming in Banach spaces", SIAM Journal of Applied Mathematics 15 (1976) 285-293. [33] J. Zowe, "A remark on a regularity assumption for the mathematical programming problem in Banach spaces", Journal of Optimization Theory and Applications 25 (1978) 375-381.

Mathematical Programming Study 19 (1982) 77-100. North-Holland Publishing Company

CHARACTERIZATIONS OF OPTIMALITY WITHOUT CONSTRAINT QUALIFICATION FOR THE ABSTRACT CONVEX PROGRAM J.M. B O R W E I N * and H. W O L K O W I C Z * * Department of Mathematics, Dalhousie University, Halifax, N.S. BJH 4H8, Canada

Received 21 June 1979 Revised manuscript received 15 July 1980 We consider the general abstract convex program (P)

minimize [(x), subject to g(x) E - S,

where f is an extended convex functional on X, g : X ~ Y is S-convex, S is a closed convex cone and X and Y are topological linear spaces. We present primal and dual characterizations for (P). These characterizations are derived by reducing the problem to a standard Lagrange multiplier problem. Examples given include operator constrained problems as well as semiinfinite programming problems. Key words: Cone-convex, Locally Convex Topological Vector Space, Optimality Conditions, Subdifferential, Directional Derivative, Faithfully Convex, Lagrange Multipliers, Slater's Condition, Semi-infinite Programs.

1. Introduction W e c o n s i d e r the a b s t r a c t c o n v e x p r o g r a m minimize

[(x),

(P) s u b j e c t to

g(x) E-S,

w h e r e f : X ~ R U {oo} is c o n v e x , g : X -~ Y, X and Y are real locally c o n v e x (Hausdorff) spaces, S is a closed c o n v e x c o n e and g is S - c o n v e x . R e c e n t l y Ben-Israel et al. [3] have p r e s e n t e d a c h a r a c t e r i z a t i o n of optimality, without a constraint qualification, in the case that the constraint g is given b y the finite n u m b e r of real-valued constraints g k ( x ) < - O , k = 1 . . . . . m. T h e y relied heavily on the c o n v e x i t y properties of the f u n c t i o n s and, in particular, t h e y used the c o n e of directions o f c o n s t a n c y of the ' e q u a l i t y ' constraints (see e.g. A b r a m s and K e r z n e r [1]). M a n y p e o p l e h a v e c o n s i d e r e d optimality criteria for the a b s t r a c t p r o g r a m (P) (see e.g. H o l m e s [14], K u r c y u s z [15], Z o w e and K u r c y u s z [25], L u e n b e r g e r [16] and N e u s t a d t [17] and the r e f e r e n c e s therein). Their criteria required a constraint * Research partially supported by NRC A4493. Presently at the Department of Mathematics, Carnegie-Mellon University, Pittsburgh, PA, U.S.A. ** Research partially supported by NSERC A 3388. Presently at the Department of Mathematics, The University of Alberta, Edmonton, Alta., Canada. 77

78

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

qualification. Craven and Zlobec [8] have extended the work in [3] in order to get a characterization of optimality for (P) that does not require any constraint qualification. They, however, required the following assumptions: (i) the cone S has nonempty interior (this automatically guarantees that the dual cone has a compact base; thus, when they choose a compact subset of the dual cone, they might as well choose a base); (ii) the feasible set contains a relative radial (core) point; (iii) continuity and differentiability properties; and (iv) the infimum is attained. They also used the cone of constancy of the 'equality' constraints in a redundant manner. In this paper we give a characterization which avoids the above-mentioned assumptions. Rather than attempt to extend the results in [3], our results are based on reducing (P) so as to be able to apply the 'Standard Lagrange Multiplier theorem'. We then give two classes of optimality criteria. The organization of the paper is as follows. Section 2 presents several preliminary definitions and results. In particular, Lemma 2.2 finds a 'Slater point' for any compact subset of the 'nonequality' constraints; Lemma 2.3 and Corollary 2.1 characterize the existence of a compact base and Slater's condition; and Lemma 2.5 gives a dual relationship between the cone of subgradients and the linearizing cone. Section 3 presents the 'Standard Lagrange Multiplier theorem' for program (P) with the added constraint x E O, where 12 is a convex subset of X (see Theorem 3.1). Several different types of optimality criteria are given. These criteria use directional derivatives, subgradients and the Lagrangian function. Section 4 presents the complete characterization of optimality without any constraint qualification (see Theorem 4.1 and Corollary 4.1). These results are derived using the results in Section 3. Several corollaries are also given, including the result which leads to the BBZ conditions [3], (see Corollary 4.2 and the following remarks). Section 5 contains several examples and applications which illustrate the theory developed in the first four sections.

2. Preliminaries In this section we present some preliminary definitions and results needed in the sequel. We consider the c o n v e x p r o g r a m

(P)

minimize

/(x),

subject to g ( x ) ~ - S, where [ : X ~ R U {oo} is convex, g : X ~ Y , X and Y are real locally convex (Hausdorff) spaces, S is a closed convex cone in Y and g is S-convex, i.e. S + S C S, AS C S for all positive ~ and for all x, y in X,

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

g(hx + (1 - h)y) - hg(x) - (1 - h)g(y) ~ - S

79

for all 0 < ~, < 1.

We let

F = {x E X : g ( x ) ~ - S } denote the feasible set of (P). X * and Y* will denote the topological duals of X and Y respectively, both equipped with the w*-topology (see e.g. [19]). If K c X (resp. Y), then K 4 = {~b E X * (resp. Y*): ~bk -- 0 for all k ~ K} is the polar (cone) of K. When A C X * (resp. Y*) then its polar is A + = { x E X (resp. Y): ~bx --- 0 for all 4~ CA}. The polar is always a closed c o n v e x set. Moreover, K 44 = ~

K

(2.1)

and

,22, where cone K denotes the closure of the convex cone generated by K, K C X (or X*), K~ C X are closed c o n v e x cones and ~ 9 denotes finite sums (see e.g. [6, 10]). In addition, it is easy to see that qg(. ) is a real-valued c o n v e x function for each q E S +. We now choose ~ C S 4 so that ~ is a generating set for S 4, i.e. S4 = ~

~.

(2.3)

Note that (2.3) is equivalent to ~+ = S. The following l e m m a is clear. L e m m a 2.1. Let x E X and Q c S 4. Then

qg(x) <- 0 for all q E Q,

if and only if g(x) E - Q+,

(2.4)

qg(x) <- 0 for all q ~ ~,

if and only if x E F.

(2.5)

Thus ~ m a y be considered the indexing set for the (convex) constraints

gq(x)=qg(x)<-O,

qE~.

Furthermore, we denote the partial feasible sets by

F ~ = {x E X : qg(x) <-0 for all q E Q}. The set of binding constraints at a ~ F, with respect to a set Q c 8 4, is Q ( a ) = {q E Q: qg(a) = 0}, An important subset of P, independent of x, is the equality set

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

80

= = {q E ~ : qg(x) = 0 for all x ~ F). This is the set of indices q for which the constraint qg(x) vanishes on the entire feasible set. The set ~= was used in [1, 3] to characterize optimality for the convex program (P), with a finite number of constraints. It was then used by Craven and Zlobec [8] in their optimality criteria for the program (P). We now let ~<= ~\~=. We will need the following property of ~<. As mentioned above, we shall assume that Y* has the w*-topology. Lemma 2.2. Suppose that K C ~< is compact in Y*. Then there exists ~ E X

such that 2~F

and

qg(:2)
Proof. For x E F, let U(x) = {q ~ Y*: qg(x)<0}. By continuity of the linear functionals g(x), U(x) is open. Moreover, K C U x ~ U(x), since K C ~<. Therefore, by compactness, we can find x~, x2..... x, E F such that

K C 0 U(x~). i=1

Let --

n

=

Xi.

Then ~ E F and

I

"

<-n ~ qg(xi) <0

by convexity,

for a l l q E K ,

since x~ is feasible for each i and qg(xi)< 0 for at least one i. The next lemma is essentially as given in [24]. We have added extra details for our applications. Note that or(Y*, Y) denotes the w*-topology on Y*; ~-(Y, Y*) is the topology on Y of uniform convergence on tr(Y*, Y) compact, convex, symmetric subsets of Y*; U ~ is the bipolar U~

{y: lyu]--- 1 for all u E U};

J.M. Borwein and H. Wolkowicz/Characterizations of optimality

81

and we call Q a base for S § if Q is closed, convex, 0 ~ Q and S§

U hQ. ~>_o

L e m m a 2.3. (a) If 0 # Y0 E 1"- int S, then

Q = S § A yo~{1} is a tr-compact (convex) base .for S § (b) If Q is a tr-compact (convex) base for S § and y0 E Y satisfies 0<Sl-
for all q E Q,

(2.6)

[or some 8l > O, then y0+ ~IQ ~

y0+ U

is a ,r-neighbourhood of Y0 inside S. In particular, such a y0 always exists. Thus S has nonempty ,r-interior if and only if S § has a g-compact base. Proof. (a) Let s + # 0 lie in S § Since y0 is nonzero and in z - i n t S , we get s+y0> 0. Thus, since Q is g-closed and convex, it is a base for S § Furthermore, if we set V = (y0 - S) fq (S - y0), then V is a T-neighbourhood of 0 and, for q E Q and v E V, we get - 1 = -q(yo)

<- q ( v ) <- q ( y o ) = 1.

Thus, Q c V ~ and the Banach-Alaoglu theorem [19] implies that V ~ and therefore also Q is tr-compact. (b) By (2.6), it follows that y0+ 61Q~ Q+= S ++= S. Moreover, Q~ is a ~--neighbourhood of O. This relies on the fact that the symmetric hull of Q is still compact and has the same polar. That such a Y0 always exists follows since Q is g-compact, convex and does not contain O. We will use the above lemma in the form given in part (a) of the following corollary. The (b) part of the corollary is included to emphasize the equivalence of Slater's condition (2.8) and the semi-infinite condition (2.7). This has not always been clear in the literature [6, 24]. Note that the ~--topology is the strongest topology on Y for which Y* is still the topological dual of Y (see [19]). Thus, for most purposes, we could have given Y the T-topology from the start. Corollary 2.1. Suppose that Q is tr-compact and a generating set for S § i.e.

S § =c--o--~ Q; and either Q is convex or Y * is quasicomplete (which holds if Y is barrelled).

82

J.M. Borwein and H. Wolkowiczl Characterizations of optimality

(a) If

qg(fO < 0 for all q E Q,

(2.7)

then .r - int S ~ O and -g(~)

(2,8)

E 9 - int S.

(b) Conversely, if (2.8) holds and g(~) is nonzero, then Q = S+n{q: qg(i)=-l}

is a g-compact (convex) base for S + for which (2.7) holds. Proof. (a) Since S § = c ~ Q (and, if Q is not convex, Y* is cr-quasicomplete), we get that D = conv Q is or-compact (see [14, p. 61]); S += ~ D ; and, by (2.7), - 8 = sup dg(~) = sup qg(fO < O. dED

qEQ

In particular, 0 ~ D and D N{q: q g ( ~ ) = - 8 } result now follows from L e m m a 2.3(b). (b) Follows directly from L e m m a 2.3(a).

is a tr-compact base for S +. The

Note that e v e r y Banach space and in fact every Fr6chet space is barrelled. In these cases, Y* is quasicomplete (see e.g. [19; 14, p. 135]). Now, suppose that h : X - ~ R U {oo} is a convex function. Following [3], for each of the relations 'relation' is ' = ' , ' < ' , ' - ' , ' < ' or ' - ' , we define D~elati~

= (d E X : there exists t~ > 0 with

h(x + ad) 'relation' h(x) for all 0 < a -< c~}. These are the cones of directions of constancy, decrease, nonincrease, increase and nondecrease respectively. For simplicity of notation, we let D'relation'- x 'relation' q iX) = Dqg (x) D~elation'(x) =

n Ltqr"'relati~ qEQ

Furthermore, if Q c S § we let

go(x) = sup qg(x) q~Q

and set

D(x) = Dg~=(x).

f o r q E Y*,

for Q c Y*.

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

83

The following l e m m a gives several useful properties of the 'uniform' function go. Note that dom(h) denotes the points at which the function h is finite. Lemma 2.4. If Q c S § is compact, then: (a) go is convex and dom(g o) = X (in particular, gq is continuous on lines); (b) if in addition g is 'weakly' lower semicontinuous (i.e. qg is lower semicon-

tinuous for each q in Q) and X is barrelled, then g is continuous. Proof. (a) For 0 <- X -< 1, m a x qg(Axl + (1 - ~,)x2) -< max(Aqg(xl) + (1 - A)qg(x2)) Q O

by convexity,

_< )t max qg(xO + (1 - A) m a x qg(x2). Q O This implies that go is convex. T h a t dom(go) = X follows from c o m p a c t n e s s of

Q. (b) Since each qg( 9) is lower semicontinuous, it follows that go( " ) is lower semicontinuous, as it is the s u p r e m u m of such functions. The result now follows since any lower semicontinuous convex function is continuous at an interior point of its domain, which in this case is the whole of X (see [9]). Following Rockafellar [21], we say that a c o n v e x function h : X ~ R is faithfully convex if: h is affine on a line segment only if it is affine on the whole line containing that segment. If X = R", then it follows f r o m results in [21], that the cone of directions of c o n s t a n c y at x, D~(x), is a subspace independent of x E R" (see e.g. [3]). This has been shown directly for X a locally convex space in [23], if h is continuous. If qg is faithfully convex for each q E Q c S § then it is clear that

Dgq~o~(a) 3 D ~ ( a ) . (Note that if h is analytic, then h is faithfully convex. Thus, if g is 'weakly analytic' i.e. qg is analytic for all q E Y*, then qg is faithfully convex for all q E S + (see e.g. [21]).) Various e x a m p l e s of these cones of constancy are given in Section 5. For a c o n v e x function h : X ~ R U {oo}, we let Vh(x; d) = lim h(x + t d ) - h(x) t~o t denote the directional derivative of h at x in the direction d, while ah(x) = {4~ E X*: h(z) >- h(x) + cb(z - x) for all z E X} denotes the subdifferential of h at x. If h(x) is finite, then V h ( x ; d) exists, for all d, although it m a y be plus or minus infinity. Moreover, if h is continuous at x,

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

84

then ah(x) is a nonempty, convex compact subset of X* and Vh(x; d) = max{~bd: ~b E 0h(x)}.

(2.9)

For two convex functions hi and h2, for which there is a point at which one is finite and the other continuous, we get that

a(alhl + a2h2)(x) = alahl(X) + a2ah2(x)

for all x,

(2.10)

where al and a2 are nonnegative real scalars (see e.g. [14]). For Q c S § the linearizing cone at x E F, with respect to Q, is

Co(x)={d E X : r

<-0 for all 4~ E aqg(x) and all q U Q}.

If qg( 9) is continuous at x for all q E Q, then (2.9) implies that Co(x) = {d E X: Vqg(x; d) <-:0 for all q ~ Q}.

(2.11)

The cone o[ subgradients at x is

Bo(x) = cone U oqg(x). q~Q

In the continuous case, (2.10) implies that

Bo(x) = cone

U

aqg(x).

(2.12)

q Econv Q

The linearizing cone and the cone of subgradients have the following dual property. Lemma 2.5. Suppose that Q c S +. Then BQ(X) = -CO(x).

(2.13)

Proof. Since -C~q)(x) = aqg(x) ++ by definition, = B{q}(x)

by (2.1),

we conclude, as C{q}(X) is always closed, that

- C ~ ( x ) = - ~ C~q}(x) by (2.2), qEQ

= BQ(X). (Note that ~ 9 denotes finite sums. Moreover, we did not require continuity of

qg(" ).) This lemma will provide the link between optimality criteria which use subditferentials, and criteria which use directional derivatives.

J.M. Borwein and H. Wolkowicz/Characterizations of optimality

85

If g has a Gateaux-derivative at x, it is easy to see that the linearizing cone

Co(x) = {d E X: Vg(x)d E -Q+}.

(2.14)

This will, of course, still hold if g only has a one-sided derivative. More specifically we have the following characterization of the linearizing cone for binding constraints. Lemma 2.6. Suppose g is Gateaux differentiable at x and that Q generates S+(x) (Q++= S+(x)). Then

Co(x) = {d E X: Vg(x)d E - c o n e ( S + g(x))}. Proof. Since Q§ = (S§

(2.15)

+ it suffices by (2.14) to observe that

S+(x) = (S + g(x)) +. The result then follows on taking polars as in (2.1). Note that since S is convex, cone(S + g(x)) = T(S, - g ( x ) ) , where T ( S , - g ( x ) ) is the cone of tangent directions to S at - g ( x ) (see e.g. [11, 13, 14]).

3. Characterization of optimality with constraint qualification Our results are based on the following 'Standard Lagrange Multiplier theorem'. Recall that if Slater's condition (3.2) below holds, then we can choose the 'generating set' ~ compact and have cone ~ = S + (see Corollary 2.1). Moreover, we then have cone ~ ( a ) = S+(a).

Theorem 3.1. Consider the convex program

(p)

minimize subject to

f(x), g(x) C - S a n d x E ~ ,

where f, g and S are as above and ~ is a convex subset of X. Let F = g-~(- S) n ~ denote the feasible set of (P); let /~0 = inf{f(x): g(x) E - S and x E ~}

(3.1)

be the (finite) solution value of (P); and further, suppose that there exists E dom f n ~ n g - l ( - i n t S)

(Slater's condition).

(3.2)

86

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

Then, there exists s + ~ S + such that

/~0 = inf{f(x) + s+g(x): x E O}.

(3.3)

If in addition a E F N dom f, then we get the following four statements (i)-(iv) of optimality and their mutual relationships.

if (i) a solves (P), then (ii) there exists s +E S + such that f ( a ) = inf{f(x)+ s+g(x): x E O}

(3.4)

s +g(a) = 0

(3.5)

(complementary slackness);

which implies (iii) (a) that there exists s +E S+(a) such that the system

Vf(a;d)<0,

Vs+g(a;d)-
dEcone(~-a),

(3.6)

is inconsistent, or equivalently (with Slater' s condition), (b) the system

Vf(a;d)<0,

Vs+g(a;d)<-O

foralls+~S+(a),

dEcone(O-a),

is inconsistent, or equivalently (with Slater's condition, cone ~ ( a ) = S+(a), and q g ( . ) continuous at a for each q E ~(a)),

(c) D r ( a ) n C~a~(a) n cone(O - a) = ~,

(3.7)

which then entails (iv) (with qg( . ) continuous at a for each q E ~ ( a ) and f continuous at a) (a) (with cone ~ ( a ) = S+(a))

af(a) n (-B~,(,~(a) + (l~ - a) § # fJ, or equivalently

(b) OE a f ( a ) + as+g(a)-(12 - a) § for some s+E S+(a).

(3.8)

Conversely, without Slater's condition in the hypothesis, we have that (iv) implies both (iii) and (ii), each of which then imply (i). Hence, when Slater's condition holds and both f and g are continuous at a, we get that (i) through (iv) are equivalent.

Proot. The proof of (3.3) and of (i) implies (ii) follows from [16, p. 217], with

J.M. Borwein

and H. Wolkowicz/

Characterizations

of optimality

87

TaMe 1. H y p o t h e s e s r e q u i r e d in T h e o r e m 3.1 implies implied by

a optimal

Lagrangian

Directional derivatives

(i)

(ii)

(iii)(a)

(iii)(b)

(iii)(r

(iv)(a)

(iv)(b)

Slater's

Slater's

Slater's

Slater's qg cont.

Slater's qg cont. f cont.

Slater's qg cont. [ cont.

(i)

subdifferentials

c~q = S

c~a = S

(ii)

qg cont. c~a = S

qg cont. f cont.

qg cont. f cont.

c~a = S

Slater's (iii)(a)

gq cont. cga = S

Slater's qg cont. f cont.

Slater's qg cont. f cont.

c~a = S

Slater's (iii)(b)

Slater's

qg cont.

Slater's qgcont, f cont.

c~a = S

c~a = S

Slater's gq cont.

Slater's qg cont. f cont.

Slater's

(iii)(c)

Slater's qg cont.

f cont.

Slater's qg cont.

f cont.

c~a = S

(iv)(a)

qg cont. c~a = S

(iv)(b) c8~a = S

qg cont. c~a = S

Slater's: Slater's condition q g cont.: q g ( 9) c o n t i n u o u s at a, for all q E ~ ( a )

f cont.: f c o n t i n u o u s at a. c ~ a = S: c o n e ~ ( a ) = S + ( a ) .

12 t-1 dom f replacing 12. Note that the proof is valid if we consider Y with the z-topology, since its topological dual remains unchanged. Table 1 gives the hypotheses required in the proof of Theorem 3.1. (ii) ~ (iii)(a): Suppose that d E cone(12- a). Then there exists [ > 0 such that a + td E 12, for all 0 < t -< t-. Therefore, (3.4) implies that t-l[f(a + t d ) - f(a)] + t-l[s+g(a + t d ) - s§ for all 0 < t -< {. Now lim t-~[f(a + t d ) - f(a)] = Vf(a; d) tJ, o

and lira t-l[s +g(a + t d ) - s +g(a)] = Vs +g(a; d)

>-O,

88

J.M. Borwein and H. Wolkowiczl Characterizations of. optimality

both exist since p~ = f ( a ) and s+g(a) are finite. Thus ~Tf(a ; d) + Vs +g(a; d) >>-0

(with the proviso that - ~ + ~-> 0). We now conclude that (3.6) is inconsistent. (Note that s § ~ S§ by (3.5).) (iii)(a) ~ (iii)(b): Obvious. (iii)(b) ~ (iii)(c): Let us first show that (3.12)

C~a~(a) = Cs+~a~(a).

Recall that S+(a)= cone ~(a). That Cs+(a}(a) C C~(a~(a) is clear. To prove the converse, suppose that d ~ C ~ a ~ ( a ) and s + E S + ( a ) = c o n e ~ ( a ) . Then s += ~,~=1 aiqi for some ai -> 0 and qi E ~ ( a ) and Vs+g(a; d) = sup{~bd: ~b ~ as+g(a)}

by (2.9),

= sup{~bd: ~b c 0 ~ a,q,g(a)} sup{~bd: ~b E ~ aicgq,g(o)}

by (2.10),

-< ~ ai sup{~bid: rki ~ aqig(a)} = ~, aiVqig(a ; d) <- O.

This implies that Ceone~(a)(a)3 C~a>(a). The result now follows since D?(a) ={d: Vf(a; d ) < 0 }

(see e.g. [5]).

(iii)(c)ff (iii)(a): Suppose that (3.7) holds but a is not optimal. Then there exists a feasible point x such that f ( x ) < f(a). Thus d = x - a E D~(a) t3 C~t~(a) N cone(O - a),

(3.13)

which is a contradiction. Therefore a solves (P). We have seen that this, with Slater's condition, implies part (a). (iii) ~ (iv)(b): From the above proof, we have that (iii) f f (i) ~ (ii). Therefore, there exists s +E S§ such that f ( a ) =/~0 = inf{f(x) + s +g(x): x E O}

and Pshenichnii's condition [14, p. 87] implies that 0 E O(f + s+g)(a) - ( 0 - a) +,

which by (2.10) yields part (b). Note that if D = X, then f need not be assumed continuous. (iv)(b) ~ (iv)(a): It is sufficient to show that Os+g(a) C B ~ ( a ) .

(3.14)

J.M. Borwein and H, Wolkowicz] Characterizations of optimatity

89

This is clear since s + ~ S+(a)= cone ~ ( a ) and q g ( . ) is continuous at a for all q E ~(a). (iv)(a) ~ (iv)(b): Let ~b E B ~ ) ( a ) . It is sufficient to show that

cb E ~s§ g(a)

for some s § E S+(a).

N o w ~b = ~/~=1 ai4~i for some ai >- O, ~pi E apig(a) and pi E ~ ( a ) . Therefore

4~i(x - a) <- pig(x) - pig(a) for all x ~ X and all i = 1. . . . . k. This implies that

~_, ai6i(x - a) <--~'~ otipi(g(x) - g(a)), i.e. ~ ai~.bi ~- O(~, aipi)g(a). ~Ve can now choose s + = ~ aipi. (iv)(b) f f (ii): This follows f r o m the Pshenichnii condition. (iii) ~ (i): See the proof of (iii)(c) ~ (iii)(a).

4. Characterization of optimality without constraint qualification Recall that ~ is a generating set for S + and ~ = ~ = U ~ < . We now divide up the set ~ < so that we can apply the 'Standard Lagrange Multiplier theorem'. Choose the set Q c ~<

(4.1)

c o m p a c t in Y* and let ~ be a remainder, i.e. S + ~ ~ 3 ~<\Q.

(4.2)

In order to apply Corollary 2.1, we will assume throughout that either Q is c o n v e x or Y is barrelled and (cone Q ) ( a ) = cone(Q(a)). The notion of dividing up the indexing set of constraints was previously used in [7, 81. Recall that, for Y/C ~ , we denote the 'partial feasible sets"

F a = {x E X : q g ( x ) - < 0 for all q E O} and the 'uniform' function

gn(x) =

sup

OEFI

qg(x).

Note that if ga(x) = 0, then

D~a(a ) = c o n e ( F ~ - a).

(4.3)

The following theorem and corollary characterize optimality for (P), without any constraint qualification. One m a y of course choose ~ convex and closed. Under our assumptions conv Q is still c o m p a c t and contained in ~<. Thus, to

J.M. Borwein and H. Wolkowicz[Characterizations of optimality

90

simplify reading the t h e o r e m and its p r o o f , the reader m a y a s s u m e that b o t h and Q are c o n v e x and thus delete c o n v and replace c o n e b y cone. If Q is finite dimensional, the situation is even simpler since c o n v Q is a l w a y s c o m p a c t . Theorem 4.1. Consider the original convex program (~). Let the sets Q and ~t be

as above. Let /x0 = inf{f(x): g(x ) ~ - S} ~ oo

(4.4)

be the (finite) solution value o f (P) and assume that d o m F ~ F. Then there exists q E cone Q such that /~0 = inf{f(x) + qg(x): x E F ~ n F ~ } .

If in addition a E F n d o m f (and for simplicity qg( 9) is continuous at a for each q ~ Q ( a ) ; and f ( . ) is continuous at a), then the following four statements (i)-(iv) are equivalent: (iii) (i) a solves (P); (ii) there exists q ~ c o n e Q such that f ( a ) = inf{f(x) + qg(x): x E F ~ n F ~ } ,

qg(a) = 0; (iii) (a) there exists q ~ cone Q(a) such that

the system Vf(a;d)~O,

Vqg(a;d)<-O,

dED(a) ncone(F ~-a)

is inconsistent, or equivalently (b) the system V f ( a ; d) ~ O, Vqg(a ; d) <- 0 for all q ~ c o n e ( Q ( a ) ) , d E D ( a ) N c o n e ( F ~ - a)

is inconsistent, or equivalently (c) D~(a) n C .... (o(a))(a) n D(a) N c o n e ( F ~ - a) -- 0; (iv) (a)

af(a) n (-Bconvto(a))(a) + (D(a) n c o n e ( F ~ - a)) § ~ 0, or equivalently

(b) 0 ~ Of(a) + Oqg(a) - (D(a) n c o n e ( F ~ - a)) § for some q ~ c o n e Q(a).

J.M. Borwein and H. Wolkowiczl Characterizations of optimality

91

Proof. By L e m m a 2.1, we can rewrite (P) as: minimize

/(x),

subject to

qg(x)<-O

for all q ~ Q,

qg(x) <- 0

for all q ~ ~ ,

qg(x)<-O

for a l l q E ~ = .

(4.5)

This is equivalent to minimize

f(x),

subject to

g(x)c:_-Q+ and x E F ~~ n F ~.

This problem now satisfies all the conditions of the 'Standard Lagrange Multiplier t h e o r e m ' , i.e. g is Q*-convex since S = S +* c Q*, F ~- n F ~ is convex since S C ~ + and S C ~=+; and L e m m a 2.2 and Corollary 2.1 imply that there exists E F n dom f with 0 ~ g(~) E - int(Q+). We can now apply T h e o r e m 3.1 with f~ = F ~ - N F ~ ; S replaced b y Q+; and S + replaced by cone Q. (Note that 0~convQ by L e m m a 2.2; c o n e c o n v Q is closed; and c o n e ( Q ( a ) ) = (cone Q)(a).) It only remains to show that

(D(a) n c o n e ( F ~ - a)) § = ((F ~- n F ~) - a) +.

(4.6)

In fact, let us show the stronger statement D ( a ) O c o n e ( F ~ - a) = D g ~ ( a ) n c o n e ( F ~ - a) = c o n e ( ( F ~~ n F ~) - a).

(4.7)

That the inclusion C holds a b o v e is clear since D ( a ) C c o n e ( F ~'" - a), (4.3) holds and since for c o n v e x sets A~, A2 containing a, c o n e ( A ~ - a ) n c o n e ( A 2 - a ) agrees with c o n e ( A t O A 2 - a ) . T o p r o v e the converse, suppose that d E cone((F ~; n F ~) - a). Then d E c o n e ( F ~ - a) = Dg~~ and we need only show that

d E D(a).

(4.8)

Let x = a + a d E F ~" N F ~, ot > 0, and choose ~ to satisfy

EF

and

g(~) E - i n t Q+.

(4.9)

This ~ always exists by L e m m a 2.2 and Corollary 2.1 and m o r e o v e r satisfies go(~) < 0.

(4.10)

N o w let xA=,~x+(1-X)~

for0-<~,
Then, by convexity, xA E / F ~- n F ~ and

go(x~) < 0

for small ~, > 0

(see L e m m a 2.4(a)). Thus x, E F for small ~, > 0, which implies that g~-(x,) = 0

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

92

for small A > 0. Now, since g~=(xx) <-0 for all 0 <- )t -< 1, and g~,~ is convex, we must have g~:(xx) = 0 for all 0 -< h -< 1. This means that g~:(x) = 0 and d ~ D(a).

Corollary 4.1. The above theorem holds if we relax the restrictions on the multipliers, i.e. if, in the statements (ii), (iii), (iv), cone Q is replaced by S* and both cone Q(a) and conv Q(a) are replaced by S§ Proof. Existence of q ~ cone Q implies existence of q E S § since Q c ~. The converse follows since the statements (ii), (iii), (iv), after the replacements, still imply (i). The above theorem and corollary characterize optimality for the convex program (P) without using any constraint qualification. Craven and Zlobec [8] have presented similar optimality conditions for (P). They did not, however, use a reduction to Slater's condition and required that the infimum be attained, int S ~ ~b and F posses a 'radial point', i.e. that the intrinsic core of F be nonempty. Moreover, the set D(a) is redundant in their optimality conditions. In the following corollary we ~ee that the set ~ may become redundant. We shall see that this leads to the so-called BBZ conditions in the case that S is polyhedral.

Corollary 4.2. In the above theorem (and corollary), suppose that aEintdomg~

and

g~(a)<0,

(4.11)

or

C c o n e ( ~ ~ U Q).

(4.12)

Then ~ (and also F ~ and c o n e ( F ~ - a)) can be omitted, thus simplifying the optimality criteria. (Note that any subset of ~ that satisfies (4.11) or (4.12) may similarly be omitted in the previous result.) Proof. Only (4.12) requires proof. Since x~F

if and only if

g(x) E - ~ +

by L e m m a 2.1,

if and only if

g(x) ~ - ( ~ = U Q)§

by hypothesis,

we see that ~ is redundant in (4.5) and thus redundant in the rest of the proof of Theorem 4.1.

Corollary 4.3. D(a) n c o n e ( F ~ - a) = {d: there exists ~ > 0 with qg(a + ad) = (or <-) qg(a) for all q E ~ = and all O
J.M. Borwein and H. Wolkowicz[Characterizations of optimality D ( a ) is convex if ~ can be omitted

93 (4.14)

ProoL Let us call the first cone on the right-hand side of (4.13) D"(a). It is immediate that

D " ( a ) C Dg~=(a). Conversely, if d lies in Dg, o ( a ) n c o n e ( F ~ - a), an examination of the proof of T h e o r e m 4.1 shows that the point xx constructed below (4.10) is feasible and hence satisfies

qg(xA)=O,

x~=h(a+ad)+(1-h)~

for all q in ~ = . Exactly as before it follows that qg(a + ad) = 0 for all a in ~ = and d lies in D~(a). Since d lies in c o n e ( F ~ - a) by (4.7) we have the desired equality. M o r e o v e r , (4.14) follows f r o m (4.7) since D ~ = ( a ) is convex. Thus we can modify the uniformity of the directions of constancy (or nonincrease). In the case that q g ( . ) is faithfully convex, for all but a finite n u m b e r of q E ~ = , we see that (4.13) implies that

D ( a ) n c o n e ( F ~ - a) = D ~ ( a ) n c o n e ( F ~ - a),

(4.15)

i.e. we can replace the uniform directions of c o n s t a n c y o v e r ~= by the intersection of the directions of constancy. N o w , in the case of a finite n u m b e r of constraints gk(x) <- O, k = 1. . . . . m, we m a y choose S = R~, the nonnegative orthant in R m, and g(x) = (gk(x)). We may now set the generating set ~ = {ei}?=l, the set of unit vectors in R m. The sets ~= and ~ < are therefore finite and so c o m p a c t . Moreover, by Corollary 4.2, we may choose Q = ~ < and ~ = 0, since ~ = ~ U ~<. First, we note that by (4.14) and (4.15),

D ~ ( a ) = D~=(a)

is convex.

(The set conv D~=(a) was needlessly considered in [3,4].) N e x t , our T h e o r e m 4.1 reduces to the so-called BBZ conditions [1, 3, 5]: a (feasible) is optimal if and only if af(a) n (-B~<(~)(a) + D(a) § ~ O. In addition, in the case that the infimum m a y not be attained, we get that

tzo = inf{f(x) + hg(x): x E F ~=}

for s o m e h ~ R 2 .

Another situation which is easily handled is the case of two cone constraints, in two spaces,

gl(x)~-SiC

Y1,

g 2 ( x ) ~ - - S 2 C "112,

where there exists a (feasible) Slater point ~ for the first constraint: gl(x) E - i n t S .

g2(x) • - $ 2 ,

J.M. Borwein and H. Wolkowicz/ Characterizations o~ optimality

94

while the second cone $2 is polyhedral, i.e. is the intersection of a finite number of half-spaces

$2 = {d E Y2: hid >- O, i = 1, 2 ..... m}. Then the cone ST has a compact base say ~t while ~2 = {hi}~"=tis a compact base for S +2. Furthermore, ~T = 0 and ~ 2 = {hi: higE(x) 0 for all feasible x}. Thus the abstract program (P) has =

S = S~x S2C Y = Y~x Y2 and the generating set for S § = S~ x S~ is = (~', u {o}) x ( ~ u {o}).

(Note that the set ~ t x ~2 is not a generating set for S § since it does not contain {0} x $2 nor St x {0}.) Corollary 4.1 now yields the characterization: a (feasible) is optimal if and only if Of(a)N ( - B e ( , ) + D ( a ) § ~ O, if and only if

0 E af(a) + 3sTgl(a) + Os~g2(a) - D ( a ) § for

some

s~fES~(a)

and

s~ ~ S~(a), where D ( a ) = D~=(a) = ie~- D~2(a)" Remark 4.1. To derive our characterizations of optimality in the above, we divided up the constraint set into three distinct parts, i.e.

~c~=uQu~, where Q c ~<, ~<\Q c ~ and Q is chosen compact in Y*. Corollary 4.2 further showed that we could throw away part (if not all) of ~ which satisfied (4.11) or (4.12). This led to the BBZ conditions in the case S was polyhedral and, in addition, gave a Lagrange multiplier relation in the case that the infimum may not be attained. If is also possible to throw away part (if not all) of ~=. This leads to 'stronger' optimality criteria as well as 'weakest constraint qualifications'. For instance, this covers situations where a Lagrange multiplier exists but ~ = ~ 0, i.e. Slater's condition fails. The details are given in [7]. The special case S = R~ is treated in [23]. Moreover, the case that S is a convex cone (not necessarily closed) in R '~ leads to very elegent characterizations of optimality without constraint qualification. These characterizations use the 'faces' of the cone S. These results are presented in a forthcoming paper.

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

95

5. Examples In this section we present several examples to illustrate the theory presented above. Example 5.1. This example first illustrates the fact that D ~ ( a ) r D ( a ) in general. (Recall that D~=(a) = i") q ~ - D~g(a) while D ( a ) = Dg~=(a).) We then see that choosing the generating set ~ in different ways can yield simpler results. In addition, the K u h n - T u c k e r conditions fail here, but we characterize optimality using T h e o r e m 4.1 (see Fig. 1). Consider the semi-infinite program minimize

f(x) = -x,

subject to

h(x, t) = max2(0, tx

(5.1) -

t 2) --<0

for all t E [0, 1].

For each x E R, g(x) = h(x, 9) is a c o n v e x continuous (differentiable) function on [0, 1], i.e. g(x) E C[O, 1]. N o w , let Y = C[0, 1] and set S = {nonnegative functions in Y}. Then we can rewrite the a b o v e semi-infinite program as the abstract convex program minimize

-x,

subject to

g(x) E - S .

(P) Let us choose the generating set = {~t : 0 ~ t -< 1},

(5.2)

the set of point measures in Y*, where Y* is the space of all Borel measures, i.e. ~,g(x) = h(x, t). Then cone ~ = S + and m o r e o v e r , Slater's condition fails. In fact

since s+g(x) --- 0, for all x E F and all (nonnegative) m e a s u r e s s + ~ S +. We now h(x,t) t=l

$7-"./ F, feasible set

O, optimum Fig. 1.

,=o

96

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

see that D ( 0 ) = D(0) + = R_, the negative real half-line, while D~3=(0)= R, the whole real line. F u r t h e r m o r e , our optimality conditions (Theorem 4.1) hold at the optimal point 0: 0 E a.f(0) + as +g(0) - (D(0)) § = - 1 - R_,

(5.3)

where we have chosen the multiplier s § = 0. L e t us now change our choice of the generating set ~. The functional y0--- 1 is in int S. Therefore, L e m m a 2.3 implies that = S + f3 yo~{1} = {probability measures} is a c o m p a c t (convex) base for S +. (Note that ~ = conv{~t: 0-< t-< 1}, by the K r e i n - M i l m a n t h e o r e m [14].) Then we actually have cone 9 = S + and, as before, f9 = = ~. But now D(0) = D~=(0) = R_, since Lebesgue m e a s u r e / x ( t ) is in ~= and DS,>(0) = R_. Note that the optimality condition 0 E Of(O) - D(0) +

still holds although the K u h n - T u c k e r conditions fail, i.e. 0 ~ a f ( 0 ) + as+gO)

for any s + ~ S+(0),

(5.4)

since a f ( 0 ) = {-1} while as+g(O)= {0} for all s + E S +.

Example 5.2. This example uses a linear operator constraint to illustrate that the K u h n - T u c k e r conditions m a y still hold at the optimum even though Slater's condition fails. N o w suppose that T is a bounded linear operator between the Banach spaces X and Y. L e t y # 0 be in ~ ( T * ) , the closure of the range of the adjoint of T in the w*-topology, and consider the program minimize

f ( x ) = yx,

subject to

Tx=b,

(5.5) xEX,

where b E ~ ( T ) . If we let g ( x ) = T x - b and S = {0}, then the a b o v e program is equivalent to the abstract convex program

(P)

minimize

yx,

subject to

g(x) ~-S.

Since int S = ~, we see that Slater's condition fails here. M o r e o v e r , S + = Y*. We now choose the generating set = B

the unit ball in Y*.

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

97

Then cone ~ = S + and ~ = = ~, since the feasible set F = ~ + N ( T ) , where ~ is any particular solution of Tx = b and JV(T) denotes the nullspace of T. Moreover, D(~) = 2((T) and B~(~)(~) = {y*T I Y* ~ Y*} = ~ ( T * ) as T is its own Fr6chet derivative. Therefore, Corollary 4.1 implies that ~ is optimal if and only if 0 e af(~) + B~(~(~) - D(~) + = y + ~ ( T * ) - Y ( T ) +

= y + ~(T*), by the Fredholm alternative. This is a special case of (2.13). Thus every feasible point ~ is optimal, since we have assumed y E ~ ( T * ) . If y ~ 3~(T*), then the problem is unbounded. If y ~ ~ ( T * ) , then the K u h n Tucker conditions hold at the optimum: 0 E af(2) + B~(~)(2) = y + ~ ( T * ) , while if y E 3 ~ ( T * ) \ ~ ( T * ) , then the K u h n - T u c k e r conditions fail. Note that even the Fritz John conditions: (5.6)

0 ~ O)tf(~) + Os+g(~)

for some h - > 0 and s+E S+(a) not both zero, may fail at the optimum. For suppose y E ~ ( T * ) \ ~ ( T * ) and ~ ( T ) = Y. (In the Banach space setting ~ ( T ) is closed exactly when ~ ( T * ) is (weak*) closed and so such a y exists exactly when T is 1-1 with dense nonsurjective range. Consider ( T x ) , = x,/n in 12 and take y = (I/n) as a specific example.) Then the Fritz John conditions fail if ,~ > 0, since the K u h n - T u c k e r conditions fail. This implies that necessarily h = 0. But Os+g(~)={s+T}~{O}

for a l l s + # 0 i n

Y*,

(5.7)

since ~ ( T ) = X, i.e. since T* is 1-1. Note that the Fritz John conditions do hold when T* is not 1-1, since we can then choose s + ~ 0 in N ( T * ) .

Example 5.3. This example shows that one should be careful when choosing the generating set ~. More precisely, we see that the assumption cone ~ ( a ) = S+(a) is needed in T h e o r e m 3.1. We again consider a semi-infinite program minimize

f ( x ) = x,

subject to

h(x,t)=tx 2-t-x<-O

for all t E [0,1],

and let g ( x ) = h(x, .) so as to formulate an abstract c o n v e x program (P). (See Example 5.1 for the definitions of X, Y, S, etc.) Note that Slater's condition i s satisfied: g(1) = - 1 E int - S and 0 is the optimal point. We first choose the generating set = { 6 t : O < t < 1}.

J.M. Borwein and H. Wolkowicz/Characterizations of optimality

98

Then ~ ( 0 ) = ~ and the optimality conditions in Theorem 3.1(iv) fail at the optimal point 0, even though Slater's condition is satisfied. Note that cone P = S § but ~ = cone ~(0) ~ S+(0). However, if we choose = {c5,: 0 --< t -< 1}, then ~(0) = {0}, B~(0)(O) = R_ and the K u h n - T u c k e r conditions hold at O: 0 ~ 0f(0) + B~,(0)(0) = {1} + R_. Example 5.4. This example shows that part (iii) of Theorem 3.1 may characterize optimality when parts (ii) and (iv) fail to, i.e. the conditions involving directional derivatives are weaker than those involving subdifferentials. The directional derivatives here coincide with the homogeneous constraint and objective functions. Let K = {~b = (~bi) ~ R2: (bE+ (qb2- 1)2-< 1},

g(x) = sup{6x: 4) E K} = N / ~ +

x2,

S~R+. Consider the program minimize

f ( x ) = xt,

subject to

g(x) E - S .

(e)

Then the point 0 solves (P) and Slater's condition fails since g(x)>-O, for all x ~ R:. Now, T h e o r e m 4.1 yields the optimality condition: D~(O) N C~(o)(O) = ~. However,

O~ Of(O)+ 3s+g(O) for any s § ~ S+(0), since 0f(0) = (1, 0) while as+g(O) = s§ This shows that part (iii) of Theorem 3.1 characterizes optimality while parts (iii) and (iv) fail to. The reason for this is that the cone of subgradients B~(0)(0) is not closed and thus not equal to - (C~(o)(O)) +.

Example 5.5 This example illustrates Corollary 4.1. Consider the semi-infinite program (see Fig. 2) minimize

f(x) = -x,

subject to

h(x,t)<-O

for a l l t E { 1 , 8 9

where

h ( x , t ) = ( ~ tx -2t2-2t

if 0 - < x - < l + t , if l + t - < x .

.... },

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

99

h(x,t)

~

t=l ~ ~ ~ ~ t=1/2

F,feasibleset _

Fig. 2. F o r e a c h x E R, g ( x ) = h(x, 9) is a s e q u e n c e in l> So, let Y = 12 a n d set S = { n o n n e g a t i v e s e q u e n c e s in Y}. W e c a n n o w r e w r i t e the a b o v e p r o g r a m in the a b s t r a c t f o r m u l a t i o n minimize

y(x),

s u b j e c t to

g(x)E-S.

(P) L e t us c h o o s e t h e g e n e r a t i n g set ~ = { e i : i = 1 , 2 , 3 .... }, the set of c o o r d i n a t e f u n c t i o n s in 12. T h e n c o n e ~ = S § S. N o t e t h a t S l a t e r ' s c o n d i t i o n fails, s i n c e int S = ~, b u t N = = tt. T o a p p l y C o r o l l a r y 4.1, w e m u s t c h o o s e the c o m p a c t set Q = { e i : i = 1,2 . . . . . k } K ~ < =

~,

while the r e m a i n d e r ~={ei:i=k+l,l+2

.... }.

Thus ( F ~ - 1) + = c o n e ( F ~ - 1) -- R_ and our optimality conditions yield: 0 E 0f(1) + Os+g(1) - ( F ~ - 1) + = { - 1 } - R_, since S+(1) = {0}, as t h e r e are no b i n d i n g c o n s t r a i n t s .

References [1] R.A. Abrams and L. Kerzner, "A simplified test for optimality" Journal of Optimization Theory and Applications 25 (1978) 161-170. [2] M.S. Bazaraa, C.M. Shetty, J.J. Goode and M.Z. Nashed, "Nonlinear programming without differentiability in Banach spaces: Necessary and sufficient constraint qualification", Applicable Analysis 5 (1976) 165-173.

100

J.M. Borwein and H. Wolkowicz/ Characterizations of optimality

[3] A. Ben-Israel, A. Ben-Tal and S. Zlobec, "Optimality conditions in convex programming", in: A. Pr6kopa, ed., Survey of mathematical programming, Proceedings of the 9th International Mathematical Programming Symposium (The Hungarian Academy of Sciences, Budapest and North-Holland, Amsterdam, 1979) pp. 137-156. [4] A. Ben-Tal, A. Ben-Israel and S. Zlobec, "Characterization of optimality in convex programming without a constraint qualification", Journal of Optimization Theory and Applications 20 (1976) 417--437. [5] A. Ben-Tal and A. Ben-Israel, "Characterizations of optimality in convex programming: The nondifferentiable case", Applicable Analysis 9 (1979) 137-156. [6] J. Borwein, "Proper efficient points for maximizations with respect to cones", SIAM Journal on Control and Optimization 15 (1977) 57-63. [7] J. Borwein and H. Wolkowicz, "Characterizations of optimality without constraint qualification for the abstract convex program", Research Report 14, Dalhousie University, Halifax, N.S. [8] B.D. Craven and S. Zlobec, "Complete characterization of optimality of convex programming in Banach spaces", Applicable Analysis 11 (1980) 61-78. [9] M. Day, Normed linear spaces, 3rd ed. (Springer, Berlin, 1973). [10] I.V. Girsanov, Lectures on mathematical theory of extremum problems, Lecture notes in economics and mathematical systems 67 (Springer, Berlin, 1972). [11] F.J. Gould and J.W. Tolle, "Geometry of optimality conditions and constraint qualifications", Mathematical Programming 2 (1972) 1-18. [12] F.J. Gould and J.W. Tolle, "A necessary and sufficient qualification for constrained optimization", SIAM Journal on Applied Mathematics 20 (1971) 164-171. [13] M. Guignard, "Generalized Kuhn-Tucker conditions for mathematical programming problems in a Banach space", SIAM Journal on Control 7 (1969) 232-241. [14] R.B. Holmes, A course on optimization and best approximation, Lecture notes in mathematics 257 (Springer, Berlin, 1972). [15] S. Kurcyusz, "On the existence and nonexistence of Lagrange multipliers in Banach spaces", Journal of Optimization Theory and Applications 20 (1976) 81-110. [16] D.G. Luenberger, Optimization by vector space methods (J. Wiley, New York, 1969). [17] L.W. Neustadt, Optimization--A theory of necessary conditions (Princeton University Press, Princeton, N J, 1976). [18] B.N. Pshenichnyi, Necessary conditions for an extremum (Marcel Dekker, New York, 1971). [19] A.P. Robertson and W.J. Robertson, Topological vector spaces (Cambridge University Press, Cambridge, 1964). [20] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, N J, 1972). [21] R.T. Rockafellar, "Some convex programs whose duals are linearly constrained", in: J.B. Rosen, O.L. Mangasarian and K. Ritter, eds., Nonlinear programming (Academic Press, New York, 1970) pp. 393-322. [22] H. Wolkowicz, "Calculating the cone of directions of constancy", Journal of Optimization Theory and Applications 25 (1978) 451-457. [23] H. Wolkowicz, "Geometry of optimality conditions and constraint qualifications: The convex case", Mathematical Programming 19 (1980) 32-60. [24] J. Zowe, "Linear maps majorized by a sublinear map", Archly der Mathematik 26 (1975) 637-645. [25] J. Zowe and S. Kurcyusz, "Regularity and stability for the mathematical programming problem in Banach spaces", Preprint No. 37 Wiirzburg (1978).

Mathematical Programming Study 19 (1982) 101-119. North-Holland Publishing Company

DIFFERENTIAL PROPERTIES OF THE MARGINAL FUNCTION IN MATHEMATICAL PROGRAMMING Jacques GAUVIN

D~partement de Math~matiques Appliqu~es, Ecole Polytechnique de Montreal, Montreal, P.Q. Canada Franqois DUBEAU Departement de Math~matiques et de Statistiques, Universit~ de Montreal, Montreal, P.Q. Canada Received 13 September 1979 Revised manuscript received 14 January 1981 This paper consists in a study of the differential properties of the marginal or perturbation function of a mathematical programming problem where a parameter or perturbation vector is present. Bounds for the Dini directional derivatives and estimates for the Clarke generalized gradient are obtained for the marginal function of the mathematical program neither assumed convex in its variables or in its parameters. This study generalizes some previously published results on this subject for the special case of right-hand side parameters or perturbations.

Key words: Nonlinear Programming, Marginal or Perturbation Analysis, Differential Stability.

1. Introduction In this p a p e r w e c o n s i d e r a m a t h e m a t i c a l p r o g r a m m i n g p r o b l e m of t h e f o r m : max s u b j e c t to

f ( x , y),

x E R n, y E R " ,

gi(x, y)~<0,

i = 1. . . . . p,

hj(x,y)=0,

j=l

(1.1)

. . . . . q,

w h e r e the c o m p o n e n t s o f the v e c t o r y c a n b e s e e n as p a r a m e t e r s o r p e r t u r b a t i o n s . F o r e a c h v a l u e o f y, the s e t o f f e a s i b l e s o l u t i o n s is

s ( y ) = {x ] g,(x, y) ~ 0,

hi(x,

y) = 0, i = 1 . . . . . p, j = 1 . . . . . q}.

T h e m a r g i n a l f u n c t i o n is the o p t i m a l v a l u e o f the p r o g r a m f o r e a c h v a l u e o f y; t h a t is,

v ( y ) = sup{y(x, y); x ~ S ( y ) } . T h e set o f o p t i m a l s o l u t i o n s is

P(y) = {~ E S(y) If(s, y) = v(y)}. 1oi

102

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

The purpose of this paper is to analyse the differential properties of the marginal function v(y) locally near some point y where v(y) is finite. First, lower and upper bound are obtained for the Dini directional derivatives of the marginal function. From the existence of these bounds, the marginal function is shown to be locally Lipschitz and estimates are then obtained for its Clarke generalized gradient. This paper is a generalization of the papers by Gauvin and Tolle [5] and Gauvin [6] where these results were obtained for the special and simpler mathematical programming problem with right-hand side parameters: max subject to

f(x),

x E R",

g~(x)<~ y~,

i = 1.....

p,

h i ( x ) = yj,

j = 1.....

q.

(1.2)

The organization of the paper is the following. First after some basic terminology, notation and preliminary results in Section 2, we study in Section 3 the continuity of the marginal function and of some point-to-set maps related to program (1.1). In Section 4, lower and upper bounds are given for the Dini directional derivatives of the marginal function. Sections 3 and 4 contain essentially a generalization to program (1.1) of results in [5]. The proofs to obtain this generalization are partially different of those in [5] and are based on some techniques and ideas from Auslender [1] where similar results are given for program (1.2) with the functions {f, g~} only assumed locally Lipschitz while the equality constraint functions {hi} are maintained continuously differentiable. Finally in Section 5 the marginal function is shown to be locally Lipschitz and estimates are given for its Clarke generalized gradient. This last section is again a generalization to program (1.1) of the results in [6] obtained for the simpler program (1.2). Fiacco and Hutzler [4] have also studied independently and essentially simultaneously the continuity of the marginal function and the bounds for its Dini directional derivatives. The methodology they used is original and quite different from ours, as will be explained in the text. They obtained essentially the same results for (1.1) with inequality constraints only, and some preliminary results for the problem including equalities.

2. Preliminaries

In this section, after some definitions, we establish, with a certain regularity condition, some fundamental lemmas which are essential in this paper. The Lagrangian function corresponding to program (1.1) is L(x,

y; u, v) = f ( x , y) - uTg(x,

y) --

vTh(x,

y),

(2.1)

J. Gauvin, F. Dubeaul Differential properties of the marginal function

103

where

g(x, h(x,

y) = y) =

[g~(x, y) . . . . . go(x, y)]T, [h~(x, y) . . . . . hq(x, y)]T.

All functions are assumed continuously differentiable. For an optimal point g ~ P(y), K(~, Y) is the set of Kuhn-Tucker vectors associated with g; that is the set of (u, v) E R ~ • R q such that (i)

VxL(g, Y; u, v) = 0,

(ii)

uigi(~, Y) = O,

(iii)

ui t> 0,

i = 1. . . . . p,

(2.2)

where Vx denotes the gradient relatively to the variables x. We denote by K(~) = U e~p~ K(~, ~) the set of all K u h n - T u c k e r vectors of program (1.1) when the parameters are at values ~. In order to obtain most results of this paper, we assume the MangasarianFromovitz regularity condition (see Mangasarian [10]). A feasible point ~ E S(~) is said to be (M-F) regular if: (i) There exists a direction ~ E R" such that Vxg,(~, ~)~ < 0 ,

i ~/(~,

9) = {i: g~(~, ~) = 0},

Vxh(~, y ) f = 0.

(ii) The Jacobian matrix [Vxh(~, 9)] has full row rank q. If g is a local maximum of program (1.1), this regularity condition is known to be necessary and sufficient to have K($, Y) nonempty and c o m p a c t (see [5, Theorem 2.2]). Following the approach of Auslender [1], we need the following implicit function result.

Lemma 2.1 (Hestenes [7, Theorem 7.1, p. 172]). If.for a function H ( a , [3) defined on S C_R q x R p with values in R ~ we have the Jacobian matrix V~H(t~,/3) with full row rank q at (6l, ~) where H ( t i , / 3 ) = 0, then there exist a continuous function ~([3) on a neighborhood N o f ~ with values in R q and an E > 0 such that

(i) a(/3)= c~, (ii) H(a([3), [3) = 0 on N, (iii) H ( a, [3) = 0, Is - a([3)l < E on N hold only in the case a = ct([3). If H (a, [3) is continuously differentiable on S, then a([3) is continuously differentiable on N.

104

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

Part (ii) of the (M-F) regularity condition gives the following result which is a slight modification of [7, Theorem 5.2, p. 163]. Lemma 2.2. If at (~, Y) the Jacobian matrix Vxh(~, Y) has full row rank q, then for any direction (L 5) E R n x R m, there exist neighborhoods U of (~, Y) and V of (~, g), a { > 0 and a continuous function

a(t;x,y;r,s)

onO=]-t,t[xUxV

with values in R ~ such that a(O;x,y;r,s)=O

onU•

V,

(2.3)

-~-(t; x, y; r, s)

(2.4)

exists and is continuous on 0 with ~t ( O ; x , y ; r , s ) = O

onU•

V.

(2.5)

Furthermore the function X ( t ; x , y; r, s) = x + a ( t ; x , y; r, s)+ tr

(2.6)

satisfies h ( X ( t ; x , y; r, s), y + ts)= = h(x, y) + t{Vxh(x, y)r + Vyh(x, y)s}

on O.

(2.7)

Proof. Consider the function H (t, oJ;x, y; r, s) = h(x + Vxh(x, y)To~ + tr, y + t s ) - h(x, y) - t(Vxh(x, y)r + Vyh(x, t)s)

(2.8)

which is continuous with values in R q. As H ( 0 , 0 ; ~ , y ; ~ , g ) = 0 and rank[V~H(0, 0; ~, Y; L 5)] = rank[Vxh(~, y)Vxh(~, y)T] = q, from L e m m a 2.1 there exists an E > 0 , a neighborhood N = ] - t , t [ x U x V of ( 0 , ~ , y ; L 5 ) and a continuous function oJ(t; x, y; r, s) such that (i) to(O;x,y;r,s)=O, (ii) H(t, t o ( t ; x , y ; r , s ) ; x , y ; r , s ) = O o n N , (iii) (d/dr)to(t,x; y; r, s) exists and is continuous on N. If we let

a(t;x, y; r, s) = Vxh(x, y)Tto(t;x, y; r, s), we then have (2.3) from (i), (2.4) from (iii) and (2.7) follows from (ii) and (2.8). Finally if we take the derivative of (ii) or (2.8) with respect to .t at (0; x, y ; r, s),

J. Gauvin, F. Dubeau/ Differential properties of the marginal [unction

105

we obtain

]

V~h(x, y) ~-~-(0; x, y; r, s ) + r + Vyh(x, y)s = Vxh(x, y)r+Vyh(x, y)s, therefore Vxh(x, y)Vxh(x, y)X ~ t (0; x, y; r, s) = 0. Since we can choose U to have rank[Vxh(x, y)] = rank[Vxh(g, 9)] = q it follows that Vxh(x, y) Vxh(x, y)~ is regular on U and we must have (d/dO to(0; x, r; r, s) = 0 and (2.5) follows. We denote by S*(y) = {x E R": gi(x, y) < O, i = 1..... p, h(x, y) = O} the strict interior of the feasible set S(y). The next lemma will be fundamental in the sequel. Lemma 2.3. If the (M-F) regularity condition is satisfied at some g ~ S(9), then for any direction g E R", there exist a { > O, a neighborhood N ( g ) of g and a continuous function X ( t ; g) on ]0, t[x N ( g ) such that X(0; s) = g, X ( t ; s ) E S*(9 + ts) with (d/dt)X(t;O) continuous on ]0, t [ x N (g). Fu'rthermore there exists an 9 > 0 such that S(y) is nonempty if ]]y - y[[ ~< 9 Proof. For any s E R m, consider the direction r ( s ) E R n defined by

r(s) = x~ - Vxh(~, y)~yh(~, y)s,

(2.9)

where Vxh(x, y) ~ = Vxh(x, 9)T[Vxh(x, 9)Vxh(x, y)T] -1

is the pseudo-inverse of the full row rank Jacobian matrix Vxh(~, Y) and where F is given by the (M-F) regularity condition. Immediately we have Vxh(g, Y) r(s) + Vyh(~, ~)s = 0.

(2.10)

For the active inequality constraints i E I(g, 9),

Vxgi(~, Y) r(s) + Vyg~(~, y)s = = XVxgi(~, 9)~ + {Vyg~(~, 9 ) - Vxg~(~, 9)Vxh(~, 9)* Vyh(~, y)Js. For s on a bounded neighborhood N ( g ) of g, we can choose A > 0 big enough to have Vxgi(~, ~) r(s) + Vygi(g, y)s < O, i E I(~, ~), s E N (g).

(2.11)

For s E N(g), r(s) is in a neighborhood of r(g), therefore we can consider the

106

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

function

X(t, s) = ~ + a(t; ~, Y; r(s), s) + tr(s),

(2.12)

where a(t;X, Y; r(s), s) is given by L e m m a 2.2. From (2.3) we have X(0, g) -- X. From (2.7) and (2.10) we have

h(X(t, s), ~ + ts) = 0 on [0, to[ x N0(g)

(2.13)

for a certain to> 0 and N0(g) C_N(g). For i ~ I(X, ~), since gi(x, ,~) < 0, by continuity there exists a 7~> 0 and Ni(g) such that

gi(X(t, s), ~ + ts) < 0 in ]0, t-i[ • Ni(g).

(2.14)

For i E I(~, y), since g~(~, Y) = 0, we have by the mean value theorem

gi(X(t, s), y + ts) = = t {Vxgi(X(~', s ) , y + ,s)[~-~ (~'; 2, Y; r(s), s)+ r(s)] +

+ Vygi(X(r, s), y + rs)s } for a certain ~-E [0, t], r depending on s. By continuity, (2.5) and (2.11), there exist ti > 0 and N~(g) such that

gi(X(t, s), y + ts) < 0 on ]0, {~[ x N~(g)

for i E I(~, 9).

(2.15)

We finally obtain the first part of the result from (2.12), (2.13) and (2.14) with t- = min{~: i = 0 . . . . . p} and N(g) = nf=o N~(g). To obtain the last part of the result we just need to consider a ball of radius e > 0 in the neighborhood t'N (g) of g = 0. The nonemptiness of S(y) near y is also demonstrated in [12, Theorem 1].

3. Continuity of the marginal function and of some related point-to-set mappings We will need some concepts from the theory of point-to-set maps. For a more detailed exposition the reader is referred to Hogan [8] or Berge [2]. In the following, F will represent a point-to-set mapping of E C_R k into the subsets of R I. Definition 3.1. The map F is upper semi-continuous at X E E, if for any open set 0 containing F(~) there is an open neighborhood N(X) of ~ such that F(x)C_ 0 for each x E N(X) n E.

J. Oauvin, F. Dubeau[Differential properties of the marginal function

I07

The concept of upper semi-continuity will be more suited to our purposes if phrased in terms of sequences.

Definition 3.2. F is said to be closed at ~ ~ E if {x~}C E, x. ~ ~, y~ ~ F ( x . ) and y. ~ ~ imply that y E F(~).

Definition 3.3. F is uniformly compact near ~ E E if there is a neighborhood N(~) of ~ such that the closure of Ux~Nt~) F ( x ) is compact. With the above definitions we can relate the concepts of closed maps and upper semi-continuity. The proof of the following proposition is found in [8].

Proposition 3.1. Let F be uniformly compact near ~ ~ E. Then F is closed at ~ if and only if F(~) is compact and F is upper semi-continuous at ~. A first result on continuity is given in the following lemma. Lemma 3.2. If S(y) is nonempty and S(y) is uniformly compact near y, then S(y) and the marginal function v(y) are upper semi-continuous at ~. Proof. Since the function gi(" ," ) and hi( 9 9) are continuous, S(y) is closed at and it follows from Proposition 3.1 that S(~) is compact and S(y) is upper semi-continuous at .~. From Hogan [8, Theorem 5] it follows that v(y) is upper semi-continuous at y. Theorem 3.3. Suppose that S(y) is nonempty, S(y) is uniformly compact near y, and the (M-F) regularity condition holds at some ~ E P ( ~ ) . Then v(y) is continuous at ~. Proof. Consider a sequence {y.), y. --} y, such that 8 = lira inf v(y) = lim v(y.). y--~

n--~

Let s. = ( y . - Y)/(IlY.- YlI) and tn = [lY.- Y]I. Without loss of generality, we can assume s.--} g, for some direction g. From L e m m a 2.3, we have the function X ( t , s) such that X(0, g) = ~ and X(tn, s.) E S*(y + t.s.) for n sufficiently large. Therefore v(y.) t> f ( X ( t . , s.), y.), I> lim f ( X ( t . , s.), y.) = f(~, y) = v(y) n-~o0

which shows that v(y) is lower semi-continuous at y. This, with L e m m a 3.2, gives the result. In the next theorems it is demonstrated that the (M-F) regularity condition is

108

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

preserved under small variations of the parameters and that the set K(y) of Kuhn-Tucker vectors is locally well behaved. Theorem 3.4. Assume the (M-F) regularity conditions holds at some ~ E P(~). Let {y.} and {x.} be sequences such that y. ~ ~, x. E P(y.) and x. ~ ~. Then for n sufficiently large (M-F) holds at x. and there exist subsequences {(u m, vm)}, {xm} with (u m, v m) ~ K(x,.; ym) such that (u m, vm)~(fl, ~) for some (ft, 6) E K(~, ~).

Proof. (Gauvin and Tolle [5, Theorem 2.7]). Since x. ~ ~, for n sufficiently large it follows that I(x., y.) C_I(~, y) and {Vxh(x., y.)} still have full row rank. Let given by (M-F) at (~, y) and let 5. = [I -Vxh(x., y.)'Nrxh(x., y.)]E

(3.1)

For n sufficiently large, we have, as ~. ~ L

Vxgi(x., y.)~. < 0,

i ~ I(~, y),

Vxh(x., y.)f. = 0.

(3.2)

Since I(x., y.) C_I(~, y), we have (M-F) satisfied at (x., y.). For each (x., y.) take (u., v.) ~ K(x., y.). Then Vxf(x,, y,) = uTVxg(x,, y.) + vTVxh(x,, y,),

(3.3)

v T = [Vxf(X,, y , ) - UTVxg(X,, y,)]Vxh(x,, y,)'~.

(3.4)

From (3.2) and (3.3), we have Vxf(x., y,,)~,, = uTVxg(X., y,,)~. ~< uTVxgi(x., Y.)L, for any i E I(~, y). Then

v~f(x., y.)~. u7 <<-Vxgi(x,., y.)~.

if i E I(~, y),

u? = 0

/ / i ~ t(~, y).

(3.5)

From (3.5) and (3.4) we have the sequence {(u", v")} bounded. Therefore there exists a subsequence converging to some (a, 6) E K(g, y). As a consequence we have:

Corollary 3.5. If S(y) is nonempty and compact and if (M-F) holds at each ~ P(y), then K ( y ) is compact. Proof (Gauvin and Tolle [5, Theorem 2.7]). Take a sequence {(u", v")}C K(y). There exists a sequence {x.}C P(y) such that (u", v " ) E K(x., ~). Since P ( y ) is compact, there exist a subsequence {xm} and a s E P ( y ) such that x,. ~ . From

J. Gauvin, F. Dubeau] Differential properties of the marginal function

Theorem 3.4, choosing ym = Y, there exists a subsequence (u k, v k) ~ K(Xk, y) such that (u k, v k ) ~ (~, ~) E K(g, Y) C_K(9).

{(uk, l)k)}

109

with

Corollary 3.6. Suppose S(~) is nonempty, S(y) is uniformly compact near (y)

and that (M-F) regularity condition holds at each ~ ~ P ( 9 ) . Then for any sequences {yn}, {xn} and {(u., v.)}, with x. E P(y.), (u., v.) E K(x., y.), and y. -->Y, there exist subsequences {Ym}, {Xm} and (u ~, v") ~ K(x,., yz) such that x,. ->2 E P(Y) and (u", vm)~(ft, ~) E K(Y,, y). Proof. (Gauvin and Tolle [5, Theorem 2.9]). Since S(y) is uniformly compact and nonempty near $, there is a subsequence {xm} and a point ~ E S(9) such that x m - ~ . Since v(y) is continuous at y, f(~, ~) = lim

f(x,.,

ym) = lim v(y.~) = v(~)

and ~ ~ P(~). The theorem is now proved by applying Theorem 3.4 to the sequence {(xm, y,,)}. The previous results can be summarised in the following theorem. Theorem 3.7. If S(y) is nonempty and S(y) is uniformly compact near (y) and if

(M-F) holds at every ~ E P(y), then for some ~ > 0, (M-F) holds at each x E P(y) with IlY - Y[[~< & Furthermore the point-to-set mapping K(y) is closed at y and uniformly compact near y and therefore upper semi-continuous at ~. It is also shown in [12, Theorem 1] that the (M-F) regularity is preserved under small perturbations.

4. Dini directional derivatives of the marginal function

The lower and upper Dini directional derivatives of v(y) at y in the direction s ~ R" are respectively D+v(y; s) = lim inf [v(y + t s ) - v(y)]/t, t~0 +

D+v(y; s) = lira sup [v(~ + ts) - v(y)]lt. t--~0+

In the next two theorems we obtain lower and upper bounds for D+v(y; s) and D+v(~; s). Theorem 4.1. If the regularity condition (M-F) is satisfied at some optimal point

E P(?), then for any direction s E R m, D+v(y; s)/>

rain

(u, v)EK(.~, ~,)

{VyL(~,~; u,v)s}.

110

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

Proof. For the given direction s E R m, the linear program rain subject to

--{gTVyg(.~, y) + 19TVyh(.~,y)}S, UTVxg(-g, y) q- t]Tvxh(x, y) = Vxf()C, y),

(4.1)

uTg(.~, V) = O, U ~>0, is feasible and bounded as the set of Kuhn-Tucker vectors K(~, y) is nonempty and bounded under the (M-F) regularity condition. Therefore the dual max r,w

subject to

Vxf(~, y)r, Vxg(~, y)r + toTg($, y) ~< -Vyg(~, y)s,

(4.2)

Vxh('2, y)r = -Vyh(3c, y)s, ~ R p,

r E R",

is also feasible and bounded. Take (r(s), w ( s ) ) an optimal solution of the dual (4.2) and define for h > 0 r(h; s) = AP+ r(s), where ~ is given by the (M-F) regularity condition. We then have for any h > 0,

Vxgi(~, y)r(h ; s) + Vygi(x, y)S < O,

i E I(~, y),

(4.3)

Vxh(~, y)r(h ; s ) + Vyh(~, y)s = 0,

(4.4)

vg(~, ~)r(X ; s) = = hVxf(2, ~)~ +

(4.5)

min

- {uXVyg(g, ~) + vVVyh(g, ~)}s.

(u, v)EK(Y,, ~)

From (4.4) and (4.3), (rOt; s), s) satisfies (2.10) and (2.11) of L e m m a 2.3 for any h > 0 and, as in L e m m a 2.3, we can show that for the function

X (t, s) = ~ + a(t; 2, y; r(h; s), s) + tr()t ; s), we have, for some { > 0, for any h > 0,

X ( t , s) E S*(y + ts),

t ~ 10, ?[.

Then

v(y+ts)>-f(X(t;s),y+ts),

t~lO,?[

and as X(O, s) = :~, v(y) = f(~, y), we have D+v(q; s) t> lira f f ( x ( t , s), y + ts) - f ( x ( o , s), y)l/t t-.O ~

Vxf(x, y)[-~-, (0, x,y, r(h, S), s)-t-r(h, S)]+ Vyf(X, y)S. LUt

.!

J. Gauvin, F. Dubeau/ Differential properties of the marginal .function

I11

From (2.5) of L e m m a 2.2 and (4.5) we obtain D+v(y; s)/> I> AV..f(~, ~)P +

min

(u, v)~K(~, ~)

= AVer(g, y)~ +

- {uTVyg(~, .~)s + vrVyh(~, ~)s}+ Vyf(~, ~)s

rain

(u, v)~K(~,y)

{VyL(~, ~); u, v)s}.

The result follows by letting X--, 0. If we assume that the (M-F) regularity holds at every optimal point g E P(.~), we obtain an upper bound for the Dini directional derivative. Examples show that this derivative can be infinite if this condition is not satisfied at one of the optimal points.

Theorem 4.2. S u p p o s e S(y) is nonempty, S(y) is uniformly c o m p a c t near y and the (M-F) regularity condition holds at every ~ E P(y). Then f o r any direction s ~ R ~', there is a ~ E P(y) such that D+v(y;s)~<

max

(u, v)EK(i, ~)

{VyL(s

Proof. Let {t.}, t . - ~ 0 +, be any sequence such that 8 = lim sup [v(~ + ts) - v(~)]/t = lira [v(y + t.S) - v(y)]/t.. 1-r +

1-r

(4.6)

From L e m m a 2.3, S(y + t.s) is nonempty for t. small and by uniform compactness of S(y) near y there exists x. U P (~ + t.s) and, again by uniform compactness of S(y) near y and continuity of v(y) at y, there exists a subsequence, still denoted by {x.}, and a ~ U P (~) such that x. ~ g. Therefore 8 = lim [f(x., y . ) - f ( ~ , ~)]/tn,

(4.7)

where y. = ~ + t.s. From Theorem 3.7, for n sufficiently large, the (M-F) regularity conditions holds at (x., y.). Take ~. given by (3.1). Then ~. ~ F where ~ is given by (M-F) at (s y) and we have V~gi(x., y.)~. < 0 ~ V~gi(~, ~)~ < 0,

i E I(~, ~),

(4.8)

V.h(x., y.)~. = 0-~ Vxh(~, y)~ = 0. Take r(-s) an optimal solution of the lincar program (4.2) for the direction -s and define rn(--S) = Vxh(xn, yn) ~' Vyh(xn, yn)s

+ [I. - V~h(x., y.)'V~h(x., y.)]r(- s).

(4.9)

112

J. Oauvin, F. Dubeau/ Differential properties of the marginal function

As n-)0% r , ( - s ) converge to r ( - s ) as we can write r ( - s ) = Vxh(~, ~))Wyh(~, y)s + [In - Vxh(Yc, y)~' Vxh(~, ~)], r ( - s )

(4.10)

(Gauvin and Totle [5, Proposition 1.1]). For any h > 0, consider ~, = h?, + rn(-s)--) ~ = h? + r ( - s )

(4.11)

and, as in L e m m a 2.2, the function X ( t ; x , y; r , - s ) = x + ot(t;x, y; r , - s ) + tr defined on 12 = [0, t [ x U(~, y ) x V(P), for some / ' > 0 where U(~, y), V(?) are neighborhoods respectively of (~, y) and ?, respectively. By (2.7) in L e m m a 2.2, we have h ( X ( t ; x, y; r, - s ) , y - ts) = = h(x, y) + t{Vxh(x, y)r - Vyh(x, y)s} If we let X , ( t ) = X ( t ; x n , yr + tns - tns = y we obtain

on 1"].

yn;Pn,--S) we have Xn(0)=xn, and as y n - t n s =

h(Xn(t~), y~) = h(xn, Yn) + tn{Vxh(x,, Y,)Pn - Vyh(xn, y~)s} = h(xn, y,) = 0

(4.12)

by (4.11), (4.9) and (4.8). For i ~ 1(4, ~), gi(X~(t,), y - ) g i ( x , Y ) < 0 , hence for n sufficiently large g~(Xn(t~),Yp)~O,

i ~ I(~, ~).

For i ~ I(~, y), g i ( X n ( t . ) , Y) = g i ( X , ( t . ) ,

yr + tns - tns)

d = g~(Xn(O), fr + t~s) + tn ~ GT(z,t,) for some "rn ~ [0, 1], where GT(t) = g~(Xn(t), y~+ (tn - t)s) with 67(0) = g~(X.(0), y. + t.s), GT(t.) = g~(X.(t.), y~). Since

-•t

GT(~t,) =

= V~(X.(~-~t.), y + ( 1 - ~-.)t.s) ~-(~-.t,; x., y., ~.,-s) + ~. -Vygi(X.('r.tn), (1 - "r~)t.s )s.

(4.13)

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

113

By (2.5) of L e m m a 2.2 and (4.11) we have lim d GT('r.t.) = Vxgi(~, 9)~ - Vygi(~, 9)s .-.~ dt = ;t Vxg~(~, 9)~ + V~g~(~, y)r(-s)- Vyg~(~, Y)s

~<XVxg~(~, 9)~ < 0 by definition of r ( - s ) and F. Therefore, for n sufficiently large,

g~(X.(t.), y) ~<0,

i ~ 1(2, 9).

(4.14)

It follows from (4.12), (4.13) and (4.14) that X . ( t . ) E S(9) and f ( X . ( t . ) , 9) <-v(y) = f($, Y), hence from (4.7), d 8 ~< - lim [f(X.(t,), Y) - f ( X . ( O ) , y + t.s)]lt. = - l i m ~ F,(.:.t.) for some r, E [0, 1], where

F.(t) = f ( X . ( t ) , Y + (t. - t)s) with F.(O) = / ( X . ( O ) , y + t.s), F.(t.) = f ( X . ( t . ) , Y). Since d

"1 fd,v -~t Fn (Tntn ) = ~ xf (Xn ( "rntn)' 9 + (1 -Tn)tnS)l-~t (~.t.; Xn' Yn , ~n, - S)+ ~'n~"

-Vy/(X.(~-.t.), Y + t.(1 - z.)s)s. Using the same arguments as with GT(~'.t.), we obtain lira d F . ( z . t . ) = ;t V j ( ~ , Y)~ + VJCr 9) r ( - s ) - Vy[(~. 9)s. n--*~

Since r ( - s ) is optimal for (4.2) we obtain 6 ~<-AVxf(2, Y)~-

rnin - - {VyL(~, Y; u, v)s}. (u, v)~K(~,,9)

Since 8 is independent of A, let A ~ 0 to obtain 6~

max

(u, v)eK(~, y)

{VyL(~,y;u,v)s}.

This, with (4.6), gives the result. Theorem 4.1 and Theorem 4.2 can be combined to yield the following result. Corollary 4.3. If S(Y) is nonempty, S(y) uniformly compact near 9 and the (M-F) regularity condition holds at every ~ ~ P(Y). then for any direction s ~ R " . we

114

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

have sup min { V y L ( x , Y; u, v ) s } ~EP(y) (u, v)EK(~, y)

~ < D + v ( y ; s )
max

~EP(~) (u, v)EK(.~,y)

{VyL(~,y; u, v)s}.

Proof. Since the (M-F) regularity condition holds at every ~ E P(y), we have,

from Theorem 4.1 and Theorem 4.2, sup

min

~eP(y) (u, v)eK(~, ~)

{VyL(g, y; u, v)s}<~D+v(y; s)

~
max

~EP(')) (u, v)EK(~, y)

{VrL(x, Y; u, v)s}.

To show that the sup can be replaced by a max in the upper bound, consider a sequence {(x., u., v.)},

x. E P(y), (u., v.) ~ K(x.; y),

such that lira VyL(x., Y; u., v.)s = sup

max

~ P ( ~ ) (u, v)EK(~, y)

n--,~

{VyL(~, Y; u, v)s}.

From Corollary 3.6, with y. = 9, there exists a subsequence {(x,., u,., vm)} such that xm --->~ E P ( y ) and (urn, Vm)-->(~, 6) E K(~, 9). Therefore, sup

max

;~eP(y) (u, v)eK(~, y)

VyL(~, Y; u, v)s = VyL(~, Y; ti, 13) s ~<

max

(u, v)eK(fc, ~)

VyL(~, Y; u, v) s

which implies that the sup is attained at some ~ E P(y). Obviously the previous argument can be used to show that inf

min

:~eP(y) (u, v)eK(~, y)

VyL(~, y; u, v) = min

min

~ P ( y ) (u, v)~K(~, ~)

VyL(~, y; u, v),

but such an argument fails to show that the sup can be replaced by a max for the lower bound of Corollary 4.3 since, for a sequence (x., u., v.)--->(fc, fJ, 6) as in the previous argument, we obtain sup

min

VrL(~, y; u, v)s = lim VyL(x., y; u., v.)s = VyL(~, y; ~, 6)s t>

min

(u, v)eK(~; ,))

VrL(~, y; u, v)s

where an opposite inequality sign is required to have the desired conclusion. The bounds given by Corollary 4.3 are sharp in the following sense: examples are known where the directional derivative exists, i.e., D+v(y; s ) = D+v(y; s), and is equal to the upper bound for one direction, to the lower bound for another

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

115

direction while for another situation neither bound is attained (see [5, Example 3.1]). If we replace the (M-F) regularity condition by the more restrictive linear independence (L-I) regularity conditions: (L-I)

the gradients {Vxgi(~, y), i E I(x, y), Vxhj(~, y), j = 1..... q} are linearly independent;

then we obtain the directional derivatives for the marginal function.

Corollary 4.4 If the hypotheses of Corollary 4.3 are satisfied with the regularity condition (L-I) instead of (M-F), then .for all direction s E R m, the directional derivatives exists and is given by Dr(y; s) = max {VyL(~, y; u, v)s}.

Proof. The result follows immediately from Corollary 4.3 by noting that (L-I) implies that the Kuhn-Tucker vector for $ is unique.

Fiacco and Hutzler [4] have studied the continuity and the bounds of the Dini directional derivatives for the marginal function by using an implicit function theorem for the equality constraints near each optimal point s E P(y) to obtain a family of locally reduced programs with inequality constraints only. Recently, the reference [9] was brought to our attention. In that paper Levitin studies the differential properties of the marginal function of a mathematical programming problem defined in Banach spaces. Translated to finite dimensional space, most of the results are similar to those of our last section but are obtained under much stronger assumptions. After assuming that the set of optimal points P(y) is not empty and bounded and that the (M-F) regularity conditions hold at every optimal point s E P(~), Levitin assumes also that the sets of Kuhn-Tucker multipliers K(~; y) are uniformly bounded for ~ E P(y). This appears as a result in our Corollary 3.5. He further assumes that the family of perturbed programs (1.1) are 'quasi-stable', a property similar to the result of our Lemma 2.3. Furthermore, it is also assumed that for all function defining programs (1.1), the gradients are locally Lipschitz continuous. The generalization of the results of this section to a program where the objective function and the inequality constraint functions are only assumed Lipschitz continuous has been recently obtained by G. Fontanie in his M.S. Thesis: "Locally Lipschitz functions and nondifferentiable programming", Technical Report 80-3, Curriculum in Operations Research and Systems Analysis, University of North Carolina at Chapel Hill, 1980, under the tutorship of J.W. Tolle.

116

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

5. The generalized gradient of the marginal function Clarke [3] has introduced the concept of the generalized gradient of a locally Lipschitz function. Definition 5.1. The marginal function v(y) is locally Lipschitz near ~ if for some neighborhood N(~) of ~ there exists a M > 0 such that for any yl, y2 ~ N(~),

Iv(y2) - v(y,)l

MllY2- Y,II.

The next theorem gives sufficient conditions to have this property. Theorem 5.1. If S(y) is nonempty, S(y) is uniformly compact near y and if the (M-F) regularity condition holds at each ~ ~ P(y), then v(y) is locally Lipschitz n e a r Y.

Proof. From Lemma 2.3. and Theorem 3.7, there exists a 6 > 0 such that for y E B(y; 8 ) = {Y l i l y 8} we have S(y) nonempty and the (M-F) regularity condition satisfied at each x E P(y). From Corollary 4.3 and the argument following it, we have at least for y ~ B(~; 6) and s ~ R m, rain

rain

x~P(y) (u, v)EK(x, y)

{~TyL(x,y; u,v)s}

<~D+v(y;s)<.D+v(y;s)~max

max

xUP(y) (u,v)~K(x,y)

{VrL(x,y;u,v)s}.

By Theorem 3.7 and the uniform compactness of S(y) we can choose 8 > 0 such that K=

U

yEB(y, 8)

K(y)

and

P=

U

yEB(y, 8)

P(y)

are compact to obtain in[

(x, y; u, v)EP•

8)•

{VyL(x, y; u, v)s} <<-D+v(y; s)

~
sup

(x, y; u, v)~PxB(~; 8)xK

{~TyL(x,y ; u, v)s}.

Hence there exists a M > 0 such that

-MINI

D+v(y, s)

D+v(y; s)

MIIsll

(51)

for any y E B(y; 8). Consider y~, y2E B(y; 8) and let y 2 - YlII' y, S = IlY2-

? = IlY2- Y4I.

From (5.1), for any t E [0, i'], the Dini directional derivatives D+v(y + ts; s) and D+v(y~ + ts; s) are uniformly bounded below and above. It follows that v(yj + ts) is Lipschitz continuous on the interval [0, i-] and therefore absolutely continuous

J. Gauvin, F. Dubeau/ Differential properties of the marginal function

117

on this interval. Its derivative exists then almost everywhere on (0, {) and we can write ?

v(y2) - v(yO = v(yl + ts) - v(yO =

di- v(yl + ts) dt. 0

Since d V(yl + ts) = Dr(y1 + ts" s) = D+v(y~ + ts" s) = D+v(y~ + ts; s) dt ' ' when it exists, we have, by (5.1), Iv(y9 - v(y,)l ~ M{ = M IlY2- Y,I[. Following Clarke [3], the locally Lipschitz function v(y) on N ( y ) has, at almost all y E N ( y ) , a gradient Vv(y). Definition 5.2 (Clarke [3]). The generalized gradient of v(y) at y, denoted ~v(y), is the convex hull of the set of limits {lira Vv(y,)} where Vv(y,)) exists and y, ~ y as n ~ . c~v(y) is a nonempty convex compact set. Definition 5.3 (Clarke [3]). The Clarke generalized directional derivative of v(y) at y in the direction s, denoted D~ s), is defined by D~

s) = lim sup [v(y + ts) - v(y)]/t.

Lemma 5.2 (Clarke [3]). If v is locally Lipschitz near y, then for any s E R m, D~

that is D~

s) = max{~ 9s ] ~ E av(y)};

s) is the support function of 9v(y).

A first estimate of the generalized gradient 9v(y) is given by the next result. Theorem 5.3. If S(y) is nonempty, if S(y) is uniformly compact near y and if the (M-F) regularity condition holds at every 2 E P ( y ) , then

v(y)cco I..~ePu(y) (u, v)eK(~, u ~) where co stands for convex hull. Proof. Let ~ E av(y) and take a sequence {y.} such that y. ~ ~ and V v ( y . ) ~ . For any direction s E R", we have, by Theorem 4.2, x. E P ( y . ) and (u., v.) K(x., y.) such that Vv(y,) 9s = D+v(y, ; s)<~ VrL(x,, y, ; u,, v,)s.

(5.2)

118

J. Gauvin, F. Dubeau] Differential properties of the marginal Junction

By Corollary 3.6, there exists a subsequence {(x~, ym; u~, vm)}, a X ~ P ( y ) and ( ~ , O ) ~ K ( ~ , y ) such that x ~ , ( u ~ , v m ) - - > ( a , ~ ) . Hence taking limit on both sides of (5.2) leads to 9s ~< V,L(~, y; t~, fi)s ~

max

(u, v)~K(~,y)

{VyL(:~,

y; u, v)s}.

(5.3)

Denote by VrL(~, ~; E) the image of E C R" x R q under the affine transformation VrL(s y; 9 9). Then (5.3) can be written ~. s <~max{~ 9s I'q ~ V y L ( L ~ ; K ( g , ~ ) ) } and

~-s ~<sup{n's In ~ ~P(~) U VyL(.~,y;K (.~,:~)]~.,

(5.4)

But, by Corollary 3.5, U~e(~)VyL(~,y;K('~,y)) is compact and the linear function rl 9s attains its supremum. Therefore by Rockafellar [13, Theorem 32,2], we have

U

,.s~<max{'0.slT~co(

V,L(~,y;K(~,y))}}

and again by Rockafeilar [13, Theorem 13.1] we have

U and the result follows. Definition 5.4. The function v(y) is said to be Clarke regular at y if the ordinary

directional derivative D r ( y ; s) exists and is equal to D~

s) for any direction

s ER". Corollary 5.4. If in Theorem 5.3, the (M-F) regularity condition is replaced by the

(L-I) regularity condition, then Ov (y)=

colrsvP(Y) O

9;

J

where (~, ~) is the unique Kuhn-Tucker vector orE; and v(y) is Clarke regular at

Proo/. From Corollary 4.4, we have for any s ~ R m, D r ( y ; s) = max {VyL(g, Y; t~, g)s}. ~P(y)

Take

n 9U

~P(~)

(VyL(~, y; a,

~)

(5.5)

J. Gauvin, F. Dubeau[ Differential properties of the marginal [unction

119

to have "O " S = VyL(x, if; a, 17)S ~< Dr(27; S) ~< D ~

s)

by [13, Theorem 13.1], -q E av(y) and [,.J {VyL(x, y; u, v)}C Ov (27). This, with Theorem 5.3, give the first part of the result. Also D%(y; s , = max{~, s ]]~ E c o { ~ U

VyL($, 27; ~, fi)}}

= max {VrL(s Y; a, t3)s} = Dv(27; s) ~P(~)

by [13, Theorem 32.2] and (5.5). Hence v(y) is Clarke regular at y. A known example (see [6, Example 1]) shown that under the (M-F) regularity condition the marginal function is not in general Clarke regular.

References [1] A. Auslender, "Differential stability in nonconvex and nondifferentiable programming", Mathematical Programming Study 10 (1979) 29-41. [2] C. Berge, Topological spaces (Oliver and Boyd, London, 1963). [3] F.H. Clarke, "Generalized gradients and applications", Transactions of the American Mathematical Society 205 (1975) 247-262. [4] A.V. Fiacco and W.P. Hutzler, "Extension of the Gauvin-Tolle optimal value differential stability results to general mathematical programs", Technical paper T-393, Institute for Management Science and Engineering, The George Washington University, Washington, DC (April 1979). [5] J. Gauvin and J.W. Tolle, "Differential stability in nonlinear programming", SIAM Journal on Control and Optimization 15 (1977) 294--311. [6] J. Gauvin, "The generalized gradient of a marginal function in mathematical programming", Mathematics of Operations Research 4 (1979) 458--463. [7] M.R. Hestenes, Optimization theory: The finite dimensional case (Wiley, New York, 1975). [8] W.W. Hogan, "Point-to-set maps in mathematical programming", S I A M Review 15 (1973) 591--603. [9] E.S. Levitin, "Differentiability with respect to a parameter of the optimal value in parametric problems of mathematical programming", Cybernetics 12 (1976) 46--60. [10] O.L. Mangasarian, Nonlinear programming (McGraw-Hill, New York, 1969). [11] S.M. Robinson, "Stability theory for systems of inequalities, Part I: Linear systems", SIAM Journal on Numerical Analysis 12 (1976) 754--769. [12] S.M. Robinson, "Stability theory for systems of inequalities, Part II: Differentiable nonlinear systems", S I A M Journal on Numerical Analysis 13 (1976) 497-513. [13] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, NJ, 1970).

Mathematical Programming Study 19 (1982) 120-139 North-Holland Publishing Company

REFINEMENTS OF NECESSARY OPTIMALITY IN NONDIFFERENTIABLE PROGRAMMING II

CONDITIONS

J.-B. H I R I A R T - U R R U T Y

Uniuersiti de Clermont-Ferrand II, 63170 Aubiire, France Received 28 December 1979 Along the lines of our earlier work under the same title, we review and sharpen necessary conditions for optimality for an optimization problem with nondifferentiable data. The relative generalized Jacobian matrix of a locally Lipschitz function and the normal subcone to a set defined by equalities are the principal concepts upon which our study is based.

Key words: Generalized Jacobian Matrix, Normal Cone, Tangent Cone, Epi-Lipschitzian Set.

O. Purpose and scope

This study is a continuation of our earlier work under the s a m e title [10]. As in this paper, we are concerned, for the m o s t part, with an optimization problem with locally Lipschitz data. We proved how the necessary conditions for optimality were sharpened by considering globally a vector-valued function F defined by means of all the functions involved in the problem. An important tool in that way was the concept of generalized Jacobian matrix of F (at x0) in Clarke's sense [3], we denote it by ~F(xo). From its basic definition (recalled at the beginning of Section 1), one can see that ~F(xo) takes into account the behaviour of F all around x0. Actually, it has appeared that, for m a n y purposes, the knowledge of all this information was not quite necessary; what is needed to be known is the contribution of F restricted to a subset Q in the construction of aCF(x0). These considerations gave rise to what we call the generalized Jacobian matrix of F relative to Q, denoted (at x0) b y ar o F(xo). This c o n c e p t which is defined through a 'lim sup'-operation on the collection F(x) I x Q) was firstly used in [10, Section 2] for particular purposes. In Section 1, we go into details of the study of ~-,o F ( . ) : conditions ensuring the connectedness of or F(xo), properties of aCoOF ( - ) as a set-valued m a p p i n g and chain rules. Of course, all these properties and calculus rules are mainly derived from those already exhibited for the usual generalized Jacobian matrix (which turns out to be ~r F ( . ) for Q the whole space). H o w e v e r , we seize the opportunity o f this review for sharpening an earlier chain rule (see T h e o r e m 1.4). Section 2 is divided into two parts. In the first part which is simply introductory, we turn our attention to two m a j o r cases where the cone of normals N ( S ; Xo) to a subset S at x0E S reduces to the positive multiples of elements of 120

J.-B. Hiriart-Urruty/Optimality conditions in nondifferentiable programming

121

the generalized gradient ads(xo) of the distance function. Moreover, the property for S being epi-Lipschitzian at x0 is rephrased in terms of relative generalized gradients of ds (Proposition 2.3). The second part is devoted to the problem of giving an estimate of N(S;xo) when S is represented as an inequality constraint

s

=

{x [ g(x)_<0},

(o.1)

and g(xo) = 0. Firstly, the general case of a lower-semicontinuous g is considered and earlier results are made more precise. Secondly, for a locally Lipschitz g, we show how our earlier assumption requiring that 0 ~ Og(Xo) (see [6, Lemma 5]) can be weakened by founding on refinements studied in Section 1. The main result (Theorem 2.6) makes conspicuous the role played by the relative generalized gradient of g. As we pointed out earlier [8, Section 3.2], the concept of normal cone is not well-adapted to handle optimization problems with equality-type constraints. In Section 3, we pursue our earlier study on the normal subcone to a set represented as one equality-type constraint by refining conditions yielding an estimate of it, and we broaden the concept to the case where the set S is defined through several equalities. The resulting cone, which is composed of a union of convex cones, appears to be for S what the normal space is for a CLmanifold. Finally, necessary conditions for optimality are derived in terms of normal subcones (Theorem 3.5).

1. The relative generalized Jacobian matrix

Let F : R P - * R q be a locally Lipschitz function; the generalized Jacobian matrix of F at x0 ~ R p, denoted by o~F(xo), is the set of matrices defined as the convex hull of the set {M I 3xi ~ xo with F differentiable at xi and ] F ( x i ) ~ M}.

(1.1)

As readily seen, this definition takes into account the behaviour of F all around x0. In many problems it has appeared that all the information is not quite necessary; what is needed is essentially the behaviour of F relative to a given subset Q c R ~. That led us to introduce in [10, Section 2] a relative generalized Jacobian matrix which can be defined on all of cl(Q) (closure of Q). Definition 1.1. The generalized Jacobian matrix of F relative to Q is defined at x0 by

w~,,o Cl{xE~nv~F (x)}, where T'(x0) denotes the collection of neighbourhoods of Xo.

(1.2)

122

J.-B. Hiriart-Urruty]Optimality conditions in nondifferentiable programming

The set of matrices defined in (1.2) will be denoted by ~_,oF(xo). In other words, ~_.oF(xo) consists of all limits of sequences of matrices Mi ~ ,,~F(xi) with x~ converging to x0 in Q. Clearly, ~-.o F(xo) is empty if x0 ~ cl(Q) and coincides with ~F(xo) whenever x0 E Q. If x0 lies on the boundary of Q (bd Q), JL.oF(xo) is, as often as not, strictly smaller than ~F(xo). The properties of the relative generalized Jacobian matrix are mainly derived from those of the generalized Jacobian matrix [3]; let us list some of them. (P0 Let {Q~ [ a ~ A} be a finite collection of subsets whose union is Q; then

y F(xo)=

U

yF(Xo).

(P2) The set-valued mapping x I:~-.oF(x) is locally bounded, that is to say: there exists a neighbourhood V of x0 and a constant K such that max{lllMIII, M ~ -~Q ~ F(x), x ~ V} ~ K.

(P3) ~-~QF ( ' ) is a compact-valued upper-semicontinuous set-valued mapping. This property, which is inherent to every 'lim sup'-operation on set-valued mappings, can be derived directly from Definition 1.1: indeed, A~_,oF(.) is the smallest upper-semicontinuous extension to cl(Q) of the set-valued mapping ~ F ( . ) restricted to Q. Property (P3) will be mainly used under the following form: if x , ~ x o in cl(Q) and if M , ~ M o with M,E,,~-.oF(x,) for all n, then

Mo E ,,~-,oF(xo). Note that, unlike ~F(xo), ~-.o F(xo) is not generally convex when x0E bd Q. However, its convex hull can be expressed in a way generalizing the basic definition (Definition 1.1) of the generalized Jacobian matrix. F is actually differentiable except on a set of null measure; let ~ denote the set of x where JF(x) is defined. Denoting by Q' the set Q n ~, the collection {JF(x)[x ~ Q', ~V(x0)} is also a filtered family. Therefore we may carry out for this collection the 'lira sup'-process as done in Definition 1.1; in accordance with our earlier notations, the resulting set-valued mapping will be denoted as x ~ J-,o' F(x). If Q is open, we have that cl(Q')= cl(Q) and the basic definition of the ,,~F(x) yields: (P4) Let Q be an open subset; then (1.3) ~

for all x0. When Q is not open, a more general relation would be that

co(

oOocoL,o 0 opon

(1.3)

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

123

In default of convexity, ~-.o F(xo) enjoys a weaker topological property, namely connectedness, under precisely a local connectedness assumption on Q at x0. Proposition 1.2. Suppose there exists a base {V,},~N .for the neighbourhood system o.f xo such that Q N V, is connected for all n. 1 Then ~_~oF(xo) is connected. Proof. Without loss of generality, we can suppose that Vn+~C V, for all n. Now, ~F(x) is nonempty and connected (since convex) for all x ~ Q N V,. Moreover, ~ F ( - ) is an upper-semicontinuous set-valued mapping. Therefore, the image set

F.=

U

~r

xEQO Vn

is connected [11, Theorem 5]. Thus, ~-.o F(xo) which is the intersection of the decreasing sequence {cl F,},~N of compact connected sets is connected. The next example illustrates some of the properties laid out above. When f is real-valued, we shall preserve the notations a-~Qf(x0) and V_~o,.f(x0) for the set of column vectors issued from {ay(x) Ix E Q} and {Vf(x) [x E Q'}. Example 1. Let f : R ~ R be defined by f(xl, x2)=lxl§ {(Xl, Xl § Ix21 0} and x0 = (0, 0). Then

Consider Q =

V,f(x~ = { ( I ) ' ( - ] ) ' (-11)' (-11)}'

(-2)}

u ~Q

-l_
Now, if Q = {(xl, x2); Xl + Ix=l > 0}, we have that a f(x0) = {(1), -1 -< a -< 1}.

-~Q

Concerning chain rules on relative generalized Jacobian matrices, we state two major results. The first one is directly derived from the corresponding one on generalized Jacobian matrices (see [4, 7, 9] and references therein). Theorem 1.3. Let f =q~ o F, where q~:R q ~ R is continuously differentiable at F(xo). Then

0 (q~o F)(xo)= ~M T" V~(F(xo))IM E ~ F(xo)l. ~,Q

t

~O

)

When x0 ~ Q, this assumption merely means that Q is locally connected at x0.

(1.4)

124

J.-B. Hiriart-Urruty] Optimality conditions in nondifferentiable programming

If we write F = (fl . . . . . fq)T the above rule provides the exact evaluation of relative generalized gradients of functions like ~?=lfi, I-I~=lfi, fdf2"'" etc. In view of assumptions which are required on q~ o F in optimization problems, it is worth mentioning the next implication. (Ps) Let ~o be continuously differentiable at F(xo) with Vq~(F(x0))#0, let ,,~-.0 F(xo) be surjective. Then 0 ~ 0-,o(q~ o F)(x0). By considering r defined by q~(Yl. . . . . yq) = yi, we have the following 'projection' property: -,Q

t

-,Q

"

J

Nevertheless, due to the possible 'interdependence' of component functions [i, we only have that -.Q

-,Q

When q~ is simply Lipschitz around x0, things are less pleasant. By applying earlier results ([7, Th6or~me 1] or [14, Theorem 4]), we obtain the following result: (P6) Let F be described as (fl . . . . . fq)T, let q~ be Lipschitz in a neighbourhood of F(xo). Then

O(q~o F)(xo)C c o ~

~Q

I./~--1

u,x~, l (ul ..... Uq)~ I

a ~0(F(x0))and

-~F(Q)

(x~' ..... x~) ~ ~t a f~(x0)}. i=l ~Q

(1.6)

Actually, we shall produce a sharper result, by expressing the result in terms of PrO F(xo) instead of X,~=l a_,q fi(xo). Theorem 1.4. Let q~ be Lipschitz in a neighbourhood o[ F(xo). Then 0 (q~o F)(Xo) C c o { M T u ~Q

I U (~

0 ~0(F(x0)) and M e ar F(xo)}. -,F(Q) --,Q

(1.7)

Proof. We begin by recalling the following mean-value theorem (see [12] and references therein). Theorem Let F : R p --~ R q be locally Lipschitz, let a, b be two distinct points in R p.

Then there exist real numbers ak, vectors ck, matrices Mk, k = 1..... q, such that >-O, ck ~ ]a, b[, Mk E o~F(ck) for all k, ~,~=l ak = 1, and

ak

F ( b ) - F ( a ) = ~ l a~M~(b - a).

(1.8)

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

125

Now let us consider a sequence {x.} converging to x0 and a sequence {h.} C R+* converging to 0. We set E. = {q~[F(x. + And)] - q~[F(x.)]}h~ ~. According to the above theorem, there exist F. ~ ]F(x.), F ( x . + And)[ and u. aq~(F.) such that E . = ( F ( x . + And) - F ( x . ) , u.)A~ ~.

(1.9)

Now, by applying the same mean-value theorem to F, we get that [ F ( x . + And) - F(x.)]h~ 1 = ~ ak..Mk..d,

(1.10)

k=l

where ak.. >--O, ~ = 1 ak.. = 1 and Mk.. E ~ F ( c k . . ) for some Ck.. E IX., X. + And[. Since the set-valued mappings ar and .,r are upper-semicontinuous, we may suppose (subsequencing if necessary) that u. ~ u E aq~(F(xo)), P

for all k (~--i ak = 1),

ak,. ~ ak

MR,. ~ Mk E ~ F ( x o )

for all k.

Consequently, we derive from (1.9) and (1.10) that lim s u p E . -< max{(MTu, d) [ u E aq~(F(xo)), M E ~r n~

Hence, a(~p o F)(xo) C co{Mru I u ~ &p(F(x0)), M E oCF(x0)},

(1.11)

and the result (1.7) is thereby proved. The next examples illustrate the foregoing results. Example 2. Let F : R 2 ~ R 2 of the form (f, _f)x where f ( x l , x2)= max(xb x2), let ~o:REoR be defined by ~0(yl, y2) = lYll+lY2[. The concerned point is here x0 = (0, 0) while Q is taken to be the whole space. We have

CF(xo)=

co{(I), (I), (ll)(I)}' {['-~ ~ 0
-ol '

Therefore, {MTu

lu E

L\

Lit/[

J

126

J.-B. Hiriart-Urruty! Optimalityconditions in nondifferentiableprogramming

Since O(~o F)(xo) = 2co{(~), (1~ (-~), (_~}, this example shows that the convex hull operation cannot be removed in (1.7).

Example 3. We slightly modify the preceding example by taking q~ defined by q~(Yl, Y2) = Yl +

lY21.Then

co{MTu I u E aq~(F(xo)), M E pF(x0)} = = { / 3 ( 1 -laO) - < a - < l ' O - < / 3 - < 2 } ' a while the convex set C defined on the right-hand side of (1.6) is larger, namely:

(I) (ll)) Formula (1.7) gives only an estimate of 0_.o(~ o F)(xo) in terms of the corresponding data on component functions; it cannot be strengthened to equality without further information on q~ or F. In view of applications we intend doing, let us particularize two situations. Let first f(x) be defined a s maxie[I,q]fi(x); then

0 f(xo) C c o / U ..aQfi(Xo)},

~Q

tiEl(x O)

(1.12)

where I(xo) denotes the set of i E [1, q] satisfying f(xo) =/i(x0). In this particular case, due to the projection property (1.5), the estimates produced by (1.6) and (1.7) are identical. Indeed we have that

co/U

o

t. iEl(x0) ~Q

i,(xo)]= J

(1.13) where ~,x0,={(X, ..... Xq)~Rq l ,x,>-O Vi, X,=O V i ~ I ( x o ) , ~ A,= I As a consequence, we have: (P7) Suppose there exists d satisfying: max{(x*, d)I x*E O

iEI(xo)}
Then 0 ~ co O-.of(xo). Now, let F be continuously differentiable at x0. Then (cf. [4, Section 13]) 0 (q~ o F)(x0) C .l'rF(xo)

~Q

a q~(F(x0)).

--,F(Q)

(1.14)

J.-B. Hiriart-Urruty/Optimality conditions in nondifferentiable programming

127

Therefore, we note: (P8) Let F be continuously differentiable at Xo. If 0 ~ O-,Fto~q~(F(xo)) and if JF(xo) is surjective, then 0 ~ 0_,o(q~ o F)(xo). Under the same assumption on JF(xo), if 0 ~ co{0-,F(o~ r then 0 ~ co{0_, 0 (q~ o F)(x0)}.

2. Normal cones to a set represented as an inequality constraint

2.1. Let S be a nonempty closed subset of R p and let x0 ~ S. Since the distance function ds(x) is null whenever x E S, the only contribution of the region S to the construction of the generalized gradient of ds at x0 amounts to: 0 E ads(xo). Thus,

where O denotes the complementary set of S and O' the set of x E O where Vds(x) exists (cf. (1.3)~ Clearly, Vds(x) is nonnull whenever x ~ O'; to be more precise, there exists a unique point ~ in S closest to x and Vds(x)= ( x - X ) l ( l l x - ~ l l ) (see [1, Proposition 2.4]). Then

coI o s xo,I-co/z where one considers all sequences {x~} converging to x0 in O such that xn has a unique closest point ~ in S. The tangent cone to S at x0 (in Clarke's sense) is a closed convex cone denoted by T ( S ; Xo) (or Ts(xo)) and defined as

T ( S ; Xo) & [Ods(xo)]~ = {d E R p I (d, x*) <- 0 for all x* ~ Ods(xo)}.

(2.3)

The tangent cone can be characterized in more ways than one; see, for example, [1, Proposition 3.7] and [8, Theorem 11. Reciprocally, the normal cone to S at x0 (in Clarke's sense) is taken to be the polar cone to T ( S ; Xo). It will be denoted by N ( S ; Xo) or Ns(xo). Thus,

N ( S ; Xo) ~- [T(S; Xo)]~ = cl{R§ Ods(xo)}.

(2.4)

This definition was Clarke's original one [1, Definition 3.1] and, in view of (2.1) and (2.2), we have: Proposition 2.1. (see [1, Proposition 3.2]).

N ( S ; x o ) = cl co{li_,mmvn(Xn- ~n)} where are considered all sequences with v. >-"O, x, converging to Xo and having closest point ~ in S.

128

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

At this stage, a question would deserve to be answered: Under what conditions does the normal cone N(S;xo) reduce to R+ ads(xo)? In deriving necessary conditions for optimality in a problem where the constraint set is S, it appears that the knowledge of R§ ads(xo) is sufficient whereas its closure is automatically introduced whenever the polarity correspondence between convex cones is invoked. The distinction between R+ ads(xo) and N(S; Xo) is cumbersome and, in view of the particular structure of ads(x0) (see (2.1) and (2.2)), one gets the impression that the two cones do coincide. We shall partially answer the question posed above by showing a number of situations where R+ ads(xo) is closed. To begin with, suppose that R§ ads(xo) is finitely generated, i.e. V ds(xo) = {x~ . . . . . x*}, --~0'

where {x$ ..... x*} is a finite set of unit vectors. This situation frequently occurs in applications. In such a case, one easily checks that R+ ads(xo), which equals R+ co{V-.o, ds(x0)}, is closed. Now we turn our attention to two major cases. The first one is the so-called tangentially regular case. Recall that the cone of adherent displacements for S from x0 (or contingent cone to S at x0) is the closed cone defined as

T(S;xo)={d I:l)t, ~ O , d , ~ d , with x0+ X,d, E S}.

(2.5)

As a general rule, T(S; xo) C T(S; x0) (see [6, Proposition 4]), and S is said to be tangentially regular at x0 if T(S; Xo) equals T(S; Xo).

Proposition 2.2. Let S be tangentially regular at x0; then N ( S ; Xo) = R§ ads(xo).

Proof. We proved earlier [13, Theorem 3] that, in such a situation, ads(xo) = N (S ; Xo) N B, where B denotes the closed unit ball in R p. Hence the result is proved. The further situation we shall deal with is the epi-Lipschitzian case. Following Rockafellar's terminology [17, Section 4], S is epi-Lipschitzian at x0 if it can be represented near xo as the epigraph of a Lipschitzian function. In order to translate this definition in terms of tangent cones, Rockafellar [17, Theorem 3] proved the following: S is epi-Lipschitzian at x0 if and only if int T(S; Xo) is nonempty. Hence, one readily deduces:

Proposition 2.3. S is epi-Lipschitzian at Xo if and only if

0

o{z

= cobo

In such a case, N(S; xo) reduces to R+ Ods(xo).

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

129

To sum up, it results that those examples (if any!) in which R+ Ods(xo) can differ from N(S; Xo) are somewhat degenerate. 2.2. For the sake of optimality conditions, we need expressions of R+ Ods(xo) or N(S; Xo) when the set S has a representation by the way of a function g. In this section, we consider more specifically a representation of S as an inequality constraint s = {x ~ a p [ g ( x ) <- %

(2.6)

To begin with, we let g be an arbitrary lower-semicontinuous (l.s.c.) function taking values in R (the extended reals). The lower-semicontinuity of g is not necessary for the next results to hold but it renders the notations simpler. The generalized gradient of g at a point x0 where it is finite can be defined in equivalent ways as:

Og(xo) = {x* I(x*, -1) E Nepig(Xo, g(x0))}, or

Og(xo) = {x* [ (x*, d) --- g~

d) for all d},

where g~ d) stands for inf{t~ l (d, #) E Tepig(x0, g(x0))}; for details, we refer to [8, 16]. g~ can also be defined in an analytic way through a formula involving limits of difference quotients of some kind; see [16]. The next result is from [8, Propositions 3 and 4].

PropoSition 2.4. Let Xo be a point satisfying g(xo) = O. Then {d ] g~

d) < 0} C T(S; Xo).

(2.7)

If, moreover, Og(xo) is nonempty and does not contain O, then N ( S ; x0) C cl{R+ 3g(xo)}.

(2.8)

Another way of expressing cl{R+ 0g(x0)} lies in decomposing it in terms of Og(xo) and its asymptotic cone. Recall that the asymptotic cone (or recession cone) of Og(xo) is the closed convex cone defined by

O~g(xo) = {d I x* + Xd E Og(xo) for all )t >_0}, where x* is an arbitrary element of ~g(xo). Hence, following the closure formula for R+ ~g(xo) (see [15, Theorem 9.6]), we can rewrite (2.8) as

N(S; x0) C R+ {0g(x0) O O~g(xo)}.

(2.9)

The assumptions of Proposition 2.4 do not secure that S is epi-Lipschitzian at x0. For that purpose, a further assumption on the function g is needed. Following Rockafellar [16, Section 5], g (l.s.c. at xo) is said to be directionaUy

130

J.-B. H i r i a r t - U r r u t y / O p t i m a l i t y c o n d i t i o n s in n o n d i f f e r e n t i a b l e p r o g r a m m i n g

Lipschitzian at Xo if there exists a direction do such that lira sup

g(x + )td)- g(x) < +~.

(2.10)

x-*xa, d-~d0 g(x)-*g(xo), ,~.0 +

In a geometric way, g directionally Lipschitzian at x0 amounts to saying that epi g is epi-Lipschitzian at (x0, g(xo)) and therefore to assuming that Tepig(Xo,g(xo)) has a nonempty interior. For several criteria guaranteeing a directionally Lipschitzian behaviour for a function, see [16, Section 6]. The next result is from [18, Theorem 5].

Proposition 2.5. Let Xo be a point satisfying g(xo)= O. Suppose that g is directionally Lipschitzian at Xo and that Og(xo) is nonempty and does not contain O. Then S is epi-Lipschitzian at Xo. When the function g is Lipschitz around xo, the above sufficient condition ensuring that N ( S ; Xo)C R+ ag(xo) (at a point where g(xo)= 0) merely amounts to (H)

0 ~ Og(xo).

This assumption and the consequent comparison results were studied in our earlier work [6, Lemma 5]. Actually, (H) turns out to be strong since ag(x0) takes into account the behaviour of g all around x0. We shall weaken (H) by considering only the behaviour of g outside S. For the rest of this section, the considered function g is assumed to be Lipschitz around x0.

Theorem 2.6. Let Xo lie on the boundary of S. We suppose that (H)

0 e o g(xo). --,0

Then R+ Ods(xo) C R+ co{.O~ g(x0)}.

(2.11)

The cone R+ Ods(xo) is a concept associated with the set S, whatever be its representation as an inequality constraint. We observe that S remains unchanged if g+= max{g, 0} is substituted for g; so, it is not surprising that the estimate (2.11) relies on an assumption which is invariant under this substitution. Moreover, according to the construction itself of the generalized gradient, one notes that

Og+(xo)= co{O, O ~ g(x0)}. Thus, in consideration of (2.11), we have that R+co[ 0~ g(x0)} = R+ Og+(xo)C R+ 0g(x0).

(2.12)

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

131

Of course, by performing the closure operation in both sides of (2.11), one gets an estimate of N(S;xo). Nevertheless, let us note the following sharpened version of Propositions 2.4 and 2.5 for the locally Lipschitz case.

Corollary 2.7. Let Xo be on the boundary of S. We assume that

0 cO{ o (X0,} Then S is epi-Lipschitzian at Xo and N ( S ; x0)C R+ ag +(xo). Proof of Theorem 2.6. Let v be a unit vector of the following form: v = lim ._= ~x. - 2,.11'

(2.13)

where x, converges to x0 in O and has closest point 2, in S. Due to the structure of ads(x0) (of. (2.1) and (2.2)), the inclusion (2.11) will be proved if all such v are proved to belong to R+ a-.o g(xo). 2, minimizes the function x ~-, ~llx,-xn 2 on S. Since 0 ~ a-~og(xo) and the set-valued mapping x ~ a-~og(x) is upper-semicontinuous (property (P3)), 0 ~ a-~o g(~,) for n large enough. Therefore, according to [10, Theorem 4.1] there exists a positive ~, and x* E O-.og(x,) satisfying x, - 2, = tz,x*.

(2.14)

Subsequencing if necessary, we may suppose that x*, converges towards a nonnull element x~ E 0--,og(xo). Consequently, we deduce from (2.13) and (2.14) that v =/~x$ for some positive/x. H e n c e the result is proved. When g turns out to be a composite function q~ o F, chain rules displayed in Section 1 give sufficient conditions ensuring (H) or (co H).

3. Equality-type constraints and normal subcones Consider an optimization problem of the form (P)

minimize

f(x),

subject to

x E S,

where S is a closed subset of R p and f a locally Lipschitz function. The basic necessary optimality conditions for (P) can be described as follows: (i) in a primal form: if x0 solves (P) locally, then r ( x 0 ; d) _> 0 for all d in the cone T(S; xo) of adherent displacements for S from x0 (see [6, Theorem 2]); (ii) in a dual form: if x0 solves (P) locally, then

0 ~ Of(xo) + R+ Ods(xo).

(3.1)

132

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiable programming

This latter result is obtained by noticing that x0 is a local minimum (on R p) of f + kds for k Lipschitz constant of f around x0 (see [2, L e m m a 2; 5, Th6or~me 5]). In fact, the condition that Of(xo) n N ( S ; xo) ~ fJ (derived from (3.1)) has been proved to be a particular case of a more general optimality condition, dualization of the primal condition (i) [cf. [6, Theorem 3]). As pointed out earlier [8, Section 3.2], the concept of normal cone is not well-adapted to handle optimization problems with equality constraints. Therefore, we shall pursue our earlier study on the normal subcone to a set [8, Section 3.2] be refining conditions yielding an estimate of it when the set is represented as one equality-type constraint, and by generalizing the concept to the case where more than one equality constraint is involved. 3.1. Let Sh be a constraint set defined as Sh = {X E R p I h(x) = 0},

(3.2)

where h is a (real-valued) continuous function. We shall begin by presenting in a unified fashion necessary conditions for x0 E Sh to solve locally (P). Starting with optimality conditions in a geometric form, we refine and cover the results established earlier for optimization problems of the above form. Let S~ and S~ denote the following subsets: S-~ = {x I h(x) >- 0},

Sh = {x I h(x) <- 0}.

(3.3)

T h e o r e m 3.1. Let Xo E S, let T§ and T_ be two closed convex cones with apex 0

and satisfying T§ C T(S-~; Xo),

T_ C T(S~; Xo).

(3.4)

A necessary condition for Xo to be a local minimum of f on S is that Of(xo) O - {N+ U N_} ~ O, where

(3.5)

N+ and N_ are the polar cones to T+ and T_ respectively.

Proof. The proof is a slight generalization of that of [8, Theorem 9]; we reproduce it for the convenience of the reader. Suppose that (3.5) is not satisfied; we have that 0 f~ Of(xo) + N+

and

0 ~ af(xo) + N_.

By the usual separation theorem for convex sets, there exist d +, d- and ~ > 0 satisfying sup{(x*, d + ) l x * E Of(xo)+ N+}-<-~, (3.6) sup{(x*, d-) I x* E Of(xo) + N'-} <- -E. The polar cones to N+ and N_ are T+ and T_ respectively, the support function

J.-B. Hiriart-Urruty/ Optimality conditions in nondifferentiab|e programming

of c~f(xo) is d ~ [~

133

d); therefore we can rewrite (3.6) as follows:

::l d + E T+, 3 d- E T- with max{f~

d+),/~

d-)} -< -~.

(3.7)

Now, according to (3.4), there exist two sequences {X+~},{~,~} converging to 0 in R* and two sequences {d~+},{d~} converging respectively towards d + and d-, such that +

+

x+~=Xo+X.d.~S~,

x~=xo+X~d~ES~

(3.8)

for all n. Since h is a continuous function, there exists a . E [0, 1] such that for every n, h(a.x+~ + (1 - a.)x~) = 0, that is to say +

x. = a . x . + (1 - a.)x~ E S.

(3.9)

The sequence of general term )t. = a.A.+ + (1 - a.)X~ converges to 0 in R*. Moreover, by subsequencing if necessary, we may suppose that ~.

= c~"x+~--, ~z ~[0, 1].

Consequently, we deduce from (3.8) and (3.9) that lim x. - x0 = ~d + + (1 - (~)d- E T ( S ; xo). The convexity of d ~ / ~ /~

d) and relations (3.7) yield that

+ + (1 - / 2 ) d - ) -< - e ,

which contradicts the necessary condition for optimality which states that f~ d) > 0 for all d E T ( S ; x0) (cf. primal condition (i)). The possibility of various choices of T§ and T_ makes Theorem 3.1 versatile. Let us show how the earlier results are covered: (i) let S be an arbitrary closed subset, let h be a positive function such that S = {x I h ( x ) = 0 (-<0)} (the distance function ds is such an example). Then, one can choose a closed convex cone T included in T(S;xo) and the optimality condition (3.5) says that O f ( x o ) N - T ~ f~. That is precisely the result of [6, Theorem 3]. (ii) Clarke's tangent cones T(S-~; Xo) and T(Sg; Xo) are instances of cones T+ and T_ satisfying (3.4). With such cones, (3.5) states that

0 E Of(xo) + M(Sh; Xo), where M(Sh; at x0, i.e.

Xo) is what we called in [8, Definition 6] the normal subcone to Sh

M(S~; Xo) = N(S~; Xo) U N ( S g ; Xo).

(3.10)

This latter example is of particular importance since Clarke's construction of

134

J.-B. Hiriart-UrrutylOptimality conditions in nondifferentiable programming

tangent cones produces cones satisfying in a systematic fashion the hypotheses of Theorem 3.1. On that account we shall be more interested in the normal subcone, especially in giving an estimate of it in terms of concepts related to the function h. The next example illustrates the versatility of the optimality condition (3.5).

Example 4 (see Fig. 1). Let h :R2--*R be defined by h(Xl, x2) = ~ xl + x2

tx, - X1/2

if x2 -< 0, if x2 >- 0,

and x0= (0,0). In view of (3.5), the larger T+ and T_ are, the sharper is the optimality condition. In this example, T(S-~; Xo) is itself c o n v e x so that we shall take T+ = T(S+h; x0). As for T_, one can choose a half-space among the following ones T -~ ={(dl, d2) ] Oldl + 02d2~ 0},

0E O

O = {(0b 02) # 0[ 0 1 ~ 0 , 0 2 ~ 01}.

Consequently, if x0 is a local minimum of f subject to h(x) = 0, there exists, for all 0 E O, x$ E Of(xo) such that - x ~ E N+U N -~ In particular, if f is C ~ at Xo, the above relation induces that -Vf(xo) must belong to N+. In view of the results of Section 2.2, we are able to give an estimate of the normal subcone M(Sh; Xo) in terms of concepts associated with h. It suffices to apply earlier results to both h and - h . For the record we note the two major results.

/ / Fig. I.

J.-B. Hiriart-Urruty/Optimality conditions in nondifferentiable programming

135

Corollary 3.2. Let h be directionally Lipschitzian at Xo with h(xo)= O. Suppose that ah(xo) is nonempty and does not contain O. Then Sh is a Lipschitzian surface near Xo and M(Sh; x0) C cl{R ah(x0)} = R{ah(x0) U 0=h(x0)}.

(3.11)

Proof. If h is directionally Lipschitz at x0, so is - h and a ( - h ) ( x 0 ) = -Oh(xo) (see [16, Theorem 3]). Hence the result is derived from Propositions 2.4 and 2.5.

Suppose now that the function h involved in the representation (3.2) of Sh is locally Lipschitz. Let O~ (resp. Oh) denote the complementary set of S~ (resp. of SD, i.e. the set of x satisfying h(x) > 0 (resp. h(x) < 0). We set Oh = O+hU Oh, SO that o h ( x o ) = ,9 h(xo)U 0 h(xo). -~oh -,o~ -,o~

As a straightforward application of Theorem 2.6 and Corollary 2.7, we get: Corollary 3.3. Let Xo lie on the boundary of Sh. If (H)

o~

a h(xo), -,oh

then M (Sh; Xo) C cI{R+ ah+(xo) U R+ ah-(xo)}.

(3.12)

Now, if

then Sh is a Lipschitzian surface near Xo and M ( Sh; Xo) C R+ {ah +(xo) U ah-(xo)} C R ah (xo).

(3.13)

As expected, the assumption as well as the result of the corollary above are invariant if - h is substituted for h and they cover what was laid out earlier by considering h = g+. Even more, the results are not meaningless when applied to fhl; but, in such a case, we are caught in the trap since M(Slhl; Xo) = N(Sh; Xo) and R a Ihl(x0) both 'blow up'. 3.2. Let now h = (h~ ..... h,,) T be a vector-valued continuous function and therefore Sh be defined by m equalities, Sh = {X ~ R p I h~(x) = 0 . . . . . hm(x) = 0}.

Denoting ~,, the collection of subsets of {1..... m}, we define a particular 1 covering {0 h}1e~', of Oh (complementary set of Sh) as follows: 0~, = {x E Oh ] hi(x) <- 0 V i ~ I, hi(x) >- 0 V i ~ I}.

(3.14)

136

J.-B. Hiriart- Urruty/ Optimality conditions in nondifferentiabte programming

In view of what will be considered below, we may restrict ourselves to the collection 9 o of subsets I yielding n o n e m p t y O~. Definition 3.4. L e t x0 ~ bd S; we call the normal subcone to Sh at Xo the following nonempty closed cone

(3.15)

M ( Sh ; XO) = I,J~ N ( Sh ; Xo), IE~ m ~o~ where

~O~

--,

When there is only one function h involved, the covering of Oh reduces to {O~, Og}. Moreover, the closest points of x E O~ in S~ and Sh are the same (likewise mutatis mutandis when x E Og). Thus, N(Sg; xo) is nothing more than N-,o~ (Sh; x0) and the definition (3.15) reduces to our earlier definition of normal subcone (cf. (3.10)). As for the normal cone to Sh at x0, it follows from its definition that N ( S h , Xo) = el{co M ( S h ; Xo)},

as it was already observed in the one-equality case [8, Proposition 5]. The calculation of M ( S h ; Xo) is not as cumbersome as it might appear at the first glance, even if OhI is not open as a general rule. Referring to (1.3) ~ and (1.3), we observe that for any open subset O containing Oh~ we have the following estimate: N (Sh; x0)C el co/lim v . ( x , - ~,)} "~ O ~

k n--~

(3.16) '

where are considered all sequences with v , - 0, x, converging to x0 in O and having closest point ~ in S. The next simple example will provide a feeling for the nature of the different /V-~oh (Sh; x0). Example 5 (see Fig. 2). Let h = (ht, h2)T:R3--~R 2 be defined by

hl(x~, x2, x3) = x~ + x~ + Ix21+ x3, h2(x~, x2, x3) = x~. At x0 = (0, 0, 0), the 'normal cone relative to 0~', N-.oh (Sh;xo) is of the following form

N o / ( S h ; x o ) = { X u + ~ v + ~ w IX ---0,~ ->0, ~->01, with

J.-B. Hiriart- Urruty / Optimality conditions in nondifferentiable programming

U =

U =

,

(1) 0 ,

v =

,

v=

w

,

=

1=0,

if I = { 1 } , -

(-i),

w=

-1 1l =

if

w=

0

U -----

-

137

v=

(~ -1

-

w =

,

ifl={2},

(~ 1

if I = {1,2}.

1

G e n e r a l i z i n g the o p t i m a l i t y c o n d i t i o n d e r i v e d w h e n o n l y o n e f u n c t i o n h was i n v o l v e d [8, T h e o r e m 9], w e h a v e :

X3

/ t~ x

F i g . 2.

X2

138

J.-B. Hiriart-Urruty/Optimality conditions in nondifferentiable programming

Theorem 3.5. A necessary condition for Xo ~ bd S to be a local minimum of f on S is that there exists I E ~ o such that 0 ~ Of(xo) + _NoR(Sh; x0).

(3.17)

Proof. By substituting for f, if necessary, a Lipschitz function coinciding with f around x0 (cf. [13]), we may suppose that (i) f is Lipschitz on all of R p, (ii) limllxll-,~ f(x) = + oo. Moreover, there is no loss of generality in considering x0 as a global isolated minimum of f on S. Let k be chosen so that k>s u ff(x)-f(y)

y}.

x0 is clearly the unique (global) minimum of f + kds on the whole space. Now, we set for ~ > O, S, = {x ~ R p I d s ( x ) --- E},

and f,(x) = f ( x ) + k max{O, ds(x) - e}. f, coincides with f on S, and one easily checks that ds, = max{O, ds - ~}. Let x, minimize the perturbed function f, on RP; then necessarily x, E S, and x,-*x0 when e ~ O. Necessary conditions for optimality yield that there exists k, E [0, k] such that 0 ~ Of(x,) + k, Ods(x,).

(3.18)

We now consider all the above having been done for a sequence {En} converging to 0. Subsequencing if necessary, we may suppose that x,, E O~ for a certain I ~ ~9~ Then, by invoking the upper-semicontinuity of the set-valued mappings Of and ads, we derive from (3.18) that

0 ~ Of(xo) + _Nh (Sh; Xo). Hence the result (3.17) is proved. Remark. In order to maintain a certain coherence in the concepts we have introduced, the normal subcone M(Sh;Xo) is defined as the union of convex cones N-,o~ (Sh; x0). Actually, the proof of the theorem above shows that the optimality condition could have been written with R+ 0-,o~ ds(xo) instead of

N-.o~ (Sh; Xo).

J.-B. Hiriart-Urruty[ Optimality conditions in nondifferentiable programming

139

References [1] F.H. Clarke, "Generalized gradients and applications", Transactions of the American Mathematical Society 205 (1975) 247-262. [2] F.H. Clarke, "A new approach to Lagrange multipliers", Mathematics of Operations Research 2 (1976) 165-174. [3] F.H. Clarke, "On the inverse function theorem", Pacific Journal of Mathematics 64 (1976) 97-102. [4] F.H. Clarke, "Generalized gradients of Lipschitz functionals", Mathematics Research Center, Technical Summary Report, University of Wisconsin, Madison, WI (1976). [5] J.-B. Hiriart-Urruty, "Conditions n6cessaires d'optimalit6 en programmation non diff6rentiable", Note aux Comptes Rendus de I'Acad~mie des Sciences de Paris S&ie A 283 (1976) 843-845. [6] J.-B. Hiriart-Urruty, "On necessary optimality conditions in nondifferentiable programming", Mathematical Programming 14 (1978) 73-86. [7] J.-B. Hiriart-Urruty, "Gradients g6n6ralis6s de fonctions compos6es. Applications", Note aux Comptes Rendus de l'Acad~mie des Sciences de Paris S~rie A 285 (1977) 781-784. [8] J.-B. Hiriart-Urruty, "Tangent cones, generalized gradients and methematical programming in Banach spaces", Mathematics of Operations Research 4 (1979) 79-97. [9] J.-B. Hiriart-Urruty, "New concepts in nondifferentiable programming", in: J.-P. Penot, ed., Journ~es d'analyse non convexe, Bulletin de la SocWt~ Math~matique de France M~moire 60 (1979) 57-85. [10] J.-B. Hiriart-Urruty, "Refinements of necessary optimality conditions in nondifferentiable programming I", Applied Mathematics and Optimization 5 (1979) 63-82. [11] J.-B. Hiriart-Urruty, "On the extension of two theorems in general topology. Applications", Technical Note, University of Kentucky, Lexington, KY (1978). [12] J.-B. Hiriart-Urruty, "Th6or~mes de valeur moyenne en analyse non diff6rentiable: cas des fonctions localement Lipschitz h valeurs vectorielles", Note aux Comptes Rendus de I'Acad~mie des Sciences de Paris S~.rie A 287 (1978) 751-753. [13] J.~ Hiriart-Urruty, "Extension of Lipschitz functions", Journal of Mathematical Analysis and Applications 77 (1980) 539-554. [14] R. Mifflin, "Semismooth and semiconvex functions in constrained optimization", S I A M Journal on Control and Optimization 6 (1977) 959-972. [15] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, NJ, 1970). [16] R.T. Rockafellar, "Generalized directional derivatives and subgradients of nonconvex functions", Canadian Journal of Mathematics 32 (1980) 257-280. [17] R.T. Rockafellar, "Clarke's tangent cone and the boundaries of closed sets in R n'', Nonlinear Analysis 3 (1979) 145-154. [18] R.T. Rockafellar, "Directionally Lipschitzian functions and subdifferential calculus", Proceedings of the London Mathematical Society 39 (1979) 331-355.

Mathematical Programming Study 19 (1982) 140-152. North-Holland Publishing Company NECESSARY CONDITIONS FOR E-OPTIMALITY P. L O R I D A N

Universit~ de Dijon, Dijon, France Received 4 October 1979 Revised manuscript received 18 October 1980 This paper consists in a study of necessary conditions in mathematical programming with errors, by introducing the notion of regular approximate solutions up to ~. These solutions are 'almost' stationary and we obtain Kuhn-Tucker conditions up to ~ with no constraint qualification. A duality result is given by using an e-Lagrangian functional.

Key words: Mathematical Programming, Approximate Solution, Kuhn-Tucker Conditions up to ~, e-Semiconvex Function.

I. Introduction In this paper we are concerned with necessary conditions for approximate solutions of mathematical programming problems. We first recall a result due to Ekeland [4]. Then, in Section 3 of this paper, we introduce the notion of regular approximate solutions and we demonstrate that these solutions are stationary up to E. In Section 4, we obtain also some properties by introducing the concept of ~-semiconvexity which extends the definition given by Mifflin [9]. In the next sections, we give K u h n - T u c k e r conditions up to E with no constraint qualification and we present some duality results by introducing an E-Lagrangian functional.

2. Preliminary results In order to prove the main results of this paper, we shall use the following theorem of Ekeland [4, 5]:

Theorem 2.1. Let V be a real Banach space and J a lower semicontinuous functional defined on V, with values in R U {+~}, J : V-~ R O {+o0}, not identically +oo and bounded f r o m below. Let ~ > 0 . For every point u E V satisfying J ( u ) <- inf{J(v) I v E V}+ ~, there exists some point u~ ~ V such that: J(u~) <- J ( u ) <- inf{J(v) I v E V}+ e,

(2.1)

Ilu - u~ll - V ~ ,

(2.2)

J(u~) <- J ( v ) + ~/-~ ]Iv - u,[[ f o r all v e V.

(2.3)

140

141

P. Loridan/ Necessary conditions [or e-optimality If J is Gfiteaux-differentiable, then u, satisfies :

IIY'(u,)U, -< V L

(2.4)

where J'(u,) denotes the Gfiteaux-derivative of J at u~ and [[. [[, the dual norm.

Remark 2.1. No compacity assumptions are made. So the infimum is not necessarily attained. Remark 2,2 By considering the proof given by Ekeland [4], the result (2.1) can be written more precisely: J(u,) + x / ~ Ilu - u, II-< J(u).

(2.5)

Corollary 2.1. Let K be a closed subset of V and let J : K ~ R be lower semicontinuous and bounded from below on K. If u ~ K satisfies J(u)<_ inf{J(v) [ v E K } + e, then there exists u, ~ K such that: Ilu - u,II ~ x / L J(u,) - J(v) § x/~ IIv - u, II for all v E K.

(2.6)

The proof uses Theorem 2.1, substituting the functional J by the functional H: H(v)=J(v)

for a l l v E K ,

H(v) = +~

for all v ~ K.

This corollary has been used in particular by Clarke [2].

3. Necessary conditions for regular approximate solutions. Stationarity up to We consider the mathematical programming problem: (P)

inf{J(v) I v E K}

and we denote by the following hypothesis (H): Hypothesis (H): (1) K is a non empty closed subset of V. (2) J is a real valued functional lower semicontinuous and bounded from below on K.

We begin with some definitions. 3.1. Approximate solutions: Definitions Definition 3.1. Let ~ > O. If u E K satisfies J(u) <- J(v) + ~ for all v E K, we say that u is an approximate solution up to E for the problem (P).

P. Loridanl Necessary conditions for e-optimality

142

Definition 3.2. We shall say that u is an e-quasisolution of (P) if u E K satisfies J ( u ) _< J ( v ) + IIv - ull for all v E K. Definition 3.3. An element u E K is said to be a regular approximate solution up to E for the problem (P) if the following two conditions are satisfied: (1) u is an a p p r o x i m a t e solution up to e (according to Definition 3.1), (2) u is an E-quasisolution of (P) (according to Definition 3.2). R e m a r k 3.1. Obviously, every E-quasisolution of (P) is 'locally optimal up to E'. That is to say, if u is an E-quasisolution, there exists a ball B around u with radius equal to X/~ such that J(u) <- J(v) + E for all v E B n K. With Corollary 2.1, there exists at least one regular a p p r o x i m a t e solution up to E for the problem (P). In other words, there exists an e-quasi solution u which is also 'globally optimal up to E':J(u)--< J ( v ) + E for all v ~ K.

3.2. Stationarity up to E and necessary conditions With the definition of a regular a p p r o x i m a t e solution, we shall introduce the notion of 'stationarity up to E' generalizing the classical one. We denote b y J'(u; d) the directional derivative of J at u in the direction d:

J'(u ; d) = lim J(u + td) - J ( u ) t--,o

t

1>0

and by using the cone of feasible directions to K at u, denoted b y F ( K , u) as in [1], we have the following definition:

Definition 3.4. If J has a directional derivative at u E K for e v e r y d in F ( K , u), we shall say that u is stationary up to E if J'(u ; d) + "X/-~Ildll >- 0 for all direction d ~ F ( K , u). Proposition 3.1. If the hypothesis (H) is satisfied, if J has a directional derivative at u for all [easible directions d E F ( K ; u), if u, is a regular approximate solution up to E for (P), then it is also a stationary point up to E. Proof. By using Corollary 2.1 with v = u, + td, where d is some feasible direction at u, and t > 0 such that u, + td E K, we obtain: J(u~) <_J(u, + td) + x/-~ tlldll. Letting t ~ O, we h a v e obviously:

J'(u,; d) +

Ildll --- 0

and the proof is achieved.

P. Loridan [ Necessary conditions .for e-optimality

143

Now, following [10], we shall extend the notion of an a p p r o x i m a t e stationary point for the case where K has an e m p t y interior. A vector y is called a tangent direction to K at u if there exists a sequence {uk}C K satisfying uk# u, Ilu - ull-~0 and Uk - - U

Ilu - ull

y

when k + ~ .

The sequence {uk} is said to define the tangent direction y. As in [10], the set of all tangent directions to K at u will be denoted Y ( K , u). It is noted that the cone of tangents to K at u is the cone consisting of non negative multiples of directions y U Y ( K , u). N o w , we first recall the notion of y-derivative as defined in [10].

Definition 3.5. L e t y ~ Y ( K , u). If there exists a real n u m b e r ~J(u, y) such that lim

J ( u 0 - J(u)

= ~J(u, y)

for e v e r y sequence { k } c K which defines y, then 8J(u, y) is called the yderivative of J at u. By using Definition 3.5, we introduce another concept of stationarity up to 9 with the next definition.

Definition 3.6. If J has a y-derivative at u E K for e v e r y y E Y ( K , u), we shall say that u is a stationary point up to 9 if ~J(u, y ) + ~ / ~ - > 0

for all y E Y ( K , u).

Proposition 3.2. If the hypothesis (H) is satisfied, if J has a y-derivative at u for every y E Y ( K , u), if u~ is a regular approximate solution up to 9 for the problem (P), then u, is also a stationary point up to ~, according to Definition 3.6. Proof. L e t {Uk} be a sequence which defines the direction y at u~. F r o m Corollary 2.1, with v = Uk, we have:

J(u,) <- J(uk) +

Ilu - u, ll.

Then, dividing b y I[uk - u,][ and letting k ~ + ~ , we obtain obviously:

M(u~, y) + X/~-> 0

for all y E Y ( K , u).

Remark 3.2. In the case where K = V and J has a G~teaux-derivative J'(u~) at u,, then, from T h e o r e m 2.1, we have

IIJ'(u,)ll,

VZ

144

P. Loridan / Necessary conditions for e-optimality

It is another necessary condition in the case where (P) is an unconstrained optimization problem and we shall say again that u, is a stationary point up to e. 4. e-semiconvex functionals In order to obtain a characterization for e-quasisolutions of (P), we now introduce the concept of 'e-semiconvexity' by extending the definition given by Mifflin [9]. We define the generalized directional derivative J~ d) at u, in the direction d as in [2] by: J~

; d ) = lim sup J(u + h + td) - J(u + h) t-,o+ t h-*O

In the sequel, we shall refer to the following definition (with E > 0). Definition 4.1. L e t K be a subset of V; J : V ~ R is E-semiconvex at u E K if: (1) J is Lipschitz on a ball about u, (2) J is ~-quasidifferentiable (or ~-regtrlar), that is to say J'(u, d) exists and 0_< J~ d ) - J'(u; d) <-X/~ Ildll for each d E V, (3) u + d E K and J'(u; d) + X/~ Ildll - 0 imply that J(u + d) + X/~ Ildll -> J(u). Remark 4.1. A c o n v e x functional on V is also E-semiconvex. For E = 0, we have the concept of semiconvexity defined by Mifflin [9]. By including the notion of E-quasidifferentiability, we obtain the next proposition similar to [9, Theorem 8]. Proposition 4.1. If J is E-semiconvex on a convex set K, u E K and u + d E K, then

J(u + d) + x/~ Ildll -< 1(u) implies that J ' ( u ; d) + ~/~ [Id[[-< O.

Proof. The proof is similar to the one given in [9]. We suppose, for contradiction purposes, J(u + d) + X/-~ Ildll -< 1(u) and J'(u; d) + X/~ I[dll > 0. Then, there exists t>0suchthatt
+ V ~ Ildll > 0,

and by setting f ( t ) = J(u + td) + X/-~ t[ldl], we obtain f(t) > [(0). But we have supposed f(1) -< f(O). So if /- maximizes the continuous function f ( t ) over [0, 1], we obtain:

f(1) = 1(u + d) § V ~ Ildll ~ J(u) = f(o) < f({).

P. Loridan [ Necessary conditions for e-optimality

145

Then, by the maximality of {, we have

J'(u + i-d; d) + ~/~ Ildll -< 0,

(4.1)

J'(u + ? d ; - d ) + x/~lldll ~ 0 .

(4.2)

By the E-quasidifferentiability of J, we obtain from (4.1) and (4.2):

J~

+ ?d; d)<-J'(u + Ed; d ) + x/~lldll ~ 0 ,

(4.3)

J~

+ Yd; - d) <- J'(u § ?d; - d) + X/~ Ildll --- 0.

(4.4)

As in [9], we have obviously J~

+ {d; d) = J~

+ i-d; - d ) = 0, and we obtain:

J'(u + ?d; d) + V ~ Ildll -- 0. Finally, we have J(u + d) + x/~ Ildll -> Y(u + {d) + V-~ Ildll

(4.5)

by using arguments similar to those given in [9]. The inequality (4.5) gives f(1) - [(t-), which contradicts f(1) -< .f(0) < .f(i-). Corollary 4.1. If J is e-semiconvex on K, if the hypothesis (H) is satisfied, i.f u E K verifies J(u) <- J(v) + 9 all v ~ K, there exists an approximate regular solution up to 9 denoted by u, such that:

J'(u; u, - u ) + x/~ Ilu,- ull-< 0. Proof. Use Remark 2.2 and Proposition 4.1 with d = u, - u. Proposition 4.2. If the hypothesis (H) is satisfied (with K convex), if J is 9 -semiconvex on K, then every 9 u, of (P) is characterized by

1'(u,; v - u,) + x/~ Ilv - u, II -> 0

for all v E K.

Proof. Obviously, if u, is an 9 u, is a stationary point up to 9 according to Proposition 3.1, with d = v - u,, v E K, and so we have

J'(u,; v - u , ) + x / ~ l l v - u, ll---0

for all v E K

This condition is sufficient from the definition of the ~-semiconvexity (J is 9 -semiconvex at e v e r y point of the c o n v e x K).

5. Generalized Kuhn-Tucker conditions up to E

Hypothesis (H'): Let V be a real Banach space. Let J be a real valued functional defined on V, locally Lipschitz and bounded from below on V. We are concerned with the

P. Loridan/ Necessary conditions for ~-optimality

146

following mathematical programming problem:

(P)

InfU(v) I v ~ K}.

K is a non empty subset of V defined by: K = {v E V [ Gi(v) -< 0, 1 -< i -< n ; Hi(v) = O, 1 <- j <- p}, where Gi and Hj are real valued functionals defined on V and locally Lipschitz, l<_i<_n, l_<j___p. Let E > 0. We introduce the e-feasible set K,,

K, = {v ~ V IG,(v)<-~/-~,I <- i < - n ; -~/-~ <<_Hj(v) <_~/-~, 1 <- j -
(1) u, E K , , (2) J(u,) <- J(v) + e for all v E K, (3) J(u,) <- J(v) + V-~ IIv - u, II for all v E K. Remark 5.1. If u~ E K, then u, is a regular approximate solution up to e, according to Definition 3.3. Lemma 5.1. Let f be a locally Lipschitz functional defined on V. Let h be the functional defined by h(v) = [f(v)] 2 for all v E V. Then the generalized gradient of h at v, denoted Oh(v), satisfies the following inclusion:

Oh(v) c 2f(v)Of(v). Proof. It is a consequence of [2, Proposition 10].

Remark 5.2. In the sequel, we shall also use this lemma with a functional f ( v ) = m a x { O , g ( v ) } , where g is a locally Lipschitz functional on V. Then, following [2, Proposition 9] with h(v) = If(v)] 2, we shall have:

Oh(v) C/{0} [2g(v) ag(v)

if g(v) <-0, if g(v) > O.

So, we can write Oh(v) C 2 max{0, g(v)} 9 ag(v). Theorem 5.1. Suppose the hypothesis (H') is satisfied. Then there exists an

almost regular approximate solution uo up to e, and real numbers )~i(e)>-O, l <- i <- n, /xi(e), 1 - < j - < p , such that: Ai(e) > 0

for all i E I(e) = {i I 0 < Gi(u~) -< ~/~},

(5.1)

hi(e) = 0

if Gi(u,)-< 0,

(5.2)

P. Loridan/ Necessary conditionsfor ~-optimality OE OJ(u,)+ie~,~,)h,(~) OG,(u~)+ ~ l.q(e)c3Hj(u~)+~f~ B*,

147 (5.3)

where B* is the unit ball in V*, the topological dual space of V. Proof. We introduce a penalized functional (see [61):

J,,,(v) = J(v) + ~" ril [max{0, G,(v)}]2+ P 1 [Hi(v)] 2, si

where ri and s i are real positive numbers, 1 - < i -< n, 1-< j-< p, and we consider the mathematical programming problem: (P~,J

inf{s

l v a V}.

Obviously, J~,s is a locally Lipschitz functional defined on V, bounded from below. Then, by using Theorem 2.1, there exists u~ ~ V (uE depending on r and s) such that: J,,s(u,) -< Jr.s(v) + ~ for all v E V,

-u,II

for all v E V.

(5.4) (5.5)

With inequality (5.4), we obtain also: J(u,)-< s and, with

J(v) + 9

for all v E K

inLevJ(v) <-J ( u J , we have: ~ ~il [max{0'

Gi(uE)}]2+~ 1 <[Hj(u')]2 ' -- veKinfJ ( v ) - ~evinfJ(v)+e.

So, max{0, Gi(u,)} ~ 0 if ri ~ 0, 1 --- i -< n, and Hi(u,) -->0 if sj ~ 0, 1 -< j -< p. Consequently, there exists ri(e), 1-< i -< n, and sj(e), 1 <-j <-p, such that ri -< r~(e) implies that max{0, G~(u,)} -< X/~, 1 -< i -< n, sj -< sj(e) implies that [Hi(u,)] 2 -< e, 1 -< j -< p, that is to say, -X/~--< Hi(u,) -< ~/~,

l<--j<_p.

So, with such a choice of ri, 1 < i <- n, and sj, 1 < j - p, we can define an element u, E K, which is an almost regular approximate solution, according to Definition 5.1 (by using the inequalities (5.4) and (5.5)). Besides, u, satisfies the necessary condition (see [2]) 0 E Os + X/-~B*. With L e m m a 5.1 and the results of [2], we then deduce

0~- OJ(u')+2~--l= G~f(U')ri OGi(u,)+2~= ~l Hi(u,)OHi(u,)+V~B. ' where

G~(v) = max{O, Gi(v)}, 1 -< i -< n.

148

P. Loridan[ Necessary conditions [or e-optimality

Finally, we obtain the result (5.3) by setting: ~j( 9 = 2 Hi(u,),

1 -< j a p,

)~i(9 - 2G?(u,),

1 -< i -< n.

sj

ri

The conditions (5.1) and (5.2) are then trivially satisfied. Remark 5.3. Conditions (5.1) to (5.3) are called generalized Kuhn-Tucker conditions up to e. For related work we refer to [2] where Clarke derives necessary conditions in Lagrange multiplier form for a local solution of (P) by making use of a normality condition. Remark 5.4. If we consider the problem (Q): (Q)

inf{J(v) I v ~ K fq C},

where C is a closed subset of V, if the hypothesis (H') is satisfied, then, by using the results of Clarke [2] about the distance function dc(v) and the normal cone Nc(v), it is easy to demonstrate the following theorem: Theorem 5.2. Let 9 be a real positive number. There exists an almost regular approximate solution u, (up to e) /or the problem (Q), real numbers ~i(9 1 -< i -< n,/xj(e), 1 <- j <- p, and a point z, in V* such that: (1)

x,(~) = 0

i/

G~(u,) -< 0,

(2)

X~(e) > 0

/or all i ~ I(e) = {i [ 0 < Gi(u,) -< X/~},

(3)

z, E

(4)

- z , ~ Nc(U,).

OJ(u,)+ i ~l(e ~, ) ~.i(e) OGi(u,)+ TI= ff~, I~j(e) OHj(u,)+VeB*,

Remark 5.5. In the differentiable case we obtain, with the next theorem, K u h n Tucker conditions up to e with no constraint qualification [8]. For related work, we refer to [4] where regularity assumption is made. Theorem 5.3. I/ the hypothesis (H') is satisfied and i/ J, Gi, 1-- O, 1 <- i <- n, I~j(e), 1 <- j <- p, such that •i(E) > 0

for all i ~ I(e),

x,(~) = 0

i / O ~ ( u , ) -< 0,

(5.6)

P. Loridan [Necessary conditions for E-optimality

149

Remark 5.6. (1) Theorem 5.3 is not necessarily a corollary of Theorem 5.1 but the proof is similar. It is also valid when the hypothesis 'locally Lipschitz' is deleted and replaced by 'J, G~, l<_i<_n, and H~, 1 - < j - < p , are lower semi continuous functionals'. (2) Theorem 5.3 can be considered as a corollary of T h e o r e m 5.1 when the following hypothesis is satisfied: J is bounded from below on V, J, Gi, 1 -< i -< n, and H i, 1 -< j -< p, are continuously differentiable on V. In effect, with these conditions, the hypothesis (H') is verified, and we derive the inequality (5.6) from condition (5.3) by making use of the following property [2]: if a functional [ admits a continuous derivative f', then the generalized gradient af(v) = ff'(v)}.

6. Lagrangian duality up to 9

We consider now the problem (P) with no equality constraints, that is to say:

K ={v ~ V JGi(v)<-O, 1 <-i <-n}. The e-feasible set is K, = {v E V [ Gi(v) <- X/~, 1 <- i <- n}. With this problem (P), we associate the Lagrangian functional n

L(v,h)=J(v)+~hiGi(v)

for a l l v E V a n d h i E R ,

1--
We denote by h the vector h = (h~, h2, ..., h,). We define an E-Lagrangian functional by setting:

L,(v, A; z, I~) = L(v, A) + ~/~ IIv - zll- x/~ where (z,/x) E V • R" and IIx choose Ilxll. = ETa, Ix, I.

- NI,,

11. denotes a norm on R". In the sequel, we shall

Definition 6.1. L e t K * be K * = {h E R" ] hi - 0, 1 -< i -< n}. We shall say that (a,)~) is a quasi saddlepoint for the r L, on V x K * if a E V, h E K * and

L,(0, h ; a, h) -< L,(a, h; a, h) -< L,(v, h; a, h) for all v E V and h E K*, that is to say: L(a, A) - V ~ IIx -

11. -< L(a,

~) <- L(v,

h) + V-~ IIv - all.

If the hypothesis (H') is satisfied, there exists u, E K, and hi(e), 1 - < i -< n, verifying Theorem 5.1. Let us denote by A(E) the vector ()~l(E), A2(r . . . . . h,(r Then, we have the following theorem:

P. Loridan[Necessary conditions for e-optimality

150

Theorem 6.1. If the hypothesis (H') is satisfied, if J'(u,, d) exists and J~ d) = J'(u,, d) for each d E V, if G~(u,, d) exists and G~ d) = G~(u,, d), 1 <- i <- n, for each d ~ V (J and G~ are said to be quasidifferentiable as in [9]), if the functional J(v)+~,~=lX~(()Gi(v) is e-semiconvex at u , ~ V, then (u,,)t(e)) is a quasi saddlepoint for the e-Lagrangian L, on V x K*. Proof. Let I be I = {1, 2 ...... n} and I(e) = {i ~ I [ Gi(u,) > 0}. With Theorem 5.1,

~,i(~) = 0 if i~ I(E). So we have: n

n

and, since ~ , , )

X~G~(u,)~ 0 for all X E K*, we obtain the following inequality:

n

(x,(~) - x,)a,(u,) -> ,~,~ (x,(~) - x,)C,(u,). By using Gi(u,) ~- X/~, i ~ I(e), we then deduce:

n

We finally obtain: n

( x , ( ~ ) - x,)G,(u,) >_ - V - ~ IIx - x(~)llo

for all ;~ E K * ,

n

J(uA + ~= X,(e) Gi(u,) - L(u,, X) - X/~ IX - ,~(e)lln for all X ~ K*. This gives the first inequality for a quasi saddlepoint. We now use the result (5.3) by noting that there exists z* E 0J(uA, z~, E 0G~(u~), 1 --- i -< n, and b* ~ B* such that n

z* + ~ ;~i(e)z~,, +X/~ b* = 0. By using the characterization of the generalized gradient of Clarke [2] and with the hypothesis of quasidifferentiability for d and Gi, we finally obtain: n

n

--- -x/~lldll for all d E V, where ( . , .) denote the canonical bilinear form on V* x V, where V* is the topological dual space of V. By using the definition of E-semiconvexity at u,, with d = v - u,, v ~ V, we then deduce:

P. Loridan/ Necessary conditions for ~-optimality n

151

n

s ( v ) + ~ h,(e) c,(v) + x G IIv - u, ll-> J(u~) § ~ h,(e) C,(u,) So we obtain the second inequality for a quasi saddlepoint:

L(v, h ( e ) ) + V ~ l l v - u, ll-> L(u,, h(e))

for

all v E V

and the proof is achieved. We also have, trivially, the following result: Corollary 6.1. If the hypothesis (H') is satisfied, if J and Gi, 1 <- i <- n, are convex functionals, then (u,, h(e)) is a quasi saddlepoint for the e-Lagrangian on VxK*. Theorem 6.2. If (u,, h(e)) is a quasi saddlepoint for the e-Lagrangian on V x K*, then u, is such that: (1)

J(u~) <- J(v) + X/~ [Iv - u~[[ for all v verifying Gi(v) <- Gi(u~), 1 <- i <- n,

(2)

Gi(u,) <- \ / ~ ,

(3)

hi(e) ~ 0

i <- i <- n,

implies -'X/-~ <- Gj(u,).

Proof. From the definition of a quasi saddlepoint, we have: n

s(u,) + ~ hi(,) Gi(u,) <- s(v) + ~ hi(e) Gi(v) + v'~ IIv - u, II and we then deduce the inequality (1). Now, we use n

s(u.) +

n

hici(u.)-v

n

Ihi-

for all h ~ K*, and, in particular, with hk = 1 + hk(e) and hi = hi(e) for all iS k, we obtain:

hkGk(u,) - ~/~--< h~(e) Gk(u,),

ak(u,) <--X/~. This result can be obtained for all k, 1 -< k -< n. So we have demonstrated (2). Finally, if h i ( e ) ~ 0 for some j, let us choose h i such that 0 < hi < hi(e) and hi = hi(e) for i ~ j. We obtain:

hj Gj(u,) - x G (hi(E) - hi) --< hi(e) Gj(u,) and the next result:

-~/~ - G~(u,).

152

P. Loridan/ Necessary conditions for e-optimality

Remark 6.1. Theorem 6.2. is a little a n a l o g o u s w i t h t h e ' e p s i l o n t h e o r e m ' of E v e r e t t [3].

References [1] M. Bazaraa and C. Shetty, "Foundations of optimization", in: Lecture notes in economics and mathematical systems 122 (Springer, Berlin, 1976). [2] F. Clarke, "A new approach to Lagrange multipliers", Mathematics of Operations Research 1 (1976) 165-174. [3] H. Everett, "Generalized Lagrange multiplier method for solving problems of optimum allocation of resources", Operations Research 11 (1963) 399-417. [4] I. Ekeland, "On the variational principle", Journal of Mathematical Analysis and Applications 47 (1974) 324-353. [5] I. Ekeland and R. Temam, Analyse convexe et probl~mes variationnels (Dunod, Paris, 1974). [6] A. Fiacco and G. McCormick, Nonlinear programming (Wiley, New York, 1968). [7] P. Loridan, "Solutions approch6es de probl/~mes d'optimisation", Communication au Colloque d'Analyse Num6rique, Imbours (1977). [8] P. Loridan, "Conditions de Kuhn et Tucker ~t ~ pros pour des solutions approch6es de probl~mes d'optimisation avec contraintes", Comptes rendus Acad~.mie des Sciences de Paris 285 (1977) 449-450. [9] R. Mifflin, "Semismooth and semiconvex functions in constrained optimization", SIAM Journal on Control and Optimization 15 (1977) 959-972. [10] J. Zang, E. Choo and M. Avriel, "On functions whose stationary points are global minima", Journal of Optimization Theory and Applications 22 (1977) 195-208.

Mathematical Programming Study 19 (1982) 153-166 North-Holland Publishing Company

CHARACTERIZATIONS OF BOUNDED SOLUTIONS OF LINEAR COMPLEMENTARITY PROBLEMS* O.L. MANGASARIAN Computer Sciences Department, University of Wisconsin Madison, WI 53706, U.S.A. Received 28 December 1979 Revised manuscript received 19 November 1980

A number of equivalent characterizations for the existence and boundedness of solutions of the linear complementarity problem: Mx + q >-O, x ~ O, xT(Mx + q) = 0 where M is an n x n real matrix and q is an n-vector, are given for the case when M is copositive plus. The special case when M is skew-symmetric covers the linear programming case. One useful characterization of existence and boundedness of solutions is given by solving a simple linear program. Other important characterizations are the Slater constraint qualification and the stability condition that for all arbitrary but sufficiently small perturbations of the data M and q which maintain copositivity plus, the perturbed linear complementarity problem is solvable and its solutions are uniformly bounded. An interesting sufficient condition for boundedness of solutions is that the linear complementarity problem have a nondegenerate vertex solution. Another result is that the subclass ~ of copositive plus matrices for which the linear complementarity problem has a solution for each q in R", that is ~ C Q, coincides with the subclass of copositive plus matrices for which the linear complementarity problem has a nonempty bounded solution set for each q in R".

Key words: Linear Complementarity Problem, Solution Set, Boundedness, Copositive-plus Matrices.

1. Introduction The principal purpose of this paper is to give a number of characterizations for the existence and boundedness of solutions of the linear complementarity problem of finding an x in the n-dimensional real Euclidean space R" such that Mx+q>-O,

x>-O,

xT(Mx+q)=O,

(1)

where M is a given n x n real matrix, q is a given vector in R" and T denotes the transpose. We shall refer to this problem as LCP(M, q). Existence of solutions for the linear complementarity problem has been investigated by many authors [3, 5, 7, 8, 15]. One well-known existence result [2, 5] is that if the feasible region of (1), S(M, q) = {x I Mx + q >- O, x >- 0}

(2)

is nonempty, then the solution set of (1), S(M, q) = {x I Mx + q >- O, x >- O, xT(Mx + q) = 0} * Research supported by National Science Foundation Grant MCS--7901066. 153

(3)

154

O.L. Mangasarian/ Bounded solutions o[ linear complementarity problems

is nonempty when the matrix M is copositive plus, that is, (a) (b)

x - 0 implies xTMx >-0 (copositive), x>-O, xTMx=O i m p l y ( M + M X ) x = 0 (plus).

(4)

Copositive plus matrices include positive semidefinite matrices and positive matrices. Other existence results are given in [5, 11, 12, 17]. Global uniqueness of solution of the linear complementarity problem has been investigated in among others [1, 14, 15] and local uniqueness in [13]. Robinson [20] and Doverspike [4] have characterized nonemptiness and boundedness of S(M, q) by the nonemptiness of S(/f/, ~]) for all (/~/, q) sufficiently close to (M, q) for the cases when M is positive semidefinite and copositive plus respectively. In addition, for positive semidefinite M, Robinson has further characterized the nonemptiness and boundedness of S(M, q) by the nonemptiness and uniform boundedness of ~(~7/, 4) for positive semidefinite ~7/. One of our characterizations, Theorem 2(xvi), extends this result of Robinson to the copositive plus M and ~/. Another characterization of the nonemptiness and boundedness of S(M, q), Theorem 2(x), extends the corresponding boundedness results of Williams [23] for a dual pair of linear programming problems to the linear complementarity problem with a copositive plus matrix. A useful feature of another characterization, Theorem 2(vii), is that without knowing whether (1) has a solution and without knowing any of its solutions we can determine if its solution set S(M, q) is nonempty and bounded from the following equivalence

S(M,q) is nonempty} ~ [ M a x { e T u l M T u _' 0 , qTu<_0, < u>_0}=0 and bounded copositive~ plus

(5)

where e is a vector of ones in R n. Note that the right-hand side of the equivalence (5) can be easily checked by solving a simple linear programming problem. Another interesting characterization of the nonemptiness and boundedness of S(M, q) for copositive plus matrices, Theorem 2(iii), is that the feasible region S(M, q) be stable [19] or equivalently that it satisfies the Slater constraint qualification [9]. The principal results of the paper are contained in Theorem 2 which gives a number of equivalent characterizations for the nonemptiness and boundedness of the solution set S(M, q). The first eight characterizations of Theorem 2 are stability or constraint qualification conditions for the feasible region S(M, q) and they do not require any assumptions on the matrix M. The last eight characterizations of Theorem 2, however, make essential use of the copositivity plus of the matrix M. The equivalence between (ix) and (xv) of Theorem 2 is due to Doverspike [4] and is stated separately as Theorem 1. Corollary 1 shows that whenever a linear complementarity problem with a copositive plus matrix has a nondegenerate vertex solution, its solution set must be bounded. Corollary 2 establishes the characterization (5) stated above. Corollary 3 characterizes the

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

155

subclass of copositive plus matrices which is in Q, that is the class of matrices for which the linear complementarity problem has a solution for each q in R". Corollary 4 specializes Theorem 2 to the case of a symmetric M and uses the same condition as that of [10, Thoerem 2.2] which ensures the boundedness of the iterates of the algorithms of [10]. We briefly describe now the notation of this paper. All matrices and vectors are real. For the m x n matrix A, row i is denoted by Ai and the element in row i and column j by Aij. For x in the real n-dimensional Euclidean space R ", element j is denoted by xj. All vectors are column vectors unless transposed by the superscript T. For I C {1..... m} and J C {1..... n}, A~ denotes the submatrix of A with rows Ai, i E / , A~j denotes the submatrix of A with elements A0, i E / , j E J, and xj denotes x~, i E J. Superscripts such as A ~, x ~denote specific matrices and vectors and usually refer to elements of a sequence. For simplicity we shall write A iT for (Ai) T. The Euclidean norm (xTx) in of a vector x in R" will be denoted by [[xl[ and the corresponding induced matrix norm maxllxll=~llAxl]will be denoted by IIAII. The vector e will denote a vector of ones usually in R n. A partition {I, J} of the set of integers {1. . . . . n} is defined as I C{1 . . . . . n}, J C { 1 ..... n } , I tA J ={1 . . . . . n} and l A J = O .

2. Principal results We begin with a theorem which was established by Robinson [20] for the case when M is positive semidefinite and extended by Doverspike [4] to Eaves' class L of matrices [5] which includes the copositive plus case. This theorem will be utilized in establishing one of the equivalences of Theorem 2. Theorem 1. (see [4]). Let M be an n x n copositive plus matrix and let q be in R n. The following are equivalent: (i) The solution set S(M, q) of the linear complementarity problem (1) is nonempty and bounded. (ii) There exists an ~ > 0 such that the solution set S(I(,I, (1) of the perturbed linear complementarity problem LCP(Jf/, (1) is nonempty for max{ll/f/- MI[, I1(1 - qll} < ,.

Our principal result which follows establishes the equivalence of a number of conditions for the nonemptiness and boundedness of the solution set S(M, q) of the linear complementarity problem (1). Theorem 2. For any n x n matrix M and any vector q in R ~ the statements (i) to (~,iii) below are equivalent. If in addition M is copositive plus, then the statements (i) to (xvi) below are equivalent.

156

O.L. Mangasarian/ Bounded solutions o[ linear complementarity problems

(i) The system Mx+q~>O,

x>-O,~>-O,

has a solution (x, ~) in R ~+l. (ii) The system Mx+q>O,

x>-O,

has a solution x in R ~. (iii) The system Mx+q>O,

x>O,

has a solution x in R ~. (iv) For each h in R ~ the system Mx+q+yh>-O,

x ->0, 3, > 0 ,

has a solution (x, 3') in R n§ (v) There exists a ~ > 0 such that the system Mx + fl >_O,x >-O, has a solution x for each F:l in R ~ such that UFt- qll <- 8. (vi) There exists an 9 > 0 such that the system lf/lx + r >-O, x >-O, has a solution x for each n • n matrix 1Q1and each ~1 in R n such that max(llM

- MII, 114 - qll}

-< ~.

(vii) The system MTu<_O,

q~u<_O,

O<_u~O,

has no solution u in R n (viii) For each (a, a) in R ~§ the set (u I M r u <- a, qXu <- a, u >- O} is empty or bounded. (ix) The solution set S(M, q) of the linear complementarity problem (1) is nonempty and bounded. (x) {x I M x >- O, qTx <-0, ( M + MT)x = O, 0----- x ~ O} = O.

(xi) For some or each solution ~ of the linear complementarity problem (1) the set {x I M x + q >- O, x >- O, qTx ~ O, ( M + MT)(x - ~) = O} is bounded.

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

157

(xii) {(x, u ) [ M x + MTu < q , x <--u } ~ O. (xiii) For each h in R" there exists a positive y such that LCP(M, q + yh) is solvable. (xiv) There exists a ~ > 0 such that LCP(M, q) is solvable for each (1 in R" such that 114 - qll-< (xv) There exists an 9 > 0 such that LCP(AT/,q) is solvable for each n x n matrix 101 and each (1 in R" such that max{llM

- MII, 114 - Oil}-< 9

(xvi) There exist 9 > 0 and a > 0 such that LCP0~7/, q) is solvable for each n x n copositive plus matrix !(/1 and each (t in R" satisfying

max{ll~/- MII, 114 -

qll} -< 9

and furthermore IlxU-< ~ for all x in S(1(/1, q). Proof. (i) 9 (ii). If x solves the system of (ii), then x and ~ = 1 solve the system

of (i). (i)ff (ii). Let (R, 2) satisfy MR + q~ > 0, R > 0, ~-> 0. Because the open set {(x, ~) I Mx + q~ > 0} contains (R, 2) it must also contain (R, ~ + 3) for some sufficiently small positive 8 and hence MR + q(~+ 3 ) > 0 . The point R[(~+ 3) solves the system of (ii). (ii) ~ (iii). Obvious. (ii) ff(iii). Let R satisfy MR + q > 0, R -> 0. Because the open set {x I Mx + q > 0} contains R it must also contain R + ~e for some sufficiently small positive & The point R + ~e solves the system of (iii). (i)r (vii). This follows from Motzkin's theorem of the alternative [9]. (iv) r (vii). Condition (vii) is equivalent to the system MTu<--O,

qTu<--O,

u>--O,

gTu
not having a solution u for each g in R". This in turn is equivalent, by Motzkin's theorem of the alternative, to the system My+q~+g>-O,

y---O,~->O,

having a solution (y, ~) in R "+~ for each g in R". By defining g = q + h, x = y/(1 + ~), ~/= 1/(1 + if) we have that (iv) follows from this last result, while the converse follows by defining h = g, y = x/y, and ~ = 1[7. (iv) ~ (v). Take y < 8/11hl[ if h # 0, otherwise take 7 = 1. ( v ) ~ (vi). Set ~ = E and M = M. (vi) 9 (vii) Suppose not, then there exists a sequence of n x n matrices {M i} and a sequence of vectors {qi} in R", i = 1,2 ..... converging to M and q respectively such that Mix + qi >-0, x >-0, has no solution for i = 1,2 .... By

158

O.L Mangasarian/Bounded solutions of linear complementarityproblems

Motzkin's theorem of the alternative this is equivalent to MiTt/ m~O, qiTu < 0 , u ->0, having a solution u i for i = 1,2 ..... Since ui~ 0 it follows that MiT U i

^

" Wi

Ui

Hence there exists an accumulation point ~ such that MTI~0,

qT/~<0,

O# U ~0,

which contradicts (vii). (vii) ~ (viii) Suppose not, then there exists an (a, a) in R "+1 such that the set {U I MTu'<-- a, qTu <--Or, U >--0}

is nonempty and unbounded. Hence there exists a sequence of {u~}, i = 1,2 .... , such that ui~ 0, []u~[[-~ and

MTu'llluql <--allluql,

qTu'llluql

lllu'll,

<-

u'/llu'll >- O,

i = 1, 2 . . . . .

Consequently there exists an accumulation a such that MTa--<0,

qTti--<0,

0#~-->0,

which contradicts (vii). ( v i i ) r (viii) We shall prove the contrapositive implication. Let MTa <--0, qTti <--0, 0 --< a r 0, then for A > 0, Aa is unbounded as A ~ oo and MT(A•) --- 0 = a,

qT(Mi) -----0 = a,

Aa -- 0.

(iv) ~ (xiii) Obvious. (iv) ~ (xiii) Because M is copositive plus and the feasible region S ( M , q + 3'h) is nonempty it follows by Lemke's algorithm [3, 8] that LCP(M, q + ~/h) has a solution. (v) ~ (xiv) Obvious. (v) # (xiv) Because M is copositive plus and the feasible region S ( M , ft) is nonempty it follows by Lemke's algorithm that LCP(M, c]) has a solution. (ix) c> (xv) This follows from Theorem 1. (vii)C> (x) By recalling that M is copositive plus the contrapositive implications follow from the following

MTu ~ 0 MTu <--0 ] 0 --< i~TMTu < 0 qTu -<0 I C> qTu -<0

o u__o]

O#u>O

r

I (M+MT)u=O qTu ~ 0 ~Mu = - M r u ~ 0 t

0~ u >0

(ix) r (x) We shall prove the contrapositive implication. Suppose that S(M, q) is empty or unbounded. If it is empty, then because M is copositive plus it follows by Lemke's algorithm that the feasible region S(M, q) is empty and consequently the system

O.L. Mangasarian/Bounded solutions of linear complementarity problems Mx+q~>-O,

159

x-0,~>0,

has no solution (x, ~) in R "§ By Motzkin's theorem of the alternative it follows that there exists a u satisfying

MTu <--O, u >--O, qTu
q/llx'll- 0,

x'/llx'll >- O,

(x'T/IIx'II)(Mx'/IIx'II + q/llx'll) = O,

i -- 1, 2 . . . . .

Hence there exists an accumulation point S such that MS -> 0, 0 r ~ --- 0, STMS = 0. Since qTx'/IIx'I=--x'TMx'/IIx'II<--O, it follows that qTs--<0. Because M is copositive plus (M + MT)~ = 0 follows from sTM~ = 0 and ~ -> 0. The existence of this ~ satisfying the conditions MS --- 0, qT~ _____0, (M + MT)~ = 0, 0--< ~ 0, contradicts (x). (ix) ~ (x) We shall prove the contrapositive implication. Let x satisfy

Mx>_O,

qTx<--O,

(M+MT)x=O,

O<_x~O,

and let ~ be a solution of LCP(M, q). Then for any )t > 0,

M(~+hx)+q>_O,

~+hx->0,

and

0 <--(2 + Ax)T(M(~ + hx) + q) = ~T(M~ + q) + AsT(M + MT)x + h2xTMx + AqTx = AqTx <--O. Hence ~ + hx solves LCP(M, q) and is unbounded as h ~oo. Hence S(M, q) is unbounded whenever it is nonempty. (x)co(xii) B y Tucker's theorem of the alternative [9] condition (x) is equivalent to the system

MTv-q~+(M+MT)z
v>_O,~>_O,

having a solution (v, z, r in R 2n+1, and because the set {(v, z, ~)[ M T v - qr (M + MT)z < 0} is open this is equivalent to

Mz+MT(v+z)
v>-O,

having a solution (v, z) in R 2". Defining x = z and u = v + z gives (xii). (x) f f (xi) We shall establish (xi) for each solution of LCP(M, q) by proving the contrapositive implication. So let ~ be some solution of LCP(M, q). Then the set defined in (xi) is nonempty because M is copositive plus, qT~ = _ ~TM~ __<0 and hence ~ is in the set. Let this set be unbounded. Then there exists a

160

O.L. Mangasarian[ Bounded solutions of linear complementarity problems

sequence of points in R", {x'}, i = 1, 2 . . . . . such that x i r 0,

Ilxill-~oo and

Mxilllx'll + qlllxiU >- O, xilllxi u >- O, qTx'IIIx'II <--O, (M + MT)(Xi/IlxilI- ~/llx'll) = 0, i = 1, 2 ..... Hence there exists an accumulation point ~ satisfying M~->0,

0#~---0,

qT~__<0, ( M + M T ) g = 0 .

This is a negation of (x). (x) 9 (xi) We shall establish (x), with (xi) holding for some solution of LCP(M, q), by proving the contrapositive implication. So let g be any solution of LCP(M, q). Let x satisfy

Mx>_O,

qTx<--O,

(M+MT)x=O,

O<_x:~O.

Then for any h > 0, s + hx is unbounded as h ~ 0% M(~ + )~x) + q --- 0, ~ + hx -> 0, qT(~ + ~X) ------~TM~ --<0 and (M + MT)(~ + hx - ~) = 0. Hence the set defined in (xi) is unbounded. This is a negation of (xi) with ~ taken as some solution of LCP(M, q). (xiv) 9 (xvi) Set 8 = e a n d / f / = M. (x) f f (xvi) Since we have already established the equivalence of (x) and (xv) we have from (xv) that for some e > 0, S(/f/, ~]) is nonempty for max{liM - MII, II0- qU} <-~. S u p p o s e now that (xvi) does not hold. Then there exist sequences {llx'll}-, ~, {M', qi}--,(M, q) such that xi:~ 0 and M' are copositive plus and

Mixi+qi>_O,

xi>_O, xi'r(Mixi+qi)=O.

Consequently, i

qi

x iT M i x i R

Xi

N---0, 1

q

iT X i

"

MiX i

iT Xi

q

=~

Since {llxill}~, it follows that there exists an accumulation point z of the bounded, sequence {xl/llxill} such that

Mz>-O, O:~z>-O,

qTz<-O, zTMz=O,

which contradicts (x). We note that the implication (vii) ~ (vi) also follows from [19, Part I] and the equivalence (vii)r (viii) follows from [22, Chapter 8]. We have included simple proofs of these relations here to make this paper more self-contained and accessible. The following corollary gives among other things a simple sufficient condition for the boundedness of the solution set of the linear complementarity problem

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

161

(1) when M is copositive plus. The first part of this corollary which is established from elementary arguments of perturbation theory of linear equations can also be established by using much more general perturbation results for complementarity problems [21].

Corollary 1. Let the linear complementarity problem (1) have a nondegenerate vertex solution ~ that is ~ + MS + q > 0 and ~ is a vertex of the feasible region S ( M , q), then assertion (xv) of Theorem 2 holds. If in addition M is copositive plus, assertions (i) to (xvi) of Theorem 2 hold as well.

Proof. Since ~ is a nondegenerate vertex solution of (1) there exists a partition {I, J} of {1..... n} such that Mjj is nonsingular and Mjj~j + qj = 0, MIj~j + qt > O, ~:

> 0.

By the Banach perturbation lemma [16, p. 45] for any n • n matrix ~/ satisfying Ilgt:~ - M~II < IIM;;I1-1, M,~ is nonsingular and

IIM Y;II-<

IIM SII 1 - IIMy'll II~t~: - M~AI'

11~;) - M Y;II <-

IIM Y;ll=ll~t. - M,: II 1 - IIMY'II I1~ - M~AI"

Hence for a sufficiently small perturbation of M and q, say 114- qll} -< ~ for some ~ > 0, we have i - = ~j - MY)(~Ij - q j ) - ( M y ] ~: = - M -jjqj

max{ll-~-MII,

MY))?t: >O

and =(MIj~j + ql) + MIj(~j - ~j) + ( M . - M , : ) ( - M y ) C ~ ) + ~ , - q,) > O.

Hence (~j, 0I) is a nondegenerate vertex solution of LCP(_~/, t~) when max{H/f/Mll, II~ - qll}-< ~, and consequently (xv) of Theorem 2 holds. By Theorem 2, the other assertions (i)-(xvi) hold also when M is copositive plus. The following example shows that the nondegenerate vertex condition is not necessary for the boundedness of the solution set.

Example 1.

I)

(:11)

162

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

Here M is positive and hence copositive plus. In fact M is also positive semidefinite. The solution set S(M, q) = {(xl, x2) ] xl + x2 = 1, xl -> 0, x2 >- 0} is bounded but does not contain a nondegenerate vertex. The following two examples show that when M is not copositive plus the existence of a nondegenerate vertex solution to the linear complementarity may or may not imply boundedness of the solution set. Example 2. ~ ' = (-11

~)'

q = ('0)"

Here M is not copositive plus and the unbounded solution set S(M, q)-{(1, 0)} O {(xt, x2) I xt = 0, x2 > 0} contains a nondegenerate vertex, (1, 0). Example 3.

Again M is not copositive plus, but the bounded solution set S(M, q ) = {(1, 0)} t2 {(Xl, x2) [ xl = 0, 0 -< x2 -< 1} contains a nondegenerate vertex, (1, 0). The following example shows that the copositivity plus assumption on AT/ in Theorem 2(xvi) cannot be dropped. Example 4. M=(_0

~),

0--(1%)

q--(-~),

Here M is skew-symmetric and hence is positive semidefinite but /~/ is not copositive plus. For this problem S(M, q) = {(1, 1)} and S(/f/, q) = {(1, 1), (0, 1 + 1/,)} which is not bounded f o r , E (0, ~) for any ~ > 0. However, if we take the copositive plus perturbation k-l+,

~

'

q=q=

'

0<'-<4'

which however is not positive semidefinite for positive ,, we have that - {( 1 6 + 4 , 1 6 - 2 0 , "~'~ S(M, q) = 16 - 16, + ,2, 16 - 16, + e2}J which is bounded by the Euclidean ball of radius 7 around the origin. The following corollary, which establishes the equivalence given by (5) in the introduction, follows by paraphrasing Theorem 2(vii) as a linear programming condition (6).

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

163

Corollary 2. Let M be copositive plus. The linear complementarity problem (1) has a nonempty bounded solution set if and only if

max{ eTu I MXu <- O, qXu <- O, u ~ 0} = 0.

(6)

tt

The following corollary characterizes the subclass of compositive plus matrices for which the linear complementarity problem has a solution for each q in R ~, that is those copositive plus matrices which are in the class Q (see [6]). The equivalence between (ii) and (iii) below has been given by Pang [18, Theorem 11] for a copositive plus M.

Corollary 3. Let M be an n x n matrix. Statements (i) and (ii) below are equivalent. Statements (i) to (iv) below are equivalent when M is copositive plus. (i) For each q in R n the system Mx+q>_O,

x>-O,

has a solution x in R ~. (ii) The system M x >O,

x >_O,

has a solution x in R ~. (iii) For each q in R ~ the solution set S(M, q) of the linear complementarity problem (1) is nonempty, that is M E Q. (iv) For each q in R ~ the solution set S(M, q) of the linear complementarity problem (1) is nonempty and bounded.

Proof. (i) f f (ii) Take q = -e. (i) ~ (ii)ff (iv) If MR > 0 and i - > 0, then obviously for each q in R" there exists a )t > 0 such that M()t~) + q > 0 and A~ - 0. By the equivalence of (ii) and (ix) of Theorem 2 it follows that the solution set S(M, q) is nonempty and bounded for each q in R ~. (iii) ~ (iv) Obvious. (i) ~ (iii) Obvious. The following result complements [10, Theorem 2.2] for symmetric copositive plus matrices. In a related result [18, Corollary 10] Pang has shown that when M is in Q, symmetric and copositive plus, then it is strictly copositive, that is xTMx > 0 for 0 ~ x -- 0.

Corollary 4. Let M be a symmetric copositive plus matrix. Assertions (i) to (xvi) of Theorem 2 hold i[ and only i[ M x + q > 0 has a solution x in R ~.

Proof. When M is symmetric Theorem 2(x) is equivalent to MTu=O,

qru<_O,

O<_u~O,

hasnosolutionuER

~.

164

O.L. Mangasarianl Bounded solutions of linear complementarity problems

By Tucker's theorem of the alternative [9] this is equivalent to

Mx + q~ > 0,

has a solution (x, ~) E R n+l.

~ -->0,

Because the set {(x, ~) I Mx + q~ > 0, (x, ~) ~ R"+I} is open, the last condition is equivalent to the set {(x, ~) [ Mx + q~ > 0, ~ > 0, (x, ~) ~ R "+1} being nonempty, which in turn is equivalent to Mx + q > 0 having a solution x in R". Hence Theorem 2(x) holds if and only if Mx + q > 0 has a solution x in R". The corollary now follows from Theorem 2. Remark 1. The equivalence between (ix) and (x) for the case when M is a skew-symmetric and hence copositive plus matrix degenerates to Williams' characterization [23, Theorem 3] of bounded solutions to a pair of dual programs. For the pair of dual linear programs (a)

min{c~Y I Ay -> b, y ~ 0},

(b)

max{bTz [ ATz <--c, z >--0},

y

(7) Z

Williams' boundedness characterization for both the primal and dual solutions sets is that (a)

Ay ~ 0,

cTy --<0,

0 # y -->0,

has no solution y

(8)

and (b)

AXz<-O,

bTz>--O, O~z>--O,

has no solutionz.

We establish now that the negation of (8) is equivalent to the negation of Theorem 2(x) with M=

(o

0

'

q=

(c) -b'

x=

(,)

.

The negation of (8) is (a)

Ay-0,

cry-<0,

0~y->0,

hasasolutiony (9)

or

(b)

AXz<-O,

bXz>-O,

0~z->0,

hasasolutionz.

We show now that (9) is equivalent to

Ay>-O,

AXz<-O,

-cXy+bXz>-O,

has a solution (y, z)

O~(y,z)->O, (10)

which is the negation of Theorem 2(x) for the linear programming case. If (9a) holds take z = 0 in (10), if (9b) holds take y = 0 in (10), and hence (9) implies (10). Suppose now (10) holds. If either y or z is zero, then (9) holds. If both y and z are nonzero,

O.L. Mangasarian/ Bounded solutions of linear complementarity problems

165

then again (9a) holds when cTy--<0, and (9b) holds when cTy > 0 because bTz >-- cVy > 0. H e n c e (9) and (10) are equivalent and T h e o r e m 2(x) degenerates to Williams' condition (8) when M and q are specialized to the linear programming case. It is worthwhile to point out an interesting subtlety in connection with conditions (8). Taken together, conditions (8a) and (8b) are equivalent to T h e o r e m 2(x) and hence guarantee the existence and b o u n d e d n e s s of the solution sets to b o t h of the dual linear p r o g r a m s of (7). H o w e v e r , taking (8a) or (8b) one at a time merely guarantees the b o u n d e d n e s s but not the existence of a solution set to the corresponding linear program. For example, for A = 0, c = - 1 , b = - 1 , condition (8a) is satisfied b y y = 1, but the linear program (7a) has no solution because its objective is u n b o u n d e d below. R e m a r k 2. F r o m the proof that (ix) f f (x) in T h e o r e m 2 we conclude that if the linear c o m p l e m e n t a r i t y problem (1) with a copositive plus M has an unbounded solution set, then each of its solutions ~ lies on a ray of solutions ~ + Ax where A > 0 and x is any nonzero element of the c o n v e x cone {x ] Nix >- O, qTx <--O, ( M + M T ) x = O, X >--0}.

Remark 3. It can be shown that Cottle's theorem 3.1 (from [2]) relating a solution ray of LCP(M, q) and the c o m p l e m e n t a r y cones of M follows f r o m the equivalence of (x) and (xiii) of T h e o r e m 2.

Acknowledgment I am indebted to Dr. W a y n e HaUman for pointing out the counter-example of [6] which shows, a m o n g other things, that LCP(~7/, q) m a y not be solvable for all (/~7/, q) sufficiently close to (M, q) even if LCP(M, q) is solvable for all ~ in R".

References [1] R.W. Cottle, "Nonlinear programs with positively bounded Jacobians", SIAM Journal on Applied Mathematics 14 (1966) 147-158. [2] R.W. Cottle, "Solution rays for a class of complementary problems", Mathematical Programming Study 1 (1974) 59-70. [3] R.W. Cottle and G.B. Dantzig, "Complementary pivot theory of mathematical programming", Linear Algebra and its Applications 1 (1968) 103-125. [4] R. Doverspike, "Some perturbation results for the linear complementarity problem", Mathematical Programming 23 (1982) 181-192. [5] B.C. Eaves, "The linear complementarity problem", Management Science 17 (1971) 612-634. [6] L.M. Kelly and L.T. Watson, "Erratum: some pertubation theorems for Q-matrices", SIAM Journal on Applied Mathematics 34 (1978) 320-321. [7] C.E. Lemke, "Bimatrix equilibrium points and mathematical programming", Management Science I1 (1965) 681-689.

166

O.L. Mangasarian f Bounded solutions o{ linear complementarity problems

[8] C.E. Lemke, "On complementary pivot theory", in: G.B. Dantzig and A.F. Veinott Jr., eds., Mathematics of the decision sciences, Part 1 (American Mathematical Society, Providence, RI, 1968) pp. 95-114. [9] O.L. Mangasarian, Nonlinear programming (McGraw-Hill, New York, 1969). [10] O.L. Mangasarian, "Solution of symmetric linear complementarity problems by iterative methods", Journal of Optimization Theory and Applications 22 (1977) 465-485. [11] O.L. Mangasarian, "Characterizations of linear complementarity problems as linear programs", Mathematical Programming Study 7 (1978) 74-87. [12] O.L. Mangasarian, "Simplified characterizations of linear complementarity problems solvable as linear programs", Mathematics of Operations Research 4 (1979) 268-273. [13] O.L. Mangasarian, "Locally unique solutions of quadratic programs, linear and nonlinear complementarity problems", Mathematical Programming 19 (1980) 200-212. [14] N. Megiddo and M. Kojima, "On the existence and uniqueness of solutions in nonlinear complementarity theory", Mathematical Programming 12 (1977) 110-130. [15] K.G. Murty, "On the number of solutions to the complementarity problem and spanning properties of complementary cones", Linear Algebra and its Applications 5 (1972) 65-108. [16] J.M. Ortega and W.C. Rheinboldt, Iterative solution of nonlinear equations in several variables (Academic Press, New York, 1970). [17] J.-S. Pang, "A note on an open problem in linear complementarity", Mathematical Programming 13 (1977) 360-363. [18] J.-S. Pang, "On Q-matrices", Mathematical Programming 17 (1979) 243-247. [19] S.M. Robinson, "Stability theory for systems of inequalities, Part I: Linear systems, Part II: Differentiable nonlinear systems", SIAM Journal on Numerical Analysis 12 (1975) 754-769 and 13 (1976) 497-513. [20] S.M. Robinson, "Generalized equations and their solutions, Part I: Basic theory", Mathematical Programming Study 10 (1979) 128-141. [21] S.M. Robinson, "Strongly regular generalized equations", Mathematics of Operations Research 5 (1980) 43-62. [22] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, NJ 1970). [23] A.C. Williams, "Marginal'values in linear programming", Journal of the Society for Industrial and Applied Mathematics 11 (1963) 82-94.

Mathematical Programming Study 19 (1982) 167-199 North-Holland Publishing Company

ON R E G U L A R I T Y CONDITIONS IN M A T H E M A T I C A L PROGRAMMING Jean-Paul PENOT D~.partement de Math~.matiques, Universit~ de Pau, 64000 Pau, France Received 3 October 1980 Revised manuscript received 15 March 1982

Constraint qualifications are revisited, once again. These conditions are shown to be reminiscent of transversality theory. They are used as a useful tool for computing tangent cones, by the means of generalized inverse function theorems. The finite dimensional case is given a special treatment as the results are nicer and simpler in this case. Some remarks on the nondifferentiable case are also presented.

Key words: Nonlinear Programming, Implicit-inverse Function Theorem, Polar Cones, Tangent Cones, Multipliers, Constraint Qualification, Farkas' Lemma.

O. Introduction

Nonlinear programming can be viewed as the problem of minimizing a functional f under implicit as well as explicit constraints. More precisely, we consider the following classical nonlinear programming problem (P)

minimize f(x), subject to x E A = B fq h-~(C).

Here B (resp. C) is a subset of a Banach space X (resp. Y) and f : X ~ R , h:X~Y (or only h : B ~ Y ) . Taking for C a closed convex cone, or if Y = Y~ x Y2, the product of {0} and a closed convex cone C2 in I"2, we obtain the usual formulation with inequality (and/or equality) constraints. We do not (always) make differentiability assumptions on f, but it is not the chief aim of this paper to relax differentiability assumptions (for such a purpose see [7, 8, 13, 17, 25, 29, 30, 46-48, 54, 55, 61, 63]). We mainly focus our attention on regularity (or qualification) conditions which enable one to write a necessary condition of optimality in terms of the explicit constraint B, eliminating the implicit constraint h-~(C), at the cost of perturbing the objective function f by y* o h, where y* E Y* is a Lagrange-Kuhn-Tucker multiplier. This problem has received much attention (see [1,2,4,6,9,15,24,33, 34,37,41,53,65,71] for a sample of a multitude of papers). Here we focus our attention on the existence of true (or normal) Lagrange multipliers (with the exception of Theorem 4.8). The case of Fritz John multipliers is treated in [34] along similar lines. The best results known to us are those of Guignard [24] (see also 167

168

Jean-Paul Penot[On regularity conditions

[9, 11, 22, 23, 44]) on one hand and those of Duong and Tuy [16] and Robinson [58] on the other hand. Here we try to relate these two lines of thought and present simple proofs of their fundamental results. Our point of view consists in reducing the whole question to the study of simple rules for calculating tangent cones. We point out an analogy with transversality theory; this connection with one of the main pillars of differential topology is not fortuitous. As our purpose here is to shed light on this parallel, we do not insist on the connection with another (fast growing) topic: the generalized differential calculus. This can be made through epigraphs and indicator functions, if one consents to leave differentiable or Lipschitzian functions ([29, 30, 46-48, 54, 61, 63]). The organization of this paper is as follows. We first recall general optimality conditions in terms of tangent and normal cones and introduce the idea of tangential approximation to a set or a function at some point. The second section is devoted to the calculus of polar cones and its applications to multipliers. In Section 3 we focus our attention on regularity or transversality conditions and their significance. We then specialize our study in Section 4 to the case of finite dimensional constraints, while the following section is devoted to the general case. We end this paper with some special cases in which differentiability or regularity conditions can be relaxed. Throughout the paper X and Y denote Banach spaces unless otherwise specified. For any subset A of X and r > 0 we denote by Ar the set of points of A whose norm is not greater than r. Given x ~ X, d(x, A): = inf{d(x, a) I a ~ A} and the Hausdorff distance d(A, B) between A and another subset B of X is d(A, B): = max(e(A, B), e(B, A)) with e(A, B): = sup(d(a, B) I a E A). The closure (resp. interior) of a subset A of X is denoted by cl A or A (resp. int A or _~); its closed c o n v e x hull is denoted by c--o(A).

1. Tangential approximations and optimality conditions We believe that classical tangent cones (and the corresponding subdifferentials) are still the best tool for what concerns necessary conditions. This is not only attested by the numerous papers using them (or derived variants) but proved by elementary examples. H o w e v e r the usual tangent cone concept suffers from the unpleasant drawback of lack of convexity. There are two ways of avoiding this difficulty. The first one consists in restricting one's attention to tangentially convex sets and tangentially c o n v e x functions; these classes include the convex and the smooth classes. The second one follows an idea of Guignard [24] (for subsets) and Pshenichnyi [55] (for functions) which consists in choosing convex cones (resp. convex functions) replacing the given data. This treatment is versatile enough to encompass at the same time the tangentially convex situation and the peritangent-peridifferential approach of Clarke [13] (see also

Jean-Paul Penot/ On regularity conditions

169

[7, 8, 25, 29, 30, 53, 61, 63]) in which convexity is automatically brought into the picture, even when the situation is naturally nonconvex. We recall the following classical definition which dates back to the thirties, with Bouligand, if not earlier. It is also useful in other topics as flow invariance for dynamical systems, or fixed point theory [18, 38, 49]. Definition 1.1. The tangent cone TaA to a subset A of X at a point a in the closure of A is the set of right derivatives of continuous curves c : [ 0 , 1]-~X emanating from a with sufficiently many values in A (more precisely, such that 0 is an accumulation point of c-~(A)) for which the right derivative exists. Equivalently v E TaA iff there exists a sequence (t., an) in (0, +oo) x A with limit (0, a) such that v = lim t-,l(a, - a) or iff 1 lim inf 7d(a + tv, A ) = O. t-~O+

We shall also use the radial tangent cone T~A to A at a, which is the set of v E X such that a + t,v E A for some sequence (t,) in (0, +oo) with limit 0. Obviously Tr~A C TaA, and if A is starlike at a E A, in particular if A is convex, then T a A - - c l ( T ~ A ) , hence T~A and T~aA have the same polar cone which is called the normal cone to A at a: N~A = (T~A) ~ = {x* E X [ (x*, x) <- 0 Vx E TaA}. We call A tangentially convex at a if T~A is convex; convex subsets as well as submanifolds with or without boundary are tangentially convex. In [14,52] regularity conditions are given which ensure that A is tangentially convex. Definition 1.2. T w o mappings g and h from a subset A of X into Y are said to be tangent at a E A if (g(x) - h(x))[Ix - a I converges to 0 as x converges to a in A-~ {a}. T h e y are said to be strictly tangent if I g ( x ) - h ( x ) - g ( y ) + h ( y ) l / I x - y] converges to 0 as x and y converge to a in A with x # y. A mapping h : A - ~ Y is said to be semi-differentiable at a if there exists a continuous positively homogeneous mapping g : X ~ Y which is tangent to x ~-* h(a + x) - h(a) at 0 on A - a. Then g]TaA is uniquely defined, as is easily seen. We call h differentiable at a if g can be chosen in the set L(X, Y) of linear continuous maps from X into Y. Similar definitions can be given for strict (semi)-differentiability. For an extended real valued function f : X ~ R" = R U {+oo} with a E dom f = [-t(R) the preceding concepts can be generalized with the notion of Dini derivative: Of(a, x) = lim inf t-'[f(a + t v ) - f ( a ) ] (t, v)-*(O+, x)

and dr(a, x) = -_d(-f)(a, x) (which involves a lira sup instead of a lim inf). These

Jean-Paul Penot/ On regularity conditions

170

elements of R = R U {-oo} U {+~} are always defined so that

Of(a) = {x* E X * IV x

x dr(a, x) >- (x*, x)}

has always a meaning, even if void. In fact we have the following elementary lemma [47].

Lemma 1.3. If f : X ~ R " attains its minimum at a E X, and if Of(a, x) >- 0 for each x E X, hence 0 E Of(a).

f(a)< +oo, then

We get in this way an elementary f r a m e w o r k which encompasses at the same time the classical differential calculus and the subdiferential calculus for convex functions. H o w e v e r , the computing rules (or an easy direct analysis) give only the following optimality condition which includes the classical normality condition (N)

- f ' ( a ) ~ N~A.

Lemma 1.4. [47, Proposition 4.1, Theorem 4.6]. If f:X-->R" attains a finite (local) minimum on A at a ~ A, then for each v E T~A dr(a, v)>-O. If T~A is convex, if dr(a,. ) is convex and finite at 0 there exists some y* E Of(a) with (iq)

- y * ~ NaA.

There is an analogous statement with d_f(a,-) and (IaA) ~ where IaA = X - ~ To(X "~ A) is the interior tangent cone (or the cone of 'interior displacements') but we can assert that there is a y * E O_f(a) A ( - N a A ) only when _0f(a) = af(a); this is the case when f is semi-diferentiable at a. Note that the assumption dr(a, 0 ) < + ~ (resp. 0f(a, 0 ) > - ~ ) is lacking in [47 T h e o r e m 4.6]. Some flexibility can be introduced by the way of the following notions of approximation. Note that they play a key role in sufficient conditions of optimality [40]. Many variants [39, 40, 43] of these notions exist in the literature, the closest one being [10]; however, the differentiability assumption in [10] is more restrictive.

Definition 1.5. A subset C of X is an approximation to a subset A of X at a point a E A if there exists a map 4> : C --> A tangent to Ix (the identity of X ) at a with ~b(a) = a. It is a strict approximation to A at a if ~b is strictly tangent to Ix at a. We call C a continuous approximation of A at a if ~b is continuous on a neighborhood of a. In fact it suffices to define ~b in a neighborhood of a in C. The following characterization can be useful. Note also that tk can be extended by Ix on X ~, C so that the extended map is still differentiable at a (but not necessarily con-

Jean-Paul Penot] On regularity conditions

171

tinuous elsewhere); of course a strict approximation is a continuous approximation. Lemma 1.6. C is an approximation to A at a iff lim

d(x, A)/d(x, a) = O.

x~a

x~C~{a}

Proof. As d(x, A ) <- d(x, 4~(x)), the condition is necessary. It is sufficient as ~b(x) can be arbitrarily chosen in A ( x ) = {y E A [ d(x, y) -< d(x, A ) + d(x, a)2}. The following result is an obvious consequence of the definitions. Lemma 1.7. If C is an approximation to A at a, then TaC C TaA. We are particularly interested in finding approximations to A at a of the form C = a + L, where L is a convex cone. It is shown in the companion paper [50] that any convex subset A of X has a + TaA as an approximation if X is finite dimensional [50, L e m m a 4.7]; a somewhat related result holds if A has a non void interior [50, L e m m a 4.8]. Several examples are also given in [50]; they include manifolds with or without boundary or corners (which have been called cyrtohedra in [62]). The idea of the following definition is due to Pshenichnyi [55]. Definition 1.8. A function g : X - - * R is an (upper) approximation to a function F : X -* R" at a E dom f if for each x E X g(a + x) >- f ( a ) + df(a, x) and g(a) =

f(a). Of course, f : x ~ f ( a ) + df(a, x - a) itself is an approximation to f at a. When f is locally Lipschitzian around a, one can take the function x f ( a ) + f~ x - a), where f~

v) = lim sup t-l[f(x + tv) - f(x)]. t~O+, X'-'}O

There are close connections between the two notions of approximation we introduced. L e t us note the following ones in which G+(f) denotes the epigraph of f. Proposition 1.9. (a) Suppose f is semi-differentiable at a (or more generally, that d_f(a, . ) = df(a, .)). If p : X ~ l q is a sublinear functional such that, for g ( x ) = p(x - a) + f(a), G+(g) is a convex approximation to G+(f) at (a, f(a)), then g is an upper approximation to f at a.

Jean-Paul Penot/ On regularity conditions

172

(b) Conversely, suppose X is finite dimensional and p is a l.s.c, sublinear functional such that g(x) = p(x - a) + f ( a ) is an (upper) approximation to f at a. Then G+(g) is an approximation to G+(f) at (a,f(a)). It is a continuous opproximation if f and p are continuous. Proof. W i t h o u t loss of generality, we s u p p o s e a = 0, f ( a ) = O. (a) L e t ~b=(~bl, q b 2 ) : X x R ~ X x R with ~b(0, 0) = (0, 0), D~(O,O)=Ix• defines the a p p r o x i m a t i o n of G+(f) b y G+(g) at (a, f ( a ) ) = (0, 0). F o r e a c h v E X and e a c h t > 0 small e n o u g h we h a v e cb(tv, p(tv)) E G+(f) or

62(tv, tp(v)) ->/(~bl(tv ; p(tv))). As h ( t ) : = f(61(tv, p(tv))) - 62(tv, tp(v)) is null for t = 0 we h a v e D+h(O) - 0 : as t ~ 6~(tv, p(tv)) is a c u r v e tangent to v at t = 0+, and d/dtlt=o+cb2(tv, tp(v)) = p ( v ) , we h a v e

0 >- D+h(O) >- Of(O, v) - p(v). Since dr(0, v) = 0f(0, v), p is an u p p e r a p p r o x i m a t i o n to f at 0. (b) W e define ~b : G + ( p ) ~ G+(f) b y 6(x, r) = (x, r + i f ( x ) - p(x)) +) w h e r e t + = max(t, 0) f o r t E R. T h e n ~b(0, 0) = (0, 0), 4~(G+(p)) C G+ff). L e t us show that ~b is H a d a m a r d - d i f f e r e n t i a b l e at (0, 0), h e n c e Fr6chet-differentiable if X is finite dimensional, with ~b'(0, 0) = Ix• This m e a n s that f o r e v e r y s e q u e n c e s (r,), (x~), (t,) with limits r, x, 0 in R, X and (0, + ~ ) r e s p e c t i v e l y , we m u s t h a v e lim t-.%(t.r., t.x.) = (r, x). This a m o u n t s to lira t~[.f(t, x.) - p(t. x.)] + = 0 or

lim sup(t-,~f(t,x,) - p(x,)) <- O. But

lim sup(t~lf(t.x.) - p(x.)) <- lira sup t~lf(t, x . ) - lim inf p ( x . )

<- dl(O, x) - p(x) <- 0 b y a s s u m p t i o n , as p is l.s.c, at x. T h e following result is not m u c h m o r e t h a n a rewriting of L e m m a 1.4. Proposition 1.10. If f : X ~ R "

attains a finite (local) minimum on A C X at a E A, if C is a convex approximation to A at a and if g is a convex

Jean-Paul Penot/ On regularity conditions

173

approximation to f at a, finite and continuous at some point v E C then 0 E Og(a) + N.C.

In particular, if C = a + P , with P a convex cone, there exists y* E Og(a) with - y * ~ p0. In fact any closed convex subcone P of T , A can be used, for instance the peritangent cone ~raA (or tangent cone in the sense of Clarke [13, 61]) of A at a. Similarly, if one takes for g the c o n v e x approximation x ~-> f ( a ) + f ~ x - a) one gets a g ( a ) = 8/(a), the periderivative of f at a (or generalized gradient of Clarke [13]). However, taking P as large as possible and g as close as possible to f yields better results. We give only two elementary examples.

Example 1.11. (The Christmas garland). L e t us consider the one dimensional unconstrained problem of minimizing the function f:R--> R given by f ( x ) = I x l - Isin x I.

Using the necessary condition 0 E ~f(a) in terms of the periderivative of f we find an infinite number of candidates (the set ,rZ) whereas the inclusion

0 ~ 0I(a) has only a = 0 as solution, and 0 is the global minimizer of [.

Example 1.12. L e t X = R 2, A = R • {0} O {0} x R, and let f be any continuously differentiable function on X. The necessary condition 0 E 8[(0) + (ffoA) ~ = f'(O) + X *

using the peritangent cone is satisfied by any f, whereas the necessary condition -f'(O) E (ToA) ~

is satisfied only if f'(O) = O. A similar result holds for A = {x E R2 [ max(x 1, X 2) ~---0}.

2. Generalized Farkas' lemma and multipliers In this section we consider the general nonlinear programming problem (P); h is supposed to be differentiable at a, a local solution to problem (P).

Definition 2.1. L e t g be an upper c o n v e x approximation to y at a, a local solution to problem (P). A continuous linear form y* on Y is said to be a multiplier for

174

Jean-Paul Penot] On regularity conditions

problem (P) at a (with respect to g) if y* E Nh(a)C and (Mg)

0 E Og(a) + y* o h'(a) + NaB.

Again Og(a) denotes the subdifferential of g in the sense of convex analysis. If f is differentiable at a, this can be written, with g(x) = f ( a ) + ['(a)(x - a): (M)

0 ~ f'(a) + y* o h'(a) + NaB.

In other words, if y* is a multiplier, a is a critical point with respect to B of the perturbed function f + y* o h: the implicit constraint has disappeared. We shall obtain the existence of multipliers under a general regularity condition with the help of a generalization of the Farkas' lemma. This is a very classical device; however we need to improve our knowledge of the computing rules for polar cones. We begin with the following well-known lemma. Lemma 2.2. Let M (resp. P) be a closed convex cone in X (resp. Y ) and let u ~ L(X, Y ) with transposed map u* E L ( Y * , X*). Then the polar cone Q0 of Q = M n u-l(P) is the weak*-closure of M ~ u*(P~ This result follows easily from the bipolar theorem applied to the cone L = {(x, u(x)) I x E M, u(x) E P}. Now we prove that under a general condition we have Q0 = M0+ u,(p0); this condition is much more general than the classical one u ( M ) A i n t P ~ 0, as it does not require that int P g 0. The proof we give below is an adaptation of the (incomplete) proof of [34, Theorem 2.1]; for a quite different proof see [50]. Theorem 2.3. Let X, Y, M, P, Q be as in the preceding lemma. Suppose u ( M ) - P = Y. Then

(M n u-I(P)) 0= M ~

u*(P~

Proof. Using the lemma, it suffices to show that M ~ u*(P ~ is weak*-closed. We invoke the Krein-~mulian theorem t o show this by proving that for any closed ball B* with center 0 and radius r in X* the set B* n (M~ u*(P~ is weak*-closed in B*. Let x* ~ B* be the weak*-limit o f a net (x*)i~z c B* with x~f=w~f,+u*(~),

w*EM ~

~Ep

~

Given any y ~ Y we can find x E M and z ~ P with y = z - u(x). Hence

(z~,, y)

= ( ~ , z - u ( x ) ) <- ( ~ , - u ( x ) ) = (-u*(~),

x ) -- ( ~

- x~,, x ) <- (-x~,, x ) <- rlxl.

Similarly, we get that ( z ~ , - y ) is bounded above. Hence (~)iel is scalarly (or

Jean-Paul Penot] On regularity conditions

175

simply) bounded in Y*, hence strongly bounded b y the Banach-Steinhaus theorem. By the Bourbaki-Alaoglu theorem it has a weak*-limit point z*. Obviously z* ~ pO and w~ = x*, - u * ( ~ ) has x* - u*(z*) as a weak*-limit point, as u* is weak*-continuous. As M ~ is weak*-closed, we get x* - u*(z*) E M ~ and x* E B* tq (M~ + u*(P~ as was to be shown. Let us observe that the same result holds if u ( M ) - P is a closed vector subspace of Y. In the following restatement, X and Y are preordered by the convex cones M and P. Corollary 2.4 (Farkas' lemma). If X , Y, M, P are as above with u ( M ) - P = Y, then the following two assertions on f ~ X * are equivalent: (a) x >- O, u(x) >- 0 imply f ( x ) <- 0 (b) there exists x* ~ M ~ and y* ~ pO with f = x* + y* o u. Now we can apply the rule we just obtained to the following result which supplements Guignard's necessary condition [20] by giving a concrete instance where her assumptions are satisfied.

Theorem 2.5. Let a be an optimal solution to (P) where h is differentiable with derivative u at a, and f is differentiable at a. Let Q = u-l('~(Th(~)C)) and let G be a closed convex cone in X such that Q n G c co(TaA),

u(G) - co(Th(a)C) = Y.

Then there exists y* E Nh(a)C such that (G)

0 ~ ['(a) + y* o h'(a) + G ~

Proof. Using T h e o r e m 2.3 and the fact that for any cone P, p0 = (~--~p)O, where co P denotes the closed convex hull of P, we get - f ' ( a ) ~ NaA = [c-o(TaA)]~C (Q f3 G) ~ = G ~ = O~

u*(-~(Th(a)C) ~

U*(Nh~ajC).

Note that instead of c--o(Th(a)C) we might have used any other closed c o n v e x cone H ; then y* would have been in H ~ When TaB is c o n v e x the preceding result can be specialized to G = TaB so that we obtain a true multiplier.

Theorem 2.6. Let a be an optimal solution of (P), where h and f are differentiable at a. Suppose B (resp. C) are tangentially convex at a (resp. h(a)) and the

176

Jean-Paul Penot/ On regularity conditions

following regularity conditions hold (R)

h ' ( a ) ( T ~ B ) - Th(o)C = Y,

(L)

TaB n h'(a)-~(Th(~)C) = T~A.

Then there exists a multiplier y* at a for (P). In fact, if TaB and Th(~)C are not c o n v e x , (R) and (L) can be replaced b y

(P~)

h'(a)(co(T~B)) - co(Th(a)C) = Y,

(Lc)

-~(T,~B) n h'(a)-~(~(Th(a~C)) C -~o(TaA).

In a similar way, we obtain:

Proposition 2.7. L e t a be an optimal solution to (P), where h is differentiable at a. Suppose f has an upper convex approximation g at a which is finite and continuous at some v E TaA. Suppose B and C are tangentially c o n v e x at a and h(a) respectively and that (R) and (L) hold. Then there exists a multiplier for problem (P) at a with respect to g.

3. Transversality and regularity conditions At this stage we m a y ask whether conditions (R) and (L) are independant, and, more precisely, whether the linearizing condition (L) is a c o n s e q u e n c e of the regularity condition (R). We shall show in the next section that this is the case when Y is finite dimensional, and in Section 5 that the slight variant (R r)

h'(a)(TraB)- T~h(~)C = y

of condition (R) implies (L) when Y is an arbitrary Banach space and B and C are arbitrary closed c o n v e x subsets. H o w e v e r this is not true for condition (R) itself. In fact the following example shows that this 'pathological' situation is widely distributed o v e r the class of infinite dimensional separable Banach spaces.

Example 3.1. L e t X = Y be an infinite dimensional separable B a n a c h space; let h = Ix and let C be a closed c o n v e x subset without interior which is not contained in some closed hyperplane. For instance if X has a base (en) such that the coordinates mappings Pn are continuous, we can take for C the closed c o n v e x cone generated b y the en's. T h e n C has a non support point a [32, p. 111]. Thus T~C = X : otherwise a non void open convex cone would be separable from T~C D C - a , a contradiction. But T~aC# X as the core (algebraic interior) of C coincides with the interior of C [32, p. 139] so that C - a is not absorbent. Thus there exists b E X ~ { a } , such that B n C = {a}, with B = [a, b]. Condition

177

Jean-Paul Penot/ On regularity conditions

(R) is satisfied but, for A = B A C

we do not have (L) as T a A = { O }

while

TaB n T~C = R+(b - a).

This example shows that the transversality condition TaB - T~C = X

which ensures the intersection p r o p e r t y T a ( B n C ) = TaB n TaC

when B and C are submanifolds of X ([21,31,35], for instance) is sufficient when B and C are just c o n v e x subsets, L e t us o b s e r v e that peritangent cone here would not i m p r o v e the situation as it coincides tangent cone in the c o n v e x case and does not even satisfy the J'aA C ff~B w h e n A C B. H e n c e we introduce the following definition. Definition 3.2. I f B (resp. Gfiteaux-differentiable at a transversal) to C at a (in X = Y and h = Ix, B and

no more using the with the inclusion

C) is a subset of X (resp. Y) and if h :B ~ Y is E B N h-l(C), h is said to be r-transversal (or radially symbols h & ~ C at a) if h ' ( a ) ( T r B ) - Trh(a)C = Y. If C are said to be r-transversal at a: B ~ r c at a itt

T ~ B - TraC = X .

When the radial tangent cones are replaced by the tangent cones, and h is Hadamard-differentiable we use the classical notation h ~B C of differentiable topology [3, 21, 31]. The following l e m m a shows that two c o n v e x subsets of X are r-transversal at any c o m m o n point as soon as they are r-transversal at some point of their intersection. L e m m a 3.3. G i v e n h. B , C as a b o v e , w i t h B a n d C c o n v e x , t h e n h ~ ' s C 0 E c o r e ( h ' ( a ) ( B - a ) + h ( a ) - C). In p a r t i c u l a r , i[ X = Y a n d h = Ix, B ,5 r C iff 0 ~ core(B - C). Proof. Suppose

0 E core(h'(a)(B - a) + h(a)-

C), i.e., that

iff

Y = (0, + ~ ) ( h ' ( a )

( B - a ) + h ( a ) - C). Then Y C h'(a)((O, +oo)(B - a)) - (0, + ~ ) ( C - h ( a ) )

and h ,Ix~ C. Conversely, suppose u ( T ~ B ) y U Y can be written y = u(r(b - a))-

T i C = Y , with c = h ( a ) ,

u = h ' ( a ) . T h e n any

s ( d - c)

with r, s in (0, +~), b E B, d E C. Then, for any t > max(r, s) y = t ( u ( r t - ~ ( b - a)) - s t - l ( d - c)) E t ( u ( B - a ) - ( C - c))

and Y = (0, + ~ ) ( h ' ( a ) ( B - a ) - C + h ( a ) ) .

178

Jean-Paul Penot[ On regularity conditions

Thus we get an interpretation of Robinson's regularity condition [58] in terms of radial transversality. In Section 5 we give a direct proof of the fact that this condition (R) implies the linearizing property (L)

TaA = TaB n h'(a)-~(Th~a)C)

when B and C are convex and h is strictly differentiable. On the other hand it is well known that (L) holds true when B and C are differentiable submanifolds and h &B C [3, 21, 31, 35]. A number of classical results of convex analysis can be cast into the framework of r-transversality. We give here only one such restatement; for other important instances connected with subdifferential calculus see [50]. Theorem 3.4. multifunction jection p : X x neighborhood

[16, 58, 64]. Let X and Y be Banach spaces and let F : X --, Y be a with closed convex graph G. If for some g = (a, b ) ~ G the proY~Y maps T~a,b)G onto Y, then F is open at (a,b): for any N of a, F ( N ) is a neighborhood of b.

The analogy with the classical open mapping theorem is made clear; this may suggest other results in this direction. Now we shall use this result to study the relationship of condition (R) with the boundedness of the set K of Lagrange-Kuhn-Tucker multipliers. This problem has already been considered by Gauvin [19], Gauvin and Tolle [20] in the finite dimensional case and by Zowe and Kurcyusz [71] in the infinite dimensional case. The novelty here lies in the fact that f is nondifferentiable and that the boundedness of K is obtained under the condition (R) instead of the stronger condition (Rr). Let us start with a simple result. Lemma 3.5. Given convex cones M in X, N = - P in Y, and u E L ( X , Y), the following two conditions are equivalent. (5) cl(u(M) + P) = Y; (b) u*-~(M ~ n pO = {0}.

Proof. It is easily verified that ( u ( M ) + p)O = u.-~(MO) n pO.

Then, the bipolar theorem gives the result. Theorem 3.6. Let F * be a bounded convex subset of X*, and let M (resp. P ) be a convex cone in X (resp. Y ) and u ~ L ( X , Y). Suppose

K = {y* E p0 [ 3 w* E F*: u*(y*) + w* ~ M ~ = p0 n u * - l ( M ~ F*) is non void. Then the conditions (~) and (b) of the lemma are equivalent to condition (c) and are implied by condition (d) below.

179

Jean-Paul Penot / On regularity conditions

I f M and P are closed condition (d) is a c o n s e q u e n c e o f condition (a) (a) u ( M ) + P = Y ; (c) the a s y m p t o t i c cone T ~ K = {y* ~ Y* I V Y* E K V t - 09* + ty* E K} o f K is

{0}; (d) K is bounded. (2) r

(b) <=> (c) ~

(d) ~

(a).

Proof. It is clear that (d) implies (c). To p r o v e the other implications let us first

observe that if B is a bounded c o n v e x set and if C is a closed c o n v e x cone, then, ifB+C~0 T=(B + C) = C.

The inclusion C C T~(B + C) is obvious. Conversely, if v ~ T~(B + C), then there exists a sequence (b,, c,) in B • C and a E B + C such that a + nv = bn + c..

H e n c e v = lim n-~(b. - a + c.) = lira n-~c. ~ cl C = C. L e t Y* be an arbitrary element of K. We know [60] that T ~ K = { y * E Y * ]V t > - O g * + t y * E K } .

Thus T=K = {y* E Y* ] V t ~ 0 y* + ty* ~ p0, u*(y*) + tu*(y*) E M ~ - F*} = T~P ~ n u*-~(T~(M ~ - F * ) ) = p0 N

u*-l(M~

and (b) and (c) are equivalent. When M and P are closed, the multifunction F : X - - - ~ Y given by F ( x ) = u ( x ) + P if x ~ M , F ( x ) = 0 if x E X - - - M has a closed c o n v e x graph G and 0 E c o r e ( F ( X ) ) if u ( M ) + P = Y. Using T h e o r e m 3.4 we can find r > 0 such that any y U Y~ can be written y = u(x) + z

withx~Mr=MnX, z~Pr=PnYr. L e t y* and y* be arbitrary elements of K. L e t w* and if* in F * be such that x* = u * ( y * ) + w* ~ M ~

:~*=u*(9*)+ff*EM

~

Then, for any y = u ( x ) + z in Y~, with x E Mr, z E Pr, we have ( y * - 9", y) = ( y * - 9", u(x)) + ( y * - 9", z ) = (x* - ~*, x ) -

( w * - ~ * , x ) + ( y * - 9", z )

_< (~*, - x ) + (~* - w*, x) + (9", - z ) -< r[~*[ + dr + r[9*[

180

Jean-Paul Penot/ On regularity conditions

where d is the diameter of F*. Then [y* - y*[ = SUpy~v,(y* - y * , y ) is bounded.

Corollary 3.7. Let g be a continuous convex upper approximation to f at a, a local solution to problem (P). Suppose B and C are tangentially convex at a and h(a) respectively and h is differentiable at a. Suppose the set K of multipliers with respect to g at a is non void. Then (a) if K is bounded, then h ' ( a ) ( T a B ) - Th(~)C is dense in Y ; (b) conversely, if h ' ( a ) ( T o B ) - Th(o)C is dense in Y, then T~K = {0}; (c) /f h ' ( a ) ( T a B ) - Th(~)C = Y, then K is bounded.

Proof. This follows from Theorem 3.6 and the fact that F * = Og(a) is bounded as g is convex and continuous. This corollary can be applied when f is tangentially convex and f is semiderivable at a in the following sense: Of(a," ) = dr(a,. ). As we have just seen, the regularity condition (R) is almost equivalent (and, when Y is finite dimensional, is actually equivalent) to the boundedness of the set K of multipliers, provided K is non void. Now, our purpose consists in showing that K is non void as soon as (R) is satisfied and Y is finite dimensional or, when (R ~) is satisfied. Before turning to this aim, let us relate conditions (R) and (R r) and relate them to a classical Slater type condition. Our assumptions are slightly weaker than Zowe's ones in [70, 71]; see also [34, 53]. The following lemma [70, Theorem 2.2] will be useful; we present a simple proof for the sake of completeness.

Lemma 3.8. I f N and P are two cones o[ a t.v.s, and if P is convex with a non void interior, then int(N + P ) = N + int P.

Proof. As N + int P is open and included in N + P, we have int(N + P ) D N + int P. Let x ~ int(N + P ) and let q E int P. There exists t > 0 small enough so that x - tq E int(N + P). Let (n, p) E N x P be such that x - tq = n + p. Then x = n + p + tq E N + P + i n t P C N + i n t P

as P + int P C int(P + P ) = int P since P is a convex cone.

Proposition 3.9. (a) (R r) implies (R) and is equivalent to it if B and C are convex and if Y is finite dimensional or if Z : - h ' ( a ) ( T r B ) - Trhca)C contains a finite codimensional subspace W of Y. (b) If h ' ( a ) ( T ~ B ) and Trh(a)C are convex and dense in h'(a)(TaB) and Th(a)C

Jean-Paul Penot/ On regularity conditions

181

respectively and if one o f the latter cones has a non void interior, then (R~ and (R) are equivalent. (c) I f B and C are closed convex subsets and if one o f them has a non void interior, (R r) and (R) are equivalent. (d) If C is a c o n v e x cone (R r) can be written h ' ( a ) ( T ~ B ) + R h ( a ) - C = Y. (e) I f B and C are c o n v e x (R r) is equivalent to R o b i n s o n ' s condition (Ro)

0 E c o r e ( h ' ( a ) ( B - a) + h ( a ) - C)

(f) I f C is a c o n v e x cone with non e m p t y interior, (R r) is equivalent to (S)

there exists v ~ T r B such that h ( a ) + h ' ( a ) v E int C.

Proof. (a) The implication (R r) ~ (R) is immediate as T~B C TaB and TrcC C TcC with c = h(a). When B and C are convex and (R) holds Z is dense in Y as T~C.=cI(T~C) and f ( c l S ) C c l f ( S ) for a continuous map f. If p : Y ~ Y [ W denotes the canonical projection, this argument shows that p ( Z ) is dense in Y / W hence coincides with Y [ W [60, Corollary 11.7.3]. Hence Z = p - ' ( p ( Z ) ) = Y and (W) holds. (b) Under the assumption of (b) and condition (R), the convex cone Z is dense in Y, hence coincides with Y as int Z = int cl Z = int Y = Y. (c) Suppose B and C are convex and (R) holds. If int C is non empty, assertion (b) can be applied. If i n t B is non empty, then i n t T ~ B = int(0, + ~ ) ( B - a) also is non empty. Let us apply Theorem 3.4 with F ( x ) = h ' ( a ) ( x ) - (C - c) for x E X and g = (b, h'(a)(b)), where b is some point of int B. As (0, + ~ ) ( F ( X ) -

h ' ( a ) ( b ) ) = h ' ( a ) ( X ) - TrcC = Y,

F is open at g: for some r > 0 we have F ( B ) D h ' ( a ) ( b ) + Yr.

Hence Z -- h'(a)((O, + ~ ) ( B - a)) - (0, + ~ ) ( C - c) D F ( B ) - h ' ( a ) ( a )

has a non void interior. It follows as in (b) that Z = Y. (d) This rewritting follows from T~cC = (0, + ~ ) ( C - c) = C - (0, + ~ ) c = C + [0, +~)c

(e) Follows from lemma 3.3.

- (0, +~)c

= C -

Rc.

182

Jean-Paul Penot/ On regularity conditions

(f) Using l e m m a 3.8 we see that int Z = h ' ( a ) ( T ~ B ) - int(C - Rc) = h ' ( a ) ( T r B ) + R c - int C. If (S) holds, 0 in interior to Z, hence Z = Y and (R r) is satisfied. Conversely, if (R r) holds, int Z = int Y = Y contains 0, so that there exist w E T r B and r U R with h ' ( a ) w + rc E int C. If r > l, we take v = r-~w, so that h ( a ) + h ' ( a ) v = r-~(h'(a)w + rc) E int C whereas forr-
4. Finite dimensional constraints The purpose of this section is to show that condition (R) implies the linearizing property (L) w h e n Y is finite dimensional and B and C are c o n v e x (or, more generally, continuously convex approximated). Our main tool consists in the following open mapping theorem. Although close to the classical implicit function theorem and its relatives, as given in [27], it does not a p p e a r in the required generality in the long string of results of [27]. Theorem 4.1. L e t A be a convex subset o f a n o r m e d vector s p a c e (n.v.s.)X, let Y be a finite dimensional n.v.s, and let f : A ~ Y be continuous on a neighborhood o f a E A and differentiable at a. I f f ' ( a ) ( T a A ) = Y, then there exist a neighborhood V o f b = f ( a ) , a m a p p i n g g : V ~ A semi-differentiable at b, with g ( b ) = a, f o g = Iv. In particular f is open at a. ProoL Without loss of generality, we suppose a = 0, b = 0. W e first o b s e r v e that [ ' ( a ) ( T ~ A ) is a dense c o n v e x cone in Y, hence is the whole space [51, Corollary 11.7.3]. We choose a basis (bl . . . . . b~) of Y and we identify Y with R ~ normed b y lYl = ET=, lY, I if y = ET=, y,b,. Replacing bi b y tb~ with t > 0, small enough if necessary, we can find ai E A

(i = 1. . . . . 2n) with f'(0)(a~) = b~,

f ' ( 0 ) ( a ~ + . ) = - b~,

i = 1 . . . . . n.

Then we define h : Y ~ X b y h

yibi =

y~ ai + i=l

yTai+, i=l

with t § = max(t, 0), t - = m a x ( - t , 0) for t E R. The unit ball Y~ of Y is m a p p e d b y h into A. M o r e o v e r , writing f ( x ) = f'(O)(x) + r(x)

Jean-Paul Penot/ On regularity conditions

183

with limlxl~0+.x~A r(x)/lx I = 0, we see that k = f o h is differentiable at 0 with derivative Iy. L e t p E (0, 1) be such that for e a c h y E Yp

Ik(y)- yl

lyl

and k is c o n t i n u o u s on Yp. L e t v U V = Yr

W e define k~ : Yp ~ Y b y

k~(y) = v + y - k(y). T h e n ko is c o n t i n u o u s and m a p s Yp into itself as Iko(Y)l --- Ivl + ly - k(y)l -< 89 + 89 <- p. T h e B r o u w e r fixed-point t h e o r e m e n s u r e s that k~ has a fixed point y~ in Yo; then y~ = v + Yo - k(yv) or v = k(yo). If we set g(v) = h(y,) for v E V, we get g(v) E A,

lyvl---21vl as I,-yol 89

l(g(v))=l(h(y~))=v,

L e t us s h o w t h a t g is semi-differentiable at b = 0, with s e m i - d e r i v a t i v e h, b y p r o v i n g that the m a p p i n g v ~ yo is differentiable at 0 with d e r i v a t i v e Iv. F o r that p u r p o s e , we a s s o c i a t e with e a c h ~ E (0, ~ a 3, E (0, p) such that I k ( y ) - Yl - < ' I y l T a k i n g y = Yv, with

Iv1-89

for

lyl- 3,.

we get

Iv - Y~I -< "ly~l-< 2,1vl so that v ~ y~ is differentiable at 0 with Iy as derivative. R e m a r k 4.2. If a b e l o n g s to the interior of A, we get the classical o p e n m a p p i n g t h e o r e m . In this case an o p e n n e s s e s t i m a t e can be given. Putting

c(f'(a)) = sup{inf{[xl: f'(a)x = y}: y ~ Y,} it is not difficult to s h o w that for e a c h c > c(f'(a)) and e a c h r > 0 small e n o u g h one has

f(a + X , ) ~ f ( a ) + Y,. In this case, or, m o r e generally, w h e n e v e r ['(a)((-TaA) fl TaA)) = Y, g can be c h o s e n to be differentiable at f(a). R e m a r k 4.3. (a) O n e can extend T h e o r e m 4.1 to a p a r a m e t r i z e d v e r s i o n with essentially the s a m e proof. (b) It suffices to s u p p o s e f is H a d a m a r d - d i f f e r e n t i a b l e at a instead of F r e c h e t differentiable at a. M o r e o v e r X could be a n y locally c o n v e x space. Corollary 4.4. Let X, Y be n.v.s, with Y ]inite dimensional and let B C X with a

convex continuous approximation a + M at a E B. If f : B --* Y is continuous and differentiable at a with f'(a)(ToM)= Y, then there exists a neighborhood V of

184

Jean-Paul Penot/ On regularity conditions

f ( a ) in Y and a mapping g: V o B g(f(a)) = a.

semi-differentiable at f ( a ) with f o g - ~ I v ,

Proof. Again we s u p p o s e a = 0, f ( a ) = 0. L e t $ : M ~ B be an a p p r o x i m a t i o n m a p p i n g with $ ' ( 0 ) = Ix. T h e n fl = f o~b is differentiable at 0, c o n t i n u o u s in a n e i g h b o r h o o d of 0 and satisfies fi(O)(ToM)= f'(O)(ToM) = Y. H e n c e there exists a n e i g h b o r h o o d V of 0 in Y and g ~ : V ~ M semi-differentiable at 0 with g l ( 0 ) = 0 , f ~ o g ~ = I v ; thus g = ~ b O g l satisfies g ( 0 ) = 0 , f o g = f o ~ b O g l = f ~ ~

Iv. T h e following result is r e m i n i s c e n t of the classical implicit f u n c t i o n t h e o r e m and will be useful f o r our purposes. T h e o r e m 4.5. Let A be a convex subset of a n.v.s. X, let Y be a finite dimensional n.v.s, and let f : A ~ Y be continuous on a neighborhood of a E A and differentiable at a. If f ' ( a ) ( T a A ) = Y, the subset S = [-l(f(a)) of A admits the convex approximation Z = (a + N ) n A at a, with N = K e r f'(a). In particular, if N n TaA = c l ( N n T~A), then

TaS = N n TaA. Proof. As in the p r o o f of T h e o r e m 4.1. we s u p p o s e a = 0 , b = 0 and we c o n s t r u c t a right i n v e r s e h : Y ~ X of f'(0) such that h ( Y O C A, h is positively h o m o g e n e o u s and t h e r e exists m --- 1 with h ( Y O C Xm. L e t us set, for (t, y, z) E (0, 1] • Y~ x Z~ k,(y, z) = k(t, y, z) = t - l f ( t h ( y ) + (1 - t)z). Then, for each t ~ (0, 1], kt is differentiable at (0, 0) and Dk,(0, 0) = (Iy, 0). In fact, if r ( x ) = f ( x ) - f ' ( 0 ) x and if a : R + ~ R + is a n o n d e c r e a s i n g f u n c t i o n with a ( 0 ) = lira,_.0 a(r) = 0 and

Ir(x l--< (Ixl)lxl for each x ~ X small enough, we h a v e [kt(y, z) - y[ = [t-~r(th(y) + (1 - t)z)[

<- t-la(tm[y[ + (1 - t)lzD(tm[y ] + (1 - t)[z D <- t-'ma(m[y[ + [zl)(ly[ + [z[). L e t ~ > 0 be such that 2 m a ( 2 m s ) <- 1 f o r s E [0, 8]. T h e n , f o r e a c h ( y , z ) E Y~ x Z~, c h o o s i n g t = tz := 8-11z1, we h a v e

<~+ ~_~_ 2m -< ~;

[t-lr(th(y) + (1 - t)z) I -< (2mt)-~(mtly[ + ]z]) - ~

Jean-Paul Penot/ On regularity conditions

185

hence, b y B r o u w e r ' s fixed point t h e o r e m , there exists Yz E Y~ with

yz = -t-~r(t~h(yz) + (1 - L)z),

t~ =

-'lzl,

or

y(8-1lzlh(yz) + (1 -

s-'lzDz) = o.

L e t us show that 4~ : Z~ ~ S given b y

~(z) = a - ' [ z l h ( y J + (1 - 8-'[zl)z = tzh(yz) + (1 - tz)z is differentiable at 0 with d e r i v a t i v e Iz. But [r

- z]-< t~mlyz [ + t~Iz[

and tz]yzl - [r(tzh(yJ + (1 - tJz)] < a(mtzly~] + [zl)(mt~[yz[ + Izl). As ly~[ -< 8, Iz[ -< 8, we h a v e a(mtz]yz[ + [z]) -< (2m) -1, h e n c e LIy~[-< 2a(mLiyz[ + ]zl)]z[ -< m-'lzl and

mt~ly~l <- 2m,~(2lz[)lz[ so that

Ir

zl ~ 2m~(21zl)lzl + ~-'lzl ~

and ~b has d e r i v a t i v e Iz at 0. It follows that T~Z C T~S. As T~Z = c l ( T r Z ) and T r z = N n T~A, if N n T~A = c l ( N n T r A ) we get

T~Z = N n T,A. As the inclusion T~S C N n T~A a l w a y s holds true, the following inclusions are equalities: N n T~A = T~Z c T~S c N n T~A. R e m a r k . W h e n T~A n ( - T ~ A ) is m a p p e d o n t o Y b y f ' ( a ) , h c a n be t a k e n to be linear. T h e n X c a n be split into X = N 9 P, with P - - h ( Y ) , and c o r r e s p o n d ingly, ~b can be written ~b = (~bN, 4~e). As 4~N is close to Iz, d~P is a kind of implicit function. In fact, w h e n A is a n e i g h b o r h o o d of a and X is finite dimensional, ~bN has a local right i n v e r s e ~bN and ~be o ON is an implicit function: f(z, ~bp o ~bN(z)) = 0 for z E Z close to a.

Corollary 4.6. Let X, Y be n.v.s, with Y finite dimensional. Let B C X with a convex continuous approximation a + M at a, let h : B ~ Y be continuous and differentiable at a and C c Y with a convex continuous approximation h(a) + N

186

Jean-Paul Penol/ On regularity conditions

at h ( a ) E C. Then L = a + M n h ' ( a ) - l ( N ) is a convex approximation to A = B n h-l(C) at a provided h ' ( a ) ( T o M ) - ToN = Y. In particular, if B and C are convex and Y finite dimensional, condition (R) implies condition (L).

Proof. Again we s u p p o s e a = 0, c = h(a) = 0. L e t D = B x C C X x Y w h i c h is c o n t i n u o u s l y a p p r o x i m a t e d at (a, c) b y M x N via ~b x t~, ~b : M ~ B, ~b : N --> C being the a p p r o x i m a t i o n m a p p i n g s of B and C respectively. L e t f:D---> Y be given b y

I(x, y)

=

h(4,(x))-

~(y),

so that f'(a, c)(T F := [-1(0) with K = {(x, y) E M • N I h'(a)(x) - y = 0}. L e t us write O(x, y) = (Ox(x, y), Or(x, y))

for

(x, y) ~ K,

X(x) = &(Ox(x, h'(a)x))

for

x E L.

Since for e a c h (w, z) ~ F = / - 1 ( 0 ) we h a v e ~b(w) E A (as ~b(w) ~ B, ~(z) E C and ~b(w)E h-l(O(z))) f o r e a c h x E L we h a v e (x, h ' ( a ) x ) E K and (Ox(x, h'(a)x), Oy(x, h'(a)x)) ~ F h e n c e X(x) E A. M o r e o v e r X ( 0 ) = ~b(0)= 0, and X is differentiable at 0 with d e r i v a t i v e Ix as ok'(O) = Ix, 0~(0, O) = (Ix, 0).

Corollary 4.7. Let X , Y be n.v.s, with Y finite dimensional. Let B C X (resp. C C Y ) , h : B ~ Y be differentiable at a E A = B n h-~(C). Suppose B (resp. C) is continuously approximated at a (resp. h(a)) by a + TaB(resp. h(a) + Th
Theorem 4.8. Let B be a convex subset of a n.v.s. X, let C be a convex cone of a finite dimensional n.v.s. Y, and let h : B --> Y be continuous and differentiable at a E A = B n h-l(C). I f f : A ~ R is differentiable at a and if f ( a ) = inf [ ( A ) , then there exists a non-null element (a, y*) of [0, +~) x C o with af'(a) + y* o h'(a) ~ NaB,

(y*, h(a)) = O.

Jean-Paul Penot/ On regularity conditions

187

This statement, although not the m o s t general, includes the relation a f ' ( a ) + y* o h'(a) = 0 when B = X, and yields the classical f o r m with equalities and inequalities constraints: for Y = R", C = ( ( - ~ , 0))" we can write (a, y) = (a, a~ . . . . . a~) with a - 0 , a l - > 0 . . . . . a , - - - 0 , t~ihi(a)= 0 for i = 1. . . . . n, a + a l + . . . + a , > - O . Proof. L e t M = TaB, N = Th(~)C, P = - N , u = h'(a). If u ( M ) - N = Y we can take a = 1 using T h e o r e m 2.6 and Corollary 4.6. If u ( M ) - N ~ Y, it is a proper c o n v e x cone and c l ( u ( M ) - N ) ~ Y b y [51]. Using L e m m a 3.5 we find y* E p0 ~ {0} with u*(y*) E M ~ Then it suffices to take a = 0, as y* o h'(a) = u*(y*) and M ~ = NaB.

5. The infinite dimensional case

This section is devoted to the p r o o f that the transversality condition (R r) implies the linearizing property (L) under convexity conditions on B and C. Our a p p r o a c h is similar to the a p p r o a c h of Duong and T u y [16] and Robinson [58]; h o w e v e r we proceed more directly as we are not specially concerned with the perturbation theory studied by these authors. In fact our proofs are closer to [16] than to [58] and we show how to complete the results of [16] up to our aim. Another a p p r o a c h based on N e w t o n ' s method is given in [50]; it bears close connections with Michel's results [41]. The need of a simple proof of the linearizing p r o p e r t y (L) has been felt b y several authors; a m o n g them Alt and Nieuwenhuis were kind enough to c o m m u n i c a t e to the author their own work while the first version of this p a p e r was being submitted. We shall need the following result which implies that a bounded (vectorvalued) c o n v e x function is Lipschitzian. It could be deduced f r o m [59, T h e o r e m 2], but we give a simple direct proof. T h e o r e m 5.1. Let X and Y be two normed vector spaces and let F : X - - ~ Y be a multifunction with convex graph. Let a E X, r > O, m > 0 be such that f o r each x ~ a + X, there exists y E F ( x ) with lyl <- m. Then [or each s E (0, r), f o r each xl, x2 in a + Xs, and each Y2E F(x2) one has

d(y=, f ( x , ) )

m +

r-s

lY2lx 1 - x21.

Proof. L e t x~,x2 in a + X s and y E E F ( x : ) ; we m a y suppose x~#x2. As the function t ~ Ix:+ t ( X l - x : ) - a l converges to +oo as t ~ + ~ and takes the value Ix1-al < r for t = 1, there exists some t o > 1 such that Ixo-al = r for x0 = x2+to(X~-X2). L e t y 0 ~ F ( x 0 ) with lY01-<m. As F has a c o n v e x graph, y~=

Jean-Paul Penot!On regularity conditions

188

Y2+ to'(Y0- Y2) belongs to F(xO. Hence

d(y2,F(x,)) < l y 2 - Y,I = t o ' l y 0 - .2. "' '= ~ iXo_ x 2 1 t.. . 0 _ y2l -

<

ly01 + lY21

m + lY21

- I x o - a l - l x 2 - a l Ix~-xd<- r - s

Ix~-x,I.

Corollary 5.2. A muitivalued mapping F : X --~ Y whose graph is convex is locally Lipschitzian on the interior of its domain if it is locally bounded there. Proof. Let a K int dom F ; take r > 0 and m > 0 such that a + 3(, C dom F and F(a + X , ) C Y,,. Then, for s E (0, r), for every x~, x2 in a + Xs and each y2E F(x2) we have by the preceding result d(y2, F(xO) <- 2m(r - s)-llx2 - Xll. Taking the lowest upper bound on Y2~ F(x2) and changing x, and x2 we get

d(F(xO, F(x2)) <- 2m(r - s)-l[x2 - x,[. The following result is due to Robinson [59] (see also [42]). For the sake of completeness we present a proof which parallels very simply the proof of the classical P i c a r d - B a n a c h fixed-point theorem. Moreover we shall use the estimate of Remark 5.4 which differs slightly from the one given in [59].

Lemma 5.3. Let B be the open ball with center Xo and radius p in a complete metric space M and let F : B ~ M be a k-Lipschitzian multifunction with closed (non empty) values. I f k E (0, 1) and if d(xo, F(xo)) < (1 - k)p, then F has a tixed point ~ E B: ~ E F(~). Proof. Let c E (k, 1) with d(x0, F(xo)) < (1 - c)p. We can find x, ~ F(xo) such that d(xo, x O < ( 1 - c ) p . Moreover, for every x , x ' in B, y E F ( x ) we can find y ' E F(x') with d(y, y') - cd(x, x'), as a distinction between the case x = x' and the case d(x, x') > 0 shows. Suppose we have defined a sequence (x.) in B for n - p with x. E F(x,_,) for 1 - n - p, d(xn, xn-O <- cd(x._,, x~-2) for 2-< n - p. Then we choose xp+~E F(xp) with d(xp§ xp) <- cd(xp, xp-O, so that d(xp§ xp) <- cPd(x,, Xo) and

d(xp+l, Xo) <~(1 + c +... + cP)d(xl, Xo) < p and xp+, ~ B. The C a u c h y sequence (Xn) has a limit ~ and

d ( ~ , x o ) < - ( 1 - c ) - I d ( x l , xo)
gEB.

Jean-Paul Penot/ On regularity conditions

189

Finally d(~, F ( g ) ) -< lim[d(~, x,) + d(x,, F(x,_l)) + d ( F ( x , _ 0 , F(~))] = 0; n

hence ~ U F(~). R e m a r k 5.4. F o r any 8 > 0 we can find x E B with ~ ~ F ( ~ ) and 1+8 d(xo, ~,) <- -1-U--~d(xo, F(xo)) as can be seen b y replacing p b y p'

min(o, ~1 -+k 8 d(xo, F(xo))

if d(xo, F(xo)) > 0 or taking ~ = x0 if d(xo, F(xo)) = O. Theorem 5.5. Let B and C be closed convex subsets of X and Y respectively, let g :B ~ Y and a E B with g(a) E C be such that: (a) g ( B ) - C is absorbing in Y (i.e., Y = (0, +~)(g(B) - C)); (b) H = {(x, g(x) - y) I (x, y) E B • C} is closed convex. Then there exists q/ > 0 such that for each neighborhood Bo of a in B and for each continuous h : B o ~ Y verifying h(a) = g(a) and (c) d = h - g is ;t-Lipschitzian on Bo, with ;t < 3,-1, there exists neighborhoods U, V, W of a, O, h(a) in Bo, Y and C respectively such that for each (u, v, w) ~ U x V x W and each 8 > 0 the equation h(x) ~ v + C

has at least one solution x in Bo sati#ying Ix - u I - ~/(1 + 8)(1 - Xy)-tl v + w - h(u)l. Proof. A s s u m p t i o n (b) ensures that the multifunction G : B - - ~ Y given by G(x) = g(x)- C

has a closed c o n v e x graph H. If p : X x Y ~ Y denotes the canonical projection, assumption (a) can be written Y = (0, +oo)G(B) = p(T(~,o)H). We m a y suppose B is bounded since B m a y be replaced b y Bi = B f-1(a + X 0 as

Tt~,o)H = T~a,o)(H tq B1 x Y ) . Using T h e o r e m 3.4 and Corollary 5.2 we can find /3 > 0 and ~/> 0 such that Y~ C int G ( B ) and such that G -~ is ~/-Lipschitzian on Ya. L e t ;~ E (0, ~/-1), let a ~ (0, l) be such that B A (a + X~) C Bo and a)t
190

Jean-Paul Penot/ On regularity conditions

Let h : B o ~ Y satisfying (c) and h ( a ) = g ( a ) . We define U, V, W in such a way that for each (u, v, w) E U x V x W we have lu-al<-,~-p,

Ivl---/3- ~x,

I v l + l h ( u ) - w l < - ( 1 - X ~ / ) p ~ -' 9

L e t (u, v, w) E U • V x W be given and let B. = B n ( u +Xp). For each x E B , we have Ix - al -< a , hence x E B0 and Iv - d ( x ) l ~ Ivl + x l x - a l - < / 3 ,

hence P(x) = G-~(v - d(x))

defines on B. a multifunction with closed non empty values. Moreover, for every x and x' in B, we have d(P(x), P(x')) ~ rid(x)-

d ( x ' ) l - < ~Xlx - x'l.

As g(u) - w E G ( u ) - C we have u E G - l ( g ( u ) - w) hence d(u, P(u)) <- d(G-l(g(u) - w),

(~-1(I)

--

d(u)))

--- , / I g ( u ) - w - v + d ( u ) l -< r i b ( u ) - v - w I < (1 - )tv) p.

Using L e m m a 5.3. and Remark 5.4, for any ~ > 0 we can find x E B. such that Ix - ul -<(1 + 8)(1 - )tT)-ld(u, P ( u ) ) and x E P(x). This inclusion is equivalent to v'd(x)Eg(x)-C

or

h(x)Ev+C

and the result follows, using the estimate on d(u, P(u)). Now we give sufficient conditions for verifying assumptions (a), (b), (c). Assumption (c) is obviously satisfied if h is strictly tangent to g provided B0 is a small enough neighborhood of a. Assumption (b) is satisfied if g E L(X, Y), or, more generally, if Y is preordered by a closed convex cone P with C - P C C (for instance if C = - P is a closed c o n v e x cone) and g:B--* Y is c o n v e x and continuous. In fact, in this case n = G+(g) - {0} x C where G+(g) = {(x, y) I x E B, y 5 g(x) + P} is the epigraph of g. Finally, assumption (a) is satisfied when g is given by g(x) = s(x - a) + h(a)

Jean-Paul Penot/ On regularity conditions

191

where s : T~B ~ Y is sublinear with r e s p e c t to the ordering of Y and y = s(T~aB)- Trh~,)C.

In fact, this relation can be written Y = s((O, +~)(B - a)) - (0, +~)(C - h(a)) or

Y = (0, +oo)(s(B - a) + P ) - (0, +~)(C - h(a)) or

Y = (0, +~)(s(B - a) + P - C + h(a))

by L e m m a 3.3 and the convexity of s ( B - a) + P and C - h(a), so that Y = (0, +~)(g(B) - C).

We sum up this discussion in the following statement.

Corollary 5.6. Let B and C be convex subsets of X and Y respectively, Y being preordered by a closed convex cone P such that C - P C C. Let h : B ~ Y be a mapping which is strictly semi-differentiable at a E A = B n h - l ( c ) with a continuous sublinear mapping s : T~B -> Y as derivative at a. If s ( T ~ B ) - Trht,)C = Y, then there exist k > 0 and neighborhoods U, V, W of a, 0 and h(a) in B, Y and C respectively such that for each (u, v, w) E U x V x W there exists x ~ B with h(x) E v + C,

Ix - u]<-k[v + w - h(u)[.

Taking u = a and w = h(a) we get the following openness result.

Corollary 5.7. Under the assumptions of Corollary 5.6 the relation H : B - ~ Y given by H ( x ) = h ( x ) - C is open at (a, 0): .for each neighborhood N of a in B, H ( N ) is a neighborhood of 0 in Y. In fact the inequality of Corollary 5.6 with u = a, w - - h ( a ) shows that if N contains B n (a + X,), H ( N ) contains Yk-lp provided p is small enough. In our last statement, we use the peritangent cone (or tangent cone in the sense of Clarke) to a subset A of X a t ' a : it is the set ~raA of vectors v such that for each sequence (t,, a,) of (0, + ~ ) • A with limit 0 there exists a sequence (v,) with limit v such that a. + t.v. ~ A for each n [7, 13, 29, 30, 61, 63]. It is easily seen that ~raA C TaA and that ~raA is convex. H o w e v e r ~raA is not an isotone function of A and cannot be considered as a close approximation to A - a; in [52] we have s h o w n that ~r,A is the stable part of T , A (when X is finite dimensional or some technical assumptions hold).

Corollary 5.8. Under the assumptions of Corollary 5.6 there exist k > 0 and a

Jean-Paul Penot[ On regularity conditions

192

neighborhood U of a in B such that for each b E U we have, with A =

B n h-~(C), d(b, A) <-- k d ( h ( b ) ,

C),

Moreover T . A = 3-aA = TaB n s-l(Th(a)C). Proof. L e t k' > k and let r > 0 with (h(a) + Y2r) n C C W. We m a y suppose U is small enough, so that h ( U ) C h(a) + Y,. T h e n for each u E U there exists w ~ W with

d(h(u), w) <- k ' k - l d ( h ( u ) , W ) = k ' k - l d ( h ( u ) , C) as d ( h ( u ) , c ) > - r > - d ( h ( u ) , h ( a ) ) > - d ( h ( u ) , W ) for each u ~ U and each c C -~ W. Using Corollary 5.6 we can find x ~ B with h(x) E C and

Ix - ul <- klw -

h(u)l-< k ' d ( h ( u ) ,

C).

As k' ~ (k, +oo) is arbitrary, we get the result with k instead of k'. For any vector x ~ T~A we have x E TaB and

s(x) = lira t ~ ( h ( a + t.x.) - h(a)) ~ Th~a~C where (t., x.) is a sequence in (0, +oo) x X with limit (0, x) such that a + t.x. A C h-~(C) for e a c h n ~ N. As 3aA C T~A it suffices to show that any x E TaB O s-I(Th~a~C) belongs to 3"aA. L e t (t., a.) be a sequence in (0, +oo) • A with limit (0, a). It suffices to p r o v e that lim t ~ d ( a . + t.x, A ) = 0. As B and C are convex, we have TaB = 3"aB, T,~.~C = 3-h~a~C, so that there exists (x.) and (y.) with limits x and s(x) respectively in X and Y such that

a. + t.x. ~ B,

h(a,) + t.y. ~ C

for each n ~ N. For n large enough we h a v e a. + t.x. E U and

t-.~d(a. + tax, A ) <- kt-,~d(h( a. + t.x.), C) + d(x., x) <- kt-.~lh(a. + t.x.) - h(a.) - t.y. I + d(x., x). As h has s as a strict derivative at a, the right hand side m e m b e r of this inequality has limit 0, hence x E 3aA.

Corollary 5.9. Let B and C be subsets of X and Y respectively, let h : B ~ Y be strictly differentiable at a ~ A = B O h-~(C). If B (resp. C) has a closed convex set M (resp. N ) as a strict approximation at a (resp. h(a)) and if h ' ( a ) ( T r M ) T rh~a~N= Y, then TaM N h'(a)-l(Th{a~N) C TaA C TaB N h'(a)-l(Th(a)C). In particular, if B and C are closed convex subsets, condition (W) implies condition (L).

Jean-Paul Penot/ On regularity conditions

193

Proof. Without loss of generality we suppose a = O, h(a) = O. L e t k : M x N ~ Y be given b y k(x, y) = h ( ~ b ( x ) ) - t/,(y) where th : M ~ B, ~b: N ~ C define the approximations of B and C. Then k is strictly differentiable at (0, O) and k'(O, O)(T~(M • N ) ) = h'(O)(T~oM) - TroN = Y. Using Corollary 5.8 we get that the tangent cone to D = k-~(0) at 0 is given by ToD = To(M x N ) D k'(0)-l(0). Thus, if x E T o M n h'(O)-l(ToN) we h a v e (x, h'(O)x) E To(M • N ) = ToM x ToN and (x, h'(O)x) ~ k'(0)-l(0), so that there exists a sequence (t., x., y.) in (0, + ~ ) x D with lira t-.J(x., y.) = (x, h'(O)x). Then h(g~(x.)) = $ ( y . ) E C so that a. = tb(x.) E A and lim t-.~a. = $'(0)x = x; we have shown that x E ToA. The preceding p r o o f shows that it suffices to suppose that h and tb are differentiable, with $ ' ( 0 ) = Ir0M and that $ and h o ~b are strictly differentiable.

6. Some special cases It m a y happen that the c o m p u t a t i o n of tangent cones is m a d e easy by some peculiarities of the problem at hand. In this section we present two special cases which are outside the scope of our general treatment. The case of constraints in a functional case will be treated elsewhere. Our first result in this direction is an easy generalization of [2] (cf. also [1]): under a Slater type assumption which implies that Thr has a non void interior we get the linearizing property (L) without supposing h is strictly differentiable at a. In this statement we use the notion of convex core Q of a cone Q introduced to us b y Giner: Q = Q a Q, where for A, B contained in X A-~ B = { c E X I V b E B ,

b+cEA}.

Proposition 6.1. S u p p o s e B and C are subsets o f n o r m e d vector spaces X and Y respectively and h : B ~ Y is (Hadamard)-differentiable at a E A = B n h-l(C). If there exists ~ E ~aB n h'(a)-~(LC * TcC) with c = h(a), and IcC = Y -- T c ( Y ~ C), then (L)

T . A = TaB n h'(a)-l(TcC).

Proof. If L denotes the right hand side of this relation, we h a v e only to show that any v E L belongs to TaA. L e t us first suppose that v ~ TaB n h'(a)-l(I~C). Then there exists a sequence (to, b,) with limit (0, a) in (0, + oo)x B such that v = lira t:l(b, - a).

Jean-Paul Penot[ On regularity conditions

194

As h'(a)v = lim t:l(h(b,) - h(a)) E I~C, for n large enough we have h(b.) E C, hence b, ~ A, so that v E T,A. Now if v is an arbitrary element of L, for any sequence (r.) in (0, +w) with limit 0, v, = v + r,*3 belongs to TaB and h'(a)-~(IcC), hence v, E T , A by what precedes. As TaA is closed, we get v = lim v. E T~A. E x a m p l e 6.2. If X = Y = R 2, el = (1, 0), e2 = (0, 1), h = Ix, B = (X1 + el) U (X1 - e 0 =

{(x, y}

R 21 (Ixl - 1)2 + y2_< 1}, c -- {(x, y) R 21 y -< Ixl}, then ToA = C as the assumption of Proposition 6.1 is satisfied with ~3= - e2 and ~ooB = X , IoC*- ToC = {(x, y) ~ R2I y < - I x l } R e m a r k 6.3. The assumption T,"B O h'(a)-~(/~C ~' TcC) • ~ is satisfied if (a) TaB n h'(a)-l(int T ~ ) ~ ~;

(b) TaB is convex; (c) IcC = int TcC.

In fact, in this case ~aaB = TaB and int ~ C I~C* T~C as any 13E int T ~ is such that for some neighborhood V of 0 in Y one has .3 + v + w E T~C for every v ~ V, w E T~C, hence .3 + w ~ IcC and ,3 E I~C ~ T~C. Let us observe that (a) is satisfied if (c) holds and (a') TaB O h'(a)-~(IcC) # ~ and T~C is convex. It has been observed in [70, 34, L e m m a 4.1] that condition (a') is equivalent to condition (R) when C is a convex cone with a non empty interior. Our second result involves a convexity condition on h. Following [20, 9] we call a subset S of a normed vector space E (locally) pseudo-convex at a E S if S C a + co TaS (resp. S n V c + co(TaS) for some neighborhood V of a). Suppose X and Y are normed vector space, B is a subset of X, C is a closed convex cone in Y and h : B ~ Y is any mapping. We define h to be (locally) epi-pseudo-convex at a E B if its epigraph G+(h) = {(x, y) E X x Y [ y E h(x) + C} is (locally) pseudo-convex at (a, h(a)) (when (Y, C) = (R, R§ and h is differentiable at a we get a stronger notion than the usual pseudo-convexity condition). 6.4. Suppose h : B ~ Y is (Hadamard)-differentiable at a E A = B n h-~(C) where C is a closed convex cone in Y and B is an arbitrary subset of X. If h is locally epi-pseudo-convex at a, then, with c = h(a), Proposition

Tr~B n h'(a)-~(TrC) C T~A. Moreover, if TaB = TaB and T i C = TcC, then TaA = TaB n h'(a)-~(TcC).

Jean-Paul Penot[ On regularity conditions

195

ProoL It is easily seen that if Y is preordered by C one has T(a,c~a+(h) = a+(h'(a)). For any v E T~B N h'(a)-l(T~C) there exists ( t , ) C (0, + ~ ) with lim t, = 0 such that a + t , v ~ B , c+t,h'(a).vEC for each n ~ N . As ( a + t , v , h ( a + t , v ) ) G+(h) n V, where V is some neighborhood of (a, c) and n is large enough, and as V n G+(h) C (a, c) + G+(h'(a)) we get

h(a + t,v) ~ c + t,h'(a)v + C C C for n large, hence v E TrA. The condition TcC = T r c is satisfied if C is a polyhedral c o n v e x set or if C is the positive cone in Lp(S) for some p E [1, ~) and some m e a s u r e d space S. The case of constraints in some functional space will be studied elsewhere in more details. Finally we show that when A is p s e u d o c o n v e x at a the existence of a multiplier for a n y problem (P) is equivalent to the following condition (L*) dual

to (L): (L*)

NaA = NaB + h'(a)*(N,C)

with c = h(a). This result generalizes [34, T h e o r e m 3.2] in which B = X and h is affine. When B = X and h is c o n v e x and differentiable, condition (L*) coincides with Raffin's regularity condition [56]. Also note that b y T h e o r e m 2.3, (L*) is a consequence of (L) and (R).

Proposition 6.5. Let B and C be arbitrary subsets of X and Y respectively. Suppose h : B --* Y is (Hadamard)-differentiable at a ~ A = B n h-~(C) and A is pseudoconvex at a. Then condition (L*) is equivalent to the following property: for any f in the set F ( A , a) of functions on A which are (Hadamard)-differentiable at a and have a local minimum at a there exists a multiplier y* ~ Nh~a~Cfor f at a:

(M)

0 ~ f ' ( a ) + y* o h'(a) + NaB.

Proof. If (L*) holds and if f K F ( A , a ) we have -f'(a)~NaA= (NcC) ~ h'(a) + NaB, hence (M). Conversely, suppose (M) is satisfied for any f E F ( A , a). L e t x* E NaA. As A is p s e u d o c o n v e x at a, for any x E A we have x - a E co(TaA) hence (x*, x a) -< 0. Then - x * ~ F ( A , a) so that there exists y* E NcC, w* E NaB with 0 = - x * + y* o h'(a) + w*.

196

Jean-Paul Penot[On regularity conditions

H e n c e x* E NaB + h'(a)*(NcC). As the inclusion NaA ~ NaB + h'(a)*(NcC)

is always true, condition (L*) is satisfied. A related result is shown in [22] without the pseudoconvexity assumption on A but with a weaker conclusion than (M).

7. Conclusion

We have investigated the existence of multipliers in a nonlinear programming problem (P) formulated in a general setting. This can be treated as an immediate application of the condition (L*) which pertains to the calculus of normal cones (Proposition 6.5). In turn (L*) is shown to be a consequence of conditions (L) and (R) (Theorem 2.3). Condition (L) is a linearizing property dealing with tangent cones; condition (R) is a transversality condition. Although (R) itself does not suffice to imply (L) (Example 3.1), a slight variant of it, condition (Rr), does imply (L) under convexity conditions (Corollary 5,9). We thus get a new insight into the well-known constraint qualification condition of Robinson [58] which is another form of (R~). Moreover (R) implies (L), hence the existence of multipliers, in several cases of interest: when B and C are submanifolds of X and Y respectively [35], or when Y is finite dimensional and B and C are convex (Corollary 4.6), or when B and C are closed c o n v e x and one of them has a non void interior (Proposition 3.9(c)). H e n c e it can be concluded that the new regularity (or transversality) condition (R) directly or indirectly plays a key role in mathematical programming.

References

[1] J.M. Abadie, Probl&mesd'optimisation (Institut Blaise Pascal, Paris, 1965). [2l J.M. Abadie, "On the Kuhn-Tucker theorem", in: J. Abadie, ed., Nonlinear programming (North Holland, Amsterdam, 1967)pp. 19-36. [3] R. Abraham and J. Robbins, Transversal mappings and flows (Benjamin, New York, 1967). [4] C.C. Agunwamba, "Optimality condition: Constraint regularization", Mathematical Programming 13 (1977) 38-48. [5l W. Alt, "Stabilit~it mengenwertiger Abbildungen mit Anwendungen auf nichtlineare Optimierungsprobleme", Bayreuther Mathematische Schriflen 3 (1979) 1-107. [6] K.J. Arrow, L. Hurwicz and H. Uzawa, "Constraint qualifications in maximization problems", Naval Research Logistics Quarterly 8 (1961) 175-191. [7l J.-P. Aubin, "La th6orie des sous-gradients g6n6ralis6s', Annales des Sciences Math~.matiques du Qu~.bec 2 (1978) 197-252. [8] J.-P. Aubin and F.H. Clarke, "Multiplicateurs de Lagrange en optimisation non convexe et applications", Comptes Rendus de I'Acaddmie des Sciences A 285 (1977) 451-454.

Jean-Paul Penotl On regularity conditions

197

[9] P.J. Bender, "An application of Guignard's generalized Kuhn-Tucker conditions", Journal of Optimization Theory and Applications 25 (1978) 585-589. [10] V.G. Boltyanski, "The method of tents in the theory of extremal problems", Russian Mathematical Surveys 30 (1975) 1-54. [11] J. Borwein, "Weak tangent cones and optimization in a Banach space", SIAM Journal on Control and Optimization 16 (1978) 512-522. [12] M. Canon, C. Cullum and E. Polak, "Constrained minimization problems in finite dimensional spaces", SIAM Journal on Control and Optimization 4 (1966) 528-547. [13] F.H. Clarke, "Generalized gradients and applications", Transactions of the American Mathematical Society 205 (1975) 247-262. [14] B. Cornet, "A remark on tangent cones", Working paper University of Paris IX (July, 1979). [15] A.Y. Dubovitskii and A.A. Milyutin, "Extremum problems with constraints", Doklady Akademii Nauk SSSR 149 (1963) 759-762. [16] Pham Canh Duong and Hoang Tuy, "Stability, surjectivity, and local invertibility of non differentiable mappings", Acta Mathematica Vietnamica 3 (1978) 89-105. [17] I. Ekeland, "Nonconvex optimization problems", Bulletin of the American Mathematical Society 1(1979) 443-474. [18] S. Gautier and J.P. Penot, "Ferm6s invariants par un syst~me dynamique", Comptes Rendus de l'Acad~mie des Sciences A 276 (1973) 1457-1460. [19] J. Gauvin, "A necessary and sufficient regularity condition to have bounded multipliers in nonconvex programming", Mathematical Programming 12 (1977) 136-138. [20] J. Gauvin and J. W. Tolle, "Differential stability in nonlinear programming", SIAM Journal on Control and Optimization 15 (1977) 294-311. [21] M. Golubitsky, V. Guillemin, Stable mappings and their singularities (Springer, Berlin, 1973). [22] F.J. Gould and J. Tolle, "A necessary and sufficient qualification for constrained optimization", SlAM Journal on Applied Mathematics 20 (1971) 164-172. [23] F.J. Gould and J. Tolle, "Geometry of optimality conditions and constraint qualifications", Mathematical Programming 2 (1972) 1-18. [24] M. Guignard, "Generalized Kuhn-Tucker conditions for mathematical programming problems in a Banach space", SIAM Journal on Control 7 (1969) 232-241. [25] J. Gwinner, "Contribution ~ la programmation non diff6rentiable darts les espaces vectoriels topologiques", Comptes Rendus de l'Acad~mie des Sciences A 289 (1979) 523-526. [26] H. Halkin, "A satisfactory treatment of equality and operator constraints in the DubovitskiiMilyutin optimization formalism", Journal of Optimization Theory and Applications 6 (1970) 138-149. [27] H. Halkin, "Implicit functions and optimization problems without continuous differentiability of the data", SIAM Journal on Control and Optimization 12 (1974) 229-236. [28] M.R. Hestenes, Optimization theory: The finite dimensional case (Wiley, New York, 1975). [29] J.B. Hiriart-Urruty, "New concepts in nondifferentiable programming", Bulletin Soci~.t~ Math~matique de France M~moire 60 (1979) 57-85. [30] J.B. Hiriart-Urruty, "Tangent cones, generalized gradients and mathematical programming in Banach spaces", Mathematics of Operations Research 4 (1979) 79-97. [31] M.W. Hirsch, Differential topology (Springer, Berlin, 1976). [32] R. Holmes, Geometric functional analysis and its applications (Springer, Berlin, 1975). [33] H.W. Kuhn and A.W. Tucker, "Nonlinear programming", in: J. Neyman, ed., Proceedings of the second Berkeley symposium on mathematical statistics and probability (University of California Press, Berkeley 1951) pp. 481-492. [34] S. Kurcyusz, "On the existence and non existence of Lagrange multipliers in Banach spaces", Journal of Optimization Theory and Applications 20 (1976) 81-110. [35] S. Lang, Introduction to differentiable manifolds (Interscience, New York, 1962). [36] I. Lasiecka, "Generalization of the Dubovitskii-Milyutin optimality conditions", Journal of Optimization Theory and Applications 24 (1978) 421-436. [37] O.L. Mangasarian, S. Fromowitz, "The Fritz-John necessary optimality conditions in the presence of equality and inequality constraints", Journal of Mathematical Analysis and Applications 17 (1967) 37-47. [38] R.H. Martin, Jr., Nonlinear operators and differential equations in Banach spaces (WileyInterscience, New York, 1976).

198

Jean-Paul Penot/ On regularity conditions

[39] D.H. Martin, R.J. Gardner and G.G. Watkins, "Indicating cones and the intersection principle for tangential approximants in abstract multipliers rules", Journal of Optimization Theory and Applications 33 (1981) 515-537. [40] H. Maurer and J. Zowe, "First and second-order necessary and sufficient optimality conditions for infinite-dimensionalprogramming problems", Mathematical Programming 16 (1979) 98-110. [41] P. Michel, "Probl6me des in6galit6s. Applications h la programmation et au contr61e optimal", Bulletin Soci~.t~ Math~.matique de France 101 (1973) 413-439. [42] S.B. Nadler, Jr., "Multivalued contraction mappings" Pacific Journal of Mathematics 30 (1969) 475-488. [43] L.W. Neustadt, Optimization (Princeton University Press, Princeton, NJ, 1976). [44] J.W. Nieuwenhuis, "Another application of Guignard's generalized Kuhn-Tucker conditions", Journal of Optimization Theory and Applications 30 (1980) 117-125. [45] J.W. Nieuwenhuis, "About a local approximation theorem and an inverse function theorem", Journal of the Australian Mathematical Society 22 (B) (1980) 185-192. [46] J-P. Penot, "Sous-diff6rentiel de fonctions num6riques non convexes", Comptes Rendus de l'Acad~.mie des Sciences 278 A (1974) 1553-1555. [47] J.-P. Penot, "Calcul sous-diff6rentiel et optimisation', Journal Functional Analysis 27 (1978) 248-287. [48] J.-P. Penot, "The use of generalized subdifferential calculus in optimization theory", Third Symposium on Optimization Research, Mannheim (1978); Methods of Operations Research 31 (Verlags gruppe Athen/ium, Hain, Scriptor, Hanstein, Berlin, 1979) pp. 495-511. [49] J.-P. Penot, "Fixed point theorems without convexity", Bulletin Soci~t~ Math~matique de France M~.moire 60 (1979) 129-152. [50] J.-P. Penot, "On the existence of Lagrange multipliers in nonlinear programming in Banach spaces in Optimization and Optimal Control, Lecture Notes in Control and Information Sciences 30 (Springer, Berlin, 1981) pp. 89-104. [51] J.-P. Penot, "Inversion ~ droite d'applications non lin6aires. Applications", Comptes Rendus de l'Acad~mie des Sciences A 290 (1980) 997-1000. [52] J.-P. Penot, "A characterization of tangential regularity", Nonlinear Analysis, Theory, Methods and Applications 5 (1981) 625-653. [53] J.-C. Pomerol, "Contributions ~ la programmation math6matique: Existence de multiplicateurs de Lagrange et stabilit6", Th~se Universit6 Paris VI (1980). [54] B.N. Pshenichnyi, Necessary conditions for an extremum (Dekker, New York, 1971). [55] B.N. Pshenichnyi, "Necessary conditions of the extremum for non smooth functions", Kibernetika 6 (1977) 92-95 [English translation: Cybernetics 13 (1977) 886-891]. [56] C. Raflin, "Sur les programmes convexes d6finis dans des espaces vectoriels topologiques", Annales Institut Fourier (1970) 457-491. [57] S.M. Robinson, "An inverse function theorem for a class of multivalued functions", Proceedings of the American Mathematical Society 41 (1973) 211-218. [58] S.M. Robinson, "Stability theory for systems of inequalities, part II: Ditterentiable nonlinear systems", Journal Numerical Analysis 13 (1976) 497-513. [59] S.M. Robinson, "Regularity and stability for convex multivalued functions", Mathematics of Operations Research 1 (1976) 130-143. [60] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, NJ, 1970). [61] R.T. Roekafellar, "Directionally lipschitzian functions and subditterential calculus", Proceedings of the London Mathematical Society 39 (1979) 331-355. [62] J. E. Spingarn, "Generic conditions for optimality in constrained minimization problems", Ph.D. Thesis, University of Washington, Seattle, WA (1977). [63] L. Thibault, "Sur les fonctions compactement lipschitziennes et leurs applications: programmation math6matique, contr61e optimal, esp6rance conditionnelle", Th~se Universit6 de Montpellier (1980). [64] C. Ursescu, "Multifunctions with convex closed graph", Czechoslovak Mathematical Journal 25 (1975) 438--441. [65] P.P. Varaiya, "Nonlinear programming in Banach space", SIAM Journal on Applied Mathematics 15 (1967) 284-293. [66] A. Wierzbicki, "Maximum principle for semi-convex performance functionals", Siam Journal on Control and Optimization 10 (1972) A.A~A59.

Jean-Paul Penot[ On regularity conditions

199

[67] S. Zlobec, "Asymptotic Kuhn-Tucker conditions for mathematical programming problems in a Banach space", SIAM Journal on Control 8 (1970) 505-512. [68] S. Zlobec, "Extensions of asymptotic Kuhn-Tucker conditions in mathematical programming", S I A M Journal on Applied Mathematics 21 (1971) 448-460. [69] S. Zlobec, "A class of optimality conditions for mathematical programming problems in Banach spaces", Glasnik Mathematicki 7 (1972) 127-137. [70] J. Zowe, "A remark on a regularity assumption in mathematical programming", Journal of Optimization Theory and Applications 25 (1978) 375-382. [71] J. Zowe and S. Kurcyusz, "Regularity and stability for the mathematical programming problems in Banach spaces", Applied Mathematics and Optimization 5 (1979) 49-62.

Mathematical Programming Study 19 (1982) 200-221 North-Holland Publishing Company

GENERALIZED EQUATIONS AND THEIR SOLUTIONS, PART Ih APPLICATIONS TO NONLINEAR PROGRAMMING* S t e p h e n M. R O B I N S O N University of Wisconsin, Madison, WI 53706, U.S.A. Received 3 March 1980 Revised manuscript received 14 January 1981 We prove that if the second-order sufficient condition and constraint regularity hold at a local minimizer of a nonlinear programming problem, then for sufficiently smooth perturbations of the constraints and objective function the set of local stationary points is nonempty and continuous; further, if certain polyhedrality assumptions hold (as is usually the case in applications), then the local minimizers, the stationary points and the multipliers all obey a type of Lipschitz condition. Key words: Nonlinear Programming, Generalized Equations.

1. Introduction This p a p e r d e a l s w i t h t h e s t a b i l i t y o f s o l u t i o n s a n d m u l t i p l i e r s of n o n l i n e a r p r o g r a m m i n g p r o b l e m s w h e n t h e d a t a o f t h e p r o b l e m s are s u b j e c t e d to s m a l l p e r t u r b a t i o n s . T h e p r o b l e m w i t h w h i c h w e shall d e a l m a y b e f o r m u l a t e d b y i n t r o d u c i n g f u n c t i o n s f a n d g f r o m an o p e n c o n v e x set ~ C R" to R a n d R m r e s p e c t i v e l y , a c l o s e d c o n v e x set C C R n a n d a c l o s e d c o n v e x c o n e Q c R m. T h e p r o b l e m of i n t e r e s t is t h e n minimize s u b j e c t to

f(x), g ( x ) E Q~

(1.1)

XEC, w h e r e Q~ d e n o t e s t h e p o l a r cone o f Q : Q~ : = {y E R m [ (q, y> ~ 0 f o r e a c h q E Q}. If w e a s s u m e t h a t f a n d g are F r 6 c h e t d i f f e r e n t i a b l e a n d t h a t c e r t a i n r e g u l a r i t y c o n d i t i o n s are satisfied, it c a n b e s h o w n [17] t h a t with e a c h l o c a l m i n i m i z e r x o f (1.1) t h e r e a r e a s s o c i a t e d o n e o r m o r e multipliers u E R ~ s u c h t h a t (x, u) s a t i s f y * Sponsored by the United States Army under Contract No. DAAG29-75-C-0024 and by the National Science Foundation under Grant No. MCS-7901066. Parts of the research for this paper were carried out at the D6partement de Math6matiques de la D6cision, Universit6 Paris-IX Dauphine, with financial support from the Centre National de la Recherche Scientifique, and at the Departamento de Matem~ticas y Ciencias de la Computaci6n, Universidad Sim6n Bolivar, Caracas, Venezuela, with support from the United Nations Educational, Scientific and Cultural Organization under Proyecto UNESCO VEN-77-002. The author greatly appreciates the hospitality and support extended by the institutions cited. 200

S.M. Robinson/Generalized equations, Part H

201

the necessary optimality conditions

0 ~ ~x(X, u) + Oq,c(X),

0 E - ~,(x, u) + a~bo(u),

(1.2)

where ~(x, u ) : = f ( x ) + ( u , g ( x ) ) is the standard Lagrangian, and where the subscript x or u denotes partial differentiation with respect to that variable. The notation 0qJ denotes the normal cone at a point: thus,

{{y

ER"l(y,c-x)~<0foreachcEC}

Oq,c(x):=

0

ifxEC, if x ~ C;

for a convex cone like Q, the description of aS is even simpler:

oq,o(u)={{vEO~

if u ~ O , if u ~ Q .

Any point x which, with some u, satisfies (1.2) is called a stationary point of (1.1); there may well exist stationary points which are not local minimizers. What we shall show here is that if x0 is a local minimizer of (1.1) at which f and g are twice continuously Fr6chet differentiable and at which certain regularity conditions hold (specifically, the second-order sufficient condition and constraint regularity), and if [ and g are smoothly perturbed, then the set of stationary points near x0, regarded as a multifunction (multivalued function) of the perturbations, is (nonempty and) continuous at x0. Further, we show that the set of multipliers is upper semicontinuous, where these continuity properties are as defined by Berge [1]. The main results are proved in Section 3, after a review in Section 2 of the necessary regularity conditions. In Section 4, we show that if C and Q are assumed to be polyhedral, as will be the case in most applications, then even stronger results can be established: the distance from a stationary point of the perturbed problem to x0, or from an associated multiplier to the set of multipliers associated with x0, obeys a kind of Lipschitz condition. Examples are given, both in Section 3 and in Section 4, to show that certain stronger statements, although plausible, are not generally true. In several parts of this paper we use as a device for simplifying and motivating results the concept of a generalized equation. These objects, introudced in Part I of this paper [19], are relations of the form

0 E F(z) + T(z), where F : R k - ~ R k and T is a closed multifunction from R k to itself (often a normal-cone operator). For example, if we rewrite (1.2) in terms of f and g, and use the fact that a~bcxo(X, u) = aqlc(X) x OqJo(u), we obtain the generalized equation r.f,(x) + g,(x),u] + a~bc•

0 E L

-

g(x)

J

u),

(1.3)

S.M. Robinson[ Generalized equations, Part H

202

and this relation will be used in several ways in our analysis. Here and in what follows, the prime symbol will denote partial differentiation with respect to the variable x. Many papers have already been written about various stability questions connected with (1.1), often in the special case in which C = R" and Q = R k x R 1 (where R k is the non-negative orthant in Rk). This special case formulates the standard nonlinear programming problem minimize subject to

f(x),

gi(x)~
(1.4)

( i - - k + l ..... k+l).

We shall not attempt to review all of these papers here; rather, we mention only a few which illustrate the different types of assumptions that have been imposed. Fiacco and McCormick [3, Sections 2.4, 5.2] formulated a basic technique for analyzing (1.4) under the assumptions of strict complementary slackness, linear independence of the gradients of the binding constraints, and the second-order sufficient condition. This technique was refined, to deal with general perturbations, in [15] and in [4]; the basic tool, in all three cases, was the standard implicit-function theorem. If the assumption of strict complementary slackness is dropped, then the standard implicit-function theorem can no longer be used. However, if the linear independence assumption and a somewhat strengthened form of the secondorder sufficient condition are retained, then one can still show that the stationary point and associated multiplier are (locally) single-valued functions of the perturbations, and that they are locally Lipschitzian if appropriate continuity assumptions are made on the problem functions. This was shown for the problem (1.4) in [20], as a by-product of a general implicit-function theorem for generalized equations. A similar result, without the Lipschitz continuity, was established by Kojima [10, Theorem 6.4] as part of his investigation of 'strongly stable' solutions of (1.4). If one weakens the hypotheses still further by dropping the assumption of linear independence, then the appropriate condition to assume in its place is regularity of the constraints [18] (see Section 2; in the case of (1.4) this is the constraint qualification of Mangasarian and Fromovitz [12]). With this assumption, together with the strengthened form of the second-order sufficient condition previously mentioned, Kojima [10, Theorem 7.2] proved the existence of a locally unique stationary point which is continuous under small perturbations. Under similar hypotheses, Levitin [11, Theorem 4] stated that if the minimizer exists for small perturbations, then various properties, including Lipschitz continuity, followed. However, in Section 4 below we give an example which appears to satisfy Levitin's hypotheses but for which local Lipschitz continuity does not hold. Finally, Sargent [23] has studied the existence and continuity of

S.M. Robinson/Generalized equations, Part II

203

local minimizers under constraint regularity; however, his methods are quite different from those of the other papers mentioned here, and it appears that some of the results in [23] may not be completely correct. We begin our analysis, in the next section, by reviewing a generalized form of the well-known second-order sufficient condition, and exploring some of its properties. From here on, all norms will be Euclidean unless stated to be otherwise.

2. A review of the second-order sufficient condition

The second-order sufficient condition is a very well-known regularity condition in nonlinear programming. It is discussed for the standard nonlinear programming problem (1.4) in [3, Section 2.3]; versions adapted to problems in more general spaces are given by, e.g., Guignard [6], Maurer and Zowe [13] and Maurer [14]. In this section we exhibit a form of this condition suitable for the problem (1.1). Although we have not seen this particular form in the literature, it is not likely to be very surprising to anyone familiar with the field. What may be less familiar, however, is its motivation in terms of generalized equations and its use to prove isolation results like Theorem 2.4. To develop a second-order sufficient condition for (1.1), we return to the generalized equation (1.3) which formulates the necessary optimality conditions. We have shown in [19, 20] that important aspects of the behavior of a generalized equation are captured in its linearization about a given point. To make use of this linearization in the present case, we assume that (x0, u0) is a point satisfying (1.2); the linearized form of (1.2) (or equivalently, of (1.3)) at (x0, u0) is then

1+ [

- Le,(x0, u0)]

1 x-

u0j

Simplifying this expression, we obtain 0 e [rLe'(x~ g(xo)U~ + [rsf"(x~ g,(xo)U~ g,(xo).]rXo jLu -- X0]u0+ &bc•

u).

(2.1)

Examination of the form of (2.1) leads us to the realization that it formulates the necessary optimality conditions for a certain quadratic programming problem, namely

f'(Xo)(X - Xo) + 89 - Xo, 5g"(Xo, Uo)(X - x0)), subject to g(xo) + g'(xo)(X - xo) ~ Q~ x E C.

minimize

(2.2)

The quadratic programming problem (2.2) is not derived by any straightforward approximation of the functions in (1.1) (note that the second-order term in the

204

S.M. Robinson/Generalized equations, Part H

objective function contains, not f"(x0), but ~g"(x0, u0)). H o w e v e r , it has been found to be an appropriate approximation of (1.1) for numerical purposes [9, 26]. Since the derivation through linearization of (1.2) is straightforward, we shall refer to (2.2) as the linearization, or linearized form, of (1.1). If we are looking for a condition to impose on a stationary point (x0, u0) of (1.1) to ensure that x0 will be a strict local minimizer of (1.1), an obvious approach to take is to impose some such conditions on (2.2). In fact, the simplest way of obtaining a plausible set of conditions is to consider feasible directions of (2.2) at x0 and to place some restriction on these. Thus, if we consider vectors h E R" such that x0+ h ~ C and g(xo)+ g'(xo)h E QO, we know (and can prove easily from the fact that (x0, u0) satisfies (1.2); see below) that f'(xo)h ~ O. If f'(xo)h > 0, then if h is small enough, the first-order term in the objective of (2.2) will be dominant and the objective value at x0 + h will be strictly greater than that at x0. On the other hand, if f'(xo)h = 0, then the only way to obtain strict increase in the objective function is to resort to the second-order term and to require that (h, ~"(Xo, uo)h ) > O. We can now use these observations to formulate an appropriate set of conditions. To do so, we note that the requirements on h involving the objective function and the Lagrangian of (2.2) are independent of the scale of h, but those involving the constraints (i.e., that x0 + h ~ C and g(xo) + g'(xo)h E QO) are not. To simplify the latter, we enlarge slightly the class of vectors h considered by requiring that h E Tc(xo) and g'(xo)h E Too(g(xo)), where Tc(xo) denotes the tangent cone to C at x0 (i.e., the polar of the normal cone aOc(Xo)), and similarly for Q~ Note that, since C and Q~ are convex, any h satisfying the earlier requirements will necessarily satisfy the latter. For small enough h, the converse is true if C and Q are polyhedral, but generally not otherwise. The steps just described therefore lead us to the following general secondorder sufficient condition: Definition 2.1. Suppose (x0, u0) is a point satisfying (1.2). The second-order su17icient condition holds at (Xo, Uo) with modulus /x > 0 if for each h ~ Tc(xo) with

g'(xo)h ~ TQo(g(xo)),

f'(xo)h = O,

one has (h, Sf"(Xo, uo)h ) >! I~ Ilhll 2. We remark that it would not have changed Definition 2.1 if we had written f'(xo)h ~ O, since that inequality implies f'(xo)h = 0 in the presence of (1.2). To see this, note that h and - [ f ' ( x 0 ) + g'(xo)*Uo] belong respectively to the tangent and normal cones to C at x0. Thus, 0 ~< (f'(x0) + g'(xo)*Uo, h)

= f'(xo)h + (Uo, g'(xo)h).

S.M. Robinson[ Generalized equations, Part H

205

However, since g(xo) belongs to the normal cone to Q at Uo, Uo also belongs to the normal cone to Q~ at g(xo) (see [22, Corollary 23.5.4]). As g'(xo)h belongs to the corresponding tangent cone, we have (u0, g'(xo)h)<<-0 and thus f'(xo)h >>-O. Because of this, in what follows we shall sometimes just show that a vector h satisfies f'(xo)h <<-0 before applying Definition 2.1. In this form, the condition has been discussed for the standard problem (1.4) by Han and Mangasarian, who have shown [7, Theorem 3.5] that for that case it is equivalent to the standard second-order sufficient condition as given in, e.g., [3]. It is thus reasonable to suppose that the condition of Definition 2.1 will enable us to show that some positive definite quadratic form minorizes the objective function of (1.1) on that portion of the feasible set near x0, and takes a minimum equal to f(xo) at x0, since such results have been established for other forms of the condition (cf. [8, Theorem 7.3; 24, Theorem 4.43; 13, Theorem 5.2]). Since we shall need this result in Section 3, we establish it in the following theorem. Theorem 2.2. S u p p o s e that (x0, Uo) satisfies (1.2), that .f and g are twice Fr~chet differentiable at x0, and that the second-order sufficient condition holds there with modulus p. Then f o r each ~ E [0, p) there exists a neighborhood V of Xo such that if x E C f3 V and g(x) ~ QO, then f ( x ) > f(xo) + 89162 IIx - Xoll2 or x = xo.

Proof. Suppose that for some E/> 0 there is a sequence {x~}C C converging to x0 with xi # x0, g(x,) E QO and f(x,) ~ p. We can suppose without loss of generality that { ( x i - Xo)/llxg- x0]l} converges to some h E R"; evidently Ilhl] = 1 and h E Tc(xo). We have for each i Q~ ~ g(x,) = g(xo) + g'(xo)(X, - Xo) + o(Hx~- x0[[),

(2.3)

so g'(xo)h = lim IIx, - Xoil-~[g(xi) - g(x0)] E Tqo(g(xo)). i---~oe

We have previously remarked that in such a case f'(xo)h >>-O, but since for each i we have f(xo) + 89

IIx, -

xoll =/> f(xi) = f(xo) + f'(xo)(xi - xo) + o(l[xi - xoll),

(2.4)

it follows upon division by [Ix~- Xol[ and passage to the limit that also f'(xo)h <<-O, and therefore that h satisfies the hypothesis of the second-order sufficient condition. H o w e v e r , we have from (2.3) and (2.4) that since (Uo, g(xi))<~ O, f(xo) + 89 - xoll 2/> f(xi) >! ~(xi, uo) = ~g(xo, uo) + ~'(xo, uo)(x~ - xo) + 89

- xo, ~e"(xo, uo)(x, - xo)) + o(llx, - xoll) 2

f(xo) + 89 - Xo, ~"(Xo, Uo)(Xi- Xo))+ o(llx, - xoUb. Subtracting f(xo), dividing by 89IIx,

-

xoll = and passing to the limit, we find that

, i> (h, ~e"(Xo, uo)h) I> p Ilhll2 = p,

and the theorem now follows by contraposition.

206

S.M. Robinson~Generalized equations, Part H

Theorem 2.2 shows that a stationary point of (1.1) satisfying the second-order sufficient condition is a strict local minimizer. A reasonable question to ask is whether it is also an isolated local minimizer: that this, whether there is some neighborhood of xo containing no other local minimizer of (1.1). As we shall now see, this is not so, even for very simple problems. Consider, for instance, the problem minimize

89 2,

subject to

x 6sin(I/x) = 0

/sin(I/0):= 0].

(2.5)

Here C = R = Q ; the feasible region is {0}U{(nTr) - 1 1 n = - 1 , - + 2 .... }. The second-order sufficient condition is satisfied at the origin; however, the origin is a cluster point of the feasible region and every feasible point is a local minimizer. One might reply that such bad behavior is not very surprising since (2.5) is a bad problem. This is quite true, and we shall see that a result of the type we seek can indeed be proved if we exclude bad problems: that is, if we impose a constraint qualification. The qualification we shall use is the requirement that the constraints g(x) E Q~ x~C

of (1.1) be regular at x0 in the sense of [18]; that is, that 0 ~ int{g(x0) + g'(xo)(C - xo) - Q~

(2.6)

We have shown in [17] that (2.6) is a sufficient qualification for the derivation of optimality conditions; and in fact for the standard problem (1.4), the condition (2.6) is equivalent to the well-known constraint qualification of Mangasarian and Fromovitz [12]. We shall now show that if the constraints of (1.1) are regular at a point x0 which, together with some u0, satisfies (1.2), then the second-order sufficient condition ensures that x0 is actually an isolated stationary point of (1.1). In order to prove this, it will be convenient to establish first certain continuity results that we shall use here and later in the paper. We therefore introduce at this point the perturbed optimization problem minimize

f ( x , p),

subject to

g ( x , p ) E Q~

(2.7)

x~C,

where p is a perturbation parameter belonging to a topological space P, and x is the variable in which the minimization is done. The functions f and g are defined from D x P to R and R m respectively, where 1-2, Q and C are as previously defined. We shall identify (1.1) with the particular case of (2.7) arising when p is

S.M. Robinson/Generalized equations, Part II

207

some fixed P0 ~ P ; thus x0 is a stationary point of (2.7) for p = p0. Our interest here will be in predicting, from information in (1.1), aspects of the behavior of (2.7) when p varies near P0, such as solvability, location of minimizers, etc. In all of what follows we make the blanket assumption that for each p ~ P, [ ( . , p) and g ( . , p) are Fr~chet differentiable on s that [, g, [' and g' are continuous on g] x P, and that f ( - , po) and g( 9 po) are twice continuously Fr6chet differentiable on g~. The stationary-point conditions for (2.7) are

0 E f'(x, p) + g'(x, p)*u + O~bc(X), 0 E - g(x, p) + a~0o(u),

(2.8)

and we define U0: = {u E R m [ (x0, u) satisfies (2.8) for p = P0}. We note that by the results of [16], if the constraints

g(x, p) ~ QO, x ~ C,

(2.9)

of (2.7) are regular at x0 for the fixed value p = p0, then there are neighborhoods M1 of x0 and N1 of p0 such that for any (x, p ) E M1 x N1 satisfying (2.9), the system (2.9) is regular at x (for the given, fixed, value of p). We shall now show that in addition the multipliers in (2.8) are uniformly bounded, and in fact the set of all such multipliers is an upper semicontinuous function of (x,p). This extends a result in [ll], also given in [5]. Theorem 2.3. If the system (2.9) is regular at Xo for p = P0, then there exist

neighborhoods ME of Xo and N2 of Po, such that if U : M E X N 2 ~ R m and SP : N 2 ~ M 2 are multifunctions defined by U(x, p) := {u E R m [ (x, u, p) satisfies (2.8)} for (x, p) E ME X N2, SP(p ) := {x ~ ME I for some u, (x, u, p) satisfies (2.8)} for p E N2, then U and SP are upper semicontinuous. Proof. We first show that U is locally bounded at (Xo,Po). Assume, on the contrary, that there are sequences {xi} C g~ and {pi} c P converging to x0 and p0 respectively, and a sequence {ui} C R m with limi_.~llu~l]= + 0% such that for each i the triple (x~, ui, p~) satisfies (2.8). With no loss of generality we can suppose that uJllu~ll converges to some y. For each i, (2.8) implies that u~ E Q (a cone) and (ui, g(x~, p~)) = 0; dividing by Ilu~lland passing to the limit we find that y E Q and (y, g(x0, p0)) = 0. Again from (2.8), we have for each i,

f'(Xi, pi) + g'(xi, pi)*Ui E -- Oqic(Xi); again dividing by ]lui[I and passing to the limit, using the fact that OqJcis a closed multifunction whose values are cones, we obtain g'(xo, po)*y E-Oq~c(Xo). By regularity, for some e > 0 there is a point x~ E C with g(xo, po)+g'(xo, po)

208

S.M. Robinson/Generalized equations, Part H

(x~ - x0) + ey E Q~ Using our information about y, we find that 0 >t (y, g(Xo, Po) + g'(x0, p0)(x, - x0) + ey) dyll 2 > 0, a contradiction. It follows that there must exist neighborhoods M 2 of x0 in 12 and N2 of P0 in P, and a compact set K C R m, such that if (x,p)~ M 2 x N2, then U(x, p) C K. Without loss of generality we can suppose that M 2 and N2 are small cnough so that M 2 is compact and that for (x, p) e M 2 x N2 the values llg(x,p)ll, IIf'(x, P)[I and IIg'(x, P)ll all satisfy some uniform bound. Now let

h(x, p, u) := [ f'(x, p) + g'(x, p ) * u ] ; I

- g(x,

p)

we note that there is some compact set L such that if (x, p, u ) E M2x N2 x K, then h(x, p, u) E L. Now denote by G the graph of the multifunction 3qJc x 3~bo, and define H := {(x, p, u, v) [ (x, p) E M2 • N2, (x, p, u, v) ~ graph h, (x, u, - v) E G}. The continuity assumptions and the properties of Otkc and O~o imply that H is closed in M2 x N: • R ~' x R "§ However, if (x, p, u, v) ~ H, then (x, p) ~ M2 x N2 and (x, u, p) satisfies (2.8), so that u ~ K and therefore v E L. As K and L are compact it follows that the projection of H on the space of the first three, o r o f the first two, components of (x, p, u, v) is closed. The first of these projections is the graph of the multifunction U, and the second is that of SP-% Thus U and SP are closed; however, as the image of U is contained in the compact set K and that of SP in the compact set M2, it follows that U and SP are actually upper semicontinuous. This completes the proof. The next theorem gives conditions under which a stationary point of (l.l) is isolated. In fact, we shall prove isolation for stationary points of an extended form of (1.1), namely: minimize subject to

f ( x ) + ~(x - x0, E(x - Xo)), g(x) ~ QO

(2.10)

x ~C, where E is any sufficiently small n x n symmetric matrix. The presence of E may seem superfluous, but we shall use this form of the theorem in Section 4. In the meantime, readers who do not care about E may take it to be zero, in which case (2.10) becomes (1.1). Theorem 2.4. Suppose that [ and g are twice continuously Fr~chet differentiable on 12, and that the regularity condition (2.6) holds at a stationary point Xo of

S.M. Robinson/Generalized equations, Part H

209

(1.1). Suppose also that for each Uo such that (Xo, Uo) satisfies the stationary conditions (1.2), the second-order sufficient condition of Definition 2.1 holds for (1.1) at (Xo, Uo) with some positive modulus. Then there exist a neighborhood W of Xo and a positive number ~, such that for each n x n symmetric matrix E with IIEI]< e, and for each stationary point x of (2.10), either x ~ W or x = Xo. Proof. By contraposition. Suppose that there are sequences {xi} of points in R n -~ {x0} and {Ei} of n x n matrices such that {xi} converges to x0, {E~} converges to the zero matrix (denoted by 0), and for each i, xi is a stationary point of (2.10): that is, for some ui, 0 E f'(xi) + (xi - xo)Ei + g'(xi)*ui + a~bc(Xi), 0 E - g(xD + 0tpQ(ui).

(2.11)

Now regard P as the space of n • n symmetric matrices under the Euclidean norm, and take po = 0. Applying T h e o r e m 2.3, we find that there are neighborhoods W0 of x0 and V0 of 0 (in P ) such that if x i E W o and E~EV0, then ui E U ( x , E~); further, the multifunction U is locally bounded. Thus the u~ are uniformly bounded, so by passing to subsequences if necessary we can assume that: (a) for each i, xi E Wo, Ei ~ Vo and ui E U (xi, El), (b) (x, - xo)/llx, - xoll converges to some h E R n ; (c) ui converges to some u0. We note that by closure of U we have uo~ U(xo, O), which is the set of admissible multipliers for (1.1) at xo. Now from (2.11) we find that for each i, g(xO ~ QO and x~ E C; reasoning as in the proof of T h e o r e m 2.2 we can establish that h ~ Tc(xo), g'(xo)h E Too(g(xo)) and ['(xo)h >~O. From (2.11) we see that for each i, 0 = (ui, g(xi)) = (ui, g(xo))+ (ui, g'(xo)(Xi - Xo)) + o(llx, - xoll), where we have used the fact that the u~ are uniformly bounded. However, as ui E Q and g(xo) E Qo we have (u, g(xo)) <- 0 and therefore 0 <~(ui, g'(xo)(Xi - Xo)) + o(llx, - xd[). Dividing by []xi - x0[[ and passing to the limit, we obtain (2.12)

(Uo, g'(xo)h) >10. Using (2.11) again we find that for each i,

f'(xd

+ (x~ - xo)E~ + g ' ( x i ) * u ~ ~ -

a~c(x~),

S.M. Robinson/Generalized equations, Part H

210

so that 0 >! f ' ( x i ) ( x i - Xo) + (xi - Xo, Ei(xi - Xo)) + (ui, g'(xi)(xi - Xo)),

and upon division by

IIx,- x011and

passage to the limit we find, using (2.12), that

.f'(xo)h <~0 and therefore that h satisfies the hypothesis of the second-order

sufficient condition. Now for fixed i > 0 consider the scalar function s defined for t E [0, 1] by --

X0

-

g(x,)

where (xt, ut):= ( 1 - t)(xo, u0)+ t(xi, ui). This function is continuous on [0, 1] and (continuously) differentiable on (0, 1). Further, since (x0, u0) satisfies (1.2) and (xi, ui) satisfies (2.11), we have s(0)~ 0 ~ s(1). By the mean-value theorem, there is then some t E (0, 1) (depending upon i), with

0~>s'(t)=

x~-xo u~ -

uo

[~e"(x,,u,)+E~ ' l_

-

g'(x,)

g'(x,)*][x~ 0

II_ui -

uo

= (xi - Xo, [~"(xt, ut) + Ei](xi - Xo)).

(2.13)

Dividing (2.13) by Ilxi - x0112and passing to the limit, we find a contradiction to the second-order sufficient condition and thereby complete the proof. We have thus shown that when regularity is added to the second-order sufficient condition, the stationary points x0 of (1.1) are isolated. It is an easy consequence of this fact that if the conditions of Theorem 2.4 hold in some compact region of R n, that region contains only finitely many stationary points, if any. However, this result says nothing about the behavior of stationary points, or minimizers, of the perturbed problem (2.7). Such behavior is treated in the next section.

3. Local solvability of perturbed nonlinear programming problems In this section we establish two continuity properties of the perturbed problem (2.7). The notation remains the same as in Section 2; in addition, we denote the distance from a point a to a set A by d [ a , A ] := inf{lla - a'll I a E A} (+ oo if A = ~), and we write B for the Euclidean unit ball whether in R" or R " ; the space involved should be clear from the context. Our first theorem shows that if the second-order sufficient condition holds at a local minimizer at which the constraints are regular, then that local minimizer persists under small perturbations.

Theorem 3.1. S u p p o s e t h a t f o r p = po, (2.7) satisfies the s e c o n d - o r d e r sufficient c o n d i t i o n at Xo a n d s o m e uo E Uo a n d its c o n s t r a i n t s are r e g u l a r at Xo.

S.M. Robinson[Generalized equations, Part H

211

Then for each neighborhood M of Xo in 0 there is a neighborhood N of po such that if p E N, then (2.7) has a local minimizer in M. Proof. By hypothesis, for p = po the constraints of (2.7) are regular at Xo. By [18, Theorem 1] there are neighborhoods M0 and No of Xo and p0 respectively, and a constant ~, such that for each x E C n M0 and p E No,

d[x, F(p)] ~< ~d[g(x, p ), Q~ where F(p) := {x E C [ g(x, p) E Q~ L e t E0 be the modulus for the second-order sufficient condition at (x0, u0); let E E (0, e0) and choose a positive 8 so that (x0 + 28B)C M0 n M n v, where V is the neighborhood of x0 given by Theorem 2.2 for the chosen ~. Select a neighborhood N6 of po with N6 C No, and some positive a, such that ff p E N6 and if xl, x2 E xo + 26B with IlXl- x211~ 5, then II/(xl, p ) - [ ( x 2 , po)l[ < E~2/16 =:/3. Next, find a neighborhood N of P0 with N C N~ and such that if p E N, then

C sup{llg(x, p) - g(x, p0)ll I x ~ x0 + ~B) ~<min{a, ~8}. Denote/(x0, p0) by qb0. Now choose any p E N. If F ( p ) n {x I IIx - x011-- ~ is not empty, let x' be any point in it. One has x' E C N {x0 + ~B} and g(x', p) E Q~ so

d[x', F(p0)] ~< ~d[g(x', po), Q~ ~< ffllg(x', p0)- g(x', p)ll ~<min{a, ~}. Thus there is some x ~ F(po) with IIx'-x~ll~<min{a, 89 and therefore with IIx;- xoll >I IIx'- xoll- llx'- x~ll/> ~. Accordingly, by Theorem 2.2 we have

f(x~, Po) >1 60 + ~ ( ~ ) 2 = cko + 2/3. Also,

IIx6- x011~< IIx'- xoll § IIx'- gill ~ ~ § ~ = ~, so x; ~ x0 + 28B; as IIx6- x'll ~ ~ we have IIf(x', p) - S(x6, p0)ll < ~. Therefore

f(x', p) >! f(x6, Po) -IIf(x', p) - f(x6, > &0 + 2/3 -/3 = 6o +/3.

po)l] (3.1)

It is also true that

d[xo, F(p)] ~< ~d[g(xo, p ), OO] < ~llg(x0, p) - g(xo, po)ll ~<min{a, 89 Accordingly, there is some x " ~ F ( p ) with

IIx0- x"ll ~<min{a, ~}.

Thus I~(x", p)

-

[(x0, p0)ll
(3.2)

Putting (3.1) and (3.2) together, we see that a minimizer of the function f ( . , p)

212

S.M. Robinson[ Generalized equations, Part H

on the set F(p)O (x0+ 8B) exists (by compactness and the fact that x" belongs to F ( p ) n (x0+ 6B)), but that no such minimizer lies on F ( p ) n {x I IIxTherefore each such minimizer is in fact a local minimizer of f ( . , p) on F(p): that is, a local minimizer of (2.7) which belongs to M. This completes the proof. Our main result on continuity of local minimizers and stationary points is given in the next theorem. For the multifunction SP defined in Theorem 2.3 and the set M3 defined below, SP n M3 denotes the multifunction defined by (SP n M3)(p) := SP(p) n M3. Theorem 3.2. Suppose that [or p = p0, for some Xo E O and each u ~ U(x0, p0), (2.7) satisfies the second-order sufficient condition at (xo, u), and that its constraints are regular at Xo. Then there exist neighborhoods M3 o[ Xo and N3 of P0 such that if the multifunction L M : N3--~ M3 is defined by LM(p) := {x E

M31x is a local minimizer of

(2.7)},

(3.3)

then SP N M3 is continuous at P0 and for each p E N3 one has 0 ~ L M (p ) C SP (p). Proof. Let M2 and W be the neighborhoods of x0 produced by Theorems 2.3 and

2.4 respectively; define M3 := M2N W; suppose in addition that M3 is bounded and is small enough so that for some neighborhood N4 of p0, any p E N4 and any x E M3 satisfying (2.9), the system (2.9) is regular at x. Let N2 be the neighborhood of p0 given by Theorem 2.3; let N5 be the neighborhood obtained by applying Theorem 3.1 to (2.7) with M = M3, and define N3 : = N2 n N4 o Ns. Define LM as in (3.3). We shall first show that L M and SP n M3 are lower semicontinuous at P0. By Theorem 2.4, SP(po) n M3 = {x0}; also, for any p E N3 if x ~ LM(p), then, since regularity of (2.9) is a sufficient constraint qualification [17], we have x SP(p) O M3. Thus if p ~ N3, then ~b~ LM(p) C SP(p) O M3. However, since xo ~ LM(po) by Theorem 2.2, we actually have LM(po) = SP(po) O M3 = {x0}. Let S be any open set in M3 with S O LM(p0) ~ ~; then obviously (SP n M3)(p0) c S. Now by applying Theorem 3.1 to S we can find some neighborhood N6 of p0 such that N6CN3 and if p E N 6 , then L M ( p ) A S ~ O . Hence L M is lower semicontinuous at P0, but since L M C SP O M3 and LM(po) = (SP n M3)(p0), we see that SP n M3 is also lower semicontinuous at p0. But SP is upper semicontinuous from N2 to M2, so SP n M3 is upper semicontinuous from N3 to M3, and thus SP n M3 is actually continuous at P0. This completes the proof. Theorem 3.2 shows that that portion of the set of stationary points near x0 is continuous at P0, but it gives no measure of how it depends on p. In the next section, we show that if the sets C and Q are polyhedral, then the dependence of

S.M. Robinson/Generalized equations, Part II

213

S P and U upon p can be measured by the dependence of the problem functions and their derivatives upon p: in particular, if the latter are Lipschitzian, then SP is 'upper Lipschitzian'. In the meantime, however, we present an example, taken from [20], to show that L M may not be single-valued near p0 and that the inclusion of L M in S P may be strict. Consider the quadratic programming problem minimize

89

x22)- ~lxi,

subject to

-x~+2x2~<0, - xl - 2x2 ~< 0,

where ~1 is the parameter. For "0 = 0 this problem has a unique minimizer at the origin; for any positive ~ it has local minimizers at 27(2, -+ 1) and a saddle point at (77,0). Thus in this case L M is multivalued and is strictly contained in SP whenever "0 > 0.

4. Upper Lipschitz continuity in the polyhedral case Throughout this section we shall assume that C and Q are polyhedral convex sets, and we shall see that in this case we can prove stronger results than we did in Section 3. Specifically, we shall prove that the solutions and multipliers of (2.7), regarded as multifunctions of p, are locally upper Lipschitzian at p0 under appropriate continuity assumptions on the problem functions. We recall that if F is a multifunction from R n to R m, F is said to be locally upper Lipschitzian at z0 E R" if there are a neighborhood Z of z0 and a constant h such that for each z E Z, F ( z ) C F(zo) + A [[z - z0llB. We showed in [21] that any polyhedral multifunction (one whose graph is the union of finitely many polyhedral convex sets) is locally upper Lipschitzian everywhere with a uniform constant. We begin by formulating a general continuity result for nonlinear generalized equations. It says that if a certain condition is satisfied by the linearizations of the nonlinear problem, then the latter has, at least locally, an upper Lipschitzian inverse (which, of course, may not be single-valued and might even be empty). As in Section 3, for a set S and a multifunction F, we define the multifunction F N S by (F Iq S)(x) := F ( x ) A S. The norm on a product space is taken to be the maximum of the norms of the c o m p o n e n t spaces.

Theorem 4.1. Let F be a continuously Fr~chet differentiable function from an open convex set ~F C R" to R n. Let R be any polyhedral multifunction from R" to R n, and define H := F + R. For Xo E ~ and x E R" define LFxo(x) := F(xo) + F'(xo)(X - Xo). Suppose there is some compact set Xo C ~ such that (i) for each Xo and xl in Xo, the restrictions of LFxo and LFxl to Xo are the same;

S.M. Robinson/Generalized equations, Part H

214

(ii) for some positive ~/ and each Xo E Xo, X , n (LFxo + R)-~(0) = Xo, where Xv := Xo + vB C ~. Then there is some positive 8 such that the multifunction H - ~ A X~ is locally upper Lipschitzian at O, with (H -1 n Xs)(0) = Xo. Proof. For x0 ~ Xo, define Txo to be (LF,~ + R ) -~. Let E : = 89 We first show that there exist a constant h and some positive ~ such that for any Xo ~ Xo, T~oO X, is upper Lipschitzian on "oB with modulus ~,. Choose any XoE Xo; then the sum LF~o+ R is polyhedral and by [21, Proposition l / t h e r e are a constant ~,(xo) and a positive ~(Xo) such that T~ is upper Lipschitzian on ~(xo)B with modulus ~,(xo). If we take ~(Xo) small enough, it follows from the fact that (T~o n X,)(0) = Xo that the multifunction T~o n X, is upper Lipschitzian on ~1(xo)B with modulus h (Xo). Now, using the continuity assumptions, choose a neighborhood No of xo small enough that for any x ~ Xo n No, (a) X(x0)llF'(x) -

F'(xo)ll < 89

(b) for any x ' ~ X,, IlLF~o(x')- LFx(x')l] <~89 Choose any x E Xo n No, and any q E 89 and also

If w E (Tx O X,)(q), then w ~ X,

q E LFx(w) + R ( w ) , SO

q + LF,~(w) - LFx(w) ~ LF~o(w) + R ( w ) and thus w E (T~o O X,)[q + LFxo(W) - LF,(w)]. However,

IIq + LF~o(W)-

LFx(w)ll <~Ilqll + IILF~o(w)- LFx(w)ll -< ~n (Xo) + ~n (xo) = n (Xo),

so by the upper Lipschitz continuity we have

w ~ Xo+ X(xo)Ilq + LF~o(w) -

LF~(w)IIB.

Now let xl be a point of Xo with d/w, Xo] = []w - Xll[. By hypothesis, since Xo, x and xl are in Xo, we have L F ~ ( x 0 = LF~(x3. Therefore IlLFxo(w) - LF~(w)II = IlLFxo(x3 - LFx(xO + [F'(xo) - F'(x)l(w - x,)ll -< I I f ' ( x o ) - F'(x)llllw - xlll, and thus

d/w, Xo] <~ X(Xo)llqll + X(Xo)W'(Xo)- F'(x)llllw - x,II -< X(xo)llqll + 89 Xol.

S.M. Robinson/Generalized equations, Part H

215

Therefore d[w, X0] ~ 2X(x0)llqll, and so (T, n X J ( q ) c (T, n X,)(0) + 2x(xo)llqllB, which proves that Tx O X , is upper Lipschitzian on ~l(xo)B with modulus 2h(x0). An elementary compactness argument now shows that for some h and some > 0, for any x ~ X0, Tx O X, is upper Lipschitzian on ~B with modulus h. An argument from the continuity and compactness assumptions shows that one can choose a positive 8 ~< min{E, 71} so that if/3 ~< & then for any y E X, and z E X0 with [lY - zll ~< ~, max{l, x } l l F ( y ) - F ( z ) - V ' ( z ) ( y - z)ll ~< 89 - zll. Now let q E JrlB and let x ~ ( H -~ n X~)(q). Then q E F ( x ) + R ( x ) , so if x~ ~ Xo with IIx- xdl-- dtx, Xol w e have q + LFx,(x) - F ( x ) E LFx,(x) + R(x), and therefore x E (Tx, N X,)[q + LFx,(X) - F(x)]. However,

IIq + L f x , ( x ) - F(x)ll ~ Ilqll + 89 - Xlll so by upper Lipschitz continuity of Txl n X,, d[x, Xo] ~< x

Ilqll + 89

X0l

implying d[x, X0] ~ 2X Ilqll. Hence

(H -1 n xs)(q) c x0 + 2x IlqllB

(4.1)

for each q E lrlB. Now let x0 ~ Xo. By hypothesis 0 E (LFxo + R)(xo) = F(xo) + R(xo) = H(xo), so X o C ( H - I A X D ( O ) . On the other hand, by applying (4.1) with q = 0 we see that ( H -~ n XD(0) c Xo, so that ( H -~ n X~(0) = X0 and H -~ n x8 is upper Lipschitzian on 89 with modulus 2h. This completes the proof. This result will allow us to measure the displacement in a stationary point x and multipliers u in terms of perturbations in the problem functions and their derivatives. We show this next, and then proceed to develop a simpler measure of displacement for the case in which the functions involved are Lipschitzian in p. In what follows, we write U0 for U(xo, po), as we did in Section 2.

216

S.M. Robinson/Generalized equations, Part H

Theorem 4.2. A s s u m e the hypotheses of Theorem 3.2, and suppose also that C and Q are polyhedral. Then there are a neighborhood N7 of po and a constant Ix such that [or each p E N7, each x E S P ( p ) and each u E U(x, p), d[(x, u), {x0}• U0] ~
u)]

~< ~llf(x, u, p0)- f(x, u, p)ll,

(4.2)

where

ff'(x, p) + g'(x, p)* u ] F(x, u, p) := I_ - g(x, p) "

Proof. We shall apply Theorem 4.1 to the function F(x, u) given by F(x, u, po). We take R to be 9$c• and as a candidate for X0 we take the set {Xo}X U0, letting g' be the set 12 x R ' . To verify hypothesis (i), we have to show that if u0, u~, u2 ~ U0, then LF~xo,,o~(Xo, u:) = LF~xo,,l~(Xo, u:).

However, an algebraic computation show~ that each of these quantities is equal to

f'(x0, po) + g'(xo, p0)*u2] - g ( x o , p0)

J'

so that (i) is satisfied. For hypothesis (ii), we suppose that u0 E U0 and that for some x and u, 0 E (LF~x0,uo~+ a~kc• u). Then x is a stationary point, and u is an associated multiplier, for the quadratic programming problem (cf. (2.2)): minimize f'(xo, po)(X - Xo) + 89 - xo, ~"(Xo, uo, po)(x - Xo)), subject to

g(xo, p0) + g'(xo, po)(X - Xo) E QO,

(4.3)

x~C.

However, (4.3) satisfies the second-order sufficient condition (at Xo and any multiplier, the modulus being independent of the multiplier), and its constraints are regular; both of these properties follow from the corresponding properties of the nonlinear problem (2.7). Also, note that if for some u' E R" we substitute in (4.3) the objective function

Xo)) = Uo, po)(X - xo))

f'(xo, po)(X - xo) + ~(x - xo, ~e"(Xo, u', po)(x = f ' ( x o , p o ) ( X - Xo) + 89 - xo, ~g"(xo,

then the quantity in square brackets is an n x n symmetric matrix which will be small if u' is close to uo. Applying Theorem 2.4, we conclude that there is a neighborhood M4(xo), depending on u0, such that for each u' close to u0, and for

S.M. Robinson/Generalized equations, Part H

217

each pair (x,u) with OE[LF~xo,,,)+O~c• either x ~ M 4 or x =Xo. An elementary argument from the compactness of Uo (Theorem 2.3) then shows that M4 can be taken to be independent of Uo. If we now take ~/ to be small enough that Xo+ yB C M4, then for each Uo ~ Uo, if (x, u ) E X~ n [LF(,o,,o)+ OqJc• then x E M4, so x = Xo, SO U E Uo by inspection. On the other hand, if u~ E U0, then clearly (Xo,Ul) E X , and we can also verify that OELF(,o,,o~ (x0, u~) + a~kc• u0. It follows that for each Uo~ Uo, Xv O [LF(xo.uo)+ O~c•

= {Xo}x Uo = Xo.

Thus hypothesis (ii) of Theorem 4.1 is satisfied, and we can conclude that for some positive 8, and some ball W about the origin in R "§ the multifunction [(F(.,., Po)+ a~bcxo)-' n X~ is upper Lipschitzian on W, with [(F(.,., Po) + OOc•

n X~](0) = {xo}x Uo.

Let the upper Lipschitz modulus be/~. Since SP and U are upper semicontinuous at po by Theorem 2.3, we can find neighborhoods N7(po) and Ms(xo) such that M5 C Xo+ ~B and (a) for x ~ M 5 and p E N 7 , U ( x , p ) C Uo+ ~B; (b) for p ~ NT, S P ( p ) C Ms; (c) for x ~ M5 and p E N7, and any u E U(x, p), F(x, u, Po) - F(x, u, p) E W. Let p E N7; let x E SP(p) and u E U(x, p). Then (x, u) E X~ by (a) and (b). If we take v to be the projection of the origin on F(x, u, po)+ ad/c• u), then we have by (2.8), 0 E F(x, u, p) + a~c•

u),

so that F(x, u, Po) - F(x, u, p) E F(x, u, Po) + a~c•

u);

therefore IHI ~ IIF(x, u, Po) - F(x, u, P)II and h e n c e v e W. A s (x, u) E [ F ( - , . , Po) + a~c•

-~ n X~](v),

we have by upper Lipschitz continuity

d[(x, u), {xo} • Uo] ~ ~lloll = ~d[0, F(x, u, Po) + aOC•

tzllF(x, u, po)- F(x, u, P)II, which completes the proof.

u)]

218

S.M. Robinson/Generalized equations, Part H

The bound (4.2) can be simplified if we assume more structure in the perturbations. Corollary 4.3. A s s u m e the hypotheses of Theorem 4.2. Suppose that P is a subset of a linear space, and that [', g and g' are Lipschitzian on ll x P. Then for some constant ~ the bound (4.2) can be replaced by d[(x, u), {x0}x U0] ~
(4.4)

Proof. We have, for the quantities in (4.2),

F(x, u, po) - _,,,,,., u, ,,, =

L

, , o , - S,,,x, , ) l + [,,,,.,<, ,,,o>- g,(x, p ) l . u 1 - [(g(x, po)- g(x, p)] J

As x E SP(p) and is thus near x0, and as u is uniformly bounded for (x, p) near (x0, p0), we find that for some L, liE(x, u, p0) - F ( x , u, P)II ~< LIIp - poll,

and (4.4) then follows from (4.2). The bound (4.4) applies to the distance from x to the point x0, and from u to the set U0. One might wonder whether, if additional hypotheses were added to guarantee that S P would be a single-valued function on a neighborhood of p0, one could establish some stronger property such as local Lipschitz continuity. Such speculation is encouraged by the existence of some known results that seemingly point in this direction. For example, Kojima has shown in [10] that, for the special problem (1.4), a stronger second-order condition than that of our Definition 2.1, together with regularity of the constraints, implies that SP is single-valued and continuous near p0. We showed in [20] that if the constraint qualification is also strengthened from the regularity assumption used in the present paper (and by Kojima) to the assumption that the gradients of the binding constraints are linearly independent, then SP is actually locally Lipschitzian at p0. These known facts motivate the following rather natural question: Can one find an appropriate combination of a constraint qualification (weaker than linear independence) and a second-order condition, under which S P - - o r even LMmwill be a single-valued function locally Lipschitzian at P0? We shall give here an example to show that under a constraint qualification stronger than regularity, together with the strongest possible second-order condition, the answer to this question is no. Specifically, we shall study the problem of projecting the origin onto a convex polytope minimally represented by a set of linear constraints, and we shall show that the unique solution of this problem is not Lipschitzian under small perturbations in the constraints. However, in order to make the nature of this example clear we first briefly explain the term 'minimally represented.' A set of m linear (inequality and/or equality) constraints in n variables is

S.M. Robinson/Generalized equations, Part H

219

called a minimal representation of a convex polytope S C R" if (i) S is the solution set of the constraints, and (ii) there is no system of fewer than m linear constraints in R n whose solution set is S. A number of authors have studied this idea; the paper of Telgen [25] explains it and gives several results about such representations. The main result of [25] says, in our terminology, that a system of linear constraints is a minimal representation of its solution set if and only if it is regular and contains no redundant inequality constraints (i.e., inequality constraints which could be individually deleted without changing the solution set). Thus, the requirement of minimal representation is a stronger constraint qualification than regularity. For the example, consider the family of convex polytopes S(b, c) defined for each pair (b, c) of real numbers by the linear system A(b)x >! a(c), where

A(b) :=

I0101 [1 ,

1

0

1

1

0

1

a(c) :=

9

1

b

l+c

It is easy to check that S(0, 0), which is the set {x x R4[x3 i> 1 + max([x~[, [x21)}, is minimally represented by the system A(O)x >i a(O). Now, the projection of the origin on S(b, c) is the unique solution of the convex quadratic programming problem minimize

llxl]2,

subject to

A(b)x >I a(c).

(QP(b, c))

Of course, the objective function of QP(b, c) is a positive definite quadratic form. However, if one considers the vectors x(b, C) :~--"(0, 0, 1, 0) T "1- cb-l(o, 0, 0, 1) T,

u(b, c) := (0.5, 0.5, 0, 0)T+ cb-2( - 1, - 1, 1, 1)T, one finds that for b > 0 and 0 <~ c ~<~b2,

A(b)x(b, c) = a(c), x(b, c) = u(b, c )A(b ),

(4.5)

u(b,c)>-O. The conditions (4.5) imply that x(b, c) is the unique solution of QP(b, c), On the other hand, it is clear that for any positive b and any el, c2 E [0, 89

IIx(b, cO - x(b, c2)11-- b - ' l c , - c21 = b-'ll[A(b), a(c,)] - [A(b), a(cg]ll, where we have normed the space of problem data [A, a/ by taking the square

220

S.M. Robinson/Generalized equations, Part II

r o o t of the s u m o f s q u a r e s of all e l e m e n t s . T h i s m e a n s t h a t t h e s o l u t i o n o f Q P ( b , c), r e g a r d e d as a f u n c t i o n of t h e d a t a [A, a], c a n n o t b e L i p s c h i t z i a n on a n y n e i g h b o r h o o d o f [A(0), a(0)]. This s e t t l e s in the n e g a t i v e a c o n j e c t u r e of D a n i e l [2], and also a p p e a r s to c o n t r a d i c t a r e s u l t s t a t e d b y L e v i t i n [11, T h e o r e m 4]. A n e a r l i e r v e r s i o n of this e x a m p l e w a s c o m m u n i c a t e d to L e v i t i n b y t h e a u t h o r in 1977, w i t h a r e q u e s t f o r c l a r i f i c a t i o n , b u t as o f 1980 no r e p l y h a d b e e n received.

References [1] C. Berge, Topological spaces (Macmillan, New York, 1963). [2] J.W. Daniel, "Stability of the solution of definite quadratic programs", Mathematical Programming 5 (1973) 41-53. [3] A.V. Fiacco and G.P. McCormick, Nonlinear programming: Sequential unconstrained minimization techniques (Wiley, New York, 1968). [4] A.V. Fiacco, "Sensitivity analysis for nonlinear programming using penalty methods", Mathematical Programming 10 (1976) 287-311. [5] J. Gauvin, "A necessary and sufficient regularity condition to have bounded multipliers in nonconvex programming", Mathematical Programming 12 (1977) 136--138. [6] M. Guignard, "Generalized Kuhn-Tucker conditions for mathematical programming problems in a Banach space", SIAM Journal on Control 7 (1969) 232-241. [7] S.-P. Han and O.L. Mangasarian, "Exact penalty functions in nonlinear programming", Mathematical Programming 17 (1979) 251-269. [8] M.R. Hestenes, Optimization theory: The finite-dimensional case (Wiley, New York, 1975). [9] N.H. Josephy, "Newton's method for generalized equations", Technical Summary Report No. 1965, Mathematics Research Center, University of Wisconsin, Madison, WI (June 1979). [10] M. Kojima, "Strongly stable stationary solutions in nonlinear programs", in: S.M. Robinson, ed., Analysis and computation of fixed points (Academic Press, New York, 1980) pp. 93-138. [11] E.S. Levitin, "On the local perturbation theory of a problem of mathematical programming in a Banach space", Soviet Mathematics Doklady 16 (1975) 1354-1358. [12] O.L. Mangasarian and S. Fromovitz, "The Fritz John necessary optimality conditions in the presence of equality and inequality constraints", Journal of Mathematical Analysis and Applications 17 (1967) 37--47. [13] H. Maurer arid J. Zowe, "First and second-order necessary and sufficient optimality conditions for infinite-dimensional programming problems", Mathematical Programming 16 (1979) 98-110. [14] H. Maurer, "First and second-order sufficient optimality conditions in mathematical programming and optimal control", Schriftenreihe des Rechenzentrums der Universit~it Miinster Nr. 38 (August 1979). [15] S.M. Robinson, "Perturbed Kuhn-Tucker points and rates of convergence for a class of nonlinear-programming algorithms", Mathematical Programming 7 (1974) 1-16. [16] S.M. Robinson, "Stability theory for systems of inequalities, Part I: Linear systems", SIAM Journal on Numerical Analysis 12 (1975) 754-769. [17] S.M. Robinson, "First order conditions for general nonlinear optimization", SIAM Journal on Applied Mathematics 30 (1976) 597--607. [18] S.M. Robinson, "Stability theory for systems of inequalities, Part II: Differentiable nonlinear systems", SIAM Journal on Numerical Analysis 13 (1976) 497-513. [19] S.M. Robinson, "Generalized equations and their solutions, Part I: Basic theory", Mathematical Programming Study 10 (1979) 128-141. [20] S.M. Robinson, "Strongly regular generalized equations", Mathematics of Operations Research 5 (1980) 43-62. [21] S.M. Robinson, "Some continuity properties of polyhedral multifunctions", Mathematical Programming Study 14 (1981) 206-214. [22] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, N J, 1970).

S.M. Robinson[ Generalized equations, Part H

221

[23] R.W.H. Sargent, "On the parametric variation of constraint sets and solutions of minimization problems", Department of Chemical Engineering and Chemical Technology, Imperial College, London (July 1979). [24] J. Spingarn, "Generic conditions for optimality in constrained minimization problems", Dissertation, Department of Mathematics, University of Washington, Seattle, WA (August 1977). [25] J. Telgen, "Minimal representation of convex polyhedral sets", Working Paper No. 88, College of Business Administration, University of Tennessee, Knoxville, TN (February 1980). [26] R.B. Wilson, "A simplicial algorithm for concave programming", Dissertation, Graduate School of Business Administration, Harvard University, Boston, MA (1963).

Mathematical Programming Study 19 (1982) 222-240 North-Holland Publishing Company

KUHN-TUCKER

MULTIPLIERS

AND NONSMOOTH

PROGRAMS

J.-J. S T R O D I O T a n d V. H i e n N G U Y E N

Department of Mathematics, Facult~s Universitaires de Namur, Namur, Belgium Received 2 February 1980 Revised manuscript received 15 May 1981

In this paper we consider a nonconvex and nondifferentiable (but locally Lipschitz) program with inequality constraints. We study the question of existence of bounded Kuhn-Tucker multipliers for this program in not assuming that a constraint qualification such as HiriartUrruty's condition (U) is satisfied. The present work may be viewed as a complement of the investigations concerned with optimality conditions in nonsmooth optimization by Clarke and Hiriart-Urruty. Our results are shown, when specialized to the case of convexity, to yield naturally a complete characterization of optimality of Ben-Israel, Ben-Tal and Zlobec. Our approach here is new and enables us to treat the general class of locally Lipschitz problems. The newness of the approach is demonstrated to be effective for the convex case.

Key words: Kuhn-Tucker-Lagrange Multipliers, Optimality Conditions, Qualification, Generalized Gradient, Nondifferentiable Programming.

Constraint

1. Introduction W e c o n s i d e r the p r o b l e m : (P0)

minimize s u b j e c t to

f(x), fi(x)<-O x~S,

fori=l

.... , p ,

a n d , for a feasible s o l u t i o n ~0 to (P0), the f o l l o w i n g c o n d i t i o n , w h i c h is k n o w n as the

generalized K u h n - T u c k e r condition: (K-T0)

t h e r e e x i s t scalars Xi -- 0 for i E I0 s u c h that

0 ~ of(~o) + ~, x~ of~(yCo)+ N(S; ~0). Here the functions f and f~ are locally Lipschitz on R n (n-dimensional Euclidean space). These conditions on f and f~ imply that they are differentiable almost everywhere in the sense of Lebesgue measure, so that the generalized gradients of Clarke [5-7] Of(~o) and Of~(~o)are nonempty. S is a nonempty closed subset of R" and N(S; ~o) denotes the (generalized) normal cone to S at ~0 (cf. [5, 6]). I0 denotes the set of active constraint indices at ~0 defined by I 0-- {i [fi(x0) ----0, i = 1.... , p}.

(We assume familiarity with general information about the generalized calculus d e v e l o p e d b y C l a r k e for locally L i p s c h i t z f u n c t i o n s ; see the r e f e r e n c e s g i v e n 222

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

223

above. However, for the readers' convenience we shall provide some definitions and results relevant to the present discussion in the next section.) It is well known that x0 solves (P0) locally does not necessarily imply (K-T0). In other words, as stated, the condition (K-T0) is not actually necessary for optimality. It is only necessary under an additional regularity hypothesis--a constraint qualification--that in practice is often difficult or impossible to check. In the context of locally Lipschitz programs, there are only a few constraint qualifications which have been studied. The most useful qualification for us in what follows is the one due to Hiriart-Urruty [8]: (UQ

there exists a vector do ~ T ( S ; x0) such that fi(xo, do) < 0 for all i E Io, 0

--

.

where T ( S ; x0) denotes the (generalized) tangent cone to S at Xo, and f~ do) is the generalized directional derivative of fi at x0 in the direction do. This condition (U~0) besides ensuring the satisfaction of the above condition (K-T0) for local optimality of Xo in (P0) has been shown to be the weakest possible constraint qualification, in the sense that it is both necessary and sufficient for the boundedness of the set K*o of all K u h n - T u c k e r multiplier vectors ()q)i~0 for (P0) defined by K~* = {(h,),~Xo[(K-To) holds} (see Nguyen et al. [ 10]). In addition to being of interest in itself, this property is quite useful in certain stability and duality questions for nonconvex problems in mathematical programming. For a proof that (K-T0) is necessary when s satisfies (UQ, we refer to Hiriart-Urruty [8, Theorem 6], which has been pointed out by Clarke to the authors to be an easy consequence of [6, Theorem 1] when (U1o) holds at ~o. We will now resume our discussion of ~.the locally Lipschitz program (P0) as follows for future reference: " Xo solves (Po) locally,~ f (K-'To) holds, (UQ is satisfied at XoJ f f [K*o is bounded. Remark. If S = R", then T(S; s

(1.1)

= R" and N ( S ; Xo) = {0}.

There are examples of locally Lipschitz programs where constraint qualifications such as (UQ do not, or cannot, hold. Therefore, it is natural to consider the question: Let (P0) be a locally Lipschitz program and suppose we do not have (UQ; can we get an analogue of the implication (1.1) where we replace the condition (K-T0) by some other conditions? In other words, we are interested in finding (if they exist) necessary conditions for optimality for (P0) which require neither (U~o) nor any other such constraint qualification. Of course, for obvious reasons, it is of interest to get conditions that are similar to (K-To). The inspiration behind this paper is the desire for an answer to the above interesting question in the case where

224

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

S = R" in (P0)- In this sense, the present work may be viewed as a complement of the investigations concerned with optimality tests in nonconvex nondifferentiable programming by Clarke [6] and Hiriart-Urruty [8]. On the other hand, in the mathematical programming literature there are a series of recent papers [1--4, 14] which propose to obviate the need for formal constraint qualifications. All of these works require the convexity hypothesis on the functions f and f~ delimiting the problem, and these authors limit themselves (like here) to the problem (P0) when S = R ". The various situations can be adequately summarized in a diagram:

Convex Nonconvex

Differentiable

Nondifferentiable

[4, 2, 1] (*)

[1, 3, 14] (**)

Note that the lower corners (*) and (**) are empty. It is the purpose here to fill out this diagram by treating the lower right-hand corner (**), i.e., the case of n o n c o n v e x and nondifferentiable (but locally Lipschitz) functions. We note at this point that Zlobec and Ja~obson in [16] have recently studied the program (P0) with a n o n c o n v e x objective function and convex constraints. In order to make this paper reasonably self-contained, we begin in Section 2 with a brief review of the necessary background on generalized gradients. Our main results concerning the question of existence of bounded K u h n - T u c k e r multiplier vectors in the absence of prior constraint qualifications are contained in Section 3. Section 4 is given to illustrate our method at work in the pure c o n v e x case by showing that the characterization of optimality of Ben-Israel, Ben-Tal and Zlobec (see [2, Theorem 5.1] and [3, Theorem 5.3]) is merely a consequence of our results. To conclude this introduction, let us point out that our approach to characterizations of optimality is new here and differs radically from that of the authors cited above, besides treating the general class of locally Lipschitz problems. The newness of the approach is demonstrated to be effective for the c o n v e x case.

2. Clarke's generalized gradient In this brief review we are concerned with a fixed locally Lipschitz function g:Rn-->R and a fixed nonempty closed subset E of RL We remind the reader that g is said to be locally Lipschitz if any point ~ in R ~ admits a neighborhood W such that, for some constant C, we have.

I g ( y ) - g(z)l - c Ily - zll

for all y and z E W,

where I1"11denotes the usual norm in R n.

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

225

The generalized gradient of g at ~ E R n is defined as ag(~) = co{6 ~ R n I 6 = lim Vg(xj), {xj}= sequence j-,o~

converging to

such that g is differentiable at each xj E R"}, where co denotes 'convex hull', and Vg(xj) is the gradient of g at xj (see [5, 6]). It plays the same role for locally Lipschitz functions as the gradient for continuously differentiable functions. Indeed, similar rules of calculus hold, and ag(~) boils down to {Vg(~)} when g is of class C 1, and to the set of subgradients of g at J/(see [ l i d if g is convex. Then the generalized directional derivative of g at g in the direction d E R", denoted by g0(~; d), may be described as the support function of ag(g) evaluated at d, i.e.,

g0(~; d) = max{(~, d) l ~ E ~g(~)},

(2.1)

where (., .) denotes the usual inner product. The function d ~g0(~; d) is a finite positively homogeneous convex function. If the classical directional derivative of g at ~ in the direction d defined by g'(~; d) = lim[g(g + t d) - g(YO]/t ~0

exists and coincides with g0(g; d) for each d, then g is said to be regular at ~ (see [7]). Let ~ E E. The (generalized) tangent cone to E at g in the sense of Clarke, denoted by T(E; ~), is the set of all directions v E R" such that, whenever one has sequences {tj} ~ 0 and {xj}~g with xj E E, there exists a sequence {vj}~v with xj + tj vj ~ E for all j (see [9, 12]). Then the (generalized) normal cone to E at ~, denoted by N ( E ; ~), may be defined as the negative polar cone of T(E; ~), i.e.,

N ( E ; ~) = [ T ( E ; g ) ] - =

{u E R n [ (u, v) <--OV v E T(E; ~)},

which generalizes the usual normal space obtained if E is a C]-manifold or the normal cone in the sense of convex analysis if E is convex. In the sequel frequent reference is made to the following result due to Hiriart-Urruty [8, L e m m a 5]: Lemma 1 (Hiriart-Urruty). Let Q be

Q = {x e a" [ g(x) <- o} and ~ be such that g(g)= O. Suppose that there is a vector d E R n satisfying gO(~; d) < O. Then {d ~ R" I gO(g; d) < O}C int T(Q; ~),

where int denotes 'interior'.

226

J.-J. Strodiot and V. Hien Nguyen/Kuhn-Tucker multipliers and nonsmooth programs

Finally, let E1 and E2 be subsets of R", and let ~ E Et N E 2. Suppose that T(E1; ~) (3 int T(E2; 2) # 0 and that E 2 is closed. Two inclusions for tangent and normal cones that we shall need in what follows are due to Rockafellar [13, Corollaries 3 and 4, Section 4]. They are: L e m m a 2 (Rockafellar). With the above hypotheses, the following inclusions

hold: T(El t3 E2, x) D T(E~; ~) N T(E2, ~),

(2.2)

N(E~ M E2; x) C N(El; ~) + N(E2; ~).

(2.3)

and

Moreover, if

E l

and

E 2 are

convex, then equality holds in (2.2) and (2.3).

3. Main results

We derive in this section our principal results concerning the question of existence of bounded Kuhn-Tucker multiplier vectors in the absence of prior constraint qualifications. The locally Lipschitz program considered here is (P)

minimize subject to

f(x), fi(x)-<0

fori=l

. . . . . p.

Observe that (P) is nothing but the problem (P0) introduced in Section 1 with the restriction that S = R n. The set Q of feasible solutions of (P) Q ={x E R " [ f , ( x ) < _ O V i = 1.... ,p}

(3.1)

is assumed nonempty. Let ~ E Q be a fixed feasible solution at which the set of active constraint indices is I = {i [ f i ( x ) = O V i

= 1.... ,p}.

For an arbitrary subset L of I, let us denote by L c the complement of L in I: L c = I - L = { i ] i E I and i ~ L}, and by QL the feasible set associated with the constraints in L: QL = {X E Rn ] fi(x) ~ 0 V i E L}. Conventions. (i) If L = O, the empty set, then we put

QL = Q ~ = R ~.

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

227

(ii) If L = {i} or {i, j}, then we will denote Q{~}and Q{~,j}simply by Q~ and Q~j, respectively. Also we will write Qic instead of Q{~r. Now, let L C I and partition the constraints f~. . . . . fi. . . . . fp according to whether the index i does or does not belong to LC: {1 . . . . . p} = L c t_J ({1 . . . . . p} .. Lr With this partition, we can formulate the problem (P) as follows: (PD

minimize subject to

[(x), /i(x)<_O xEQLo.

for i ~ { 1 . . . . . p } ~ L ~,

Notice that in terms of this partition the set of active constraint indices at 4, for (PL), is nothing but the subset L. Since (PL) is of the same type as the problem (P0) of the introduction, it is natural to introduce, along with Hiriart-Urruty, the following condition (UL) associated with (PL): (UD

there exists a vector a E T(QLo; 4) such that f~ for a l l i E L .

Let DE be DL = {d ~ R n 1[~

d) < 0 V i E L}.

(3.2)

Then (U.D can be expressed as follows: (UL) holds at 4 ~ D L t3 T(QLc; 4) g 0. Conventions. (i) If L = 0, then we set DL = D~ = R n. (ii) If L = {i} or {i, j}, then we will denote D0} and D0,i} simply by Di and D~j, respectively.

Define ~ to be the collection of all subsets L of the set I such that (UL) is satisfied at 4: ~g= {L C I [ (UL) holds at 4},

(3.3)

or equivalently = {L C I [ DL D T(QL,; 4) ~ $}.

(3.4)

With these preliminaries, we are now in a position to state the following simple but fundamental theorem. Theorem 3. In problem (P) suppose that f and fi, i = 1..... p, are locally Lipschitz. Let 4 be optimal for (P). Then for all L E g,

228

J.-J. Strodiot and V. Hien Nguyen/Kuhn-Tucker multipliers and nonsmooth programs

(K-TL)

there exist scalars ~i >- 0 for i E L such that 0 E Of(s + ~ P.i Ofi('2) + N(QLc; x). iEL

Moreover, the set K * of all Kuhn-Tucker multiplier vectors for (PL), defined by K* = {(tZ,),zL [ (K-TL) holds},

(3.5)

is a bounded subset of R ILl, where ILl denotes the number of elements in L. Conversely, the following implication holds: LCI } K* # 0 and bounded ::>L ~ ~g. Proof. This follows immediately from Theorem 6 in [8] and the Theorem in Section 2 of [10]. Remark. With the conclusion of Theorem 3, we can rewrite the definition of as follows: = {L C I I K* # 0 and bounded}, where K* is defined as in (3.5). In light of this theorem, it is possible to gain insight to the proposed question of existence of bounded Kuhn-Tucker multiplier vectors for (P) by studying the mathematical structure of the collection ~. It is easy to check that the relation ' C ' is a partial order in the collection ~. Therefore, it could happen that ~ has an element Lm such that L C Lm for every L in f~. In that case we say that Lm is the largest element of ~. The antisymmetry of an order implies that if ~ has a largest element, then it has only one. Observe that there always exist maximal elements of ~, not necessarily unique, which coincide with the largest element if it exists. Also observe that these maximal elements of ~ are not necessarily disjoint, as will be shown in Example 3 below. Recall (see Remark in Section 1) that the constraint qualification condition (Ut) associated with (P) is defined as follows: (UI)

there exists a vector do E R" such that fi(~;d0)
Also recall that if (Ut) is satisfied at ~, then the set K* of all Kuhn-Tucker multiplier vectors for (P), defined by K* = {(X,),z, ] (K-T,) holds},

(3.6)

is a nonempty bounded subset of R Ptl, where (K-Tp is the generalized KuhnTucker condition for (P):

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

(K-T1)

229

there exist scalars Ai -> 0 for i E I such that

OE c3f(~)+ ~ A , Of,(2). iEl

When L = I, the qualification (UL) for (PL) coincides with (UI), and the generalized K u h n - T u c k e r condition (K-TL) for (PL) is identical with (K-T~), and consequently, the set K * of all bounded K u h n - T u c k e r multiplier vectors for (PL) is equal to K*, because in this case T(QLc; g) = T(Qo; ~) = T(R"; ~) = R" and N(QLc; ;/) = N(Qo; ~) = N ( R " ; ;/) = {0}. Also notice that if we do have (U~) for (P), then I = Lm ; that is, I in this case is the largest element of the collection ~. (But, let us emphasize that condition (U~) is not assumed to hold in the present work.) The question to be answered then boils down to the problem of finding (if it exists) the largest element L,, of the partially ordered set ~. To this we now turn. Note immediately that the class ~ is not empty because we always have that the empty set, 0, belongs to ~. Next, it must be emphasized that the collection ~, as described in (3.3) or (3.4), may be the singleton {0}; that is, the set whose only member is the empty set 0. As an illustration, let us consider the following numerical example. Example 1. Consider the problem: minimize f(x) = - x over all x E R such that fl(x) -----0 and rE(X) -< 0, where

fl(x) =

t0

if x - 0 , if - 1 -< x -< 0, if x - < - l ;

tx+l

f2(x)=

x-1 0

when x -> 1, whenx-
It can be verified that the point ~ = 0 is the optimal solution. At this point, the constraints fl and f2 are active. Using the definition (3.1) of (DE), it follows that Dl

= DE = 0.

From this it can be easily checked that ~g = {0}. Noting that for the case where ~ = {0} we have, by convention,

~, ~, of,(x) = iE0

{0},

230

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

the optimality condition (K-TL) reads as follows: 0 E Of(4) + N(QI ; 4). So, if ~ is the singleton {0}, then the generalized Kuhn-Tucker condition is of no practical value in obtaining useful information about K u h n - T u c k e r multiplier vectors. Hence, to have an interesting case we suppose in the sequel that ~g~ {0}. Of course, it is clear that such an assumption is often difficult or impossible to verify. However, what we want here is to exclude the uninteresting case ~ = {0} from our study, hence the hypothesis made. The following theorem shows that the collection ~g has the 'hereditary' property, and that the set M (the precise definition follows) is almost the largest element of ~. Theorem 4. (a) The following implication holds : BC

::),BE~g.

(b) Let M be M = {i E I I {i} E ~}.

(3.7)

Then the following implication holds : L~g~LCM. Proof. To prove part (a) let A E ~ and B C A. Then, according to the definition (3.4) of ~,

DAn T(QAo; 4) ~ O. As Da = DB N DA.B, this gives that DB fq DA.8 fq T(QAo; g) g 0.

(3.8)

In particular, DA_a ~ 0,

(3.9)

and DA.B A T(QAc; 4) ~ O.

(3.10)

In view of (3.9), we can apply L e m m a 1 to obtain that DAuB C int T(QA~a; ~).

(3.11)

Using this and (3.10), T(QAc; ~) A int T(QA~s; ~) ~ O. We now invoke L e m m a 2 to deduce that this intersection is contained in the

J.-J. Strodiot and V. Hien Nguyen/Kuhn-Tucker multipliers and nonsmooth programs

231

tangent cone T(QAc n QA-B, X) which is nothing but T(Qs~; ~). Applying (3.11) we have that DA .s n T(QAc; ~) C T(Qac; ~). But (3.8) implies that DB n T(QBc; ~) ~ O, which is precisely equivalent to the fact that B E ~. This proves part (a). To prove part (b) let L E ~. It follows at once from part (a) that {i} E ~ for each i ~ L. Hence L C M, giving part (b). Example 2. We now illustrate the above theorem by the following simple example. Consider the program: minimize

f(xl, x2) = x~ + x2 over all (Xl, x2) E R 2 such that .fl(x,, x2) = IXll- 2x: -< 0, 12(x,, x 9 = x ~ - x, <- O, f 3 ( x l , x 9 = (xl - x 9 2 <- O,

It can be easily verified that the point ;~ = (0, 0) is optimal. At this point, the constraints fl, f2 and f3 are active. Using the definitions of DL and T(QLc; x), it follows that

D, = {(d,, d=) ~ R21 Id,I < 2d2} and T(Q23; .~) = {(v, v) E

R~[ v -> o}.

Thus D~ n T(Q23; ~) # 0, which means that {1} E *. Also, it is easy to show in a similar fashion that {2} E * and {3} ~ *. By definition, we have M ={1, 2}. The reader can easily convince himself that = {0, {1}, {2}, {1, 2}}. Note that in this example we have M E ~. The above result is almost the result that that the set M, defined in (3.7), belongs to above example, the reader may be tempted general. Unfortunately, this turns out to be example.

we want. The part that is lacking is the collection ~. Motivated by the to believe that M E ~ does hold in false as indicated by the following

232

J.-J. Strodiot and V. Hien Nguyent Kuhn-Tucker multipliers and nonsmooth programs

Example 3. Consider the program: minimize

f(xj, x2)= x2+ x~ over all (x~, x2)~ R 2 such that fl(x~, x2) = rain{x1, x2}-< 0, f2(x~, x 9 = x~ -<0, f3(xl, X2) ------X2 ~ 0, f4(xl, X2) = X1 + X2--~ 0.

The optimal solution to this program is ~ = (0, 0). At this point, the constraints f~, f2, f3 and f4 are active. The reader can check that {1} ~ ~, {2} E ~, {3} ~ ~ and {4} ~ ~. It is clear that M = {2, 3, 4}. But, on the other hand, by definition, 9234 = {(Xl, X2) ~ R 2 [ Xl < 0, xz > 0, xl + xz < 0}

and

T(Q1; ~) = {(xb x2) e R2! xl -- 0, x2 -< 0}. Combining these two results gives that D234 A T(QI; ~) = 0;

and this is precisely the fact that M does not belong to ~. In fact, = {0, {2}, {3}, {4}, {2, 3}, {2, 4}, {3, 4}}. There are conditions that are both necessary and sufficient for the set M to belong to the collection ~. We shall see such conditions shortly. But, at this point, it might be noted that the following relation is always true.

Proposition 5. The largest element Lm of ~, if it exists, is equal to the set M. Proof. This follows at once from the fact that Lm (if it exists) is equal to the union of the elements of ~. Theorem 6. Let FM be Ft~ = {d e R"

I f~

d) <- ovi ~ M}.

Then the following three statements are equivalent: (a) the set M is the largest element of the collection ~ ; (b) for all i E M, MCcLcI

(c) for all i E M,

Di n T(OM~;~) n FM~ 0.

(3.12)

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

233

Convention. If M = 0, then we put F~ = Fo = R". Proof. (a) implies (b). Suppose that M is the largest element of ~. Then M E ~. By the definition of g,

D~ n T(QMc; '2) # O. As DM C FM, we have for each i E M and each L with M c C L,

0 ~ DM n T(QMo; ~) c D i n DLnt~ n T(Q~tc; "2) n FM.

(3.13)

In particular, DLnM and DLnM n T(QMc; "2) are not empty. Hence, by L e m m a 1,

T(QM~; s n i n t T(QLnu ; '2) # 0. In view of this, L e m m a 2 implies that the following inclusion holds: DLnM n T(Qu~; s

T(QL; x).

(3.14)

Combining (3.13) and (3.14) gives that

0 # DM n T(QMc; "2) c D~ n T(QL; "2) n FM. From this the result now follows. (b) implies (c). This is clear. (c) implies (a). To prove part (a) it suffices to show that M E ~ because of Proposition 5. By hypothesis, for each i E M, there exists a direction di ~ R ~ such that

d~ E D~ n T(QM~; "2) n FM. Let d = ~ieMd~. Since f0('2; .) is a finite positively homogeneous convex function, we have for each i E M, .fi~163d) -< f0('2; dl) + ~, [o('2; dj). jeM

(3.15)

For the first term in the right-hand side of (3.15) we use the fact that dg E D~. We have [7(s d~)<0. For the second term we observe that di E FM implies that f0(~; dj)-<0. It follows that

f0('2; d) < 0. Therefore, since i E M was arbitrary, the direction d belongs to Du. On the other hand, since the tangent cone T(Qu~; s is a convex cone, the definition of d implies that the direction d does belong to this cone. Combining this with the first part of the proof we see that

Du n T(QM~; s # O. That is, M E ~.

234

J.-J. Strodiot and V. Hien NguyenJ Kuhn-Tucker multipliers and nonsmooth programs

Corollary 1. If T(Q~; '2) C T(QMc; '2) A FM

(3.16)

holds, then the set M is the largest element of the collection ~. Proof. Let i E M. Then {i} E ~, or equivalently, Di N T(Qic; '2) ~ 0. Using Lemmas 1 and 2, we conclude that D~ A T(QI; '2) ~ 0. This, together with the hypothesis (3.16), gives that D~ A T(QMc; "2) A FM~ O. We now invoke the theorem to complete the proof. In the case where the functions f~, i E / , are assumed to be regular at '2 (see Section 2), the above hypothesis (3.16) may be simplified by means of the following fact. Lemma 7. If the functions f~, i E I, are regular at ,2, then the following inclusion holds: T(Q,; '2) c F,, where Fr is defined as in (3.12) but with M replaced by L Proof. Let d ~ T(Qt; '2) and let {tj} $ 0. The definition of the tangent cone T(Q, ; '2) shows that there exists a sequence {dj} -~ d such that '2 + tj d i E QI for all j (see Section 2). Therefore, for each i E L

f,('2 + tj d ) - f,('2) = f,('2 + tj d) <_f,('2 + tj d ) - f,('2 + tj dj)

tj

tj

tj

Since the functions f~ are locally Lipschitz (with constants Ci), we have that the right-hand side of this inequality is less than or equal to C~ [Id- dill. So, in the limit j ~ 0% lim li('2 + tj d) - f,('2) _< 0. ti

i-~

That is, for all i E I, f~('2; d) -< O. By regularity of fi, i E I, at '2, we have f~

for all i ~ I .

Hence, by definition, d ~ Ft.

J.-J. Strodiot and V. Hien Nguyenl Kuhn-Tucker multipliers and nonsmooth programs

235

Corollary 2. Suppose that the functions fi, i E I, are regular at ~. If T(Q~ ; g) C T(QMc; ~)

(3.17)

holds (in particular, such is the case when M = I), then the set M is the largest element of the collection ~g. Proof. This follows at once from L e m m a 7 and Corollary 1. Remark. There are examples which show that the conclusion of Corollary 2 does not hold in the absence of the hypothesis (3.17). It should be noted that Q~ c QMc does not necessarily imply that the inclusion T(Q~; i ) C T(QMc;i) holds true. For example, let us consider the program: minimize

f(Xl, x2) = xt + x2 over all (xt, x2) E

R2

such that

fl(x~, x~) = x ~ - x2 <-O, f2(xl, x2) = x ; - x2 <- O,

f3(xt, x2) = min{x 2, x2r} -< 0. The reader can easily convince himself that ~ = (0, 0), I = {1, 2, 3} and M = {1, 2}. We thus have Q, = {0} x [0, oo[,

Qu c = Q3 = {(xl, x2) E R E[ xl = 0 or x2 = 0}. Observe that Q~ c Qua. It follows immediately from the definition of tangent cone that T ( Q , ; 4) = {0} x [0, ~[, T(QM~; x) = {(0, 0)}. Therefore, T(Q~; ~) is not contained in T(QMc; g). Finally, observe that all of the functions f~, i E I, are regular at g in this example. For later use, we consider the implication of Corollary 2 in the case of convexity.

Corollary 3. Suppose that the functions fi, i E L are convex. Then the set M is the largest element of the collection ~. ProM. Since Q~ c QMc and these sets are convex by hypothesis, it is well known that T(Q~; ~) C T(QMc; ~). Noting that a c o n v e x function is regular, it becomes clear that the desired result is merely a consequence of Corollary 2.

236

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

4. T h e c o n v e x c a s e

In this section we establish that our optimality condition is not only necessary, but also sufficient in the case of convexity, and show how the complete characterization of optimality of Ben-Israel, Ben-Tal and Zlobec (see [2, Theorem 5.1] and [3, Theorem 5.3]) for convex programs appears in a natural way as a special case of our previous results. Let (P) be the problem defined as in Section 3. But, now we suppose that the problem functions f, ft . . . . . fp are convex. With these hypotheses and notations, we have the following result: Theorem 8. Let ~ be a feasible point to (P). Then the following two statements are equivalent: (a) Y` is optimal for (P); (b) There exist scalars Ixi >- 0 for all i ~ M, the largest element of ~g, such that

0 E Of(Y,) + ~, ~, Of~(y`)+ N(QMc; y`).

(4.1)

i~M

Proof. (a) implies (b). This is clear from Corollary 3 and T h e o r e m 3. (b) implies (a). Recall that Q denotes the feasible set of (P); see (3.1). Since the function f is c o n v e x and the set Q is a convex subset of R", Y` will be optimal for (P) if f'(y`; x - Y`) -> 0

for all x E Q.

Let x E Q. As the set QMc is also convex, x -y` E T(QMo; s Using (4.1), (2.2) and the definition of normal cone (see Section 2), we obtain that f ' ( y ` ; x - y ` ) + ~, ~ i f ~ ( y ` ; x - ~ ) - > 0 . i~M

On the other hand, the segment [~; x - Y`] is contained in Q; and consequently,

f~(y`;x-g)-----O

for each i E M .

Hence f'(y`; x - Y`) -> O, and Y` is optimal. We will now establish two 'explicit' formulas for M c and N(Quc; Y`), which will imply that T h e o r e m 8 includes, as a special case, [2, T h e o r e m 5.1] and [3, Theorem 5.3].

The complement of the largest element M of ~ in I is equal to the set of constraints which are identically zero on the entire feasible set:

L e m m a 9.

M c = {i [ i = 1. . . . . p; f,(x) = OVx ~ Q}.

(4.2)

Remark. The set of the right-hand side of (4.2) was introduced by Abrams and

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

237

Kerzner in [1] where the reader can also find an algorithm for constructing this set in the case of differentiability. See also [15] for another computational method. Proof. For simplicity of notation, let us denote by J= the set in the right-hand side of (4.2). M c is contained in J =. Let i ~ M c, and let x E Q, x ~ 4. Since Qi c is convex, the direction d - - x - ~ belongs to the tangent cone T(Qic; 4). By the definition of M c, d E Di. H e n c e f~(~, d) -> 0. It follows now from [11, Theorem 23.1] that f~(~+td)>0 t

for a l l t > 0 .

In particular, if we take t = 1, then f~(~ + d)>-0, or equivalently, f~(x)>-O. The opposite inequality is immediate because x E Q. Hence f~(x) = 0. That is, i E J=. J= is contained in M ~. Let i ~ J~. Then, of course, i E I. Suppose that i ~ Me; that is,

Di f) T(QLG 4) ~ O. Take an element d in this intersection. Since Di~ 0, by L e m m a 1, 19, C int

T(Q~; 4).

It follows from L e m m a 2 that d E T(Qx; 4). Then by using the definition of tangent cone in the sense of convex analysis, we have that there exist sequences {tj} ~ 0 and {dj} ~ d such that ~ + ti dj ~ QI for all j. Moreover, as {tj dj} ~ 0 and fk(~) < 0 for all k ~ I, we can deduce from the continuity of fk, k ~ I, that + tj dj ~ Q

for j large enough.

So, Q being convex, the segment [4; ~ + tjdj] is contained in Q. Since f~(x)= 0 for all x E Q, the directional derivative f~(~; dj) -- 0

for j sufficiently large.

In the limit j -~ ~, we get f'~(~; d) -- 0, and thus d E D~. But, this contradicts the assumption that d ~ D~. Hence J ~ C M L Lemma 10. The normal cone N(QM~; ~) to Qu~ at ~ is equal to the negative polar cone of the intersection iEM c

where D~(~) is the cone of directions of constancy of f~ at 4: D~(~) = {d ~ R ~ I there exists r > 0 such that f,(~ + ~d) =f,(~) V a ~ ]0, ~[}.

238

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

Proof. For simplicity of notation, let us set

N(QMo;4) is contained in [D~c(2)]-. It is clear that each direction of constancy of fi, i E M c, belongs to the tangent cone T(QM0; 4). Hence, to conclude, it suffices to take the negative polar cone for obtaining that

[D~c(2)]- ~ [T(QMc; 4)]- = N(QMc; 4). [D~c(2)]- is contained in N(QMc; 4). Let us consider a d E [D~e(2)]-. By the definition of normal cone in the sense of convex analysis (see [11]), we have to prove that ( y - 2 , d)-<0

for a l l y E Q M %

which is equivalent to the fact that d E N(QMc; 2). Therefore, let y U Quc and consider a point, say z, of the segment [4; y]. Since QMc is convex, z E QMc. Now we have two possibilities: either z E Q or z ~ Q. If z E Q, then we can easily deduce from L e m m a 9 that .fi(z)= 0 for all i ~ M c. If z ~ Q, then we claim that the same property still holds in this case. Let us admit this fact for the time being. So, we have that f~(z) = O, i ~ M ~, for all z E [2; y]. But this means that y - 2 E D~ff~). Hence ( y - 2 , d)___0 for all y E Q M c because d ~ [ D ~ o ( 2 ) ] - ; that is, d N ( Q ~ ; ~). To finish the proof, let us prove the claim. Put K = {1. . . . . p} -- ML Since z ~ Q and z E QMc, this set K is not empty and satisfies the following relation: For each i ~ K, there is a w i E Q such that [~(w~) < O. Then, by convexity of [i, the point a, = (I/IKI)~K w~belongs to Q and verifies the inequality: .f~(ff~)< 0

for all i E K.

Therefore, by continuity of [i, we have f~(w)<0

foralli~K

a n d a l l w E B ( C v , E),

where B(ff, E) denotes the open ball centered at ~ and of radius E > 0. Consider now the segments [z; ~i,] and B(~i,; ~) f3 [z; ~]. By construction of z and ff~,the segment [z; ff:] is contained in QM0 and does not reduce to a singleton. As B(~,, ~)C QK, the segment B(~, ~)tq [z; ~] is contained in Q and is of the type ]~; ~]. So, by L e m m a 9, f~ is identically zero on ]~; .~] for all i E ML From

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

239

this it follows that fi(z)=O

for alliEM

c.

(If not, that is, s u p p o s e fi(z) < 0 f o r s o m e i E M c, then w e could have a c o n v e x f u n c t i o n fi w h i c h is zero on ]~; if] and takes a negative value at z ~ ~; this is impossible.) The claim is established, and the p r o o f is finished. To c o n c l u d e the paper, we p r e s e n t the following result w h i c h is an immediate c o n s e q u e n c e o f T h e o r e m 8 and L e m m a s 9 and 10. Corollary (Ben-Israel, Ben-Tal and Zlobec). L e t ~ be a feasible point to the c o n v e x p r o g r a m (P). T h e n ~ is optimal to (P) if and only if there exist scalars Ixi >-O, i E M, such that 0 E 3f(X) + ~ , tx, Of,(~) + [D;=(J/)]-, iEM

where M = I \ Y = and J= is defined as in the right-hand side o f (4.2). So, b y an entirely different t e c h n i q u e , we h a v e obtained, as a special c a s e of our results, the c o m p l e t e c h a r a c t e r i z a t i o n of optimality in [2, T h e o r e m 5.1] and [3, T h e o r e m 5.3].

Acknowledgment The a u t h o r s wish to t h a n k J e a n - B a p t i s t e H i r i a r t - U r r u t y ( C l e r m o n t - F e r r a n d ) for suggesting s o m e i m p r o v e m e n t s in the presentation.

References [1] R. Abrams and L. Kerzner, "A simplified test for optimality", Journal of Optimization Theory and Applications 25 (1978) 161-170. [2] A. Ben-Israel, A. Ben-Tal and S. Zlobec, "Optimality conditions in convex programming", in: A. Pr6kopa, ed., Survey of mathematical programming (North-Holland, Amsterdam, 1979) pp. 153-169. [3] A. Ben-Tal and A. Ben-Israel, "Characterizations of optimality in convex programming: The nondifferentiable case", Applicable Analysis 9 (1979) 137-156. [4] A. Ben-Tal, A. Ben-Israel and S. Zlobec, "Characterization of optimality in convex programming without a constraint qualification", Journal of Optimization Theory and Applications 20 (1976) 417--437. [5] F.H. Clarke, "Generalized gradients and applications", Transactions of the American Mathematical Society 205 (1975) 247-262. [5] F.H. Clarke, "A new approach to Lagrange multipliers", Mathematics o[ Operations Research 1 (1976) 165-174. [7] F.H. Clarke, "Generalized gradients of Lipschitz functionals", Mathematics Research Center Technical Summary Report No. 1687, University of Wisconsin, Madison (1976).

240

J.-J. Strodiot and V. Hien Nguyen/ Kuhn-Tucker multipliers and nonsmooth programs

[8] J.-B. Hiriart-Urruty, "On optimality conditions in nondifferentiable programming", Mathematical Programming 14 (1978) 73-86. [9] J.-B. Hiriart-Urruty, "Tangent cones, generalized gradients and mathematical programming in Banach spaces", Mathematics of Operations Research 4 (1979) 79-97. [10] V.H. Nguyen, J.-J. Strodiot and R. Mifflin, "On conditions to have bounded multipliers in locally Lipschitz programming", Mathematical Programming 18 (1980) 100-106. [11] R.T. Rockafellar, Convex analysis (Princeton University Press, Princeton, N J, 1970). [12] R.T. Rockafellar, "Clarke's tangent cones and the boundaries of closed sets in R"", Nonlinear Analysis 3 (1979) 145-154. [13] R.T. Rockafellar, "Directionally Lipschitzian functions and subdifferential calculus", Proceedings of the London Mathematical Society 39 (1979) 331-355. [14] H. Wolkowicz, "Geometry of optimality conditions and constraint qualifications: The convex case", Mathematical Programming 19 (1980) 32-60. [15] S. Zlobec and B. Craven, "Stabilization and calculation of the minimal index of binding constraints", Mathematische Operationsforschung und Statistik, Series Optimization 12 (1981) 203-220. [16] S. Zlobec and D.H. Jacobson, "Minimizing an arbitrary function subject to convex constraints", Utilitas Mathematica 17 (1980) 239-257.