Applied Numerical Mathematics 61 (February 2011)

Editors-in-Chief Robert Beauwens Facultés des Sciences Appliquées, Service de Métrologie Nucléaire, Université Libre de ...

Author: Robert Beauwens | Martin Berzins

33 downloads 505 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Editors-in-Chief Robert Beauwens Facultés des Sciences Appliquées, Service de Métrologie Nucléaire, Université Libre de Bruxelles, Ave. F.D. Roosevelt 50, 1050 Brussels, Belgium

Martin Berzins School of Computing, University of Utah, 50 S. Central Campus Dr., Rm 3190,Salt Lake City, UT 84112-9205, USA

Founding Editor and Editor Emeritus

Editor Emeritus

R. Vichnevetsky

Joseph E. Flaherty

Senior Editors M. Ainsworth Department of Mathematics Strathclyde University 26 Richmond Street Glasgow G1 1XH, UK Randolph Bank Department of Mathematics University of California at San Diego La Jolla, CA 92093-0001 USA

Claude Brezinski Laboratoire d’Analyse Numérique et d’Optimisation UFR de Mathématiques Pures et Appliquées Université des Sciences et Technologies de Lille 59655 Villeneuve d’Ascq Cedex France J.J. Dongarra Computer Science Department University of Tennessee Knoxville, TN 37996-1301, USA

Peter K. Jimack School of Computing University of Leeds Leeds, LS2 9JT, UK

J.M. Sanz-Serna Facultad de Ciencias Departamento de Matemática Aplicada y Computación Universidad de Valladolid Valladolid, Spain

Ernest Mund Service de Métrologie Nucléaire Université Libre de Bruxelles Ave. F.D. Roosevelt 50 1050 Brussels, Belgium

Editorial Board Alvin Bayliss Department of Engineering Science and Applied Mathematics Northwestern University Evanston, IL 60201, USA Stefania Bellavia Dipartimento di Energetica “Sergio Stecco” University of Florence via C. Lombroso 6/17 50134 Firenze, Italy J.C. Butcher Department of Mathematics The University of Auckland Private Bag 92019, Auckland, New Zealand Eric de Sturler Department of Mathematics Virginia Tech 544 McBryde Hall Blacksburg, VA 24061-0123, USA Dinh Nho Hào Hanoi Institute of Mathematics Vietnam Academy of Science and Technology 18 Hoang Quoc Viet Road, 10307 Hanoi, Vietnam

Jens Lang Department of Mathematics Darmstadt University of Technology Darmstadt 64289, Germany Dr. Torsten Linß Anne Gelb Institut für Numerische Mathematik Department of Mathematics and Technische Universität Dresden Statistics D-01062 Dresden Arizona State University Germany Tempe, AZ 85287-1804, USA Yvan Notay Norbert Heuer Brussels Free University Pontificia Universidad Católica de Chile Faculty of Applied Sciences Santiago, Chile ULB, CP 165/84 Ave. F.D. Roosevelt 50 Willem Hundsdorfer Brussels 1050 MAS Center for Mathematics and Computer Belgium Science Jorg Peters P.O. Box 94079 Dept of C.I.S.E. 1090 GB Amsterdam, The Netherlands University of Florida CSE Bldg M.Y. Hussaini Gainesville, FL 32611-6120, USA Program in Computational Science Alfio Quarteroni and Engineering Institute of Analysis and Scientific Florida State University Computing 411 Dirac Science Center Library CMCS – Modelling and Scientific Tallahassee, FL 32306-3075, USA Computing Zdzislaw Jackiewicz Ecole Polytechnique Federale Department of Mathematics de Lausanne (EPFL) Arizona State University Av. Piccard, Station 8 Tempe, Arizona 85287, USA 1015 Lausanne, Switzerland Alan Feldstein Department of Mathematics Arizona State University Tempe, AZ 85287, USA

G. Richter Department of Computer Science Rutgers University New Brunswick, NJ 08903, USA Adrian Sandu Department of Computer Science Virginia Polytechnic Institute Blacksburg, VA 24061, USA Roger Temam Mathematics Department Indiana University Rawles Hall, 831 East Third Street Bloomington, IN 47405, USA S. Tsynkov Department of Mathematics North Carolina State University Box 8205 Raleigh, NC 27695, USA J. Vignes Laboratoire d’Informatique de Paris 6 U. Pierre et Marie Curie 4 Place Jussieu 75230 Paris Cedex 05, France Dongbin Xiu Department of Mathematics Purdue University, West Lafayette IN 47907, USA

© 2010 IMACS. Published by Elsevier B.V. All rights reserved Publication information: Applied Numerical Mathematics (ISSN 0168-9274). For 2011, volume 61 is scheduled for publication. A combined subscription to Applied Numerical Mathematics and Mathematics and Computers in Simulation at reduced rate is available. Subscription prices are available upon request from the Publisher or from the Elsevier Customer Service Department nearest you or from this journal’s website (http://www.elsevier.com/locate/apnum). Further information is available on this journal and other Elsevier products through Elsevier’s website: (http://www.elsevier.com). Subscriptions are accepted on a prepaid basis only and are entered on a calendar year basis. Issues are sent by standard mail (surface within Europe, air delivery outside Europe). Priority rates are available upon request. Claims for missing issues should be made within six months of the date of dispatch. Orders, claims, and journal enquiries: please contact the Elsevier Customer Service Department nearest you: St. Louis: Elsevier Customer Service Department, 3251 Riverport Lane, Maryland Heights, MO 63043, USA; phone: (877) 8397126 [toll free within the USA]; (+1) (314) 4478878 [outside the USA]; fax: (+1) (314) 4478077; e-mail: [email protected]. Oxford: Elsevier Customer Service Department, The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK; phone: (+44) (1865) 843434; fax: (+44) (1865) 843970; e-mail: [email protected]. Tokyo: Elsevier Customer Service Department, 4F Higashi-Azabu, 1-Chome Bldg, 1-9-15 Higashi-Azabu, Minato-ku, Tokyo 106-0044, Japan; phone: (+81) (3) 5561 5037; fax: (+81) (3) 5561 5047; e-mail: [email protected]. Singapore: Elsevier Customer Service Department, 3 Killiney Road, #08-01 Winsland House I, Singapore 239519; phone: (+65) 63490222; fax: (+65) 67331510; e-mail: [email protected]. Advertising information: If you are interested in advertising or other commercial opportunities please e-mail [email protected] and your enquiry will be passed to the correct person who will respond to you within 48 hours.

∞ The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper). Published monthly

0168-9274/07/$36.00

Printed in The Netherlands

Applied Numerical Mathematics 61 (2011) 149–159

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

Stein implicit Runge–Kutta methods with high stage order for large-scale ordinary differential equations A. Bouhamidi ∗ , K. Jbilou L.M.P.A, Université du Littoral, 50 rue F. Buisson BP699, F-62228 Calais-Cedex, France

a r t i c l e

i n f o

Article history: Received 14 July 2008 Received in revised form 22 May 2010 Accepted 20 August 2010 Available online 15 September 2010 Keywords: Ordinary differential equations Runge–Kutta Block Krylov subspace Stein matrix equations

a b s t r a c t The Runge–Kutta method is one of the most popular implicit methods for the solution of stiff ordinary differential equations. For large problems, the main drawback of such methods is the cost required at each integration step for computing the solution of a nonlinear system of equations. In this paper, we propose to reduce the cost of the computation by transforming the linear systems arising in the application of Newton’s method to Stein matrix equations. We propose an iterative projection method onto block Krylov subspaces for solving numerically such Stein matrix equations. Numerical examples are given to illustrate the performance of our proposed method. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

1. Introduction In this paper we consider the numerical solution of initial value problems for a system of ordinary differential equations (ODE), i.e. initial value problems (IVPs), of the form

y (t ) = f (t , y (t )), t ∈ [t 0 , T ], y (t 0 ) = y 0 ∈ Rm ,

(1)

where y : [t 0 , T ] → Rm and f : [t 0 , T ] × Rm → Rm . We assume that the function f is suﬃciently differentiable and that the usual Lipschitz condition, which guarantees the existence and uniqueness of a solution y of (1), holds. Among the numerical techniques commonly considered for the eﬃcient solution of initial value ordinary differential equations are the Implicit Runge–Kutta (IRK) schemes (see, for example [6,15,16] and references therein). The use of IRK methods leads to systems of nonlinear equations usually solved by Newton-type methods. For an s-stage IRK method and a differential system of dimension m, the obtained nonlinear system is of size ms × ms and its solution by Newton-type methods requires hight computation and cpu-time when s 2 and m is large. Many subclasses of implicit schemes, such as diagonally implicit multistage integration methods [1,7,24], singly implicit Runge–Kutta schemes, and mono-implicit schemes, have been developed to attempt to reduce these costs. It is also possible to proceed by block, see [11,13,17,19]. The block implicit methods have been recently implemented in parallel computation by other authors (see [26,27]). For large problems, Krylov subspace methods such as the GMRES algorithm [22] could also be used; see [4]. In this paper, we will exploit the special structure of the matrices of the linear system arising at each step of the Newton method. Thus, by using some properties of the Kronecker product, the linear systems could be transformed into Stein matrix equations. Then block Arnoldi method is proposed for solving the Stein matrix equations. This allows one, to use implicit Runge–Kutta methods for ordinary differential equations in large dimensional space.

*

Corresponding author. E-mail addresses: [email protected] (A. Bouhamidi), [email protected] (K. Jbilou).

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.08.010

150

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

Let A = (ai j ) and B = (b i j ) be m × s and n × q matrices respectively. The Kronecker product of the matrices A and B is deﬁned as the mn × sq matrix A ⊗ B = (ai j B ). The vec operator transforms a matrix A of size m × s to a vector a = vec( A ) of size ms × 1 by stacking the columns of A. Some properties of the Kronecker product are given in [18]. In this paper we will use the following relations

vec( A X B ) = B T ⊗ A vec( X ),

(2)

( A ⊗ B )(C ⊗ D ) = ( AC ) ⊗ ( B D ).

(3)

The well known Frobenius norm denoted by . F is given by Z F = Z ) where tr ( Z ) denotes the trace of the square matrix Z . The paper is organized as follows. In Section 2 we recall the Implicit Runge–Kutta methods, we discuss the approximate Jacobian matrix used here and we show how the Newton method is transformed into Stein matrix equations. The block Arnoldi method for solving nonsymmetric Stein equations is proposed in Section 3. Some numerical results illustrating the effectiveness of our proposed approach are given in Section 4. tr( Z T

2. Implicit Runge–Kutta methods and Stein matrix equations In this section we recall some backgrounds about the implicit Runge–Kutta (IRK) methods for the numerical approximation of ODEs (1) and we show how to derive the Stein matrix equation arising at each integration step. Many of the given results concerning the IRK methods are classical and can be found in several books, see for example [5,6,15,16]. Let tn = t 0 + nh, for n = 0, . . . , N, be a discretization by the N + 1 points t 0 , . . . , t N of the interval [t 0 , T ], where h is the stepsize such that Nh = T − t 0 . The Implicit Runge–Kutta methods of s stages advances from (tn , yn ) to (tn+1 , yn+1 ) by using the following numerical scheme

yi = y n + h

s

ai , j f (tn + c j h, y j ),

i = 1, . . . , s

(4)

j =1

y n +1 = y n + h

s

b j f (tn + c j h, y j ).

(5)

j =1

Here yn approximates y (tn ) and the m-dimensional stage vectors yi approximate y (tn + c i h). The IRK methods (4)–(5) have order p (see [15]) if for some K and small enough h, we have

y (tn + h) − yn+1 K h p +1 , where . is any norm deﬁned on Rm . We will assume that s 2 for the remainder of the article. Let b = (b1 , . . . , b s ) T be A = (ai j )i , j =1,...,s be the RK coeﬃcient matrix. The vector c is the weight vector, c = (c 1 , . . . , c s ) T be the node vector, and let Ae = c with e = (1, . . . , 1) T ∈ Rs . The Runge–Kutta coeﬃcients are usually given by the Butcher-tableau as follows such that

c A b

T

c 1 a11 . . . a1s

.. = .

.. .

c s as1 b1

..

. ... ...

.. . ass bs

Using tensor notations, the one step IRK methods (4)–(5) can be written as

y = (e ⊗ yn ) + h( A ⊗ I m ) F (tn , y),

(6)

yn+1 = yn + h(b ⊗ I m ) F (tn , y),

(7)

where I m is the m × m identity matrix, y = (y1T , . . . , ysT ) T ∈ Rms is the stage vector, and the function F : [t 0 , T ] × Rms → Rms is given by

F (t , y) = f (tn + c 1 h, y1 ) T , . . . , f (tn + c s h, ys ) T

T

(8)

.

By solving the nonlinear system given in (6) we obtain y and we compute yn+1 from (7). Let R n : R deﬁned by

ms

R n (y) = −y + (e ⊗ yn ) + h( A ⊗ I m ) F (tn , y).

→R

ms

be the function

(9)

Hence, (6) is equivalent the following nonlinear system of equations

R n (y) = 0.

(10)

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

151

Therefore, applying Newton’s method to, we get the iterations

y(0) initial guess y(k+1) = y(k) − [ J R n (y(k) )]−1 R n (y(k) ),

k = 0, 1 , . . . ,

(11)

where J R n (y) is the Jacobian matrix of size ms × ms of the function R n evaluated at y. An easy computation gives

J R n (y) = − I ms + h( A ⊗ I m ) J F (tn , y), where J F (tn , y) is the Jacobian matrix of size ms × ms of the function F evaluated at (tn , y). It is also easy to obtain

J F (tn , y) = diag J f (tn + c 1 h, y1 ), . . . , J f (tn + c s h, ys ) ,

∂f

where J f (t , y ) = ∂ y i (t , y ) 1i , j m with f = ( f 1 , . . . , f m ) T , is the Jacobian matrix of size m × m of the function f . We j approximate the Jacobian matrices J f (tn + c i h, yi ), for i = 1, . . . , s by a same matrix J n which is may be the Jacobian matrix J f (t , y ) at some intermediate point, usually the point (tn , yn ). Therefore, the Jacobian matrix J F (tn , y) can be approximated by

J n = diag[ J n , . . . , J n ] = I s ⊗ J n . s times

It follows that the Jacobian matrix J R n (y) can be approximated by the following matrix

J R n = − I ms + h( A ⊗ I m )( I s ⊗ J n ). Using the property (3), we obtain

J R n = − I ms + A ⊗ (h J n ).

(12)

Now, the Newton scheme (11) can be replaced by the modiﬁed one, given by

y(0) initial guess 1 (k) y(k+1) = y(k) − J− R n R n (y ),

k = 0, 1 , . . . .

(13)

1 (k) The vector-increment x(k) = J− R n R n (y ) is obtained as the solution of the following ms × ms linear system

A ⊗ (h J n ) − I ms x(k) = R n y(k) . m× s

Let Rn : R

m× s

→R

(14)

be the residual matrix-mapping deﬁned by

Rn (Y ) = −Y + yn e T + hF (tn , Y ) AT , where the matrix-mapping F : [t 0 , T ] × Rm×s → Rm×s is derived from F n as follows

F (t , Y ) = f (t + c 1 h, y1 ), . . . , f (t + c s h, ys ) ∈ Rm×s . It is easy to see that for any vector y ∈ Rms and a matrix Y ∈ Rm×s with y = vec(Y ), we have R n ( y ) = vec(Rn (Y )). Let X (k) and Y (k) be the m × s matrices such that x(k) = vec( X (k) ) and y(k) = vec(Y (k) ). Therefore from the relations (13), the approximations Y (k) are derived from the iterations

Y (0) initial matrix guess Y (k+1) = Y (k) − X (k) , k = 0, 1, . . . , kmax,

(15)

where X (k) is the solution of the following k-th Stein Matrix equation obtained from the relation (2)

(h J n ) X (k) A T − X (k) = Rn Y (k) . Using again (2), the relation (7) can also be given as

yn+1 = yn + hF (tn , Y )b, where Y = Y (kmax) is obtained from the iterations (15). The computations are summarized in the following algorithm.

(16)

152

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

Algorithm 1. AT . – Choose a tolerance tol and an integer max. Set = 0 and B = – For n = 0, . . . , N 1. Choose an initial guess Y and a matrix Z = Y . Compute E = Y − Z and E F . 2. While < max and E F > tol do 2-1. Set Z = Y . 2-2. Set A n = h J n and C n = Rn (Y ). 2-3. Solve the Stein matrix equation A n X B − X = C n . 2-4. Compute Y = Y − X and put = + 1. 2-5. Compute the error E = Y − Z and the norm E F . End While. 3. Compute yn+1 = yn + hFn (Y )b. – End For. For small problems, the nonsymmetric Stein matrix equation, in Step 2.3 of Algorithm 1, can be solved by a direct method or by transforming this matrix equation to a linear system using the Kronecker product. For large problems, these two approaches could not be used. In this case, we will propose an iterative projection method for solving nonsymmetric Stein matrix equations. 3. A numerical method for solving large nonsymmetric Stein matrix equations In this section we will study and propose a numerical method for solving the following Stein matrix equation

AXB − X = C

(17)

where A ∈ Rm×m , B ∈ Rs×s , C ∈ Rm×s and X ∈ Rm×s . We assume here that the integer m is large and the integer s is of moderate size. The matrix equation (17) plays an important role in linear control and ﬁltering theory for discrete-time large-scale dynamical systems and other problems; see [9,10,25,14] and the references therein. They also appear in image restoration techniques [3,8] and in each step of Newton’s method for discrete-time algebraic Riccati equations [18]. Eq. (17) is also referred to as discrete Sylvester equation. When the matrices are not large, direct methods based on the Schur decomposition could be used for solving the matrix equation (17); see [2]. Notice also that by using (2), the matrix equation (17) can be formulated as the following ms × ms linear system of equations

A ⊗ B T − I ms vec( X ) = vec(C ).

(18)

Krylov subspace methods such as the GMRES algorithm [22] could be used to solve the linear system (18). The matrix equation (17) has a unique solution if and only if λi ( A )λ j ( B ) = 1 for all i = 1 . . . , m; j = 1, . . . , s where λi ( A ) is the i-th eigenvalue of the matrix A. This will be assumed through this paper. In particular, if ρ ( A )ρ ( B ) < 1 where ρ ( A ) denotes the spectral radius of the matrix A, Eq. (17) has a unique solution. Notice that this last condition is always satisﬁed if the step size h is small enough. We present here a Galerkin projection method based on the block Arnoldi algorithm [21,23]. We consider the case where the m × s matrix C is of full rank and s m. We also assume that the matrix B is nonsingular which is the case for Labatto IIIA and IIIB methods. 3.1. The block Arnoldi algorithm In this section, we recall the block Arnoldi process applied to the matrix A and starting with the m × s orthonormal matrix V 1 . The block Krylov subspace Kk ( A , V 1 ) = Range([ V 1 , A V 1 , . . . , A k−1 V 1 ]), is the subspace generated by the columns of the matrices V 1 , A V 1 , . . . , A k−1 V 1 . The block Arnoldi algorithm constructs the blocks V 1 , . . . , V k whose columns form an orthonormal basis of the block Krylov subspace Kk ( A , V 1 ). The algorithm is described as follows:

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

153

Algorithm 2 (The block Arnoldi algorithm). 1. Choose a unitary m × s matrix V 1 . 2. For j = 1, . . . , k W j = AV j, for i = 1, 2, . . . , j, H i , j = V iT W j , W j = W j − V j H i, j ,

end for. Q j R j = W j (QR decomposition), V j +1 = Q j ; H j +1, j = R j . End For. The blocks V 1 , . . . , V k constructed by Algorithm 1 have their columns mutually orthogonal provided that the upper triangular matrices H j +1, j are of maximum rank. If H j +1, j = 0 then K j is invariant under A. Notice that a beakdown could may occur if for some j, the block H j +1, j is nonzero and is singular. To cure this problem, one can use a sequential version (see [20]) of Algorithm 2 with a deﬂation procedure. k denotes the (k + 1)s × ks upper band-Hessenberg matrix whose nonzero entries h i , j , for i = 1, . . . , (k + 1)s and Let H k by deleting the last s-rows and H k+1,k j = 1, . . . , ks are deﬁned by Algorithm 1. Hk is the ks × ks matrix obtained from H is the s × s submatrix of the last s-rows and the last s-columns of Hk . The matrix Vk is deﬁned by Vk = [ V 1 , . . . , V k ] where V i , i = 1, . . . , k is the i-th block constructed by the block Arnoldi algorithm. From the block Arnoldi algorithm we can deduce the following relations

A Vk = Vk Hk + V k+1 H k+1,k E kT ;

k , A Vk = Vk+1 H

(19)

and

Hk = VkT A Vk

and

VkT Vk = I k ,

(20)

where E k is the matrix of the last s columns of the ks × ks identity matrix I ks . 3.2. The block Arnoldi algorithm for Stein equations In this section, we will apply the block Arnoldi algorithm to produce approximate solutions to the Stein matrix equation (17). The idea is to project the Stein equation (17) onto the block Krylov subspace and then solve, at each iteration, the obtained low-dimensional equation. Let A be the linear operator from Rm×s onto Rm×s deﬁned as follows

A : X −→ A( X ) = A X B − X .

(21)

Then the Stein equation (17) can be written as

A( X ) = C .

(22)

We will solve the problem (22) which is equivalent to the initial problem (17). Starting from an initial guess X 0 and the corresponding residual R 0 = C − A X 0 B + X 0 , the block Arnoldi Stein method constructs, at step k, the new approximation X k such that (i )

(i )

(i )

X k − X 0 = Z k ∈ Kk (A, R 0 );

i = 1, . . . , s ,

(23)

with the orthogonality relation (i )

R k ⊥ Kk (A, R 0 );

i = 1, . . . , s ,

(i )

(24) (i )

where R k is the i-th component of the residual R k = C − A( X k ) and X k We give the following result which is easy to prove [12].

is the i-th of column X k .

Theorem 3.1. Let A be the operator deﬁned by (21) and assume that R 0 is of full rank. Then

Kk (A, R 0 ) = Kk ( A , R 0 ). Using this last property, the relations (23) and (24) are written as (i )

(i )

(i )

X k − X 0 = Z k ∈ Kk ( A , R 0 ),

(25)

154

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

and (i )

R k ⊥ Kk ( A , R 0 );

i = 1, . . . , s .

(26)

Assume that R 0 is of rank s and let R 0 = V 1 U 1 be the Q R decomposition of R 0 where the m × s matrix V 1 is orthogonal and U 1 is s × s upper triangular. Now as the columns of the matrix Vk (constructed by the block Arnoldi algorithm) form a basis of the block Krylov subspace Kk ( A , R 0 ), the relation (25) implies that X k = X 0 + Vk Y k where Y k is a ks × s matrix. The orthogonality relation (26) implies that

VkT ( R 0 − A Vk Y k B + Vk Y k ) = 0. Hence, using (20), we ﬁnally obtain the low-dimensional Stein equation

Hk Y k B − Y k = C˜

(27)

with C˜ = E˜ 1 U 1 where E˜ 1 is the ks × s matrix whose upper s × s principal block is the identity matrix. The matrix equation (27) will be solved by using a direct method such as the Hessenberg–Schur method [10]. We assume that during the iterations λi (Hk )λ j ( B ) < 1 and this implies that Eq. (27) has a unique solution. Let us give now an expression of the residual norm that can be used to stop the iterations in the block-Arnoldi Stein algorithm without having to compute an extra product with the matrix A. Theorem 3.2. At step k, the norm of the residual R k is given by

R k F = H k+1,k E kT Y k B F = H k+1,k Y˜ k B F , where Y˜ k is the s × s matrix corresponding to the last s rows of the matrix Y k . Proof. At step k, the residual R k = C − A X k B + X k , with X k = X 0 + Vk Y k , is expressed as

R k = R 0 − A Vk Y k B + Vk Y k and from the relation A Vk = Vk Hk + V k+1 H k+1,k E kT , it follows that

R k = Vk [C˜ − Hk Y k B + Y k ] − V k+1 H k+1,k E kT Y k B . Therefore using (27) and the fact that the matrix V k+1 is orthogonal the result follows.

2

The next result shows that the approximate solution X k is an exact solution of a perturbed Stein matrix equation. Theorem 3.3. Assume that k steps of the block Arnoldi Stein method have been run and let X k be the obtained approximate solution to (17) given by X k = X 0 + Vk Y k , where Y k satisﬁes (27). Then X k is a solution of the perturbed problem

( A − Fk ) X B − X = C − Fk X0 B , with F k = V k+1 H k+1,k V kT and F k F = H k+1,k F . Proof. Multiplying on the left Eq. (27) by the matrix Vk we get

Vk Hk Y k B − Vk Y k = Vk C˜ . Using the relation A Vk = Vk Hk + V k+1 H k+1,k E kT and the fact that Vk is orthogonal it follows that

A Vk Y k B − V k+1 H k+1,k E kT VkT Vk Y k B − Vk Y k = Vk C˜ . Then as X k = X 0 + Vk Y k , Vk E k = V k and Vk C˜ = R 0 , we get

( A − F k ) Xk B − Xk = C − F k X 0 B where F k = V k+1 H k+1,k V kT and then F k F = H k+1,k F .

2

Note that when H k+1,k = 0, F k = 0 and hence X k is the exact solution of the Stein matrix equation (17). The blockArnoldi algorithm for solving (17) is summarized as follows:

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

155

Algorithm 3 (The block Arnoldi algorithm for Stein equations). 1. Choose a tolerance tol, an initial guess X 0 , an integer kmax and set k = 1. 2. Compute R = C + X 0 − A X 0 B and R 0 = V 1 U 1 : QR decomposition. 3. While k < kmax and R F > tol do – apply Algorithm 1 to the pair ( A , V 1 ) to generate the blocks V 1 , . . . , V k+1 and the block Hessenberg matrix Hk . – Solve by a direct method (the Schur method) the low-order Stein equation Hk Z B − Z = C˜ . – Compute X = X 0 + Vk Z . – Compute R F , by using Theorem 2. – Set k = k + 1. End While. To save CPU time and memory requirement, Algorithm 3 will be used in a restarted mode. This means that we restart the algorithm every k1 iterations were k1 is a ﬁxed integer. The restarted block Arnoldi Stein algorithm will be denoted by BlArnoldi–Stein(k1 ). 4. Numerical examples All the numerical experiments were performed in Matlab 7.0.4, on Windows XP system running on Intel(R) Core(TM) 2 Duo CPU 3.00 GHz with 3.23 GB RAM. In all our experiments, a maximum number of 20 iterations was allowed for both the block Arnoldi and for the Newton method. For the following tests, the fourth stage Implicit Runge–Kutta method (s = 4) of Kuntzmann–Butcher was implemented. This method is of order p = 2s = 8 with the following Butcher-tableau (see [15] and references therein)

ω1 ω1 − ω3 + ω4 ω1 − ω3 − ω4 ω1 − ω5 − ω2 ω1 − ω3 + ω4 ω1 ω1 − ω5 ω1 − ω3 − ω4 + ω2 ω1 + ω3 + ω4 ω1 + ω5 ω1 ω1 + ω3 − ω4 + ω2 ω1 + ω5 ω1 + ω3 + ω4 ω1 + ω3 − ω4 ω1 2ω1 2ω1 2ω1 2ω1

1 2 1 2 1 2 1 2

− ω2

where

ω1 =

ω1 =

1 8 1 8

√ −

144

√ +

30

30

144

ω2 =

,

ω2 =

,

2

35 15 − 2 30

2

35

,

ω3 = ω2

√

1

ω5 = ω2 − 2ω3 ,

√

15 + 2 30

1

,

1 6

ω3 = ω2

1 6

√ +

24

√ −

30

,

30 ,

24

ω4 = ω2

ω4 = ω2

1 21

1 21

√

+

5 30 168

√

−

5 30 168

, ,

ω5 = ω2 − 2ω3 .

Also, the third stage Implicit Runge–Kutta (RADAU-IIA) with (s = 3) was tested in Experiment 2.2 of Example 2. The 3-stage RADAU-IIA method is of order p = 2s − 1 = 5 and its Butcher tableau is given in [15] by

4−

√ ( 6)

10 4+

√

88 − 7 (6) 360

√ √ (6) 296 + 169 (6)

10 1

1800 16 −

√

(6)

36 16 −

√

36

√

1800

√

225

88 + 7 (6)

√ −2 − 3 (6)

360

225

16 +

√

(6)

36

(6)

√

296 − 169 (6) −2 + 3 (6)

16 +

√

1 9

(6)

36

1 9

4.1. Example 1: The nonlinear multi-mode problem The system (1) studied in our ﬁrst example is a nonlinear multi-mode problem which is given by

156

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

Table 1 Results for Experiment 1 and Experiment 2. Dimension

Experiment 1.1: Stiff ratio S R 1011

1000 2000 3000 6000 8000 10 000

Experiment 1.2: Stiff ratio S R 105

Time (sec)

Relative error

Time (sec)

Relative error

41 138 245 730 1230 2630

1.92e−08 8.46e−08 9.82e−08 6.92e−09 3.32e−07 5.45e−08

23 75 164 623 1116 1665

1.15e−10 1.25e−10 9.83e−11 1.61e−10 1.05e−10 1.24e−10

f : [t 0 , T ] × Rm → Rm , where the i-th component of f is given by

f i (t , y ) = g i (t ) + λi y i +1 y i − g i (t ) , f m (t , y ) = gm (t ) + λm ym − gm (t ) .

1 i m − 1,

The exact solution is y (t ) = g (t ) where the i-th component of g is given by

2π i

g i (t ) = a + cos t +

m

e −t .

(28)

For this example, we set t 0 = 0, T = 10 and N = 100 where N is the number of steps. The eigenvalues λi are chosen non-positive. The stiffness of the problem is due to the distribution of the eigenvalues λi , and the stiff ratio S R is given by

SR =

max1i m |λi | min1i m |λi |

.

Experiment 1.1. For this test, the eigenvalues are chosen as λ1 = λ2 = −109 and for i = 3, . . . , m, the eigenvalues λi are all equal to −0.01. In this case the stiffness phenomena appears with a stiff ratio S R = 1011 . Experiment 1.2. For our second numerical test, we set λ1 = λ2 = −103 and the other eigenvalues remain equal to −0.01. In this case the stiff ratio is S R = 105 and it is relatively smaller than the one in Experiment 1.1. In Table 1, we listed the results for Experiment 1 and Experiment 2. We used different values of the dimension m and we reported the relative error norms and the required CPU-time for each value of m. The relative error norm is given by Y − Y F /Y F where Y is the matrix whose columns are the exact vector-solutions and Y is the matrix whose columns are the computed approximate vector-solutions. For these two experiments, the Stein matrix equation in Step 2.3 of Algorithm 1 was solved by Block Arnoldi–Stein(k1 ) with k1 = 5 and the inner iterations were stopped when the relative norm of the residual was less than tol = 10−7 . 4.2. Example 2 In this example, the nonlinear function f : [t 0 , T ] × Rm → Rm , is given by f (t , y ) = ( f 1 (t , y ), f 2 (t , y ), . . . , f m (t , y )) T where

⎧ m 2 ⎪ ⎨ f (t , y ) = g (t ) − λ ( y − g (t )) + g j (t ) y j − g j (t ) , 1 i m − 1, i i i i i j = i +1 ⎪ ⎩ f m (t , y ) = gm (t ) − λm ym − gm (t ) .

(29)

The vector-valued function g is given by (28). The solution of the Cauchy problem (1) with the initial condition y 0 = g (0) is the one already given in Example 1: y (t ) = g (t ). The Jacobian is the triangular upper matrix deﬁned as

⎛

⎜ ⎜ ⎜ J f (t , y ) = ⎜ ⎜ ⎝

−λ1

p 1 (t )

.. . .. .

−λ2 .. . .. .

0

0

0

p 2 (t ) p 2 (t )

..

. .. . ...

⎞ . . . pm−1 (t ) . . . pm−1 (t ) ⎟ ⎟ .. .. ⎟ ⎟, . . ⎟ .. . pm−1 (t ) ⎠ 0 −λm

where p i (t ) = 2g i +1 (t )( y i +1 − g i +1 (t )) for i = 1, . . . , m − 1. For this experiment, we used different values of the size m (m = 800, m = 1200, m = 3000 and m = 7000). For this example, we set t 0 = 0, T = 10, the number of steps N = 100. The

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

157

Table 2 The obtained errors and the cpu-time (in parentheses) with Stein Block GMRES, GMRES and the Matlab’s backslash methods. Dimension

Stein block Arnoldi

GMRES

Direct

800 1200 1500 3000 7000

4.43 × 10−08 7.83 × 10−08 1.96 × 10−07 4.05 × 10−07 6.96 × 10−07

1.64 × 10−08 (511) 2.45 × 10−08 (1095)

1.64 × 10−08 (2210) 2.45 × 10−08 (6806)

(75) (178) (249) (746) (3587)

∗ ∗ ∗

∗ ∗ ∗

Table 3 Results for Method 1 and Method 2. Dimension

Kuntzmann–Butcher

RADAU-IIA

800 1200 1500 3000 7000

4.43 × 10−08 7.83 × 10−08 1.96 × 10−07 4.05 × 10−07 6.96 × 10−07

1.10 × 10−05 7.20 × 10−06 6.77 × 10−06 2.78 × 10−06 7.14 × 10−07

(75) (178) (248) (746) (3587)

(80) (181) (253) (783) (3720)

π ) + α for i = 1, . . . , m. The values of stepsize h is h = ( T − t 0 )/ N = 10−1 . The eigenvalues λi = (β − α ) cos2 ( im α = 107 and β = 1.2 × 107 .

α and β were

Experiment 2.1. We compared the results obtained with Stein Block GMRES, the GMRES for the linear system (14) and the direct Matlab’s backslash. The experiments are done with the 4-stage IRK Kuntzmann–Butcher method. In Table 2, we listed the obtained errors and in parentheses the corresponding times (in seconds). For m = 1500, and due to the large size (ms = 6000) of the linear system (14), the direct Matlab’s backslash was unable to solve this system. We notice that for large values of the dimension space m 3000, the GMRES and the direct Matlab’s backslash methods could not be run on our computer. This is due to the impossibility of forming the Kronecker product used in these last two methods. For the GMRES algorithm, we ﬁxed the maximum number of iterations to mmax = 100. In Table 2, the symbol ∗ means that the method doesn’t work on our computer (problem of storage). We notice that for large values of m, the error increases and this is due in part to the block Arnoldi. To remedy this problem, Experiment 2.2. In this experiment, we compared the performances of the fourth stage Implicit Runge–Kutta of Kuntzmann– Butcher (Method 1) and the third stage Implicit Runge–Kutta RADAU-IIA (Method 2) when both methods are implemented using our Newton-Block Arnoldi Stein algorithm (Algorithms 1, 2 and 3). In Table 3 we listed the obtained errors and the corresponding cpu-times required for convergence. 4.3. Example 3: Heat equation An important class of stiff problems with very large dimensions is derived from some partial differential equations. For example, consider the parabolic problem

⎧ ∂ 2 u (t , x) ∂ u (t , x) ⎪ ⎪ = c2 + g (t , x), (t , x) ∈ [t 0 , T ] × [α , β], ⎨ ∂t ∂ x2 u (t , α ) = u (t , β) = 0, t ∈ [t 0 , T ], ⎪ ⎪ ⎩ u (t 0 , x) = y 0 (x), x ∈ [α , β]. Let y i (t ) be the approximation of the exact values u (t , xi ), and replace

∂ 2 u (t ,xi ) with the approximation ∂ x2

∂ 2 u (t , xi ) u (t , xi + k) − 2u (t , xi ) + u (t , xi − k) /k2 , ∂ x2 where xi = xi −1 + k, the parameter k is the stepsize on the x-axis, k = (β − α )/(m + 1), m ∈ N, with x0 = α and xm+1 = β . The vector y (t ) = ( y 1 (t ), . . . , ym (t )) T is the exact solution of the following problem

with

y (t ) = f (t , y (t )), t ∈ [t 0 , T ], y (t 0 ) = y 0 ,

158

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

Fig. 1. Exact solution (left), computed function (right).

f (t , y ) = B y + g(t ),

(m + 1) B = c2 tridiag(1, −2, 1), (β − α )2 T g(t ) = g (t , x1 ), . . . , g (t , xm ) , 2

and the vector y0 = ( y 0 (x1 ), . . . , y 0 (xm )) T . B are The eigenvalues of the matrix

+ 1)2 iπ 2 , λi = −4c sin 2(m + 1) (β − α )2 2 (m

for i = 1, . . . , m. β−α

When m increases the stiff ratio also increases. As an example, we consider the case where c = π ,

g (t , x) = e −t sin

π (x − α ) sin(t − t 0 ), β −α

and y 0 (x) = 0. The exact solution is given by

π (x − α ) u (t , x) = e sin 1 − cos(t − t 0 ) . β −α −t

For the ﬁrst test, we set m = 4000, t 0 = 0, T = 1 and N = 100. In this case the stiff ratio is S R 1.6 × 106 and the obtained error is Y − Y F /Y F 6.84 × 10−8 , where Y = ( y i , j ) 1iN is the computed matrix and Y = (u (t i , x j )) 1iN is the exact 1 j m

1 j m

matrix. For the second test, we set m = 25 and N = 30. The stiff ratio is S R 3.8 × 102 and the error is Y − Y F /Y F 4.02 × 10−6 . The exact solution u (t , x) is given in the left side of Fig. 1 and the computed approximation is in the right of Fig. 1. 5. Conclusion In this paper, we proposed a numerical scheme for solving stiff ordinary differential equations in large dimensional spaces. We considered the one step implicit Runge–Kutta method. This method requires, at each integration step, the solution of a nonlinear system of equations. The Newton method was then applied to solve these nonlinear systems and this requires, at each iteration of Newton’s method, the computation of the solution of large special linear systems. Using some properties of the Kronecker product, we transform these linear systems to nonsymmetric Stein matrix equations. Then, we proposed a block Arnoldi method to approximate the solution of these matrix equations. We gave some theoretical results such as an eﬃcient and economical way for computing the residual norms without computing the approximation at each iteration and a perturbation result.

A. Bouhamidi, K. Jbilou / Applied Numerical Mathematics 61 (2011) 149–159

159

Acknowledgements The authors would like to thank the anonymous referees for their useful comments and helpful suggestions. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27]

R. Alexander, Diagonally implicit Runge–Kutta methods for stiff ODES, SIAM J. Numer. Anal. 14 (1977) 1006–1021. A.Y. Barraud, A numerical algorithm to solve A T X A − X = Q , IEEE Trans. Autom. Contr. AC-22 (1977) 883–885. A. Bouhamidi, K. Jbilou, Sylvester Tikhonov-regularization methods in image restoration, J. Comput. Appl. Math. 206 (1) (2007) 86–98. P.N. Brown, A.C. Hindmarsh, L.R. Petzold, Using Krylov methods in the solution of large-scale differential-algebraic systems, SIAM J. Sci. Comput. 15 (1994) 1467–1488. L. Brugnano, Trigiante, Solving Differential Problems by Multistep Initial and Boundary Value Methods, Gordon and Breach, Amsterdam, 1998. J.C. Butcher, The Numerical Analysis of Ordinary Differential Equations, Wiley, Chichester, 1987. J.C. Butcher, Diagonally implicit multistage integration methods, Appl. Numer. Math. 11 (1993) 347–363. D. Calvetti, L. Reichel, Application of ADI iterative methods to the restoration of noisy images, SIAM J. Matrix Anal. Appl. 17 (1996) 165–186. B.N. Datta, Krylov-subspace methods for large scale matrix problems in control, Gen. Comput. Syst. 19 (2003) 125–126. B.N. Datta, Numerical Methods for Linear Control Systems, Elsevier Academic Press, 2004. K. Dekker, Partitioned Krylov subspace iteration in implicit Runge–Kutta methods, Linear Algebra Appl. 431 (2009) 488–494. A. El Guennouni, K. Jbilou, A.J. Riquet, Block Krylov subspace methods for solving large Sylvester equations, Numer. Alg. 29 (2002) 75–96. C.W. Gear, Simultaneous numerical solutions of differential-algebraic equations, IEEE Trans. Circuit Theory CT-18 (1) (1971) 89–95. K. Glover, D.J.N. Limebeer, J.C. Doyle, E.M. Kasenally, M.G. Safonov, A characterisation of all solutions to the four block general distance problem, SIAM J. Control Optim. 29 (1991) 283–324. E. Hairer, S.P. Nørsett, G. Wanner, Solving Ordinary Differential Equations I. Nonstiff Problems, 2nd Revised Editions, Springer Series in Computational Mathematics, vol. 8, Springer-Verlag, Berlin, 1993. E. Hairer, G. Wanner, Solving Ordinary Differential Equations II. Stiff and Differential Algebraic Problems, 2nd Revised Editions, Comput. Math., vol. 14, Springer-Verlag, Berlin, 1996. L.O. Jay, Inexact simpliﬁed Newton iterations for implicit Runge–Kutta methods, SIAM J. Numer. Anal. 38 (2000) 1369–1388. P. Lancaster, L. Rodman, Algebraic Riccati Equations, Clarendon Press, Oxford, 1995. J.B. Rosser, A Runge–Kutta for all seasons, SIAM Rev. 9 (3) (1967) 417–452. A. Ruhe, Rational Krylov sequence methods for eigenvalue computations, Linear Algebra Appl. 58 (1984) 391–405. Y. Saad, Iterative Methods for Sparse Linear Systems, PWS Press, New York, 1995. Y. Saad, M.H. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statis. Comput. 7 (1986) 856–869. M. Sadkane, Block Arnoldi and Davidson methods for unsymmetric large eigenvalue problems, Numer. Math. 64 (1993) 687–706. P.J. Van Der Houwen, B.P. Sommeijer, N. Huu Cong, Parallel diagonally implicit Runge–Kutta–Nyström methods Source, Appl. Numer. Math. Arch. 9 (2) (1992) 111–131. P. Van Dooren, Gramian based model reduction of large-scale dynamical systems, in: Numerical Analysis, Chapman and Hall, CRC Press, London, 2000, pp. 231–247. D. Voss, S. Abbas, Block predictor–corrector schemes for the parallel solution of ODEs, Comp. Math. Appl. 33 (1997) 65–72. D. Voss, P.H. Muir, Mono-implicit Runge–Kutta schemes for of initial value ODES the parallel solution, J. Comp. Appl. Math. 102 (1999) 235–252.

Applied Numerical Mathematics 61 (2011) 160–169

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

Economical Runge–Kutta methods with strong global order one for stochastic differential equations F. Costabile, A. Napoli ∗ Department of Mathematics, University of Calabria, 87036 Rende (Cs), Italy

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 22 July 2009 Received in revised form 22 June 2010 Accepted 2 September 2010 Available online 6 September 2010

Economical Runge–Kutta schemes for the numerical solution of Stratonovich stochastic differential equations are proposed. The methods have strong global order 1. Numerical stability is studied and some examples are presented to support the theoretical results. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

Keywords: Stochastic differential equations Stochastic Taylor expansion Mean-square stability

1. Introduction Stochastic differential equations (SDEs) are used for the description of many real-life phenomena in different ﬁelds, including biology and physics, population dynamics, economics and ﬁnance. In fact if more realistic models are requested, stochastic effects need to be taken into account. Unfortunately, in many cases analytic solutions of SDEs are not available, thus numerical methods are needed to approximate them. In this paper we consider the scalar autonomous Stratonovich SDE [8]

dy (t ) = a y (t ) dt + b y (t ) ◦ dW t ,

t0 t T ,

y (t 0 ) = y 0 ,

(1)

where W = { W t , 0 t T } denotes a standard Wiener process. The functions a and b are the drift and the diffusion coeﬃcients respectively, and we assume that they are deﬁned and measurable in [t 0 , T ] × R and satisfy both Lipschitz and linear growth bound conditions in y. These requirements ensure existence and uniqueness of solution of the SDE (1). The integral formulation of (1) can be written as

t y (t ) = y 0 + t0

a y (s) ds +

t

b y (s) ◦ dW (s),

(2)

t0

where the ﬁrst integral is a regular Riemann–Stieltjes integral and the second one is a Stratonovich stochastic integral with respect to the Wiener process W (t ).

*

Corresponding author. E-mail addresses: [email protected] (F. Costabile), [email protected] (A. Napoli).

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.09.001

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

161

By truncating the stochastic Taylor expansion for the process (2), different numerical methods can be constructed [8]. But the computational cost can be high due to the proliferation of elementary derivatives. In order to derive derivative-free methods, the extension of classical Runge–Kutta methods to stochastic differential equations has been introduced (see [8], which includes also an extensive bibliography). In [3] the authors gave an overview of the methods of Runge–Kutta type for SDEs studied until then. For deterministic equations it is known that the classical Runge–Kutta methods are expensive in terms of function evaluations. In order to reduce the number of function evaluations, the so-called economical Runge–Kutta methods have been proposed (see [5] and the references therein). In [6] the authors extended the idea of deterministic economical Runge–Kutta methods to the solution of SDEs, by constructing a weak second order Runge–Kutta type method. In this paper we propose some classes of explicit three-stage economical Runge–Kutta methods with strong global order 1. One of these methods can be considered as the corresponding economical version of the so-called EM2, presented by Soheili and Namjoo in [9]. Although the presented formulas are related to scalar SDEs, they apply to multidimensional SDEs. In order to facilitate the reading of the work, in Section 2 we quote deterministic economical Runge–Kutta methods and in Section 3 stochastic Runge–Kutta methods. In Section 4 we propose new families of methods. Numerical stability for the proposed schemes is studied in Section 5, where the domains of stability are obtained and shown in some ﬁgures. In the last section numerical examples are given which compare the proposed method to EM2 presented in [9] and conﬁrm the theoretical results. 2. Deterministic economical Runge–Kutta scheme A deterministic explicit s-stage Runge–Kutta method for the numerical solution of the differential equation y (t ) = f (t , y ) with initial condition y (t 0 ) = y 0 is

⎧ ⎪ ⎨ ⎪ ⎩

y n +1 = y n + h n

s

n = 0, 1 , . . . , N − 1 ,

b i K i ,n ,

(3)

i =1

y 0 = y (t 0 ),

where b i ∈ R, s is the number of stages, hn = tn+1 − tn , and

K i ,n = f tn + c i hn , yn + hn

i −1

ai j K j ,n ,

c 1 = 0.

(4)

j =1

Usually c i satisfy the condition c i =

i −1

j =1 ai j ,

i = 1, . . . , s.

In the equivalent form, Eqs. (3) and (4) can be written as

⎧ i −1 ⎪ ⎪ ⎪ ⎪ Y in = yn + hn ai j f tn + c j hn , Y nj , ⎪ ⎨ j =1

s ⎪ ⎪ ⎪ ⎪ y = y + h b i f tn + c i hn , Y in , n + 1 n n ⎪ ⎩

(5) n = 0, 1 , . . . , N − 1 .

i =1

If a Runge–Kutta method with order p 3 is such that b1 = 0, c s = 1, then it is said to be of class A p [5]. For a method belonging to class A p , in [5] the authors proposed the corresponding economical method

⎧ ⎪ ⎨ ⎪ ⎩

y n +1 = y n + h n y 0 = y (t 0 ),

where

s

b i K i ,n ,

n = 0, 1 , . . . , N − 1 ,

(6)

i =2

K i ,n = f tn + c i hn , yn + hn K s,−1 = K 1,0 = f (t 0 , y 0 )

i −1

ai j K j ,n + ai1 K s,n−1

,

j =2

(7)

and showed that the order does not change. In the equivalent form the method is

⎧ i −1 ⎪ ⎪ n ⎪ Y = y + h ai j f tn + c j hn , Y nj + hn ai1 f tn + c 1 hn , Y sn−1 , ⎪ n n ⎨ i j =2

s ⎪ ⎪ ⎪ y = y + h b i f tn + c i hn , Y in , ⎪ n n ⎩ n +1 i =2

Y −1 s

=

Y 10

= y0 .

(8) n = 0, 1 , . . . , N − 1 ,

162

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

3. Stochastic Runge–Kutta schemes The general form of an s-stage stochastic Runge–Kutta scheme for the solution of (1), in the case of one Wiener process, is given by

⎧ s s ⎪ ⎪ ( 0) (1 ) ⎪ Y = y + Z a ( Y ) + Z i j b(Y j ), ⎪ n i j ij ⎪ ⎨ j =1

i = 1, 2, . . . , s ,

j =1

(9)

s s ⎪ ⎪ ( 0) (1 ) ⎪ ⎪ y = y + z a ( Y ) + z j b(Y j ), n + 1 n j ⎪ j ⎩ j =1

j =1

where Z (1) and z(1) are respectively arbitrary matrix and vector whose elements are random variables and Z (0) and z(0) are respectively the parameter matrix and vector associated with the deterministic components. If both Z (0) and Z (1) are strictly lower triangular then (9) is said to be explicit, otherwise it is implicit. The scheme (9) can be represented in the tableau form Z (0) T z(0)

Z (1) z(1)

T

In the area of SDEs there are two ways to measure accuracy: strong convergence and weak convergence. In the ﬁrst case sample path trajectories of the numerical approximation are calculated; in the case of weak approximation only some of the moments may be of interest. Of course, for problems involving direct simulation it is important to determine trajectories of the approximation which are close to the exact solution. t −t

Deﬁnition. (See [8].) If y N is the numerical approximation of y (t N ) after N steps with constant stepsize h = N N 0 , then y is said to converge strongly to y with strong global order p if there exist k > 0 (independent of h) and δ > 0 such that for the mean value E (| y N − y (t N )|) the following relation holds

E y N − y (t N ) kh p ,

h ∈ (0, δ).

We remember that, if X is a random variable deﬁned on the probability space (Ω, A , P ), the expected value or mean value of X is

E(X) =

X dP. Ω

Note that this deﬁnition is for global order. The equivalent of local order means that a method of order p would have 1

local error which behaves like O (h p + 2 ). Often in literature there is no distinction between local and global order. Thus, for example, the same method in [1] and [4] is claimed to have different orders. In [2] the authors found a relationship between global and local order. In what follows we refer to the convergence theorem in [2]. Analogously with the deterministic case, the technique for obtaining the order conditions consists in matching the truncated Runge–Kutta scheme with the stochastic Taylor series expansion of the exact solution over one step assuming exact initial values (see, for example, [8]). By successive applications of Itô formula, the Stratonovich Taylor expansion of (1) is given by

y (t ) = y 0 + a J 0 + b J 1 + aa J 00 + ab J 01 + ba J 10 + bb J 11

2 + a aa + a J 000 + a ab + a b J 001 + a a b + a b J 010 2 2 + b aa + a J 100 + a bb + b J 011 + b ab + a b J 101 2 + b a b + a b J 110 + b bb + b J 111 + R ,

where R is the remainder term and J j 1 , j 2 ,..., jk is the Stratonovich multiple integral where integration is with respect to ds if j i = 0 or ◦ dW (s) if j i = 1. Thus, for example,

t s2 s J 101 =

◦ dW (s1 ) ds ◦ dW (s2 ). t0 t0 t0

In [1] the development of order conditions via tree theory allows the construction of more general Runge–Kutta schemes.

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

163

A class of SRK methods given by (9) can be characterized by T

Z ( 0) = h A ,

z ( 0) = h α T ,

T

Z (1 ) = J 1 B ,

z (1 ) = J 1 γ T ,

(10)

where A = (ai j ) and B = (b i j ) are s × s real matrices, while α T = (α1 , . . . , αs ) and γ T = (γ1 , . . . , γs ) are row vectors in Rs . In [9] a class of 3-stage explicit strong global order 1 SRK methods with (10) is constructed. 4. Economical stochastic Runge–Kutta methods As in the deterministic case [5], if we save one function call for each step by using information from the previous step, (0) we obtain an economical Runge–Kutta method for SDEs. Let us consider 3-stage SRK methods such that z1 = 0. The following scheme

⎧ i −1 i −1 ⎪ ⎪ ( 0) n ( 0) n −1 (1 ) n ⎪ + + Y = y + Z a Y Z a Y Z i j b Y nj , ⎪ n s j i j i1 i ⎪ ⎨ j =2

i = 1, 2, . . . , s ,

j =1

(11)

s s ⎪ ⎪ ⎪ ( 0) (1 ) ⎪ ⎪ zi a Y in + zi b Y in ⎩ y n +1 = y n + i =2

i =1

is an explicit economical Runge–Kutta type method (EcSRK in what follows) for SDEs. It can be also written in the form

⎧

i −1 i −1 ⎪ ⎪ ( 0) n ( 0) n −1 (1 ) n ⎪ n ⎪ K i = a yn + Z i j K j + Z i1 K s + Z i j K j , i = 1, 2, . . . , s , ⎪ ⎪ ⎪ ⎪ j =2 j =1 ⎪ ⎪ ⎪

⎪ i −1 i −1 ⎨ n ( 0) n ( 0) n −1 (1 ) n K i = b yn + Z i j K j + Z i1 K s + Z i j K j , i = 1, 2, . . . , s , ⎪ ⎪ ⎪ j =2 j =1 ⎪ ⎪ ⎪ ⎪ s s ⎪ ⎪ ( 0) n (1 ) n ⎪ ⎪ y = y + z K + zi K i . n ⎪ i i ⎩ n +1 i =2

(12)

i =1

Let us consider the EcSRK method (11) with Z (0) , Z (1) , z(0) and z(1) deﬁned as in (10) and the Stratonovich Taylor expansion of (11)

yn+1 = yn + ha(α2 + α3 ) + J 1 b(γ1 + γ2 + γ3 ) + h J 1

α1 = 0. Let’s now compare

γ2a21 + γ3 (a31 + a32 ) ab

+ J 1 h α2 b21 + α3 (b31 + b32 ) a b + J 12 γ2 b21 + γ3 (b31 + b32 ) bb b b2 2 + J 13 γ2 b221 + γ3 (b31 + b32 )2 + J 13 γ3 b32 b21 b b + · · · 2

with the expansion of the exact solution

2

y (t ) = y 0 + a J 0 + b J 1 + aa J 00 + ab J 01 + ba J 10 + bb J 11 + b bb + b

J 111 + · · · .

Taking into account that (see [8]) for the multiple Stratonovich integrals the following formulae hold

J 00 =

h2 2

J 10 = J 1 h − J 01 ,

,

J 11 =

1 2

J 12 ,

J 111 =

1 6

J 13 ,

the method has strong global order 1 if

⎧ α + α = 1, 2 3 ⎪ ⎪ ⎪ ⎪ γ + γ 1 2 + γ3 = 1, ⎪ ⎪ ⎪ ⎪ 1 ⎪ ⎪ γ2a21 + γ3 (a31 + a32 ) = , ⎪ ⎪ ⎪ 2 ⎪ ⎪ ⎪ 1 ⎪ ⎪ ⎨ α2 b21 + α3 (b31 + b32 ) = , 2

⎪ 1 ⎪ ⎪ ⎪ ⎪ γ2 b21 + γ3 (b31 + b32 ) = 2 , ⎪ ⎪ ⎪ ⎪ 1 ⎪ ⎪ γ3 b32 b21 = , ⎪ ⎪ ⎪ 6 ⎪ ⎪ ⎪ ⎪ ⎩ γ b2 + γ (b + b )2 = 1 . 2 21

3

31

32

3

(13)

164

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

In fact, if the seven equations with eleven unknowns (13) are satisﬁed, then both the mean square conditions (strong local order 1) and the mean conditions (mean local order 1)

2 12

E L (t )

= O h 1 .5 ,

E L (t ) = O h2

hold, where L (t ) = y (t ) − yn+1 is the local truncation error (see [2]). The solutions of system (13) are:

• The four-parameter families if b32 = 41γ3 , γ3 = 34 , 48b232 γ3 − 12b32 − 1 0, L=

γ32 48b232 γ3 − 12b32 + 1 .

Family A1.

⎧ − L + γ3 (1 + 3b32 (8b32 γ3 − 2) + L ) (3b32 γ3 − 1)( L − γ3 ) ⎪ ⎪ , , α3 = ⎪ α2 = ⎪ 6b γ ( 4b γ − 1 ) 6b32 γ3 (4b32 γ3 − 1) ⎪ 32 3 32 3 ⎪ ⎪ ⎪ ⎪ 1 1 ⎪ ⎪ γ2 = (−1 + 6b32 )γ3 + L , ⎪ ⎨ γ1 = 2 − (1 + 6b32 )γ3 − L , 2

2

(14)

(1 − 2(a31 + a32 )γ3 )(γ3 − 6b32 γ3 + L ) ⎪ a21 = , ⎪ ⎪ ⎪ 12b232 γ32 (4γ3 − 3) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ γ3 − 12b232 γ32 − L 1 ⎪ ⎪ , b31 = . ⎩ b21 = 2 6b32 γ3

12b32 γ3

Family A2.

⎧ L + γ3 (1 + 24b232 γ3 − 3b32 (2 + γ3 − L )) (3b32 γ3 − 1)( L + γ3 ) ⎪ ⎪ , , α3 = ⎪ α2 = ⎪ ⎪ 6b γ ( 4b γ − 1 ) 6b32 γ3 (4b32 γ3 − 1) 32 3 32 3 ⎪ ⎪ ⎪ ⎪ 1 1 ⎪ ⎪ γ2 = (−1 + 6b32 )γ3 − L , ⎪ ⎨ γ1 = 2 − (1 + 6b32 )γ3 + L , 2

2

(15)

(2(a31 + a32 )γ3 − 1)(γ3 (6b32 − 1) + L ) ⎪ ⎪ a21 = , ⎪ ⎪ 12b232 γ32 (4γ3 − 3) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ γ3 − 12b232 γ32 + L 1 ⎪ ⎪ , b31 = . ⎩ b21 = 2 6b32 γ3

12b32 γ3

• The three-parameter families if γ3 = 34 . Family B1. (If b32 = 41γ .) 3

⎧ 1 ⎪ ⎪ a21 = a21 , ⎨ γ1 = 4 , ⎪ 1 1 ⎪ ⎩ α3 = , 3+ 4 1 − 3b32

a31 =

2 3

− a32 ,

γ 2 = 0,

α2 =

b31 =

2 3

3b32 12b32 − 4

− b32 ,

,

b21 =

(16)

2 9b32

.

Family B2.

⎧ 9b32 ⎪ ⎪ , ⎨ γ1 = 1 − ⎪ 3 1 ⎪ ⎩ α3 = − + , 4

2

a31 =

2

3

− a32 + a21 (1 − 6b32 ), 3

γ2 = (6b32 − 1),

3b32

b31 = −

4

• The three-parameter family if b32 =

α2 = 2 3

+

7

4 2

9b32

−

1 3b32

− b32 ,

, b21 =

2 9b32

(17)

.

1 . 4γ3

Family C.

⎧ 1 ⎪ ⎪ ⎨ γ1 = − γ3 , 4 1

⎪ ⎪ ⎩ α3 = , 4

a21 = 3

γ2 = , 4

2 3

4

− (a31 + a32 )γ3 , 3

b31 = −

1 4γ 3

Thus the following theorem has been proved:

,

b21 =

3

α2 = , 4

2 3

.

(18)

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

165

Theorem 1. The EcSRK method (11) with Z (0) , Z (1) , z(0) and z(1) deﬁned as in (10) and α1 = 0 has strong global order 1 if one of the set of relations (14), (15), (16), (17), (18) is satisﬁed. Remark 1. The order of the economical method is the same of that of the classical method. In [9] the authors proposed a strong global order 1 method, called EM2, which corresponds to the following tableau: 0 2 h 3 −h

0

0

0 2 J1 3 − J1

0

0

0

h 3 h 4

0 1 h 4

0

0

0

0

0

J1 3 J1 4

0 1 J1 4

The corresponding EcSRK method, which we call EcEM2, can be obtained from (18) with

⎧ n Y = yn , ⎪ ⎪ ⎪ 1 ⎪ 2 ⎪ 2 ⎪ ⎪ Y 2n = yn + ha Y 3n−1 + J 1 b Y 1n , ⎨ 3 3 ⎪ Y 3n = yn − ha Y 3n−1 + ha Y 2n − J 1 b Y 1n + J 1 b Y 2n , ⎪ ⎪ ⎪ ⎪ 1 ⎪ 3 1 3 ⎪ ⎩ yn+1 = yn + ha Y 2n + ha Y 3n + J 1 b Y 2n + J 1 b Y 3n . 4

4

4

γ3 = 14 , a31 = −1, a32 = 1,

(19)

4

From Theorem 1 EcEM2 also has strong global order 1. Other economical Runge–Kutta type methods with strong global order 1 can be obtained from (14)–(18). For example, from (17) if γ3 = 34 , b32 = 23 , a21 = 0, a32 = 14 , an eﬃcient method is derived, which we call EcCN

⎧ n Y 1 = yn , ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ Y n = y n + 1 J 1 b Y n , ⎪ ⎪ 1 ⎨ 2 3 1 1 2 5 ⎪ Y 3n = yn + ha Y 3n−1 + ha Y 2n − J 1 b Y 1n + J 1 b Y 2n , ⎪ ⎪ 12 4 4 3 ⎪ ⎪ ⎪ ⎪ n 1 n n 9 5 ⎪ ⎩ yn+1 = yn + ha Y − ha Y − 2b Y + J 1 b Y n + 3 J 1 b Y n . 2 3 1 2 3 4

4

4

(20)

4

5. Stability regions In order to study the stability regions for EcSRK methods we consider the Itô test equation

dy = λ y dt + μ y dW t ,

t > t 0 , λ, μ ∈ C,

(21)

with nonrandom initial conditions y (t 0 ) = y 0 ∈ R, y 0 = 0. Deﬁnition. A numerical solution { yn }n∈N generated by a scheme with equidistant stepsize applied to the test equation (21) is called asymptotically mean square stable if limn→∞ E [| yn |2 ] = 0. When we apply to (21) the scheme (19) in the form (12) we obtain the difference equation

yn+1 = P yn + Q K 3n−1 ,

(22)

K 3n = L yn + M K 3n−1 with K 3−1 = 0 and

⎧ J 12 μ2 J 13 μ3 h 2 λ2 3 1 2 1 2 ⎪ 2 2 ⎪ ⎪ P = 1 + h λ + + J μ + λ μ + J λ μ + + λ μ + , h J h h J 1 1 1 ⎪ ⎪ 4 4 6 2 3 1 6 ⎪ ⎪ ⎪ ⎪ h2 λ h 3 λ2 h J 1μ 1 2 1 2 2 ⎪ ⎪ ⎪ ⎨ Q = 4 + 6 + 4 + 3 h J 1 λμ + 6 h J 1 μ , 2 J 2 μ2 ⎪ 2 ⎪ ⎪ L = λ 1 + h λ + h J 1 λμ + 1 , ⎪ ⎪ 3 3 ⎪ ⎪ ⎪ ⎪ ⎪ 2h2 λ 2h J 1 μ ⎪ ⎪ ⎩ M = λ −h + + . 3

3

(23)

166

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

If we put

un = yn , K 3n−1

T

,

Eq. (22) takes the form

un+1 = Aun , where

A=

P L

Q . M

When we calculate the components of the second moment of un , the following one-step difference equation is obtained:

X n +1 = Ω X n , where

⎡ Xn = ⎣

X n1

⎡

⎤

X n2 ⎦ X n3

=

⎤

E (un1 )2

⎣ E (un2 )2 ⎦ E (un1 un2 )2

and Ω is the stability matrix. Under the pth matrix norm · p it is evident that limn→∞ X n = 0 if Ω p < 1. The entries of the stability matrix Ω can be determined by direct computation. Theorem 2. When method (12) is applied to the test equation (21), the stability matrix Ω is given by

⎤ Ω11 Ω12 Ω13 Ω = ⎣ Ω21 Ω22 Ω23 ⎦ , Ω31 Ω32 Ω33 ⎡

where

Ω11 = 1 +

1

h5 λ4 μ2 + 2h λ + μ2 +

36 h 6 λ4 h3 μ2

1

5 2

1 6

h2 λ 9λ + 19μ2 +

2

1

4

1 48 2

h4 λ3 3λ + 20μ2 +

1 16

+ + h λ λ + 2μ + h λ λ + 4μ , 36 16 12 16 1 2 1 5 3 1 1 2 2 Ω13 = h 3λ + 5μ + h λ 3λ + 8μ + h4 λ2 11λ + 24μ2 + h3 λ 20λ + 39μ2 , 6 36 24 24 1 4 2 Ω21 = λ2 + h3 λ4 μ2 + hλ2 3λ + 2μ2 + h2 λ3 3λ + 4μ2 , 9 3 3 4 4 4 4 3 2 2 2 2 Ω22 = h λ + h λ − h λ 3λ − μ , 9 9 4 2 2 2 2 Ω23 = −2hλ − h λ λ + 2μ2 + h3 λ3 3λ + 4μ2 , 3 9 1 1 1 1 Ω31 = λ + h4 λ4 μ2 + h3 λ3 λ + 4μ2 + hλ 12λ + 7μ2 + h2 λ2 15λ + 26μ2 , 9 4 6 12 1 3 2 1 5 4 1 4 2 2 Ω32 = − h λ + h λ + h λ μ , 4 9 3 1 1 1 2 Ω33 = −hλ − h λ λ − 4μ2 + h4 λ3 λ + 2μ2 + h3 λ2 5λ + 6μ2 . 12 3 6 Ω12 =

The scheme is stable in the mean square sense with respect to · ∞ if

max{ A , B , C } < 1, where

A = |Ω11 | + |Ω12 | + |Ω13 |, B = |Ω21 | + |Ω22 | + |Ω23 |, C = |Ω31 | + |Ω32 | + |Ω33 | in which Ωi j are as given in Theorem 2.

h3 λ2 8λ + 29μ2 ,

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

167

Fig. 1. Mean square stability regions of EcEM2 method.

Fig. 2. Mean square stability regions of EcCN method.

If we restrict attention to λ, μ ∈ R, the region of mean square stability of the scheme (19) can be obtained for several values of h (Fig. 1). With the same technique used above, the regions of mean square stability of scheme EcCN (20) for several values of h are shown in Fig. 2. 6. Numerical results In this section numerical results from the implementation of EcEM2 and EcCN proposed in this paper are compared to those from the implementation of EM2 method [9]. The three methods will be implemented with constant step size on three problems for which the exact solution in terms of a Wiener process is known. In Section 4 we said that in scheme (9) at each step we use a function call of the previous step. Therefore the cost of both EcEM2 and EcCN is less than the cost of EM2 method. All the computations were done on a PC with Core 2 processor using Matlab 7.0 and 1000 independent simulations 1 1 1 1 1 , 50 , 100 , 200 , 400 . In order to simulate the Gaussian variable J 1 (500 in Example 3) were generated for stepsizes h = 25 with distribution N (0, h) the Matlab random number generator randn is used. For each step size the same Brownian path is followed. The implementation determines the average error for each stepsize h at the end of the interval of integration. This error and the computational work (the number of function evaluation nfc) for each problem are summarized in Tables 1, 2 and 3.

168

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

Table 1 Error and computational work in the approximation of (21). h

EM2

Economical methods

Error 1 25 1 50 1 100 1 200 1 400

nfc

EcEM2 error

EcCN error

nfc

7.24908e−3

294 000

7.33767e−3

3.38547e−4

3.72861e−3

594 000

3.74273e−3

1.19079e−4

495 001

1.84692e−3

1 194 000

1.85186e−3

4.29900e−5

995 001

9.45799e−4

2 394 000

9.47236e−4

1.62784e−5

1 995 001

4.55096e−4

4 794 000

4.55308e−4

5.38715e−6

3 995 001

245 001

Table 2 Error and computational work in the approximation of (24). h

EM2

Economical methods

Error 1 25 1 50 1 100 1 200 1 400

nfc

EcEM2 error

EcCN error

nfc

7.54615e−2

144 000

7.51431e−2

7.23452e−2

120 001

2.94867e−2

294 000

2.94685e−2

3.88896e−2

245 001

9.64905e−3

594 000

9.67940e−3

9.33214e−3

495 001

4.07348e−3

1 194 000

4.04165e−3

4.94036e−3

995 001

1.75723e−3

2 394 000

1.76208e−3

2.73337e−3

1 995 001

Example 1. The ﬁrst test equation is the linear test equation (21) which is used to check the stability properties of SRK methods. Its exact solution is

y (t ) = y 0 exp

1 λ − μ2 t + μ W (t ) 2

which is sometimes called Brownian motion. Table 1 lists the results for λ = −0.8, μ = −0.5, at t = 2. Example 2. Consider the SDE [9]

dy = −α 1 − y 2 dt + β 1 − y 2 ◦ dW ,

t ∈ [0, 1],

y (0) = 0

(24)

with the solution

y (t ) = For

(1 + y 0 ) exp(−2αt + 2β W (t )) + y 0 − 1 . (1 + y 0 ) exp(−2αt + 2β W (t )) + 1 − y 0

α = 1.0, β = 2.0 the results are given in Table 2.

Example 3. This two-dimensional SDE is taken from [7]

1 dy 1 = − β 4 y 2 dt + β 2 y 2 dW , 2 1 dy 2 = − y 1 dt + y 1 dW . 2

(25)

For the initial value [0, 2], the actual solution is

y 1 (t ) = −β exp −β W (t ) + β exp β W (t ) ,

y 2 (t ) = exp −β W (t ) + exp β W (t ) . Here we used 500 independent simulations. If t ∈ [0, 1], for β = 0.4 the results at t = 1 are given in Table 3.

F. Costabile, A. Napoli / Applied Numerical Mathematics 61 (2011) 160–169

169

Table 3 Error and computational work in the approximation of (25). h

EM2

EcEM2

Error 1 25 1 50 1 100 1 200 1 400

nfc

Error

nfc

6.23434e−2

144 000

6.48433e−2

120 002

5.93177e−2

294 000

6.49988e−2

245 002

6.54008e−2

594 000

6.39327e−2

495 002

6.47541e−2

1 194 000

6.47382e−2

995 002

6.29907e−2

2 394 000

6.38862e−2

1 995 002

Table 4 Error and computational work in the approximation of (26). h

EcEM2 Error

1 25 1 50 1 100 1 200 1 400

nfc

8.05177e−1

192 001

7.83249e−1

392 001

7.37925e−1

791 001

7.03005e−1

1 592 001

7.06341e−1

3 192 001

Remark 2. We observe that the entire analysis here presented is relating to SDEs with one Wiener process. The straightforward extension of EcEM2 method to two Wiener processes, in the actual formulation, performs poorly. In fact, if, for example, we consider the following SDE

3 dy = − y dt + y ◦ dW 1 + y ◦ dW 2 , 2 y (0) = 1

(26)

with the solution

3 y (t ) = exp − t + W 1 (t ) + W 2 (t ) y 0 2

we have the results in Table 4. 7. Conclusions and future work In this paper we have extended the idea of deterministic economical Runge–Kutta methods to the solution of SDEs, by constructing some classes of explicit three-stage economical Runge–Kutta methods with strong global order 1. As we can see from the above examples, the proposed methods are less expensive in terms of function evaluations than the classical ones. Moreover, the new EcCN method in some cases is also more eﬃcient than the existing EM2 as far as the magnitude of the error is concerned, in the case of one Wiener process. The extension of the proposed method to SDEs with two Wiener processes, doesn’t work well, as Remark 2 shows. For this reason, it is necessary to look for methods that will be better suited for problems with multi-Wiener processes. This extension is under study. Moreover, it is left for future work to determine other economical stochastic Runge–Kutta methods with order greater than one. References [1] K. Burrage, P.M. Burrage, High strong order explicit Runge–Kutta methods for stochastic ordinary differential equations, Appl. Numer. Math. 22 (1996) 81–101. [2] K. Burrage, P.M. Burrage, Order conditions of stochastic Runge–Kutta methods by B-series, SIAM J. Numer. Anal. 38 (5) (2000) 1626–1646. [3] K. Burrage, E. Platen, Runge–Kutta methods for stochastic differential equations, Ann. Numer. Math. 1 (1994) 63–78. [4] P.M. Burrage, Runge–Kutta methods for stochastic differential equations, PhD thesis, Dept. Maths., University of Queensland, Australia, 1999. [5] F. Costabile, R. Caira, M.I. Gualtieri, Economical Runge–Kutta method, Rend. Mat. 15 (VII) (1995) 57–77. [6] F. Costabile, A. Napoli, Economical Runge–Kutta methods for numerical solution of stochastic differential equations, BIT 48 (3) (2008) 499–509. [7] J.G. Gaines, T.J. Lyons, Variable step size control in the numerical solution of stochastic differential equations, SIAM J. Appl. Math. 57 (5) (1997) 1455– 1484. [8] P.E. Kloeden, E. Platen, Numerical Solution of Stochastic Differential Equations, Springer, Berlin, 1992. [9] A.R. Soheili, M. Namjoo, Strong approximation of stochastic differential equations with Runge–Kutta methods, World J. Modell. Simul. 4 (2008) 83–93.

Applied Numerical Mathematics 61 (2011) 170–180

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

MLPG method for two-dimensional diffusion equation with Neumann’s and non-classical boundary conditions S. Abbasbandy ∗ , A. Shirzadi Department of Mathematics, Imam Khomeini International University, Ghazvin 34149, Iran

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 24 June 2010 Received in revised form 29 August 2010 Accepted 10 September 2010 Available online 17 September 2010 Keywords: MLPG method Neumann’s boundary conditions Non-classical integral boundary condition Finite differences Heat equation Parabolic partial differential equations

In this paper, a meshless local Petrov–Galerkin (MLPG) method is presented to treat parabolic partial differential equations with Neumann’s and non-classical boundary conditions. A diﬃculty in implementing the MLPG method is imposing boundary conditions. To overcome this diﬃculty, two new techniques are presented to use on square domains. These techniques are based on the ﬁnite differences and the Moving Least Squares (MLS) approximations. Non-classical integral boundary condition is approximated using Simpson’s composite numerical integration rule and the MLS approximation. Two test problems are presented to verify the eﬃciency and accuracy of the method. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

1. Introduction The MLPG method was ﬁrst proposed by S.N. Atluri and T.L. Zhu [5,6] in 1998. Because of great freedom in choosing test function, there exist several kind of MLPG methods [4]. The method has been developed and explained with more details in two books [2,3]. The method is based on the local weak form and the moving least squares (MLS) approximation, and so is a truly numerical meshless method for solving boundary value problems. The main advantage of this method is that it does not need a “mesh”, either for purposes of interpolation or integration. In spite of great beneﬁts in using MLPG method, there are also some demerits arising in implementation of the method. For example, the nature of complicated non-polynomial shape functions may cause large cost when implementing numerical integration scheme [21]. For a review of meshless methods and their computer implementation aspects the reader is referred to [24]. For more useful applications of this method see [18,23,7,31,34,12,1,33,25,36,22]. Problems with non-local boundary conditions arise in the modeling of various processes in science and engineering such as heat transfer [10,8,9,20,35,14,38,19], control theory [37], thermoelasticity [15,16], medical science [11] and so on. The purpose of this article is to present very eﬃcient meshless method for solving the following two-dimensional timedependent diffusion equation

∂ u ∂ 2u ∂ 2u = 2 + 2 ∂t ∂x ∂y

(1.1)

with initial condition

u (x, y , 0) = f (x, y ),

*

0 x, y 1,

Corresponding author. Tel.: +98 912 1305326; fax: +98 281 3780040. E-mail address: [email protected] (S. Abbasbandy).

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.09.002

(1.2)

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

171

and boundary conditions

∂ u (0, y , t ) = g 0 ( y , t ), 0 t T , 0 y 1, ∂x ∂ u (1, y , t ) = g 1 ( y , t ), 0 t T , 0 y 1, ∂x u (x, 1, t ) = h1 (x, t ), 0 t T , 0 x 1, u (x, 0, t ) = h0 (x)μ(t ),

0 t T , 0 x 1,

(1.3) (1.4) (1.5) (1.6)

and the integral condition

1 1 u (x, y , t ) dx dy = m(t ), 0

0 x 1,

(1.7)

0

where f , g 0 , g 1 , h0 , h1 and m are known functions, while the functions u and μ are unknown. The boundary condition (1.6) is variable separable, with spatial dependence given by h0 (x) and time dependence given by μ(t ). Dehghan [17] proposed second-order ﬁnite difference schemes for the numerical solutions of the above mentioned problem. In comparison with the numerical solutions obtained by Dehghan, our results are better. It should be noted that the MLPG has been successfully applied to heat conduction problems with Dirichlet or Neumann’s boundary conditions in many papers such as [26,30,28,32,27,29]. Under consideration problem in this paper has a nonclassical boundary condition (1.7). Also, to impose the Neumann’s boundary condition, we use a ﬁnite difference scheme in combination with the classical MLPG which by our knowledge is a new idea in application of the MLPG. The organization of this paper is as follows. Section 2, presents a brief introduction on the moving least squares (MLS) approximations which is a well-known meshless interpolation scheme. In Section 3, the ﬁnite difference approximation schemes which are used in this study are introduced. Section 4 discusses on the local weak formulation of the problem and the numerical implementation of the method. Two different algorithms for imposing boundary conditions are also presented in this section as two subsections. Numerical examples are given in Section 5. Finally, our conclusions are given in Section 6. 2. The MLS approximation scheme A meshless method uses a local interpolation or approximation to represent the trial function with the values of the unknown variable at some nodes. The moving least squares (MLS) approximation is used in the current work. Consider a sub-domain Ωx , the neighborhood of a point x and denoted as the domain of deﬁnition of the MLS approximation for the trial function at x, which is located in the problem domain Ω . To approximate the distribution of function u in Ωx , over a number of randomly located nodes xi , i = 1, 2, . . . , n, the moving least squares approximant u h (x) of u, ∀x ∈ Ωx , can be deﬁned by

u h (x) = p T (x)a(x)

∀ x ∈ Ωx ,

(2.1)

where p T (x) = [ p 1 (x), p 2 (x), . . . , pm (x)] is a complete monomial basis of order m; and a(x) is a vector containing coeﬃcients a j (x), j = 1, 2, . . . , m, which are functions of the space coordinates x. For example, for a 2-D problem, p T (x) = [1, x, y ] and p T (x) = [1, x, y , x2 , xy , y 2 ], for linear basis (m = 3) and quadratic basis (m = 6), respectively. The coeﬃcient vector a(x) is determined by minimizing a weighted discrete L 2 norm, deﬁned as

J (x) =

n

w i (x) p T (xi )a(x) − uˆ i

2

T = P . a(x) − uˆ W P . a(x) − uˆ ,

(2.2)

i =1

where w i (x) is the weight function associated with the node i, with w i (x) > 0 for all x in the support of w i (x), xi denotes the value of x at node i, n is the number of nodes in Ωx for which the weight functions w i (x) > 0, the matrices P and W are deﬁned as

⎞

⎛

p T (x1 ) ⎜ pT (x2 ) ⎟

P=⎝

··· p (xn ) T

⎠ n×m

,

W=

w 1 (x)

··· 0

... 0 , ··· ··· . . . w n (x)

ˆ T = [uˆ 1 , uˆ 2 , . . . , uˆ n ]. Here it should be noted that uˆ i , i = 1, 2, . . . , n, in (2.2) are the ﬁctitious nodal values, and not the and u nodal values of the unknown trial function u h (x) in general. The stationarity of J in (2.2) with respect to a(x) leads to the ˆ following linear relation between a(x) and u

ˆ, A(x)a(x) = B(x)u

(2.3)

172

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

where the matrices A(x) and B(x) are deﬁned by

A(x) = P T WP = B(x)P =

n

w i (x)p(xi )p T (xi ),

i =1

(2.4)

B(x) = P T W = w 1 (x)p(x1 ), w 2 (x)p(x2 ), . . . , w n (x)p(xn ) .

(2.5)

The MLS approximation is well deﬁned only when the matrix A in (2.3) is non-singular. It can be seen that this is the case if and only if the rank of P equals m. A necessary condition for a well-deﬁned MLS approximation is that at least m weight functions are non-zero (i.e., n > m) for each sample point x ∈ Ω and that the nodes in Ωx will not be arranged in a special pattern such as on a straight line. Here a sample point may be a nodal point under consideration or a quadrature point. Solving for a(x) from (2.3) and substituting it into (2.1) gives a relation which may be written as the form of an interpolation function similar to that used in FEM, as

ˆ= u h (x) = Φ T (x) . u

n

φi (x)uˆ i ,

u h (xi ) ≡ u i ,

x ∈ Ωx ,

(2.6)

i =1

and essentially u i = uˆ i , where

Φ T (x) = pT (x)A−1 (x)B(x),

(2.7)

or

φi (x) =

m

p j (x) A−1 (x)B(x)

j =1

ji

.

Usually φi (x) is called the shape function of the MLS approximation corresponding to nodal point y i . From (2.5) and (2.7), it may be seen that φi (x) = 0 when w i (x). In practical applications, w i (x) is generally chosen such that it is non-zero over the support of nodal points y i . The support of the nodal point y i is usually taken to be a circle of radius r i , centered at y i . The fact that φi (x) = 0, for x not in the support of nodal point y i preserves the local character of the moving least squares approximation. Let C q (Ω) be the space of qth continuously differentiable functions on Ω . If w i (x) ∈ C q (Ω) and p j (x) ∈ C s (Ω), i = 1, 2, . . . , n and j = 1, 2, . . . , m, then φi (x) ∈ C r (Ω) with r = min(q, s). The partial derivatives of φi (x) are obtained as

φi ,k =

m

p j ,k A−1 B

j =1

ji

1 + p j A−1 B,k + A− ,k B ji ,

(2.8)

1 −1 −1 in which A− ,k = (A ),k represents the derivative of the inverse of A with respect to xk , which is given by A,k = −A−1 A,k A−1 , where ( ),i denotes ∂( )/∂ xi . In this paper the Gaussian weight function is used as

w i (x) =

⎧ di 2 ri 2 ⎪ ⎨ exp[−( ci ) ]−exp[−( ci ) ] ⎪ ⎩

r 2

1−exp[−( ci )] i

0,

, 0 di r i , di r i ,

where di = x − xi , c i is a constant controlling the shape of the weight function w i and r i is the size of the support domain. The size of support, r i , of the weight function w i associated with node i should be chosen such that r i should be large enough to have suﬃcient number of nodes covered in the domain of deﬁnition of every sample point (n m) to ensure the regularity of A. A very small r i may result in a relatively large numerical error in using Gauss numerical quadrature to calculate the entries in the system matrix. On the other hand, r i should also be small enough to maintain the local character of the MLS approximation. For useful researches on error analysis of the MLS approximation and meshless methods see [13, 39,40]. 3. Finite difference approximations In the current work, we employ a time-stepping scheme to overcome the time derivative. For this purpose we consider the following ﬁnite difference approximation for the time derivative operator

∂ u (x, y , t ) ∼ 1 k+1 u (x, y ) − uk (x, y ) . = ∂t t

(3.1)

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

173

Also we employ the following approximation by using the Crank–Nicholson technique

∇ 2 u (x, y , t ) =

1 2

∇ 2 uk+1 (x, y ) + ∇ 2 uk (x, y ) ,

(3.2)

where uk (x, y ) = u (x, y , kt ). Setting the difference schemes (3.1) and (3.2) into (1.1) yields

uk+1 (x, y ) −

t

∇ 2 uk+1 (x, y ) = uk (x, y ) +

2

t 2

∇ 2 uk (x, y ).

(3.3)

To treat the Neumann’s boundary conditions, we use the following ﬁnite difference schemes which are of order O (h 3 )

∂ uk (x, y ) = ∂x x=0 ∂ uk (x, y ) = ∂x x=1

1 h 1

−

11 6

11

h

6

uk (0, y ) + 3uk (h, y ) −

3 2

uk (1, y ) − 3uk (1 − h, y ) +

uk (2h, y ) + 3 2

1 3

uk (3h, y ) ,

uk (1 − 2h, y ) −

1 3

(3.4)

uk (1 − 3h, y ) .

(3.5)

4. Local weak formulation and numerical implementation The MLPG method constructs the weak form over local sub-domains such as Ωs , which is a small region taken for each node in the global domain Ω and could be of any geometric shape and size. In this paper they are taken to be of circular shape. Because the weak form is constructed over local sub-domains, the formulation is called the “local weak formulation”. The local weak form of Eq. (3.3) for xi = (xi , y i ) ∈ Ωsi can be written as follows

u k +1 −

t 2

t 2 k ∇ 2 uk+1 v dx = uk + ∇ u v dx,

(4.1)

2

Ωsi

Ωsi

where v is a test function. Using the divergence theorem, (4.1) yields the following expression

uk+1 v dx +

t

∇ uk+1 ∇ v dx −

2

Ωsi

Ωsi

t

v

2

∂ u k +1 ds = ∂n

∂Ωsi

uk v dx − Ωsi

t

∇ uk ∇ v dx +

2

t

Ωsi

v

2

∂ uk ds, ∂n

(4.2)

∂Ωsi

where Ωsi is a circle centered at xi of radius r0 , ∂Ωsi is the boundary of Ωsi , n = (n1 , n2 ) is the outward unit normal to the boundary ∂Ωsi , and

∂u ∂u ∂u = n1 + n2 ∂n ∂x ∂y is the normal derivative, i.e., the derivative in the outward normal direction to the boundary ∂Ωsi . The test function in each sub-domain is chosen as the Heaviside step function

v (x) =

1, x ∈ Ω s , 0,

x∈ / Ωs ,

so, ∇ v = 0 and the local weak form (4.2) is transformed into the following simple local integral equation

Ωsi

uk+1 dx −

t

2 ∂Ωsi

∂ u k +1 ds = ∂n

uk dx + Ωsi

t

2

∂ uk ds. ∂n

(4.3)

∂Ωsi

Consider N regularly located nodes on the boundary and domain of the problem so that the distance between two consecutive nodes in each direction is constant and is equal to h, see Fig. 1. Note that this assumption is necessary because we want to use the composite Simpson’s numerical integration rule to approximate the integral boundary condition. ˆ k (the approximation of μ(kt )) are known, Assuming that uˆ i (the approximation of u (xi , kt )), for i = 1, 2, . . . , N and μ ˆ (k+1) . So we have N + 1 unknowns and to compute these unknowns our aim is to compute uˆ ki +1 , for i = 1, 2, . . . , N and μ we need N + 1 equations. As it will be described below, corresponding to each node we obtain one equation. So we can obtain N equations from nodes and we need one other equation. One equation can be obtained from the non-local boundary condition. For nodes which are located in the interior of the domain, i.e., for xi ∈ interior Ω, from (4.3) and using the MLS approximation (2.6), we have the following linear equations

174

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

Fig. 1. M × M = N regularly located nodes in the interval [0, 1] × [0, 1] and their numeration.

N

φ j dx uˆ kj +1 −

j =1

Ωsi

N N N ∂φ j ∂φ j t t φ j dx uˆ kj + ds uˆ kj +1 = ds uˆ kj . 2 ∂n 2 ∂n j =1

∂Ωsi

j =1

Ωsi

(4.4)

j =1

∂Ωsi

For nodes on boundaries, we present two algorithms to impose boundary conditions as follows. 4.1. Algorithm A (a1) For nodes xl = (0, yl ) on the left vertical boundary (0 yl 1), using (3.4) and the MLS approximation, we have

nl nl+1 nl+2 nl+3 l +1 k +1 3 l +2 k +1 1 l +3 k +1 ∂ uk+1 (xl ) 1 11 l k+1 ≈ φi x uˆ i + 3 φi x uˆ i − φi x uˆ i + φi x uˆ i − + O h3 ∂x h 6 2 3 i =1 i =1 i =1 i =1 l = g 0 x , (k + 1)t , with xl+a = (ah, yl ). (a2) For nodes xl = (1, yl ) on the right vertical boundary (0 yl 1), using (3.5) and the MLS approximation, we have

nl nl−1 nl−2 nl−3 l −1 k +1 3 l −2 k +1 ∂ uk+1 (xl ) 1 11 l k+1 1 l −3 k +1 ≈ φi x uˆ i − 3 φi x uˆ i + φi x uˆ i +− φi x uˆ i + O h3 ∂x h 6 2 3 i =1 i =1 i =1 i =1 = g 1 xl , (k + 1)t , with xl−a = (1 − ah, yl ). (a3) For nodes xl = (xl , 1) on the top horizontal boundary (0 xl 1), using (1.5) and the MLS approximation, we have l

n

φi xl uˆ ki +1 = h1 xl , (k + 1)t .

i =1

(a4) For nodes xl = (xl , 0) on the bottom horizontal boundary (0 xl 1), using (1.6) we have l

n

k +1 ˆ φi xl uˆ ki +1 − h0 xl μ = 0.

i =1

Using Simpson’s composite numerical integration rule and the MLS approximation, the double integral in (1.7) is approximated in the following way:

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

1 1 u 0

k +1

(x, y ) dx dy ≈

N

d j ukj +1

j =1

0

=

N

dj

j =1

n

j

φi x

uˆ ki +1

i =1

=

n N i =1

j

d j φi x

175

uˆ ki +1 = m (k + 1)t ,

j =1

where d j ’s are Simpson’s composite numerical integration rule coeﬃcients. Summarizing what has been mentioned up to now, we have the following system of linear equations:

A−

t 2

B

Uˆ k+1

μˆ k+1

k t Uˆ = C+ + F, B μˆ k 2

(4.5)

where A = (ai , j ), B = (b i , j ), and C = (c i , j ) are square ( N + 1) × ( N + 1) matrices, Uˆ k = (uˆ ki ) where uˆ ki ≈ u (xi , kt ) and F = ( f i ) is an ( N + 1)-vector. Assuming the nodes are xi , i = 1, 2, . . . , N, the entries of these matrices are as follows. If xl ∈ interior Ω then

al, j = cl, j =

bl , j =

φ j dx, Ωsl

∂φ j ds. ∂n

∂Ωsl

If xl belongs to the boundary (a1) described above, then

3 1 φi xl + 3φi xl+1 − φi xl+2 + φi xl+3 , 6 2 3 f l = hg 0 xl , (k + 1)t .

al,i = −

11

If xl belongs to the boundary (a2) described above, then

3 1 φi xl − 3φi xl−1 + φi xl−2 − φi xl−3 , 6 2 3 f l = hg 1 xl , (k + 1)t .

al,i =

11

(4.6) (4.7)

If xl belongs to the boundary (a3) described above, then

al,i = φi xl ,

f l = h1 xl , (k + 1)t .

(4.8)

If xl is belong to the boundary (a4) described above, then

al,i = φi xl ,

al, N +1 = −h0 xl , (4.9) n and ﬁnally, a N +1,i = ( j =1 d j φi (x j )), for i = 1, 2, . . . , N , where d j ’s are Simpson’s composite numerical integration rule coeﬃcients, f N +1 = m((k + 1)t ), and all other entries of all the above mentioned matrices are zero. 4.2. Algorithm B (b1) For nodes xl = (0, yl ) on the left vertical boundary (0 yl 1), using (3.4), we have

3 1 ∂ uk+1 (xl ) 1 11 − uˆ k+1 xl + 3uˆ k+1 xl+1 − uˆ k+1 xl+2 + uˆ k+1 xl+3 + O h3 = g 0 xi , (k + 1)t . ≈ ∂x h 6 2 3 (b2) For nodes xl = (1, yl ) on the right vertical boundary (0 yl 1), using (3.5), we have

3 1 ∂ uk+1 (xl ) 1 11 k+1 l ≈ uˆ x − 3uˆ k+1 xl−1 + uˆ k+1 xl−2 − uˆ k+1 xl−3 + O h3 = g 1 xi , (k + 1)t . ∂x h 6 2 3 (b3) For nodes xl = (xl , 1) on the top horizontal boundary (0 xl 1), using (1.5), we have

uˆ k+1 xl = h1 xl , (k + 1)t . (b4) For nodes xl = (xl , 0) on the bottom horizontal boundary (0 xl 1), using (1.6) we have

ˆ k +1 = 0. uˆ k+1 xl − h0 xl μ It should be noted that in (b1)–(b4), uˆ i is the ﬁctitious nodal value at node i; not real one. The numerical integration procedure used is Simpson’s composite numerical integration rule, which has a truncation error of order four. The double integral in (1.7) is approximated in the following way

176

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

Table 1 The obtained results for Example 1; A and B means that the results are obtained using Algorithms A and B, respectively, described in Section 4. t

μ∞

μ∞

A

B

0.1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9 1 .0

3.8421 × 10−4 2.6722 × 10−4 4.6597 × 10−4 4.5792 × 10−4 6.5479 × 10−4 7.1481 × 10−4 9.5007 × 10−4 1.0898 × 10−3 1.3963 × 10−3 1.6448 × 10−3

3.3749 × 10−4 2.0447 × 10−4 3.9075 × 10−4 3.6584 × 10−4 5.4217 × 10−4 5.7746 × 10−4 7.8212 × 10−4 8.8485 × 10−4 1.1011 × 10−3 1.3010 × 10−3

1 1 0

uk+1 (x, y ) dx dy ≈

N

(3, 3)-ADI μ, ×10−3

μ, ×10−3

−1.8 −2.1 −2.2 −2.4 −2.5 −2.6 −2.6 −2.3 −2.0 −1.9

5 .1 5 .2 5 .5 5 .6 5 .7 6 .0 6 .2 5 .9 5 .7 5 .6

d j uˆ kj +1 = m (k + 1)t ,

(5, 5)-Implicit

u R ×10−5

u R ×10−5

A

B

7.4647 6.2374 6.6427 6.3851 6.5447 6.4290 6.5090 6.4502 6.4926 6.4609

7.2451 6.0482 6.5067 6.2274 6.3951 6.2773 6.3572 6.2993 6.3405 6.3099

(4.10)

j =1

0

where d j ’s are Simpson’s composite numerical integration rule coeﬃcients. Summarizing what has been mentioned up to now, we have the following system of linear equations:

A−

t 2

B

Uˆ k+1

μˆ k+1

k t Uˆ = C+ + F, B μˆ k 2

(4.11)

where A = (ai , j ), B = (b i , j ), and C = (c i , j ) are square ( N + 1) × ( N + 1) matrices, Uˆ k = (uˆ ki ) where uˆ ki ≈ u (xi , kt ) and F = ( f i ) is an ( N + 1)-vector. Assuming the nodes are xi , i = 1, 2, . . . , N, the entries of these matrices are as follows. If xl ∈ interior Ω then

al, j = cl, j =

∂φ j ds. ∂n

bl , j =

φ j dx, Ωsl

∂Ωsl

If xl is belong to the boundary (b1) described above then

al,l = −

11 6

,

al,l+1 = 3,

f l = hg 0 xl , (k + 1)t .

3 al,l+2 = − , 2

al,l+3 =

1 3

,

If xl is belong to the boundary (b2) described above then

al,l =

11 6

,

al,l−1 = −3,

al,l−2 =

3 2

,

1 al,l−3 = − , 3

f l = hg 1 xl , (k + 1)t .

(4.12) (4.13)

If xl is belong to the boundary (b3) described above then

al,l = 1,

f l = h1 xl , (k + 1)t .

(4.14)

If xl is belong to the boundary (b4) described above then

al,l = 1,

al, N +1 = −h0 xl .

(4.15)

a N +1, j = d j , for j = 1, 2, . . . , N, where d j ’s are Simpson’s composite numerical integration rule coeﬃcients and f N +1 = m((k + 1)t ). Finally, all other entries of all above mentioned matrices are zero. 5. Numerical experiments Two test problems are presented in this section. The numerical solutions using Algorithms A and B are presented in Tables 1 and 2. The domain and boundary integrals are evaluated with 2 and 8 point Gauss quadrature rule, respectively. For the MLS approximations, the quadratic basis is used for both examples. The relative error which is reported in the tables is deﬁned as follows:

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

177

Table 2 The obtained results for Example 2; A and B means that the results are obtained using Algorithms A and B, respectively, described in Section 4. t 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 .0

μ ∞

μ∞

u R

A

B

A

u R B

3.1856 × 10−4 1.5341 × 10−4 3.1479 × 10−4 2.2718 × 10−4 3.5791 × 10−4 2.9816 × 10−4 4.1966 × 10−4 3.7957 × 10−4 4.9873 × 10−4 4.7623 × 10−4

3.2031 × 10−4 1.5718 × 10−4 3.1807 × 10−4 2.3135 × 10−4 3.6212 × 10−4 3.0314 × 10−4 4.2488 × 10−4 3.8559 × 10−4 5.0516 × 10−4 4.8354 × 10−4

7.8204 × 10−5 6.2455 × 10−5 6.9105 × 10−5 6.4530 × 10−5 6.7723 × 10−5 6.5127 × 10−5 6.7131 × 10−5 6.5479 × 10−5 6.6799 × 10−5 6.5695 × 10−5

7.7972 × 10−5 6.1775 × 10−5 6.8484 × 10−5 6.3860 × 10−5 6.7089 × 10−5 6.4463 × 10−5 6.6491 × 10−5 6.4819 × 10−5 6.6156 × 10−5 6.5038 × 10−5

Fig. 2. Exact and approximate solutions obtained by Algorithm A at t = 0.5 for Example 1.

N ˆ i )2 1 (u i − u u R = i = . N 2 i =1 u i

Example 1. For the ﬁrst test problem consider (1.1)–(1.7) with

f (x, y ) = exp(x + y ), h0 (x) = exp(x),

g 0 ( y , t ) = exp( y + 2t ),

g 1 ( y , t ) = exp(1 + y + 2t ), m(t ) = exp(2t ) exp(2) − 2 exp(1) + 1 ,

h1 (x, t ) = exp(1 + x + 2t ),

for which the exact solution is

u (x, y , t ) = exp(x + y + 2t ),

μ(t ) = exp(2t ).

The calculations are performed using r0 = 0.5h, r i = 4r0 and c i = 0.05, where h is the distance between two consecutive nodes in each direction and r0 is the radius of the local sub-domain. Table 1 presents the results obtained for u and μ at several time levels with t = 0.1 and N = 441 nodes. In the tables, columns which represent the results obtained via (3, 3)-ADI and (5, 5)-Implicit ﬁnite difference schemes, are results obtained by Dehghan [17] and are presented here only for comparison. Also Figs. 2 and 3 show the exact and approximate solutions in both algorithms. Example 2. For the second test problem consider (1.1)–(1.7) with

f (x, y ) = (1 + y ) exp(x), h0 (x) = exp(x),

g 0 ( y , t ) = exp( y + 2t ), 3 m(t ) = exp(1) − 1 exp(t ), 2

g 1 ( y , t ) = (1 + y ) exp(1 + t ),

h1 (x, t ) = 2 exp(x + t ),

178

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

Fig. 3. Exact and approximate solution obtained by Algorithm B at t = 1.0 for Example 1.

Fig. 4. Exact and approximate solution obtained by Algorithm A at t = 0.5 for Example 2.

for which the exact solution is

u (x, y , t ) = (1 + y ) exp(x + t ),

μ(t ) = exp(t ).

The calculations are performed using r0 = 0.5h and r i = 4r0 and c i = 0.025, where r0 is the radius of the local subdomain. The results obtained for u and μ for several time levels with t = 0.1 and N = 1089 nodes are shown in Table 2. Also Figs. 4 and 5 show the exact and approximate solutions in both algorithms. 6. Conclusions In this paper an MLPG method was proposed for the study of parabolic partial differential equations with Neumann’s and non-classical boundary conditions. Two new techniques, namely Algorithms A and B, were proposed to impose the Neumann’s boundary conditions on square domains. Numerical results revealed that these algorithms are of same accuracy (at least for our examples), whereas Algorithm A is consistent while B is more simple. We proposed four point ﬁnite

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

179

Fig. 5. Exact and approximate solution obtained by Algorithm B at t = 1.0 for Example 2.

differences schemes for Neumann’s boundaries, but two or three point schemes can also be implemented in similar way. The non-classical integral boundary condition was discretized by using Simpson’s composite numerical integration rule and the resulting discretized equation was approximated by the MLS approximations. As it is shown in the numerical examples, in comparison with the existing methods, our proposed method has higher accuracy. Also, by experience we understood that for problems with Neumann’s boundary conditions, our proposed algorithms have higher precision than the classical MLPG. Implementing our proposed method to solve other problems with Neumann’s boundary conditions can be further research opportunity for interested researchers. Acknowledgements The authors would like to express their thankfulness to the anonymous referees whose constructive comments improved the quality of this paper. References [1] M.K. Alves, R. Rossi, A modiﬁed element-free Galerkin method with essential boundary conditions enforced by an extended partition of unity ﬁnite element weight function, Internat. J. Numer. Methods Engrg. 57 (2003) 1523–1552. [2] S.N. Atluri, The Meshless Method (MLPG) for Domain and BIE Discretizations, Tech. Science Press, 2004. [3] S.N. Atluri, S. Shen, The Meshless Local Petrov–Galerkin (MLPG) Method, Tech. Science Press, 2002. [4] S.N. Atluri, S. Shen, The meshless local Petrov–Galerkin (MLPG) method: a simple and less costly alternative to the ﬁnite element and boundary element methods, Comput. Modeling Engrg. Sci. 3 (2002) 11–51. [5] S.N. Atluri, T.L. Zhu, A new meshless local Petrov–Galerkin (MLPG) approach in computational mechanics, Comput. Mech. 22 (1998) 117–127. [6] S.N. Atluri, T.L. Zhu, A new meshless local Petrov–Galerkin (MLPG) approach to nonlinear problems in computer modeling and simulation, Comput. Modeling Simulation Engrg. 3 (1998) 187–196. [7] R.C. Batra, M. Porﬁri, D. Spinello, Free and forced vibrations of a segmented bar by a meshless local Petrov–Galerkin (MLPG) formulation, Comput. Mech. 41 (2008) 473–491. [8] J.R. Cannon, Y. Lin, An inverse problem of ﬁnding a parameter in a semi-linear heat equation, J. Math. Anal. Appl. 145 (2) (1990) 470–484. [9] J.R. Cannon, Y. Lin, A.L. Matheson, The solution of the diffusion equation in two-space variables subject to the speciﬁcation of mass, Appl. Anal. 50 (1993) 1–19. [10] J.R. Cannon, J. van der Hoek, Diffusion subject to speciﬁcation of mass, J. Math. Anal. Appl. 115 (1986) 517–529. [11] V. Capsso, K. Kunisch, A reaction–diffusion system arising in modeling man-environment diseases, Quart. Appl. Math. 46 (1988) 431–449. [12] T. Chen, I.S. Raju, A coupled ﬁnite element and meshless local Petrov–Galerkin method for two-dimensional potential problems, Comput. Methods Appl. Mech. Engrg. 192 (2003) 4533–4550. [13] Rongjun Cheng, Yumin Cheng, Error estimates for the ﬁnite point method, Appl. Numer. Math. 58 (2008) 884–898. [14] Y.S. Choi, K.Y. Chan, A parabolic equation with nonlocal boundary conditions arising from electrochemistry, Nonlinear Anal. Theory Methods Appl. 18 (4) (1992) 317–331. [15] W.A. Day, Existence of a property of solutions of the heat equation subject to linear thermoelasticity and other theories, Quart. Appl. Math. 40 (1982) 319–330. [16] W.A. Day, A decreasing property of solutions of a parabolic equation with applications to thermoelasticity and other theories, Quart. Appl. Math. XLIV (1983) 468–475. [17] M. Dehghan, Second-order schemes for a boundary value problem with Neumann’s boundary conditions, J. Comput. Appl. Math. 138 (2002) 173–184.

180

S. Abbasbandy, A. Shirzadi / Applied Numerical Mathematics 61 (2011) 170–180

[18] M. Dehghan, D. Mirzaei, Meshless Local Petrov–Galerkin (MLPG) method for the unsteady magnetohydrodynamic (MHD) ﬂow through pipe with arbitrary wall conductivity, Appl. Numer. Math. 59 (2009) 1043–1058. [19] G. Fairweather, R.D. Saylor, The reformulation and numerical solution of certain nonclassical initial–boundary value problems, SIAM J. Sci. Statist. Comput. 12 (1991) 127–144. [20] Y. Lin, An inverse problem for a class of quasilinear parabolic equations, SIAM J. Math. Anal. 22 (1991) 146–156. [21] A. Mazzia, M. Ferronato, G. Pini, G. Gambolati, A comparison of numerical integration rules for the meshless local Petrov–Galerkin method, Numer. Algorithms 45 (2007) 61–74. [22] D. Mirzaei, M. Dehghan, Meshless local Petrov–Galerkin (MLPG) approximation to the two dimensional sine-Gordon equation, J. Comput. Appl. Math. 233 (2010) 2737–2754. [23] D. Mirzaei, M. Dehghan, A meshless based method for solution of integral equations, Appl. Numer. Math. 60 (2010) 245–262. [24] V.P. Nguyen, T. Rabczuk, S. Bordas, M. Duﬂot, Meshless methods: A review and computer implementation aspects, Math. Comput. Simulation 79 (2008) 763–813. [25] L.F. Qian, R.C. Batra, Three-dimensional transient heat conduction in a functionally graded thick plate with a higher-order plate theory and a meshless local Petrov–Galerkin method, Comput. Mech. 35 (2005) 214–226. [26] J. Sladek, V. Sladek, S.N. Atluri, Meshless local Petrov–Galerkin method for heat conduction problem in an anisotropic medium, CMES Comput. Model. Eng. Sci. 6 (2004) 309–318. [27] J. Sladek, V. Sladek, Ch. Hellmich, J. Eberhardsteiner, Heat conduction analysis of 3D axisymmetric and anisotropic FGM bodies by meshless local Petrov–Galerkin method, Comput. Mech. 39 (2007) 323–333. [28] J. Sladek, V. Sladek, J. Krivacek, Ch. Zhang, Local BIEM for transient heat conduction analysis in 3-D axisymmetric functionally graded solids, Comput. Mech. 32 (2003) 169–176. [29] J. Sladek, V. Sladek, C.L. Tan, S.N. Atluri, Analysis of transient heat conduction in 3D anisotropic functionally graded solids by the MLPG, Comput. Modeling Engrg. Sci. 32 (2008) 161–174. [30] V. Sladek, J. Sladek, M. Tanaka, Ch. Zhang, Transient heat conduction in anisotropic and functionally graded media by local integral equations, Engrg. Anal. Bound. Elem. 29 (2005) 1047–1065. [31] J. Sladek, V. Sladek, P.H. Wen, Y.C. Hon, Inverse fracture problems in piezoelectric solids by local integral equation method, Engrg. Anal. Bound. Elem. 33 (2009) 1089–1099. [32] J. Sladek, V. Sladek, Ch. Zhang, Transient heat conduction analysis in functionally graded materials by the meshless local boundary integral equation method, Comput. Mater. Sci. 28 (2003) 494–504. [33] J. Sladek, V. Sladek, Ch. Zhang, M. Schanz, Meshless local Petrov–Galerkin method for continuously nonhomogeneous linear viscoelastic solids, Comput. Mech. 37 (2006) 279–289. [34] V. Vavourakis, D. Polyzos, A MLPG (LBIE) numerical method for solving 2D incompressible and nearly incompressible elastostatic problems, Comm. Numer. Methods Engrg. 24 (2008) 281–296. [35] S. Wang, The numerical method for the conduction subject to moving boundary energy speciﬁcation, Numer. Heat Transfer 130 (1990) 35–38. [36] Q.X. Wang, H. Li, K.Y. Lam, Analysis of microelectromechanical systems (mems) devices by the meshless point weighted least-squares method, Comput. Mech. 40 (2007) 1–11. [37] S. Wang, Y. Lin, A ﬁnite difference solution to an inverse problem determining a control function in a parabolic partial differential equations, Inverse Problems 5 (1989) 631–640. [38] S. Wang, Y. Lin, A numerical method for the diffusion equation with nonlocal boundary speciﬁcations, Internat. J. Engrg. Sci. 28 (1990) 543–546. [39] C. Zuppa, Error estimates for moving least square approximations, Bull. Braz. Math. Soc. (N.S.) 34 (2003) 231–249. [40] Carlos Zuppa, Good quality point sets and error estimates for moving least square approximations, Appl. Numer. Math. 47 (2003) 575–585.

Applied Numerical Mathematics 61 (2011) 181–200

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

A posteriori error estimates for optimal distributed control governed by the evolution equations ✩ Chunguang Xiong a,∗ , Yuan Li b a b

Department of Mathematics, Beijing Institute of Technology, Beijing, 100081, China Department of Mathematics, Chinese People’s Armed Police Forces Academy, Hebei, 065000, China

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 22 June 2009 Received in revised form 6 September 2010 Accepted 17 September 2010 Available online 12 October 2010 Keywords: Optimal control Evolution equation A posteriori error estimate Method of lines FEM

We describe a technique for a posteriori error estimates suitable to the optimal control problem governed by the evolution equations solved by the method of lines. It is applied to the control problem governed by the parabolic equation, convection-diffusion equation and hyperbolic equation. The error is measured with the aid of the L 2 -norm in the space– time cylinder combined with a special time weighted energy norm. Crown Copyright © 2010 Published by Elsevier B.V. on behalf of IMACS. All rights reserved.

1. Introduction The optimal control or design and the ﬁnite element method are crucial to many engineering applications. It is essential to apply eﬃcient numerical methods in solving the optimal control problem. Finite element approximation of the optimal control problems has been an important and hot topic in engineering design work, and has been extensively studied in literature [13,17,14,21,18]. For the optimal control problems governed by elliptic or parabolic state equations, many authors did much work, e.g. [5,20,7,19,22,15]. Recently a number of techniques for the a posteriori error estimates have been developed. A posteriori error estimate is used for assessing numerical results and adjusting the meshes. The subject has been treated by many authors in various means. In [4,6,12] the elliptic problems are analyzed. And parabolic problems are investigated in [8,23,25,26,1,24,10,11,16]. In [3], the a posteriori error estimates are analyzed for parabolic problems discretized with respect to time only. However, in [8,23,25,26,1,24] they gave error estimates for parabolic problems semi-discretized in space while the time remains continuous. This technique is called the method of lines, which is a widespread method of numerical computations and converts evolution equation into a system of usually stiff ODEs, i.e., the original PDE is approximated by a system of ODEs. Whereas ODEs can be solved by very eﬃcient solvers, e.g., LSODE. The a posteriori error estimates for partial differential equations by the method of lines are the subject of literature [25,26,1,24,9]. Moreover, the original PDE can be replaced by another one (for example, for ﬁnding numerical solutions to stationary problems of mathematical physics one can apply the stabilization method). In the present paper we discuss the a posteriori error estimate analysis of the method of lines. The error of the optimal control governed by the parabolic equations is estimated in L 2 -norm over the space–time cylinder combined with a special ✩

*

This subject is supported partially by NSFC No. 10932002 and No. 10972031. Corresponding author. E-mail addresses: [email protected] (C. Xiong), [email protected] (Y. Li).

0168-9274/$30.00 Crown Copyright doi:10.1016/j.apnum.2010.09.004

©

2010 Published by Elsevier B.V. on behalf of IMACS. All rights reserved.

182

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

time weighted energy norm. Moreover, we pay attention to linear hyperbolic and convection-diffusion problems. And we can obtain that the quality of the a posteriori error estimates for the optimal control governed by the evolution equations is the same as the optimal control governed by the elliptic ones. In [2] the ideas of our paper are further discussed for parabolic equations of the heat transfer. The organization of the paper is as follows. In Section 2, we formulate the continuous forms of the control problems governed by the parabolic equations in a weak sense. From the weak form, we derive an integral identity which is the basis of a posteriori error estimates. In Section 3, we state the discretization of continuous problems by the method of lines. In Section 4, we prove that the a posteriori error upper bound is bounded in terms of residuals and inter-element jump about control problems governed by the parabolic equations. In Section 5, the method of lines is applied to the control problems governed by the convection-diffusion equation. In Section 6, it is concerned to the application of the method of lines to the control problems governed by the hyperbolic equations. 2. Continuous problem of the parabolic equation In this section we study the method of lines about the convex optimal control problems governed by the parabolic equation. The problem we consider here is as follows: Find u ∈ M = L 2 ( Q uT ) : Q uT → R , Q uT = Ωu × (0, T ) such that

min

u∈M

g ( y ) + h(u ) dx

Ω

subjects to

⎧

d ⎪ ∂ ∂y ∂y ⎪ ⎪ − a = f + Bu (t , x) ∈ Q T , ⎪ i j ⎨ ∂t ∂ xi ∂xj i , j =1 ⎪ ⎪ y (x, 0) = y 0 (x) x ∈ Ω, ⎪ ⎪ ⎩ y (x, t ) = 0 x ∈ ∂Ω, t ∈ (0, T ),

(2.1)

where d 1 is an integer number and denotes the space dimension. Ω ⊂ R d and Ωu ⊂ R d are the bounded domains with the Lipschitz continuous boundary ∂Ω and ∂Ωu , respectively. u = u (x, t ), x ∈ Ωu , t ∈ (0, T ), 0 < T < ∞, and y = y (x, t ), x ∈ Ω, t ∈ (0, T ), 0 < T < ∞. For simplicity we set Y = H 01 (Ω) and make the following fundamental assumptions: ¯ n×n , and there exists a constant α > 0 such that A1. A (x) = (ai j )n×n ∈ (C ∞ (Ω))

v T A v α | v |2 . A2. There exists a constant C > 0 such that

( Bu , v ) = u , B ∗ v C u 2 L ( Q uT ) v L 2 ( Q T ) ,

u ∈ M, v ∈ Y ,

where B ∗ is the adjoint operator of B. A3. There exists a constant C > 0 such that

g ( y ) − g ( v ), q C y − v 2 L ( Q T ) q L 2 ( Q T ) , g ( y ) − g ( v ), y − v 0, y , v ∈ Y .

y, v , q ∈ Y ,

A4. There exists a constant C > 0 such that

h (u ) − h ( w ), u − w C u − w 2L 2 ( Q ) , uT

u, w ∈ M .

If y = y (x, t ), then y (t ) denotes the function deﬁned on Ω such that y (t )(x) = y (t , x) for x ∈ Ω and t ∈ (0, T ). We will denote the Bochner space of square integrable maps of the interval H 01 (Ω) by L 2 (0, T ; Y ). We assume that

y 0 ∈ L 2 (Ω). The weak form of the convex optimal control problem is posed as follows: Find a function u ∈ M = L 2 ( Q uT ) such that

min u∈M

Ω

g ( y ) + h(u ) dx

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

183

subjects to

⎧ ⎪ ⎪ ⎨ d ⎪ ⎪ ⎩

dt

y v dx +

d Ω i , j =1

Ω

∂ y ∂v ai j dx = ∂ x j ∂ xi

( f + Bu ) v dx ∀ v ∈ Y , Ω

y (0) = y 0 .

It is well known (see, e.g., [13]) that the control problem (2.1) has the unique solution ( y , u ), and a pair ( y , u ) is the solution of (2.1) if and only if there is a co-state p ∈ Y such that the triplet ( y , u , p ) satisﬁes the following equivalent problem

⎧

d ⎪ ∂ ∂y ⎪ ∂y ⎪ = f + Bu − a ⎪ i j ⎪ ⎪ ∂t ∂ xi ∂xj ⎪ i , j = 1 ⎪ ⎪ ⎪ ⎪

⎪ d ⎪ ⎨ ∂p ∂ ∂p = g ( y) − ai j − ∂t ∂ xi ∂xj ⎪ i , j =1 ⎪ ⎪ ⎪ ⎪ ⎪ h ( u ) + B ∗ p, w − u 0 ⎪ ⎪ ⎪ ⎪ ⎪ y (x, 0) = y 0 (x), p (x, T ) = 0 ⎪ ⎪ ⎩ y (x, t ) = 0

(t , x) ∈ Q T ,

(t , x) ∈ Q T , ∀w ∈ M, x ∈ Ω, x ∈ ∂Ω, t ∈ (0, T ).

We give the weak form of the above equivalent problem,

⎧ d ⎪ d ∂ y ∂v ⎪ ⎪ y v dx + a dx = ( f + Bu ) v dx ∀ v ∈ Y , ⎪ i j ⎪ ⎪ dt ∂ x j ∂ xi ⎪ ⎪ i , j = 1 ⎪ Ω Ω Ω ⎪ ⎪ ⎨ d d ∂ p ∂q − pq dx + a dx = g ( y )q dx ∀q ∈ Y , i j ⎪ ⎪ dt ∂ x j ∂ xi ⎪ i , j = 1 ⎪ Ω Ω Ω ⎪ ⎪ ⎪ ⎪ ⎪ h (u ) + B ∗ p , w − u 0 ∀w ∈ M, ⎪ ⎪ ⎩ y (0) = y 0 , p ( T ) = 0.

(2.2)

(2.3)

With the help of techniques of existence and uniqueness proof about solution ( y , p ) of (2.2) and (2.3), it can be shown ∂y ∂p that ∂ t , ∂ t ∈ L 2 (0, T ; Y ). In what follows we assume the stronger regularity of y and p such that

∂y ∂p , ∈ L 2 ( Q T ). ∂t ∂t Then (2.2) and (2.3) can be rewritten as

∂y v dx + ∂t

Ω

d Ω i , j =1

∂ y ∂v ai j dx = ∂ x j ∂ xi

( f + Bu ) v dx ∀ v ∈ Y , a.e. t ∈ (0, T )

(2.4)

Ω

and

−

∂p q dx + ∂t

Ω

d

ai j

Ω i , j =1

∂ p ∂q dx = ∂ x j ∂ xi

g ( y )q dx

∀q ∈ Y , a.e. t ∈ (0, T ).

(2.5)

Ω

Now we modify the weak forms (2.4) and (2.5). Firstly, we take arbitrarily v , q ∈ L 2 (0, T ; Y ) in (2.4) and (2.5) for a.e. t ∈ (0, T ). Secondly, we integrate from 0 to τ ∈ (0, T ) with respect to t. Last, we carry out the integration over (0, T ) with respect to τ . So we get the following identities:

T τ 0

∂y v dx dt dτ + ∂t

0 Ω

T T − 0

τ Ω

T τ d 0

∂p q dx dt dτ + ∂t

ai j

0 Ω i , j =1

T T d 0

τ

Ω i , j =1

∂ y ∂v dx dt dτ = ∂ x j ∂ xi

ai j

T τ ( f + Bu ) v dx dt dτ 0

∂ p ∂q dx dt dτ = ∂ x j ∂ xi

∀ v ∈ L 2 (0, T ; Y ),

(2.6)

0 Ω

T τ 0

g ( y )q dx dt dτ

∀q ∈ L 2 (0, T ; Y ).

0 Ω

Note that under the assumptions, the integrations in (2.6) and (2.7) make sense for v , q ∈ L 2 (0, T ; Y ).

(2.7)

184

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

By the Fubini’s Theorem, we have the following identity:

T τ

T

ϕ (t ) dt dτ = 0

0

ϕ ∈ L 1 (0, T ).

( T − t )ϕ (t ) dt , 0

Take

∂ϕ Y = ϕ ∈ L 2 (0, T ; Y ), ∈ L2( Q T ) , ∂t

V = L 2 (0, T ; Y ),

Y ⊂ V.

We can deﬁne the bilinear maps B and B ∗ : Y × V −→ R,

T τ B( y , v ) = 0

0 Ω

T τ = 0

B ∗ ( p , q) = −

∂y v dx dτ dt + ∂t

0 Ω T T

0

∂y v dx dτ dt + ∂t

T (T − t )

τ Ω

=− τ Ω

∂p q dx dτ dt + ∂t

T

t

0 Ω

0

T

g ( y )q dx dτ dt =

τ Ω

0

0

0 Ω

0

τ Ω

∂ p ∂q dx dt . ∂ x j ∂ xi

( f + Bu ) v dx dt ,

g ( y )q dx dt

(T − t )

d Ω i , j =1

∂p q dx dτ dt + ∂t

−

∂ p ∂q dx dτ dt ∂ x j ∂ xi

∀v , q ∈ Y .

Ω

0

T T

ai j

∂ y ∂v dx dt , ∂ x j ∂ xi

Ω

So we can rewrite the weak forms: T τ T

∂y v dx dτ dt + ∂t

ai j

t 0

ai j

Ω i , j =1

(T − t )

T T

d

τ Ω i , j =1

0

( f + Bu ) v dx dτ dt =

d

T d

It follows from Fubini’s Theorem that T τ T 0

T

0

∂p q dx dτ dt + ∂t

∂ y ∂v dx dτ dt ∂ x j ∂ xi

Ω i , j =1

0

ai j

0 Ω i , j =1

0

T T 0

T τ d

T d t

ai j

Ω i , j =1

0

ai j

∂ y ∂v dx dt = ∂ x j ∂ xi

∂ p ∂q dx dt = ∂ x j ∂ xi

T (T − t ) 0

T t 0

( f + Bu ) v dx dt ∀ v ∈ Y , Ω

g ( y )q dx dt

∀q ∈ Y .

Ω

For convenience, we deﬁne two linear maps Fu and G y : V −→ R

T Fu ( v ) =

(T − t ) 0

T ( f + Bu ) v dx dt ,

G y (q) =

t 0

Ω

g ( y )q dx dt ,

Ω

and simplify (2.6) and (2.7) as follows:

B( y , v ) = Fu ( v ),

∀v ∈ Y , B ( p , q) = G y (q), ∀q ∈ Y .

(2.8)

∗

The bilinear forms B ( y , v )

(2.9)

and B ∗ ( p , q) have the following property – coercivity.

Theorem 1. Assume that matrix A is positive deﬁnite. Then B ( y , v ) and B ∗ ( p , q) are coercive in the following sense,

T B( v , v )

v 2L 2 ( Q ) T

( T − t )| v |2H 1 (Ω) dt −

+α 0

T 2

v (0)22

L (Ω)

∀v ∈ Y ,

(2.10)

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

B∗ (q, q) q2L 2 ( Q

T T

T q( T )22 L (Ω) 2

t |q|2H 1 (Ω) dt −

+α )

185

∀q ∈ Y .

(2.11)

0

Proof. Take y = v in (2.8), so we have

T τ B( v , v ) = 0

∂v v dx dt dτ + ∂t

T (T − t )

0 Ω

d

ai j

Ω i , j =1

0

∂v ∂v dx dt . ∂ x j ∂ xi

By the Newton–Leibniz Theorem, for the ﬁrst term, we have

T τ 0

T τ

∂v 1 v dx dτ dt = ∂t 2

0 Ω

0

∂ v2 1 dx dτ dt = ∂t 2

0 Ω

v 2 dx dt −

T 2

v (0)22

L (Ω)

0 Ω

1

= v 2L 2 ( Q 2

T

T)

T

−

2

2

v (0) L 2 (Ω) .

Applying the positive deﬁnite of the matrix A (x) and Fubini’s Theorem, for the second term, we get

T (T − t )

d

∂v ∂v dx dt α ∂ x j ∂ xi

ai j

Ω i , j =1

0

T (T − t )

2 T d ∂v dx dt = α ( T − t )| v |2H 1 (Ω) dt . ∂ xi

Ω i =1

0

0

And the ﬁrst result follows. Now, we turn to the second one.

T T

∗

B (q, q) = − 0

τ Ω

∂q q dx dτ dt + ∂t

T d t

0

ai j

Ω i , j =1

∂q ∂q dx dt . ∂ x j ∂ xi

Applying the same method above, for the ﬁrst term, we have

T T

∂q 1 q dx dτ dt = − ∂t 2

− 0

τ Ω

T T τ Ω

0

=

1

q2L 2 ( Q ) T 2

T

∂ q2 1 dx dτ dt = ∂t 2

q2 dx dt −

T q( T )22 L (Ω) 2

0 Ω

2 T − q( T ) L 2 (Ω) . 2

For the second term, it follows from the positive deﬁnite of A (x) and Fubini’s Theorem that

T d t

0

Ω i , j =1

∂q ∂q ai j dx dt α ∂ x j ∂ xi

We obtain the second result.

2 T T d 2 ∂q t dx dt = α t q(t ) H 1 (Ω) dt . ∂ xi Ω i =1

0

0

2

For simplicity, we introduce the weighted norms in the space V besides the standard norm v 2L 2 (0. T ;Y ) =

T v V 1 = α

2 1 ( T − t ) v (t ) H 1 (Ω) dt +

v 2 dx dt

2

v V 2 = α

,

QT

0

T

12

2 t v (t ) 1

H (Ω)

dt +

1

12

2

v dx dt

2

.

QT

0

(2.10) and (2.11) are rewritten as

2 1 v V 1 B( v , v ) + v (0) L 2 (Ω) , 2

2 1 qV 2 B∗ (q, q) + q( T ) L 2 (Ω) . 2

T 0

v 2H 1 (Ω) dt,

186

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

3. The discrete problem In this section we discuss the problem (2.1) by the ﬁnite element method of lines. Let Th and Fh denote the partitions of Ω and Ωu into the disjoint open regular n-simplicity K , respectively. Every element has at most one face on ∂Ω or ∂Ω u . Any two elements have at most either one common vertex, or a whole edge, or a whole face. Every element K in triangulation (or rectangularity) partition Th is aﬃne equivalent to one of several reference elements. We assume that the partitions Th and Fh have standard properties used in the ﬁnite element method (see, e.g. [9]). Γh denotes union of the boundaries of all elements on partition Th , i.e., Γh = K ∈Th S ∈∂ K S . For each t ∈ [0, T ] the solution triplet ( y , u , p ) ∈ Y × M × Y will be approximated by the triplet ( y h , u h , p h ) ∈ Y h × M h × Y h , here we deﬁne Y h and M h

¯ yh |∂Ω = 0, yh | K ∈ P r ( K ), ∀ K ∈ Th , Y h = y h ∈ C (Ω);

M h = u h ∈ L 2 (Ωu ), u h ∈ P m ( K u ), ∀ K u ∈ Fh ,

where P r ( K ) and P m ( K u ) are the polynomial spaces on K and K u respectively, i.e.,

P r ( K ) = span x|α | , 0 |α | r ,

P m ( K u ) = span x|α | , 0 |α | m .

We deﬁne the ﬁnite element spaces:

∂ ϕh Yh = ϕh ∈ L 2 0, T ; Y h , ∈ L2( Q T ) , ∂t

Vh = L 2 0, T ; Y h .

The approximation of the optimal control problem is as follows,

⎧ d ⎪ d ∂ yh ∂ v h ⎪ ⎪ y v dx + a dx = ( f + Buh ) v h dx ∀ v h ∈ Yh , ⎪ ij h h ⎪ ⎪ dt ∂ x j ∂ xi ⎪ ⎪ i , j = 1 Ω Ω Ω ⎪ ⎪ ⎪ ⎪ d ⎪ ⎪ d ∂ p h ∂ qh ⎪ ⎪ ⎨− p h qh dx + ai j dx = g ( y h )qh dx ∀qh ∈ Yh , dt ∂ x j ∂ xi i , j =1 Ω Ω Ω ⎪ ⎪ ⎪ ⎪ T ⎪ ⎪ ⎪ ⎪ h (u ) + B ∗ p , w − u dt 0 ⎪ ∀wh ∈ Mh , h h h h U ⎪ ⎪ ⎪ ⎪ ⎪ ⎪0 ⎪ ⎩ y h (0) = y h0 , p h ( T ) = 0, where y h0 is the approximation of y 0 in Yh , i.e., y h0 = πhα y 0 . We deﬁne two linear functionals

(T − t ) 0

t 0

( f + Buh ) v h dx dτ dt ∀ v h ∈ Yh , Ω

T G yh (qh ) =

πhα : Y → Yh is standard Lagrange interpolant operator.

T Fu h ( v h ) =

(3.1)

g ( y h )qh dx dτ dt

∀qh ∈ Yh .

Ω

So we reformulate the discrete problems as below by these denotations, i.e.,

B( y h , v h ) = Fuh ( v h ) ∀ v h ∈ Yh ,

(3.2)

B∗ ( p h , qh ) = G yh (qh ) ∀qh ∈ Yh .

(3.3)

4. A posteriori error estimators 4.1. A posteriori error estimates of the state equation As usual, we split the error

B y I , v = Fuh ( v ),

ε y = y − yh = y − y I + y I − yh = e y + η y , where y I is the solution to the following equation

∀v ∈ Y .

(4.1)

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

187

Theorem 2. Let us assume that y and y I are the solutions to (2.8) and (4.1) respectively. For the error e y = y − y I we have

y − yI

C u − uh L 2 ( Q uT ) + V1

T 2

y0 − y I

0 L 2 (Ω) .

Proof. Let v = y − y I in (2.10) and use Theorem 1 and Young’s inequality, we have

T

1 2

y − y I 22

L (Q T

+α )

2 2 T ( T − t ) y − y I H 1 (Ω) dt − y 0 − y 0I L 2 (Ω) B y − y I , y − y I = B (u − uh ), y − y I 2

0

C u − uh L 2 (( Q uT ) y − y I L 2 ( Q C u − uh 2L 2 (( Q

T)

1

uT

2 + y − y I L2 ( Q ) . ) 4

T

We can conclude by the weighted norm and Cauchy inequality, i.e.,

y − yI

V1

C u − uh L 2 ( Q uT ) +

T 2

y0 − y I

2

0 L 2 (Ω) .

Let v ∈ Y . It follows from (2.8) and (4.1) that

B y I , v − B( y h , v ) = Fuh ( v ) − B( y h , v ) = R( v ),

(4.2)

where R( v ) is the residual, which has the following orthogonality,

R( v h ) = 0 ∀ v h ∈ Yh . We set h S = diam( S ) and denote a unit normal on S by n S . We deﬁne the jump of function S (x) of two adjoint elements by

ϕ in a common boundary

[ϕ ]| S (x) = lim ϕ (x + εn S ) − ϕ (x − εn S ) ∀x ∈ S . ε →0

It is well known that the jump depends on the orientation of n S , but the expression of [ϕ ]| S niS (i = 1, . . . , d) is independent on the orientation of n S . We take

r K ( y h ) = f + Bu h −

d ∂ yh ∂ ∂ yh − ai j , ∂t ∂ xi ∂xj

J S ( yh ) = −

i , j =1

d

ai j

i , j =1

∂ yh i n . ∂xj S

By Gauss formula, we obtain

T R( v ) =

(T − t ) r K (t ) v (t ) dx + J S (t ) v (t ) ds dt . S ∈Γh S

K ∈T K

0

Let v = η y in (4.2) and apply Theorem 1. We get the following lemmas (see [3]). Lemma 1. For the error η y = y I − y h , we have the upper bound,

2

η y V 1

1 2

η y (0)22

K ∈Th

T +

(T − t ) 0

L

T + (K )

(T − t ) 0

h K rk (t ) K ∈Th

L2 (K )

−1 h (η y − v h ) K

1 1 h 2 J s (t ) 2 h− 2 (η y − v h ) 2 dt ∀ v h ∈ Yh . S S L (S) L (S)

S ∈Γh

L2 (K )

dt

(4.3)

188

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

Lemma 2. For the error η y = y I − y h , we have the a posteriori error estimate η y V 1 C ξ . (1) The case d = 1,

η y (0)22 + C ( T − t ) h K r K (t )22 dt ; ξ =T L (K ) L (K ) T

2

K ∈Th

(4.4a)

K ∈Th

0

(2) The case d 2,

η y (0)22 + C 1 ( T − t ) h K r K (t )22 dt ξ =T L (K ) L (K ) T

2

K ∈Th

K ∈Th

0

T + C2

(T − t )

1 h 2 J S (t )22 dt . S L (S)

(4.4b)

S ∈Γh

0

Proof. The proof of Lemma 2 follows from Lemma 1 and Cauchy–Schwartz inequality. Because geometric structure of high ¯ , the jump terms are not generally dimensional vector space is different from 1-dimensional vector space, H 1 (Ω) C (Ω) vanished. So we use the Clement’s interpolation instead of the Lagrange interpolation. 2 We summarize the above results and obtain the following result: Theorem 3. Assume that y and y h are the solutions to the problems (2.8) and (3.1), respectively. For the error ε y = y − y h we have

ε y V 1 C ξ +

2

y0 − y I

T

0

L 2 (Ω)

+ u − uh L 2 (ΩuT )

(4.5)

Proof. Using the triangle inequality, we have

ε y V 1 e y V 1 + η y V 1 .

(4.6)

It follows from Theorem 2, the weighted norm and Cauchy–Schwartz inequality that

e y V 1 C

T 2

y0 − y I 0

+ u − uh L 2 (ΩuT ) . L 2 (Ω)

By (4.4), (4.6) and (4.7), we obtain the inequality.

(4.7)

2

4.2. A posteriori error estimate of co-state equation As usual, we split the error

B∗ p I , q = G yh (q),

ε p = p − ph = p − p I + p I − ph = e p + η p , where p I is the solution to the following equation

∀q ∈ Y .

(4.8)

Theorem 4. Assume that p and p I are the solutions to (2.9) and (4.8) respectively. For the error e p = p − p I we have

p − pI

V1

C y − yh L2 ( Q T ) +

T 2

p ( T ) − p I ( T ) L 2 (Ω) .

Proof. Let q = p − p I in (2.11). It follows from (2.9), (4.8), Theorem 1 and Young’s inequality that

1 2

p−

2 p I L2 ( Q ) T

T +α

2

0

∗

2 2 T ( T − t ) p − p I H 1 (Ω) dt − p ( T ) − p I ( T ) L 2 (Ω)

I

I

B p − p I , p − p = g ( y ) − g ( yh ), p − p p − pI 2 C y − yh 22 C y − yh 2 L (Q T )

L (Q T )

L (Q T )

2 1 + p − p I L2 ( Q ) . 4

T

(4.9)

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

189

We can get the conclusion by the weighted norm and Cauchy inequality, i.e.,

p − pI

T

C y − yh L2 ( Q T ) +

V1

2

p ( T ) − p I ( T )

L 2 (Ω)

.

2

Let q ∈ Y . It follows from (3.2) and (4.8) that

B∗ p I , q − B∗ ( p h , q) = G yh (q) − B∗ ( p h , q) = R∗ (q).

(4.10)

It is obvious that the residual R∗ (q) has orthogonality, i.e.,

R∗ (qh ) = 0,

∀qh ∈ Yh .

Now we set

d ∂ ph ∂ ∂ ph r K ( ph ) = g ( yh ) − − − ai j , ∂t ∂ xi ∂xj ∗

d

J ∗S ( p h ) = −

i , j =1

ai j

i , j =1

∂ ph i n . ∂xj S

Applying Gauss formula on every element, we obtain

R∗ (q) =

T

r ∗K (t )q(t ) dx +

t

J ∗S (t )q(t ) ds dt .

(4.11)

S ∈ h S

K ∈T K

0

Theorem 5. For the error η p = p I − p h , we have the upper bound: 2

η p V 2

1 2

η p ( T )22

L (K )

K ∈T

T h K r ∗ (t ) + t K

K ∈T

0

L2 (K )

−1 h (η p − qh ) K

L2 (K )

dt

T 1 2 ∗ 1 h J (t ) 2 h− 2 (η p − qh ) 2 dt ∀qh ∈ Y h . + t S S S L (S) L (S)

(4.12)

S ∈Γh

0

Proof. We set q = η p in (2.11) and apply (4.9)–(4.10) and orthogonality of R∗ . Then

η p 2V 2 R∗ (η p ) +

1 2

η p ( T )22

= R∗ (η p − qh ) + T =

1 2

η p ( T )22

K ∈Th K

T h K r ∗ (t ) t K

0

L (K )

K ∈Th

1 η p ( T )22 (T − t ) rk∗ (t )(η p − qh ) dx + J ∗S (t )(η p − qh ) ds dt + L (K )

0

L (K )

K ∈Th

K ∈Th

L2 (K )

−1 h (η p − qh ) K

1 η p ( T )22 + ∀qh ∈ Yh . L (K ) 2

2

S ∈Γh S

dt + L2 (K )

K ∈Th

T 1 2 ∗ 1 h J (t ) 2 h− 2 (η p − qh ) 2 dt t S s S L (S) L (S) 0

S ∈Γh

2

K ∈Th

When the techniques in [3] and Section 4.1 are used, we can obtain the following result. Theorem 6. Assume that all the conditions of Lemmas 1–5 hold. For the error η p = p I − p h , we have the a posteriori error estimate

η p V 2 C ρ .

(1) The case d = 1,

2 η p ( T )22 + C ∗ t ρ =T h K r ∗K (t ) L 2 ( K ) dt ; L (K ) T

2

K ∈Th

0

K ∈Th

(4.13a)

190

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

(2) The case d 2,

1 2 ∗ 2 η p ( T )22 + C ∗ t h K r ∗ (t )22 dt + C ∗ t h J (t ) 2 dt . ρ =T K 1 2 S S L (K ) L (K ) L (S) T

T

2

K ∈Th

K ∈Th

0

(4.13b)

S ∈Γh

0

Theorem 7. Assume that p and p h are the solutions to (2.9) and (4.8), respectively. For the error ε p = p − p h we have

ε p V 2 C

T 2

p ( T ) − p I ( T )

L 2 (Ω)

+ y − yh L2 ( Q T ) + ρ .

(4.14)

Proof. Using the triangle inequality, we have

ε p V 2 e p V 2 + η p V 2 . It follows from Theorem 4, the weighted norm and Cauchy–Schwartz inequality that

e p V 1 C

T 2

p ( T ) − p I ( T )

+ y − y h L2 ( Q T ) . L 2 (Ω) 2

By (4.13) and the two inequalities above, we obtain the inequality.

Lemma 3. Let ( y , u , p ) and ( y h , u h , p h ) be solutions to the co-state problems (2.8)–(2.9) and (3.2)–(3.3), respectively. Assume that (h (uh ) + B ∗ ph )| K u ∈ H 1 ( K u × I ) and that there exits w ∈ M h such that

T T ∗ h (u h ) + B p h , w − u U dt C h K h (u h ) + B ∗ p h H 1 ( K ) u − u h L 2 ( K u ) dt . u 0

0

K

Then

u − uh L 2 ( Q uT )) C θ + p I − ph L 2 ( Q ) ,

(4.15)

T

where θ 2 =

T 0

K

h2K |h (u h ) + B ∗ p h |2H 1 ( K ) dt. u

Proof. It follows from assumptions A2 and A4 in Section 2 and Young’s inequality that

T c u − u h 2L 2 ( Q ) uT

h (u ), u − u h dt −

0

T

h (u h ), u − u h dt

0

T −

T

∗

B p , u − u h dt +

0

B p h , u − u h dt −

0

T −

h (u h ) + B ∗ p h , w − u

=−

U

∗

dt +

h (u h ) + B p h , w − u

T

U

dt +

0

0

T

T

h (u h ) + B ∗ p h , w − u

U

0

B ∗ ( p − p h ), u − u h dt

dt +

B

∗

B ∗ p I − p h , u − u h dt

0

2

T

uT )

.

I g y

0

2

T

2

I

p − p h , u − u h dt −

2 c C θ + p I − ph L 2 ( Q ) + u − uh 2L 2 ( Q We get the desired result.

h (u h ) + B ∗ p h , u − u h dt

0

T

0

T

0

−

T

∗

− g ( y ), y I − y dt

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

191

Theorem 8. Let ( y , u , p ) and ( y h , u h , p h ) be solutions to the control problem (2.8)–(2.9) and (3.2)–(3.3), respectively. Assume that all conditions of Lemma 3 hold. Then

u − uh L 2 ( Q uT ) + y − yh V 1 + p − ph V 2

T T I y0 − y 2 p ( T ) − p I ( T ) 2 . C ξ +ρ +θ + + 0 L (Ω) L (Ω) 2

2

Proof. From Theorem 6 and Lemma 3, we have the inequality

u − uh L 2 ( Q uT ) C (θ + ρ ). Using the triangular inequality, Theorem 2, Lemma 2, Lemma 3 and Theorem 6 by the standard techniques, we have the inequality

y − yh V 1 y − y I V 1 + y I − yh V 1 T y0 − y I u − u h 2 + L ( Q uT )

C (ξ + ρ + θ) +

0 L 2 (Ω)

2

T 2

+ Cξ

y0 − y I

0 L 2 (Ω) .

We can similarly prove the inequality

p − ph V 2 p − p I V 2 + p I − ph V 2 T p ( T ) − p I ( T ) y − yh 2 + L (Ω T )

C (ξ + ρ + θ) +

2

T 2

p ( T ) − p I ( T )

+ Cρ T y0 − y I +

L 2 (Ω)

L 2 (Ω)

2

0 L 2 (Ω) .

2

Then the estimate follows.

5. Convection diffusion problem Now we consider the control problem governed by convection diffusion equation

min u∈M

g ( y ) + h(u ) dx

Ω

subjects to

⎧ ∂y ⎪ ⎪ − y + b∇ y = f + Bu (t , x) ∈ Q T , ⎨ ∂t y (x, 0) = y 0 (x) x ∈ Ω, ⎪ ⎪ ⎩ y (x, t ) = 0 x ∈ ∂Ω, t ∈ (0, T ),

(5.1)

where Ω and B have the same properties as in Section 2. b = (b1 , b2 , . . . , bd ) : Q T → R d . For simplicity we assume that div b = 0. We have known the problem (5.1) is equivalent to the following problem:

⎧ ∂y ⎪ ⎪ − y + b∇ y = f + Bu ⎪ ⎪ ⎪ ⎪ ∂t ⎪ ⎪ ⎪ ⎨ − ∂ p − p − b∇ p = g ( y ) ∂t ⎪ h (u ) + B ∗ p , w − u 0 ⎪ ⎪ U ⎪ ⎪ ⎪ ⎪ y (x, 0) = y 0 (x), p (x, T ) = 0 ⎪ ⎪ ⎩ y (x, t ) = 0, p (x, t ) = 0

(t , x) ∈ Q T , (t , x) ∈ Q T , ∀w ∈ M, x ∈ Ω, x ∈ ∂Ω, t ∈ (0, T ).

(5.2)

192

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

The weak solution of problem (5.2) is deﬁned as a triplet ( y , u , p ) such that

⎧

∂y ⎪ ⎪ − b ∇ y v dx + ∇ y ∇ v dx = ( f + Bu ) v dx ⎪ ⎪ ⎪ ∂t ⎪ ⎪ Ω Ω Ω ⎪ ⎪ ⎪ ⎨ ∂p − b∇ p q dx + ∇ p ∇ q dx = g ( y )q dx − ⎪ ∂t ⎪ ⎪Ω Ω Ω ⎪ ⎪ ⎪ ⎪ ∗ ⎪ h ( u ) + B p , w − u 0 ⎪ U ⎪ ⎩ y (x, 0) = y 0 (x), p (x, T ) = 0

∀v ∈ V , ∀q ∈ V ,

(5.3)

∀w ∈ M, x ∈ Ω.

We assume that f ∈ L 2 ( Q T ), b ∈ L 2 (0, T ; W 1,∞ (Ω))d , and y 0 ∈ L 2 (Ω) and reformulate similarly problem (5.3) as in Section 2. Using Fubini’s Theorem, we obtain that

⎧ T τ T T ⎪ ⎪ ∂y ⎪ ⎪ τ − ( T − t ) b ∇ y v dx dt + ( T − t ) ∇ y ∇ v dx dt v dx dt d ⎪ ⎪ ∂t ⎪ ⎪ ⎪0 0 Ω Ω Ω 0 0 ⎪ ⎪ ⎪ ⎪ T ⎪ ⎪ ⎪ ⎪ ⎪ = ( T − t ) ( f + Bu ) v dx dt ∀v ∈ Y , ⎪ ⎪ ⎪ ⎨ 0

Ω

⎪ T T T T ⎪ ⎪ ∂p ⎪ ⎪ ⎪ q dx dt dτ − t b∇ pq dx dt + t ∇ p ∇ q dx dt − ⎪ ⎪ ∂t ⎪ ⎪ τ ⎪ Ω Ω Ω 0 0 0 ⎪ ⎪ ⎪ ⎪ T ⎪ ⎪ ⎪ ⎪ ⎪ = t g ( y )q dx dt ⎪ ⎩ 0

∀q ∈ Y .

Ω

We introduce the bilinear maps B and B ∗ : Y × V → R,

T τ B( y , v ) = 0

B ∗ ( p , q) = −

∂y v dx dt dτ − ∂t

0 Ω

(T − t ) 0

T T 0

T

τ Ω

∂p q dx dt dτ − ∂t

b∇ y v dx dt +

(T − t ) 0

Ω

T

∇ y ∇ v dx dt , Ω

T b∇ pq dx dt +

t 0

T

0

Ω

∇ p ∇ q dx dt ,

t Ω

and the linear maps F and G : V → R

T Fu ( v ) =

(T − t ) 0

T ( f + Bu ) v dx dt ,

G y (q) =

Ω

t 0

g ( y )q dx.

Ω

The continuous problems and discrete problems can be rewritten as

B ( y , v ) = Fu ( v ) B∗ ( p , q) = G y (q) ∀ v , q ∈ Y , h (u ) + B ∗ p , w − u U 0 ∀ w ∈ M ,

(5.4) (5.5)

and

B ( y h , v h ) = Fu h ( v h ) B∗ ( p h , qh ) = G yh (qh ) ∀qh , v h ∈ Yh , h (u h ) + B ∗ p h , w h − u h U 0 ∀ w h ∈ M h . Let us establish the coercivity of the bilinear maps B and B ∗ .

(5.6) (5.7)

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

193

Theorem 9.

T

1

y 2L 2 ( Q ) T 2

B( y , y ) =

( T − t )| y |2H 1 (Ω) dt −

+

T 2

y (0)22

∀y ∈ Y ,

L (Ω)

(5.8)

0

∗

B ( p, p) =

T

1

p 2L 2 ( Q ) T 2

T

t | p |2H 1 (Ω) dt −

+

p ( T )22

∀p ∈ Y .

L (Ω)

2

(5.9)

0

Proof. Let v = y and q = p into B and B ∗ respectively. We obtain

T τ

∂y y dx dt dτ − ∂t

B( y , y ) = 0

=

0 Ω

1

T (T − t ) 0

T y dx dt −

2

2

by ∇ y dx dt +

y (0)

T L 2 (Ω)

∂p p dx dt dτ − ∂t

B ( p, p) = − 0

=

1

τ Ω

|∇ y | dx dt −

T 2

p dx dt −

2

2

p ( T )

Ω

T L 2 (Ω)

(5.10a)

∇ p ∇ p dx dt

t 0

Ω

T

by ∇ y dx dt , Ω

T b∇ pp dx dt +

t

(T − t ) 0

T 0

T

0 Ω

T T

∇ y ∇ y dx dt Ω

2

+

0 Ω

∗

(T − t ) 0

Ω

T

2

T

+

0 Ω

T 2

|∇ p | dx dt − 0 Ω

bp ∇ p dx dt .

t 0

(5.10b)

Ω

It follows from Gauss formula, assumption condition div b = 0 and boundary condition that

T (T − t ) 0

by ∇ y dx dt = 0,

(5.11a)

Ω

T bp ∇ p dx dt = 0,

t 0

(5.11b)

Ω

(5.10) and (5.11) yield the result.

2

We introduce the weighted norms in the space V 2

v V 1 =

T

1

v 2L 2 ( Q ) T 2

+

( T − t )| v |2H 1 (Ω) dt ,

(5.12)

t | v |2H 1 (Ω) dt .

(5.13)

0 2

v V 2 =

T

1

v 2L 2 ( Q ) T 2

+ 0

Assume that y I and p I satisfy

B y I , v = Fuh ( v ),

B∗ p I , q = G yh (q) ∀q, v ∈ Y .

(5.14)

Theorem 10. Let ( y , p ) and ( y I , p I ) be the solutions to (5.4) and (5.14), respectively. For the errors e y = y − y I and e y = p − p I we have

T

1

e y 2L 2 ( Q ) T 4

( T − t )|e y |2H 1 (Ω) dt u − uh 2L 2 ( Q

+

uT )

+

T 2

y 0 − y I 22

0 L (Ω) ,

0

1 4

T e p 2L 2 ( Q

T)

+ 0

2 T t |e p |2H 1 (Ω) dt y − y h 2L 2 ( Q ) + p ( T ) − p I ( T ) L 2 (Ω) , T 2

194

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

or simple forms by the weighted norm deﬁnitions:

e y 2V 1 C u − uh 2L 2 ( Q

uT )

e p 2V 2 C y − yh 2L 2 ( Q

T)

+

+

CT

y 0 − y I 22

0 L (Ω) ,

2 CT 2

p ( T ) − p I ( T )22

L (Ω)

.

Proof. It can be proved by using similar techniques used in the proof of Theorem 2 and Theorem 4.

2

We will derive an a posteriori error bound in terms of the residual

r K ( y h ) = f + Bu h −

∂ yh + yh − b∇ yh , ∂t

∂ ph r ∗K ( p h ) = g ( y h ) + + ph + b∇ ph , ∂t ∂ yh ∂ ph J S ( yh ) = , J ∗S ( p h ) = . ∂n S ∂n S Lemma 4. For the errors η y = y I − y h and η p = p I − p h , we have the upper bounds:

1 2

T η y 2L 2 ( Q

T

( T − t )|η y |2H 1 (Ω) dt = η y 2V 1

+ )

T

η y (0)22

2

T +

h K r K (t )

(T − t )

K ∈Th

0

T +

(T − t )

2

T η

2 p L2 ( Q ) T

t |η p |2H 1 (Ω) dt = η p 2V 2

+

−1 h (η y − v h ) K

L2 (K )

dt

1 1 h 2 J s (t ) 2 h− 2 (η y − v h ) 2 dt ∀ v h ∈ Yh , S S L (S) L (S)

T 2

0

η p ( T )22

L (K )

K ∈Th

T h K r K (t ) + t 0

L2 (K )

S ∈Γh

0

1

L (K )

K ∈T

0

K ∈Th

L2 (K )

−1 h (η p − qh ) K

L2 (K )

dt

T 1 2 1 h J s (t ) 2 h− 2 (η p − qh ) 2 dt ∀qh ∈ Yh . + t S S L (S) L (S) 0

S ∈Γh

Lemma 5. We have the a posteriori error estimates

η y V 1 C ξ

and

η p V 2 C ρ ,

where (1) the case d = 1,

2 T 2 hK ξ =T η y (0) L 2 ( K ) + C ( T − t ) √ r K (t ) 2 2

K ∈Th

L (K )

K ∈Th

0

2 T 2 hK ∗ √ r K (t ) ρ =T η p (T ) L2 (K ) + C t 2 2

K ∈Th

0

K ∈Th

L (K )

dt ;

dt ,

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

195

(2) the case d 2,

2 T hK 2 √ r K (t ) ξ =T η y (0) L 2 ( K ) + C 1 ( T − t ) 2 2

K ∈Th

η p ( T )22

ρ2 = T

L

K ∈Th

+ C 1∗ (K )

2 T hK √ r K (t ) t 2 0

dt + C 2

L (K )

K ∈Th

0

T

dt + C 2∗

L (K )

K ∈Th

(T − t ) 0

h S 12 J s (t )22 dt , L (S)

S ∈Γh

T h S 12 J s (t )22 dt . t L (S) 0

S ∈Γh

Theorem 11. Assume that ( y , u , p ) and ( y h , u h , p h ) are solutions to the control problems (5.4) and (5.6), respectively. Assume that all conditions of Lemma 3 hold. Then

u − uh L 2 ( Q uT ) + y − yh V 1 + p − ph V 2 T p ( T ) − p I ( T ) 2 C (θ + ρ + ξ ) + C 2

L (Ω)

+ y 0 − y 0I L 2 (Ω) .

Proof. Applying Theorem 10, Lemmas 3–5 and Young’s inequality, it can be proved by the similar techniques used in the proof of Theorem 8. 2 6. Hyperbolic problem In the section the problem we consider is the following optimal control problem governed by the hyperbolic equation: Find u : Q uT → R , Q uT = Ωu × (0, T ) such that

min u∈M

g ( y ) + h(u ) dx

Ω

subjects to

⎧

d 2 ⎪ ∂ ∂y ⎪ ⎪∂ y − ⎪ a = f + Bu ij ⎪ ⎪ ∂ xi ∂xj ⎨ ∂t2

(t , x) ∈ Q T ,

i , j =1

⎪ ⎪ ⎪ y (x, 0) = y 0 (x), ⎪ ⎪ ⎪ ⎩ y (x, t ) = 0

∂y (x, 0) = y 1 (x) x ∈ Ω, ∂t x ∈ ∂Ω, t ∈ (0, T ).

In order to discuss solution existence and uniqueness of the problem, we assume that ¯ . 1. A (x) = (ai j (x))d×d is the symmetric matrix, i.e., ai j (x) = a ji (x), and ai j (x) ∈ C 1 (Ω) 2. A (x) is the positive deﬁnite bound matrix, i.e., there exist two positive constants c 0 and c 1 such that

c 0 |ξ |2 ξ T A (x)ξ c 1 |ξ |2 , where x, ξ, ∈ R d . ∂f 3. We assume that f , ∂ t ∈ C ( Q¯ T ), y 0 ∈ Y , y 1 ∈ L 2 (Ω). The equivalent problem of the optimal control problem is

⎧

d ⎪ ∂ ∂y ∂2 y ⎪ ⎪ ⎪ − a = f + Bu i j ⎪ ⎪ ∂ xi ∂xj ∂t2 ⎪ ⎪ i , j =1 ⎪ ⎪ ⎪ ⎪

d ⎪ 2 ⎪ ∂ ∂p ⎪ ⎨∂ p − a = g ( y) i j ∂ xi ∂xj ∂t2 i , j =1 ⎪ ⎪ ⎪ ⎪ ⎪ h ( u ) + B ∗ p, w − u 0 ⎪ ⎪ ⎪ ⎪ ⎪ ∂y ⎪ ⎪ y (x, 0) = y 0 (x), (x, 0) = y 1 (x) ⎪ ⎪ ∂t ⎪ ⎩ y (x, t ) = 0

(t , x) ∈ Q T ,

(t , x) ∈ Q T , ∀w ∈ M, x ∈ Ω, x ∈ ∂Ω, t ∈ (0, T ).

196

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

The weak form of the equivalent problem is

⎧ d ⎪ ∂2 y ∂ y ∂v ⎪ ⎪ v dx + a dx = ( f + Bu ) v dx ⎪ ij ⎪ ⎪ ∂ x j ∂ xi ∂t2 ⎪ i , j = 1 ⎪ Ω Ω ⎪ ⎪Ω ⎪ ⎪ d ⎪ 2 ⎪ ∂ p ∂q ⎪ ⎨ ∂ p q dx + ai j dx = g ( y )q dx ∂ x j ∂ xi ∂t2 ⎪ Ω Ω i , j =1 Ω ⎪ ⎪ ⎪ ∗ ⎪ h (u ) + B p , w − u 0 ⎪ ⎪ ⎪ ⎪ ⎪ ∂y ⎪ ⎪ (x, 0) = y 1 (x) ⎪ y (x, 0) = y 0 (x), ⎪ ⎪ ∂t ⎩ y (x, t ) = 0

∀ v ∈ Y , a.e. t ∈ (0, T ),

∀q ∈ Y , a.e. t ∈ (0, T ), ∀w ∈ M, x ∈ Ω, x ∈ ∂Ω, t ∈ (0, T ).

By Fubini’s theorem, we change the weak form into

⎧ T τ T d ⎪ 2 ⎪ y ∂ y ∂v ∂ ⎪ ⎪ v + a dx dt d τ = ( T − t ) ( f + Bu ) v dx dt dτ ⎪ ij ⎪ ⎪ ∂ x j ∂ xi ∂t2 ⎨ 0

i , j =1

0 Ω

0

Ω

⎪ T T 2 T d ⎪ ⎪ p ∂ p ∂ q ∂ ⎪ ⎪ ⎪ q + a dx dt d τ = t g ( y )q dx dt dτ ij ⎪ ⎩ ∂ x j ∂ xi ∂t2 0

i , j =1

τ Ω

0

So we can introduce the bilinear maps B

T τ B( y , v ) = 0

0 Ω

T T

B∗ ( y , v ) =

∂2 y v dx dt dτ + ∂t2

0

τ Ω

0

∂2 p q dx dt dτ + ∂t2

0 Ω i , j =1

0

∂ y ∂v dx dt dτ , ∂ x j ∂ xi

ai j

τ Ω i , j =1

F and G .

∂ p ∂q dx dt dτ , ∂ x j ∂ xi

T Fu ( v ) =

(T − t ) 0

( f + Bu ) v dx dt , Ω

T G y (q) =

ai j

T T d

∀q ∈ L 2 (0, T ; Y ).

Ω

and B ∗ , and the linear maps

T τ d

∀ v ∈ L 2 (0, T ; Y ),

t 0

g ( y )q dx dt .

Ω

Then continuous form and semi-discretion form can be rewritten by the denotation respectively

B( y , v ) = Fu ( v ),

B∗ ( p , q) = G y (q) ∀ v , q ∈ Y

(6.1)

and

B∗ ( p h , qh ) = G yh (qh ) ∀ v h , qh ∈ Y h .

B( y h , v h ) = Fuh ( v h ),

(6.2)

We assume that y I and p I satisfy

B y I , v = Fu h ( v )

B∗ p I , q = G yh (q) ∀ v , q ∈ Y .

(6.3)

Let v ∈ Y . It follows from (6.1) and (6.3) that

B y I , v − B( y h , v ) = Fuh ( v ) − B( y h , v ) = R( v ), where R( v ) is the residual, which has the orthogonality, i.e.,

R( v h ) = 0 ,

∀vh ∈ Y h .

(6.4)

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

197

We deﬁne local residuals and jumps,

d ∂ 2 yh ∂ ∂ yh r K ( y h ) = f + Bu h − − ai j , ∂ xi ∂xj ∂t2 i , j =1

d ∂ ∂ ph ∂ 2 ph , r¯ K ( p h ) = g ( y h ) − − ai j ∂ xi ∂xj ∂t2 i , j =1

d

J S ( yh ) = −

∂ yh i n , ∂xj S

ai j

i , j =1

T R( v ) =

(T − t )

ai j

i , j =1

∂ ph i n , ∂xj S

rk (t ) v (t ) dx +

J S (t ) v (t ) ds) dt .

S ∈Γh S

K ∈Th K

0

d

¯J S ( ph ) = −

In the sequel, we use the notation v˙ = ∂∂vt . Theorem 12. Assume that ( y , p ) and ( y I , p I ) are the solutions to (6.1) and (6.3), respectively. For the errors e y = y − y I and e p = p − p I we have

1

˙e y 2L 2 ( Q ) T 4

c0

+

T |e y |2H 1 (Ω) dt u − uh 2L 2 (Ω

2

uT )

+

T C1 T e˙ y (0)22 e y (0) 2 1 , + L (Ω) H (Ω) 2 2

0

1

˙e p 2L 2 ( Q ) T 4

c0

+

T |e p |2H 1 (Ω) dt y − yh 2L 2 (Ω) +

2

T C1 T e˙ p ( T )22 e p ( T ) 2 1 . + L H (Ω) (Ω) 2 2

0

Proof. Let v = e˙ y and q = e˙ p in (6.1) and (6.3), respectively, and use integration by parts in time. Then we get

T τ B(e y , e˙ y ) = 0

=

1

i , j =1

0 Ω

T

d ∂ 2e y ∂ e˙ y ∂e y e˙ y + ai j dx dτ dt ∂ xi ∂xj ∂t2

τ

∂ e˙ 2y

2

∂t 0

+

i , j =1

0 Ω

1

˙e y 2L 2 ( Q 2

T

− )

d

∂ ∂e y ∂e y ai j dx dτ dt ∂ t ∂ xi ∂ x j

T e˙ y (0)22 + c0 L (Ω) 2

T |e y |2H 1 (Ω) dt −

c1 T e y (0) 1 . H (Ω) 2

0

Furthermore, we have

B(e y , e˙ y ) = (u − uh , e˙ y ) u − uh ˙e y . From the above two formulas and Young’s inequality, we have

1

˙e y 2L 2 ( Q ) T 4

+

c0

T |e y |2H 1 (Ω) dt u − uh 2L 2 (Ω ) +

2

u

2 C1 T T e y (0) 2 1 + e˙ y (0) L 2 (Ω) . H (Ω) 2 2

0

2

Applying the same method, we obtain the second inequality. We introduce the weighted semi-norms in the space V 2

| v |V 1 =

1

v˙ 2L 2 ( Q ) T 2

T + c0

(T 0

− t )| v |2H 1 (Ω) dt ,

2

| v |V 2 =

1

v˙ 2L 2 ( Q ) T 2

T t | v |2H 1 (Ω) dt .

+ c0 0

198

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

Theorem 13. Assume that ( y I , p I ) and ( y h , p h ) are the solutions to (6.2) and (6.3), respectively. For the errors η p = p I − ph we have

|η y |2V 1

T 2

η y (0)2 1

H (Ω)

+

C1 T

K ∈Th

+2

L2 (K )

K

+ L2 (K )

1 h 2 J S (0) ηh (0) − v h (0) 2 K L (S)

S ∈Γh

h K r K − ( T − t )˙r K (t ) 2 h−1 (η y − v h ) 2 dt K L (K ) L (K )

1/2 −1/2 h J − ( T − t ) ˙J S (t ) L 2 ( S ) h S (η y − v h ) L 2 ( S ) dt ∀ v h ∈ C 1 [0, T ], Y h , S

+

S ∈Γh

0

T 2

L (Ω)

−1 h η y (0) − v h (0)

K ∈Th

0

T

|η p |2V 2

η˙ y (0)22

2

h K r K (0) +T T

η y = y I − yh and

η p ( T )2 1

H (Ω)

+

C1 T 2

h K r¯ K ( T ) +T K ∈Th

η˙ p ( T )22

L2 (K )

L (Ω)

−1 h ηh ( T ) − qh ( T ) K

T

h K r¯ K − t ∂ r¯ K (t ) +2 ∂t K ∈Th

0

T ¯ 1/2 h ¯J − t ∂ J S (t ) + S ∂t S ∈Γh

0

L2 (K )

L2 ( S )

1 h 2 ¯J S ( T ) ηh ( T ) − qh ( T )

+ L2 (K )

−1 h (η p − qh ) K

K

S ∈Γh

L2 (K )

−1/2 h (η p − qh ) S

L2 ( S )

L2 ( S )

dt

∀qh ∈ C 1 [0, T ], Y h .

dt

Proof. We can see the ﬁrst inequality in [3]. We prove only the second inequality. From Theorem 12, we similarly obtain that ∗

B (η p , η˙ p )

1

2

˙ p 2L 2 ( Q ) T

η

T

2 − η˙ p ( T ) 2

L (Ω)

+ c0

|η

2 p | H 1 (Ω) dt

−

c1 T 2

η p ( T )

H 1 (Ω)

.

(6.5)

0

It follows from (6.1)–(6.3), integration by parts and Cauchy–Schwartz inequality that

B∗ (η p , η˙ p ) = R(η˙ p ) = R(η˙ p − q˙ h )

=

T t

K ∈Th K

0

= −T

r¯ K (t )(η˙ p − q˙ h ) dx +

¯J S (t )(η˙ p − q˙ h ) ds dt

S

S

¯J S ( T ) η p ( T ) − qh ( T ) ds r¯ K ( T ) η p ( T ) − qh ( T ) dx +

K ∈Th K

+

S ∈Γh S

T

r¯ K (t ) − t

0

S ∈Γh S

K ∈Th K

h K r¯ K ( T ) T K ∈Th

¯ ∂ r¯ K (t ) ¯J S (t ) − t ∂ J (t ) (η p − qh ) ds dt (η p − qh ) dx + ∂t ∂t

L2 (K )

−1 h ηh ( T ) − qh ( T ) K

T

h K r¯ K − t ∂ r¯ K (t ) + ∂t 0

K ∈Th

S ∈Γh

L2 (K )

T T ¯ 1/2 h ¯J − t ∂ J S (t ) + S ∂t 0

1 2 ¯ + h K J S ( T ) ηh ( T ) − qh ( T ) L 2 ( S ) L2 (K )

S ∈Γh 0

The second result follows from (6.5) and (6.6).

−1 h (η p − qh ) K

L2 ( S )

2

L2 (K )

−1/2 h (η p − qh ) S

dt

L2 ( S )

dt .

(6.6)

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

199

Lemma 6. We have a posteriori error estimates of the form

|η y |V 1 ξ,

| η p |V 2 ρ ,

where

2 ξ 2 = T η y (0) L 2 ( Q

T

1 ∗ h K r K (0)22 + T C ∗∗ h 2 J S (0) 2 η y (0) 2 1 + T ( c + 1 ) + T C 1 S H (Ω) L (K ) ) L (Ω) S ∈Γh

K ∈Th

T T 1/2 2 2 ∗ ∗∗ h +C h K r K − ( T − t )˙r K (t ) L 2 ( K ) dt + C J − ( T − t ) ˙J S (t ) L 2 ( S ) dt , S 0

K ∈Th

0

2

2

ρ 2 = T η p ( T )L 2 ( Q T ) + T (c1 + 1) η p ( T ) H 1 (Ω) + T C ∗ + C∗

T 0

2

h K r¯ K − t ∂ r¯ K (t ) 2 ∂t

dt + C ∗∗

L (K )

K ∈Th

S ∈Γh

h K r¯ K ( T )22 K ∈Th

T 0

1 h 2 ¯J S ( T ) 2 S L (S)

+ T C ∗∗

L (K )

S ∈Γh

2 1/2

¯ h ¯J − t ∂ J S (t ) 2 S ∂t

dt .

L (S)

S ∈Γh

Proof. We can ﬁnd the ﬁrst inequality in [3] and only prove the second inequality. Let qh = πhα η p = Clement interpolation in the second inequality of Lemma 5. Using Cauchy inequality, we have

T ˙ p 2L 2 ( Q ) T

η

+ c0

t |η

2 p | H 1 (Ω) dt

2 T η p ( T ) 2

L (Ω)

0

+ c ∗∗

H (Ω)

+ 2T c ∗

S ∈Γh

c

∗

S ∈Γh

1/2

2

h K r¯ K − t ∂ r¯ K (t ) 2 ∂t

T 0

dt

L (K )

K ∈Th

1/2 T

2 T ¯ 1/2 ∗∗ h ¯J − t ∂ J S (t ) + C 2 S ∂t 0

L

+ C ∗∗ T

We obtain the desired results.

0

|η p | H 1 ( K ) dt

K ∈Th

2 h K r¯ K ( T )22 + (c 1 + 1) T η p ( T ) H 1 (Ω) + T C ∗ L (K ) (Ω)

1 h 2 ¯J S ( T ) 2 + C ∗ S L (S)

S ∈Γh

K ∈Th

T 0

2 h K r¯ K − t ∂ r¯ K (t ) 2 ∂t

S ∈Γh

L (S)

dt +

dt

L (K )

K ∈Th

2 T ¯ 1/2 h ¯J − t ∂ J S (t ) 2 S ∂t 0

dt

L (S)

S ∈Γh

2 T η p ( T ) 2

+C

L (K )

K ∈Th

∗∗

h K r¯ K ( T )22

1/2 1/2 1 h 2 ¯J S ( T )22 η p ( T ) 2 S L (S) L (S)

+2

2 + c 1 T η p ( T ) 1

c0

T |η p |2H 1 (Ω) dt .

2 0

2

Remark 1. In proof of Lemma 6, we aren’t concerned with difference of interpolations in different dimension space, i.e., interpolation operator πhα is different in the different dimension space. For example, for the case d = 1, interpolation operator is the piecewise linear Lagrange interpolation. However, for the case d 2, we use Clement interpolation, because ¯ and Lagrange interpolation cannot be used. H 1 (Ω) C (Ω)

¯ for the case d 2, the jump terms in the boundary Remark 2. Because we use linear interpolation and H 1 (Ω) C (Ω) of the inner element may not be vanished, but the case d = 1, the jump is vanished in the nodes. So the result in the lemma should be changed. Namely, for the case d = 1, c ∗ = 0. For the case d 2, c ∗∗ = C = 0, which only depends on c 1 and c 0 .

200

C. Xiong, Y. Li / Applied Numerical Mathematics 61 (2011) 181–200

Theorem 14. Let ( y , u , p ) and ( y h , u h , p h ) be solutions to (6.1) and (6.2), respectively. Assume that conditions of Lemma 3 hold. Then

T p ( T ) − p I ( T ) 1 εu L 2 ( Q uT ) + |ε y |V 1 + |ε p |V 2 C (θ + ξ + ρ ) + C + y 0 − y 0I H 1 (Ω) H (Ω) 2 + y˙ 0 − y˙ 0I L 2 (Ω) + p˙ ( T ) − p˙ I ( T ) L 2 (Ω) where θ is deﬁned in Lemma 3. Proof. The proof follows from Lemma 3, Theorem 12, Lemma 6 and Young’s inequality.

2

Acknowledgements We are indebted to our advisors Prof. Qun Lin and Prof. Tao Tang for their help. References [1] S. Adjerid, J.E. Flaherty, Y.J. Wang, A posteriori error estimates with the ﬁnite element of lines for one dimensional parabolic systems, Tech. report 91-1 Troy, NY, Dept. of Computer Science, Rensselaer Polytechnic Institute, 1991. [2] I. Babuska, S. Ohnimus, A posteriori error estimation for the semidiscrete ﬁnite element method of parabolic differential equations, Preprint, TICAM, Univ. of Texas at Austin, 1999. [3] I. Babuska, Milsolav Feistauer, Pavel Solin, On one approach to a posteriori error estimates for evolution problems solved by the methods of lines, Numer. Math. 89 (2001) 225–256. [4] R.E. Bank, A. Weiser, Some a posteriori error estimators for elliptic partial differential equations, Math. Comp. 44 (1985) 283–301. [5] R. Becker, H. Kapp, Optimization in PDE models with adaptive ﬁnite element discretization, Tech. report 98-20 (SFB 359) IWR, University of Heidelberg, Germany, 1998. [6] R. Becker, R. Rannache, A feed-back approach to error control in ﬁnite element methods: Basic analysis and examples, Preprint 96-52 (SFB 359) Heidelberg, 1996. [7] R. Becker, H. Kapp, R. Rannacher, Adaptive ﬁnite element methods for optimal control of partial differential equations: Basis concept, SIAM J. Control Optim. 1 (2000) 113–132. [8] M. Beiterman, I. Babuska, An adaptive method of lines with error control for parabolic equations of the reaction-diffusion type, J. Comput. Phys. 63 (1986) 33–66. [9] P. Ciarlet, The Finite Element Method for Elliptic Problem, North-Holland, Amsterdam, 1978. [10] K. Eriksson, C. Johnson, Adaptive ﬁnite element methods for parabolic problems I: A linear model problem, SIAM J. Numer. Anal. 28 (1991) 43–77. [11] K. Eriksson, C. Johnson, Adaptive ﬁnite element methods for parabolic problems IV: Nonlinear problems, SIAM J. Numer. Anal. 32 (1995) 1729–1749. [12] K. Eriksson, C. Johnson, Adaptive ﬁnite element methods for linear elliptic problems, Math. Comp. 50 (1998) 361–383. [13] F.S. Falk, Approximation of a class of optimal control problems with order of convergence estimates, J. Math. Anal. Appl. 44 (1973) 28–47. [14] D.A. French, J.T. King, Approximation of an elliptic control problem by the ﬁnite element method, Numer. Funct. Anal. Optim. 12 (1991) 299–314. [15] H. Fu, H. Rui, A priori error estimates for optimal control problems governed by transient advection-diffusion equation, J. Sci. Comput. 38 (2009) 290–315. [16] R.M. Furzeland, J.G. Verwer, P.A. Zegeling, A numerical study of three moving grid methods for one dimension partial differential equations which are based the methods of lines, J. Comput. Phys. 89 (1990) 349–388. [17] G. Knowles, Finite element approximation of parabolic time optimal control problems, SIAM J. Control Optim. 20 (1982) 414–427. [18] W. Liu, H. Ma, T. Tang, N. Yan, A posteriori error estimates for discontinuous Galerkin time-stepping method for optimal control problems governed by parabolic equations, SIAM J. Numer. Anal. 42 (3) (2004) 1032–1061. [19] W.B. Liu, D. Tiba, Error estimates for the ﬁnite element approximation of a class of nonlinear optimal control problems, Numer. Funct. Anal. Optim. 22 (2001) 953–972. [20] W.B. Liu, N.N. Yan, A posteriori error estimator for convex boundary control problems, SIAM J. Numer. Anal. 39 (2001) 73–99. [21] K. Malanowski, Convergence of approximations vs. regularity of solutions for convex, control constrained optimal controlsystem, Appl. Math. Optim. 8 (1) (1982) 69–95. [22] D. Meidner, B. Vexler, Adaptive space–time ﬁnite element methods for parabolic optimization problems, SIAM J. Control Optim. 4 (1) (2007) 116–142. [23] P.K. Moore, A posteriori error estimation with ﬁnite element semi- and fully distribute methods for nonlinear parabolic equations in one space dimension, SIAM J. Numer. Anal. 31 (1994) 149–169. [24] K. Segeth, Grid adjustment based on a posteriori error estimates, Appl. Math. 38 (1993) 488–504. [25] K. Segeth, A posteriori error estimates for parabolic differential systems solved by the ﬁnite element of lines, Appl. Math. 39 (1994) 415–443. [26] K. Segeth, A posteriori error estimates with the ﬁnite element of lines for a nonlinear parabolic equations in one space dimension, Preprint, Mathematical Institute of the Academy of Sciences, Prague, 1998.

Applied Numerical Mathematics 61 (2011) 201–215

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

H 1 -second order convergent estimates for non-Fickian models S. Barbeiro, J.A. Ferreira ∗ , L. Pinto CMUC, Department of Mathematics, University of Coimbra, 3001-454 Coimbra, Portugal

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 5 January 2010 Received in revised form 8 September 2010 Accepted 22 September 2010 Available online 29 September 2010 Keywords: Non-Fickian models Finite difference method Piecewise linear ﬁnite element method Supraconvergence Superconvergence

In this paper we study numerical methods for integro-differential initial boundary value problems that arise, naturally, in many applications such as heat conduction in materials with memory, diffusion in polymers and diffusion in porous media. Here, we propose ﬁnite difference methods to compute approximations for the continuous solutions of such problems. We analyze stability and study convergence for those methods. Supraconvergent estimates are obtained. As such methods can be seen as lumped mass methods, our supraconvergent result corresponds to a superconvergent property in the context of ﬁnite element methods. Numerical results illustrating the theoretical results are included. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

1. Introduction We consider the semi-discretization of the integro-differential equation

∂u (t ) + Au (t ) = ∂t

t B (s, t )u (s) ds + f (t ),

t ∈ (0, T ],

(1)

0

where u (t ) denotes a function deﬁned on [a, b] when t ∈ [0, T ] is ﬁxed, A and B (s, t ) represent the following operators

∂ ∂u ∂ a2 (x) (x, t ) + a1 (x)u (x, t ) + a0 (x)u (x, t ), ∂x ∂x ∂x ∂ ∂u ∂ B (s, t )u (x, t ) = − b2 (s, t , x) (x, t ) + b1 (s, t , x)u (x, t ) + b0 (s, t , x)u (x, t ), ∂x ∂x ∂x Au (x, t ) = −

for x ∈ (a, b), s, t ∈ (0, T ]. We assume that (1) is complemented with homogeneous Dirichlet boundary conditions and an initial condition u (x, 0) = u 0 (x), x ∈ [a, b]. For the coeﬃcient functions we assume some smoothness that will be speciﬁed later in Section 2. Integro-differential equations of type (1) arise in several applications as for instance transport in heterogeneous media [7,22,30] and heat propagation in materials with memory [23]. Transport in media or heat propagation phenomena are traditionally modeled by the convection-diffusion equation

∂u + ∇. J = f , ∂t

*

Corresponding author. E-mail addresses: [email protected] (S. Barbeiro), [email protected] (J.A. Ferreira), [email protected] (L. Pinto).

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.09.005

(2)

202

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

where u denotes the concentration, J represents the mass ﬂux and f denotes the reaction term. In (2) J can be expressed as

J = J adv + J dif + J dis ,

(3)

J adv = uv

(4)

where

represents the advection mass due to the ﬂuid velocity v,

J dif = − D m ∇ u

(5)

denotes the mass ﬂux due to molecular diffusion, being D m the effective molecular diffusion coeﬃcient, and J dis satisﬁes the so-called Fick’s law

J dis = − D d ∇ u and represents the dispersive mass ﬂux associated with random deviations of ﬂuid velocities within the porous space from their macroscopic value v. In the deﬁnition of J dis , D d denotes the dispersive tensor. Combining (2) with (3) we obtain the parabolic equation

∂u + ∇.(uv ) = ∇. ( D m I + D d )∇ u + f , ∂t

(6)

where I is the identity tensor. Eq. (6) gives good accurate results in laboratory environment for perfectly homogeneous media. Nevertheless when nonhomogeneous media are considered, deviations of the Fickian behavior are observed. In this case the main sources for such deviations are the small-scale and large-scale heterogeneities [22,30]. From the theoretical point of view, as Eq. (6) is of parabolic type, it induces inﬁnite propagation speed. In order to circumvent the pathologic behavior of the convection-diffusion-reaction (6) several approaches were proposed in the literature. Ref. [30] summarizes some of them. One possible approach, that leads to a simpliﬁed version of Eq. (1), is to consider that the dispersive mass ﬂux satisﬁes the following differential equation

τ

∂ J dis (x, t ) + J dis (x, t ) = − D d ∇ u (x, t ), ∂t

(7)

where τ is a delay parameter [28]. We remark that the left hand side of (7) is a ﬁrst order approximation of the left hand side of

J dis (x, t + τ ) = − D d ∇ u (x, t ), which means that the dispersion mass ﬂux at the point x at the time t + τ depends on the gradient of the concentration at the same point but at a delayed time. From (2)–(7) and considering non-reactive ﬂows, we obtain the hyperbolic equation

∂ ∂ ∂ 2u 1 ∂u 1 1 1 + ∇. (uv ) + + ∇.(uv ) = ∇.( D m ∇ u ) + ∇.( D m ∇ u ) + ∇.( D d ∇ u ). 2 ∂t τ ∂t τ ∂t τ τ ∂t

(8)

To avoid the mixed derivatives that arise in Eq. (8) we point out that (7) leads to

J dis (t ) = −

1

t

τ

e−

t −s

τ

D d ∇ u (s) ds,

(9)

0

provided that J dis (0) = 0. Combining the partition (3), where J adv , J dif and J dis are given by (4), (5) and (9), respectively, with (2) we obtain the integro-differential equation

∂u 1 + ∇.(uv ) − ∇.( D m ∇ u ) = ∂t τ

t

e−

t −s

τ

∇.( D d ∇ u )(s) ds + f .

(10)

0

This equation is an example of the integro-differential equation (1). Initial boundary value problems deﬁned with integro-differential equations of type (1) have been studied numerically. We mention without being exhaustive [26,27,33,37] for the study of the semi-discrete approximations for the solution when ﬁnite element method is considered. Generally, in those papers, it is shown that several results known for the semidiscrete approximations for the solution of Fickian parabolic problems also hold for the semi-discrete approximation for the solution of (1). For instance, it is established for the two-dimensional version of (1) that, under convenient assumptions

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

203

on the partition of the domain, the semi-discrete approximation deﬁned using piecewise linear ﬁnite element method is second order convergent with respect to the L 2 -norm and it is ﬁrst order convergent with respect to the H 1 -norm. These results were also considered in [31] for a semi-discrete lumped mass approximation but with respect to discrete norms and assuming that the solution of the continuous problems are smooth enough. Second order estimates for the semi-discrete approximation with respect to the L 2 -norm were obtained for ﬁnite volume approximations in [13] and [14] provided that the regularity with respect to space of the exact solution is u (t ) ∈ H 3 and t ∂u ( u ( s ) + 3 0 ∂ t (s)3 ) ds < ∞, t ∈ [0, T ]. The previous smoothness assumptions were weakened in [32] where second order

estimates were also obtained. Here the authors established an upper bound for the L 2 -norm of the error of the ﬁnite volume t approximation provided that the terms u (t )2 , 0 (u (s)22 + s2 ∂∂ut (s)22 ) ds, t ∈ [0, T ], are ﬁnite. Integro-differential problems deﬁned by Eq. (1) can be rewritten as equivalent systems composed by a partial differential equation involving only a time derivative and an integro-differential equation involving only partial derivatives with respect to the space variables and the integral term. This approach was used for instance in [15] where mixed ﬁnite element methods were studied. Finite difference methods (FDM) for initial boundary value problems (IBVP) deﬁned by (1) were considered by the authors recently in [1,5,6,17], where schemes presenting the same qualitative behavior of the correspondent continuous models with respect to stability were proposed. In [17], Eq. (1) was considered with the coeﬃcient functions deﬁned by a2 = a1 = a0 = 0, s−t

b1 = b0 = 0 and b2 (s, t , x) = τ e − τ and it was shown that the character of (1) is related with both parabolic and hyperbolic type. Applications of integro-differential models on drug release were considered in [2,3,16,17]. The aim of the present paper is to study a semi-discretization for the IBVP (1) with homogeneous Dirichlet boundary conditions. The method that we propose can be obtained combining a piecewise linear ﬁnite element semi-discretization with a quadrature rule in space. This method can be seen as a lumped mass method. The stability and convergence analysis of the semi-discrete solution will be presented. Concerning the error estimates, we establish a second order convergence estimate with respect to both L 2 -norm and H 1 -norm. This last result shows that our method possesses a supraconvergent behavior, that is, the convergence order is greater than the order of the truncation error. Supraconvergent ﬁnite difference schemes have been considered in the literature for elliptic and parabolic equations. Without being exhaustive we mention [4,10–12,18–21,24,25,29]. In Section 4 we prove that the L 2 -norm of the gradient of the error is second order convergent, being this property known as supercloseness of the gradient [34]. The results obtained in [4] play a central role in the proof of the convergence result. We point out that second order estimates with respect to H 1 -norm were also obtained, in [8], for a ﬁnite element approximation based on piecewise quadratic elements instead of piecewise linear elements. The paper is organized as follows. In Section 2 we introduce the method, the basic deﬁnitions and the notation used. Section 3 focuses on the stability analysis of the semi-discrete solution. The convergence analysis is presented in Section 4. In Section 5 we illustrate the second order convergence of our method with some numerical results. Finally, in Section 6, we provide some conclusions. Finally, we remark that our future goal is to analyze fully discrete methods for initial boundary value problems deﬁned with integro-differential equations of type (1) as presented, for instance, in [1,3,6,9]. β

2. A fully semi-discrete Galerkin approximation We start with the Galerkin formulation of our IBVP and its discretization by linear ﬁnite elements with quadrature. By H 0r (a, b), r ∈ N0 , we denote the usual Sobolev spaces with the usual norms .r , r ∈ N0 being H 00 (a, b) = L 2 (a, b). We consider in L 2 (a, b) the usual inner product (. , .)0 . By L 2 (0, T ; H 0r (a, b)) we denote the space of Bochner-measurable functions v : [0, T ] → H 0r (a, b) such that

T

v (t )2 dt

(11)

r

0

is ﬁnite. Lets consider the dual space of L 2 (0, T ; H 01 (a, b)) given by L 2 (0, T ; H −1 (a, b)) where H −1 (a, b) denotes the dual space of H 01 (a, b). We deﬁne

dg W (0, T ) = g ∈ L 2 0, T ; H 01 (a, b) such that ∈ L 2 0, T ; H −1 (a, b) , dt

which is a Hilbert space (see Theorem 25.4 of [36]). For f ∈ L 2 (0, T ; H −1 (a, b)) and u 0 ∈ L 2 (a, b), we consider the following variational formulation of our problem: ﬁnd u ∈ W (0, T ) such that

⎧ ⎪ ⎪ du ⎨

t

(t ), v + a u (t ), v =

⎪ dt ⎪ ⎩ u (0) = u 0 ,

b s, t , u (s), v ds + f (t ), v 0

0

a.e. in (0, T ), for all v ∈ H 01 (a, b),

(12)

204

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

where . , . denotes the duality pairing between H −1 and H 01 (a, b), a(. , .) and b(s, t , . , .) are the bilinear forms deﬁned by

a( v , w ) = a2 v , w

0

− a1 v , w 0 + (a0 v , w )0 for v , w ∈ H 01 (a, b),

(13)

and

b (s, t , v , w ) = b 2 (s, t ) v , w

0

− b1 (s, t ) v , w 0 + b0 (s, t ) v , w 0 for v , w ∈ H 01 (a, b).

(14)

The coeﬃcient functions of the integro-differential equation (1) are assumed to be smooth enough, e.g., a2 , b2 (s, t ) ∈ C [a, b], a0 , a1 , b0 (s, t ), b1 (s, t ) ∈ W 2,∞ (a, b), for s, t ∈ (0, T ]. The discretization of (12) is obtained in the following way. We ﬁrst introduce in [a, b] the grid

Ih := {a = x0 < x1 < · · · < xN −1 < xN = b}, where h is the vector of mesh-sizes h j = x j − x j −1 , j = 1, . . . , N. By W h,0 := {u h , v h , w h , . . .} we denote the space of realvalued grid functions deﬁned on Ih being null on {a, b}. In W h,0 we introduce the inner product

( v h , w h )h =

N −1

for v h , w h ∈ W h,0 ,

h i +1/2 v h (xi ) w h (xi ),

(15)

i =1

where h i +1/2 = (h i + h i +1 )/2. The norm induced by the inner product (15) is denoted by .h . In W h,0 we also consider the discrete H 1 -norm

2 w h 21 = w h h2 + ( P h w h ) 0 ,

w h ∈ W h,0 ,

(16)

N

where P h denotes the piecewise linear interpolation operator, P h : W h,0 → C [a, b], P h v h (x) = i =0 v h (xi )φi (x), x ∈ [a, b], being {φi , i = 0, . . . , N } the usual base of hat functions. The discrete version of L 2 (0, T ; H 01 (a, b)) is denoted by L 2 (0, T ; W h,0 ) and it is the space of abstract functions w h : [0, T ] → W h,0 such that

T

w h (t )2 dt

(17)

1

0

is ﬁnite. Let R h be the operator of pointwise restriction to the grid Ih . We now introduce the approximation of the bilinear forms a(. , .) and b(s, t , . , .). By ah (. , .) and bh (s, t , . , .) we denote the following bilinear forms

ah ( v h , w h ) = M (a2 )( P h v h ) , ( P h w h )

0

− M P h ( R h a1 v h ) , ( P h w h ) 0 + ( R h a0 v h , w h )h ,

(18)

for v h , w h ∈ W h,0 , and

bh (s, t , v h , w h ) = M b2 (s, t ) ( P h v h ) , ( P h w h )

0

− M P h R h b1 (s, t ) v h , ( P h w h ) 0 + R h b0 (s, t ) v h , w h h , (19)

for v h , w h ∈ W h,0 . In (18) and (19) we use the notation M (q)(x) = q(xi +1/2 ) for x ∈ [xi , xi +1 ], i = 0, . . . , N − 1. We assume that ah (. , .) is continuous

ah ( v h , w h ) αc P h v h 1 P h w h 1 ,

for all v h , w h ∈ W h,0 ,

(20)

and elliptic in the sense that

ah ( v h , v h ) αe P h v h 21 ,

for all v h ∈ W h,0 .

(21)

We also suppose that bh (s, t , . , .) is bounded uniformly with respect to s, t, that is,

bh (s, t , v h , w h ) bc P h v h 1 P h w h 1 ,

for all v h , w h ∈ W h,0 , s, t ∈ [0, T ].

(22)

Let g be a function deﬁned on [a, b]. We denote by ( g )h the grid function

( g )h (xi ) =

1 h i +1/2

xi +1/2

g (x) dx,

i = 1, . . . , N − 1,

(23)

xi −1/2

( g )h (x0 ) = ( g )h (xN ) = 0, where xi +1/2 = xi +

h i +1 , 2

xi −1/2 = xi −

hi 2

.

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

205

Let W h∗,0 be the dual space of W h,0 . Considering

du h Wh (0, T ) = w h ∈ L 2 (0, T ; W h,0 ) such that ∈ L 2 0, T ; W h∗,0 , dt

the semi-discrete variational problem has the form: ﬁnd u h ∈ Wh (0, T ) such that

⎧ ⎪ ⎪ ⎨ duh ⎪ ⎪ ⎩

dt

t

+ ah uh (t ), v h =

(t ), v h h

bh s, t , u h (s), v h ds +

f (t ) h , v h

h

,

a.e. in (0, T ) for all v h ∈ W h,0 ,

(24)

0

u h (0) = u 0,h ,

where . , . h denotes the duality paring between W h,0 and W h∗,0 , u 0,h ∈ W h,0 is an approximation of u 0 . The semi-discrete variational problem (24) is equivalent to a standard ﬁnite semi-discretization of (1) on non-uniform grids which we derive in what follows. We use the divided differences

( D c v h )(xi ) =

v h ( x i +1 ) − v h ( x i −1 ) x i +1 − x i −1

( D v h )(xi ) =

,

v h (xi +1/2 ) − v h (xi −1/2 ) xi +1/2 − xi −1/2

and

v h ( x i +1 ) − v h ( x i )

( D v h )(xi +1/2 ) =

x i +1 − x i

,

where v h (xi +1/2 ) is used as far as it makes sense. Now choosing v h ∈ W h,0 to vanish in all but one grid point in Ih \ {a, b} and collecting the terms arising in (24), we obtain for u h ∈ Wh (0, T ), the following ordinary differential problem

⎧ ⎪ ⎪ ⎨ duh

t (t ) + A h uh (t ) =

dt ⎪ ⎪ ⎩ u h (0) = u 0,h ,

B h (s, t )u h (s) ds + ( f (t ))h ,

(25)

0

where

A h v h = − D (a2 D v h ) + D c (a1 v h ) + a0 v h and

B h (s, t ) v h = − D b 2 (s, t ) D v h + D c b 1 (s, t ) v h + b 0 (s, t ) v h . We remark that P h u h is an approximation for the weak solution deﬁned by (12), being u h the ﬁnite difference solution deﬁned by (25). 3. The stability analysis In what follows we establish the stability of the solution u h deﬁned by (25) with respect to perturbations of the initial condition u 0,h . In the analysis, a stronger regularity assumption on u h deﬁned by (25) is required. In order to specify the required regularity for u h we introduce the space C m ([0, T ]; W h,0 ), m ∈ N0 , of abstract functions v h : [0, T ] → W h,0 such that

dm v h dt m

: [0, T ] → W h,0 is continuous. Here we consider in W h,0 the norm .h induced by the inner product (15).

Theorem 1. Let us suppose that ah (. , .) and bh (s, t , . , .) satisfy (21) and (22), respectively. If the solution u h of (25) is in C 1 ([0, T ]; W h,0 ) then

uh (t )2 +

t

h

t P h uh (s)2 ds e Ct C uh (0)2 + f (s)2 e −C s ds , h 1 0

0

t ∈ [0, T ],

(26)

0

with

b2 T

max 1, 2c 2 C= min{1, 2(αe − 2 )}

(27)

for all = 0 such that

αe − 2 > 0.

(28)

206

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

Proof. Taking in (24) v h = u h (t ) we establish

t

1 d 2 dt

uh (t )2 + αe P h uh (t )2 bc h

P h uh (s) P h uh (t ) ds + 1 f (t ) 2 + uh (t )2 . h 1 1 h h

1

2

0

As we have

t

P h uh (s) P h uh (t ) ds 1 1 1 2

t

4

0

for all

P h uh (s) ds 1

2

2 + 2 P h uh (t )1 ,

0

= 0, we deduce d

2 uh (t )2 + 2 αe − 2 P h uh (t )2 bc h

dt

1

t

2 2

P h uh (s) ds 1

2

2 2 + f (t ) h h + uh (t )h .

(29)

0

Using the inequality

t

P h uh (s) ds 1

2

t T

0

P h uh (s)2 ds 1

0

in (29) and integrating from 0 to T we get

uh (t )2 + 2(αe − 2 )

t

h

P h uh (s)2 ds

t

1

0

f (s) 2 ds + uh (0)2 h h h

0

+

bc2 T 2

t s

t

0

uh (s)2 ds. h

1

2 0

Choosing

P h uh (μ)2 dμ ds +

(30)

0

satisfying (28) we obtain

uh (t )2 +

t

h

P h uh (s)2 ds C

t

( f (s))h 2 ds + uh (0)2 h

1

0

0

t s

+C 0

h

P h uh (μ)2 dμ + uh (s)2 ds h 1

(31)

0

with C deﬁned by (27). Finally considering the Gronwall lemma in (31) we conclude (26).

2

Theorem 2. Let us suppose that ah (. , .) satisﬁes (21), bh (s, t , . , .) satisﬁes (22),

bh (t , t , v h , v h ) βe P h v h 21 , and

for all v h ∈ W h,0 ,

∂ bh ( s , t , v , w ) h h βd P h v h 1 P h w h 1 , ∂t

(32)

for all v h , w h ∈ W h,0 , s, t ∈ [0, T ].

(33)

If the solution u h of (25) is in C 1 ([0, T ]; W h,0 ), then

t t duh 2 2 P h uh (t )2 ds (s) ds + P h uh (t ) 1 + 1 h

0

e

0

max{1, αc }

Ct

min{1, αe − η2 , 2(βe − 2 )}

for and η such that

P h uh (0)2 +

t

1

0

f (s)2 e −C s ds ,

t ∈ [0, T ],

(34)

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

αe − η2 > 0,

207

βe − 2 > 0,

(35)

with

b2 T

β2 T

max ηc 2 , 2d 2 C= . min{1, αe − η2 , 2(βe − 2 )}

(36) du h (t ) dt

Proof. Considering in (24) v h replaced by

we obtain

t duh 2 duh du h du h dt (t ) + ah uh (t ), dt (t ) = bh s, t , uh (s), dt (t ) ds + f (t ) h , dt (t ) . h

As

d dt

ah u h (t ), u h (t ) = 2ah u h (t ),

du h dt

(37)

h

0

(t )

and

t

d

dt

t

bh s, t , u h (s), u h (t ) ds = bh t , t , u h (t ), u h (t ) + 0

bh s, t , u h (s),

t ∂ bh (t ) ds + s, t , u h (s), u h (t ) ds, dt ∂t

du h

0

0

we deduce that

2 t t d 1 d ∂ bh (t ) + bh s, t , u h (s), u h (t ) ds − s, t , u h (s), u h (t ) ds ah u h (t ), u h (t ) 2 dt 2 dt dt ∂ t h 1 du h

0

0

2 1 − bh t , t , uh (t ), uh (t ) + ( f (t ))h h .

(38)

2

Using inequalities (32) and (33) in (38) we establish

1 duh 2 dt

2 1 d (t ) ah u h (t ), u h (t ) + βe P h u h (t )1 + 2 dt

h

t

d

t

bh s, t , u h (s), u h (t ) ds + βd

dt 0

P h uh (s) ds P h uh (t ) + 1 ( f (t ))h 2 . h 1 1 2

0

Consequently, as

t βd

T βd2 P h uh (s) ds P h uh (t ) 1 1 2

t

0

P h uh (s)2 ds + 2 P h uh (t )2 1

4

1

0

= 0, we have duh 2 + d ah uh (t ), uh (t ) + 2 βe − 2 P h uh (t )2 ( t ) dt 1 dt

holds for any

h

2

d

t

β2 T bh s, t , u h (s), u h (t ) ds + d 2

t

2

dt 0

P h uh (s)2 ds + f (t ) 2 . h h 1

(39)

0

The integration from 0 to t of the inequality (39) leads to

t t duh 2 P h uh (s)2 ds ds + ah uh (t ), uh (t ) + 2 βe − 2 ( s ) ds 1 h

0

t 2 0

bh s, t , u h (s), u h (t ) ds +

0

βd2 T 2 2

t s

P h uh (μ)2 dμ ds + ah uh (0), uh (0) +

t

1

0

0

0

f (s) 2 ds. h h

(40)

208

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

Combining (40) with (21) and (22) we obtain

t t duh 2 P h uh (s)2 ds ds + αe − η2 P h uh (t )2 + 2βe − 2 ( s ) ds 1 1 h

0

bc2 T

η

t

0

β2 T P h uh (s)2 ds + d 1

2

t s

2

P h uh (μ)2 dμ ds + αc P h uh (0)2 + 1

2

0

0

t

0

( f (s))h 2 ds, h

1

0

and consequently

t t 2 duh 2 P h uh (t ) ds (s) ds + P h uh (t ) 1 + 1 h

0

t s C

0

P h uh (μ)2 dμ + P h uh (s)2 1

0

ds +

1

0

max{1, αc }

P h uh (0)2 +

min{1, αe − η2 , 2(βe − 2 )}

t

1

f (s) 2 ds h h

0

(41) for , η satisfying (35) and with C deﬁned by (36). Applying the Grownwall lemma to the inequality (41) we conclude (34).

2

4. A supraconvergent estimate In this section we derive an estimate for the error P h eh (t ) = P h R h u (t ) − P h u h (t ). Following [35] we split the error P h eh (t ) in the following form

P h eh (t ) = P h R h u (t ) − P h u˜ h (t ) + P h u˜ h (t ) − P h u h (t ) =: P h ρh (t ) + P h θh (t )

(42)

where u˜ h (t ) is solution of the discrete variational problem

ah u˜ h (t ), v h = gh (t ), v h

h

for all v h ∈ W h,0 ,

,

(43)

with

t gh (t ) =

B ( s , t )u ( s )

ds − h

du dt

(t ) + f (t ) h .

(44)

h

0

As we have successively

2

αe P h ρh (t )1 ah ρh (t ), ρh (t ) = ah R h u (t ), ρh (t ) − gh (t ), ρh (t ) h = ah R h u (t ), ρh (t ) − Au (t ) h , ρh (t ) = ah R h u (t ), ρh (t ) −

N −1

xi +1/2

(a)

Au (x, t ) dx ρh (xi , t ) =: τh

h

ρh (t ) ,

i =1 xi −1/2

we conclude that

P h ρh (t )2 1 τ (a) ρh (t ) . h 1

(45)

αe

A bound for P h ρh (t ) is obtained using Lemma 1 and (45). The proof of this lemma as well as the proofs of the following Lemmas 2, 3 and 4 differ in minor details from the proof of Theorem 3.1 of [4]. Lemma 1. If ah (. , .) is continuous in W h,0 × W h,0 and satisﬁes (21), u ∈ L 2 (0, T ; H r +1 (a, b)), r ∈ {1, 2}, and (a)

then, for the functional τh

∈ L 2 (0, T ; L 2 (a, b)),

the following inequality holds:

N (a) τ ( v h ) C u (t )2 1+r h2r i h H (x i =1

for v h ∈ W h,0 .

du dt

1/2 i −1 , x i )

P h v h 1 ,

r ∈ {1, 2},

(46)

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

209

Proposition 1. Under the assumptions of Lemma 1, the error P h ρh (t ) satisﬁes the estimate

N P h ρh (t ) C u (t )2 1+r h2r i H 1 (x i =1

An estimate for P h have successively

αe P h

dρh dt

dρh (t )1 dt

1/2 ,

i −1 , x i )

r ∈ {1, 2}.

(47)

can be obtained following the procedure used on the estimation of P h ρh (t )1 . In fact we

2 du du˜ h dρh (t ) a R ( t ) − ( t ), ( t ) h h 1

= ah

dt

dt

dt

dρh du dρh Rh (t ), (t ) − B (t , t )u (t ) h , (t ) dt

dt

dt

h

t

2 dρh d u df dρh ∂ B (s, t )u (s) ds, − (t ) + (t ) + (t ) , (t ) ∂t dt dt dt 2 h h h h dt h 0 du du dρh dρh = ah R h (t ), (t ) − A (t ) , (t ) dt

= ah

dt

(d)

αe P h

dρh dt

dt

dt

h

h

xi +1/2 N −1 du dρh du dρh Rh (t ), (t ) − A (x, t ) dx ( xi , t )

=: τh that is

dt

dρh dt

dt

dt

i =1 xi −1/2

(t ) ,

dt

2 (d) dρh (t ) τ ( t ) . h

(48)

dt

1

Lemma 2. If ah (. , .) is continuous in W h,0 × W h,0 and satisﬁes (21), the following inequality holds:

du dt

N 1/2 2 (d) 2r du τ ( v h ) C h i (t ) P h v h 1 , h dt H 1+r (xi −1 ,xi )

∈ L 2 (0, T ; H r +1 (a, b)), r ∈ {1, 2}, then, for the functional τh

(d)

r ∈ {1, 2},

(49)

i =1

for v h ∈ W h,0 . From Lemma 2 and the inequality (48) we obtain the next proposition. Proposition 2. Under the assumptions of Lemma 2, the error P h

N 2 dρh 2r du Ph ( t ) C h ( t ) i 1+r dt dt 1

(b)

Lemma 3. The functional τh

τh(b) (t , v h ) =

H

i =1

t

dρh (t ) dt

satisﬁes the estimate

1/2 ,

r ∈ {1, 2}.

(50)

( x i −1 , x i )

deﬁned by

bh s, t , R h u (s), v h − B (s, t )u (s), v h

h

ds,

0

for v h ∈ W h,0 , satisﬁes

(b) τ (t , v h ) C

t N

h

0

i =1

u (s)2 1+r h2r i H ( x i −1 , x i )

provided that u ∈ L (0, T ; H r +1 (a, b)), r ∈ {1, 2}. 2

1/2 ds P h v h 1 ,

r ∈ {1, 2},

(51)

210

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

Lemma 4. If

du dt

∈ L 2 (0, T ; H r +1 (a, b)), r ∈ {1, 2}, then, for

τh(u) (t , v h ) = R h

du

−

(t ), v h

dt

du dt

h

(t ) , v h , h

h

with v h ∈ W h,0 , we have

N 1/2 du 2 (u ) 2r τ (t , v h ) C h i (t ) P h v h 1 , h dt H 1+r (xi −1 ,xi )

r ∈ {1, 2}.

(52)

i =1

In what follows we use the notation

τh(,br) (t ) = C

t N i =1

0

and

τh(,ur) (t ) = C

N i =1

u (s)2 1+r h2r i H ( x i −1 , x i )

dt

ds

1/2

2

du (t ) h2r i

1/2

H 1+r (xi −1 ,xi )

.

The following lemma plays a central role in the main result of this paper. Lemma 5. If ah (. , .) and bh (s, t , . , .) satisfy (21) and (22), respectively, and u ,

θh (t )2 + 2 αe − 3 2

t

h

2 P h θh (s)2 ds bc T 1

2

t s

dt

∈ L 2 (0, T ; H 0r +1 (a, b)), then the following holds

P h eh (μ)2 dμ ds + θ(0)2 h

1

2

0

du

0

0

t dρh 2 + τh,r (s)2 ds, + 2 ( s ) ds 2 h 1

(53)

0

for t ∈ [0, T ] and for any

(b) (u ) 0, with τh,r (s) = τh,r (s) + τh,r (s). = dθh dt

Proof. It is easy to show that

dθh dt

=

(t ), v h

du˜ h dt

h

(t ) is a solution of the discrete variational problem

t

+ ah uh (t ), v h −

(t ), v h h

bh s, t , u h (s), v h ds −

f (t ) h , v h

h

0

a.e. in (0, T ), for all v h ∈ W h,0 .

(54)

Considering in (54) the deﬁnition (42) of θh (t ) and using (43) we obtain

dθh dt

=

(t ), v h h

du˜ dt

− ah θh (t ), v h −

(t ), v h h

t +

t

B (s, t )u (s) h ds, v h

bh s, t , u h (s), v h ds −

du dt

(t ) , v h

0

h

h

h

0

which is equivalent to

dθh dt

t

+ ah θh (t ), v h =

(t ), v h h

bh s, t , eh (s), v h ds − 0

(u )

(b)

dρh dt

+ τh ( v h ),

(t ), v h

(55)

h

for v h ∈ W h,0 and with τh (t , v h ) = τh (t , v h ) − τh (t , v h ). We now ﬁx v h = θh (t ) in (55). As τh (t , θh ) τh,r (t ) P h θh (t )1 , using the same kind of arguments that were used in the stability analysis it can be shown that

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

t

1 d 2 dt

2 θh (t )2 + αe P h θh (t )2 bc T h 1 2

P h eh (s)2 ds + η2 P h θh (t )2 1

4η

211

1

0

+ for

2 + 2 P h θh (t )2 + 1 τh,r (t )2 + σ 2 P h θh (t )2 , ( t ) 1 1 2 2 dt 4 4σ h 1 dρh

= 0, σ = 0, η = 0 and for t ∈ [0, T ], which is equivalent to t 2 2 d θh (t )2 + 2 αe − 3 2 P h θh (t )2 bc T P h eh (s)2 ds + 1 dρh (t ) + τh,r (t )2 , h 1 1 dt 2 2 2 2 dt 0

when

(56)

h

= η = σ are considered. Integrating (56) we establish (53). 2

The main theorem is established now. Theorem 3. Let u be the solution of the variational problem (12) and P h u h its approximation deﬁned by (24). Then, under the assumptions of Lemma 5, the error P h eh (t ) = P h R h u (t ) − P h u H (t ), for t ∈ [0, T ], satisﬁes the estimate

t

P h eh (s)2 ds e Ct

1

αe − 3

1

0

t +

e −C s

0

2

θh (0)2 h

dρh 2 2 2 (s) + τh,r (s) + 2 P h ρh (s) 1 ds , (αe − 3 2 ) 2 ds h 1

(57)

for r ∈ {1, 2}, where C is deﬁned by

C=

bc2 T

(58)

(αe − 3 2 ) 2

and is such that

αe − 3 2 > 0.

(59)

Proof. The error P h eh satisﬁes the following

t

P h eh (s)2 ds 2

t

1

0

P h ρh (s)2 ds + 2

t

1

0

P h θh (s)2 ds.

(60)

1

0

satisfying (59), we obtain t t s t 1 bc2 T P h eh (s)2 ds P h eh (μ)2 dμ ds + 2 P h ρh (s)2 ds 1 1 1 αe − 3 2 2 0 0 0 0 t 2 dρh 2 1 1 2 + θ(0) h + 2 ds (s) + τh,r (s) ds . αe − 3 2 h

Using (53) in (60) and choosing

(61)

0

Finally the application of the Gronwall lemma to (61) leads to (57). Combining Theorem 3 with Proposition 1, with the deﬁnitions of

2

τh(b) (t ) and τh(u) (t ) we conclude the following result.

Corollary 1. Let u be the solution of the variational problem (12) and P h u h its approximation deﬁned by (24). If ah (. , .) and bh (s, t , . , .) satisfy (21) and (22), respectively, then, for r ∈ {1, 2}, there exists a positive constant C such that

t 0

N t 2r P h eh (s)2 ds C u (s)2 r +1 hi H (x 1 i =1

0

du 2 + ( t ) dt r +1 H

i −1 , x i )

2

ds + u (t ) H r +1 (x

2 + P h u (0) − P h uh (0)

( x i −1 , x i )

for t ∈ [0, T ] provided that u , du ∈ L 2 (0, T ; H r +1 (a, b)). dt

h

i −1 , x i )

212

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

Fig. 1. Numerical solution of (1) with the coeﬃcients (62) and with the conditions (63).

5. Numerical results In the numerical results that we present in this section, the ordinary differential problem (25) is integrated using the implicit Euler method. We introduce the uniform time grid {tn , n = 0, . . . , M } with step-size t, tn = n t and M t = T , and by unh we denote the numerical solution obtained at time level tn . The error that we compute in what follows is the maximum of P h enh 1 = P h u (tn ) − P h unh 1 for n = 0, . . . , M, which is denoted by P h eh 1 . Example 1. Let us consider in Eq. (1)

a2 (x) = 0.5,

a1 (x) = a0 (x) = 0,

b2 (s, t , x) = −

0.5 − t −s e τ ,

τ

b1 (s, t , x) = b0 (s, t , x) = 0,

(62)

τ = 0.01, f = 0, and boundary and initial conditions u (0, t ) = 1,

u (1, t ) = 0,

t > 0,

u (x, 0) = 0,

x ∈ [0, 1],

(63)

and T = 0.1. This IBVP can be used to model the diffusion of a substance in the space domain [0, 1] which is initially empty, with a constant source in the left hand side and assuming that the substance that arrives to the right hand side is immediately removed. In Fig. 1 we plot the numerical solution for several time levels computed using a uniform mesh in [0, 1] with h = 10−2 , t = 10−4 , and uh (0) = R h u (0). Furthermore, the solution of the semi-discrete problem (25) was obtained with the implicit Euler being the integral term discretized using a rectangular rule. The same procedure is used in the next examples. In Table 1 we illustrate the convergence order established in Corollary 1. In this table we present the convergence rates given by

Ph

rate =

eh 1

ln P 1 e 1 h2 h2 1

h1

ln

h2

where h1 and h2 = h21 are consecutive step-sizes that are contained in the ﬁrst column of this table. The error is computed using a numerical solution obtained with a much ﬁner mesh, taking h = 3.09e−0.5. Example 2. Let us consider Eq. (1) with Ω = (0, 1),

a2 (x) = x + 1,

a1 (x) = x + 1,

a0 (x) = 0,

b2 (t , s, x) = e −

t −s

τ

,

b1 (s, t , x) = b0 (s, t , x) = 0,

(64)

and τ = 0.01, T = 0.1. The reaction term f and the initial condition u 0 are such that the IBVP deﬁned with (1) has the solution u (x, t ) = tx(x − 1) cos(x).

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

213

Table 1 Convergence rates for (1) with the coeﬃcients (62) and with the conditions (63). h

N

Error

Rate

1.000e−02 5.000e−03 2.500e−03 1.250e−03 6.250e−04 3.125e−04 1.563e−04

1.000e+02 2.000e+02 4.000e+02 8.000e+02 1.600e+03 3.200e+03 6.400e+03

3.796e−06 9.484e−07 2.370e−07 5.916e−08 1.470e−08 3.590e−09 8.188e−10

2.00 2.00 2.00 2.01 2.03 2.13 –

Fig. 2. ln( P h eh 1 ) versus ln(hmax ) and the least square straight line.

We consider a set of 451 random grids in [0, 1] and for the time integration we take t = 2 × 10−6 . We plot in Fig. 2 the logarithm of the error P h eh 1 versus the logarithm of the maximum step-size. The straight line plotted in this ﬁgure is the least-squares ﬁt to the points (ln(hmax ), ln( P h eh 1 )). As the slope of this straight line is 2.0107, the numerical results obtained conﬁrm the estimate given in Corollary 1. Example 3. Let us consider Eq. (1) with Ω = (0, 1),

a 2 = a 1 = 1,

a 0 = 0,

b2 (s, t , x) = e x−

t −s

τ

sin(x),

b1 (s, t , x) = e −

t −s

τ

x2 ,

b0 (s, t , x) = 0,

(65)

and τ = 0.01, T = 0.1. Let f and u 0 be such that the IBVP deﬁned with (1) has the solution u (x, t ) = tx(x − 1) cos(x). In Fig. 3 we plot the logarithm of the error P h eh 1 versus the logarithm of the maximum step-size for a set of 451 random grids considered in [0, 1], when t = 2 × 10−6 . The least-squares straight line ﬁtting the points (ln(hmax ), ln( P h eh 1 )) is also plotted in this ﬁgure. As the slope of this straight line is 1.998 we conclude that these results illustrate the estimates given in Corollary 1. 6. Conclusions In this paper a semi-discretization of the integro-differential problem (1) with Dirichlet boundary conditions was studied. It was shown that the semi-discrete approximation presents convergence of order r with respect to the norm .1 provided

214

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

Fig. 3. ln( P h eh 1 ) versus ln(hmax ) and the least square straight line.

that u (t ) and ∂∂ut (t ) are in H r +1 (a, b) for r ∈ {1, 2}. The semi-discretization studied can be seen as a standard ﬁnite difference discretization and as a lumped mass semi-discretization, and so the convergence estimates established can be seen as both supraconvergent and supercloseness estimates. It is known that the semi-discrete approximation for the Fickian parabolic problem correspondent to (1), deﬁned using the piecewise linear ﬁnite element method has second convergence order with respect to the L 2 -norm provided the solution is in H 2 (a, b) [33]. The smoothness required in the present paper is essential to conclude the unexpected convergence order because it was established considering the norm .1 . Acknowledgements The authors gratefully acknowledge support of the Centre for Mathematics of University of Coimbra and the projects PTDC/Mat/74548/2006 and UTAustin/MAT/0066/2008. References [1] A. Araújo, J.R. Branco, J.A. Ferreira, On the stability of a class of splitting methods for integro-differential equations, Appl. Numer. Math. 59 (2009) 436–453. [2] S. Barbeiro, J.A. Ferreira, Integro-differential models for percutaneous drug absorption, Int. J. Comput. Math. 84 (2007) 451–467. [3] S. Barbeiro, J.A. Ferreira, Coupled vehicle–skin models for drug release, Comput. Methods Appl. Mech. Engrg. 198 (2009) 2078–2086. [4] S. Barbeiro, J.A. Ferreira, R.D. Grigorieff, Supraconvergence of a ﬁnite difference scheme for solutions in H s (0, L ), IMA J. Numer. Anal. 25 (2005) 797– 811. [5] J.R. Branco, J.A. Ferreira, A singular perturbation of the heat equation with memory, J. Comput. Appl. Math. 218 (2008) 376–394. [6] J.R. Branco, J.A. Ferreira, P. de Oliveira, Numerical methods for the generalized Fisher–Kolmogorov–Petrovskii–Piskunov equation, Appl. Numer. Math. 57 (2007) 89–102. [7] H.-T. Chen, K.-C. Liu, Analysis of non-Fickian diffusion problems in a composite medium, Comput. Phys. Comm. 150 (2003) 31–42. [8] C. Chuanmiao, S. Tsimin, Finite Element Methods for Integrodifferential Equations, World Scientiﬁc Publishers, 1998. [9] E. Cuesta, C. Lubich, C. Palencia, Convolution quadrature time discretization of fractional diffusion-wave equations, Math. Comp. 75 (2006) 673–696. [10] F. de Hoog, D. Jackett, On the rate of convergence of ﬁnite difference schemes on nonuniform grids, J. Aust. Math. Soc. B 26 (1985) 247–256. [11] E. Emmrich, Supraconvergence and supercloseness of a discretization for elliptic third kind boundary value problems on polygonal domains, Comput. Methods Appl. Math. 7 (2007) 153–162. [12] E. Emmrich, R.D. Grigorieff, Supraconvergence of a ﬁnite difference scheme for elliptic boundary value problems of the third kind in fractional order Sobolev spaces, Comput. Methods Appl. Math. 6 (2006) 154–177. [13] R.E. Ewing, R.D. Lazarov, Y. Lin, Finite volume element approximations of nonlocal in time one-dimensional ﬂows in porous media, Computing 64 (2000) 157–182.

S. Barbeiro et al. / Applied Numerical Mathematics 61 (2011) 201–215

215

[14] R.E. Ewing, R.D. Lazarov, Y. Lin, Finite volume element approximations of nonlocal reactive ﬂows in porous media, Numer. Methods Partial Differential Equations 16 (2000) 258–311. [15] R.E. Ewing, Y. Lin, T. Sun, J. Wang, S. Zhang, Sharp L 2 -error estimates and superconvergence of mixed ﬁnite element methods for non-Fickian ﬂows in porous media, SIAM J. Numer. Anal. 40 (2002) 1538–1560. [16] J.A. Ferreira, P. de Oliveira, Memory effects and random walks in reaction-transport systems, Appl. Anal. 86 (2007) 99–118. [17] J.A. Ferreira, P. de Oliveira, Qualitative analysis of a delayed non Fickian model, Appl. Anal. 87 (2008) 873–886. [18] J.A. Ferreira, R.D. Grigorieff, On the supraconvergence of elliptic ﬁnite difference schemes, Appl. Numer. Math. 28 (1998) 275–292. [19] J.A. Ferreira, R.D. Grigorieff, Supraconvergence and supercloseness of a scheme for elliptic equations on nonuniform grids, Numer. Funct. Anal. Optim. 27 (2006) 539–564. [20] P.A. Forsyth, P.H. Samon, Quadratic convergence for cell-centered grids, Appl. Numer. Math. 4 (1988) 377–394. [21] R.D. Grigorieff, Some stability inequalities for compact ﬁnite difference operators, Math. Nachr. 135 (1986) 93–101. [22] S.M. Hassahizadeh, On the transient non-Fickian dispersion theory, Transp. Porous Media 23 (1996) 107–124. [23] D.D. Joseph, L. Preziosi, Heat waves, Rev. Mod. Phys. 61 (1989) 41–73. ´ L.D. Ivanovic, ´ E.E. Sülli, Convergence of ﬁnite difference schemes for elliptic equationswith variable coeﬃcients, IMA J. Numer. Anal. 7 [24] B.S. Jovanovic, (1987) 301–305. [25] H.O. Kreiss, T.A. Manteuffel, B. Swartz, B. Wendroff, A.B. White, Supraconvergent schemes on irregular grids, Math. Comp. 45 (1986) 105–116. [26] Y. Lin, Semi-discrete ﬁnite element approximations for linear parabolic integro-differential equations with integrable kernels, J. Integral Equations Appl. 10 (1998) 51–83. [27] Y. Lin, V. Thomée, L.B. Wahlbin, Ritz–Volterra projections to ﬁnite-element spaces and applications to integrodifferential and related equations, SIAM J. Numer. Anal. 28 (1991) 1047–1070. [28] C. Maas, A hyperbolic dispersion equation to model the bounds of a contaminated groundwater body, J. Hydrol. 226 (1999) 234–241. [29] T.A. Manteuffel, A.B. White Jr., The numerical solutions of second order boundary value problems, Math. Comp. 47 (1986) 511–535. [30] S.P. Neuman, D.M. Tartakovski, Perspectives on theories of non-Fickian transport in heterogeneous medias, Adv. Water Resour. 32 (5) (2009) 678–680. [31] A.K. Pani, T.E. Peterson, Finite element methods with numerical quadrature for parabolic integro-differential equations, SIAM J. Numer. Anal. 33 (1996) 1084–1105. [32] R.K. Sinha, R.E. Ewing, R.D. Lazarov, Some new error estimates of a semidiscrete ﬁnite volume method for a parabolic integro-differential equation with nonsmooth initial data, SIAM J. Numer. Anal. 43 (2006) 2320–2344. [33] V. Thomée, N.-Y. Zhang, Error estimates for semidiscrete ﬁnite element methods for parabolic integro-differential equations, Math. Comp. 53 (1989) 121–139. [34] L.B. Wahlbin, Superconvergence in Galerkin Finite Element Methods, Lecture Notes in Math., vol. 1605, Springer, Berlin, 1995. [35] M.F. Wheeler, A priori L 2 error estimates for Galerkin approximation to parabolic partial differential equations, SIAM J. Numer. Anal. 10 (1973) 559– 723. [36] J. Wloka, Partial Differential Equations, Cambridge University Press, 1987. [37] N.-Y. Zhang, On fully discrete Galerkin approximation for partial integro-differential equations of parabolic type, Math. Comp. 60 (1993) 133–166.

Applied Numerical Mathematics 61 (2011) 216–228

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

Some Goldstein’s type methods for co-coercive variant variational inequalities M. Li a,1 , L.-Z. Liao b,2 , X.M. Yuan b,∗,3 a b

School of Economics and Management, Southeast University, Nanjing 210096, PR China Department of Mathematics, Hong Kong Baptist University, Hong Kong, PR China

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 6 December 2008 Received in revised form 24 August 2010 Accepted 5 October 2010 Available online 14 October 2010

The classical Goldstein’s method has been well studied in the context of variational inequalities (VIs). In particular, it has been shown in the literature that the Goldstein’s method works well for co-coercive VIs where the underlying mapping is co-coercive. In this paper, we show that the Goldstein’s method can be extended to solve co-coercive variant variational inequalities (VVIs). We ﬁrst show that when the Goldstein’s method is applied to solve VVIs, the iterative scheme can be improved by identifying a reﬁned stepsize if the involved co-coercive modulus is known. By doing so, the allowable range of the involved scaling parameter ensuring convergence is enlarged compared to that in the context of VVIs with Lipschitz and strongly monotone operators. Then, we show that for such a VVI whose co-coercive modulus is unknown, the Goldstein’s method is still convergent provided that an easily-implementable Armijo’s type strategy of adjusting the scaling parameter self-adaptively is employed. Some numerical results are reported to verify that the proposed Goldstein’s type methods are eﬃcient for solving VVIs. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

Keywords: Co-coercive Goldstein’s method Optimal step-size Variant variational inequality

1. Introduction The ﬁnite-dimensional variational inequality problem, denoted by VI(Ω, F ), is to ﬁnd a vector u ∈ Ω such that

( v − u ) T F ( u ) 0,

∀ v ∈ Ω,

(1)

where Ω is a nonempty closed convex set in R n and F is a mapping from R n into itself. We refer to [4,8,20,31] for the pivotal roles of VIs in various ﬁelds such as economics, transportation, engineering, etc. This paper considers the so-called variant variational inequality VVI(Ω, F ), which is closely related to VI(Ω, F ). In particular, VVI(Ω, F ) is to ﬁnd a vector u ∈ R n such that

F (u ) ∈ Ω,

T

v − F (u )

u 0,

∀ v ∈ Ω.

(2)

Throughout, we assume that the solution set of (2), denoted by S ∗ , is nonempty. The necessity of studying VVI(Ω, F ) can be justiﬁed in the literatures, e.g., [13,14,23,32–34].

* 1 2 3

Corresponding author. E-mail addresses: [email protected] (M. Li), [email protected] (L.-Z. Liao), [email protected] (X.M. Yuan). This author was supported by the SRFDP grant 200802861031 and the NSFC grants 10926147 and 11001053. This author was supported in part by grants from Hong Kong Baptist University and the Research Grant Council of Hong Kong. This author was supported in part by FRG/08-09/II-40 from HKBU and the NSFC grant 10701055.

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.10.001

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

217

In general, it is not possible to identify analytic solutions of VI(Ω, F ) and VVI(Ω, F ) without speciﬁc structures of F and Ω . Therefore, researchers are particularly interested in designing numerical algorithms to approach solutions of VI(Ω, F ) and VVI(Ω, F ), subject to certain criteria. No doubt that the monotonicity of F plays a crucial role in designing numerical algorithms for solving VI(Ω, F ) and VVI(Ω, F ). Among existing numerical methods for solving VIs (see the comprehensive survey [7]) is the classical Goldstein’s method [10,22]. With the restrictions that F is Lipschitz continuous and strongly monotone, starting from any u 0 ∈ Ω , the Goldstein’s method iteratively updates uk according to the formula

(Goldstein’s method)

uk+1 = P Ω uk − βk F uk ,

(3)

where βk > 0 is the scaling parameter (also can be regarded as the step-size) and P Ω ( v ) denotes the projection of v onto Ω under the Euclidean norm, i.e.,

P Ω ( v ) = argmin u − v | u ∈ Ω . The Goldstein’s method converges globally with linear speed under the condition that ∞

∞

2s

k =1

k =1

L2

0 < β L inf {βk } sup{βk } βU <

(4)

,

where L is the Lipschitz constant and s is the strong monotonicity constant of F . The iterative scheme of the Goldstein’s method is extremely simple and easily implementable, and this method requires little storage during iterations. In particular, (3) is an explicit scheme in the sense that the unknown variable uk+1 appears only in the left-hand side of (3). The sequent development on the Goldstein’s method includes the extragradient method and its variants [16–19,21,29], whose main common characteristic is to relax the strong monotonicity of F to monotonicity with the price of operating an additional projection onto Ω at each iteration. It is [13] that ﬁrstly extended (3) to solve VVI(Ω, F ) with Lipschitz continuous and strongly monotone F . The corresponding scheme is:

(He–Goldstein method)

u k +1 = u k −

1 k F u − P Ω F uk − βk uk .

βk

(5)

Similar to (4), the scaling parameters {βk } in (5) should also be restricted in certain ranges, relying on the Lipschitz constant and strong monotonicity constant of F . Recently, the He–Goldstein method was developed in [26] with the improvement of relaxing the strong monotonicity assumption on F to monotonicity, by operating a descent step at each iteration whose descent direction is derived from the original Goldstein’s type method (5). In addition to the monotonicity, the co-coercivity of F also arises widely in the VI and VVI models in various applications. Therefore, the corresponding co-coercive VI(Ω, F ) and VVI(Ω, F ) (i.e., F is co-coercive in (1) and (2), respectively) also deserve intensive investigation, see e.g. [11,24,27,28,30,35–37]. Recall that F is called co-coercive if there exists a positive constant τ (co-coercive modulus) such that

2 (u − v )T F (u ) − F ( v ) τ F (u ) − F ( v ) ,

∀u , v ∈ R n .

(6)

Note that the co-coercivity is also referred to the strong monotonicity of F −1 in [9] and the strong-F -monotonicity in [27, 28]. In fact, the co-coercivity implies the Lipschitz continuity (with constant 1/τ ) and monotonicity (but not necessarily strong monotonicity). As studied in [7,30], the classical Goldstein’s method (3) can also be extended to solve co-coercive VI(Ω, F ) provided that the scaling parameters {βk } satisfy the following rule: ∞

∞

k =1

k =1

0 < β L inf {βk } sup{βk } βU < 2τ .

(7)

Later, an improved Goldstein’s type method was presented in [11] for co-coercive VIs arising in transportation by integrating reﬁned step-sizes. We refer to [24] for the most recent development on the Goldstein’s type method for solving co-coercive VI(Ω, F ). Then, a natural question arises: Is it possible to extend the Goldstein’s type method (5) to solve co-coercive VVI(Ω, F )? The intuition is fairly positive, and the goal of this paper is to give the aﬃrmative answer. After reviewing some preliminaries in Section 2 and providing the motivation in Section 3, in Section 4 we show that when the involved co-coercive modulus of a VVI is known, the Goldstein’s method can be improved by identifying a reﬁned step-size at each iteration, as inspired by [11]. Hence, an improved Goldstein’s type method is proposed for co-coercive VVIs with known co-coercive modulus. As a by-product, the improved Goldstein’s type method enlarges the allowable range of the scaling parameter {βk } required conventionally by Goldstein’s type methods for Lipschitz continuous and strongly monotone VVIs. In Section 5, we concentrate on the case where the co-coercive modulus is unknown. For such VVIs, we show that the Goldstein’s method still works, provided that an easily-implementable Armijo’s type strategy of adjusting the scaling parameter {βk } self-adaptively is adopted. Finally, in Section 6, we report some numerical results to verify the eﬃciency of the proposed Goldstein’s type methods for VVIs. Numerical comparisons with some existing methods are also reported in this section.

218

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

2. Preliminaries In this section, we provide some preliminaries that are useful for the following analysis. First, some important inequalities with respect to the projection operator are listed, and the proofs are available in many monographs such as [7]. Lemma 1. Let Ω be a closed convex set in R n , then we have

T

∀ v ∈ R n , u ∈ Ω; 2 v − P Ω ( v ) − w − P Ω ( w ) v − w 2 − P Ω ( v ) − P Ω ( w )2 , P Ω ( v ) − P Ω ( w )2 ( v − w )T P Ω ( v ) − P Ω ( w ) , ∀ v , w ∈ R n ; v − P Ω ( v ) − w − P Ω ( w ) v − w , ∀ v , w ∈ R n . v − P Ω (v )

u − P Ω ( v ) 0,

(8) n

∀v , w ∈ R ;

(9) (10) (11)

Remark 1. The inequality (10) shows that the projection mapping P Ω (·) : R n → R n is co-coercive with the modulus 1. The VVI(Ω, F ) is characterized by the following projection equation (see e.g. [14] for a proof)

F (u ) = P Ω F (u ) − β u ,

(12)

where β is a positive constant. Denote

r (u , β) =

1

β

F (u ) − P Ω F (u ) − β u

(13)

as the scaled residue of the projection equation (12). Then, for any solution point u ∗ ∈ S ∗ , we have

r u ∗ , β = 0. Therefore, solving VVI(Ω, F ) is equivalent to ﬁnding a zero point of r (u , β). By the deﬁnition of (13), for any given u ∈ R n , the magnitude r (u , β) depends on the value of β . The following lemma is useful for the convergence analysis of the algorithm. Lemma 2. For any u ∈ R n and β˜ β > 0, we have

r (u , β) ˜ r (u , β).

(14)

2

Proof. See a similar proof in [26].

The following result is used widely in the literature, see e.g. [6,14], and the proof is referred to Theorems 8.52 and 8.55 in [1]. Lemma 3. Let {bk }k∞=1 be a positive series and bk ∈ (0, 1) for all k. If

∞

k=1 (1 − bk )

> 0, then

∞ k=1 bk < +∞ and thus limk→∞ bk = 0; ∞ 2. k=1 (1 + cbk ) < ∞ for any c > 0. 1.

3. Motivation Note that the Goldstein’s type method (5) can be represented equivalently by

uk+1 = uk − r uk , βk .

(15)

To discuss the possibility of applying this scheme to solve co-coercive VVI(Ω, F ), we ﬁrst provide the following lemma, which is similar to Lemma 12.1.7 in [7]. Lemma 4. Let u ∗ ∈ S ∗ be a solution point of co-coercive VVI(Ω, F ). For any u ∈ R n and β > 0, we have

u − u∗

T

r (u , β) 1 −

1

4β τ

where τ is the co-coercive modulus of F .

r (u , β)2 ,

(16)

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

219

Proof. Since u ∗ is a solution and P Ω [ F (u ) − β u ] ∈ Ω , it follows from the deﬁnition of VVI(Ω, F ) (2) that

T β u∗ P Ω F ( u ) − β u − F u ∗ 0. Setting v =

(17)

F (u ) − β u in the inequality (8), because F (u ∗ ) ∈ Ω , we have

F (u ) − β u − P Ω F (u ) − β u

T

P Ω F (u ) − β u − F u ∗

0.

(18)

Adding (17) and (18) and using the notation of residue r (u , β), we get

T F (u ) − F u ∗ − β r (u , β) 0, β r (u , β) − β u − u ∗ and consequently

u − u∗

T

2

r (u , β) r (u , β) +

1

β

u − u∗

T

F (u ) − F u ∗

−

1

β

F (u ) − F u ∗

T

r (u , β).

(19)

Using the co-coercivity of F , we obtain

u − u∗

T

2

r (u , β) r (u , β) +

τ

β

2 F (u ) − F u ∗ −

1

β

F (u ) − F u ∗

T

r (u , β)

2 2 2 τ ∗ 1 1 − √ r (u , β) r (u , β) + F (u ) − F u = r (u , β) − 4β τ β 2 βτ 1 r (u , β)2 , 1− 4β τ 2

then the lemma is proven.

Therefore, for any β > 1/(4τ ), the right-hand side of (16) is non-negative. Since (u − u ∗ ) is the gradient of the unknown distance function u − u ∗ 2 /2 at the point u, Lemma 4 shows that −r (u , β) is a descent direction of u − u ∗ 2 /2 at the / S ∗ . This observation justiﬁes not only the idea of extending the Goldstein’s type method (5) to solve point u, as long as u ∈ co-coercive VVI(Ω, F ), but also the possibility of identifying a reﬁned step-size along the descent direction −r (u , β). The following corollary provides the strategy of choosing {βk } when the Goldstein’s type method (5) is applied to solve VVI(Ω, F ). Corollary 1. Let u ∗ ∈ S ∗ be a solution point. When the Goldstein’s type method

uk+1 = uk − r uk , βk

is applied to solve co-coercive VVI(Ω, F ), we have

k k +1 ∗ 2 ∗ 2 u −u u −u − 1−

1 2βk τ

k r u , βk 2 ,

(20)

where τ is the co-coercive modulus of F . Proof. It follows directly from Lemma 4.

2

Based on (20), it is easy to derive that if {βk } satisﬁes the following condition:

1 2τ

∞

∞

k =1

k =1

< β L inf {βk } sup{βk } βU ,

(21)

then we have

k +1 k ∗ 2 ∗ 2 u −u u −u − 1−

1 2β L τ

k r u , βU 2 ,

(22)

which implies immediately that the sequence {uk } generated by the Goldstein’s method (5) is Fejér monotone with respect to the solution set S ∗ . Hence, convergence of the sequence {uk } generated by the scheme (5) with the requirement (21) for co-coercive VVI(Ω, F ) is readily derived, see e.g. [3]. Note that F is co-coercive if it is Lipschitz continuous (with constant L) and strongly monotone (with modulus s), and the co-coercive modulus is s/ L 2 . Therefore, by substituting τ = s/ L 2 in (21), the following condition on {βk } required in [13] for solving Lipschitz continuous and strongly monotone VVI(Ω, F ) is recovered:

220

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

L2 2s

∞

∞

k =1

k =1

< β L inf {βk } sup{βk } βU .

(23)

As we shall show soon, the allowable range of {βk } indicated in (21) can be enlarged provided that the co-coercive modulus τ is known. 4. An improved Goldstein’s type method for co-coercive VVIs with known co-coercive modulus In this section, we shall show that when the co-coercive modulus is known, the modiﬁcation of seeking an appropriately chosen step-size along the descent direction −r (uk , βk ) cannot only reﬁne the original Goldstein’s method for co-coercive VVIs, but also enlarge the allowable range of {βk } implied by (21). More speciﬁcally, an improved version of the Goldstein’s method for such co-coercive VVIs whose co-coercive modulus is known is as follows. Algorithm 1: An improved Goldstein’s type method for co-coercive VVIs Initialization: Choose βU > β L > 1/(4τ ), γ ∈ (0, 2), Step 1. For the given pair (uk , βk ), compute

r uk , βk =

ε > 0, u 0 ∈ R n , β0 ∈ [βL , βU ] and k = 0.

1 k F u − P Ω F uk − βk uk .

(24)

βk

If r (uk , βk ) ε , terminate the iteration with the approximate solution uk . Otherwise, go to Step 2. Step 2. Generate the new iterate uk+1

uk+1 := uk − γ αk r uk , βk

with αk := 1 −

1 4βk τ

.

(25)

Step 3. Adjust βk

⎧ 1 ⎪ ⎨ max{β L , 2 βk }, βk+1 = min{βU , 6 βk }, 5 ⎪ ⎩ βk

if ωk < 12 ; if ωk > 32 ; otherwise,

where

ωk =

F (uk+1 ) − F (uk ) . βk uk+1 − uk

Set k := k + 1 and go to Step 1. Remark 2. Note that the restriction on the step-sizes {βk } in (25) is βk ∈ [β L , βU ] ⊂ (1/(4τ ), +∞), which is weaker than the previously-derived condition (21). Remark 3. How to adjust the parameters {βk } self-adaptively plays an important role for accelerating convergence of this type of methods. Our strategy of adjusting {βk } is similar to the technique presented in [16], which is in the spirit of balancing the quantities βk uk+1 − uk and F (uk+1 ) − F (uk ) during iterations, i.e., βk is adjusted such that

βk uk+1 − uk ≈ F uk+1 − F uk .

We ﬁrst investigate the technique of choosing the reﬁned step-size along the descent direction −r (uk , βk ), and then prove the convergence of the new method. To justify the strategy of choosing the step-size αk as in (25), we use

uk+1 (α ) := uk − α r uk , βk

(26)

to denote the temporary point taking α as the step-size along −r (uk , βk ), then the following lemma motivates us to identify the reﬁned step-size in (25) along this direction. Lemma 5. For given uk and βk > 0, let uk+1 (α ) be deﬁned in (26). Then we have

2 2 Θk (α ) := uk − u ∗ − uk+1 (α ) − u ∗ Φk (α ), where u ∗

∈

S ∗ and

2

Φk (α ) := −α + 2 1 −

1 4βk τ

2 α r uk , βk .

(27)

(28)

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

221

Proof. Because uk+1 (α ) = uk − α r (uk , βk ), we obtain

k +1 u (α ) − u ∗ 2 = uk − u ∗ 2 − 2α uk − u ∗ T r uk , βk + α 2 r uk , βk 2 , and consequently

T 2 Θk (α ) = 2α uk − u ∗ r uk , βk − α 2 r uk , βk . Since u ∗ is a solution, it follows from (16) that

Θk (α ) 2 1 −

1

4βk τ

2 2 α r uk , βk − α 2 r uk , βk ,

which proves the assertion (27).

(29)

2

Note that Φk (α ) is a lower-bound of Θk (α ), which is the progress obtained by the temporary point uk+1 (α ). Therefore, it is reasonable to choose such optimal values of α that maximize Φk (α ) in order to accelerate convergence of Algorithm 1. This obversion explains why we choose

αk = 1 −

1 4βk τ

(30)

,

the value maximizing the quadratic function Φk (α ). On the other hand, from numerical point of view, it is necessary to attach a relaxation factor to the reﬁned step-size αk obtained theoretically to achieve faster convergence, see e.g. [5,12]. The following theorem concerns how to choose the relaxation factor. Theorem 1. Let u ∗ be an arbitrary point in S ∗ , γ ∈ (0, 2) and βk ∈ [β L , βU ] ⊂ (1/(4τ ), +∞). For the given uk , let uk+1 = uk+1 (γ αk ) = uk − γ αk r (uk , βk ), i.e., the step-size along the descent direction −r (uk , βk ) is γ αk , then we have

k +1 2 2 2 − γ k u u − uk+1 2 , − u ∗ u k − u ∗ −

(31)

k u − uk+1 γ r uk , βk .

(32)

γ

and

Proof. Using (30), by a simple manipulation we can get

k u − uk+1 = γ αk r uk , βk γ r uk , βk .

(33)

From (28) and (30), we have

2

2

Φk (γ αk ) = −γ (αk ) + 2γ 1 − =

(2 − γ )(γ αk )2

=

2−γ

γ

γ

1 4βk τ

2 αk r uk , βk

2 r uk , βk

2

u k − u k +1 .

(34)

The assertion follows from (27), (34) and (33) directly. Remark 4. Theorem 1 shows theoretically that any (in practice, γ ∈ [1, 2) is preferable).

2

γ ∈ (0, 2) guarantees that the new iterate makes progress to a solution

Theorem 1 indicates that the sequence {uk } generated by Algorithm 1 is Fejér monotone with respect to S ∗ . Hence, the convergence of the proposed improved Goldstein’s type method for co-coercive VVIs is readily obtained, see e.g. Theorem 2.1 in [25] and [3]. Theorem 2. The sequence {uk } generated by Algorithm 1 converges to a solution of VVI(Ω, F ).

222

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

5. A self-adaptive Goldstein’s type method for VVIs with unknown co-coercive modulus As analyzed in the previous section, the estimation of the modulus τ is required to determine appropriate step-sizes for implementing the Goldstein’s type methods (15) and (25) to solve co-coercive VVIs. In many situations, the task of estimating the value of τ is not accomplishable. Therefore, in this section, we will present a practical variant of the Goldstein’s type method (15) for solving such co-coercive VVIs whose co-coercive modulus is unknown. The new Goldstein’s type method adopts a self-adaptive strategy, which is in the manner of Armijo’s rule (see [2]) and analogous to that in [15], to determine the step-sizes. Thus, the estimation of the value of τ is avoided. This method can be regarded as a generalization of the method in [15] for co-coercive VVIs. Algorithm 2: A self-adaptive Goldstein’s type method for co-coercive VVIs Initialization: Choose Step 1. Compute

r uk , βk−1 =

μ > 1, δ ∈ (0, 2), ε > 0, u 0 ∈ R n , β−1 > 0 and k = 0. 1 k F u − P Ω F uk − βk−1 uk .

βk−1

(35)

If r (uk , βk−1 ) ε , terminate the iteration with the approximate solution uk . Otherwise, go to Step 2. Step 2. Generate the new iterate uk+1 : ﬁnd the smallest non-negative integer lk (begin with lk = 0) such that βk = μlk βk−1 and

uk+1 = uk − r uk , βk ,

(36)

which satisﬁes

F uk − F uk+1 2 T 2 − δ. βk uk − uk+1 F uk − F uk+1

(37)

Set k := k + 1 and go to Step 1. Remark 5. Recall the co-coercivity of VVI (6). It is easy to verify that the condition (37) is fulﬁlled as long as βk 1/(τ (2 − δ)). In fact, without knowing the exact value or accurate estimation of the co-coercivity modulus τ , appropriate step-sizes {βk } of Algorithm 2 can still be determined in ﬁnitely-many steps due to the adoption of the Armijo’s rule. Remark 6. On the other hand, the feature of the Armijo’s rule ensures that {βk } is a non-decreasing sequence. Thus, the sequence {βk } generated by Algorithm 2 is bounded below. Recall that solving the VVI (2) is equivalent to ﬁnding a zero point of r (u , β) deﬁned in (13). We shall show that

r (uk , βk ) → 0 for the sequence {(uk , βk )} generated by Algorithm 2. Theorem 3. Let the sequence {(uk , βk )} be generated by Algorithm 2, then we have

k +1 r u , βk+1 2 r uk , βk 2 − δ τ F uk − F uk+1 2 . βk

(38)

Proof. First, according to the deﬁnition of r (u , β) (see (13)) and Eq. (36), we have

r uk+1 , βk =

1 k +1 − P Ω F uk+1 − βk uk+1 F u

βk

(39)

and

0=

1

βk

−βk uk+1 −

k F u

− βk uk − P Ω F uk − βk uk .

(40)

Adding (39) and (40), we have

r uk+1 , βk =

1 k+1 − βk uk+1 − P Ω F uk+1 − βk uk+1 F u

βk −

1 k F u − βk uk − P Ω F uk − βk uk .

βk

Second, setting v = F (uk+1 ) − βk uk+1 and w = F (uk ) − βk uk in (11) and using (41), we derive that

(41)

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

223

k+1 k r u , βk u − uk+1 − 1 F uk − F uk+1 . βk

(42)

Then, it follows from (42) that

k+1 2 k r u , βk u − uk+1 2 − 2 uk − uk+1 T F uk − F uk+1 + 1 F uk − F uk+1 2 . 2 βk βk

(43)

From (37), we obtain

1 k F u − F uk+1 2 2 − δ uk − uk+1 T F uk − F uk+1 . 2 βk

βk

Substituting this into (43) and using (6), we have

k+1 2 k r u , βk u − uk+1 2 − δ uk − uk+1 T F uk − F uk+1 βk k 2 δ τ k k + 1 F u − F u k +1 2 . u − u − βk

(44)

Recall that βk+1 βk . So, it follows from Lemma 2 that

k +1 r u , βk+1 r uk+1 , βk .

(45)

With the fact r (uk , βk ) = uk − uk+1 in (36), the assertion (38) is derived immediately from (44) and (45).

2

For the sequence {(uk , βk )}, since r (uk , βk ) = uk − uk+1 , we can assume that uk+1 = uk for any k. Otherwise, r (uk , βk ) = 0, uk is the solution, and then the sequence is terminated. Therefore, we can deﬁne

ηk :=

F (uk ) − F (uk+1 ) . βk uk − uk+1

(46)

The following lemma indicates that the sequence generated by Algorithm 2 holds a similar (but weaker) contractive property to the proximal point algorithm for VVIs that was developed in [23] (see Section 3 of [23]). Lemma 6. Let u ∗ ∈ S ∗ , {(uk , βk )} be generated by Algorithm 2 and ηk be deﬁned in (46). Then, we have

2 2 1 − 2ηk2 uk+1 − u ∗ uk − u ∗ −

1 2

u k − u k +1 2 .

(47)

Proof. It follows from (13) and (36) that

βk uk+1 = P Ω F uk − βk uk − F uk − βk uk . Moreover, it is obvious that

−βk u ∗ = F u ∗ − βk u ∗ − P Ω F u ∗ − βk u ∗ , where u ∗ ∈ S ∗ . Adding the above two identities together, we obtain

k +1 2 2 βk u − u ∗ = P Ω F uk − βk uk − F uk − βk uk − P Ω F u ∗ − βk u ∗ − F u ∗ − βk u ∗ . Setting v = F (u ) − βk u and w = k

k

k +1 2 2 2 βk u − u ∗ F uk − F u ∗ − βk uk − u ∗ − P Ω F uk − βk uk − P Ω F u ∗ − βk u ∗ . Note that

P Ω F u ∗ − βk u ∗ = F u ∗ . On the other hand, it follows from (13) and (36) that

(48)

F (u ∗ ) − βk u ∗ in (9) and using (48), we have

P Ω F uk − βk uk = F uk + βk uk+1 − uk . Therefore from the above two equations and (49), we obtain that

(49)

224

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

k +1 2 2 2 βk u − u ∗ F uk − F u ∗ − βk uk − u ∗ − F uk − F u ∗ − βk uk − uk+1 2 2 T = βk uk − u ∗ − βk uk − uk+1 − 2βk uk+1 − u ∗ F uk − F u ∗ . Recall that

0 −2βk uk+1 − u ∗

T

F u ∗ − F u k +1 .

Adding the above two inequalities together, we have

k +1 2 2 2 T βk u − u ∗ βk uk − u ∗ − βk uk − uk+1 − 2βk uk+1 − u ∗ F uk − F uk+1 . If F (uk ) = F (uk+1 ), from the deﬁnition of ηk we have Otherwise, ηk = 0. According to the Cauchy–Schwarz inequality, we have

−2v T w 2 v T w 2 v w v 2 + w 2 , Since

√

(50)

ηk = 0. Then the assertion (47) follows from (50) immediately.

∀v , w ∈ Rn.

√

ηk = 0, by setting v = 2ηk βk (uk+1 − u ∗ ) and w = [ F (uk ) − F (uk+1 )]/( 2ηk ) in the above inequalities, we have T 2 2 1 −2βk uk+1 − u ∗ F uk − F uk+1 2ηk2 βk uk+1 − u ∗ + 2 F uk − F uk+1 . 2ηk

Therefore, (50) implies that

k +1 2 2 2 2 1 k βk u F u − F u k +1 2 . − u ∗ βk uk − u ∗ − βk uk − uk+1 + 2η2 βk uk+1 − u ∗ + k

Using (46), the assertion (47) follows from the above inequality immediately.

2ηk2

2

The next theorem shows that any accumulation point of the sequence {(uk , βk )} generated by Algorithm 2 offers a solution of the co-coercive VVI(Ω, F ). Theorem 4. Let {(u ∗ , β ∗ )} be an accumulation point of the sequence {(uk , βk )} generated by Algorithm 2. Then u ∗ is a solution point of the co-coercive VVI(Ω, F ). Proof. As we have proved in Theorem 3, {r (uk , βk )} is non-increasing. So, if limk→∞ r (uk , βk ) = 0, there must exist a positive scalar ω such that

lim r uk , βk = ω > 0.

(51)

k→∞

Recall that r (uk , βk ) = uk − uk+1 . It follows from Theorem 3 and (46) that

F (uk ) − F (uk+1 )2 r (uk+1 , βk+1 )2 1 − (δ τ βk ) 1 − (δ τ βmin )ηk2 . k 2 r (u , βk ) βk [uk − uk+1 ]2 Consequently, we have ∞ ∞ r (uk+1 , βk+1 )2 1 − (δ τ βmin )ηk2 . k 2 r (u , βk )

k =1

(52)

k =1

It is from (51) and (52) that ∞

1 − (δ τ βmin )ηk2

k =1

ω2 r (u 1 , β

2 1 )

> 0.

(53)

By the choice of bk = (δ τ βmin )ηk2 , it follows from Lemma 3 and (53) that ∞

ηk2 < +∞,

k =1

lim

k→∞

ηk2 = 0 and

∞ k =1

Since limk→∞ ηk2 = 0, there is a k0 > 0 such that

2ηk2

1 2

,

∀k k0 .

1 + c ηk2 < ∞,

∀c > 0.

(54)

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

225

Thus, for this k0 > 0, we have

1

1 1 − 2ηk2

1 + 4ηk2 ,

∀k k0 .

(55)

Therefore, it follows from Lemma 6 and (55) that

k +1 2 2 1 2 u − u ∗ 1 + 4ηk2 uk − u ∗ − uk − uk+1 , 2

∀k k0 .

(56)

Obviously, (56) implies that k k +1 2 2 u − u∗ 1 + 4ηl2 uk0 − u ∗ l=k0

∞

2 1 + 4ηl2 uk0 − u ∗ := E ,

∀k k0 .

(57)

l=k0

∞

2 l=k0 (1 + 4 l )

Recall that

η < +∞ (see (54)). We have E > 0. Then, it follows from (56) and (57) that

∞ 1

2

∞ (56) 2 2 u k − u k +1 2 4 ηk2 uk − u ∗ + uk0 − u ∗

k=k0

k=k0

(57)

Using the facts that

∞

k=k0

4

∞

2 k

η + 1 E.

(58)

k=k0

ηk2 < +∞ and r (uk , βk ) = uk − uk+1 , it follows from (58) that

lim r uk , βk = 0,

k→∞

which is a contradiction with (51). Hence, limk→∞ r (uk , βk ) = 0. Recall that solving VVI(Ω, F ) is equivalent to ﬁnding a zero point of r (u , β). Thus, if {(u ∗ , β ∗ )} is an accumulation point of the sequence {(uk , βk )} generated by Algorithm 2, then u ∗ is a solution point of the co-coercive VVI(Ω, F ). 2 6. Numerical experiments In this section, we apply the proposed Goldstein’s type methods to solve some test problems, and compare them numerically with some existing methods. In particular, we test the following problem which has been studied in [13]:

min

1 2

x − c 2

s.t. Ax ∈ B ,

(59)

where A ∈ R m×n , c ∈ R n and B ⊂ R m is a closed convex set. We assume that the solution set of the inclusion Ax ∈ B is not empty. As analyzed in [13], this problem is equivalent to the following m-dimensional VVI:

A A T y ∗ + Ac ∈ B ,

T ξ − A A T y ∗ + Ac y ∗ 0,

∀ξ ∈ B;

in the sense that the solution of (59) can be obtained immediately via x∗

(60)

=

A y∗ T

+ c, where y ∗ is a solution of (60).

To compare with the method in [13], we form the test problem in the same way as described in [13]:

A := U Σ V T

with U = I m − 2

uu T

u 2

and

V = In − 2

vvT

v 2

;

and the Householder matrix Σ = diag(σk ) is an m × n diagonal matrix where the vectors u, v and c contain pseudo random numbers in the following fashion:

u 1 = 13846; u i = (31416u i −1 + 13846) mod 46261,

i = 2, . . . , m ;

v 1 = 13846; v j = (42108v j −1 + 13846) mod 46273,

j = 2, . . . , n ;

226

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

c 1 = 13846; c i = (45278c i −1 + 13846) mod 46219,

i = 2, . . . , n .

The closed convex set B in (59) is

B := z ∈ R m z a ,

where a is a given constant. In the test problems we set σk = cos lk+π1 + 1, k = 1, . . . , l = min{m, n}. The singular values of matrix A tend to cluster at the endpoints of the interval [0, 2]. By denoting

F ( y ) = A A T y + Ac

and

Ω = B,

(61)

the problem (60) can be compactly rewritten into the following variant variational inequality (VVI(Ω, F )):

T ξ − F ( y ) y 0,

F ( y ) ∈ Ω,

∀ξ ∈ Ω.

(62)

Theorem 5. The mapping F ( y ) in (62) is co-coercive. In detail, for all y¯ , y˜ ∈ R m we have

2 ( y¯ − y˜ )T F ( y¯ ) − F ( y˜ ) τ F ( y¯ ) − F ( y˜ ) ,

(63)

where τ = 1/{maxk {σk2 }}. Proof. First, from (61) we have

( y¯ − y˜ )T F ( y¯ ) − F ( y˜ ) = ( y¯ − y˜ )T A A T ( y¯ − y˜ )

(64)

F ( y¯ ) − F ( y˜ )2 = ( y¯ − y˜ )T A A T A A T ( y¯ − y˜ ).

(65)

and

Note that A = U Σ V T , then we obtain

A AT = U Σ V T V Σ U T = U Σ 2U T

(66)

and

A AT A AT = U Σ 2U T

U Σ2U T = U Σ4U T .

(67)

It follows from (64) and (66) that

T ( y¯ − y˜ )T F ( y¯ ) − F ( y˜ ) = ( y¯ − y˜ )T U ΣΣ U T ( y¯ − y˜ ) = Σ U T ( y¯ − y˜ ) Σ U T ( y¯ − y˜ ) .

(68)

Similarly, from (65) and (67), we have

F ( y¯ ) − F ( y˜ )2 = Σ U T ( y¯ − y˜ ) T Σ 2 Σ U T ( y¯ − y˜ ) .

(69)

Obviously, since Σ = diag(σk ), the following inequality always holds

max k

T T Σ U T ( y¯ − y˜ ) Σ U T ( y¯ − y˜ ) Σ 2 Σ U T ( y¯ − y˜ ) ,

σk2 Σ U T ( y¯ − y˜ )

∀ y¯ , y˜ ∈ R m ,

that is

2 ( y¯ − y˜ )T F ( y¯ ) − F ( y˜ ) τ F ( y¯ ) − F ( y˜ ) , where

∀ y¯ , y˜ ∈ R m ,

τ = 1/ maxk {σk2 }. The proof is complete. 2

During the numerical implementation, we take the same stopping criterion as in [13]:

A A T y + Ac − a 5 · 10−6 and a

r ( y , 1) a

5 · 10−6 .

As pointed out in [13], A A T y ∗ + Ac = a when Ac > a (otherwise y ∗ = 0 is the trivial solution). Therefore, we test the problem with different a < Ac . All the codes were written in MATLAB and run on an HP Compaq 6910p notebook with a 2.0 GHz processor, duo cores, and 2.00 GB of memory. We compare the proposed methods with the Goldstein’s type methods in [13]. All methods start

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

227

Table 1 Comparison of the new methods with the method in [13]. m = 200,000, n = 100,000

a

Method in [13] 0.05 Ac 0.10 Ac 0.15 Ac

Algorithm 1

Algorithm 2

No. of It.

CPU (s)

No. of It.

CPU (s)

No. of It.

CPU (s)

537 184 99

20.09 6.77 3.67

433 150 81

17.37 6.00 3.24

417 144 78

18.18 6.40 3.49

Table 2 Comparison of Algorithm 2 to the method in [13]. m = 10,000, n = 10,000

a

Method in [13] 0.05 Ac 0.10 Ac 0.15 Ac

Algorithm 2

No. of It.

CPU (s)

No. of It.

CPU (s)

1346 1345 1345

1.35 1.37 1.35

419 142 79

0.48 0.17 0.09

No. of It.

CPU (s)

No. of It.

CPU (s)

1344 1343 1343

24.92 24.83 25.04

417 144 78

8.91 3.14 1.72

Table 3 Comparison of Algorithm 2 to the method in [13]. m = 100,000, n = 200,000

a

Method in [13] 0.05 Ac 0.10 Ac 0.15 Ac

Algorithm 2

the iterations from y 0 = 0. For the Goldstein’s type method [13], βk ≡ 2.5 (as in [13]). For Algorithm 1, we take β0 = 1/τ , β L = 1/(2τ ), βU = 1000/τ and γ ≡ 1.99. For Algorithm 2, let β−1 = 1.0, δ = 0.2, μ = 1.6 and

tk :=

F ( yk ) − F ( yk+1 )2 . βk ( yk − yk+1 )T [ F ( yk ) − F ( yk+1 )]

(70)

Too large values of βk , however, usually lead to extremely slow convergence according to our numerical experiments. Thus, it is necessary to avoid this situation. To do so, a strategy of decreasing βk is proposed in practice. If tk < 1.0, the value of βk is reduced according to βk+1 := βk ∗ 0.8. This adjustment only needs to be applied for ﬁnitely many of times. Hence, convergence of Algorithm 2 remains effective. We report the iteration numbers and CPU time of these three methods for various scenarios of (m, n, a) in Tables 1–3. As shown in Table 1, the proposed Goldstein’s type methods are both eﬃcient for solving co-coercive VVIs. To achieve an approximate solution with the same accuracy, they need less time compared with the method in [13]. On the other hand, compared with Algorithm 1, Algorithm 2 requires a bit more time even though it needs less iterations. This is due to the computation involved in the Armijo’s type procedure for identifying appropriate values of βk ’s in the implementation of Algorithm 2. Nevertheless, as we emphasized previously, Algorithm 2 is implementable for the case where the co-coercive modulus of VVIs is unknown. In the following, we implement the method in [13] and Algorithm 2 to solve some scenarios where values of the involved co-coercive modulus of the tested example are unknown. The initial iterations for implementing these two methods are both y 0 = 0. For the method in [13], we let βk ≡ 1.5. For Algorithm 2, we let β−1 = 1.5, δ = 0.2, μ = 1.6. Some numerical results are reported in Tables 2 and 3. Tables 2 and 3 show that Algorithm 2 is eﬃcient for solving co-coercive VVIs when the co-coercive modulus is unknown. References [1] T.M. Apostol, Mathematical Analysis, 2nd ed., Addison–Wesley Publishing Company, 1981 (ﬁfth printing). [2] L. Armijo, Minimization of functions having continuous partial derivatives, Paciﬁc J. Math. 16 (1966) 1–3. [3] H.H. Bauschke, P.L. Combettes, A weak-to-strong convergence principle for Fejér monotone methods in Hilbert spaces, Math. Oper. Res. 26 (2) (2001) 248–264. [4] S. Dafermos, A. Nagurney, Supply and demand equilibration algorithms for a class of market equilibrium problems, Transport. Sci. 23 (2) (1989) 118–124. [5] Y.H. Dai, Y.X. Yuan, Alternate minimization gradient method, IMA J. Numer. Anal. 23 (2003) 377–393. [6] J. Eckstein, D.P. Bertsekas, On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program. 55 (1992) 293–318.

228

M. Li et al. / Applied Numerical Mathematics 61 (2011) 216–228

[7] F. Facchinei, J.S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, Springer Series in Operations Research, SpringerVerlag, 2003. [8] M.C. Ferris, J.S. Pang, Engineering and economic applications of complementarity problems, SIAM Rev. 39 (1997) 669–713. [9] D. Gabay, Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, in: Applications of the Method of Multipliers to Variational Inequalities, North-Holland, Amsterdam, 1983. [10] A.A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc. 70 (1964) 709–710. [11] D.R. Han, H.K. Lo, Solving non-additive traﬃc assignment problems: A descent method for co-coercive variational inequalities, Eur. J. Oper. Res. 159 (2004) 529–544. [12] B.S. He, Solving a class of linear projection equations, Numer. Math. 68 (1) (1994) 71–80. [13] B.S. He, A Goldstein’s type projection method for a class of variant variational inequalities, J. Comput. Math. 17 (4) (1999) 425–434. [14] B.S. He, Inexact implicit methods for monotone general variational inequalities, Math. Program. 86 (1999) 199–217. [15] B.S. He, X.Z. He, H.X. Liu, T. Wu, Self-adaptive projection method for co-coercive variational inequalities, Eur. J. Oper. Res. 196 (1) (2009) 43–48. [16] B.S. He, X.M. Yuan, J.Z. Zhang, Comparsion of two kinds of prediction–correction methods for monotone variational inequalities, Comput. Optim. Appl. 27 (3) (2004) 247–267. [17] A.N. Iusem, An iterative algorithm for the variational inequality problem, Comput. Appl. Math. 13 (1994) 103–114. [18] A.N. Iusem, B. Svaiter, A variant of Korpelevich’s method for variational inequalities with a new search strategy, Optimization 42 (1997) 309–321. [19] E.N. Khobotov, Modiﬁcation of the extra-gradient method for solving variational inequalities and certain optimization problems, USSR Comput. Math. Math. Phys. 27 (5) (1989) 120–127. [20] D. Kinderlehrer, G. Stampacchia, An Introduction to Variational Inequalities and Their Applications, Academic Press, New York, 1980. [21] G.M. Korpelevich, The extragradient method for ﬁnding saddle points and other problems, Ekon. Math. Metody 12 (1976) 746–756; English translation: Matecon 13 (1977) 35–49. [22] E.S. Levitin, B.T. Polyak, Constrained minimization problems, USSR Comput. Math. Math. Phys. 6 (1966) 1–50. [23] M. Li, L.-Z. Liao, X.M. Yuan, The variant VI model for second-best road pricing, manuscript, 2007. [24] M. Li, L.-Z. Liao, X.M. Yuan, A modiﬁed projection method for co-coercive variational inequalities, Eur. J. Oper. Res. 189 (2) (2008) 310–323. [25] M. Li, L.-Z. Liao, X.M. Yuan, Proximal point algorithms for general variational inequalities, J. Optim. Theory Appl. 142 (2009) 125–145. [26] M. Li, X.M. Yuan, An improved Goldstein’s type method for a class of variant variational inequalities, J. Comput. Appl. Math. 214 (1) (2008) 304–312. [27] T.L. Magnanti, G. Perakis, A unifying geometric solution framework and complexity analysis for variational inequalities, Math. Program. 71 (1995) 327–352. [28] T.L. Magnanti, G. Perakis, The orthogonality theorem and the strong-f-monotonicity condition for variational inequality algorithms, SIAM J. Optim. 7 (1997) 248–273. [29] P. Marcotte, Application of Khobotov’s algorithm to variational inequalities and network equilibrium problems, Inf. Syst. Oper. Res. 29 (1991) 258–270. [30] P. Marcotte, J.H. Wu, On the convergence of projection methods: Application to the decomposition of aﬃne variational inequalities, J. Optim. Theory Appl. 85 (2) (1995) 347–362. [31] A. Nagurney, Network Economics, A Variational Inequality Approach, Kluwer Academics Publishers, Dordrecht, 1993. [32] M.A. Noor, Quasi variational inequalities, Appl. Math. Lett. 1 (1988) 367–370. [33] J.V. Outrata, J. Zowe, A Newton method for a class of quasi-variational inequalities, Comput. Optim. Appl. 4 (1995) 5–21. [34] J.S. Pang, J.C. Yao, On a generalization of a normal map and equation, SIAM J. Optim. 33 (1995) 168–184. [35] P. Tseng, Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming, Math. Program. 48 (1990) 249–264. [36] Y.B. Zhao, D. Li, Monotonicity of ﬁxed point and normal mappings associated with variational inequality and its application, SIAM J. Optim. 11 (2001) 962–973. [37] D.L. Zhu, P. Marcotte, Co-coercivity and its role in the convergence of iterative schemes for solving variational inequalities, SIAM J. Optim. 6 (1996) 714–726.

Applied Numerical Mathematics 61 (2011) 229–240

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

Waveform relaxation method for stochastic differential equations with constant delay ✩ Zhencheng Fan Department of Mathematics, Minjiang University, Fuzhou 350108, China

a r t i c l e

i n f o

Article history: Received 23 July 2009 Received in revised form 7 August 2010 Accepted 5 October 2010 Available online 16 October 2010 Keywords: Waveform relaxation methods Superlinear convergence Stochastic differential equations Stochastic delay differential equations Mean square convergence

a b s t r a c t This paper extends the waveform relaxation method to stochastic differential equations with constant delay terms, gives suﬃcient conditions for the mean square convergence of the method. A lot of attention is paid to the rate of convergence of the method. The conditions of the superlinear convergence for a special case, which bases on the special splitting functions, are given. The theory is applied to a one-dimensional model problem and checked against results obtained by numerical experiments. © 2010 Published by Elsevier B.V. on behalf of IMACS.

1. Introduction The waveform relaxation (WR) method, also called dynamic iteration or Picard–Lindelöf iteration, is a highly parallel iterative method for numerically solving large-scale systems of ordinary differential equations and ﬁrst proposed by Lelarasmee et al. [10] for analyzing large-scale integrated circuit. It has been observed by many authors that the approach based on the WR method can be quite competitive with the classical methods based, for example, on discrete variable methods such as Runge–Kutta, linear multistep or predictor–corrector methods for differential systems. The WR method has been applied successfully to the linear problems. Nevanlinna [14] has proved superlinear convergence of dynamic iterations for linear differential systems on ﬁnite time interval. Jan Jansen et al. [7] deal with the acceleration of the standard WR method by successive overrelaxation techniques. Sand and Burrage [16] focus on the approach to the parallelism and present a highly parallel method. Nichols [15] ﬁrst proposed the two-stage iterative method in 1973, then it has been further studied by other authors (see [5,18,21]). The WR method has been also applied extensively to more general, time-dependent coeﬃcient problems and to nonlinear problems. It has been extended to, for example, differential-algebraic systems of neutral type [1], integro-algebraic systems and differential-algebraic systems (see [2] and [8]), systems of second kind Volterra integral equations [3], stiff nonlinear ordinary differential equations [6], functional-differential equations [22], neutral delay differential equations [19], Volterra type systems of neutral differential-functional equations [20]. In these references, the error estimates are derived, which imply the linear convergence of WR methods. In particular, [3] and [20] obtain the conditions of superlinear convergence for some special cases. Stochastic differential equations (SDEs) are increasingly used nowadays to model real-world phenomena by including effects of random perturbations on the time evolution of the state of the system (see [4] and [11]). Most of the SDEs cannot

✩ This work is partially supported by the NSF of P.R. China (No. 10901036) and the Science and Technology Project from the Education Ministry of Fujian in P.R. China (No. JA09192). E-mail address: [email protected].

0168-9274/$30.00 © 2010 Published by Elsevier B.V. on behalf of IMACS. doi:10.1016/j.apnum.2010.10.003

230

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

be solved explicitly and many attempts have been made to develop the eﬃcient numerical methods (see [9] and [12]). To our best knowledge, the unique work of applications of the WR method to SDEs is [17] in which L p convergence of WR methods for numerical solving of systems of ordinary stochastic differential equations (SODEs) is studied and the suﬃcient conditions for the linear L p convergence of the method are given. Such conditions show that the Lipschitz-continuity of the coeﬃcients of SODEs and the form of its splitting into subsystems are crucial to establish the convergence of the method. However, we have two reasons to believe that the conditions should be relaxed. First, even the conditions don’t hold for some linear systems, and second the conditions heavily depend on the spectral radius of the large matrixes, which is obtained hardly. Stochastic delay differential equations (SDDEs) can be viewed as generalizations of SODEs. In many scientiﬁc ﬁelds, such as ﬁnance, biology, mechanics, and ecology, SDDEs are often used to model the corresponding systems. In recent years, there has been growing interesting in studying such equations. For the research in theoretical solutions of SDDEs, one can refer to [11,13] and the references therein. In convenience, here we will concentrate on SDEs with constant delay terms. In fact, the results of this paper can be extended without any diﬃculty to the more general stochastic system in which the delay is time dependent (see [11, p. 156]). The purpose of this paper is to extend the WR method to SDEs with constant delay. We will prove the WR method for SDEs with constant delay is convergent in mean square sense under Lipschitz conditions. The superlinear convergence of the WR method with the special splitting functions will be also derived. 2. Convergence of WR method Throughout this paper, we let (Ω, F , {Ft }t 0 , P ) be a complete probability space with a ﬁltration {Ft }t 0 satisfying the usual conditions (i.e. it is increasing and right continuous while F0 contains all P -null sets). Let W = { W (t ), t 0}, W (t ) = ( W t1 , . . . , W td )T , be a d-dimensional Brownian motion deﬁned on the probability space. Let | · | denote the Euclidean vector norm as well as the matrix trace norm (see [11]). Let r be a positive integer and τ > 0. Let C ([−τ , 0]; Rr ) denote the family of continuous function ϕ from [−τ , 0] to Rr with the norm ϕ = sup−τ θ0 |ϕ (θ)|. Let B (C ([−τ , 0], Rr )) denote Borel

σ -algebra of C ([−τ , 0], Rr ) and L 2F0 ([−τ , 0]; Rr ) denote the family of (F0 , B(C ([−τ , 0], Rr )))-measurable C ([−τ , 0]; Rr )-

valued random variables ξ such that E ξ 2 < ∞ (see [13]). We consider the following stochastic delay differential equation

dX (t ) = f t , X (t ), X (t − τ ) dt + g t , X (t ), X (t − τ ) dW (t ), X (t ) = ξ(t ),

t ∈ (0, T ],

t ∈ [−τ , 0],

(2.1) r ×d

where T > 0, ξ = (ξ(t ))−τ t 0 ∈ L F ([−τ , 0]; R ), f : [0, T ] × R × R → R and g : [0, T ] × R × R → R are the given 0 continuous functions. Let the continuous functions F : [0, T ] × Rr × Rr × Rr × Rr → Rr and G : [0, T ] × Rr × Rr × Rr × Rr → Rr ×d denote the splitting functions for Eq. (2.1), that is 2

r

r

r

r

r

r

f t , X (t ), X (t − τ ) = F t , X (t ), X (t ), X (t − τ ), X (t − τ ) , g t , X (t ), X (t − τ ) = G t , X (t ), X (t ), X (t − τ ), X (t − τ ) . In this section we shall always assume that

F (t , x1 , x2 , x3 , x4 ) − F (t , y 1 , y 2 , y 3 , y 4 )2 K |x1 − y 1 |2 + |x2 − y 2 |2 + |x3 − y 3 |2 + |x4 − y 4 |2

(2.2)

G (t , x1 , x2 , x3 , x4 ) − G (t , y 1 , y 2 , y 3 , y 4 )2 K |x1 − y 1 |2 + |x2 − y 2 |2 + |x3 − y 3 |2 + |x4 − y 4 |2

(2.3)

and

for some K > 0 and any t 0, x1 , x2 , x3 , x4 , y 1 , y 2 , y 3 , y 4 ∈ Rr . The form of a WR method considered in this section is

dX (k+1) (t ) = F t , X (k+1) (t ), X (k) (t ), X (k+1) (t − τ ), X (k) (t − τ ) dt

+ G t , X (k+1) (t ), X (k) (t ), X (k+1) (t − τ ), X (k) (t − τ ) dW (t ),

X

(k+1)

(t ) = ξ(t ),

t ∈ (0, T ],

t ∈ [−τ , 0],

(2.4)

where k = 0, 1, . . . and the initial approximation X (0) (t ) to X (t ) satisﬁes X (0) (t ) = ξ(t ), t ∈ [−τ , 0] and X (0) (t ) = ξ(0), t ∈ (0, T ]. Consider the equation

dX (t ) = F t , X (t ), Y (t ), X (t − τ ), Y (t − τ ) dt + G t , X (t ), Y (t ), X (t − τ ), Y (t − τ ) dW (t ), X (t ) = ξ(t ),

t ∈ [−τ , 0],

t ∈ (0, T ], (2.5)

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

231

where Y : [−τ , T ] × Ω → Rr is a measurable {Ft }-adapted process such that E (sup−τ t T |Y (t )|2 ) < ∞. Using the same techniques as in the proof of Theorem 5.2.2 in monograph [11] we can easily obtain the following theorem. Theorem 2.1. Assume that (2.2) and (2.3) hold. Then there exists a unique solution ( X (t ))−τ t T to Eq. (2.5) and the solution satisﬁes E (sup−τ t T | X (t )|2 ) < ∞. The solution mentioned in this paper is the strong solution whose deﬁnition can be found on p. 149 in Ref. [11]. This theorem implies that under (2.2) and (2.3) there exists a unique solution to Eq. (2.4) with any nonnegative integer k and the solution satisﬁes E (sup−τ t T | X (k+1) (t )|2 ) < ∞. In this section, we discuss the convergence of WR method. To do this, the following lemma is useful. Lemma 2.2. Let 0 < b < 1. Assume that there exist the positive real numbers ak , B k , k = 0, 1, . . . such that limk→∞ ak = 0 and B k+1 bB k + ak . Then limk→∞ B k = 0. Proof. By B k+1 bB k + ak we can obtain that B k+1 bk+1 B 0 + limk→∞

k

i =0 b

k −i

k

ai = 0. By limn→∞ an = 0 we have that for any given

i =0 b

k −i

ai . Clearly, it is enough to prove that

> 0, there is a positive constant N such that = 0 we have that for the above , there is a positive

N

if n > N, then |an | < 2(1−b) . Let B = i =0 b N −i ai . By limk→∞ bk− N ˜ > N such that if k > N, ˜ then bk−N < . Thus if k > N, ˜ then constant N 2B

k N k k k −i b ai = bk −i a i + bk −i a i bk − N B + bk −i < 2(1 − b) i =0

i =0

i = N +1

and the proof is complete.

i = N +1

2

By this lemma we will show the mean square convergence of WR method (2.4) under Lipschitz conditions (2.2) and (2.3). Theorem 2.3. Assume that (2.2) and (2.3) hold. Then the sequence ( X (k) (t ))−τ t T (k = 0, 1, . . .) produced by the WR method (2.4) converges uniformly to the solution ( X (t ))−τ t T of (2.1) in mean square, that is

lim E

k→∞

2 sup X (k) (t ) − X (t ) = 0.

−τ t T

Proof. For simplicity, we denote by

F (s) F s, X (s), X (s), X (s − τ ), X (s − τ ) , G (s) G s, X (s), X (s), X (s − τ ), X (s − τ ) ,

F (k) (s) F s, X (k+1) (s), X (k) (s), X (k+1) (s − τ ), X (k) (s − τ ) , G (k) (s) G s, X (k+1) (s), X (k) (s), X (k+1) (s − τ ), X (k) (s − τ ) . Eqs. (2.1) and (2.4) can be rewritten as

t X (t ) = X (0) +

t F (s) ds +

0

X (t ) = ξ(t ),

G (s) dW (s),

t ∈ (0, T ],

0

t ∈ [−τ , 0]

(2.6)

and

X (k+1) (t ) = X (k+1) (0) +

t

t F (k) (s) ds +

0

X

(k+1)

(t ) = ξ(t ),

G (k) (s) dW (s),

t ∈ (0, T ],

0

t ∈ [−τ , 0],

where k = 0, 1, 2, . . . , X (0) (t ) = ξ(t ), t From (2.6) and (2.7) we have

(2.7)

∈ [−τ , 0] and X (0) (t ) = ξ(0), t

∈ (0, T ]. Deﬁne e (k) (t ) X (k) (t ) − X (t ), k

= 0, 1, 2, . . . .

232

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

e (k+1) (t ) =

t

t

F (k) (s) − F (s) ds +

0

G (k) (s) − G (s) dW (s),

t ∈ (0, T ]

(2.8)

0

and

e (k+1) (t ) = 0,

t ∈ [−τ , 0]

(2.9)

for k = 0, 1, 2, . . . . Using the element inequality |a + b| 2|a| + 2|b| , the Hölder inequality and (2.8) we get 2

2

2

2 t 2 t (k+1) 2 ( k ) ( k ) e (t ) 2 F (s) − F (s) ds + 2 G (s) − G (s) dW (s) 0

t 2t

0

2 t (k) 2 ( k ) F (s) − F (s) ds + 2 G (s) − G (s) dW (s) .

0

0

From the above inequality we have

2 sup e (k+1) (t ) 2t 1

0t t 1

t1 0

t 2 (k) 2 ( k ) F (s) − F (s) ds + 2 sup G (s) − G (s) dW (s) , 0t t 1 0

which together Doob’s Martingale inequality (see e.g. [12, Theorem 1.3.8 on p. 14]) imply

E

2 sup e (k+1) (t ) 2t 1

0t t 1

t1

2 E F (k) (s) − F (s) ds + 8

t1

0

2

E G (k) (s) − G (s) ds.

(2.10)

0

From (2.2), (2.3) and (2.10) we have

E

2 sup e (k+1) (t ) (2t 1 + 8) K

0t t 1

t1

2 2 2 2 E e (k+1) (s) + e (k) (s) + e (k+1) (s − τ ) + e (k) (s − τ ) ds.

(2.11)

0

From (2.9) and (2.11) we have

E

2 sup e (k+1) (t ) (2t 1 + 8) K

0t t 1

t1

2 2 E e (k+1) (s) + e (k) (s) ds

0

2 2 (2t 1 + 8) K t 1 E sup e (k+1) (t ) + E sup e (k) (t ) 0t t 1

(2.12)

0t t 1

for any t 1 ∈ (0, τ ]. Using (2.12) we can obtain

E

2 sup e (k+1) (t )

0t t 1

2 (2t 1 + 8) K t 1 E sup e (k) (t ) 1 − (2t 1 + 8) K t 1 0t t 1

(2.13)

(2T +8) K T

for any t 1 ∈ (0, (2τ +1 8) K ). Let T 1 = min{τ , 3(2τ 1+8) K }, then 1−(2T1 +8) K 1T ∈ (0, 12 ]. Using (2.12) and (2.13) we deduce 1 1

lim E

k→∞

2 sup e (k) (t ) = 0.

(2.14)

0t T 1

From (2.6) and (2.7) we get

X (k+1) (t ) − X (t ) = X (k+1) ( T 1 ) − X ( T 1 ) +

t

t

F (k) (s) − F (s) ds +

T1

G (k) (s) − G (s) dW (s)

for t T 1 .

T1

Using the element inequality |a + b + c | 3(|a| + |b| + |c | ) and the Hölder inequality we can obtain from the above inequality 2

(k+1) 2 2 e (t ) 3e (k+1) ( T 1 ) + 3(t − T 1 )

2

t T1

2

2

2 t (k) (k) F (s) − F (s)2 ds + 3 G (s) − G (s) dW (s) for t T 1 . T1

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

233

So for any t 2 T 1 we have

t2

2 2 sup e (k+1) (t ) 3e (k+1) ( T 1 ) + 3(t 2 − T 1 )

T 1 t t 2

T1

2 t (k) 2 ( k ) F (s) − F (s) ds + 3 sup G (s) − G (s) dW (s) . T 1 t t 2 T1

Taking the expectation on the both sides of the above inequality and using Doob’s Martingale inequality, (2.2) and (2.3) we have

E

2 2 sup e (k+1) (t ) 3E e (k+1) ( T 1 ) + 3(t 2 − T 1 )

T 1 t t 2

t2 T1

·

2

E G (k) (s) − G (s) ds

T1

2 3E e (k+1) ( T 1 ) + 3(t 2 − T 1 ) + 12 K t2

t2

2 E F (k) (s) − F (s) ds + 12

2 2 2 2 E e (k+1) (s) + e (k) (s) + e (k+1) (s − τ ) + e (k) (s − τ ) ds.

(2.15)

T1

Let t 2 ∈ [ T 1 , T 1 + τ ] and A (3(t 2 − T 1 ) + 12) K (t 2 − T 1 ). By (2.9) and (2.15) we have

E

2 2 sup e (k+1) (t ) 3E e (k+1) ( T 1 ) + A E

T 1 t t 2

+A E So for 0 < A < 1 we have

E

2 sup e (k+1) (t )

T 1 t t 2

1− A

+ Deﬁne T 2 min{ T 1 + τ , T 1 +

E

3K (3τ +12)

2 sup e (k+1) (t )

T 1 t t 2

2 sup e (k) (t )

T 1 t t 2

0t T 1

2

E e (k+1) ( T 1 ) + E

A

E

2 sup e (k) (t )

1− A T 1 t t 2 (k+1) 2 2 . sup e (t ) + E sup e (k) (t )

1− A

1

T 1 t t 2

0t T 1

A

2 sup e (k+1) (t ) + A E

2 2 . sup e (k+1) (t ) + E sup e (k) (t )

3

0t T 1

0t T 1

}. For t 2 ∈ [ T 1 , T 2 ] we have 0 A 13 , 0

A 1− A

1 2

and 0 1−1 A 32 , so that

2 9 2 1 sup e (k) (t ) + E e (k+1) ( T 1 ) E 2 2 T 1 t t 2 +

2 1 2 1 sup e (k+1) (t ) + E sup e (k) (t ) . E 2 2 0t T 1 0t T 1

It then follows from (2.14) and Lemma 2.2 that

lim E

k→∞

2 sup e (k+1) (t ) = 0.

T 1 t T 2

Deﬁne T n+1 min{ T n + τ , T n +

lim E

sup

k→∞

Then

T n t T n+1

lim E

k→∞

1 }, 3K (3τ +12)

(k+1) 2 e (t ) = 0,

n = 1, 2, . . . . Similarly

n = 2, 3, . . . .

2 sup e (k+1) (t ) = 0

0t T

and the proof is complete.

2

3. Superlinear convergence of WR method In this section the form of the WR method considered is

dX (k+1) (t ) = Fˆ t , X (k+1) (t ), X (k) (t ), X (k) (t − τ ) dt + Gˆ t , X (k+1) (t ), X (k) (t ), X (k) (t − τ ) dW (t ), X

(k+1)

(t ) = ξ(t ),

t ∈ [−τ , 0],

k = 0, 1 , . . . ,

t ∈ (0, T ], (3.1)

234

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

and the initial approximation X (0) (t ) to X (t ) satisﬁes X (0) (t ) = ξ(t ), t ∈ [−τ , 0] and X (0) (t ) = ξ(0), t ∈ (0, T ]. Here the splitting functions Fˆ : [0, T ] × Rr × Rr × Rr → Rr and Gˆ : [0, T ] × Rr × Rr × Rr → Rr ×d satisfy

Fˆ t , X (t ), X (t ), X (t − τ ) = f t , X (t ), X (t − τ ) , Gˆ t , X (t ), X (t ), X (t − τ ) = g t , X (t ), X (t − τ ) . In this section we assume that there exist positive constants K 1 , K 2 , K 3 , K 4 , K 5 , K 6 such that for any x, y , z, xˆ , yˆ , zˆ ∈ Rr

(x − xˆ )T Fˆ (t , x, y , z) − Fˆ (t , xˆ , y , z) K 1 |x − xˆ |2 , Fˆ (t , x, y , z) − Fˆ (t , x, yˆ , zˆ )2 K 2 y − yˆ 2 + K 3 z − zˆ 2

(3.2)

Gˆ (t , x, y , z) − Gˆ (t , xˆ , yˆ , zˆ )2 K 4 |x − xˆ |2 + K 5 | y − yˆ |2 + K 6 z − zˆ 2 .

(3.3)

and

Denote by ( X (t ))−τ t T and ( X (k) (t ))−τ t T the solution of (2.1) and (3.1), respectively and let e (k) (t ) = X (k) (t ) − X (t ), −τ t T . Clearly, e (k) (t ) = 0 for t ∈ [−τ , 0]. Theorem 3.1. Assume that (3.2) and (3.3) hold. Then the sequence ( X (k) (t ))−τ t T (k = 0, 1, . . .) produced by the WR method (3.1) is superlinearly convergent, that is

2 sup E X (k+1) (s) − X (s) e

t 0

K˜ 1 ds

0st

t k +1

(k + 1)!

2 ( K˜ 2 + K˜ 3 )k+1 max E e (0) (s) , 0st

t ∈ [0, T ],

where ( X (t ))−τ t T is the solution of (2.1), k = 0, 1, 2, . . . , K˜ 1 = 2K 1 + K 4 + 1, K˜ 2 = K 2 + K 5 and K˜ 3 = K 3 + K 6 . Proof. In a similar way as in the proof of Theorem 2.3, we denote by

Fˆ (s) Fˆ s, X (s), X (s), X (s − τ ) , Gˆ (s) Gˆ s, X (s), X (s), X (s − τ ) ,

Fˆ (k) (s) Fˆ s, X (k+1) (s), X (k) (s), X (k) (s − τ ) , Gˆ (k) (s) Gˆ s, X (k+1) (s), X (k) (s), X (k) (s − τ ) . From (2.1) and (3.1) we can obtain for t ∈ [0, T ]

de (k+1) (t ) = Fˆ (k) (t ) − Fˆ (t ) dt + Gˆ (k) (t ) − Gˆ (t ) dW (t ). By virtue of Itô formula we have

2 2 T (k) de (k+1) (t ) = 2 e (k+1) (t ) Fˆ (t ) − Fˆ (t ) dt + Gˆ (k) (t ) − Gˆ (t ) dt T + 2 e (k+1) (t ) Gˆ (k) (t ) − Gˆ (t ) dW (t ),

t ∈ [0, T ],

which can be written as

(k+1) 2 (k+1) 2 e (t ) = e (0) +

t

2 e

(k+1)

T (s) Fˆ (k) (s) − Fˆ (s) ds +

0

t +

t

T (k) 2 e (k+1) (s) Gˆ (s) − Gˆ (s) dW (s)

0

(k) Gˆ (s) − Gˆ (s)2 ds,

t ∈ [0, T ].

0

Taking the expectation on the both sides of the above equality we have

2 2 E e (k+1) (t ) = E e (k+1) (0) +

t

2E e

(k+1)

T (s) Fˆ (k) (s) − Fˆ (s) ds +

0

t

2

E Gˆ (k) (s) − Gˆ (s) ds,

t ∈ [0, T ],

0

and so that for t ∈ [0, T ]

dE |e (k+1) (t )|2 dt

2 T = 2E e (k+1) (t ) Fˆ (k) (t ) − Fˆ (t ) + E Gˆ (k) (t ) − Gˆ (t ) .

(3.4)

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

235

By (3.2) and (3.3) we can derive that for t ∈ [0, T ]

T

e (k+1) (t )

T Fˆ t , X (k+1) (t ), X (k) (t ), X (k) (t − τ ) − Fˆ t , X (t ), X (k) (t ), X (k) (t − τ ) T + e (k+1) (t ) Fˆ t , X (t ), X (k) (t ), X (k) (t − τ ) − Fˆ t , X (t ), X (t ), X (t − τ ) 2 1 2 1 2 1 2 (3.5) K 1 e (k+1) (t ) + e (k+1) (t ) + K 2 e (k) (t ) + K 3 e (k) (t − τ ) ,

Fˆ (k) (t ) − Fˆ (t ) = e (k+1) (t )

2

2

2

and

(k) Gˆ (t ) − Gˆ (t )2 K 4 e (k+1) (t )2 + K 5 e (k) (t )2 + K 6 e (k) (t − τ )2 .

(3.6)

Substituting (3.5) and (3.6) into (3.4) we have that

⎧ (k+1) (t )|2 2 2 2 ⎪ ⎨ dE |e K˜ 1 E e (k+1) (t ) + K˜ 2 E e (k) (t ) + K˜ 3 E e (k) (t − τ ) , dt ⎪ (k+1) 2 ⎩ E e (0) = 0, k = 0, 1, . . . ,

t ∈ (0, T ],

where K˜ 1 = 2K 1 + K 4 + 1, K˜ 2 = K 2 + K 5 and K˜ 3 = K 3 + K 6 . Using Gronwall’s inequality we can obtain

2

E e (k+1) (t )

t e

t

K˜ 1 ds1

s

2

2

K˜ 2 E e (k) (s) + K˜ 3 E e (k) (s − τ )

t ∈ [0, T ],

ds,

0

which yields

2 t ˜ max E e (k+1) (s) e − 0 K 1 ds1

t

0st

e−

s

K˜ 1 ds1

0

2

2

K˜ 2 E e (k) (s) + K˜ 3 E e (k) (s − τ )

ds

0

t

2

K˜ 2 max E e (k) (s1 ) e −

s 0

K˜ 1 ds1

0 s 1 s

ds

0

t +

K˜ 3

2

s −τ

max

E e (k) (s1 ) e −

K˜ 1 ds1

and suppose that

0 s 1 s −τ

0

K˜ 1 ds1 −

e

s

s −τ

K˜ 1 ds1

ds,

t ∈ [0, T ].

0

Denote by (k+1) (t ) = max0st E |e (k+1) (s)|2 e − above inequality can be rewritten as

(k+1)

(t ) K˜ 2

t

(k)

˜ (s) ds + K˜ 3 e − K 1 τ

0

t 0

(l) (s) = 0 for s 0 and l = 0, 1, 2, . . . . Then the

t

(k) (s − τ ) ds, t ∈ [0, T ], 0

from which we obtain for t ∈ [0, T ], k = 0, 1, 2, . . .

(k+1)

(t ) ( K˜ 2 )2

t

s

ds 0

(k−1)

0

t

s ds

0

t

s ds

0

ds1

+ 3( K˜ 2 ) K˜ 3 e

s ds 0

(k−1) (s1 − 2τ ) ds1

0

s1

0 2

t 0

˜ 2 + K˜ 3 e − K 1 τ

( K˜ 2 )3

˜ (s1 ) ds1 + 2 K˜ 2 K˜ 3 e − K 1 τ

(k−2) (s2 ) ds2

0

− K˜ 1 τ

t

s ds

0

s1 ds1

0

0

(k−2) (s2 − τ ) ds2

(k−1) (s1 − τ ) ds1

236

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

˜ 2 + 3 K˜ 2 K˜ 3 e − K 1 τ

t

s ds

0

˜ 3 + K˜ 3 e − K 1 τ

k +1 i =0

s1

ds

ds1 0

(k−2) (s2 − 3τ ) ds2

0

······

C ki +1 ( K˜ 2 )k+1−i K˜ 3 e

······ ······ t s − K˜ τ i

C ki +1 ( K˜ 2 )k+1−i ( K˜ 3 )i

ds

1

t

t k +1

(k + 1)!

The theorem is proved.

0

0 sk−1

ds1 · · ·

ds

2 t max E e (0) (u ) (k + 1)! 0u t

······ ······ sk−1 ds1 · · · (0) (sk − i τ ) dsk

s

0 k +1

(k−2) (s2 − 2τ ) ds2

0

s

0

k +1 i =0

······

0

t 0

······

s1 ds1

0 k +1 i =0

max

0u sk −i τ

2

˜

E e (0) (u ) e −sk K 1 dsk

0

C ki +1 ( K˜ 2 )k+1−i ( K˜ 3 )i

2 ( K˜ 2 + K˜ 3 )k+1 max E e (0) (u ) . 0u t

2

Remark 3.2. In this section a new criterion is used to measure the difference between the true solution ( X (t ))−τ t T and the WR iteration ( X (k) (t ))−τ t T , which seems to be reasonable for numerical approximation since sup0t T E | X (k) (t ) − X (t )|2 = 0 implies that P ( X (k) (t i ) = X (t i ), ∀t i ∈ [0, T ], i = 1, 2, . . .) = 1.

4. Numerical experiments In this section we shall discuss a numerical example to illustrate our theory. Consider one-dimensional system given by

dX (t ) = − X (t ) + X (t − 1) dt + X (t ) + X (t − 1) dW (t ), X (t ) = 1 + t ,

t ∈ (0, T ],

t ∈ [−1, 0]

(4.1)

and apply the WR method (2.4) and (3.1) with the different splitting functions to this equation. These splitting functions are (f1) (f2) (f3) (f4) (f5) (f6)

F (t , x, x, y , y ) = −0.1x − 0.9x + 0.1 y + 0.9 y, G (t , x, x, y , y ) = 0.1x + 0.9x + 0.1 y + 0.9 y; F (t , x, x, y , y ) = −0.5x − 0.5x + 0.5 y + 0.5 y, G (t , x, x, y , y ) = 0.5x + 0.5x + 0.5 y + 0.5 y; F (t , x, x, y , y ) = −0.9x − 0.1x + 0.9 y + 0.1 y, G (t , x, x, y , y ) = 0.9x + 0.1x + 0.9 y + 0.1 y; Fˆ (t , x, x, y ) = −0.1x − 0.9x + y, Gˆ (t , x, x, y ) = 0.1x + 0.9x + y; Fˆ (t , x, x, y ) = −0.5x − 0.5x + y, Gˆ (t , x, x, y ) = 0.5x + 0.5x + y; Fˆ (t , x, x, y ) = −0.9x − 0.1x + y, Gˆ (t , x, x, y ) = 0.9x + 0.1x + y.

To verify Theorem 2.3 and Theorem 3.1, we will compute the errors

k = E

2 sup X (k) (t ) − X (t )

0t 10

and

2 δk = sup E Xˆ (k) (t ) − X (t ) , 0t 10

where ( X (t ))−τ t T is the analysis solution of Eq. (4.1), ( X (k) (t ))−τ t T is the analysis solution of the WR method (2.4)

with splitting functions (f1) or (f2) or (f3) and ( Xˆ (k) (t ))−τ t T is the analysis solution of the WR method (3.1) with splitting functions (f4) or (f5) or (f6). In our test these analysis solutions will be replaced by their numerical approximations. Denote (k) (k) by X h (t i ), X h (t i ) and Xˆ h (t i ) the numerical approximations of X (t i ), X (k) (t i ) and Xˆ (k) (t i ), respectively, which are generated by the Euler method with step-size h = 0.001. Here t i = ih, i = 0, 1, . . . , N, N = T /h, k = 0, 1, . . . . Then we can estimated the errors k and δk in the following way

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

237

Table 1 The errors of the WR method (2.4) with splitting functions (f1). k

k

k

k

k

k

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

7.9373e+003 7.1583e+003 6.6290e+003 1.1387e+004 3.7531e+004 1.3300e+005 4.1890e+005 1.1216e+006 2.5167e+006 4.6905e+006 7.2453e+006 9.3150e+006 1.0050e+007 9.2195e+006 7.3558e+006 5.1780e+006 3.2641e+006 1.8473e+006 9.2277e+005

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

3.9719e+005 1.4129e+005 4.1542e+004 1.0563e+004 2.6774e+003 808.9498 277.05577 88.9766 23.3085 4.6723 0.9547 0.2883 0.0876 0.0176 0.0024 6.3343e−004 2.0828e−004 3.6081e−005 3.6229e−006

39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

1.1571e−006 3.5134e−007 3.7584e−008 5.1747e−009 1.9450e−009 3.4297e−010 1.8886e−011 7.1709e−012 2.0220e−012 1.1776e−013 2.2978e−014 8.4713e−015 6.9663e−016 6.7482e−017 2.7549e−017 3.0067e−018 1.7026e−019

Table 2 The errors of the WR method (2.4) with the splitting functions (f2). k

k

k

k

k

k

1 2 3 4 5 6 7 8 9 10

6.0803e+003 2.7182e+003 734.5479 118.9471 16.5902 3.1894 0.5906 0.1170 0.0253 0.0058

11 12 13 14 15 16 17 18 19 20

0.0014 3.4518e−004 8.5170e−005 2.0024e−005 4.4622e−006 9.1540e−007 1.7126e−007 2.8831e−008 4.3614e−009 5.9505e−010

21 22 23 24 25 26 27 28 29

7.4656e−011 8.7255e−012 9.6397e−013 1.0056e−013 9.8149e−015 8.5282e−016 6.5986e−017 4.7046e−018 3.2610e−019

Table 3 The errors of the WR method (2.4) with the splitting functions (f3).

k ≈

M 1

M

j =1

δk ≈ max

l=1,..., N

k

k

k

k

k

k

1 2 3 4

723.1812 18.4135 0.1790 0.0010

5 6 7 8

8.1357e−006 5.4237e−008 2.9750e−010 1.4220e−012

9 10 11

5.9926e−015 4.1110e−017 1.4201e−019

max

l=1,..., N

(k) X (tl , ω j ) − X h (tl , ω j )2 , h

M 1 (k) Xˆ (tl , ω j ) − X h (tl , ω j )2 , h M j =1

where M denotes the number of computed paths and M = 106 . In our calculations, the stopping criterion of WR methods is k 10−18 or δk 10−18 . The computer simulation results are shown in Tables 1–6 and Figs. 1–3. The convergence results in Tables 1–3 and Fig. 1 are in agreement with the theoretical results provided in the statement of Theorem 2.3. However, the rates of convergence observed from Tables 4–6 and Fig. 3 are lower than those following from the given estimates in Theorem 3.1, especially, when the number of iteration k is large. The reason may be that the difference between the (k) numerical approximation { Xˆ h (t i ), i = 0, 1, . . . , N } and the true solution { Xˆ (k) (t ), 0 t T } are more and more large as number of iteration k increases.

238

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

Table 4 The errors of the WR method (3.1) with splitting functions (f4). k

δk

δk+1 /δk

k

δk

δk+1 /δk

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

2.5873e+003 2.5292e+003 2.6250e+003 4.6688e+003 1.7988e+004 7.1785e+004 2.4948e+005 7.1802e+005 1.6906e+006 3.2648e+006 5.2285e+006 7.0325e+006 8.0466e+006 7.9314e+006 6.8077e+006 5.2864e+006 3.6130e+006 2.1334e+006 1.0668e+006 4.4060e+005 1.4545e+005 3.6768e+004 7.3785e+003 1.9562e+003 665.7808 235.0211 74.8563

0.9775 1.0379 1.7786 3.8528 3.9907 3.4754 2.8781 2.3545 1.9312 1.6015 1.3450 1.1442 0.9857 0.8583 0.7765 0.6834 0.5905 0.5001 0.4130 0.3301 0.2528 0.2007 0.2651 0.3403 0.3530 0.3185 0.2112

28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

15.8128 3.2471 0.7468 0.2395 0.0647 0.0121 0.0018 5.5047e−004 1.6537e−004 2.5909e−005 2.7516e−006 1.0774e−006 2.8939e−007 2.7974e−008 4.1366e−009 1.8124e−009 2.8192e−010 1.3555e−011 7.0467e−012 1.8246e−012 9.4664e−014 2.1171e−014 8.1275e−015 6.0613e−016 5.8635e−017 2.7317e−017 2.7644e−018 1.4479e−019

0.2053 0.2300 0.3207 0.2702 0.1873 0.1476 0.3077 0.3004 0.1567 0.1062 0.3915 0.2686 0.0967 0.1479 0.4382 0.1555 0.0481 0.5198 0.2589 0.0519 0.2236 0.3839 0.0746 0.0967 0.4659 0.1012 0.0524

Table 5 The errors of the WR method (3.1) with splitting functions (f5). k

δk

δk+1 /δk

k

δk

δk+1 /δk

1 2 3 4 5 6 7 8 9 10 11 12 13 14

2.5728e+003 2.3635e+003 1.7152e+003 854.0249 323.0428 70.7973 24.5389 5.6281 1.6473 0.4766 0.1277 0.0310 0.0070 0.0014

0.9187 0.7257 0.4979 0.3783 0.2192 0.3466 0.2294 0.2927 0.2893 0.2680 0.2424 0.2247 0.2028 0.1842

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

2.5997e−004 4.3235e−005 6.4650e−006 8.7014e−007 1.0556e−007 1.1634e−008 1.1861e−009 1.1330e−010 1.0070e−011 8.2838e−013 6.2387e−014 4.3675e−015 2.7493e−016 1.5960e−017 9.4026e−019

0.1663 0.1495 0.1346 0.1213 0.1102 0.1019 0.0955 0.0889 0.0823 0.0753 0.0700 0.0629 0.0581 0.0589

Table 6 The errors of the WR method (3.1) with splitting functions (f6). k

δk

δk+1 /δk

k

δk

δk+1 /δk

1 2 3 4 5 6 7

2.5675e+003 2.3053e+003 1.5298e+003 705.3177 209.5295 42.5481 2.3908

0.8979 0.6636 0.4611 0.2971 0.2031 0.0562 0.0120

8 9 10 11 12 13 14 15

0.0287 1.7779e−004 1.3685e−006 5.1562e−009 1.9710e−011 7.5590e−014 2.3283e−016 6.0729e−019

0.0062 0.0077 0.0038 0.0038 0.0038 0.0031 0.0026

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

239

Fig. 1. The errors of the WR method (2.4) with the special splitting functions, left: (f1), middle: (f2), right: (f3).

Fig. 2. The errors of the WR method (3.1) with the special splitting functions, left: (f4), middle: (f5), right: (f6).

Fig. 3. The rate of convergence of the WR method (3.1) with the special splitting functions, left: (f4), middle: (f5), right: (f6).

Acknowledgements I am grateful to the anonymous referees for their very valuable comments. References [1] Z. Bartoszewski, M. Kwapisz, Convergence of waveform relaxation methods for differential-algebraic systems, SIAM J. Numer. Anal. 33 (1996) 2303– 2317. [2] Z. Bartoszewski, T. Jankowski, M. Kwapisz, On the convergence of iterative methods for general differential-algebraic systems, J. Comput. Appl. Math. 169 (2004) 393–418. [3] M.R. Crisci, N. Ferraro, E. Russo, Convergence results for continuous-time waveform methods for Volterra integral equations, J. Comput. Appl. Math. 71 (1996) 33–45. [4] T.C. Gard, Introduction to Stochastic Differential Equations, Marcel Dekker, New York, 1988. [5] R. Garrappa, An analysis of convergence for two-stage waveform relaxation methods, J. Comput. Appl. Math. 169 (2004) 377–392. [6] K.J. in’t Hout, On the convergence of waveform relaxation methods for stiff nonlinear ordinary differential equations, Appl. Numer. Math. 18 (1995) 75–190. [7] J. Jansen, S. Vandewalle, On SOR waveform relaxation methods, SIAM J. Numer. Anal. 34 (1997) 2456–2481. [8] Y.L. Jiang, A general approach to waveform relaxation solutions of differential-algebraic equations: the continuous-time and discrete-time cases, IEEE Trans. Circuits Syst. I. 51 (2004) 1770–1780. [9] P.E. Kloeden, E. Platen, The Numerical Solution of Stochastic Differential Equations, Springer, Berlin, 1992. [10] E. Lelarasmee, A.E. Ruehli, A.L. Sangivanni-Vincentelli, The waveform relaxation method for time-domain analysis of large scale integrated circuits, IEEE Trans. CAD IC Sys. 1 (1982) 131–145. [11] X.R. Mao, Stochastic Differential Equations and Their Applications, Horwood Publishing Limited, New York, 1997. [12] G.N. Milstein, Numerical Integration of Stochastic Differential Equations, Kluwer Academic Publishers Group, Dordrecht, 1995. [13] S.E.A. Mohammed, Stochastic Functional Differential Equations, Pitman Publishing Limited, London, 1984. [14] O. Nevanlinna, Remarks on Picard–Lindelöf iteration, Part I, BIT 29 (1989) 328–346. [15] N.K. Nichols, On the convergence of two-stage iterative processes for solving linear equation, SIAM J. Numer. Anal. 10 (1973) 460–469.

240

Z. Fan / Applied Numerical Mathematics 61 (2011) 229–240

[16] J. Sand, K. Burrage, A Jacobi waveform relaxation method for ODEs, SIAM J. Sci. Comput. 20 (1999) 534–552. [17] H. Schurz, K.R. Schneider, Waveform relaxation methods for stochastic differential equations, Int. J. Numer. Anal. Model. 3 (2) (2006) 232–254. [18] J. Wang, Z.Z. Bai, Convergence analysis of two-stage waveform relaxation method for the initial value problems, Appl. Math. Comput. 172 (2006) 797–808. [19] W.S. Wang, S.F. Li, Convergence of waveform relaxation methods for neutral delay differential equations, Math. Comput. Modelling 48 (2008) 1875– 1887. [20] S.L. Wu, C.M. Huang, Convergence analysis of waveform relaxation methods for neutral differential-functional systems, J. Comput. Appl. Math. 223 (2009) 263–277. [21] S. Zhou, T.Z. Huang, Convergence of waveform relaxation methods for Hermitian positive deﬁne linear systems, Appl. Math. Comput. 203 (2008) 943–952. [22] B. Zubik-Koval, S. Vandewalle, Waveform relaxation for functional-differential equations, SIAM J. Sci. Comput. 21 (1999) 207–226.

Applied Numerical Mathematics 61 (2011) 241–265

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

A Hamilton–Jacobi–Bellman approach to optimal trade execution ✩ Peter A. Forsyth David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada N2L 3G1

a r t i c l e

i n f o

Article history: Received 27 June 2009 Received in revised form 11 August 2010 Accepted 5 October 2010 Available online 14 October 2010 Keywords: Optimal execution Mean-variance tradeoff HJB equation Semi-Lagrangian discretization Viscosity solution

a b s t r a c t The optimal trade execution problem is formulated in terms of a mean-variance tradeoff, as seen at the initial time. The mean-variance problem can be embedded in a linear–quadratic (LQ) optimal stochastic control problem. A semi-Lagrangian scheme is used to solve the resulting nonlinear Hamilton–Jacobi–Bellman (HJB) PDE. This method is essentially independent of the form for the price impact functions. Provided a strong comparison property holds, we prove that the numerical scheme converges to the viscosity solution of the HJB PDE. Numerical examples are presented in terms of the eﬃcient trading frontier and the trading strategy. The numerical results indicate that in some cases there are many different trading strategies which generate almost identical eﬃcient frontiers. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

1. Introduction A large institutional investor, when selling a large block of shares, is faced with the following dilemma. If the investor trades rapidly, then the actual cash received from the sale will be less than anticipated, due to the market impact of the trades. Market impact can be minimized by breaking up a large trade into a number of smaller blocks. However, in this case, the investor is exposed to the risk of price depreciation during the trading horizon. Recently, there has been considerable interest in algorithmic trading strategies. These are automated strategies for execution of trades with the objective of meeting pre-determined optimality criteria [14,15]. In this work, we consider an idealized model for price impact. In the case of selling shares, the market price will decrease as a function of the trading rate, while at the same time following a stochastic process. The optimal control problem is then to liquidate the portfolio over some ﬁxed time, and maximize the expected cash receipts while minimizing the variance of the outcome [9,1,2,26,16,28]. An alternative approach is to pose this problem in terms of maximizing a power-law or exponential utility function [21, 32,31]. Since a different objective function is used, the optimal strategies in [21,32,31] will, of course, be different from the strategy determined from the mean variance criteria. We will focus on the mean-variance approach in this work, due to its intuitive interpretation and popularity in industry. In [1], path-independent or static strategies are suggested. The optimal strategies are those which satisfy a mean-variance optimality condition, recomputed at each trade time. However, in [28], the authors acknowledge that this strategy cannot be optimal in terms of the mean-variance tradeoff as measured at the initial time. This subtle distinction is discussed in [26,27, 8]. In [8], the strategy of maximizing the mean-variance objective at the initial time is termed the pre-commitment policy, i.e.

✩ This work was supported by the Natural Sciences and Engineering Research Council of Canada, and by a Morgan Stanley Equity Market Microstructure Research Grant. The views expressed herein are solely those of the authors, and not those of any other person or entity, including Morgan Stanley. Morgan Stanley is not responsible for any errors or omissions. Nothing in this article should be construed as a recommendation by Morgan Stanley to buy or sell any security of any kind. E-mail address: [email protected].

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.10.004

242

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

once the initial strategy (as a function of the state variables) has been determined at the initial time, the trader commits to this policy, even if the optimal mean variance policy computed at a later time differs from the pre-commitment policy. This contrasts with the time-consistent policy, whereby the trader optimizes the mean-variance tradeoff at each instant in time, assuming optimal mean-variance strategies at each later instant. The advantages and disadvantages of these two different approaches are discussed in [8]. In this paper, we focus solely on the pre-commitment strategy, which is the optimal policy in terms of mean-variance as seen at the initial time. A concrete example of the sense in which the pre-commitment strategy is optimal is the following. Suppose we are in an idealized world, where all our modelling assumptions (such as the form of the price impact functions, stochastic processes, and so on) are perfect. In this world, suppose we followed the pre-commitment strategy for many thousands of different trades. We then measure the standard deviation and expected gain (relative to the initial pre-trade state) averaged over the thousands of trades. Any other trading strategy (including the time-consistent strategy) would never produce a larger expected gain for a given standard deviation compared to the pre-commitment strategy. We formulate the optimal trading problem as an optimal stochastic control problem, where the objective is to maximize the mean-variance tradeoff as measured at the initial time. The mean variance objective function can be converted to linear– quadratic (LQ) objective function using a Lagrange multiplier method [24,10,34,4,20]. Standard dynamic programming can then be used to derive a Hamilton–Jacobi–Bellman (HJB) PDE. Note that previously this method has been used mainly as a tool for obtaining analytic solutions to multi-period mean-variance investment problems. Analytic solutions are, of course, not available for many problems. In this work, we the formulate the optimal trading problem in terms of the equivalent LQ formulation. We then use a numerical method to solve the resulting HJB equation for the optimal strategy. Our main contributions in this paper are

• We formulate the numerical problem so that a single solve of the nonlinear HJB problem, and a single solve of a related linear PDE, generates the entire eﬃcient trading frontier.

• We develop a semi-Lagrangian scheme for solution of the HJB PDE and prove that this method is monotone, consistent and stable, hence converges to the viscosity solution of the HJB equation [7,5] assuming that the HJB equation satisﬁes a strong comparison principle. • We assume geometric Brownian motion for the stochastic process of the underlying asset, and a speciﬁc form for the price impact functions. However, our numerical method is essentially independent of any particular form for the price impact functions, and can be easily generalized to other stochastic processes (e.g. jump diffusion, regime switching). The technique is also amenable to implementation on multi-processor architectures. • The trading problem is originally three-dimensional. However, in some cases, the HJB PDE can be reduced to two dimensions using a similarity reduction. Our numerical formulation can be used for either the full three-dimensional case, or for cases when the similarity reduction is valid, with minor modiﬁcation. • The numerical results indicate that there are some cases there are many different trading strategies which generate almost the same eﬃcient frontier. 2. Optimal execution Let

S = Price of the underlying risky asset,

α = Number of shares of underlying asset, B = Risk free bank account.

(2.1)

At any time t ∈ [0, T ] an investor has a portfolio Π given by

Π(t ) = B + α S .

(2.2)

In order to handle both selling and buying cases symmetrically, we start off with α I > 0 shares if selling, and α I < 0 shares if buying. In other words, our objective is to liquidate a long position if selling, and to liquidate a short position if buying. More precisely

t=0

→

B = 0,

t=T

→

B = BL,

S = S0, S = ST ,

α = αI , α = α T = 0,

α I > 0 if selling, α I < 0 if buying

(2.3)

where B L is the cash which is generated by selling/buying in [0, T ), with a ﬁnal liquidation/purchase at t = T to ensure that the correct total number of shares are sold/bought. B acts as a path dependent variable which keeps track of the total receipts obtained thus far from selling/buying the underlying asset S. Our objective will be to maximize B L and minimize the risk, as measured by the variance (or standard deviation) of B L .

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

243

2.1. Problem formulation: overview There are two popular formulations of the optimal trading problem. The impulse control formulation assumes that trades only take place at discrete points in time [21,32]. However, this approach has the conceptual diﬃculty that the price impact of two discrete trades is independent of the time interval between trades. A better model would be based on impulse control (discrete trades) but include extra lag variables which would track the time interval between trades [29,16]. However, this would be computationally expensive. As a compromise, we can assume continuous trading at an instantaneous trading rate v [28,3]. This is unrealistic in the sense that real trading only takes place discretely. However, we can make the temporary price impact a function of the trade velocity, which introduces a simpliﬁed memory effect into the model, i.e. rapid trading has a larger temporary price impact than slower trading. We will use this model in the following. 2.2. Problem formulation: details Let the trading rate v be

v=

dα dt

(2.4)

,

where α is the number of shares in the portfolio (2.2). For deﬁniteness, we will suppose that S follows geometric Brownian motion (GBM), with a modiﬁcation due to the permanent price impact of trading at rate v

dS =

η + g ( v ) S dt + σ S d Z ,

η is the drift rate of S , g ( v ) is the permanent price impact,

σ is the volatility, d Z is the increment of a Wiener process.

(2.5)

We use the following form for the permanent price impact

g(v ) = κp v ,

κ p is the permanent price impact factor. We take

(2.6)

κ p to be a constant. Suppose η = 0, σ = 0 in Eq. (2.5). If X = log S, then from Eqs. (2.5)–(2.6) we have t

X (t ) − X (0) = κ p

v (u ) du

(2.7)

0

which means that X (t ) = X (0) if a round-trip trade ( eliminates round-trip arbitrage opportunities [22,3]. The bank account B is assumed to follow

dB dt

t 0

v (u ) du = 0) is executed. This form of permanent price impact

= r B − v S f ( v ),

(2.8)

r is the risk-free return, f ( v ) is the temporary price impact and transaction cost function.

(2.9)

The term v S f ( v ) represents the rate of cash expended to purchase shares at price S f ( v ) at a rate v. The temporary price impact and transaction cost function f ( v ) is assumed to be

f ( v ) = 1 + κs sgn( v ) exp

κt sgn( v )| v |β ,

κs is the bid-ask spread parameter, κt is the temporary price impact factor, β is the price impact exponent.

(2.10)

We shall refer to f ( v ) in the following as the temporary price impact function, although strictly speaking, we also include a transaction cost term as well. For various studies which suggest the form (2.10) see [25,30,3].

244

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

Given the state variables ( S , B , α ) the instant before the end of trading t = T − , then we have one ﬁnal trade (if necessary) so that the number of shares owned at t = T is α T = 0, as in Eq. (2.3). The liquidation value after this ﬁnal trade B L = Φ L ( S , α , B , α T ) is determined from a discrete form of Eq. (2.8) i.e.

B L = Φ L ( S , B , α , α T ) = B − v T ( t ) T S f ( v T ),

(2.11)

where v T is given from

vT =

αT − α ( t )T

=

−α ( t )T

(2.12)

where we can specify that the liquidation interval is very short, e.g. ( t ) T = 10−5 years. Note that effectively the liquidation value (2.11) penalizes the trader for not hitting the target α = α T at the end of trading. The optimal strategy will attempt to avoid this state (where α = α T ), hence the results are insensitive to ( t ) T if this value is selected suﬃciently small. In the case of selling, B L will be a positive quantity obtained by selling α I shares. In the case of buying, B L will be negative, indicating a cash outﬂow to liquidate a short position of α I shares (i.e. buying |α I | shares). 2.3. The optimal strategy Let v ( S , B , α , t ) be a speciﬁed trading strategy. Let E tv=(·)0 [ B L ] be the expected gain from this strategy. Deﬁne the variance of the gain for this strategy as

2

Vartv=(·)0 [ B L ] = E tv=(·)0 ( B L )2 − E tv=(·)0 [ B L ] .

(2.13)

0 The control problem is then to determine the optimal strategy v ∗ ( S , B , α , t ) such that E tv=∗ (·) [ B L ] = d, while minimizing the risk as measured by the variance. More formally, we seek the strategy v ∗ (·) which solves the problem

min Vartv=(·)0 [ B L ] = E tv=(·)0 ( B L )2 − d2

subject to

E tv=(·)0 [ B L ] = d,

(2.14)

v (·) ∈ Z ,

where Z is the set of admissible controls. We emphasize here that the expectation and variance are as seen at t = 0. Problem (2.14) determines the best strategy given a speciﬁed E tv=(·)0 [ B L ] = d. Varying the expected value d traces out a curve in the expected value, standard deviation plane. This curve is known as an eﬃcient frontier. Each point on the curve represents a trading strategy which is optimal in the sense that there is no other strategy which gives rise to a smaller risk for the given expected value of the trading gain. Consequently, any rational trader will only choose strategies which correspond to points on the eﬃcient frontier. Different traders will, however, choose different points on the eﬃcient frontier, which will depend on their risk preferences. 2.4. Objective function: eﬃcient frontier Problem (2.14) is a convex optimization problem, and hence has a unique solution. We can eliminate the constraint in problem (2.14) by using a Lagrange multiplier [24,10,34,4,20], which we denote by γ . Problem (2.14) can then be posed as [11]

max min E tv=(·)0 ( B L )2 − d2 − γ E tv=(·)0 [ B L ] − d . γ v (·)∈ Z

γ , d, this is equivalent to ﬁnding the control v (·) which solves

γ 2 . min E tv=(·)0 B L −

(2.15)

For ﬁxed

v (·)∈ Z

2

(2.16)

Note that if for some ﬁxed γ , v ∗ (·) is the optimal control of problem (2.16), then v ∗ (·) is also the optimal control of problem (2.14) with d = E tv=∗ 0 [ B L ] [24,10], where the notation E tv=∗ 0 [·] refers to the expected value given the strategy v ∗ (·). Conversely, if there exists a solution to problem (2.14), with E tv=∗ 0 [ B L ] = d, then there exists a γ which solves problem (2.16) with control v ∗ (·). We can now restrict attention to solving problem (2.16). For a given γ , ﬁnding the control v ∗ (·) which minimizes equation (2.16) gives us a single pair ( E v ∗ [ B L ], Var v ∗ [ B L ]) on the variance minimizing eﬃcient frontier. Varying γ allows us to trace out the entire frontier. Remark 2.1 (Eﬃcient frontier). The eﬃcient frontier, as normally deﬁned, is a portion of the variance minimizing frontier √ [10]. That is, given a point ( E v ∗ [ B L ], Var v ∗ [ B L ] ) on the eﬃcient frontier, corresponding to control v ∗ (·), then there exists no other control v¯ ∗ (·) such that Var v¯ ∗ [ B L ] = Var v ∗ [ B L ] with E v¯ ∗ [ B L ] > E v ∗ [ B L ]. Hence the points on the eﬃcient frontier are Pareto optimal [35]. From a computational perspective, once a set of points on the variance minimizing frontier are determined, then the eﬃcient frontier can be constructed by a simple sorting operation.

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

245

We will assume that the set of admissible controls is given by

Z ∈ [ v min , v max ],

v min 0 v max .

(2.17)

If only selling is permitted, then, for example,

v max = 0.

v min < 0,

(2.18)

v min , v max are assumed to be bounded in the following. Bearing in mind that we are going to solve problem (2.16) by solving the corresponding Hamilton–Jacobi–Bellman control PDE, we would like to avoid having to do many PDE solves. Deﬁne (assuming γ = const.)

B(t ) = B (t ) −

γ e−r (T −t ) 2

(2.19)

.

Then let

γ γ BL = ΦL S , B t = T − , α , αT = ΦL S , B t = T − , α , αT − = B L − , 2

2

(2.20)

so that problem (2.16) becomes, in terms of B L = B L − γ /2

min E t =0 B 2L .

(2.21)

v (·)∈ Z

Note (from Eqs. (2.8), (2.19)) that

dB dt

= rB − v S f (v )

(2.22)

which has the same form as Eq. (2.8). However, we now have the γ dependence appearing at t = 0. Recall from Eq. (2.3) that B (t = 0) = 0, then

t=0

→

B=

−γ e −rT 2

,

S = S0,

α = αI .

(2.23)

This is very convenient, in the PDE context. We simply determine the numerical solution for problem (2.21), which is independent of γ . We can then determine the solution for different discrete values of γ by examining the solution for different discrete values of B (t = 0). Since we normally solve the PDE for a range of discrete values of B , we can solve problem (2.21) once, and use this result to construct the entire variance minimizing eﬃcient frontier. 3. HJB formulation: overview 3.1. Determination of optimal control Let V = V ( S , B , α , τ = T − t ) = E t [B 2L ] and denote

LV ≡

σ 2 S2 2

V SS + ηS V S.

(3.1)

Assuming process (2.5), and Eqs. (2.4), (2.22), then following standard arguments [17], the solution to problem (2.21) is given from the solution to

V τ = L V + r B V B + min − v S f ( v ) V B + v V α + g ( v ) S V S , v∈ Z

with the initial condition (at

Z = [ v min , v max ]

(3.2)

τ = 0 or t = T )

V ( S , B , α , τ = 0) = B 2L ,

(3.3)

where B L is given from Eq. (2.20). Solution of this problem determines an optimal control v ∗ ( S , B , α , τ ) at each point ( S , B , α , τ ). We can use Eq. (2.19) to determine the control in terms of the variables ( S , B , α , τ ).

246

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

3.2. Determination of expected value We need to determine E tv=∗ 0 [B L ] in order to determine the pair ( E tv=∗ 0 [ B L ], E tv=∗ 0 [ B 2L ]) which generates a point on the variance minimizing eﬃcient frontier for a given γ . Let U = U ( S , B , α , τ = T − t ) = E tv ∗ [B L ]. The operator LU is deﬁned as in Eq. (3.1). Let v ∗ ( S , B , α , τ ) be the optimal control from problem (3.2). Once again, assuming process (2.5), then U satisﬁes

U τ = LU + r B U B − v ∗ S f v ∗ U B + v ∗ U α + g v ∗ S U S

(3.4)

with the initial condition

U ( S , B , α , τ = 0) = B L

(3.5)

where B L is given from Eq. (2.20). Since the most costly part of the solution of Eq. (3.2) is the determination of the optimal control v ∗ , solution of Eq. (3.4) is very inexpensive, since v ∗ is known. 3.3. Construction of the eﬃcient frontier Once we have solved problems (3.2) and (3.4) we can now construct the eﬃcient frontier. We examine the solution values at τ = T (t = 0) for the initial values of ( S , α ) of interest. Deﬁne

V 0 (B ) = V ( S = S 0 , B , α = α I , τ = T ), U 0 (B ) = U ( S = S 0 , B , α = α I , τ = T ).

(3.6)

Note that

V 0 (B ) = E tv=∗ 0 B 2L , U 0 (B ) = E tv=∗ 0 [B L ].

(3.7)

From Eq. (2.23), a value of B at t = 0 or

τ = T corresponds to the value of γ given by

γ = −2erT B.

(3.8)

Note that E tv=∗ 0 [ y (B )] for known v ∗ is given from the solution to linear PDE (3.4), with initial condition y (B ), so that E tv=∗ 0 [const.] = const. Recall B L = B L − γ /2, so that from Eqs. (3.7) we have

V 0 (B ) = E tv=∗ 0 B 2L − γ E tv=∗ 0 [ B L ] + U 0 (B ) = E tv=∗ 0 [ B L ] −

γ 2

γ2

,

4

, (3.9)

with γ = γ (B ) from Eq. (3.8). Consequently, for given B , γ is given from Eq. (3.8), then E tv=∗ 0 [ B 2L ] and E tv=∗ 0 [ B L ] are obtained from Eqs. (3.9). By examining the solution for different values of B , we trace out the entire variance minimizing eﬃcient frontier. Remark 3.1 (Generation of the eﬃcient points). As discussed in Remark 2.1, the points on the eﬃcient frontier are, in general, a subset of the points on the variance minimizing frontier. Given a set of points on the variance minimizing frontier, the points are sorted in order of increasing expected value. Then these points are traversed in order from the highest expected value to the lowest expected value. Any points which have a higher variance compared to a previously examined point are rejected. 3.4. Similarity reduction For price impact functions of the form (2.6) and (2.10), payoffs (3.3) and (3.5), and assuming geometric Brownian motion (2.5) then

V (ξ S , ξ B , α , τ ) = ξ 2 V ( S , B , α , τ ), U (ξ S , ξ B , α , τ ) = ξ U ( S , B , α , τ ).

(3.10)

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

247

Consequently,

B∗ S ∗ , B , α, τ , B

∗ B B S ∗ U ( S , B, α , τ ) = U , B , α, τ B∗ B

V ( S , B, α , τ ) =

B B∗

2

V

(3.11) (3.12)

and hence we need only solve for two ﬁxed values of B ∗ (one positive and one negative) and we can reduce the numerical computation to (essentially) a two-dimensional problem (see Section 5.1). 4. HJB formulation: details Consequently, the problem of determining the eﬃcient frontier reduces to solving Eqs. (3.2) and (3.4). 4.1. Determination of the optimal control Eq. (3.2) is

V τ = L V + r B V B + min − v S f ( v ) V B + v V α + g ( v ) S V S . v∈ Z

(4.1)

The domain of Eq. (4.1) is

( S , B, α , τ ) ∈ [0, ∞] × [−∞, +∞] × [αmin , αmax ] × [0, T ],

(4.2)

where, for example αmin = min(0, α I ), αmax = max(α I , 0) if we only allow monotonic buying/selling. We also typically normalize quantities so that |α I | = 1. For numerical purposes, we localize the domain (4.2) to

At

( S , B, α , τ ) ∈ [0, S max ] × [ B min , B max ] × [αmin , αmax ] × [0, T ].

(4.3)

α = αmin , αmax , we do not allow buying/selling which would cause α ∈/ [αmin , αmax ], so that V τ = L V + r B V B + min − v S f ( v ) V B + v V α + g ( v ) S V S , α = αmax ; Z − = [ v min , 0], v∈ Z − V τ = L V + r B V B + min − v S f ( v ) V B + v V α + g ( v ) S V S , α = αmin ; Z + = [0, v max ].

(4.4)

v∈ Z +

(4.5)

At B = Bmin , Bmax , we can assume that Eq. (3.11) holds. In which case, we can replace V B in Eq. (4.1) by

VB =

2

B

V −

S

B

V S;

B = Bmin , Bmax .

(4.6)

In general, this would be an approximation. However, in our case, Eq. (3.11) holds exactly. In fact, we will not need to consider boundary conditions at Bmin , Bmax since we will use Eq. (3.11) to effectively eliminate the B variable. We include Eq. (4.6) for generality. The initial condition is

V ( S , B , α , 0) = (B L )2 .

(4.7)

At S = 0, no boundary condition is required for Eq. (4.1), we simply solve Eq. (4.1) with L V = 0. At S → ∞, consider the cases of buying and selling separately. In the case of selling, we would normally have 0 α α I , so that α f ( v ) → 0 if ( t )T → 0 in Eq. (2.11). Hence B L B which is independent of S. For τ > 0, the optimal strategy for S large will attempt to ﬁnd the solution which minimizes B 2 , so the value will also be independent of S as S → ∞. In the case of buying (S → ∞)

2 B2L α 2 S f ( v T ) .

(4.8)

In this case, the payoff condition essentially penalizes the trader for not meeting the target value of α T = 0 the instant before trading ends when S is large. The optimal strategy would therefore be to make sure α 0 at t → T . Hence the optimal control at τ > 0 when S → ∞ should tend to force α = 0. In other words, from Eqs. (2.11), (4.8), V ( S max , B , α , τ > 0) V ( S max , B , α T , τ ) B 2 , which is independent of S. Hence, in both cases, we make the ansatz that

V S S , V S → 0;

S = S max ,

so that Eq. (4.1) becomes

(4.9)

V τ = r B V B + min − v S f ( v ) V B + v V α ; v∈ Z

S = S max .

(4.10)

Eq. (4.10) is clearly an approximation, but has the advantage that it is very easy to implement. We shall carry out various numerical tests with different values of S max to show that the error in this approximation can be made small in regions of interest.

248

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

4.2. Determination of the expected value Given the optimal trading strategy v ∗ = v ∗ ( S , B , α , τ ) determined from Eq. (4.1), the expected value U = E tv=∗ 0 [B L ] is given from Eq. (3.4)

U τ = LU + r B U B − v ∗ S f v ∗ V B + v ∗ V α + g v ∗ S V S .

(4.11)

At S = 0 we simply solve Eq. (4.11). From Eq. (4.4), at α = αmax , we must have v ∗ ( S , B , αmax , τ ) 0 hence no boundary condition is required at α = αmax . Similarly, at α = αmin , v ∗ ( S , B , αmin , τ ) 0, and no boundary condition is required at α = αmin . The boundary conditions at B = Bmin , Bmax can be eliminated using Eq. (3.12)

UB =

1

B

U−

S

B

US;

B = Bmin , Bmax .

(4.12)

However, in this paper, the similarity reduction (3.12) is exact, hence we can eliminate the B variable, and thus no boundary condition at {Bmin , Bmax } is required. Following similar arguments as used in deriving Eq. (4.10), we assume U S , U S S → 0 as S → S max , hence Eq. (4.11) becomes

U τ = rBU B − v ∗ S f v ∗ V B + v ∗ V α ;

S = S max .

(4.13)

The payoff condition is

U ( S , B , α , 0) = B L .

(4.14)

5. Discretization: an informal approach We ﬁrst provide an informal discretization of Eq. (4.1) using a semi-Lagrangian approach. We prove that this is a consistent discretization in Section A.3. Eq. (4.11) is discretized in a similar fashion. The reader is referred to the references in [12] for more details concerning semi-Lagrangian methods for HJB equations. Along the trajectory S = S (τ ), B = B (τ ), α = α (τ ) deﬁned by

dS dτ dB dτ dα dτ

= − g(v ) S , = − rB − v S f (v ) , = −v ,

(5.1)

Eq. (4.1) can be written as

max v∈ Z

DV Dτ

= LV ,

(5.2)

where the Lagrangian derivative D V / D τ is given by

DV Dτ

= V τ − V S g(v ) S − V B rB − v S f (v ) − V α v .

(5.3)

The Lagrangian derivative is the rate of change of V along the trajectory (5.1). Deﬁne a set of nodes [ S 0 , S 1 , . . . , S i max ], [B0 , B1 , . . . , B j max ], [α0 , α1 , . . . , αkmax ], and discrete times τ n = n τ . Let V ( S i , B j , αk , τ n ) denote the exact solution to Eq. (4.1) at point ( S i , B j , αk , τ n ). Let V in, j ,k denote the discrete approxima-

tion to the exact solution V ( S i , B j , αk , τ n ). We use standard ﬁnite difference methods [13] to discretize the operator L V as given in (3.1). Let (Lh V )ni, j ,k denote the

discrete value of the differential operator (3.1) at node ( S i , B j , αk , τ n ). The operator (3.1) can be discretized using central, forward, or backward differencing in the S direction to give

(Lh V )ni, j ,k = ai V in−1, j ,k + b i V in+1, j ,k − (ai + b i ) V in, j ,k ,

i < i max ,

(5.4)

where ai and b i are determined using an algorithm in [13]. The algorithm guarantees ai and b i satisfy the following positive coeﬃcient condition:

a i 0;

b i 0,

i = 0, . . . , i max .

(5.5)

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

249

The boundary conditions will be taken into account by setting

a0 = ai max = 0, b0 = b i max = 0.

(5.6)

(Lh V nj,k )i

Deﬁne the vector = then Lh is an (i max + 1) × (i max + 1) matrix such that is given by Eq. (5.4). Let v ni, j ,k denote the approximate value of the control variable v at mesh node ( S i , B j , αk , τ n ). Then we approximate

[ V 0n, j ,k , . . . , V inmax , j ,k ]t ,

V nj,k

D V / D τ at ( S i , B j , αk , τ n+1 ) by the following

DV Dτ

n+1

i , j ,k

1

τ

V in,+j ,1k − V ˆn ˆ ˆ i , j ,k

(5.7)

where V ˆn ˆ ˆ is an approximation of V ( S nˆ , Bnˆ , αnˆ , τ n ) obtained by linear interpolation of the discrete values V in, j ,k , with i j i , j ,k k +1 ( S nˆ , Bnˆ , αnˆ ) given by solving Eqs. (5.1) backwards in time, from τ n+1 to τ n , for ﬁxed v ni ,+j,1k to give (noting that g ( v ni max , j ,k ) = i

j

k

0 from Eq. (4.10))

S nˆ = S i exp g v ni ,+j ,1k τ ; i = S i ; i = i i max ,

Bnˆ j

= B j exp[r τ ] −

i < i i max ,

v ni ,+j ,1k S i

f

v ni ,+j ,1k

n +1

er τ − e g ( v i, j,k ) τ , r − g ( v ni ,+j ,1k )

αknˆ = αk + v ni,+j,1k τ .

(5.8)

Eq. (5.8) is equivalent to O (( τ )2 ) to

S nˆ = S i + S i g v ni ,+j ,1k τ + O ( τ )2 ;

i Bnˆ j

= B j + rB j −

v ni ,+j ,1k S i

f

v ni ,+j ,1k

i < i max ,

τ + O ( τ )2 ,

αknˆ = αk + v ni,+j,1k τ .

(5.9)

For numerical purposes, we use Eq. (5.8) since this form ensures, for example, that

S nˆ i

0, regardless of timestep size. We

will use the limiting form (5.9) when carrying out our consistency analysis. All the information about the price impact function is embedded in Eq. (5.8). This means that the form of the price impact functions can be easily altered, with minimal changes to an implementation. Let Z in,+j ,1k ⊆ Z denote the set of possible values for v ni ,+j ,1k such that ( S nˆ , Bnˆ , αnˆ ) remains inside the computational domain. i

In other words, v ni ,+j ,1k ∈ Z in,+j ,1k ensures that

j

k

0 S nˆ S i max , i

α0 αknˆ αkmax .

(5.10)

Note that we do not impose any constraints to ensure

Bnˆ j

∈ [Bmin , Bmax ]. We will essentially eliminate the B variable using

the similarity reduction (3.12). We approximate the HJB PDE (4.1) and the boundary conditions (4.4)–(4.5), and (4.10) by

V in,+j ,1k = At

min

v ni ,+j ,1k ∈ Z in,+j ,1k

V ˆn ˆ ˆ + τ (Lh V )ni ,+j ,1k , i , j ,k

τ 0 = 0 we have the payoff condition (3.3) 2 V i0, j ,k = (B L )i , j ,k .

n+1

v ∗ i , j ,k ∈ arg min V ˆn ˆ ˆ . i , j ,k n +1 n +1

(5.11)

v i , j ,k ∈ Z i , j ,k

(5.12)

Once the optimal control ( v ∗ )ni ,+j ,1k = v ∗ ( S i , B j , αk , τ n+1 ) is determined from the solution to Eq. (5.11), then the solution to Eq. (4.13) is given by solving the linear PDE

U in,+j ,1k = U ˆn ˆ ˆ i , j ,k

v =( v ∗ )ni ,+j ,1k

+ τ (Lh U )ni ,+j ,1k ,

(5.13)

with payoff condition

U i0, j ,k = (B L )i , j ,k .

(5.14)

250

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

5.1. Discrete similarity reduction If the similarity reduction (3.12) is valid (which is the case for the price impact functions, payoff and price process assumed in this work), we can reduce the number of nodes needed in the B direction to a ﬁnite number, independent of the mesh size. Choose B ∗ > 0, let B j ∈ Bset = {−B ∗ , +B ∗ }, i.e. we have only two nodes in the discrete B grid. Further, let B0 = −B ∗ , B1 = +B ∗ . If Bnˆ > 0 then we evaluate V ˆn ˆ ˆ , U ˆn ˆ ˆ by j

V ˆn ˆ ˆ i , j ,k U ˆn ˆ ˆ i , j ,k S ˆi ∗ =

= =

i , j ,k

B n 2 ˆj

V ˆn∗

i ,1,kˆ

B∗

Bn ˆj

U ˆn∗

i ,1,kˆ

B∗

i , j ,k

,

,

B∗ S ˆi

(5.15)

Bnˆ j

where V ˆn∗ ˆ refers to a linear interpolant of V n at the node ( S ˆi ∗ , B ∗ , αkˆ ). i ,1,k If Bnˆ < 0 then we evaluate V ˆn ˆ ˆ by j

i , j ,k

V ˆn ˆ ˆ = i , j ,k U ˆn ˆ ˆ i , j ,k S ˆi ∗ =

=

Bnˆ j

−B∗

2

Bn ˆ j

−B∗

−B∗ S ˆi Bnˆ

V ˆn∗

i ,0,kˆ

U ˆn∗

i ,0,kˆ

,

, (5.16)

.

j

Note that use of the similarity reduction as in Eqs. (5.15)–(5.16) eliminates the need for applying a boundary condition at Bmin , Bmax . We can exclude the case Bnˆ = 0 since (from Eq. (5.9)) j

n ∗ B = B 1 + O ( τ ) . ˆ

(5.17)

j

Remark 5.1 (Reduction to a two-dimensional problem). We can proceed more formally to eliminate the variable B . If the similarity reduction (3.12) is valid, then we can deﬁne a function χ ( z, α , τ ) such that

V ( S , B , α , τ ) = B 2 χ ( S /B , α , τ ) = B 2 χ ( z, α , τ ),

zmin z zmax ; z =

S

B

.

(5.18)

Substituting Eq. (5.18) into Eq. (3.2) with payoff (3.3) gives an HJB equation for χ ( z, α , τ ). However, we will not follow this approach here. From an implementation point of view, application of the similarity reduction is simply a special (trivial) case of a full three-dimensional implementation. There is no need for a separate implementation to handle the cases where the similarity reduction is valid/invalid. In addition, it is convenient to deal with the physical variables ( S , B , α ), when dealing with boundary conditions, price impact functions and so on. Finally, our convergence proofs are given for the case of the similarity reduction. However, since we use the variables ( S , B , α , τ ), these proofs can be easily extended to the case where the similarity reduction is not valid. The one complicating factor resulting from not carrying out the formal reduction to a two-dimensional problem concerns the appropriate set of test functions to use in deﬁning consistency in the viscosity solution sense. Since the problem is inherently two-dimensional, this means that the test functions should be smooth, differentiable functions ψ( z, α , τ ). We cannot use arbitrary three-dimensional test functions φ( S , B , α , τ ), but in view of Eq. (5.18) (which we use to deﬁne the interpolation operators (5.15)–(5.16)) we should use test functions of the form

φ( S , B, α , τ ) = B2 ψ( S /B, α , τ ).

(5.19)

Let x = ( S , B , α , τ ), then we can write Eq. (5.19) as

φ = φ(x) = φ x, ψ(x) = φ x, ψ( S /B, α , τ ) .

(5.20)

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

251

Table 7.1 Parameters for optimal execution example, short trading horizon. Parameter

Value

σ

1.0 1/250 years 0.0 0.0 100 1.0 0.0 2 × 10−6 0.0 1.0 Sell −1000/T 0.0 20 000 10−6 years

T

η r S0

αI κp κt κs β Action v min v max S max ( t ) T (2.12)

5.2. Solution of the local optimization problem Recall Eq. (5.11)

V in,+j ,1k =

min

v ni ,+j ,1k ∈ Z in,+j ,1k

V ˆn ˆ ˆ + τ (Lh V )ni ,+j ,1k .

(5.21)

i , j ,k

An obvious way to solve the local optimization problem is to use a standard one-dimensional algorithm. However, we found this to be unreliable, since the local objective function has multiple local minima (this will be discussed in more detail later). Instead, we discretize the range of controls. For example, consider the set of controls Z = [ v min , v max ] for a point in the interior of the computational domain. Let Zˆ = { v 0 , v 1 , . . . , v k } with v 0 = v min , v k = v max and maxi v i +1 − v i = O (h). Then, if φ is a smooth test function and f ( v ), g ( v ) are continuous functions (which we assume to be the case) then

φτ − Lφ − r BφB − min − v S f ( v )φB + v φα + g ( v ) S φ S ˆ v∈ Z − φτ − Lφ − r BφB − min − v S f ( v )φB + v φα + g ( v ) S φ S → 0; v∈ Z

as h → 0.

(5.22)

Consequently, replacing Z by Zˆ is a consistent approximation [33]. Our actual numerical algorithm uses Z in,+j ,1k ⊆ Zˆ , and the minimum in Eq. (5.21) is found by linear search. Note that this approximation would be O (h) if f ( v ), g ( v ) are Lipshitz continuous. 6. Convergence to the viscosity solution Provided a strong comparison result for the PDE applies, [7,5] demonstrate that a numerical scheme will converge to the viscosity solution of the equation if it is l∞ stable, monotone, and pointwise consistent. In Appendix A, we prove the convergence of our numerical scheme (5.11) to the viscosity solution of problem (4.1) associated with boundary conditions (4.4)–(4.5), (4.10) by verifying these three properties. The deﬁnition of consistency in the viscosity solution sense [5] appears to be somewhat complex. However, as can be seen in Appendix A, this deﬁnition is particularly useful in the context of a semi-Lagrangian discretization, since there are nodes in strips near the boundaries where the discretization is not consistent in the classical sense for arbitrary mesh/timestep sizes. 7. Optimal liquidation example: short trading horizon We use the parameters shown in Table 7.1, for an example where the entire stock position is to be liquidated in one day. Eqs. (3.2) and (3.4) are solved numerically using a semi-Lagrangian method described in Section 5. A similarity reduction is used to reduce the problem to a two-dimensional S × α grid, with two nodes (for all mesh/timestep sizes) in the B direction, as described in Section 5.1. Table 7.2 shows the number of nodes and timesteps used in the convergence study. Table 7.3 shows the value of E tv=∗ 0 [B 2L ] at t = 0, S = 100, α = 1, B = −100 for several levels of reﬁnement. Convergence appears to be at a ﬁrst order rate. Increasing the size of S max resulted in no change to the solution to eight digits. The eﬃcient frontier is shown in Fig. 7.1. This ﬁgure shows the expected average amount obtained per share versus the standard deviation. The pre-trade share price is $100. The results in Fig. 7.1 were obtained using the similarity reduction.

252

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

Table 7.2 Grid and timestep data for convergence studies. If a similarity reduction is used, then the B grid has only two nodes for any reﬁnement level. Timesteps

S nodes

α nodes

B nodes

v nodes

Reﬁnement level

25 50 100 200 400

98 195 389 777 1553

41 81 161 321 641

77 153 305 609 1217

30 59 117 233 465

0 1 2 3 4

Table 7.3 Value of E tv=∗ 0 [B2L ] at t = 0, S = 100, α = 1, B = −100. Data in Table 7.1. Discretization data is given in Table 7.2. Reﬁnement level

Value

0 1 2 3 4

1.668460 1.319408 1.176402 1.094543 1.054693

Fig. 7.1. The eﬃcient frontier for optimal execution (sell case), using the data in Table 7.1. The vertical axis represents the expected average share price obtained. Initial stock price S 0 = 100. Discretization details given in Table 7.2. Similarity reduction used.

For comparative purposes, we also show the eﬃcient frontier in Fig. 7.2, obtained using the full three-dimensional PDE (no similarity reduction). Due to memory requirements, we can only show three levels of reﬁnement. Note that the full three-dimensional PDE uses a discretization in the B direction. Recall that the use of a similarity reduction (as described in Section 3.4) effectively means that there is no discretization error in the B direction. Hence we can expect that the full three-dimensional PDE solve will show larger discretization errors, compared to the solution obtained using the similarity reduction, for the same reﬁnement level. As shown in Fig. 7.2, the full three-dimensional solution is converging to the same eﬃcient frontier as the similarity reduction solution, but more slowly and at much greater computational cost. Fig. 7.3 shows E tv=∗ 0 [B 2L ], B = −100. This value of B = −100 corresponds to γ = 200. Assuming we are at the initial point ( S = 100, B = 0, α = 1), this value of γ corresponds to the point

Expected Gain = 99.295, Standard Deviation = 0.7469

(7.1)

on the curve shown in Fig. 7.1. 7.1. Optimal strategy: uniqueness From Fig. 7.3 we can see that there is a large region for S > 100 where

V α 0;

V S 0;

V 0

(7.2)

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

253

Fig. 7.2. The eﬃcient frontier for optimal execution (sell case), using the data in Table 7.1. The vertical axis represents the expected average share price obtained. Initial stock price S 0 = 100. Discretization details given in Table 7.2. Results are obtained by solving the full three-dimensional PDE. The curve labelled “Sim Red” was computed using the similarity reduction method (as in Fig. 7.1).

Fig. 7.3. The value surface E tv=∗ 0 [B2L ], B = −100, t = 0. Data in Table 7.1.

which then implies, using Eq. (4.6), that V B 0. Hence, in the ﬂat region in Fig. 7.3, V α 0, V S 0, and V B 0. Recall Eq. (3.2)

V τ = LV + rB V B +

min

v ∈[ v min , v max ]

−v S f (v )V B + v V α + g(v ) S V S .

(7.3)

If V S = V B = V α = 0, then the optimal control can be any value v ∈ [ v min , v max ]. Clearly there are large regions where the optimal strategy is not unique. As an extreme example, one way to achieve minimal risk is to immediately sell all stock at an inﬁnite rate, which results in zero expected gain, and zero standard deviation. However, this strategy is not unique. Another possibility is to do nothing until t = T − , and then to sell at an inﬁnite rate. This will also result in zero gain and zero standard deviation. There are inﬁnitely many strategies which produce the identical result. Hence, in general, the optimal strategy is not unique, but the value function is unique.

254

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

Fig. 7.4. Optimal trading rate at t = 0.0, B = 0, α = 1, as a function of S. This is the optimal strategy for the point on the eﬃcient frontier given by Eq. (7.1). Note that the constant trading rate which meets the liquidation objective is v = −250. Data in Table 7.1. Discretization details given in Table 7.2.

7.2. Optimal trading strategy Fig. 7.4 shows the optimal trading rate at t = 0.0, B = −100, α = 1, as a function of S. This is the optimal strategy for the point on the eﬃcient frontier given by Eq. (7.1). We can interpret this curve as follows. Given the initial data (S = 100, α = 1, B = 0, t = 0), this curve shows the optimal trading rate if the asset price suddenly changes to the value of S shown. Note that this particular strategy is the rate which minimizes (2.16) for the value of γ which results in (7.1). To put Fig. 7.4 in perspective, the constant trading rate which meets the liquidation objective is v = −1/ T = −250. The optimal trading rate behaves roughly as expected [28]. As the asset price increases, the trading rate should also increase. In other words, some of the unexpected gain in stock price can be spent to reduce the standard deviation. Recall that the strategy maximizes (2.16) as seen at the initial time. However, note the sawtooth pattern in the optimal trading rate for S > 75. This does not appear to be an artifact of the discretization, since this pattern seems to persist for small mesh sizes. It is perhaps not immediately obvious how a smooth value function as given in Fig. 7.3 can produce the non-smooth trading strategy shown in Fig. 7.4. Recall that a local optimization problem (5.21) is solved at each node to determine the optimal trade rate. A careful analysis of the objective function at the points corresponding to the sawtooth pattern in Fig. 7.4 revealed that the value function was very ﬂat, with multiple local minima. Although the value function is a smooth function of S, the optimal trade amount (v t) is not a smooth function of S. This suggests that the optimal value is not very sensitive to the control at these points. 7.3. Discrete trade rates In order to explore the effect of the sawtooth pattern on the optimal trade rates, the optimal strategy was recomputed using a ﬁxed number of discrete trading rates. The rates were (in units of 1/ T )

Trade rates = {−1000, −500, −100, −50, −40, −30, −25, −20, −15, −10, −9, −8, −7, −6, −5, −4.5, −4,

−3.5, −3, −2.5, −2, −1.5, −1.25, −1.0, −0.75, −0.5, −0.25, 0}.

(7.4)

These discrete trade rates were ﬁxed, and not changed for ﬁner grids. Recall that for the continuous case, the spacing of the discrete trading rates was divided by two on each grid reﬁnement. On the ﬁnest grid (1553 × 641) the interval [− v min , v max ] was discretized using 465 nodes. Note that there are only 27 discrete trading rates in the set of nodes in Eq. (7.4). The eﬃcient frontier using both these possible sets of trading rates is shown in Fig. 7.5 (left plot). The two curves are almost indistinguishable. This has an interesting practical beneﬁt. If h is the mesh/timestep size parameter (see Eq. (A.1)), then the method developed here has complexity O (1/h4 ). One might expect a complexity of O (1/h3 ) but the need to solve the local optimization problem using a linear search generates the extra power of 1/h. However, from Fig. 7.5, it would appear that we can determine the eﬃcient frontier to a practical level of accuracy using a mesh independent set of trading rates, which would lower the complexity to O (1/h3 ).

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

255

Fig. 7.5. Left plot: the eﬃcient frontier for optimal execution (sell case), using the data in Table 7.1. The vertical axis represents the expected average share price obtained. Initial stock price S 0 = 100. The curves are computed with reﬁnement level 4 (see Table 7.2). The two curves are computed using the set of trade rates in Eq. (7.4) (Discrete Trade Rate), and the approximation to continuous trading rates obtained by discretizing [ v min , v max ] with 465 nodes (Continuous Trade Rate). Right plot: the optimal trading rates corresponding to the eﬃcient frontiers in the left plot.

Fig. 7.5 (right plot) also shows the optimal trading rates corresponding to the eﬃcient frontiers shown in Fig. 7.5 (left plot). It would appear that there are many strategies which generate very similar eﬃcient frontiers. It is likely that the sawtooth pattern in Fig. 7.4 is due to the ill-posed nature of the optimal strategy. 8. Liquidation example: long trading horizon Table 8.1 shows the data used for a second example. Note that β in Eq. (2.10) is set to β = 0.5. Similar values of β have been reported in [25]. Fig. 8.1 shows the eﬃcient frontier. Fig. 8.2 shows the optimal trading rate at t = 0.0, B = −100, α = 1, as a function of S. The trade rates are given for a point on the eﬃcient frontier corresponding to (γ = 200.83)

Expected Gain = 95.6, Standard Deviation = 3.47.

(8.1)

Once again, we see that the eﬃcient frontier is smooth, but that the optimal trading rates show the same sawtooth pattern as observed in Fig. 7.4. This indicates that the optimal trading rates are somewhat ill posed.

Table 8.1 Parameters for optimal execution example, long trading horizon. Parameter

Value

σ

0.40 1/12 years 0.10 0.05 100 1.0 0.01 0.069 0.01 0.5 Sell −25/T 0.0 20 000 10−9 years

T

η r S0

αsell κp κt κs β Action v min v max S max ( t ) T (2.12)

256

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

Fig. 8.1. The eﬃcient frontier for optimal execution (sell case), using the data in Table 8.1. The vertical axis represents the expected average share price obtained. Initial stock price S 0 = 100. Discretization details given in Table 7.2.

Fig. 8.2. Optimal trading rate at t = 0.0, B = 0, α = 1, as a function of S. This is the optimal strategy for the point on the eﬃcient frontier given by Eq. (8.1). Note that the constant trading rate which meets the liquidation objective is v = −12. Data in Table 8.1. Discretization details given in Table 7.2.

9. Conclusion We have formulated the problem of determining the eﬃcient frontier (and corresponding optimal strategy) in terms of an equivalent LQ problem. We need only solve a single nonlinear HJB equation (and an associated linear PDE) to construct the entire eﬃcient frontier. The HJB equation is discretized using a semi-Lagrangian approach. Assuming that the HJB equation satisﬁes a strong comparison property, then we have proven convergence to the viscosity solution by showing that the scheme is monotone, consistent and stable. Note that in this case, it is useful to use consistency in the viscosity solution sense [7,5] since the semi-Lagrangian method is not classically consistent (for arbitrary grid sizes) at points near the boundaries of the computational domain. The semi-Lagrangian discretization separates the model of the underlying stochastic process from the model of price impact. Changing the particular model of price impact amounts to changing a single function in the implementation. The semi-Lagrangian method is also highly amenable to parallel implementation. The eﬃcient frontiers computed using the method developed in this work are consistent with intuition. However, the optimal trading rates, as a function of the asset price at the initial time, show an unexpected sawtooth pattern for large asset prices. A detailed analysis of the numerical results shows that there are many strategies which give virtually the same value

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

257

function. Hence, the numerical problem for the optimal strategy (as opposed to the eﬃcient frontier) appears to be ill-posed. Note that this ill-posedness seems to be a particular property of the pre-commitment mean-variance objective function, and is not seen if alternative objective functions are used, such as a utility function [31] or mean-quadratic variation [19]. However, this ill-posedness in terms of the strategy is not particularly disturbing in practice. The end result is that there are many strategies which give essentially the same eﬃcient frontier, which is the measure of practical importance. This also indicates that it is possible to vary the trading rates in an unpredictable pattern, which may be useful to avoid signalling trading strategies, yet still achieve a mean variance eﬃcient result. Appendix A. Convergence to the viscosity solution of (4.1) In this appendix, we will verify that the discrete scheme (5.11) is consistent, stable and monotone, which ensures convergence to the viscosity solution of (4.1) associated with boundary conditions (4.4)–(4.5), (4.10). We will assume that the similarity reduction equations (5.15) and (5.16) are used in the following analysis. A.1. Some preliminary results It will be convenient to deﬁne S max = maxi ( S i +1 − S i ), S min = mini ( S i +1 − S i ), αmax = max j (α j +1 − α j ), αmin = mink (αk+1 − αk ). We assume that there is a mesh size/timestep parameter h such that

S max = C 1 h;

αmax = C 2 h;

τ = C 3 h ;

S min = C 1 h;

αmin = C 2 h,

(A.1)

where C 1 , C 1 , C 2 , C 2 , C 3 are constants independent of h.

If test function φ is of the forms (5.19)–(5.20), then we can write

φ S , B, α , τ , ψ( S , B, α , τ ) = B2 ψ( S /B, α , τ ),

(A.2)

where we assume that ψ( S /B , α , τ ) = ψ( z, α , τ ) is a smooth function of ( z, α , τ ), which has bounded derivatives with respect to ( z, α , τ ) on [ zmin , zmax ] × [αmin , αmax ] × [0, T ]. Note that since | B j | > 0, and B ˆj = B j (1 + O (h)), then φ has bounded derivatives with respect to ( S , B , α , τ ) for B near B0 , B1 , for h suﬃciently small, since ψ has bounded derivatives with respect to ( z, α , τ ). For more compact notation, we will also deﬁne

xni, j ,k = S i , B j , αk , τ n ,

φ S , B, α , τ , ψ( S , B, α , τ ) = φ x, ψ(x) , φin, j ,k = φ xni, j ,k = φ xni, j ,k , ψ xni, j ,k .

(A.3)

Taylor series (see [13]) gives

(Lh φ)ni, j ,k = (Lφ)ni, j ,k + O (h)

(A.4)

and if ξ is a constant, we also have (noting Eq. (A.2))

n φ x, ψ(x) + ξ i , j ,k = φin, j ,k + B2j ξ,

(A.5)

and

Lh φ(x, ψ + ξ ) ni, j ,k = (Lφ)ni, j ,k + O (h).

(A.6)

Assuming φ is of the form (A.2) and noting interpolation schemes (5.15)–(5.16) we obtain, using Eqs. (5.8)–(5.9) n +1

n+1 er τ − e g ( v i, j,k ) τ n +1 n +1 n τ , B j exp[r τ ] − v i , j ,k S i f v i , j ,k = φ S i exp g α + v τ , τ + O h2 k i , j ,k n +1 r − g ( v i , j ,k ) n +1 n+1 n +1 = φ S i + S i g v i , j ,k τ , B j + r B j − v i , j ,k S i f v i , j ,k τ , αk + v ni ,+j ,1k τ , τ n + O h2 . (A.7)

φˆn ˆ ˆ i , j ,k

v ni ,+j ,1k

Noting that

Bn 2 ˆj

Bj

= 1 + O (h)

(A.8)

and that if ξ is a constant, then the linear interpolation in Eqs. (5.15)–(5.16) is exact for constants, then we obtain

n φ x, ψ(x) + ξ ˆi , ˆj,kˆ = φ S i + S i g v ni ,+j ,1k τ , B j + r B j − v ni ,+j ,1k S i f v ni ,+j ,1k τ , αk + v ni ,+j ,1k τ , τ n + O h2 + B2j ξ 1 + O (h) .

(A.9)

258

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

A.2. Stability Deﬁnition A.1 (l∞ stability). Discretization (5.11) is l∞ stable if

n +1 V

∞

C4,

(A.10)

for 0 n N − 1 as h → 0, where C 4 is a constant independent of h. Here V n+1 ∞ = maxi , j ,k | V in,+j ,1k |. Lemma A.1 (l∞ stability). If the discretization (5.4) satisﬁes the positive coeﬃcient condition (5.5) and linear interpolation is used to compute V ˆn ˆ ˆ , then the scheme (5.11) with payoff (5.12), using the similarity reductions (5.15)–(5.16), satisﬁes i , j ,k

n V

∞

e 2rT V 0 ∞

(A.11)

for 0 n N = T / τ as h → 0. Proof. First, note that from payoff condition (5.12) we have 0 V i0, j ,k B 2L ∞ , which is bounded since the computational domain is bounded. Now, suppose that

0 V in, j ,k V n ∞ .

(A.12)

V in,+j ,k =

(A.13)

Deﬁne

min

v ni ,+j ,1k ∈ Z in,+j ,1k

V ˆn ˆ ˆ . i , j ,k

Since linear interpolation is used, then from Eq. (A.12), V in,+j ,k 0. Since v ni ,+j ,1k = 0 ∈ Z in,+j ,1k , then from Eqs. (5.8), (5.15)–(5.16)

and the fact that linear interpolation is used to compute V ˆn∗ ˆ , we have that 0 V in,+j ,k e 2r τ V n ∞ . i , j ,k Since discretization (5.4) is a positive coeﬃcient method, a straightforward maximum analysis shows that

0 V in,+j ,1k V n+ ∞ e 2r τ V n ∞ e 2rT V 0 ∞ .

2

(A.14)

1 n +1 , V in, j ,k = V i , j ,k − min V ˆn ˆ ˆ − τ (Lh V )ni ,+j ,1k i , j ,k τ v ni ,+j ,1k ∈ Z in,+j ,1k

(A.15)

A.3. Consistency Let

+1 Hni ,+j ,1k h, V in,+j ,1k , V ln,m ,p

l=i m= j p =k

where

+1 V ln,m ,p

(A.16)

l=i m= j p =k

+1 n is the set of values V ln,m , p , l = i, l = 0, . . . , i max and m = j, m = 0, . . . , j max , p = k, p = 0, . . . , kmax , and { V i , j ,k } is the set of n values V i , j ,k , i = 0, . . . , i max , j = 0, . . . , j max , k = 0, . . . , kmax . We can then deﬁne the complete discrete scheme as

+1 Gin,+j ,1k h, V in,+j ,1k , V ln,m ,p ≡ = 0.

l=i m= j p =k

, V in, j ,k

Hni ,+j ,1k

if 0 S i S i max , B j ∈ Bset ,

αmin αk αmax , 0 < τ n+1 T ,

V in,+j ,1k − ((B L )i , j ,k )2

if 0 S i S i max , B j ∈ Bset ,

αmin αk αmax , τ n+1 = 0 (A.17)

Remark A.1. We have written Eq. (A.15) as if we ﬁnd the exact minimum at each node. In practice, we ﬁnd the approximate minimum as described in Section 5.2. To avoid notational complexity, we will carry out our analysis assuming the algorithm determines the exact minimum. However, in view of Eq. (5.22), the use of the approximate minimum is a consistent approximation to the original problem, as long as the node spacing in [ v min , v max ] tends to zero as h → 0 [33].

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

259

Let Ω be the set of points ( S , B , α , τ ) such that Ω = [0, S max ] × Bset × [αmin , αmax ] × [0, T ]. The domain Ω can divided into the subregions

Ωin = [0, S max ) × Bset × (αmin , αmax ) × (0, T ], Ωαmin = [0, S max ) × Bset × {αmin } × (0, T ], Ωαmax = [0, S max ) × Bset × {αmax } × (0, T ], Ω S max = { S max } × Bset × (αmin , αmax ) × (0, T ], Ω S max αmin = { S max } × Bset × {αmin } × (0, T ], Ω S max αmax = { S max } × Bset × {αmax } × (0, T ], Ωτ 0 = [0, S max ] × Bset × [αmin , αmax ) × {0},

(A.18)

where Ωin represents the interior region, and Ωαmin , Ωαmax , Ω S max , Ωτ 0 , Ω S max αmax , Ω S max αmin denote the boundary regions. If x = ( S , B , α , τ ), let D V (x) = ( V S , V B , V α , V τ ) and D 2 V (x) = V S S . Let us deﬁne the following operators:

F in D 2 V (x), D V (x), V (x), x = V τ − L V − r B V B − min − v S f ( v ) V B + v V α + g ( v ) S V S ,

v∈ Z

F αmin D 2 V (x), D V (x), V (x), x = V τ − L V − r B V B − min − v S f ( v ) V B + v V α + g ( v ) S V S ,

v∈ Z +

F αmax D 2 V (x), D V (x), V (x), x = V τ − L V − r B V B − min − v S f ( v ) V B + v V α + g ( v ) S V S ,

v∈ Z −

F S max D 2 V (x), D V (x), V (x), x = V τ − r B V B − min − v S f ( v ) V B + v V α ,

v∈ Z

F S max αmin D 2 V (x), D V (x), V (x), x = V τ − r B V B − min − v S f ( v ) V B + v V α ,

v∈ Z +

F S max αmax D 2 V (x), D V (x), V (x), x = V τ − r B V B − min − v S f ( v ) V B + v V α ,

2

F τ 0 D V (x), D V (x), V (x), x = V

v∈ Z −

− B2L .

(A.19)

Then the problems (4.1)–(4.10) can be combined into one equation as follows:

F D 2 V (x), D V (x), V (x), x = 0 for all x = ( S , B , α , τ ) ∈ Ω,

(A.20)

where F is deﬁned by

F=

⎧ F in ( D 2 V (x), D V (x), V (x), x) ⎪ ⎪ ⎪ ⎪ 2 ⎪ ⎪ ⎪ F αmin ( D V (x), D V (x), V (x), x) ⎪ ⎪ 2 ⎪ ⎪ ⎨ F αmax ( D V (x), D V (x), V (x), x) 2

F S max ( D V (x), D V (x), V (x), x) ⎪ ⎪ ⎪ F S max αmax ( D 2 V (x), D V (x), V (x), x) ⎪ ⎪ ⎪ ⎪ ⎪ F S max αmin ( D 2 V (x), D V (x), V (x), x) ⎪ ⎪ ⎪ ⎩ F τ 0 ( V (x), x)

if x ∈ Ωin , if x ∈ Ωαmin , if x ∈ Ωαmax , if x ∈ Ω S max ,

(A.21)

if x ∈ Ω S max αmax , if x ∈ Ω S max αmin , if x ∈ Ωτ0 .

In order to demonstrate consistency, we ﬁrst need some intermediate results. For given τ , consider the continuous form of Eqs. (5.8)

Sˆ = S exp g ( v ) τ ,

r τ e − e g ( v ) τ ˆ , B = B exp[r τ ] − v S f ( v ) r − g(v ) αˆ = α + v τ , v ∈ [ v min , v max ].

(A.22)

Consider the domain

Ω Z ( τ ) ⊆ [0, S max ] × Bset × (αmin , αmax ) × (0, T ]

(A.23)

ˆ) ∈ where ( Sˆ , α / [0, S max ] × [αmin , αmax ]. In other words, for points in Ω Z , the range of possible values of v in Eq. (A.22) would have to be restricted to less than the full range [ v min , v max ] in order to ensure that 0 Sˆ S max ,

αmin αˆ αmax .

(A.24)

260

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

For example, the region

αmax − v max τ < α < αmax , αmin < α < αmin − v min τ ,

(A.25)

will be in Ω Z . In general, Ω Z will consist of small strips near the boundaries of Ω . We deﬁne the set Z (x, h) ⊆ Z such that if x ∈ Ω Z , then v ∈ Z (x, h) ensures that Eq. (A.24) is satisﬁed. We deﬁne the operator

F Z D 2 V (x), D V (x), V (x), x = V τ − L V − r B V B − min − v S f ( v ) V B + v V α + g ( v ) S V S ; v∈ Z

= V τ − r B V B min − v S f ( v ) V B + v V α ; v∈ Z

x ∈ Ω Z , S < S max

x ∈ Ω Z , S = S max .

(A.26)

Lemma A.2. For any smooth test function of the form

φ x, ψ(x) = B2 ψ( z, α , τ ), z=

S

(A.27)

B

where ψ has bounded derivatives with respect to ( z, α , τ ) for ( S , B , α , τ ) ∈ Ω , and

S i max −1 < S i max e − g ( v max ) τ

(A.28)

then

n+1 n+1 Gin,+j ,1k h, φ x, ψ(x) + ξ i , j ,k , φ x, ψ(x) + ξ l,m, p

=

⎧ ⎪ ⎪ F in + O (h) + O (ξ ) ⎪ ⎪ ⎪ ⎪ ⎪ F αmin + O (h) + O (ξ ) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ F αmax + O (h) + O (ξ ) ⎪ ⎪ ⎪ ⎪ ⎪ ⎨F + O (h) + O (ξ )

l=i m= j p =k

n , φ x, ψ(x) + ξ i , j ,k

if xni ,+j ,1k ∈ Ωin \ Ω Z , if xni ,+j ,1k ∈ Ωαmin , if xni ,+j ,1k ∈ Ωαmax , if xni ,+j ,1k ∈ Ω S max \ Ω Z ,

S max

⎪ F S max αmax + O (h) + O (ξ ) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ F S max αmin + O (h) + O (ξ ) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ F Z + O (h) + O (ξ ) ⎪ ⎪ ⎪ ⎩ F + O (ξ ) τ0

(A.29)

if xni ,+j ,1k ∈ Ω S max αmax , if xni ,+j ,1k ∈ Ω S max αmin , if xni ,+j ,1k ∈ Ω Z , if xni ,+j ,1k ∈ Ωτ 0

where ξ is a constant, and F in , F αmin , F αmax , F S max , F Z , F τ 0 , F S max αmax , F S max αmin are functions of ( D 2 φ(x), D φ(x), φ(x), x). Remark A.2. Condition (A.28) is a very mild restriction on the placement of node S i max −1 and is not practically restrictive. / ΩZ . This condition ensures that, for example, if xni ,+j ,1k ∈ Ωαmin or xni ,+j ,1k ∈ Ωαmax , then xni ,+j ,1k ∈ Proof. Consider the case x ∈ Ωin \ Ω Z . From Eqs. (A.4), (A.5), (A.6), (A.9), we obtain

n+1 1 φ x, ψ(x) + ξ i , j ,k −

τ

=

1

τ

φin,+j ,1k − φin, j ,k −

n φ x, ψ(x) + ξ ˆi , ˆj ,kˆ − τ Lh φ(x, ψ + ξ ) ni ,+j ,1k

min

v ni ,+j ,1k ∈ Z in,+j ,1k

min

v ni ,+j ,1k ∈ Z in,+j ,1k

(φ S )ni, j ,k S i g v ni ,+j ,1k τ + (φB )ni, j ,k r B j − v ni ,+j ,1k S i f v ni ,+j ,1k τ

+ (φα )ni, j ,k v ni ,+j ,1k τ + O h2 + O (hξ ) − (Lφ)ni ,+j ,1k + O (h)

(φ S )ni ,+j ,1k S i g v ni ,+j ,1k + (φB )ni ,+j ,1k r B j − v ni ,+j ,1k S i f v ni ,+j ,1k = (φτ )ni ,+j ,1k − (Lφ)ni ,+j ,1k − min v ni ,+j ,1k ∈ Z in,+j ,1k

+ (φα )ni, j ,k v ni ,+j ,1k + O (ξ ) + O (h) + O (h)

n+1 + O (ξ ) + O (h) = φτ − Lφ − min φ S S g ( v ) + φB r B − v S f ( v ) + φα v v∈ Z

i , j ,k

(A.30)

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

261

where we have taken the O (h), O (ξ ) terms out of the min since they are bounded functions of v ni ,+j ,1k (see [12]). As a result, we have

n+1 n+1 Gin,+j ,1k h, φ x, ψ(x) + ξ i , j ,k , φ x, ψ(x) + ξ l,m, p

l=i m= j p =k

n , φ x, ψ(x) + ξ i , j ,k

n+1 = F in D 2 φ(x), D φ(x), φ(x), x i , j ,k + O (h) + O (ξ ) if xni ,+j ,1k ∈ Ωin \ Ω Z . The rest of the results in Eq. (A.29) follow using similar arguments.

(A.31)

2

Recall the following deﬁnitions of upper and lower semi-continuous envelopes: Deﬁnition A.2. If C is a closed subset of R N , and f (x) : C → R is a function of x deﬁned in C , then the upper semicontinuous envelope f ∗ (x) and the lower semi-continuous envelope f ∗ (x) are deﬁned by

f ∗ (x) = lim sup f ( y ) and y →x y ∈C

f ∗ (x) = lim inf f ( y ).

(A.32)

y →x y ∈C

Lemma A.3 (Consistency). Assuming all the conditions in Lemma A.2 are satisﬁed, then the scheme (A.17) is consistent with the ˆ , τˆ ) ∈ Ω and any HJB equations (4.1), (4.4), (4.5), (4.7), (4.10) in Ω according to the deﬁnition in [7,5]. That is, for all xˆ = ( Sˆ , Bˆ , α function φ(x, ψ(x)) of the form φ(x, ψ(x)) = B 2 ψ( z, α , τ ), z = S /B , where ψ has bounded derivatives with respect to ( z, α , τ ) for ( S , B , α , τ ) ∈ Ω , and xni ,+j,1k = ( S i , B j , αk , τ n+1 ), we have

n+1

n+1

lim sup Gin,+j ,1k h, φ x, ψ(x) + ξ i , j ,k , φ x, ψ(x) + ξ l,m, p h→0 xni ,+j ,1k →ˆx ξ →0

l=i m= j p =k

n , φ x, ψ(x) + ξ i , j ,k

F ∗ D 2 φ(ˆx), D φ(ˆx), φ(ˆx), xˆ ,

(A.33)

and

n+1

n+1

lim inf Gin,+j ,1k h, φ x, ψ(x) + ξ i , j ,k , φ x, ψ(x) + ξ l,m, p h→0

xni ,+j ,1k →ˆx ξ →0

l=i m= j p =k

n , φ x, ψ(x) + ξ i , j ,k

F ∗ D 2 φ(ˆx), D φ(ˆx), φ(ˆx), xˆ .

(A.34)

Proof. According to the deﬁnition of lim inf, there exist sequences hq , i q , j q , kq , nq , ξq such that

h q → 0,

ξq → 0,

ˆ , τˆ ) xq ≡ S iq , B jq , αkq , τ nq +1 → ( Sˆ , Bˆ , α

as q → ∞,

(A.35)

and nq +1

lim inf Gi , j ,k hq , φ x, ψ(x) + ξq q q q q→∞

nq +1 i q , j q ,kq

n +1

, φ x, ψ(x) + ξq l,qm, p

n+1 n+1 = lim inf Gin,+j ,1k h, φ x, ψ(x) + ξ i , j ,k , φ x, ψ(x) + ξ l,m, p h→0

xni ,+j ,1k →ˆx ξ →0

l=i q m= j q p =kq

l=i m= j p =k

n

, φ x, ψ(x) + ξq i q, j q

q ,kq

n , φ x, ψ(x) + ξ i , j ,k .

(A.36)

Consider the case where xˆ ∈ Ωαmin i.e.

xˆ = ( S , B , αmin , τ ),

τ ∈ (0, T ]; S < S max .

(A.37)

Choose q suﬃciently large so that

0 S iq < S max ;

αmin αkq < αmax − v max ( τ )q .

(A.38)

262

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

For xq satisfying condition (A.38), and using Lemma A.2, we have

n +1 n+1 nq +1 Giq , jq ,kq hq , φ x, ψ(x) + ξq i q, j ,k , φ x, ψ(x) + ξq l,m, p q

q

q

l=i q m= j q p =kq

n

, φ x, ψ(x) + ξq i q, j q

q ,kq

⎧ 2 if xq ∈ Ωin \ Ω Z , ⎪ ⎨ F in ( D φ(xq ), D φ(xq ), φ(xq ), xq ) + O (hq ) + O (ξq ) 2 = F αmin ( D φ(xq ), D φ(xq ), φ(xq ), xq ) + O (hq ) + O (ξq ) if xq ∈ Ωαmin , ⎪ ⎩ F Z ( D 2 φ(xq ), D φ(xq ), φ(xq ), xq ) + O (hq ) + O (ξq ) if xq ∈ Ω Z .

(A.39)

For xq satisfying (A.38), since Z + ⊆ Z ⊆ Z , it follows from Eqs. (A.19) and (A.26) that

F in D 2 φ(xq ), D φ(xq ), φ(xq ), xq F Z D 2 φ(xq ), D φ(xq ), φ(xq ), xq

F αmin D 2 φ(xq ), D φ(xq ), φ(xq ), xq .

(A.40)

We then have nq +1

lim inf Gi , j ,k hq , φ x, ψ(x) + ξq q q q q→∞

nq +1 i q , j q ,kq

n+1

, φ x, ψ(x) + ξq l,m, p

l=i q m= j q p =kq

n

, φ x, ψ(x) + ξq i q, j q

q ,kq

lim inf F αmin D 2 φ(xq ), D φ(xq ), φ(xq ), xq − lim sup O (hq ) + O (ξq ) q→∞

q→∞

F ∗ D φ(ˆx), D φ(ˆx), φ(ˆx), xˆ , 2

(A.41)

where the last step follows since F αmin , F in are continuous functions of their arguments for smooth test functions, and F αmin F in . Let hq , i q , j q , kq , nq , ξq be sequences satisfying (A.35), such that nq +1

lim sup Gi , j ,k hq , φ x, ψ(x) + ξq q q q q→∞

nq +1 i q , j q ,kq

n +1

, φ x, ψ(x) + ξq l,qm, p

n+1 n+1 = lim sup Gin,+j ,1k h, φ x, ψ(x) + ξ i , j ,k , φ x, ψ(x) + ξ l,m, p h→0

xni ,+j ,1k →ˆx

l=i q m= j q p =kq

l=i m= j p =k

n

, φ x, ψ(x) + ξq i q, j q

q ,kq

n , φ x, ψ(x) + ξ i , j ,k .

(A.42)

ξ →0

Take q suﬃciently large so that condition (A.38) are satisﬁed. It follows from Eqs. (A.40) that

F Z D 2 φ(xq ), D φ(xq ), φ(xq ), xq F in D 2 φ(xq ), D φ(xq ), φ(xq ), xq ,

if xq ∈ Ω Z

(A.43)

hence nq +1

lim sup Gi , j ,k hq , φ x, ψ(x) + ξq q q q q→∞

nq +1 i q , j q ,kq

n+1

, φ x, ψ(x) + ξq l,m, p

l=i q m= j q p =kq

n

, φ x, ψ(x) + ξq i q, j q

q ,kq

lim sup F D 2 φ(xq ), D φ(xq ), φ(xq ), xq + lim sup O (hq ) + O (ξq ) q→∞

F

∗

q→∞

D φ(ˆx), D φ(ˆx), φ(ˆx), xˆ . 2

Similar arguments can be used to prove (A.33)–(A.34) for any xˆ in Ω .

(A.44)

2

Remark A.3 (Need for deﬁnition of consistency [7]). Note that in view of Eq. (A.39), there exist points near the boundaries where the discretized equations are never consistent in the classical sense with Eqs. (4.1), (4.4)–(4.5) and (4.10). Classical consistency would require that Z = ∅, which could only be achieved by placing restrictions on the timestep and ( α )min . These artiﬁcial restrictions are not required for the more relaxed deﬁnition of consistency (A.33)–(A.34). A.4. Monotonicity Using the methods in [18] it is straightforward to show that scheme (A.17) is monotone. Lemma A.4. If the discretization (5.4) is a positive coeﬃcient discretization, and interpolation schemes (5.15)–(5.16) is used with linear interpolation in the S × α plane, then discretization (A.17) satisﬁes

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

+1 Gin,+j ,1k h, V in,+j ,1k , Xln,m ,p

l=i m= j p =k

+1 Gin,+j ,1k h, V in,+j ,1k , Y ln,m ,p

263

, X in, j ,k l=i m= j p =k

, Y in, j ,k ;

for all X in, j ,k Y in, j ,k , ∀i , j , k, n.

(A.45)

n n n n Note that if the similarity reduction (3.12) is valid, then we can replace X in, j ,k by X m ,0, p , X m,1, p , and Y i , j ,k by Y m,0, p , n using Eqs. (5.15)–(5.16). Hence it follows from Lemma A.4 that the discretization is monotone in terms of X m,0, p , n n ∀m, p , n. Since Xm ,0, p , X m,1, p are essentially the discretized values of ψ( S /B , α , τ ) in Eq. (5.18), we have the precise form of monotonicity required in [7]. n Ym , 1, p , n Xm , 1, p ,

A.5. Convergence We make the assumption that there exists a unique, continuous viscosity solution to Eq. (3.2) with boundary conditions (4.4)–(4.5), (4.10), (4.7), at least in Ωin . This follows if the equation and boundary conditions satisfy a strong comparison property. Assumption A.1. If u and v are an upper semi-continuous subsolution and a lower semi-continuous supersolution of the pricing Eq. (3.2) associated with the boundary conditions (4.4)–(4.5), (4.10), (4.7), then

u v;

( S , B, α , τ ) ∈ Ωin .

(A.46)

A strong comparison result was proven in [6] for a general problem similar to Eq. (3.2). However, we violate some of the assumptions required in [6] (i.e. the domain is not smooth). We can now state the following result: Theorem A.1 (Convergence). Assume that scheme (A.17) satisﬁes all the conditions required by Lemmas A.1, A.3, A.4, and that Assumption A.1 holds, then scheme (A.17) converges to the unique, continuous viscosity solution to problem (3.2), with boundary conditions (4.4)–(4.5), (4.10), (4.7), for ( S , B , α , τ ) ∈ Ωin . Proof. This follows from the results in [7,5].

2

Remark A.4. Note that as discussed in [23], at points on the boundary where the PDE degenerates, it is possible that loss of boundary data may occur, and the solution can be discontinuous at these points. Hence, in general, we can only assume that strong comparison holds for points in the interior of the solution domain. In this situation, we should consider the computed solution to be the limit as we approach the boundary points from the interior. Appendix B. Convergence of the expected value Given the optimal control determined from the solution to Eq. (5.11), then Eq. (5.13) is a discretization of the linear PDE (4.11) with a classical solution. The discretization (5.13) is easily seen to be consistent. It is perhaps not immediately obvious that scheme (5.13) is l∞ stable, in view of the similarity reduction (5.15)–(5.16), with the control determined from Eq. (3.2). Note that |Bnˆ /B ∗ | may be greater than unity (see Eqs. (5.15)–(5.16)). However, we note that j

U in, j ,k

E tv=∗ 0 [B L ],

V in, j ,k E tv=∗ 0 (B L )2 so that if

V in, j ,k

(B.1)

is bounded, then

2

Var[B L ] = E tv=∗ 0 (B L )2 − E tv=∗ 0 [B L ] would imply a bound on (U in, j ,k )2 . Stability in the l∞ norm for U in, j ,k

0,

(B.2)

is a consequence of the following lemma.

Lemma B.1 (Stability of scheme (5.13)). If U n+1 is given by (5.13), with the discrete optimal control determined by the solution to Eq. (5.11), a positive coeﬃcient method is used to discretize the operator L as in Eq. (5.4), the discrete similarity interpolation operators are given by Eqs. (5.15)–(5.16), with linear interpolation in the S × α plane, and the payoff conditions given by Eqs. (5.12) and (5.14), then

U in, j ,k

2

V in, j ,k ;

∀i , j , k , n .

(B.3)

264

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

Proof. Deﬁne V nj,k = [ V 0n, j ,k , . . . , V i max , j ,k ]t , with Lh being the (i max + 1) × (i max + 1) matrix deﬁned in Eq. (5.4). Write Eqs. (5.11) and (5.13) as

[ I − τ Lh ] V nj ,+k 1 = V nj ,+k ;

V in,+j ,k =

min

v ni ,+j ,1k ∈ Z in,+j ,1k

[ I − τ Lh ]U nj ,+k 1 = U nj ,+k ;

V ˆn ˆ ˆ , i , j ,k

n+1

v ∗ i , j ,k ∈ arg min V ˆn ˆ ˆ , i , j ,k n +1 n +1

v i , j ,k ∈ Z i , j ,k

U in,+j ,k = U ˆn ˆ ˆ ( v ∗ )n+1 . i , j ,k i , j ,k

(B.4)

Since [ I − τ Lh ] is a diagonally dominant M matrix, and rowsum(Lh ) = 0, then

[ I − τ Lh ]−1 = G ;

G i ,l = 1; 0 G i ,l 1.

(B.5)

l

Assume (U in,+j ,k )2 V in,+j ,k , then since (Jenson’s inequality)

G i ,l U ln,+ j ,k

2

l

G i ,l U ln,+ j ,k

2

(B.6)

l

we have that (U in,+j ,1k )2 V in,+j ,1k . Using the interpolation operators (5.15)–(5.16) and the deﬁnitions of U (n+1)+ , V (n+1)+ we (n+1)+ 2 )

can see that (U i , j ,k

V i(,nj+,k1)+ . Finally, we have (U i0, j,k )2 = V i0, j,k . 2

Since V n+1 is l∞ stable from Lemma A.1, it follows from Lemma B.1 that U n+1 is l∞ stable. Remark B.1. Note that Lemma B.1 is true (in general) only if [ I − τ Lh ] is an M matrix, and linear interpolation is used in operators (5.15)–(5.16). References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28]

R. Almgren, N. Chriss, Optimal execution of portfolio transactions, Journal of Risk 3 (2000/2001 (Winter)) 5–39. R. Almgren, J. Lorenz, Bayesian adaptive trading with a daily cycle, Working paper, ETH, 2006. R. Almgren, C. Thum, E. Hauptmann, H. Li, Equity market impact, Risk (July 2005) 58–62. L. Bai, H. Zhang, Dynamic mean-variance problem with constrained risk control for the insurers, Mathematical Methods for Operations Research 68 (2008) 181–205. G. Barles, Convergence of numerical schemes for degenerate parabolic equations arising in ﬁnance, in: L.C.G. Rogers, D. Talay (Eds.), Numerical Methods in Finance, Cambridge University Press, Cambridge, 1997, pp. 1–21. G. Barles, E. Rouy, A strong comparison result for the Bellman equation arising in stochastic exit time control problems and applications, Communications in Partial Differential Equations 23 (1998) 1945–2033. G. Barles, P.E. Souganidis, Convergence of approximation schemes for fully nonlinear equations, Asymptotic Analysis 4 (1991) 271–283. S. Basak, G. Chabakauri, Dynamic mean-variance asset allocation, Review of Financial Studies 23 (2010) 2970–3016. D. Bertsimas, A. Lo, Optimal control of execution costs, Journal of Financial Markets 1 (1998) 1–50. T.R. Bielecki, H. Jin, S.R. Pliska, X.Y. Zhou, Continuous time mean-variance portfolio selection with bankruptcy prohibition, Mathematical Finance 15 (2005) 213–244. S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, New York, 2008. Z. Chen, P.A. Forsyth, A semi-Lagrangian approach for natural gas storage valuation and optimal control, SIAM Journal on Scientiﬁc Computing 30 (2007) 339–368. Y. D’Halluin, P.A. Forsyth, G. Labahn, A semi-Lagrangian approach for American–Asian options under jump diffusion, SIAM Journal on Scientiﬁc Computing 27 (1) (2005) 315–345. The Economist, The march of the robo-traders, The Economist Technology Quarterly (September 17, 2005). The Economist, Algorithmic trading: Ahead of the tape, The Economist (June 21, 2007) 85. R. Engle, R. Ferstenberg, Execution risk, Journal of Trading 2 (2) (2007) 10–20. W.H. Fleming, H.M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer, Berlin, 1993. P.A. Forsyth, G. Labahn, Numerical methods for controlled Hamilton–Jacobi–Bellman PDEs in ﬁnance, Journal of Computational Finance 11 (2007/2008 (Winter)) 1–44. P.A. Forsyth, J.S. Kennedy, S.T. Tse, H. Windcliff, Optimal trade execution: a mean quadratic variation approach, Quantitative Finance (2009), submitted for publication. C. Fu, A. Lari-Lavassani, X. Li, Dynamic mean-variance portfolio selection with borrowing constraint, European Journal of Operational Research 200 (2010) 312–319. H. He, H. Mamaysky, Dynamic trading with price impact, Journal of Economic Dynamics and Control 29 (2005) 891–930. G. Huberman, W. Stanzl, Price manipulation and quasi-arbitrage, Econometrica 72 (2004) 1247–1275. E. Jakobsen, Monotone schemes, in: R. Cont (Ed.), Encyclopedia of Quantitative Finance, Wiley, New York, 2010, pp. 1253–1263. X. Li, X.Y. Zhou, A. Lim, Dynamic mean-variance portfolio selection with no-shorting constraints, SIAM Journal on Control and Optimization 30 (2002) 1540–1555. F. Lillo, J. Farmer, R. Manttegna, Master curve for price impact function, Nature 421 (2003) 129. J. Lorenz, Risk-averse adaptive execution of portfolio transactions, Slides from a presentation, ETH Zurich. J. Lorenz, Optimal trading algorithms: Portfolio transactions, multiperiod portfolio selection, and competitive online search, PhD thesis, ETH Zurich, 2008. J. Lorenz, R. Almgren, Adaptive arrival price, in: Brian R. Bruce (Ed.), Algorithmic Trading III: Precision, Control, Execution, Institutional Investor Journals, 2007.

P.A. Forsyth / Applied Numerical Mathematics 61 (2011) 241–265

265

[29] A. Obizhaeva, J. Wang, Optimal trading strategy and supply/demand dynamics, Working paper, Sloan School, MIT, 2006. [30] M. Potters, J.-P. Bouchard, More statistical properties of order books and price impact, Physica A 324 (2003) 133–140. [31] A. Schied, T. Schoeneborn, Risk aversion and the dynamics of optimal liquidation strategies in illiquid markets, Finance and Stochastics 13 (2009) 181–204. [32] V. Ly Vath, M. Mnif, H. Pham, A model of optimal portfolio selection under liquidity risk and price impact, Finance and Stochastics 11 (2007) 51–90. [33] J. Wang, P.A. Forsyth, Maximal use of central differencing for Hamilton–Jacobi–Bellman PDEs in ﬁnance, SIAM Journal on Numerical Analysis 46 (2008) 1580–1601. [34] J. Xia, Mean-variance portfolio choice: Quadratic partial hedging, Mathematical Finance 15 (2005) 533–538. [35] X.Y. Zhou, D. Li, Continuous time mean variance portfolio selection: A stochastic LQ framework, Applied Mathematics and Optimization 42 (2000) 19–33.

Applied Numerical Mathematics 61 (2011) 266–284

Contents lists available at ScienceDirect

Applied Numerical Mathematics www.elsevier.com/locate/apnum

Two stable methods with numerical experiments for solving the backward heat equation Fabien Ternat a , Oscar Orellana b , Prabir Daripa c,∗ a b c

Institut de Recherche sur les Phénomènes Hors-Équilibre, IRPHE, Marseille, France Departamento de matemáticas, Universidad Técnica Santa María de Valparaíso, UTFSM, Chile Department of Mathematics, Texas A&M University, College Station, TX, United States

a r t i c l e

i n f o

Article history: Received 3 December 2009 Received in revised form 17 July 2010 Accepted 9 September 2010 Available online 14 October 2010 Keywords: Backward heat equation Ill-posed problem Numerical methods Crank–Nicolson method Euler scheme Dispersion relation Filtering Regularization

a b s t r a c t This paper presents results of some numerical experiments on the backward heat equation. Two quasi-reversibility techniques, explicit ﬁltering and structural perturbation, to regularize the ill-posed backward heat equation have been used. In each of these techniques, two numerical methods, namely Euler and Crank–Nicolson (CN), have been used to advance the solution in time. Crank–Nicolson method is very counter-intuitive for solving the backward heat equation because the dispersion relation of the scheme for the backward heat equation has a singularity (unbounded growth) for a particular wave whose ﬁnite wave number depends on the numerical parameters. In comparison, the Euler method shows only catastrophic growth of relatively much shorter waves. Strikingly we ﬁnd that use of smart ﬁltering techniques with the CN method can give as good a result, if not better, as with the Euler method which is discussed in the main text. Performance of these regularization methods using these numerical schemes have been exempliﬁed. © 2010 IMACS. Published by Elsevier B.V. All rights reserved.

1. Introduction The problem of heat conduction through a conducting medium occupying a space Ω subject to no heat ﬂux across the boundary of the region is formulated as follows:

⎧ ⎨ ut − ν u xx = 0, x ∈ Ω, t > 0, u | = 0, t > 0, ⎩ x ∂Ω u (x, 0) = u 0 (x), x ∈ Ω.

(1)

Here u (x, t ) is the temperature and u 0 (x) is the initial temperature distribution. This problem is known to be well-posed in the sense of Hadamard, i.e. existence, uniqueness and continuous dependence of the solution on the boundary data are well-established for this problem. The above problem is usually referred as a forward problem in the context of heat equation. The backward problem related to the heat equation refers to the problem of ﬁnding the initial temperature distribution of the forward problem from a knowledge of the ﬁnal temperature distribution v 0 (x) at time T :

*

Corresponding author. E-mail address: [email protected] (P. Daripa).

0168-9274/$30.00 © 2010 IMACS. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.apnum.2010.09.006

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

⎧ x ∈ Ω, t ∈ [0, T ], ⎨ ut − ν u xx = 0, u x |∂Ω = 0, ⎩ u (x, T ) = v 0 (x), x ∈ Ω.

267

(2)

The change of variable t → T − t leads to the following formulation of this backward problem where v (x, t ) = u (x, T − t ):

⎧ ⎨ v t + ν v xx = 0, x ∈ Ω, t ∈ [0, T ], v | = 0, ⎩ x ∂Ω v (x, 0) = v 0 (x), x ∈ Ω.

(3)

This backward problem is ill-posed on all three counts: existence, uniqueness and continuous dependence of solution on arbitrary initial data (see Nash [19], John [11], Miranker [17] and Hollig [9]), though this problem is well-posed for initial data whose Fourier spectrum has compact support (see Miranker [17]). However, in practice, an initial data cannot in general be guaranteed to have a compact support in Fourier space. When an initial data has a compact support in Fourier space, it loses this compactness in practice for a variety of reasons such as measurement error, noise in the measured data, round-off error in machine representations of such data, just to mention a few reasons. Integrating such equations by any numerical scheme further compounds this problem due to the effect of truncation error. Because of these reasons, even when a unique solution of the backward problem exists for some particular initial data, computing such a solution in some stable way has been a challenge for a long time (see Douglas and Gallie [6], John [11], Pucci [20]). A constructive approach to circumvent this computational challenge is to analyze ﬁrst the dispersion relation. The dispersion relation associated with the backward heat equation is ω = k2 , i.e. a mode with wave number k grows quadratically. This kind of catastrophic growth of short waves is also an indication that solutions (classical) of the backward problem may not always exist for all time for arbitrary initial data. This is all too well known for the backward heat equation for we know that any discontinuous temperature proﬁle gets smoothed out instantaneously by forward heat equation. Another consequence of this is the undesirable catastrophic growth of errors (in particular in high wave number modes) arising due to numerical approximation of the equation (truncation error), the machine representation of the data (roundoff error) and noise in any measured data. In this paper, computation of solutions of this ill-posed backward heat equation is undertaken on appropriately chosen space–time grid in conjunction with ﬁltering and regularization techniques. We present numerical results that show that solutions can be computed in stable ways for times longer than earlier reported by clever choice of the grids, ﬁlters, regularization term and clever dynamic application of the chosen ﬁlters. We also present detail outline of the procedures so that the computational results presented here can be reproduced by anyone interested in doing so. It is worth pointing out here that the ﬁltering techniques reported earlier in the literature with other ill-posed problems (see [13,22,4,5,8]) have been applied here successfully to this backward heat problem. 2. Numerical schemes and results The computational domain Ω is taken to be one dimensional, in particular Ω = [0, 1]. We discretize the interval [0, 1] with M subintervals x = 1/ M of equal length with grid points denoted by xm , m = 0, . . . , M. Integration in time is done in time step of t with time interval T = N × t and tn = n × t, n = 0, . . . , N. The exact value of the solution at (xm , tn ) is denoted by v (xm , tn ) and numerical value by v nm . Zero Neumann boundary conditions at both end points of the interval [0, 1] are approximated that results in the following third order accurate end point values of v for t > 0,

4v (x, t ) − v (2x, t )

+ O (x)3 , 3 4v (1 − x, t ) − v (1 − 2x, t ) + O (x)3 . v (1, t ) = 3 v (0, t ) =

(4) (5)

2.1. Euler scheme In terms of forward and backward ﬁnite difference operators D + and D − , the ﬁnite difference equation for the backward heat equation is

D t+ v nm

t

= −ν

− n D+ x D x vm

x2

,

∀m = {1, M }, ∀n > 2.

(6)

For numerical construction of solutions, it is useful to choose appropriate values of x and t so that numerical and exact dispersion relations do not deviate too much from each other over a range of participating wave numbers. Using the ansatz v nm = ρ n e i ξ m (where ρ = e βt and ξ = kπ x) in the ﬁnite difference equation (6) yields the dispersion relation,

ρ = 1 + 4ν r sin2 (kπ x/2), where r = t /x . When x → 0, we have which is same as the exact growth rate. 2

(7)

ρ ∼ 1 + (kπ ) ν t which gives, in the limit t → 0, β = ln |ρ |/t ∼ ν (kπ )2 2

268

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

2.2. Crank–Nicolson scheme The backward heat equation in this scheme is discretized as

D t+ v nm

t

=−

ν 2x2

− n +1 − n D+ + D+ x D x vm x D x vm .

(8)

For dispersion relation, same ansatz for v nm as in the Euler scheme is inserted in the ﬁnite difference equation (8). This yields the following dispersion relation ξ

ρ=

1 + 2ν r sin2 ( 2 ) 1 − 2ν r sin2

ξ

(9)

,

2

where r = t /x2 as before. When x → 0, we have

ρ∼

1 + ν (kπ )2 2t 1 − ν (kπ )2 2t

,

which gives, in the limit t → 0, β = ln |ρ |/t ∼ ν (kπ )2 which is the same as the exact dispersion relation. For r > 1/2ν , the dispersion relation has a singularity at k = ku given by

ku =

2

π x

arcsin √

x 2ν t

(10)

.

Figs. 1(a) and 1(b) compare the exact dispersion relation with the numerical ones for several values of space and time steps respectively for both the Euler and the CN schemes. The plots are log–log plots due to the large values of growth rates. Numerical dispersion plot for the CN scheme corresponding to x = 10−3 and t = 10−4 for which r > 1/2ν clearly shows the location of the singularity at ku = 45.05. Since the singularity and high values of the growth rate are very localized near a very high wave number with rest of the dispersion curves comparing favorably with the exact one, larger time steps may still be able to yield reasonably accurate solutions on the same grid x as for the other dispersion curves in the ﬁgure. We will test below whether this is indeed true or not. For the other choices of grid values used for the CN case in the ﬁgure, r is less than 1/2 (r < 1/2ν ). This ﬁgure shows that numerical dispersion curves compare favorably with the exact one up to a higher wave number for the CN scheme than for the Euler scheme. However, they all are almost same for up to a wave number approximately 25. 2.3. Numerical results Numerical experiments have been performed on many problems but the results from the ones corresponding to only the following problems are presented below for brevity. Example 1 (Single cosine mode). It is easy to see that the function

v e (x, t ) = cos(kπ x) exp −k2 π 2 ν ( T 0 − t ) ,

(11)

is the solution of the backward heat equation with initial data

v 0 (x) = cos(kπ x) exp −k2 π 2 ν T 0 .

(12)

Note that v x (x, t ) = 0 at x = 0, 1 for all t > 0. Example 2 (Gaussian). It is easy to check that

(x − 0.5)2 , exp − v (x, t ) = √ ν (5 − 4t ) 5 − 4t

1

0 t 1,

(13)

is the solution of the backward heat equation with initial data

(x − 0.5)2 1 . v (x, 0) = √ exp − 5ν 5 It follows that

v x (x, t ) =

−2(x − 0.5) (x − 0.5)2 , exp − ν (5 − 4t ) ν (5 − 4t )3/2

(14)

0 t 1,

which is not exactly zero at the end points. It can be made close to zero by choosing a small value of

ν.

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

269

Fig. 1. Comparison (in log scale) of the exact dispersion relations of the Euler and Crank–Nicolson schemes for various values of space and time steps. FDE stands for “Finite Difference Equation". Note the singularity at ku = 45.05 in the Crank–Nicolson FDE for t = 10−4 and x = 10−3 .

Example 3 (Square bump). This function is given by

⎧ ⎨ 0, 0 < x < 1 /4 , h(x) = 1, 1/4 < x < 3/4, ⎩ 0, 3 /4 > x > 1 .

(15)

The exact solution of the forward problem with this square bump initial data is given by

u (x, t ) = a0 +

∞

2 2 ak cos(kπ x)e −ν k π t ,

k =1

where

a 0 = 1 /2 ,

ak =

2 kπ

sin

3kπ 4

− sin

kπ 4

,

k 1.

It then follows that the exact solution of the backward heat equation with initial data

v (x, 0) = a0 +

∞

2 2 ak cos(kπ x)e −ν k π T 0 ,

(16)

k =1

is given by

v (x, t ) = a0 +

∞

2 2 ak cos(kπ x)e −ν k π ( T 0 −t ) ,

0 t T 0.

k =1

It is found that ﬁfty modes are more than suﬃcient to accurately represent the bump function (15). Therefore, the initial data (16) for the backward heat equation has been generated with ﬁfty modes in our applications later. First of all, we want to emphasize that we solve the backward heat equation in a ﬁnite interval. For a solution to exist, 2 following condition on initial data should be satisﬁed: “amplitude of the Fourier coeﬃcient should decay faster than e −k for a mode with wave number k for large k (see also Section 2.2 of [7])”. In Example 1, the cosine initial data has only one Fourier mode, thus the above condition is satisﬁed. It is easy to verify that the initial Gaussian data also satisﬁes this condition. For Example 3, the initial data v (x, 0) is generated using ﬁnite number of modes as mentioned above and hence this data also satisﬁes the above condition. Alternatively, one can also see that the initial data we use for these three

(2k)2 examples satisfy the Picard criterion (see [7, p. 39]) which for the problem of backward heat conduction is e | fk| < ∞ (see [7, Section 1.5]). For each of the examples above, using both the Euler and the Crank–Nicolson schemes, we compute numerical solutions v˜ (x, T ) from initial data v 0 (x) using 14-digit accurate arithmetic. We do the experiments on [0, 1] for various grid sizes and up to various time levels t. Fig. 2(a) shows plots of exact and numerical solutions based on the cosine initial data (12) with k = 1. Fig. 2(b) shows similar plots but with cosine initial data having k = 6. In both ﬁgures we see that quality of solutions at time levels t = 3.5 × 10−3 and t = 2 × 10−3 shown in Figs. 2(a) and 2(b) respectively is acceptable. For time

270

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Fig. 2. Cosine initial data (12) (Example 1). Comparison of exact (solid line) and numerical solutions (Euler in diamonds and CN in plus) for different initial data. Results are with t = 10−4 and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 3. Gaussian initial data (14) (Example 2). Comparison of exact (solid line) and numerical solutions (Euler in diamonds and CN in plus) for different initial data and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

levels beyond these the accuracy of solutions gradually deteriorates with increase in time of simulation due to growth of participating short waves present in the round-off and the truncation errors. The normalized L 2 norm of the error between the exact solution v e (., t ) and the numerical solution v (., t ) at time t, deﬁned by

e L 2 (t ) =

v (., t ) − v e (., t ) 2 ,

v e (., t ) 2

is shown in Table 1 for both cosine initial data. Figs. 3(a), 3(b) and 4 show similar plots for Gaussian initial data (14) (two different values of ) and square bump initial data (15) respectively. The normalized L 2 error norms are shown in Table 1. The evolution of the L 2 errors for these three examples is shown in Fig. 5. As expected, it grows exponentially for both the Euler and the Crank–Nicolson numerical schemes. However, a slight difference of maximum value can be noticed between them at t = 10−2 where errors are the largest with Crank–Nicolson. Next we show some results with noisy initial data. To generate noisy initial data for the three examples, a noise function is introduced into the initial condition of the backward problems as follows:

v δ (x, 0) = v (x, 0) × 1 + δ(x) ,

(17)

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

271

Fig. 4. Bump square data (15) (Example 3). Comparison of exact (solid line) and numerical solutions (Euler in diamonds and CN in plus) for M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 5. Plot of L 2 error versus time for three different examples without noise. For these plots M = 33, t = 10−4 .

Table 1 Relative error norms without ﬁltering. IC

Time

Schemes

e L2

10−4

t = 3.5 × 10−3

Euler CN

2.06 × 10−2 1.99 × 10−1

Cosine k=6

10−4

t = 2 × 10−3

Euler CN

1.45 × 10−1 4.5 × 10−1

Gaussian ν = 10−2

5 × 10−3

t = 2 × 10−1

Euler CN

5.41 × 10−2 9.85 × 10−2

Gaussian ν = 5 × 10−3

10−2

t = 8 × 10−1

Euler CN

8 × 10−2 2.88 × 10−1

Example 3 T 0 = 10−1

10−5

t = 3 × 10−3

Euler CN

4.08 × 10−3 5.04 × 10−3

Cosine k=1

t

where δ(x) is the noise generated using the MatLab function “rand” multiplied by a magnitude coeﬃcient δm :

δ(x) = δm × rand(x).

(18)

For a ﬁxed time t = 10−2 , Fig. 6 shows the plots of the L 2 error as a function of the noise parameter δm for both the Euler and the Crank–Nicolson schemes. In both cases, when the noise parameter is less than about 10−4 , the error remains at a

272

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Fig. 6. Plot of L 2 error at t = 10−2 versus noise parameter δm for three different examples. For these plots M = 33, t = 10−4 .

constant level (O (1010 ) for Euler and O (1013 ) for CN) corresponding to the values that can be observed without noise at t = 10−2 in Fig. 5. Above this value 10−4 of the noise parameter, the error grows with an increase in the noise parameter for two of the three examples as seen in Fig. 6. It should be noted that with exactly the same numerical conditions and the noise parameter, error with the Crank–Nicolson scheme is three orders of magnitude larger than that with the Euler scheme.

3. Filtering technique We have applied ﬁve different ﬁlters to control the spurious effects on the solution due to catastrophic growth of participating short wave components of the round-off and truncation errors. These low-pass ﬁlters, denoted as Φ(k; kc ), are applied on the Fourier spectrums ak of the solution at certain time intervals (see Daripa [4] and text below for their proper applications). This results in the ﬁltered spectrum ak and deﬁned by

ak (k; kc ) = Φ(k; kc )ak (k),

(19)

where ak and ak denote respectively the unﬁltered and ﬁltered Fourier coeﬃcients and kc is a parameter, called cut off wave number, on which the ﬁlter depends (see below). First of these ﬁlters is the sharp ﬁlter Φs deﬁned by

Φs (k) =

1, k k c , 0, k > k c .

(20)

We have also applied another three ﬁlters described in Appendix A. Two of these Φa (k) and Φe (k) are C ∞ ﬁlters and the other three Φi (k), i = 1, 2, 3 have varying degrees of smoothness with smoothness of the ﬁlters increasing with index i. Below, ﬁgures and tables show numerical solutions for times much longer than otherwise possible without ﬁlters. 3.1. Numerical results Figs. 7(a), 7(b), 8(a), 8(b) and 9 compare exact solutions against the numerical solutions obtained using the sharp ﬁlter. Two ways of ﬁltering have been used as it impacts on the result quality. On the one hand, the ﬁlter can be applied whenever the amplitude of the mode above the cut off (kc ) exceeds 10−5 : this method is called F1. On the other, the solution may be ﬁltered every time steps: let us call it F2. The method of ﬁltering has been reported in the results. In the tested examples, choice of the ﬁlter shape does not affect noticeably the solutions in the three examples when plotted. However, cut off wave number kc needs to be carefully selected in each case for it to be able to ﬁlter the spurious effects of computational (truncation and round-off errors) noise on the numerically constructed solutions. Errors as a function of ﬁlter type and value of the cut off wave number kc are shown in Table 2. In each case, the cut off wave number shown gives the reasonable good numerical solution. Data with other values of cut off wave numbers are not shown as these do not improve the solution. For a given precision (less than about 5 × 10−1 ), application of the ﬁlter enables computation of quality solutions for times more than what is otherwise possible without ﬁltering. The most dramatic improvement occurs with the Example 1 with k = 1, where time increases from 3.5 × 10−3 to t = 1 with the same

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

273

Fig. 7. Cosine initial data (12) (Example 1). Comparison of exact (solid line) and ﬁltered numerical solutions (Euler in diamonds and CN in plus) for different initial data and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 8. Gaussian initial data (14) (Example 2). Comparison of exact (solid line) and ﬁltered numerical solutions (Euler in diamonds and CN in plus) for different initial data and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 9. Bump square data (15) (Example 3). Comparison of exact (solid line) and ﬁltered numerical solutions (Euler in diamonds and CN in plus) for different initial data and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

274

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Table 2 Relative error norms with the three examples using ﬁltering. IC

t

Time

Cut off

Cosine k=1

5 × 10−3

t=1

kc = 2

Cosine k=6

4 × 10−4

t = 10−2

kc = 7

Gaussian ν = 10−2

10−4

t = 0.9

kc = 10

Gaussian ν = 5 × 10−3

5 × 10−2

t=1

kc = 16

Example 3 T 0 = 0.1

10−4

t = 0.095

kc = 4

Type

Schemes

e L2

F2

Euler CN

4.45 × 10−2 2.58 × 10−2

F1

Euler CN

1.72 × 10−1 5.45 × 10−2

F1

Euler CN

6.93 × 10−2 6.92 × 10−2

F1

Euler CN

1.36 × 10−1 6.33 × 10−2

F1

Euler CN

4.45 × 10−2 4.57 × 10−2

Fig. 10. Cosine initial data (12) (Example 1). Comparison of exact (solid line) and ﬁltered numerical solutions (Euler in diamonds and CN in plus) for different initial data with noise and M = 33. Noise parameter δm = 10%. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 11. Gaussian initial data (14) (Example 2). Comparison of exact (solid line) and ﬁltered numerical solutions (Euler in diamonds and CN in plus) for different initial data with noise and M = 33. Noise parameter δm = 10%. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

275

Fig. 12. Bump square data (15) (Example 3). Comparison of exact (solid line) and ﬁltered numerical solutions (Euler in diamonds and CN in plus) with noise and M = 33. Noise parameter δm = 1%. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Table 3 Relative error norms with the three examples using ﬁltering on noisy initial data. IC

t

Time

Cut off

Cosine k=1

5 × 10−3

t=1

kc = 2

Cosine k=6

4 × 10−4

t = 10−2

kc = 7

Gaussian ν = 10−2

5 × 10−3

t = 0.9

kc = 7

Gaussian ν = 5 × 10−3

10−2

t=1

kc = 10

Example 3 T 0 = 0.1

10−3

t = 0.05

kc = 4

Type

Schemes

e L2

F1

Euler CN

5.55 × 10−2 1.07 × 10−1

F2

Euler CN

1.31 × 10−1 1.06 × 10−1

F2

Euler CN

1.13 × 10−1 1.12 × 10−1

F2

Euler CN

2.41 × 10−1 2.41 × 10−1

F2

Euler CN

1.67 × 10−2 2.11 × 10−2

level of precision. Indeed, wavenumber of the mode being small, a small cut off may be applied and thus the contamination of the computation is delayed. The marginal enhancement is obtained with Example 2 with ν = 5 × 10−3 where simulation time for quality solution increases from 8 × 10−1 to 1. All the other cases are comprised between these extremes. The use of the method F1 of ﬁltering has been necessary only with Example 1. In the other cases, no enhancement has been noticed so that the method F2 has been preferred because it provides the results faster. Next we show results obtained with noisy initial data v δ (x, 0) = v (x, 0) × (1 + δ(x)), where δ(x) denotes the noise presented in Eq. (18). In the case of Example 3 with noise, the magnitude of the coeﬃcient had to be changed to δm = 1% as the value δm = 10% is comparable to the magnitude of the original initial function. Note that the function “rand” returns pseudo-random values drawn from a uniform distribution on the unit interval. Solutions analogous to those shown in Figs. 7(a), 7(b), 8(a), 8(b) and 9 but with noisy initial data are shown in Figs. 10(a), 10(b), 11(a), 11(b) and 12. Table 3 shows the various error norms with the sharp ﬁlter only because the results are similar with other ﬁlters. Similar to the case without noise, ﬁltering helps improve the accuracy of the solution at any instant of time. The noise introduces high wavenumbers in the spectrum of the initial condition of the backward problem. They contaminate the solution initially but does so even more drastically to the solution at later times. Application of the ﬁlter at every time step (method F2) has been used to compute relatively accurate solutions at later times. In simulations with noise, the ﬁlter cut off kc remains independent of the presence of noise. Next we show for all three examples, two different types of plots of L 2 error. In particular, Fig. 13 shows the L 2 error against time for both the numerical schemes. For these simulations, no random noise has been added on the initial data. Next we show results of simulations from initial data with random noise added as per ansatz (17). Fig. 14 shows L 2 error at a ﬁxed time level as a function of the noise parameter δm (see (18)). Comparison of Fig. 13 with Fig. 5 shows the effectiveness of the ﬁlters in limiting the contamination of the results by spurious growth of the high wave number modes of the round-off and discretization errors. On the other hand, comparison of Fig. 14 with Fig. 6 shows that noise levels δm < 10−4 does not affect the L 2 error for any of the examples in both the methods with this trend continuing even for some values of δm higher than 10−4 in some cases. However, as seen in these ﬁgures the error in Fig. 14 converges towards much smaller values than those obtained without ﬁlter as shown in Fig. 6. Growth of the error is clearly limited. But such

276

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Fig. 13. Plot of L 2 error versus time using ﬁlter for three different examples without noise. For these plots M = 33, t = 10−4 . Filter cut off is kc = 10, F2 method has been used where ﬁlter has been applied every 4th time step.

Fig. 14. Plot of L 2 error at t = 10−2 versus the noise parameter using ﬁlter for three different examples. For these plots M = 33, t = 10−4 . Filter cut off is kc = 10, F2 type.

control has its limitation: the ﬁlter also removes the information that is required to retrieve ﬁne scale features of solutions data when present and thus cannot be used to recover corners, discontinuities, etc. in the solutions if present. 4. Regularization technique There exist different types of regularization techniques (see [1,18,16,8,23]). Here we regularize the backward heat equation by adding a fourth order term. The resulting problem is given by

⎧ v t + ν v xx + v xxxx = 0, x ∈ Ω = [0, 1], t ∈ [0, T ], ⎪ ⎪ ⎨ v | = 0, t ∈ [0, T ], x ∂Ω ⎪ v | = 0, t ∈ [0, T ], ⎪ ⎩ xxx ∂Ω v (x, 0) = ψ(x), x ∈ Ω.

(21)

The dispersion relation of this Eq. (21)1 is given by

ω = (π k)2 ν − (π k)2 .

(22)

The most dangerous wave number kd with the maximum growth rate ωmax and the wave number k∗ of the neutral mode are given by

kd =

1

π

ν , 2

ωmax =

ν2 , 4

k∗ =

1

π

ν .

(23)

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

277

Fig. 15. Comparison of the exact (ω versus k) and numerical (|ρ |/t versus k) dispersion relations for the regularized problem for several values of space and time steps. For these plots ν = 1 and kd = 10.

An appropriate choice of small value for the regularizing parameter can signiﬁcantly curtail the spurious growth of short waves. We can see from the above formulas that we can equivalently treat kd or k∗ as regularizing parameter instead of since value of is automatically chosen if we choose either kd or k∗ . Next we discuss the schemes to be used for solving the above regularized problem. 4.1. Euler and CN schemes Euler scheme. The ﬁnite difference equation for the regularized Eq. (21) is

D t+ v nm

t

= −ν

− n D+ x D x vm

x2

−

− + − n D+ x D x D x D x vm

x4

,

∀m = {1, M }, ∀n > 2.

(24)

For numerical construction of the accurate solutions, it is also necessary to choose appropriate values of x and t so that numerical and exact dispersion relations do not deviate too much from each other over a range of participating wave numbers. Using the ansatz v nm = ρ n e i ξ m (where ρ = e βt and ξ = kπ x) in the ﬁnite difference equation (24) yields the dispersion relation

ρ = 1 + 4r sin2

ξ 2

− 16μ sin4

ξ ,

(25)

2

t . Fig. 15(a) compares the exact (see Eq. (22)) and the numerical (see Eq. (25)) dispersion relations for several x4 choices of parameter values. This ﬁgure shows that optimal step sizes are x = 10−4 and t = 10−3 when ν = 1 and

where

μ=

kd = 10. Note that with this set of parameters, the dispersion relation seems to be discontinuous in the range [12, 26] of k. In fact, this is due to the representation in log scale of the function that reaches a value close to zero. Crank–Nicolson scheme. The ﬁnite difference equation for the above regularized equation is

D t+ v nm

t

=−

ν 2x2

− n +1 − n D+ + D+ x D x vm x D x vm −

2x4

− + − n +1 − + − n D+ + D+ x D x D x D x vm x D x D x D x vm .

(26)

For dispersion relation, the same ansatz for v nm as in the Euler scheme is inserted in the ﬁnite difference equation (26) of the Crank–Nicolson scheme. This yields the following dispersion relation:

ρ=

ξ

ξ

ξ

ξ

1 + 2r sin2 ( 2 ) − 8μ sin4 ( 2 ) 1 − 2r sin2 ( 2 ) + 8μ sin4 ( 2 )

.

(27)

Fig. 15(a) compares the exact (see Eq. (22)) and the numerical (see Eq. (25)) dispersion relations for several choices of parameter values. This ﬁgure shows that the optimal step sizes are x = 10−4 and t = 10−3 when ν = 1 and kd = 10. Since the plot is in log scale, note that sometimes the growth factor is undeﬁned in a deﬁned range of wavenumber because some values of ρ are less than zero.

278

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Fig. 16. Cosine initial data (12) (Example 3). Comparison of exact (solid line) and regularized numerical solutions (Euler in diamonds and CN in plus) for different initial data and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

4.2. Numerical boundary conditions Regularization introduces a fourth order term in the equation and a second boundary condition at each boundary (see (21)). The fourth order derivative term is evaluated by ﬁnite central differences using ﬁve points: j

v xxxx (xi , t j ) =

j

j

j

j

v i +2 − 4v i +1 + 6v i − 4v i −1 + v i −2

x4

(28)

.

Because of this term in the discrete approximation of the regularized equation at the interior grid points, solutions at two j j points outside the domain, namely v −1 and v M +1 , are required. This is done the following way using the second boundary condition (21). The third order derivative term v xxx is approximated to second order accuracy as

v xxx |x=0 =

− 32 v (−x) + 5v (0) − 6v (x) + 3v (2x) − 12 v (3x) + O x2 . 3 x

Using the boundary condition v xxx = 0 and the third order accurate approximation (4) in the above formula, we obtain third order accurate formulae

v (−x)

4v (x) + 8v (2x) − 3v (3x) 9

(29)

,

and similarly at x = 1 (using Eq. (5))

v (1 + x)

4v (1 − x) + 8v (1 − 2x) − 3v (1 − 3x) 9

.

(30)

4.3. Numerical results Figs. 16(a), 16(b), 17(a), 17(b) and 18 compare the numerical solutions against the exact solutions of the regularized problem. The normalized L 2 error norms are shown in Table 4. The regularized Euler and CN schemes do not give the best results for the same parameter set. The parameters for which we obtain the best results for each of these scheme are mentioned in the caption of the ﬁgures and are also reported in Table 4. These numerical results conﬁrm that under a given tolerance of error, the simulation time can be increased signiﬁcantly using regularization. Finally, we present results of regularization of the backward problem subject to initial conditions with noise: v δ (x, 0) = v (x, 0) × (1 + δ(x)) (see Eq. (18)). The same settings as presented in the section dedicated to the ﬁltering, e.g., the magnitude of δ = 10% for all examples except for Example 3 where it is 1%, are used for these simulation with regularization. Plots are shown in Figs. 19(a), 19(b), 20(a), 20(b) and 21. Table 5 recaps the error norms. As expected, noise prevents from reaching as large simulation time as with pure initial data. As before, we present in Fig. 22 the error growth as a function of time using regularization. For these simulations, no random noise has been added on the initial data. The sensitivity of the solutions to noise has also been found to be interesting which is shown in Fig. 23. Comparing Fig. 22 with Fig. 5 and Fig. 23 with Fig. 6

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

279

Fig. 17. Gaussian initial data (14) (Example 2). Comparison of exact (solid line) and regularized numerical solutions (Euler in diamonds and CN in plus) for different initial data and M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 18. Bump square data (15) (Example 3). Comparison of exact (solid line) and regularized numerical solutions (Euler in diamonds and CN in plus) for M = 33. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Table 4 Relative error norms with regularization. IC

t

kd

Schemes

e L2

t = 10−1

kd = 4

Euler CN

3.18 × 10−2 3.17 × 10−2

10−3

t = 5 × 10−3

kd = 16 kd = 10

Euler CN

9.81 × 10−2 4.81 × 10−2

Gaussian ν = 10−2

5 × 10−2

t = 0.8

kd = 11 kd = 10

Euler CN

8.63 × 10−2 7.6 × 10−2

Gaussian ν = 5 × 10−3

10−2

t=1

kd = 28 kd = 24

Euler CN

1.53 × 10−1 1.51 × 10−1

Example 3 T 0 = 10−1

10−5

t = 5 × 10−2

kd = 6

Euler CN

2.11 × 10−2 2.12 × 10−2

Cosine k=1

10−5

Cosine k=6

Time

280

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Fig. 19. Cosine initial data (12) (Example 1). Comparison of exact (solid line) and regularized numerical solutions (Euler in diamonds and CN in plus) for different initial data with noise and M = 33. Noise parameter is δm = 10%. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 20. Gaussian initial data (14) (Example 2). Comparison of exact (solid line) and regularized numerical solutions (Euler in diamonds and CN in plus) for different initial data with noise and M = 33. Noise parameter δm = 10%. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

Fig. 21. Bump square data (15) (Example 3). Comparison of exact (solid line) and regularized numerical solutions (Euler in diamonds and CN in plus) with noise for M = 33. Noise parameter is δm = 1%. Plus symbols and diamond symbols are on top of each other wherever the contrast between diamond and plus symbols are in question in the ﬁgure.

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

281

Table 5 Relative error norms with regularization on noisy initial conditions. IC

t

kd

Schemes

e L2

t = 8 × 10−2

kd = 3

Euler CN

5.9 × 10−2 5.91 × 10−2

10−4

t = 3 × 10−3

kd = 12 kd = 12

Euler CN

8.26 × 10−2 7.13 × 10−2

Gaussian ν = 10−2

5 × 10−3

t = 0.7

kd = 8 kd = 8

Euler CN

8.54 × 10−2 8.55 × 10−2

Gaussian ν = 5 × 10−3

5 × 10−3

t = 0.8

kd = 10 kd = 10

Euler CN

8.02 × 10−2 7.91 × 10−2

Example 3 T 0 = 10−1

10−5

t = 2 × 10−2

kd = 3

Euler CN

1.21 × 10−2 1.21 × 10−2

Cosine k=1

10−5

Cosine k=6

Time

Fig. 22. Plot of L 2 error versus time using regularization for three different examples without noise. For these plots M = 33, t = 10−4 . Regularization parameter is given by kd = 10.

Fig. 23. Plot of L 2 error at t = 10−2 versus the noise parameter δm using regularization for three different examples. For these plots M = 33, t = 10−4 . Regularization parameter is given by kd = 10.

respectively, it is observed that regularization limits growth of the error with or without noise. Moreover, its effect is fully comparable to that of the ﬁlter (see Figs. 13 and 14). For our purposes below, value of the regularization parameter for which the L 2 error is least will be called optimal value, denoted by ∗ , of the parameter . The choice of the optimal value ∗ certainly depends on the noise parameter δm

282

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

Fig. 24. Plot of L 2 error at t = 2 × 10−3 versus the regularization parameter For these plots t = 10−5 and M = 33.

with the cosine initial data (k = 1) for three values of the noise parameter δm .

which is a measure of signal to noise ratio modulo some constant depending on the examples. A strategy that will allow selection of ∗ in dependency of the noise parameter δm is certainly helpful. However, it is not clear how to do this a priori. To get some insight into how to do this even a posteriori, plots of L 2 error are shown against the regularization parameter and the residual norm ( × v xxxx 2 ) in Figs. 24(a) and 24(b) respectively. The results are shown with CN scheme only and for the ﬁrst example only because general trends of the plots for other combinations of the two methods and the three examples of this paper are similar. The plots in Fig. 24(a) resemble U -curves and those in Fig. 24(b) resemble L-curves. It is worth mentioning here that L 2 errors and the residual norms were computed for decreasing values of the regularization parameter and then these plots were done. Therefore, it should be understood that the parameter decreases as any of the L-curves (including the one which looks more like a U for no noise case in Fig. 24(b)) is traced from right to left. We see from the U -curves that both, the minimal value of L 2 error (corresponding to ∗ ) and the optimal value ∗ decrease monotonically with decreasing values of the noise parameter δm . From the L-curves, same inference is drawn about the dependency of L 2 error on the noise. However, notice that the effect of decreasing away from the optimal value ∗ has much more dramatic effect on the L 2 error than on the residual. In the presence of noise, L 2 error increases rapidly with hardly any change in the residual (the L-part of the L-curves). Therefore, either of the curves can be used for choosing the optimal value ∗ . In general, smaller the magnitude of the noise, smaller the optimal value of the regularization parameter ∗ . The value of ∗ seems to remain constant when the noise parameter reaches a value less than 0.01% (ﬁgure is not shown here). Indeed, for such a value of δm < 0.01% and such time level, the error is no longer affected by the noise in agreement with the observation made in Fig. 6. As seen in the U -curves, for optimal choice ∗ of the regularizing parameter with noise level δm < 0.01% in the initial data, the regularized solution approximates the exact one having an L 2 error of the order of O (10−3 ). In concluding this section, we want to emphasize that the discussion here on U - and L-curves is based on plots made from data obtained at a speciﬁc time level. More research is needed (which will be a topic of research in the future) to determine, even a posteriori, the optimal value of the regularizing parameter in dependency of time of simulation. 5. Discussion and conclusion Two stable ways of computing solutions of backward heat equation, namely ﬁltering (direct ﬁltering of short waves) and regularization techniques (structural perturbation of the heat equation), have been proposed and discussed for their proper implementation. For each of these ways of computing stable solutions, two ﬁnite difference methods, namely the Euler method and the Crank–Nicolson (CN) method, for solving the associated initial boundary value problem have been devised. These schemes have been analyzed. In particular, (numerical) dispersion relations for these two numerical schemes associated with each of the two initial boundary value problems arising in ﬁltering and regularizing techniques respectively have been derived. Appropriate choice of parameters so that numerical dispersion relations well approximate the exact dispersion relations of the PDEs over the range of participating wave numbers is one of the important factors in devising stable ways of computing the numerical solutions of the backward heat equation. This has been one of the hallmarks of the success of these methods which has been exempliﬁed in this paper with adequate number of examples. Another important factor has been to apply the ﬁlter and set the level of the ﬁlter appropriately which are partly guided by severity of ill-posedness and partly by trial and error. We have shown here that in this way, we are able to compute stable solutions for times longer than

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

283

otherwise possible. The methods are new. It will be interesting to see whether these results compare favorably or not with other existing methods [2,3,10,12,14,15,21,24] which is a topic of future research. The ﬁltering and regularization methods are used to obtain smooth approximate solutions of ill-posed problems. The ﬁltering methods have been applied here in a way that can provide good approximate smooth solutions but falls short of providing singular solutions such as the ones with corners and discontinuities. Such corners and discontinuities are smoothed out in the solutions obtained by the way the ﬁltering techniques are applied here. Singular solutions can be obtained by better applications of the ﬁltering techniques which is diﬃcult to apply in general because the application process involves in part science and in part art (see [13]). In the regularization technique, we have provided the U -curve criterion for optimal choice of the regularizing parameter a posteriori. This optimal value is shown to decrease with decreasing noise level. Acknowledgements This paper has been made possible by a NPRP grant to one of the authors (Prabir Daripa) from the Qatar National Research Fund (a member of The Qatar Foundation) and by a SCAT grant to the other two authors (Fabien Ternat and Oscar Orellana). One of the authors (Fabien Ternat) thanks the Department of Mathematics at Texas A&M University for making his one month long summer visit to Dr. Daripa possible and enjoyable. He also thanks the Department of Mathematics at Universidad Tecnica Federico Santa Maria, Valparaiso, Chile for making his research under SCAT grant possible. We would also like to thank immensely the reviewers for their very constructive and insightful criticisms which have helps up to improve the paper. The statements made herein are solely responsibility of the authors. Appendix A. Deﬁnition of the ﬁlters used We have applied ﬁve ﬁlters one of which is described in the main body of the text and the rest four are deﬁned below: 1. Arctan ﬁlter Φa (k):

Φa (k) =

1

π

arctan −104 (k − kc ) + 0.5.

(31)

2. Three polynomial ﬁlters Φi (k): smoothness of the sharp ﬁlter deﬁned in the main body of the text can be improved by considering polynomial functions g i (see Daripa [4]):

⎧ k kc , ⎨ 1, Φi (k, p ) = 1 − g i (kˆ ), kc < k < k2 , ⎩ 0, k k2 ,

(32)

where kˆ = kk−−kkc . The smoothing functions are deﬁned respectively by: 2

c

g 1 (x) = x,

0 < x < 1, ⎧ 9 3 1 ⎪ ⎪ 0<x , ⎪ x , ⎪ ⎪ 2 3 ⎪ ⎨ 27 2 9 1 1 2 3 g 2 (k) = 9x + x − x+ , <x , ⎪ 2 2 2 3 3 ⎪ ⎪ ⎪ ⎪ 9 2 ⎪ 3 ⎩ 1 − (1 − x) , < x < 1, 2

(33)

(34)

3

⎧ 625 5 1 ⎪ ⎪ 0<x , x , ⎪ ⎪ 24 5 ⎪ ⎪ ⎪ ⎪ 625 5 3125 4 625 3 125 2 25 1 1 2 ⎪ ⎪ <x , ⎪ ⎪ − 6 x + 24 x − 12 x + 12 x − 24 x + 24 , ⎪ 5 5 ⎪ ⎨ 625 5 3125 4 4375 3 625 2 775 21 2 3 g 3 (k) = , <x , x − x + x − x + x− ⎪ 4 8 12 4 24 8 5 5 ⎪ ⎪ ⎪ ⎪ 625 3125 625 125 25 1 3 4 ⎪ 5 4 3 2 ⎪ ⎪ ⎪ 1 + 6 (1 − x) − 24 (1 − x) + 12 (1 − x) − 12 (1 − x) + 24 (1 − x) − 24 , 5 < x 5 , ⎪ ⎪ ⎪ ⎪ 4 ⎪ ⎩ 1 − 625 (1 − x)5 , < x < 1. 24

5

(35) These ﬁlters have varying degree of smoothness and how to apply these have been exempliﬁed in gory detail in Daripa [4].

284

F. Ternat et al. / Applied Numerical Mathematics 61 (2011) 266–284

References [1] K.A. Ames, L.E. Payne, Asymptotic behavior for two regularizations of the Cauchy problem for the backward heat equation, Mathematical Models and Methods in Applied Sciences 8 (1998) 187–202. [2] L.D. Chiwiacowsky, H.F. De Campo Velho, Different approaches for the solution of a backward heat conduction problem, Inverse Problem Engineering 11 (2003) 471–494. [3] D.T. Dang, H.T. Nguyen, Regularization and error estimates for nonhomogeneous backward heat problems, Electronic Journal of Differential Equations 2006-04 (2006) 1–10. [4] P. Daripa, Some useful ﬁltering techniques for ill-posed problems, Journal of Computational and Applied Mathematics 100 (1998) 161–171. [5] P. Daripa, H. Wei, A numerical study of an ill-posed Boussinesq equation arising in water waves and nonlinear lattices: Filtering and regularization techniques, Applied Mathematics and Computation 101 (1999) 159–207. [6] J.J. Douglas, T.M. Gallie, An approximate solution of an improper boundary value problem, Duke Mathematical Journal 26 (1959) 339–347. [7] H. Engl, M. Hanke, A. Neubauer, Regularization of Inverse Probems, Kluwer Academic Publishers, 2000. [8] C.-L. Fu, X.-T. Xiong, Z. Qian, Fourier regularization for a backward heat equation, Journal of Mathematical Analysis and Applications 331 (2007) 472– 480. [9] K. Höllig, Existence of inﬁnitely many solutions for a forward backward heat equation, Transactions of the American Mathematical Society 279 (1983) 299–316. [10] H. Houde, Y. Dongsheng, A non overlap decomposition method for the forward-backward heat equation, Journal of Computational and Applied Mathematics 159 (2003) 35–44. [11] F. John, Numerical solution of the equation of heat conduction for preceding times, Annali di Matematica Pura et Applicata Series IV 40 (1955) 129–142. [12] L. Kentaro, Numerical solution of backward heat conduction problems by a high order lattice-free ﬁnite difference method, Journal of the Chinese Institute of Engineers 27 (2004) 611–620. [13] R. Krasny, A study of singularity formation in a vortex sheet by the point-vortex approximation, Journal of Fluid Mechanics 167 (1986) 65–93. [14] D. Krawczyk-Stando, M. Rudnicki, Regularization parameter selection in discrete ill-posed problems – the use of the u-curve, International Journal of Applied Mathematics and Computer Sciences 17 (2007) 157–164. [15] N.S. Mera, The method of fundamental solutions for the backward heat conduction problem, Inverse Problems in Science and Engineering 13 (2005) 65–78. [16] N.S. Mera, L. Elliot, D.B. Ingham, An inversion method with decreasing regularization for the backward heat conduction problem, Numerical Heat Transfer, Part B 42 (2002) 215–230. [17] W. Miranker, A well posed problem for the backward heat equation, Proceedings of the American Mathematical Society 12 (1961) 243–247. [18] W.B. Muniz, F.M. Ramos, H.F. De Campos Velho, Entropy- and Tikhonov-based regularization techniques applied to the backwards heat equation, Computers and Mathematics with Applications 40 (2000) 1071–1084. [19] J. Nash, Continuity of solutions of parabolic and elliptic equations, American Journal of Mathematics 80 (1958) 931–954. [20] C. Pucci, Sui problemi di Cauchy non “ben posti”, Atti Accademia Nazionale dei Lincei. Rend. Classe di Scienze Fisiche, Matematiche e Naturali 18 (1955) 473–477. [21] A. Qian, C.-L. Fu, R. Shi, A modiﬁed method for a backward heat conduction problem, Applied Mathematics and Computation 185 (2007) 564–573. [22] I. Seidman, Optimal ﬁltering for the backward heat equation, SIAM Journal of Numerical Analysis 33 (1996) 162–170. [23] G. Teschke, M. Zhariy, M.J. Soares, A regularization of nonlinear diffusion equations in a multiresolution framework, Mathematical Methods in the Applied Sciences 31 (2008) 575–587. [24] X.-T. Xiong, C.-L. Fu, A. Qian, Two numerical methods for solving a backward heat conduction problem, Applied Mathematics and Computation 179 (2006) 370–377.