Recent Advances in Computational and Applied Mathematics

Recent Advances in Computational and Applied Mathematics

Theodore E. Simos Editor

Recent Advances in Computational and Applied Mathematics

Editor Theodore E. Simos Department of Mathematics College of Sciences King Saud University P.O. Box 2455 Riyadh 11451 Saudi Arabia and Laboratory of Computational Sciences Department of Computer Science and Technology University of Peloponnese 22100 Tripolis Greece [email protected]

ISBN 978-90-481-9980-8 e-ISBN 978-90-481-9981-5 DOI 10.1007/978-90-481-9981-5 Springer Dordrecht Heidelberg London New York Library of Congress Control Number: 2010938640 Mathematics Subject Classification (2000): 65, 65D20, 65Rxx, 65Lxx, 30, 30G35, 31B05, 31B35, 31, 33C05, 33C10, 33C15, 35A08, 35G15, 35Q40, 35R01, 41A60, 49Jxx, 49Nxx, 76-xx, 76A05, 81Q05, 81Qxx, 93Exx © Springer Science+Business Media B.V. 2011 Chapter 10 was created within the capacity of an US governmental employment and therefore is in the public domain. No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Cover design: WMXDesign Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

This volume includes an exciting collection of papers on computational and applied mathematics presenting the recent advances in several areas of this field. All the papers have been peer reviewed by at least three reviewers. In the paper entitled: “Fifty Years of Stiffness” by Luigi Brugnano, Francesca Mazzia and Donato Trigiante a review on the evolution of stiffness is presented. The authors also given a precise definition of stiffness which encompasses all the previous ones. In the paper entitled: “Efficient Global Methods for the Numerical Solution of Nonlinear Systems of Two point Boundary Value Problems” by Jeff R. Cash and Francesca Mazzia, the authors investigated the numerical methods for the solution of nonlinear systems of two point boundary value problems in ordinary differential equations. More specifically they answer to the question: “which codes are currently available for solving these problems and which of these codes might we consider as being state of the art”. Finally the authors included some new codes for BVP’s which are written in MATLAB. These codes was not available before and allow us for the first time in the literature the possibility of comparing some important MATLAB codes for solving boundary value problems. In the paper entitled: “Advances on collocation based numerical methods for Ordinary Differential Equations and Volterra Integral Equations” by D. Conte, R. D’Ambrosio, B. Paternoster a survey on collocation based numerical methods for the numerical integration of Ordinary Differential Equations and Volterra Integral Equations (VIEs) is presented. This survey starts from the classical collocation methods and arrive to the important modifications appeared in the literature. The authors consider also the multistep case and the usage of basis of functions other than polynomials. In the paper entitled: “Basic Methods for Computing Special Functions” by Amparo Gil, Javier Segura and Nico M. Temme, the authors given a survey of methods for the numerical evaluation of special functions, that is, the functions that arise in many problems in the applied sciences. They considered a selection of basic methods which are used frequently in the numerical evaluation of special functions. They discussed also several other methods which are available. Finally, they given examples of recent software for special functions which use the above mentioned methods v

vi

Preface

and they mentioned a list of new bibliography on computational aspects of special functions available on our website. In the paper entitled: “Melt Spinning: Optimal Control and Stability Issue” by Thomas Gotz and Shyam S.N. Perera, the authors studied a mathematical model which describe the melt spinning process of polymer fibers. The authors used Newtonian and non-Newtonian models in order to describe the rheology of the polymeric material. They also investigated two important properties, the optimization and the stability of the process. In the paper entitled: “On orthonormal polynomial solutions of the Riesz system in R3 ” by K. Gürlebeck and J. Morais, a special orthogonal system of polynomial solutions of the Riesz system in R3 is studied. This system presents a proportion with the complex case of the Fourier exponential functions {einθ }n≥0 on the unit circle and has the additional property that also the scalar parts of the polynomials form an orthogonal system. An application of the properties of the above system to the explicit calculation of conjugate harmonic functions with a certain regularity is also presented. In the paper entitled: “Brief survey on the CP methods for the Schrödinger equation” by L.Gr. Ixaru, a review of the CP methods is presented. The authors investigated, after years of research in the subject all the advantages over other methods. In the paper entitled: “Symplectic Partitioned Runge–Kutta methods for the numerical integration of periodic and oscillatory problems” by Z. Kalogiratou, Th. Monovasilis and T.E. Simos an investigation on Symplectic Partitioned Runge– Kutta methods (SPRK) is presented. More specifically they present the methodology for the construction of the exponentially/trigonometrically fitted SPRK. They applied the above methodology to methods with corresponding order up to fifth. The trigonometrically-fitted approach is based on two different types of construction: (i) fitting at each stage and (ii) Simos’s approach. The authors also derived SPRK methods with minimal phase-lag as well as phase-fitted SPRK methods. Finally, they applied the methods to several problems. In the paper entitled: “On the Klein-Gordon equation on some examples of conformally flat spin 3-manifolds” by Rolf Sören Kraußhar a review about recent results on the analytic treatment of the Klein-Gordon equation on some conformally flat 3-tori and on 3-spheres is presented. The paper has two parts. In the first part the time independent Klein-Gordon equation ( − α 2 )u = 0 (α ∈ R) on some conformally flat 3-tori associated with a representative system of conformally inequivalent spinor bundles is considered. In the second part a unified approach to represent the solutions to the Klein-Gordon equation on 3-spheres is described. The hp version of the finite element method (hp-FEM) combined with adaptive mesh refinement is a particularly efficient method. For this method a single error estimate can not simultaneously determine whether it is better to do the refinement by h or by p. Several strategies for making this determination have been proposed over the years. In the paper entitled: “A Survey of hp-Adaptive Strategies for Elliptic Partial Differential Equations” by William F. Mitchell and Marjorie A. McClain, the authors studied these strategies and demonstrate the exponential convergence rates with two classic test problems.

Preface

vii

In the paper entitled: “Vectorized Solution of ODEs in M ATLAB with Control of Residual and Error” by L.F. Shampine a study on vectorization which is very important to the efficient computation in the popular problem-solving environment M ATLAB is presented. More specifically, the author derived a new error control procedure which is based on vectorization. An explicit Runge—Kutta (7,8) pair of formulas that exploits vectorization is obtained. The new proposed method controls the local error at 8 points equally spaced in the span of a step. A new solver which is based on the above mentiobed pair and it is called odevr7 is developed. This solver is much more efficient than the solver ode45 which is recommended by M ATLAB. In the paper entitled: “Forecasting equations in complex-quaternionic setting” by W. Sprössig, the author considered classes of fluid flow problems under given initial value and boundary value conditions on the sphere and on ball shells in R3 . The author interest is emphasized to the forecasting equations and the deduction of a suitable quaternionic operator calculus. In the paper entitled: “Symplectic exponentially-fitted modified Runge–Kutta methods of the Gauss type: revisited” by G. Vanden Berghe and M. Van Daele, the development of symmetric and symplectic exponentially-fitted Runge–Kutta methods for the numerical integration of Hamiltonian systems with oscillatory solutions is studied. New integrators are obtained following the six-step procedure of Ixaru and Vanden Berghe (Exponential Fitting, Kluwer Academic, 2004). We would like to express our gratitude to the numerous (anonymous) referees, to Prof. Hélène de Rode, the President of the European Academy of Sciences for giving us the opportunity to come up with this guest editorial work. University of Peloponnese

T.E. Simos

Contents

1

Fifty Years of Stiffness . . . . . . . . . . . . . . . . . . . . . . . . . . Luigi Brugnano, Francesca Mazzia, and Donato Trigiante

2

Efficient Global Methods for the Numerical Solution of Nonlinear Systems of Two Point Boundary Value Problems . . . . . . . . . . . Jeff R. Cash and Francesca Mazzia

23

Advances on Collocation Based Numerical Methods for Ordinary Differential Equations and Volterra Integral Equations . . . . . . . . Dajana Conte, Raffaele D’Ambrosio, and Beatrice Paternoster

41

3

1

4

Basic Methods for Computing Special Functions . . . . . . . . . . . Amparo Gil, Javier Segura, and Nico M. Temme

67

5

Melt Spinning: Optimal Control and Stability Issues . . . . . . . . . 123 Thomas Götz and Shyam S.N. Perera

6

On Orthonormal Polynomial Solutions of the Riesz System in R3 . . 143 K. Gürlebeck and J. Morais

7

Brief Survey on the CP Methods for the Schrödinger Equation . . . 159 L.Gr. Ixaru

8

Symplectic Partitioned Runge-Kutta Methods for the Numerical Integration of Periodic and Oscillatory Problems . . . . . . . . . . . 169 Z. Kalogiratou, Th. Monovasilis, and T.E. Simos

9

On the Klein-Gordon Equation on Some Examples of Conformally Flat Spin 3-Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Rolf Sören Kraußhar ix

x

Contents

10 A Survey of hp-Adaptive Strategies for Elliptic Partial Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 William F. Mitchell and Marjorie A. McClain 11 Vectorized Solution of ODEs in M ATLAB with Control of Residual and Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 L.F. Shampine 13 Symplectic Exponentially-Fitted Modified Runge-Kutta Methods of the Gauss Type: Revisited . . . . . . . . . . . . . . . . . . . . . . . 289 G. Vanden Berghe and M. Van Daele Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

Contributors

Luigi Brugnano Dipartimento di Matematica, Università di Firenze, Viale Morgagni 67/A, 50134 Firenze, Italy, [email protected] Jeff R. Cash Department of Mathematics, Imperial College, South Kensington, London SW7, UK, [email protected] Dajana Conte Dipartimento di Matematica e Informatica, Università di Salerno, Fisciano, Italy, [email protected] Raffaele D’Ambrosio Dipartimento di Matematica e Informatica, Università di Salerno, Fisciano, Italy, [email protected] M. Daele Vakgroep Toegepaste Wiskunde en Informatica, Universiteit Gent, Krijgslaan 281-S9, 9000 Gent, Belgium Thomas Götz Department of Mathematics, TU Kaiserslautern, 67663 Kaiserslautern, Germany, [email protected] K. Gürlebeck Institut für Mathematik/Physik, Bauhaus-Universität Weimar, Coudraystr. 13B, 99421 Weimar, Germany, [email protected] Amparo Gil Departamento de Matemática Aplicada y CC. de la Computación, ETSI Caminos, Universidad de Cantabria, 39005 Santander, Spain [email protected] L.Gr. Ixaru Department of Theoretical Physics, “Horia Hulubei” National Institute of Physics and Nuclear Engineering, P.O. Box MG-6, Bucharest, Romania, [email protected]; Academy of Romanian Scientists, 54 Splaiul Independentei, 050094, Bucharest, Romania Z. Kalogiratou Department of Informatics and Computer Technology, Technological Educational Institute of Western Macedonia at Kastoria, P.O. Box 30, 521 00, Kastoria, Greece Francesca Mazzia Dipartimento di Matematica, Università di Bari, Via Orabona 4, 70125 Bari, Italy, [email protected] xi

xii

Contributors

Francesca Mazzia Dipartimento di Matematica, Università di Bari, Via Orabona 4, 70125 Bari, Italy Marjorie A. McClain Mathematical and Computational Sciences Division, National Institute of Standards and Technology, Gaithersburg, MD 20899-8910, USA William F. Mitchell Mathematical and Computational Sciences Division, National Institute of Standards and Technology, Gaithersburg, MD 20899-8910, USA, [email protected] Th. Monovasilis Department of International Trade, Technological Educational Institute of Western Macedonia at Kastoria, P.O. Box 30, 521 00, Kastoria, Greece J. Morais Institut für Mathematik/Physik, Bauhaus-Universität Weimar, Coudraystr. 13B, 99421 Weimar, Germany, [email protected] Beatrice Paternoster Dipartimento di Matematica e Informatica, Università di Salerno, Fisciano, Italy, [email protected] Shyam S.N. Perera Department of Mathematics, University of Colombo, Colombo 03, Sri Lanka, [email protected] Rolf Sören Kraußhar Fachbereich Mathematik, Technische Universität Darmstadt, Schloßgartenstraße 7, 64289 Darmstadt, Germany [email protected] Javier Segura Departamento de Matemáticas, Estadística y Computación, Universidad de Cantabria, 39005 Santander, Spain, [email protected] L.F. Shampine 1204 Chesterton Dr., Richardson, TX 75080, USA [email protected] T.E. Simos Laboratory of Computational Sciences, Department of Computer Science and Technology, Faculty of Science and Technology, University of Peloponnessos, 22100, Tripolis, Greece, [email protected]; Department of Mathematics, College of Sciences, King Saud University, P.O. Box 2455, Riyadh 11451, Saudi Arabia W. Sprössig Fakultaet fuer Mathematik und Informatik, TU Bergakademie Freiberg, Prueferstraße 9, 09596 Freiberg, Germany, [email protected] Nico M. Temme CWI, Science Park 123, 1098 XG Amsterdam, The Netherlands, [email protected] Donato Trigiante Dipartimento di Energetica, Università di Firenze, Via Lombroso 6/17, 50134 Firenze, Italy, [email protected] G. Vanden Berghe Vakgroep Toegepaste Wiskunde en Informatica, Universiteit Gent, Krijgslaan 281-S9, 9000 Gent, Belgium, [email protected]

Chapter 1

Fifty Years of Stiffness Luigi Brugnano, Francesca Mazzia, and Donato Trigiante

Abstract The notion of stiffness, which originated in several applications of a different nature, has dominated the activities related to the numerical treatment of differential problems for the last fifty years. Contrary to what usually happens in Mathematics, its definition has been, for a long time, not formally precise (actually, there are too many of them). Again, the needs of applications, especially those arising in the construction of robust and general purpose codes, require nowadays a formally precise definition. In this paper, we review the evolution of such a notion and we also provide a precise definition which encompasses all the previous ones.

Keywords Stiffness · ODE problems · Discrete problems · Initial value problems · Boundary value problems · Boundary value methods Mathematics Subject Classification (2000) 65L05 · 65L10 · 65L99 Frustra fit per plura quod potest per pauciora. Razor of W. of Ockham, doctor invincibilis.

Work developed within the project “Numerical methods and software for differential equations”. L. Brugnano () Dipartimento di Matematica, Università di Firenze, Viale Morgagni 67/A, 50134 Firenze, Italy e-mail: [email protected] F. Mazzia Dipartimento di Matematica, Università di Bari, Via Orabona 4, 70125 Bari, Italy e-mail: [email protected] D. Trigiante Dipartimento di Energetica, Università di Firenze, Via Lombroso 6/17, 50134 Firenze, Italy e-mail: [email protected] T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_1, © Springer Science+Business Media B.V. 2011

1

2

L. Brugnano et al.

1.1 Introduction The struggle generated by the duality short times–long times is at the heart of human culture in almost all its aspects. Here are just a few examples to fix the idea: • in historiography: Braudel’s distinction among the geographic, social and individual times;1 • in the social sphere: Societies are organized according to three kinds of laws, i.e., codes (regulating short term relations), constitutions (regulating medium terms relations), and ethical laws (long term rules) often not explicitly stated but religiously accepted; • in the economy sphere: the laws of this part of human activities are partially unknown at the moment. Some models (e.g., the Goodwin model [19]), permits us to say, by taking into account only a few variables, that the main evolution is periodic in time (and then predictable), although we are experiencing an excess of periodicity (chaotic behavior). Nevertheless, some experts claim (see, e.g., [18]) that the problems in the predictability of the economy are mainly due to a sort of gap in passing information from a generation to the next ones, i.e. to the conflict between short time and long time behaviors.2 Considering the importance of this concept, it would have been surprising if the duality “short times–long times” did not appear somewhere in Mathematics. As a matter of fact, this struggle not only appears in our field but it also has a name: stiffness. Apart from a few early papers [10, 11], there is a general agreement in placing the date of the introduction of such problems in Mathematics to around 1960 [17]. They were the necessities of the applications to draw the attention of the mathematical community towards such problems, as the name itself testifies: “they have been termed stiff since they correspond to tight coupling between the driver and the driven components in servo-mechanism” ([12] quoting from [11]). Both the number and the type of applications proposing difficult differential problems has increased exponentially in the last fifty years. In the early times, the problems proposed by applications were essentially initial value problems and, consequently, the definition of stiffness was clear enough and shared among the few experts, as the following three examples evidently show: D1: Systems containing very fast components as well as very slow components (Dahlquist [12]). D2: They represent coupled physical systems having components varying with very different times scales: that is they are systems having some components varying much more rapidly than the others (Liniger [31], translated from French). 1 Moreover, his concept of structure, i.e. events which are able to accelerate the normal flow of time, is also interesting from our point of view, because it somehow recalls the mathematical concept of large variation in small intervals of time (see later). 2 Even

Finance makes the distinction between short time and long time traders.

1 Fifty Years of Stiffness

3

D3: A stiff system is one for which λmax is enormous so that either the stability or the error bound or both can only be assured by unreasonable restrictions on h. . . Enormous means enormous relative to the scale which here is t¯ (the integration interval) . . . (Miranker [34]). The above definitions are rather informal, certainly very far from the precise definitions we are accustomed to in Mathematics, but, at least, they agree on a crucial point: the relation among stiffness and the appearance of different time-scales in the solutions (see also [24]). Later on, the necessity to encompass new classes of difficult problems, such as Boundary Value Problems, Oscillating Problems, etc., has led either to weaken the definition or, more often, to define some consequence of the phenomenon instead of defining the phenomenon itself. In Lambert’s book [29] five propositions about stiffness, each of them capturing some important aspects of it, are given. As matter of fact, it has been also stated that no universally accepted definition of stiffness exists [36]. There are, in the literature, other definitions based on other numerical difficulties, such as, for example, large Lipschitz constants or logarithmic norms [37], or nonnormality of matrices [23]. Often is not even clear if stiffness refers to particular solutions (see, e.g. [25]) or to problems as a whole. Sometimes one has the feeling that stiffness is becoming so broad to be nearly synonymous of difficult. At the moment, even if the old intuitive definition relating stiffness to multiscale problems survives in most of the authors, the most successful definition seems to be the one based on particular effects of the phenomenon rather than on the phenomenon itself, such as, for example, the following almost equivalent items: D4: Stiff equations are equations where certain implicit methods . . . perform better, usually tremendous better, than explicit ones [11]. D5: Stiff equations are problems for which explicit methods don’t work [21]. D6: If a numerical method with a finite region of absolute stability, applied to a system with any initial condition, is forced to use in a certain interval of integration a step length which is excessively small in relation to the smoothness of the exact solution in that interval, then the system is said to be stiff in that interval [29]. As usually happens, describing a phenomenon by means of its effects may not be enough to fully characterize the phenomenon itself. For example, saying that fire is what produces ash, would oblige firemen to wait for the end of a fire to see if the ash has been produced. In the same way, in order to recognize stiffness according to the previous definitions, it would be necessary to apply first one3 explicit method and see if it works or not. Some authors, probably discouraged by the above defeats in giving a rigorous definition, have also affirmed that a rigorous mathematical definition of stiffness is not possible [20]. It is clear that this situation is unacceptable for at least two reasons: 3 It

is not clear if one is enough: in principle the definition may require to apply all of them.

4

L. Brugnano et al.

• it is against the tradition of Mathematics, where objects under study have to be precisely defined; • it is necessary to have the possibility to recognize operatively this class of problems, in order to increase the efficiency of the numerical codes to be used in applications. Concerning the first item, our opinion is that, in order to gain in precision, it would be necessary to revise the concept of stability used in Numerical Analysis, which is somehow different from the homonym concept used in all the other fields of Mathematics, where stable are equilibrium points, equilibrium sets, reference solutions, etc., but not equations or problems4 (see also [17] and [30]). Concerning the second item, operatively is intended in the sense that the definition must be stated in terms of numerically observable quantities such as, for example, norms of vectors or matrices. It was believed that, seen from the applicative point of view, a formal definition of stiffness would not be strictly necessary: Complete formality here is of little value to the scientist or engineer with a real problem to solve [24]. Nowadays, after the great advance in the quality of numerical codes,5 the usefulness of a formal definition is strongly recognized, also from the point of view of applications: One of the major difficulties associated with the study of stiff differential systems is that a good mathematical definition of the concept of stiffness does not exist [6]. In this paper, starting from ideas already partially exposed elsewhere [2, 4, 26], we will try to unravel the question of the definition of stiffness and show that a precise and operative definition of it, which encompasses all the known facets, is possible. In order to be as clear as possible, we shall start with the simpler case of initial value for a single linear equation and gradually we shall consider more general cases and, eventually, we shall synthesize the results.

1.2 The Asymptotic Stability Case For initial value problems for ODEs, the concept of stability concerns the behavior of a generic solution y(t), in the neighborhood of a reference solution y(t), ¯ when the initial value is perturbed. When the problem is linear and homogeneous, the difference, e(t) = y(t) − y(t), ¯ satisfies the same equation as y(t). ¯ For nonlinear problems, one resorts to the linearized problem, described by the variational equation, which, essentially, provides valuable information only when y(t) ¯ is asymptotically stable. Such a variational equation can be used to generalize to nonlinear problems the arguments below which, for sake of simplicity, concerns only the linear case. 4 Only in particular circumstances, for example in the linear case, it is sometimes allowed the language abuse: the nonlinear case may contain simultaneously stable and unstable solutions. 5A

great deal of this improvement is due to the author of the previous sentence.

1 Fifty Years of Stiffness

5

Originally, stiffness was almost always associated with initial value problems having asymptotically stable equilibrium points (dissipative problems) (see, e.g., Dahlquist [13]). We then start from this case, which is a very special one. Its peculiarities arise from the following two facts:6 • it is the most common in applications; • there exists a powerful and fundamental theorem, usually called Stability in the first approximation Theorem or Poincaré-Liapunov Theorem, along with its corollary due to Perron7 , which allows us to reduce the study of stability of critical points, of a very large class of nonlinearities, to the study of the stability of the corresponding linearized problems (see, e.g., [9, 27, 35, 38]). The former fact explains the pressure of applications for the treatment of such problems even before the computer age. The latter one provides, although not always explicitly recognized, the mathematical solid bases for the profitable and extensive use, in Numerical Analysis, of the linear test equation to study the fixed-h stability of numerical methods. We shall consider explicitly the case where the linearized problem is autonomous, although the following definitions will take into account the more general case. Our starting case will then be that of an initial value problem having an asymptotically stable reference solution, whose representative is, in the scalar case, y  = λy, y(0) = η,

t ∈ [0, T ],

Re λ < 0,

(1.2.1)

where the reference solution (an equilibrium point, in this case) has been placed at the origin. From what is said above, it turns out that it is not by chance that it coincides with the famous test equation. Remark 1.1 It is worth observing that the above test equation is not less general than y  = λy + g(t), which very often appears in the definitions of stiffness: the t only difference is the reference solution, which becomes y(t) ¯ = 0 eλ(t−s) g(s)ds, but not the topology of solutions around it. This can be easily seen by introducing the new variable z(t) = y(t) − y(t) ¯ which satisfies exactly equation (1.2.1) and then, trivially, must share the same stiffness. Once the solution z(t) of the homogeneous equation has been obtained, the solution y(t) is obtained by adding to it y(t) ¯ which, in principle, could be obtained by means of a quadrature formula. This allows us to conclude that if any stiffness is in the problem, this must reside in the homogeneous part of it, i.e., in problem (1.2.1). 6 We omit, for simplicity, the other fact which could affect new definitions, i.e., the fact that the solutions of the linear equation can be integrated over any large interval because of the equivalence, in this case, between asymptotic and exponential stability. 7 It is interesting to observe that the same theorem is known as the Ostrowsky’s Theorem, in the theory of iterative methods.

6

L. Brugnano et al.

Remark 1.2 We call attention to the interval of integration [0, T ], which depends on our need for information about the solution, even if the latter exists for all values of t . This interval must be considered as datum of the problem. This has been sometimes overlooked, thus creating some confusion. Having fixed problem (1.2.1), we now look for a mathematical tool which allows us to state formally the intuitive concept, shared by almost all the definitions of stiffness: i.e., we look for one or two parameters which tell us if in [0, T ] the solution varies rapidly or not. This can be done easily by introducing the following two measures for the solution of problem (1.2.1):  1 1 1 T γc = |y(t)|dt, (1.2.2) max |y(t)|, κc = |η| t∈[0,T ] |η| T 0 which, in the present case, assume the values: κc = 1,

γc =

1 1 T∗ (1 − eRe λT ) ≈ = , |Re λ|T |Re λ|T T

where T ∗ = |Re λ|−1 is the transient time. The two measures κc , γc are called conditioning parameters because they measure the sensitivity of the solution subject to a perturbation of the initial conditions in the infinity and in the l1 norm. Sometimes, it would be preferable to use a lower value of γc , i.e., γc =

1 . |λ|T

(1.2.3)

This amounts to consider also the oscillating part of the solution (see also Remark 1.5 below). By looking at Fig. 1.1, one realizes at once that a rapid variation of the solution in [0, T ] occurs when kc  γc . It follows then that the parameter σc =

kc T ≡ ∗, γc T

(1.2.4)

which is the ratio between the two characteristic times of the problem, is more significant. Consequently, the definition of stiffness follows now trivially: Definition 1.3 The initial value problem (1.2.1) is stiff if σc  1. The parameter σc is called stiffness ratio. Remark 1.4 The width of the integration interval T plays a fundamental role in the definition. This is an important point: some authors, in fact, believe that stiffness should concern equations; some others believe that stiffness should concern problems, i.e., equations and data. We believe that both statements are partially correct: stiffness concerns equations, integration time, and a set of initial data (not a specific one of them). Since this point is more important in the non scalar case, it will be discussed in more detail later.

1 Fifty Years of Stiffness

7

Fig. 1.1 Solutions and values of kc and γc in the cases λ = −0.2 (left plot) and λ = −2 (right plot)

Remark 1.5 When γc is defined according to (1.2.3), the definition of stiffness continues to be also meaningful in the case Re λ = 0, i.e., when the critical point is only marginally stable. In fact, let λ = iω ≡ i

2π . T∗

Then, T , T∗ and the definition encompasses also the case of oscillating stiffness introduced by some authors (e.g., [34]). Once again the stiffness is the ratio of two times. If information about the solution on the smaller time scale is needed, an adequately small stepsize should be used. It is worth noting that high oscillating systems (with respect to T ) fall in the class of problems for which explicit methods do not work, and then are stiff according to definitions D4–D6. When λ = 0, then kc = γc = σc = 1. In the case Re λ > 0 (i.e., the case of an unstable critical point), both parameters kc and γc grow exponentially with time. This implies that small variations in the initial conditions will imply exponentially large variations in the solutions, both pointwise and on average: i.e., the problem is ill conditioned. σc = 2π

Of course, the case Re λ = 0 considered above cannot be considered as representative of more difficult nonlinear equations, since linearization is in general not allowed in such a case. The linearization is not the only way to study nonlinear differential (or difference) equations. The so called Liapunov second method can be used as well (see, e.g., [22, 27, 38]). It has been used, in connection with stiffness in [5, 13–17], al-

8

L. Brugnano et al.

though not always explicitly named.8 Anyway, no matter how the asymptotic stability of a reference solution is detected, the parameters (1.2.2) and Definition 1.3 continue to be valid. Later on, the problem of effectively estimating such parameters will also be discussed.

1.2.1 The Discrete Case Before passing to the non scalar case, let us now consider the discrete case, where some interesting additional considerations can be made. Here, almost all we have said for the continuous case can be repeated. The first approximation theorem can be stated almost in the same terms as in the continuous case (see e.g. [28]). Let the interval [0, T ] be partitioned into N subintervals of length hn > 0, thus  defining the mesh points: tn = nj=1 hj , n = 0, 1, . . . , N . The linearized autonomous problem is now: yn+1 = μn yn ,

n = 0, . . . , N − 1,

y0 = η,

(1.2.5)

where the {μn } are complex parameters. The conditioning parameters for (1.2.5), along with the stiffness ratio, are defined as: κd =

1 max |yi |, |η| i=0,...,N

γd =

N 1 1 hi max(|yi |, |yi−1 |), |η| T i=1

(1.2.6)

kd σd = . γd This permits us to define the notion of well representation of a continuous problem by means of a discrete one. Definition 1.6 The problem (1.2.1) is well represented by (1.2.5) if kc ≈ kd ,

(1.2.7)

γc ≈ γd .

(1.2.8)

In the case of a constant mesh-size h, μn ≡ μ and it easily follows that the condition (1.2.7) requires |μ| < 1. It is not difficult to recognize the usual A-stability conditions for one-step methods (see Table 1.1). Furthermore, it is easily recognized that the request that condition (1.2.7) holds uniformly with respect to hλ ∈ C− implies that the numerical method producing (1.2.5) must be implicit. What does condition (1.2.8) require more? Of course, it measures how faithT  fully the integral 0 |y(t)|dt is approximated by the quadrature formula N i=1 hi · max(|yi |, |yi−1 |), thus giving a sort of global information about the behavior of the 8 Often,

it appears under the name of one-sided Lipschitz condition.

1 Fifty Years of Stiffness

9

Table 1.1 Condition (1.2.7) for some popular methods

Method

μ

Condition

Explicit Euler

1 + hλ

Implicit Euler

1 1−hλ

Trapezoidal rule

1+hλ/2 1−hλ/2

|1 + hλ| < 1  1    1−hλ < 1  1+hλ/2    1−hλ/2 < 1

method producing the approximations {yi }. One of the most efficient global strategies for changing the stepsize is based on monitoring this parameter [3, 4, 7, 8, 32, 33]. In addition to this, when finite precision arithmetic is used, then an interesting property of the parameter γd occurs [26]: if it is smaller than a suitably small threshold, this suggests that we are doing useless computations, since the machine precision has already been reached.

1.2.2 The non Scalar Case In this case, the linearized problem to be considered is y  = Ay,

t ∈ [0, T ],

y(0) = η,

(1.2.9)

with A ∈ Rm×m and having all its eigenvalues with negative real part. It is clear from what was said in the scalar case that, denoting by (t) = eAt the fundamental matrix of the above equation, the straightforward generalization of the definition of the conditioning parameters (1.2.2) would lead to:  1 T κc κc = max  (t), γc =  (t)dt, σc = . (1.2.10) t∈[0,T ] T 0 γc Indeed, these straight definitions work most of the time, as is confirmed by the following example, although, as we shall explain soon, not always. Example 1.7 Let us consider the well-known Van der Pol’s problem, y1 = y2 , y2 = −y1 + μ y2 (1 − y12 ),

t ∈ [0, 2μ],

(1.2.11)

y(0) = (2, 0) , T

whose solution approaches a limit cycle of period T ≈ 2μ. It is also very wellknown that, the larger the parameter μ, the more difficult the problem is. In Fig. 1.2 we plot the parameter σc (μ) (as defined in (1.2.10)) for μ ranging from 0 to 103 . Clearly, stiffness increases with μ. Even though (1.2.10) works for this problem, this is not true in general. The problem is that the definition of stiffness as the ratio of two quantities may require a lower bound for the denominator. While the definition of κc remains unchanged, the definition of γc is more entangled. Actually, we need two different estimates of such a parameter:

10

L. Brugnano et al.

Fig. 1.2 Estimated stiffness ratio of Van der Pol’s problem (1.2.11)

• an upper bound, to be used for estimating the conditioning of the problem in l1 norm; • a lower bound, to be used in defining σc and, then, the stiffness. In the definition given in [2, 4], this distinction was not made, even though the definition was (qualitatively) completed by adding “for at least one of the modes”.

(1.2.12)

We shall be more precise in a moment. In the meanwhile, it is interesting to note that the clarification contained in (1.2.12) is already in one of the two definitions given by Miranker [34]: A system of differential equations is said to be stiff on the interval (0, t¯) if there exists a solution of that system a component of which has a variation on that interval which is large compared to 1t¯ , where it should be stressed that the definition considers equations and not problems: this implies that the existence of largely variable components may appear for at least one choice of the initial conditions, not necessary for a specific one. Later on, the definition was modified so as to translate into formulas the above quoted sentence (1.2.12). The following definitions were then given (see, e.g., [26]): 1 max y(t), η 0≤t≤T  T 1 y(t)dt, γc (T , η) = T η 0 κc (T , η) =

κc (T ) = max κc (T , η), η

(1.2.13)

γc (T ) = max γc (T , η) η

and σc (T ) = max η

κc (T , η) . γc (T , η)

(1.2.14)

The only major change regards the definition of σc . Let us be more clear on this point with an example, since it leads to a controversial question in the literature: i.e.,

1 Fifty Years of Stiffness

11

the dependence of stiffness from the initial condition. Let A = diag(λ1 , λ2 , . . . , λm ) with λi < 0 and |λ1 | > |λ2 | > · · · > |λm |. The solution of problem (1.2.9) is y(t) = eAt η. If σc is defined according to (1.2.10), it turns out that eAt  = eλm t and, then, γc (T ) ≈ T |λ1 m | . If, however, we take η = (1, 0, . . . , 0)T , then y(t) = eλ1 t and γc (T )

1 . Of course, by changing the initial point, one may activate becomes γc (T ) ≈ T |λ 1| each one of the modes, i.e. the functions eλi t on the diagonal of the matrix eAt , leaving silent the others. This is the reason for specifying, in the older definition, the quoted sentence (1.2.12). The new definition (1.2.14), which essentially poses as the denominator of the ratio σc the smallest value among the possible values of γc (T , η), is more compact and complies with the needs of people working on the construction of codes, who like more operative definitions. For the previous diagonal example, we have that kc continues to be equal to 1, while γc (T ) = T |λ1 1 | . Having got the new definition (1.2.14) of σc (T ), the definition of stiffness continues to be given by Definition 1.3 given in the scalar case, i.e., the problem (1.2.9) is stiff if σc (T )  1. How does this definition reconcile with the most used definition of stiffness for the linear case, which considers the “smallest” eigenvalue λm as well? The answer is already in Miranker’s definition D3. In fact, usually the integration interval is chosen large enough to provide complete information on the behavior of the solution. In this case, until the slowest mode has decayed enough, i.e. T = 1/|λm |, which implies      λ1  1 σc T = =  , (1.2.15) |λm | λm

which, when much larger than 1, coincides with the most common definition of stiffness in the linear case. However, let us insist on saying that if the interval of integration is much smaller than 1/|λm |, the problem may be not stiff even if | λλm1 |  1. The controversy about the dependence of the definition of stiffness on the initial data is better understood by considering the following equation given in [29, pp. 217–218]:        d y1 −2 1 y1 2 sin t = + , −1.999 0.999 0.999(sin t − cos t) y2 dt y2 whose general solution is         1 1 sin t y1 = c1 e−t + c2 e−0.001t + . y2 1 1.999 cos t The initial condition y(0) = (2, 3)T requires c2 = 0 and, then, the slowest mode is not activated: the solution rapidly reaches the reference solution. If this information was known beforehand, one could, in principle, choose the interval of integration 1 . This, however, does not take into account the fact that T much smaller than 0.001 the computer uses finite precision arithmetic, which may not represent exactly the initial condition η. To be more precise, let us point out that the slowest mode is not activated only if the initial condition is on the line y2 (0) − y1 (0) − 1 = 0. Any irrational value of y1 (0) will not be well represented on the computer. This is enough

12

L. Brugnano et al.

Fig. 1.3 Estimated stiffness ratio of Robertson’s problem (1.2.16)

to activate the silent mode. Of course, if one is sure that the long term contribution to the solution obtained on the computer is due to this kind of error, a small value of T can always be used. But it is rare that this information is known in advance. For this reason, we consider the problem to be stiff, since we believe that the definition of stiffness cannot distinguish, for example, between rational and irrational values of the initial conditions. Put differently, initial conditions are like a fuse that may activate stiffness. We conclude this section by providing a few examples, which show that Definition 1.3, when σc is defined according to (1.2.14), is able to adequately describe the stiffness of nonlinear and/or non autonomous problems as well. Example 1.8 Let us consider the well-known Robertson’s problem: y1 = −0.04y1 + 104 y2 y3 , y2 = 0.04y1 − 104 y2 y3 − 3 × 107 y22 , y3

t ∈ [0, T ],

= 3 × 107 y22 , T

(1.2.16)

y(0) = (1, 0, 0) .

Its stiffness ratio with respect to the length T of the integration interval, obtained through the linearized problem and considering a perturbation of the initial condition of the form (0, ε, −ε)T , is plotted in Fig. 1.3. As it is well-known, the figure confirms that for this problem stiffness increases with T . Example 1.9 Let us consider the so-called Kreiss problem [21, p. 542], a linear and non autonomous problem: y  = A(t)y,

t ∈ [0, 4π ],

y(0)

fixed,

(1.2.17)

where A(t) = QT (t)ε Q(t),

(1.2.18)

1 Fifty Years of Stiffness

13

Fig. 1.4 Estimated stiffness ratio of the Kreiss problem (1.2.17)–(1.2.19)

and

 Q(t) =

cos t − sin t

sin t cos t



 ε =

,



−1

−ε −1

(1.2.19)

.

Its stiffness ratio with respect to the small positive parameter ε, obtained by considering a perturbation of the initial condition of the form (−ε, 1)T , is plotted in Fig. 1.4. As one expects, the figure confirms that the stiffness of the problem behaves as ε −1 , as ε tends to 0. Example 1.10 Let us consider the following linear and non autonomous problem, a modification of problem (1.2.17), that we call “modified Kreiss problem”:9 y  = A(t)y,

t ∈ [0, 4π ],

y(0)

fixed,

(1.2.20)

where −1 A(t) = Q−1 ε P Qε (t), ε (t)P

and

 P=

 −1 0 , 1 1

 Qε (t) =

1

ε

esin t

esin t

(1.2.21) 

 ,

ε =

−1

 −ε −1

. (1.2.22)

Its stiffness ratio with respect to the small positive parameter ε, obtained by considering a perturbation of the initial condition of the form (−ε, 1)T , is shown in Fig. 1.5. Also in this case the stiffness of the problem behaves as ε−1 , as ε tends to 0. Remark 1.11 It is worth mentioning that, in the examples considered above, we numerically found that max η

9 This

κc (T , η) γc (T , η)

problem has been suggested by J.I. Montijano.

14

L. Brugnano et al.

Fig. 1.5 Estimated stiffness ratio of the modified Kreiss problem (1.2.20)–(1.2.22)

is obtained by considering an initial condition η in the direction of the eigenvector of the Jacobian matrix (computed for t ≈ t0 ) associated to the dominant eigenvalue. We note that, for an autonomous linear problem, if A is diagonalizable, this choice activates the mode associated with λ1 , i.e., the eigenvalue of maximum modulus of A.

1.2.3 The Non Scalar Discrete Case As for the scalar case, what we said for the continuous problems can be repeated, mutatis mutandis, for the discrete ones. For brevity, we shall skip here the details for this case, also because they can be deduced from those described in the more general case discussed in the next section.

1.3 Boundary Value Problems (BVPs) The literature about BVPs is far less abundant than that about IVPs, both in the continuous and in the discrete case. While there are countless books on the latter subject presenting it from many points of view (e.g., stability of motion, dynamical systems, bifurcation theory, etc.), there are many less books about the former. More importantly, the subject is usually presented as a by product of the theory of IVPs. This is not necessarily the best way to look at the question, even though many important results can be obtained this way. However, it may sometimes be more useful to look at the subject the other way around. Actually, the question is that IVPs are naturally a subclass of BVPs. Let us informally clarify this point without many technical details which can be found, for example, in [4]. IVPs transmit the initial information “from left to right”. Well conditioned IVPs are those for which the initial value, along with the possible initial errors, decay

1 Fifty Years of Stiffness

15

moving from left to right. FVPs (Final Value problems) are those transmitting information “from right to left” and, of course, well conditioning should hold when the time, or the corresponding independent variable, varies towards −∞. More precisely, considering the scalar test equation (1.2.1), the asymptotically stability for IVPs and FVPs requires Re λ < 0 and Re λ > 0, respectively. BVPs transmit information both ways. Consequently, they cannot be scalar problems but vectorial of dimension at least two. We need then to refer to the test equation (1.2.9). It can be affirmed that a well conditioned linear BVP needs to have eigenvalues with both negative and positive real parts (dichotomy, see, e.g., [1, 4]). More precisely: the number of eigenvalues with negative real part has to match the amount of information transmitted “from left to right”, and the number of eigenvalues with positive real part has to match the amount of information traveling “from right to left”. For brevity, we shall call the above statement continuous matching rule. Of course, if there are no final conditions, then the problem becomes an IVP and, as we have seen, in order to be well conditioned, it must have all the eigenvalues with negative real part. In other words, the generalization of the case of asymptotically stable IVPs is the class of well conditioned BVPs because both satisfy the continuous matching rule. This is exactly what we shall assume hereafter. Similar considerations apply to the discrete problems, where the role of the imaginary axis is played by the unit circumference in the complex plane. It is not surprising that a numerical method will well represent a continuous autonomous linear BVP if the corresponding matrix has as many eigenvalues inside the unit circle as the number of initial conditions and as many eigenvalues outside the unit circle as the number of final conditions (discrete matching rule). Remark 1.12 The idea that IVPs are a subset of BVPs is at the root of the class of methods called Boundary Value Methods (BVMs) which permits us, thanks to the discrete matching rule, to define high order and perfectly A-stable methods (i.e., methods having the imaginary axis separating the stable and unstable domains), which overcome the Dahlquist’s barriers, and are able to solve both IVPs and BVPs (see, e.g., [4]). Remark 1.13 From this point of view, the popular shooting method, consisting of transforming a BVP into an IVP and then applying a good method designed for IVPs, does not appear to be such a good idea. As matter of fact, even a very well conditioned linear BVP, i.e. one which satisfies the continuous matching rule, will be transformed in a badly conditioned IVP, since the matrix of the continuous IVP shall, of course, contain eigenvalues with positive real part. This will prevent the discrete matching rule to hold.

1.3.1 Stiffness for BVPs Coming back to our main question, stiffness for BVPs is now defined by generalizing the idea already discussed in the previous sections.

16

L. Brugnano et al.

As in the previous cases, we shall refer to linear problems, but the definitions will also be applicable to nonlinear problems as well. Moreover, according to what is stated above, we shall only consider the case where the problems are well conditioned (for the case of ill conditioned problems, the arguments are slightly more entangled, see e.g. [7]). Then, let us consider the linear and non autonomous BVP: y  = A(t)y,

t ∈ [0, T ],

B0 y(0) + B1 y(T ) = η,

(1.3.1)

where y(t), η ∈ Rm and A(t), B0 , B1 ∈ Rm×m . The solution of the problem (1.3.1) is y(t) = (t)Q−1 η, where (t) is the fundamental matrix of the problem such that (0) = I , and Q = Ba + Bb (T ), which has to be nonsingular, in order for (1.3.1) to be solvable.10 As in the continuous IVP case, the conditioning parameters are defined (see (1.2.13)) as: 1 max y(t), η 0≤t≤T  T 1 y(t)dt, γc (T , η) = T η 0 κc (T , η) =

κc (T ) = max κc (T , η), η

(1.3.2)

γc (T ) = max γc (T , η). η

Consequently, the stiffness ratio is defined as (see (1.2.14)): σc (T ) = max η

κc (T , η) , γc (T , η)

and the problem is stiff if σc (T )  1. Moreover, upper bounds of κc (T ) and γc (T ) are respectively given by:  1 T γc (T ) ≤  (t)Q−1 dt. (1.3.3) κc (T ) ≤ max  (t)Q−1 , 0≤t≤T T 0 Thus, the previous definitions naturally extend to BVPs the results stated for IVPs. In a similar way, when considering the discrete approximation of (1.3.1), for the sake of brevity provided by a suitable one-step method over a partition π of the interval [0, T ], with subintervals of length hi , i = 1, . . . , N , the discrete problem will be given by yn+1 = Rn yn ,

n = 0, . . . , N − 1,

whose solution is given by n−1 

yn = Ri Q−1 N η, i=0

10 Observe

B0 y0 + B1 yN = η,

QN = B0 + B1

N−1

i=0

that, in the case of IVPs, B0 = I and B1 = O, so that Q = I .

Ri .

(1.3.4)

1 Fifty Years of Stiffness

17

The corresponding discrete conditioning parameters are then defined by: 1 κd (π) = max κd (π, η), max yn , η η 0≤n≤N (1.3.5) N 1  γd (π, η) = hi max(yi , yi−1 ), γd (π) = max γd (π, η), η T η κd (π, η) =

i=1

and σd (π) = max η

κd (π, η) . γd (π, η)

According to Definition 1.6, we say that the discrete problem11 (1.3.4) well represents the continuous problem (1.3.1) if κd (π) ≈ κc (T ),

γd (π) ≈ γc (T ).

(1.3.6)

Remark 1.14 It is worth mentioning that innovative mesh-selection strategies for the efficient numerical solution of stiff BVPs have been defined by requiring the match (1.3.6) (see, e.g., [3, 4, 7, 8, 26]).

1.3.2 Singular Perturbation Problems The numerical solution of singular perturbation problems can be very difficult because they can have solutions with very narrow regions of rapid variation characterized by boundary layers, shocks, and interior layers. Usually, the equations depend on a small parameter, say ε, and the problems become more difficult as ε tends to 0. It is not always clear, however, how the width of the region of rapid variation is related to the parameter ε. By computing the stiffness ratio σc (T ), we observe that singularly perturbed problems are stiff problems. Moreover, as the following examples show, the parameter σc (T ) provides us also with information about the width of the region of rapid variation. The examples are formulated as second order equations: of course, they have to be transformed into corresponding first order systems, in order to apply the results of the previous statements. Example 1.15 Let us consider the linear singularly perturbed problem: εy  + ty  = −επ 2 cos(πt) − πt sin(πt),

y(−1) = −2,

y(1) = 0,

(1.3.7)

whose solution has, for 0 < ε  1, a turning Fig. 1.6). The exact √ point at t = 0 (see √ solution is y(t) = cos(πt) + exp((t − 1)/ ε) + exp(−(t + 1)/ ε). In Fig. 1.7 we plot an estimate of the stiffness ratio obtained by considering two different perturbations of the boundary conditions of the form (1, 0)T and (0, 1)T . 11 It

is both defined by the used method and by the considered mesh.

18

L. Brugnano et al.

Fig. 1.6 Problem (1.3.7), ε = 10−8

Fig. 1.7 Estimated stiffness ratio of problem (1.3.7)

−1 −14 The parameter ε varies √ from 10 to 10 . We see that the (estimated) stiffness parameter grows like ε −1 .

Example 1.16 Let us consider the following nonlinear problem:   π πt exp(2y) = 0, y(0) = 0, y(1) = 0. εy  + exp(y)y  − sin 2 2

(1.3.8)

This problem has a boundary layer at t = 0 (see Fig. 1.8). In Fig. 1.9 we plot an estimate of the stiffness ratio obtained by considering two different perturbations of the boundary conditions of the form (1, 0)T and (0, 1)T . The parameter ε varies from 1 to 10−8 . We see that the (estimated) stiffness parameter grows like ε −1 , as ε tends to 0. Example 1.17 Let us consider the nonlinear Troesch problem: y  = μ sinh(μy),

y(0) = 0,

y(1) = 1.

(1.3.9)

1 Fifty Years of Stiffness Fig. 1.8 Problem (1.3.8), ε = 10−6

Fig. 1.9 Estimated stiffness ratio of problem (1.3.8)

Fig. 1.10 Troesch’s problem (1.3.9), μ = 50

19

20

L. Brugnano et al.

Fig. 1.11 Estimated stiffness ratio of Troesch’s problem (1.3.9)

This problem has a boundary layer near t = 1 (see Fig. 1.10). In Fig. 1.11 we plot the estimate of the stiffness ratio obtained by considering two different perturbations of the boundary conditions of the form (1, 0)T and (0, 1)T . The parameter μ is increased from 1 to 50 and, as expected, the stiffness ratio increases as well: for μ = 50, it reaches the value 1.74 × 1012 . Acknowledgements tions.

The authors wish to thank the reviewers, for their comments and sugges-

References 1. Ascher, U.M., Mattheij, R.M.M., Russell, R.D.: Numerical Solution of Boundary Value Problems for Ordinary Differential Equations. SIAM, Philadelphia (1995) 2. Brugnano, L., Trigiante, D.: On the characterization of stiffness for ODEs. Dyn. Contin. Discrete Impuls. Syst. 2, 317–335 (1996) 3. Brugnano, L., Trigiante, D.: A new mesh selection strategy for ODEs. Appl. Numer. Math. 24, 1–21 (1997) 4. Brugnano, L., Trigiante, D.: Solving Differential Problems by Multistep Initial and Boundary Value Methods. Gordon & Breach, Amsterdam (1998) 5. Butcher, J.C.: The Numerical Analysis of Ordinary Differential Equations. Wiley, Chichester (1987) 6. Cash, J.R.: Efficient numerical methods for the solution of stiff initial-value problems and differential algebraic equations. Proc. R. Soc. Lond. A 459, 797–815 (2003) 7. Cash, J.R., Mazzia, F.: A new mesh selection algorithm, based on conditioning, for two-point boundary value codes. J. Comput. Appl. Math. 184, 362–381 (2005) 8. Cash, J.R., Sumarti, N., Mazzia, F., Trigiante, D.: The role of conditioning in mesh selection algorithms for first order systems of linear two-point boundary value problems. J. Comput. Appl. Math. 185, 212–224 (2006) 9. Corduneanu, C.: Principles of Differential and Integral Equations. Chelsea, New York (1971) 10. Crank, J., Nicolson, P.: A pratical method for numerical evaluation of solutions od partial differential equations of the heat-conduction type. Proc. Camb. Philos. Soc. 43, 50–67 (1947) 11. Curtiss, G.F., Hirshfelder, J.O.: Integration of Stiff equations. Proc. Natl. Acad. Sci. US 38, 235–243 (1952)

1 Fifty Years of Stiffness

21

12. Dahlquist, G.: Problems related to the numerical treatment of Stiff differential equations. In: Günther, E., et al. (eds.) International Computing Symposium, 1973, pp. 307–314. North Holland, Amsterdam (1974) 13. Dahlquist, G.: A special stability problem for linear multistep methods. BIT 3, 27–43 (1964) 14. Dahlquist, G.: Error analysis for a class a methods for stiff nonlinear initial value problems. In: Num. Anal., Dundee. Lect. Notes in Math., vol. 506, pp. 60–74. Springer, Berlin (1975) 15. Dahlquist, G.: On stability and error analysis for stiff nonlinear problems. Part 1. Report TritaNA-7508 (1975) 16. Dahlquist, G.: G-stability is equivalent to A-stability. BIT 18, 384–401 (1978) 17. Dahlquist, G.: 33 Years of instability, Part I. BIT 25, 188–204 (1985) 18. Galbraith, J.K.: A Short History of Financial Euphoria. Whittle Direct Book (1990) 19. Goodwin, R.H.: A growth cycle. In: Feinstein, C.H. (ed.) Socialism, Capitalism and Economic Growth. Cambridge University Press, Cambridge (1967) 20. Guglielmi, N., Hairer, E.: Stiff delay equations. Scholarpedia 2(11), 2850 (2007) 21. Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II. Springer, Berlin (1996). 2nd rev. edn 22. Hahn, W.: Stability of Motions. Springer, New York (1967) 23. Higham, D.J., Trefethen, L.N.: Stiffness of ODE. BIT 33, 286–303 (1993) 24. Hindmarsh, A.C.: On Numerical Methods for Stiff Differential Equations–Getting the Power to the People. Lawrence Livermore Laboratory report, UCRL-83259 (1979) 25. Hundsdorfer, W.H.: The numerical solution of stiff initial value problems: an analysis of one step methods. CWI Tracts 12, Amsterdam (1980) 26. Iavernaro, F., Mazzia, F., Trigiante, D.: Stability and conditioning in numerical analysis. JNAIAM 1, 91–112 (2006) 27. Lakshmikantham, V., Leela, S.: Differential and Integral Inequalities. Academic Press, New York (1969) 28. Lakshikantham, V., Trigiante, D.: Theory of Difference Equations. Numerical Methods and Applications, 2nd edn. Marcel Dekker, New York (2002) 29. Lambert, J.D.: Numerical Methods for Ordinary Differential Equations. Wiley, New York (1991) 30. Le Veque, R.J.: Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time-Dependent Problems. SIAM, Philadelphia (2007) 31. Liniger, W.: Solution Numériques des Équations Différentielle et au derivées partielle. Unpublished Lecture Notes of a course taught at Swiss Federal Institute of Technology, Lausanne, Switzerland (1972–1973) 32. Mazzia, F., Trigiante, D.: A hybrid mesh selection strategy based on conditioning for boundary value ODEs problems. Numer. Algorithms 36(2), 169–187 (2004) 33. Mazzia, F., Trigiante, D.: Efficient strategies for solving nonlinear problems in BVPs codes. Nonlinear Studies (in press) 34. Miranker, W.L.: The Computational Theory of Stiff Differential Equations. Pubblicazioni IAC Roma Ser. III N. 102 (1975) 35. Rouche, N., Mawhin, J.: Équations Differentielle Ordinaire, vol. 2. Masson et Cie, Paris (1973) 36. Shampine, L.F., Thompson, S.: Stiff systems. Scholarpedia 2(3), 2855 (2007) 37. Söderlind, G.: The logarithmic norm. History and modern theory. BIT 46, 631–652 (2006) 38. Yoshizawa, T.: Stability Theory by Liapunov’s Second Method. The Mathematical Soc. of Japan (1966)

Chapter 2

Efficient Global Methods for the Numerical Solution of Nonlinear Systems of Two Point Boundary Value Problems Jeff R. Cash and Francesca Mazzia

Abstract In this paper we will be concerned with numerical methods for the solution of nonlinear systems of two point boundary value problems in ordinary differential equations. In particular we will consider the question “which codes are currently available for solving these problems and which of these codes might we consider as being state of the art”. In answering these questions we impose the restrictions that the codes we consider should be widely available (preferably written in MATLAB and/or FORTRAN) they should have reached a fairly steady state in that they are seldom, if ever, updated, they try to achieve broadly the same aims and, of course, it is relatively inexpensive to purchase the site licence. In addition we will be concerned exclusively with so called boundary value (or global) methods so that, in particular, we will not include shooting codes or Shishkin mesh methods in our survey. Having identified such codes we go on to discuss the possibility of comparing the performance of these codes on a standard test set. Of course we recognise that the comparison of different codes can be a contentious and difficult task. However the aim of carrying out a comparison is to eliminate bad methods from consideration and to guide a potential user who has a boundary value problem to solve to the most effective way of achieving his aim. We feel that this is a very worthwhile objective to pursue. Finally we note that in this paper we include some new codes for BVP’s which are written in MATLAB. These have not been available before and allow for the first time the possibility of comparing some powerful MATLAB codes for solving boundary value problems. The introduction of these new codes is an important feature of the present paper.

Work developed within the project “Numerical methods and software for differential equations”. J.R. Cash () Department of Mathematics, Imperial College, South Kensington, London SW7, UK e-mail: [email protected] F. Mazzia Dipartimento di Matematica, Università di Bari, Via Orabona 4, 70125 Bari, Italy T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_2, © Springer Science+Business Media B.V. 2011

23

24

J.R. Cash and F. Mazzia

Keywords Two-point boundary value problems · Mesh selection · State of the art codes · Numerical comparisons Mathematics Subject Classification (2000) 65L10 · 65L06

2.1 Introduction An important task in many areas of Numerical Analysis is to carry out meaningful comparisons of numerical algorithms which attempt to achieve roughly the same well defined objectives. This is of particular interest to a user who wishes to find an appropriate code to solve his problem as well as to someone who has developed a new algorithm and wishes to compare it with existing state of the art codes. An obvious example of a particular case where such comparisons have been carried out leading to the development of codes which are very much more efficient is in the numerical solution of initial value problems of the form dy = f (x, y), dx

x ≥ a,

y(a) = ya .

(2.1.1)

The normal course of events which was followed by, for example, algorithms for initial value problems (and also by most linear algebra routines) is that first the mathematical theory behind the problems to be solved is well understood, then the mathematical theory behind the numerical algorithms is established, then the problems involved in the writing of high quality, robust codes are identified and overcome and finally some sort of numerical comparison is carried out to highlight the strengths and weaknesses of the codes. It is particularly relevant to us in this paper to focus our attention for the present on what has been done for initial value problems of the general form (2.1.1) and this we now do. Traditionally when a new code was proposed for the numerical solution of (2.1.1) the code was illustrated by comparing it, on a small set of test problems, with other codes also designed to solve (2.1.1). This proved to be less than satisfactory since there is the danger that potential disadvantages of the new code are not discovered if such limited testing is done. The first widely used test set for general initial value problems was DETEST which was proposed by Enright, Hull and Lindberg [18]. After this test set first appeared, users proposing a new method were often expected to run their code on this test set (which contains 30 problems) and this proved very useful in eliminating poor codes. However as codes became even more powerful they tended to find the problems in DETEST to be too easy. A particular cause for concern was that many of the problems in DETEST were of very small dimension and so they were often solved extremely quickly even by relatively poor codes. Following an appraisal by Shampine [35] this test set was modified to make it much more challenging but the major disadvantage still was the small dimension of the problems. A big step forward in the quality of numerical testing came with the book of Hairer and Wanner [20]. They proposed a test set which was considerably

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

25

more challenging than DETEST and, in particular, contained some problems of very large dimension (for example, the Brusselator problem on p. 151 of [20] is of dimension 32768). They also considerably advanced the methodology of testing and set new standards in attempting to make the tests as fair as possible. Based on the work of Hairer and Wanner an even more demanding test set was derived in Amsterdam by Lioen and de Swart [23]. They considerably broadened the aims and scope of the test set by adding differential algebraic equations with index ≤ 3. More recently this test set was taken over by Francesca Mazzia and her co-workers at the University of Bari [25] and it now plays a central role in providing realistic test problems for IVP solvers. It is important to realise, however, that this test set now has several additional facilities which can be extremely useful to users and which considerably strengthens the case for carrying out sophisticated numerical testing. For example, the user is easily able to run several state of the art codes either on his own test problems or on problems that appear in the test set. It also allows the user to test his own code against those appearing in the test set on really challenging problems and as output it produces highly relevant, and easy to understand, statistics. For these reasons the Bari test set now plays an important role in the development of powerful codes for the numerical solution of initial value problems of the form (1.1) and it also has several other important features available, which are much more important than was anticipated when this project was first begun. To see exactly what is available (and to note the very high level of use of this facility) the reader is urged to log into the Bari web page [25].

2.2 Boundary Value Problems Having explained what has been done in developing test sets for initial value problems of the form (2.1.1), and witnessing the central role that is taken by test sets in deriving efficient codes for initial value problems, it clearly would be a worthwhile aim to extend some of these ideas to codes for the numerical solution of two-point boundary value problems. Ideally we would like to be able to answer the question “What are the best codes currently available for solving a large class of nonlinear two-point boundary value problems and what properties of the boundary value problem need to be taken into account when deriving efficient numerical methods for its solution”. Here the situation is very different than it is for initial value problems simply because boundary value problems tend to be much harder to solve and much more diverse than initial value problems. However it could be argued that in these circumstances a user is in even more need of guidance. Due to the many forms that can be taken by two-point boundary value problems, it is clear that we will need to impose some important restrictions: firstly, on which classes of problems we will attempt to solve; secondly, on which classes of numerical methods we will consider. We need to decide for example whether or not we will consider non-separated boundary conditions, eigenvalues and other parameter dependent problems, singular problems, problems of the special form y  = f (x, y) where there is no first derivative present) and so on. What we will in fact consider in this paper is global

26

J.R. Cash and F. Mazzia

methods, i.e. not shooting methods or Shishkin methods, for first order systems of nonlinear two-point boundary value problems with separated boundary conditions. This means that we will restrict the class of problems that we are interested in to dy = f (x, y), dx

a ≤ x ≤ b,

g(y(a), y(b)) = 0.

(2.2.1)

In this paper our aim will be to identify which codes are at present suitable to be used for the numerical solution of (2.1). In order to do this, we need to reconsider the codes that were presented in a previous article [5], where one of the present authors sought to identify those codes designed for the numerical solution of two point boundary value problems of the form (2.2.1), which were in some sense efficient for the solution of this class of problems. This has naturally involved some repetition of what was considered in [5] but we feel that this is justified since it is appropriate, for the sake of completeness, for us to collect together in a single article those codes which we feel are efficient and are suitable for inclusion in a numerical comparison. Note however that all of these codes in [5] are written in FORTRAN. In due course we will hope to provide a comparison of the performance of various codes on (2.2.1) but in the present paper we will be mainly interested in identifying ‘state of the art’ codes which will be suitable for inclusion in a comparison. Some interesting thoughts concerning this can be found in [2, p. 515]. In [5] three codes were identified as having the possibility of being considered state of the art codes and these were COLNEW.f/COLSYS.f, MIRKDC.f, and TWPBVP.f/TWPBVPL.f. Also identified in [5] as being powerful continuation codes were COLMOD.f and ACDC.f [13]. The first of these two codes is based on COLSYS.f and the second is an adaptation of TWPBVPL.f. These codes allow COLSYS.f and TWPBVPL.f to be run in a continuation framework. This makes these codes much more able to deal with really difficult singular perturbation problems than is the case when continuation is not used. The important questions we need to consider in this section are: whether these codes have stabilised in the time since [5] was written; whether these new codes can still be considered as state of the art; whether new codes which are competitive with these three codes have been developed in the interim; whether more recent codes written in MATLAB are now suitable. It is important to be aware of the fact that the development of a test set for initial value problems took a lot of time, and considerable effort, and many researchers contributed to this project. We would expect the same to be the case for BVPs and so the present paper, which aims to identify which codes should have a place in a numerical comparison, should be regarded as the first step in an on going project. The first code we consider is COLSYS.f/COLNEW.f [1, 2]. This code is based on the approximation of the solution of the differential equation (2.2.1) by a piecewise polynomial and it uses collocation at Gauss points to define this polynomial uniquely. In fact, the COLSYS.f codes are applicable directly to mixed order systems (i.e., there is no need to reduce the problem to a first order system before attempting to solve it) and in what follows we describe how COLSYS.f can deal with such systems. This code was considered in [5] but for completeness we include it here. However, there is a major difference in that in [5] we considered a single high

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

27

order equation whereas here we consider a mixed order system. We consider the mixed order system of ODEs with separated boundary conditions (mi )

ui

(m1 −1)

= fi (x, u1 , . . . , u1 = fi (x, z(u)),

(md −1)

, u2 , . . . , ud

(2.2.2)

)

1 ≤ i ≤ d, a ≤ x ≤ b.

(2.2.3)

1 ≤ j ≤ m∗ ,

(2.2.4)

The boundary conditions are gj (z(u(ηj ))) = 0, where u(x) = [u1 (x), u2 (x), . . . , ud (x)]T , m∗ =

d 

mi ,

(2.2.5)

a = η1 ≤ η2 ≤ · · · ≤ ηm∗ = b,

(2.2.6)

i=1

and (m1 −1)

z(u(x)) = (u1 (x), u1 (x), . . . , u1

(m2 −1)

(md −1)

(x))T . (2.2.7) There are some important points to be noted about COLSYS.f. The first is that multipoint boundary conditions are allowed but all boundary conditions must be separated. The second point is that, as mentioned earlier, collocation codes such as COLSYS.f do not require the problem to be reduced to first order form before it can be solved. The codes COLSYS.f and COLNEW.f approximate the solution by using collocation at Gauss points. This requires (x), u2 (x), . . . , u2

u(x) ∈ C mi −1 [a, b],

(x), . . . , ud

for i = 1, 2, . . . , d.

(2.2.8)

In this approach, an approximate solution of the form uπ (x) =

M 

αj φj (x),

a ≤ x ≤ b,

(2.2.9)

j =1

is sought. Here the φj (x) are known linearly independent functions, defined on the range [a, b], and the αj are parameters that remain to be chosen. The M parameters are determined by the requirement that uπ (x) should satisfy the following M conditions: It should satisfy the m boundary conditions and it must also satisfy the ODE at M − m points in [a, b]. These M − m points are called the collocation points. A popular choice for the φj (x) is to let them be piecewise polynomial functions and this is exactly what is done in COLSYS.f. It follows that the M conditions imposed on uπ (x), to allow (2.2.9) to be uniquely defined are (1) uπ satisfies the m boundary conditions (2.2.4) (2) uπ satisfies the ODE at k points in each of the N mesh subintervals and this defines Nk + m unknowns.

28

J.R. Cash and F. Mazzia

In order that the solution should satisfy the differential equation at N k points we require that 0 = u(m) π (xij ) − f (xij , uπ (xij )),

1 ≤ j ≤ k, 1 ≤ i ≤ N.

(2.2.10)

If we define the mesh with maximum mesh size h, and we derive our collocation method so that it is based on Gauss points with s collocation points per subinterval, then the global error is uniformly O(hs ) while at the mesh points we have superconvergence and the error is O(h2s ). It is important to realise that COLSYS.f derives a continuous solution in the form of a piecewise polynomial and it also attempts to compute the error in this continuous solution. Of course, computing a continuous solution is a much more difficult task than just computing the solution at a discrete set of points as is done by several other codes. On the other hand, COLSYS.f performs the expensive task of computing a continuous solution even if the user only requires the solution to be computed at a few points. Thus, in any numerical comparison, we have to consider if the user will be provided with a continuous solution and if that is what he requires. As mentioned previously, COLSYS.f attempts to provide an estimate of the error in the continuous solution and it does this using equi-distribution. The idea is that mesh points are distributed so that an equal error is made in each mesh interval. A description of this technique can be found in [2, p. 62]. The second code highlighted in [5] was MIRKDC.f. This code is based on Mono Implicit Runge-Kutta methods [4, 11, 16] which have been widely used for the numerical solution of two point boundary value problems. These methods have the special property that they can be implemented very efficiently for boundary value problems, due to a certain explicitness appearing in the MIRK equations. In what follows, we will describe exactly what MIRK formulae attempt to achieve and we show how this is done. Following Enright and Muir [17], we rewrite MIRK schemes in the slightly different form yn+1 = yn + h

s 

bi f (xn + ci h, Yi ),

i=1

Yi = (1 − vi ) yn + vi yn+1 + h

s 

zij f (xn + cj h, Yj ), i = 1, . . . , s. (2.2.11)

j =1

This is of course just a standard implicit Runge-Kutta method which can be expressed using the celebrated Butcher notation. A convenient way of expressing this formula as an array, is to write it in the form c1 v1 z11 c2 v2 z21 .. .. .. . . .

z12 z22 .. .

... ... .. .

z1s z2s

cs vs zs1

zs2

...

zss

b1

b2

...

bs

(2.2.12)

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

29

This is written to emphasise the fact that both yn and yn+1 appear in the Runge-Kutta equations written in the form (2.2.11). The link between (2.2.12) and the standard Butcher notation is A = Z + vbT .

(2.2.13)

Having defined the Runge-Kutta method in this particular way, Enright and Muir then compute a solution of the given two point boundary value problem on a discrete set of points. Once convergence has been obtained to a discrete solution, Enright and Muir consider how this can be transformed to a continuous solution by the computation of additional function evaluations. Muir and Owren consider schemes of the form ∗

u(x) = u(xi + θ hi ) = yi + h

s 

br (θ )kr

(2.2.14)

r=1

where θ = (x − xi )/ hi ,

0 ≤ θ ≤ 1, xi ≤ x ≤ xi+1 ,

(2.2.15)

which they call CMIRK schemes [33]. Note that this defines a continuous solution on the ith subinterval. The number of stages required to compute this continuous solution is s ∗ , with s < s ∗ , and the first s stages of the interpolating polynomial (2.2.14) are the same as the first s stages of the discrete solution. The main difference between discrete MIRK schemes and continuous CMIRK schemes is that the weight coefficients of the MIRK scheme are replaced by weight polynomials in (2.2.14). Having derived a continuous solution u(x) by using the CMIRK framework (2.2.14), (2.2.15), Enright and Muir use defect control both for mesh selection and accuracy control, in order to define a continuous defect. The (continuous) defect is defined as δ(x) = u (x) − f (x, u(x))

(2.2.16)

and Enright and Muir estimate the maximum value of this defect on each subinterval. For further discussion of this defect correction approach, the reader is referred to [5, p. 13]. In summary, we should point out that COLSYS.f controls the global error while MIRKDC.f controls the defect and these two tasks are not equivalent. The theory underlying the code MIRKDC.f [17] has remained largely unchanged since the MIRKDC.f code was written. What has changed, since [17] was written, is that there has been a major update to the code rather than to the theory. Concerned by the long calling sequence of many boundary value codes (in particular calling sequences for boundary value problems tend to be much longer than for initial value problems) and worried that many potential users may be put off using standard codes because of this, Shampine and Muir wrote a MIRKDC code which they called ‘user friendly’ [37]. This new code is written in FORTRAN 90/95, it cuts down drastically on the number of calling parameters required, it extends the capabilities of the MIRKDC code to the solution of problems with unknown parameters, it is able to deal with eigenvalue problems and some singular problems, and it allows the required Jacobian matrices to be computed using numerical differences.

30

J.R. Cash and F. Mazzia

There is also a discussion explaining why it is desirable for the code to be written in FORTRAN 90/95. Also the possibility of using the code in a continuation framework is discussed. For the purpose of this paper we will consider this code to be state of the art and, for more details, the reader is referred to [37].

2.3 Deferred Correction Codes The most convenient way to describe our deferred correction approach is to write our methods in a Runge-Kutta framework. As mentioned earlier, if we are using MIRK formulae we can write our Runge-Kutta formulae in the form (2.2.11). In what follows we consider the deferred correction framework originally proposed by Fox [19]. In this approach we first need to define two Runge-Kutta methods which will be used to define our basic algorithm. The first Runge-Kutta method, which we will denote by φ, computes a cheap low order approximation to the solution of (2.2.1) while the second, denoted by ψ , computes an estimate of the local error in φ. This then allows us to define the deferred correction scheme in the basic form φ(η) = 0, φ(η) = ψ(η).

(2.3.1)

For a long time the question concerning the order of accuracy of (2.3.1) was unresolved. This problem was solved by Skeel [38] and in what follows we give his theorem. In the theorem below the first two conditions have been known for some time, it was the third condition that was elusive. However the theorem is quite intuitive if we regard the true solution as being the sum of the numerical solution and the global error. To state Skeel’s theorem consider the deferred correction scheme (2.3.1) defined on the grid π : a = x0 < x1 < · · · < xN = b.

(2.3.2)

Denote by y the restriction of the continuous solution y(x) to the final grid π . Then Skeel’s theorem says the following: Theorem Let φ be a stable numerical method and assume that the following conditions hold for the deferred correction scheme (2.3.1): η − y = O(hp ), ψ( y) − φ( y) = O(hr+p ),

(2.3.3)

ψ( w) = O(hr ), for arbitrary functions w having at least r continuous derivatives. Then if φ(η) = ψ(η) it follows that η − y = O(hr+p ).

(2.3.4)

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

31

The main problem was how to define the two operators φ and ψ. There are many ways in which the deferred correction schemes can be refined depending on the choices of φ and ψ and in what follows we use a particular form of deferred correction which was proposed by Fox [19] and later refined by Lindberg [22]. Their proposal was to consider two Runge-Kutta formulae of order i and j , respectively, where i < j . Having defined these formulae (and of course there is an extremely wide choice that we have), we consider the algorithm defined by φi (η) = 0, φi (η) = −φj (η). It is clear that the first two conditions of Skeel’s theorem are trivially satisfied if we use this deferred correction approach, with p = i and r + p = j . It is much more complicated to verify that the final condition is satisfied and this requires us to present our Runge-Kutta methods in a special way and to explain this the reader is referred to [10]. There are many ways in which these deferred correction methods can be refined. For example we could consider the deferred correction scheme φi (η) = 0, φi (η) = −φj (η),

(2.3.5)

φi (η) = −φj (η) − φk (η), where an extra symmetric Runge-Kutta formula is introduced in an attempt to define a scheme of order k where k > j . Again the order of accuracy of this scheme can be determined by Skeel’s theorem. A popular choice is to take i = 4, j = 6, k = 8. It is easy to show that if the φ are either MIRK formulae or Lobatto formulae then η , η, and η are of order 4, 6 and 8 respectively. If we use the deferred correction scheme based on MIRK methods then this defines the code TWPBVP.f and if we base our codes on Lobatto formulae then we have the code TWPBVPL.f. This leads to two widely used codes which use the basic framework (2.3.5). For an illustration of how these formulae perform on a wide range of test problems the reader is referred to the web page of one of the authors [8]. We emphasise that the schemes we have described are just some of many that can be defined in the Runge-Kutta framework. Another possibility which certainly seems worth investigating is to use methods of different orders in different parts of the mesh. Another possibility is to use fixed order but include methods of order 10 rather than stopping at order 8. An important point to note is that these schemes, which are based on Runge-Kutta methods, have the usual problem that there is no convenient error estimate available and, also, the obtained solution is a discrete one. In order to define a continuous solution, if it is needed, we have to derive a suitable interpolant and as an estimate of the error we compute φi (η) − φi (η).

(2.3.6)

32

J.R. Cash and F. Mazzia

This gives us an error estimate in the j th order solution η and this is used for grid refinement [39]. There is also an important difference in the way in which the mesh is formed when using iterated deferred correction methods. As mentioned earlier, COLSYS.f uses equi-distribution and this results in mesh points ‘sliding about’. The deferred correction methods discussed in [5], namely TWPBVPL.f which is based on Lobatto formulae and TWPBVP.f which is based on MIRK formulae, either add in or take out mesh points depending on the error estimate and this results in a lot of the mesh points not moving during a grid refinement. These codes are on the authors web page and can be considered as being state of the art codes. In particular, the code TWPBVP.f tends to be very effective for non-stiff problems while TWPBVPL.f is particularly efficient for stiff problems. The main reason for this behaviour is that the Lobatto codes, which use implicit deferred corrections, are much more reliable than the codes based on MIRK formulae which use explicit deferred corrections [12]. An important way in which deferred correction codes have changed since TWPBVP and TWPBVPL were written is that they now have the option of taking into account the conditioning of the problem. The importance of having this facility was demonstrated in a paper of Shampine and Muir [36] who wanted to test out the diagnostic capabilities of the two-point boundary value code BVP4C. They did this by considering Bratu’s problem: y  = λ exp(y), y(0) = y(1) = 0.

(2.3.7)

Davis [14] has shown that if 0 ≤ λ < λ∗ = 3.51383 . . . then there are 2 solutions to this problem and both are parabolic and are concave down in nature. If λ = λ∗ then there is just one solution and for λ > λ∗ there are no solutions. In their numerical experiments Shampine and Muir first described the performance of the code for λ = 3.45 and plotted one of the solutions which they obtained in a perfectly satisfactory way. However they did note that their estimate of the conditioning constant was quite high (the error tolerance imposed was 10−3 ) and the estimate of the conditioning constant was 3.14 × 103 . The code found this problem so easy that the solution was obtained on the initial mesh of 10 points. Having done this, Shampine and Muir solved Bratu’s problem with λ = 3.55 for which the problem has no solution. The expectation was that BVP4C would fail to find a solution and would send this message back to the user. However the code did not do this, instead it returned a solution which had all the characteristics of being parabolic in nature and concave downwards. The solver provided no warning message. However the code did need 179 points to compute the solution and the estimate of the conditioning constant was 106 which can be considered as being very large given the precision that is required. The reason for this poor performance of BVP4C is not hard to understand. The problem arises from the fact that the solution is obtained, and the mesh is refined, using a local error estimate. We do this on the assumption that if the local error estimate is sufficiently small, then the global error will also

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

33

be small. However a backward error analysis shows that this may not be the case if the problem is ill conditioned. All of this makes a very powerful case, when solving two point boundary value problems, for computing the conditioning constant of the problem as well as the solution. Without a conditioning estimate we are not able to have any confidence in a solution that we compute based on local errors and we feel that this is a really important problem which requires considerable further investigation. Early work on the estimation of conditioning for boundary value problems was carried out by Trigiante, Brugnano and Mazzia [3, 28]. The approach of these researchers was rather different from that of Shampine and Muir. The Shampine Muir approach on the detection of poor conditioning was to warn the user of this and to warn that there may be a severe loss of correct digits in the solution computed. In contrast, in a recent series of papers [6, 7] Cash, Mazzia and their co-workers derived efficient algorithms for estimating the condition numbers of BVPs and they developed a mesh choosing algorithm which takes into account the conditioning of the problem. In this approach the aim is to choose the mesh so that a local error tolerance is satisfied and also so that the problem remains well conditioned. This led to the derivation of codes TWPBVPLC.f and TWPBVPC.f which are based on Lobatto and MIRK methods respectively and which give an estimate of the conditioning of the problem along with the numerical solution computed by the code. The existence of these new codes gives the user the option of taking into account the conditioning of the problem and, if this option is used, the original codes TWPBVP.f and TWPBVPL.f require the change of just one input parameter. Extensive numerical testing on these four deferred correction codes appear on the web page of one of the authors [8]. What is found, for most problems where conditioning is not an issue, is that there is little difference between the performance of TWPBVP.f/TWPBVPC.f and TWPBVPL.f/TWPBVPLC.f. However for problems where conditioning is important there is often a considerable gain in efficiency if the codes TWPBVPC.f and TWPBVPLC.f are used. Finally, another important feature, is that the estimate of the conditioning constant can be used in a backward error analysis to ensure that the final solution, which is based on local error estimation, gives a solution with a satisfactory global error. Although considerable advances have been made in the computation of conditioning constants we feel that there is still more work to be done. However the approach of Trigiante, Brugnano, Cash and Mazzia which chooses the mesh so that a local error tolerance is satisfied and so that the conditioning of the continuous and discrete problems remains roughly the same is a very powerful one.

2.4 Boundary Value Methods In this section we describe a class of methods known as Boundary Value Methods (BVMs) [3]. These are linear multistep methods used in a special way that allows us to generate stable discrete boundary values schemes. In the case of symmetric schemes, given the grid π defined by (2.3.2) with hi = xi − xi−1 , 1 ≤ i ≤ N , the numerical scheme generated by a k-step BVM is defined by the following

34

J.R. Cash and F. Mazzia

equations: ⎧ g(y0 , yN ) = 0, ⎪ ⎪ ⎪ k−i (i) k−i (i) ⎪ ⎪ ⎪ j =−i αj +i yi+j = hi j =−i βj +i fi+j , ⎪ ⎪ ⎪ − 1 i = 1, . . . , k ⎪ 1 ⎪ ⎪ ⎪ (additional initial methods), ⎪ ⎪ ⎪ k2 ⎨ k2 (i) (i) j =−k1 αj +k1 yi+j = hi j =−k1 βj +k1 fi+j , ⎪ i = k1 , . . . , N − k2 ⎪ ⎪ ⎪ ⎪ (main methods), ⎪ ⎪ ⎪  N−i ⎪ (i) (i) ⎪ ⎪ j =N−i−k αj −N+i+k yi+j = hi N−i j =N −i−k βj −N +i+k fi+j , ⎪ ⎪ ⎪ ⎪ i = N + 1 − k2 , . . . , N ⎪ ⎩ (additional final methods),

(2.4.1)

where yi is the approximation of y(xi ), fi = f (xi , yi ), k is odd, k1 = (k + 1)/2, (i) (i) (i) (i) k2 = k − k1 and α (i) = (α0 , . . . , αk )T and β (i) = (β0 , . . . , βk )T are the coefficient vectors characterising the method. The so called “additional initial methods” and “additional final methods” i.e. k1 − 1 initial and k2 final equations, are derived by using appropriate discretisation schemes. The main code currently available, which implements these methods, is known as TOM. This is a general purpose code for the solution of BVPs and is rather different from other codes that have been derived. The first release of the code was written in MATLAB in 2003 [24], and was based on a class of symmetric Boundary Value Methods (BVMs) [3]. The Top Order Methods (TOM) are k-step linear multistep methods with the highest possible order 2k and in the code TOM we use k = 3 to give a sixth order method. A complete description of these methods, along with their stability properties, can be found in [3, Sect. 7.4]. Recently the code has been updated by implementing another class of BVMs, namely the BS linear multistep methods [30–32]. The new version of the code is available in MATLAB. The BS methods are derived by imposing the restriction that the numerical solution of the general k-step linear multistep formula is the same as is given by the collocation procedure using the B-spline basis. The coefficients are computed by solving special linear systems. The k-step BS scheme provides a C k continuous solution that is kth-order accurate uniformly in [a, b] and collocating the differential equation at the mesh points [32]. In order to introduce the continuous extension, some further notation is needed. First, we represent any spline function s of degree d = (k + 1) and knot xi , i = 0, . . . , N using the associated (dthdegree) B-spline basis Bj (x), j = −d, . . . , N − 1. This can be defined after prescribing two additional sets of d knots, {xi , i = −d, . . . , −1} (left auxiliary knots), with x−d ≤ · · · ≤ x0 , and {xi , i = N + 1, . . . , N + d} (right auxiliary knots), with xN ≤ xN+1 ≤ · · · ≤ xN+d [15]. Using this notation, the spline collocating the differential equation can be represented as follows, (y) = Q(BS) d

N−1  j =−d

μ(BS) (y)Bj , j

(2.4.2)

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

35

(BS)

where μj (y) are linear combinations of the values of yi and fi , i = 0, . . . , N in π . We note that, if the values yi and fi have been computed with a different scheme, the continuous approximation is the quasi-interpolation spline described in [26]. This means that this continuous extension could safely be used for any discretisation method. The main feature of the code is that it implements a hybrid mesh selection strategy based on both the conditioning parameters of the problem and on a suitable approximation of the local error. This strategy was introduced in [28] and was first implemented in the code TOM. Subsequently it was modified in [6, 7] for use in TWPBVPC and TWPBVPLC. As is the case for the codes based on deferred correction, by changing just one input parameter, it is possible for the mesh selection to use a standard strategy based only on the local error for the mesh selection. Instead of using a damped Newton method for the solution of the nonlinear equations, the code TOM implements a quasi-linearisation technique [27, 29, 32]. This means that a sequence of continuous linear BVPs is solved to a suitable tolerance. This allows us to use very efficiently the mesh selection based on conditioning for non linear problems as well as linear ones. We note that the conditioning parameters estimated by the code can be defined both for the continuous problem and for the discrete one giving the possibility to measure the reliability of the discrete problem with respect to the continuous one. In other words, we must be suspicious if the discrete problem provides parameters which are very different from the continuous ones, and this could be checked if the conditioning parameters do not converge. Since such parameters, for the discrete problem, depend on the chosen mesh, it is possible, for a fixed method, to vary the latter in order to be sure that the discrete parameters converge. This allows us, for example, to recognise if the solution has two or more time scales associated with it. This is the idea on which the mesh strategy is based. Since the BS are collocation methods, the code provides as output a continuous solution, which could be used to estimate the error at any chosen point, or a set of discrete points, if, for example, the user requires only the solution at the mesh points. A continuous solution is also computed, if needed, when using the TOM code, by the quasi-interpolation technique based on the BS methods described in [26].

2.5 Interpolation As mentioned earlier, the deferred correction codes TWPBVP.f and TWPBVPL.f are the only ones described in this paper which do not seek to provide a continuous solution. The obvious way of getting around this problem is to derive an a posteriori interpolant which can be computed using just a few extra function evaluations. This is rather similar to what the code MIRKDC does, although this code needs a continuous solution from the start so that it can estimate the defect. Here a discrete solution is computed first of all and this is accepted once the defect is sufficiently small. After the discrete solution has been accepted a continuous MIRK scheme is

36

J.R. Cash and F. Mazzia

formed using an approach of Muir and Owren [33]. Note that, if a continuous solution is not required, for example the solution may be required only at a few mesh points, then this interpolant is not needed for error estimation. Numerical experience has shown that it is advisable to compute an interpolant using data from only one sub-interval, and this avoids problems at the end of the mesh. If the interpolant is a local one, that is it is defined over a single interval, then the interpolant is identical over each sub-interval and it also has the desirable property that it is symmetric. The problem of deriving high order interpolants for MIRK formulae has recently been considered in some detail. It has been shown in [9] that a sixth order MIRK interpolant can be computed using just one extra function evaluation. To compute an eighth order MIRK interpolant, we need four extra function evaluations and the way in which this is done is described in [9]. By deriving an a posteriori interpolant, we are effectively introducing a continuous solution and this allows a fair comparison with other codes discussed in this paper to be carried out.

2.6 Conclusion The purpose of the present paper was to answer the question ‘Which codes for the numerical solution of two point boundary value problems of the form (2.2.1) can be considered as being state of the art’. The codes identified as falling into this class were based on: collocation methods (COLSYS/COLNEW); defect control methods based on MIRK formulae (MIRKDC); deferred correction methods (TWPBVP/TWPBVPL) and boundary value methods (TOM). There is considerable numerical evidence to suggest that these codes are amongst the most efficient global methods currently available. In addition these codes have reached what we have called a “steady state” and this makes them ideal for use in a numerical comparison. However there are two important points that we need to bear in mind. Firstly, these codes attempt to solve slightly different problems. In particular COLSYS/COLNEW and TOM attempt to define a continuous solution by computing the error in a polynomial approximation to the solution. MIRKDC also seeks to provide a continuous solution but controls the error in the defect. In contrast the deferred correction codes compute a discrete solution initialy and the continuous solution is obtained using an a posteriori interpolant. The second point to note is that it is important to take into account the conditioning of the problem when solving a problem of the form (2.2.1). A standard backward error analysis which links the global error to the local one relies on the problem being well conditioned and, if it is not, a solution which has not got the required accuracy may be accepted. A considerable amount of effort has been applied to the estimation of the conditioning of a problem and the reader is referred to [3, 6, 7, 28]. The boundary vale methods and deferred correction codes allow conditioning estimation and an estimate of the conditioning can be obtained by changing just one input parameter for these codes. We expect the estimation of the conditioning of a problem to become a very important topic in the future. In addition we feel that the

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

37

four FORTRAN codes we have discussed are a very firm basis for carrying out numerical comparisons of different methods and we hope to be able to investigate this in the near future. Finally, we list those codes in FORTRAN and MATLAB which would be candidates for use in a numerical comparison. We feel it appropriate to divide the codes into two catagories, namely those which are written in FORTRAN and those which are written in MATLAB. Generally speaking, if run time is an issue then the user should probably be using a FORTRAN code. If it is not, then the user may find a MATLAB code to be more convenient. The codes that we feel either are or will become state of the art codes are • FORTRAN codes: TWPBVPC, TWPBVPLC, ACDC, COLNEW, COLMOD, MIRKDC, BVP_SOLVER. • MATLAB codes: BVP4C, BVP5C ([21]), TOM, TWPBVPC, TWPBVPLC. Note that the MATLAB codes TWPBVPC/TWPBVPLC have only just been released but are based on widely used codes. We finish this section with the observation that codes written in languages other than FORTRAN or MATLAB are starting to appear. As an example we mention the code TWPBVP which is now available in R (package bvpSolve) [34].

References 1. Ascher, U., Christiansen, J., Russell, R.D.: Collocation software for boundary-value odes. ACM Trans. Math. Softw. 7(2), 209–222 (1981) 2. Ascher, U.M., Mattheij, R.M.M., Russell, R.D.: Numerical Solution of Boundary Value Problems for Ordinary Differential Equations. Classics in Applied Mathematics, vol. 13. SIAM, Philadelphia (1995). Corrected reprint of the 1988 original 3. Brugnano, L., Trigiante, D.: Solving Differential Problems by Multistep Initial and Boundary Value Methods. Stability and Control: Theory, Methods and Applications, vol. 6. Gordon and Breach, Amsterdam (1998) 4. Cash, J.R.: A class of implicit Runge-Kutta methods for the numerical integration of stiff ordinary differential equations. J. ACM 22(4), 504–511 (1975) 5. Cash, J.R.: A survey of some global methods for solving two-point BVPs. Appl. Numer. Anal. Comput. Math. 1(1–2), 7–17 (2004) 6. Cash, J.R., Mazzia, F.: A new mesh selection algorithm, based on conditioning, for two-point boundary value codes. J. Comput. Appl. Math. 184(2), 362–381 (2005) 7. Cash, J.R., Mazzia, F.: Hybrid mesh selection algorithms based on conditioning for two-point boundary value problems. J. Numer. Anal. Ind. Appl. Math. 1(1), 81–90 (2006) 8. Cash, J.R., Mazzia, F.: Algorithms for the solution of two-point boundary value problems. http://www.ma.ic.ac.uk/jcash/BVP_software/twpbvp.php 9. Cash, J.R., Moore, D.R.: High-order interpolants for solutions of two-point boundary value problems using MIRK methods. Comput. Math. Appl. 48(10–11), 1749–1763 (2004) 10. Cash, J.R., Silva, H.H.M.: Iterated deferred correction for linear two-point boundary value problems. Comput. Appl. Math. 15(1), 55–75 (1996) 11. Cash, J.R., Singhal, A.: High order methods for the numerical solution of two-point boundary value problems. BIT 22(2), 184–199 (1982) 12. Cash, J.R., Wright, M.H.: A deferred correction method for nonlinear two-point boundary value problems: implementation and numerical evaluation. SIAM J. Sci. Stat. Comput. 12(4), 971–989 (1991)

38

J.R. Cash and F. Mazzia

13. Cash, J.R., Moore, G., Wright, R.W.: An automatic continuation strategy for the solution of singularly perturbed linear two-point boundary value problems. J. Comput. Phys. 122(2), 266– 279 (1995) 14. Davis, H.T.: Introduction to Nonlinear Differential and Integral Equations. Dover, New York (1962) 15. de Boor, C.: A Practical Guide to Splines. Applied Mathematical Sciences, vol. 27. Springer, New York (2001). Revised edition 16. Enright, W.H., Muir, P.H.: Efficient classes of Runge-Kutta methods for two-point boundary value problems. Computing 37(4), 315–334 (1986) 17. Enright, W.H., Muir, P.H.: Runge-Kutta software with defect control for boundary value ODEs. SIAM J. Sci. Comput. 17(2), 479–497 (1996) 18. Enright, W.H., Hull, T.E., Lindberg, B.: Comparing numerical methods for stiff systems of O.D.Es. BIT 15(2), 10–48 (1975) 19. Fox, L.: The Numerical Solution of Two-Point Boundary Problems in Ordinary Differential Equations. Oxford University Press, New York (1957) 20. Hairer, E., Wanner, G.: Solving Ordinary Differential Equations. II. Springer Series in Computational Mathematics, vol. 14. Springer, Berlin (1991). Stiff and differential-algebraic problems 21. Kierzenka, J., Shampine, L.F.: A BVP solver that controls residual and error. J. Numer. Anal. Ind. Appl. Math. 3(1–2), 27–41 (2008) 22. Lindberg, B.: Error estimation and iterative improvement for discretization algorithms. BIT 20(4), 486–500 (1980) 23. Lioen, W.M., de Swart, J.J.B.: Test set for IVP solvers. Technical Report MAS-R9832. CWI, Amsterdam (1998) 24. Mazzia, F.: Software for boundary value problems. Department of Mathematics, University of Bari and INdAM, Research Unit of Bari, February 2003. Available at http://www.dm.uniba.it/mazzia/bvp/index.html 25. Mazzia, F., Magherini, C.: Test set for initial value problem solvers, release 2.4. Department of Mathematics, University of Bari and INdAM, Research Unit of Bari, February 2008. Available at http://www.dm.uniba.it/testset 26. Mazzia, F., Sestini, A.: The BS class of hermite spline quasi-interpolants on nonuniform knot distributions. BIT 49(3), 611–628 (2009) 27. Mazzia, F., Sgura, I.: Numerical approximation of nonlinear BVPs by means of BVMs. Appl. Numer. Math. 42(1–3), 337–352 (2002). Ninth Seminar on Numerical Solution of Differential and Differential-Algebraic Equations (Halle, 2000) 28. Mazzia, F., Trigiante, D.: A hybrid mesh selection strategy based on conditioning for boundary value ODE problems. Numer. Algorithms 36(2), 169–187 (2004) 29. Mazzia, F., Trigiante, D.: Efficient strategies for solving nonlinear problems in bvps codes. Nonlinear Stud. in press 30. Mazzia, F., Sestini, A., Trigiante, D.: B-spline linear multistep methods and their continuous extensions. SIAM J. Numer. Anal. 44(5), 1954–1973 (2006) (electronic) 31. Mazzia, F., Sestini, A., Trigiante, D.: BS linear multistep methods on non-uniform meshes. J. Numer. Anal. Ind. Appl. Math. 1(1), 131–144 (2006) 32. Mazzia, F., Sestini, A., Trigiante, D.: The continous extension of the B-spline linear multistep metods for BVPs on non-uniform meshes. Appl. Numer. Math. 59(3–4), 723–738 (2009) 33. Muir, P., Owren, B.: Order barriers and characterizations for continuous mono-implicit RungeKutta schemes. Math. Comput. 61(204), 675–699 (1993) 34. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2009). ISBN 3-900051-07-0 35. Shampine, L.F.: Evaluation of a test set for stiff ode solvers. ACM Trans. Math. Softw. 7(4), 409–420 (1981) 36. Shampine, L.F., Muir, P.H.: Estimating conditioning of BVPs for ODEs. Math. Comput. Model. 40(11–12), 1309–1321 (2004) 37. Shampine, L.F., Muir, P.H., Xu, H.: A user-friendly Fortran BVP solver. J. Numer. Anal. Ind. Appl. Math. 1(2), 201–217 (2006)

2 Efficient Global Methods for the Numerical Solution of Nonlinear Systems

39

38. Skeel, R.D.: A theoretical framework for proving accuracy results for deferred corrections. SIAM J. Numer. Anal. 19(1), 171–196 (1982) 39. Wright, R., Cash, J., Moore, G.: Mesh selection for stiff two-point boundary value problems. Numer. Algorithms 7(2–4), 205–224 (1994)

Chapter 3

Advances on Collocation Based Numerical Methods for Ordinary Differential Equations and Volterra Integral Equations Dajana Conte, Raffaele D’Ambrosio, and Beatrice Paternoster

Abstract We present a survey on collocation based methods for the numerical integration of Ordinary Differential Equations (ODEs) and Volterra Integral Equations (VIEs), starting from the classical collocation methods, to arrive to the most important modifications appeared in the literature, also considering the multistep case and the usage of basis of functions other than polynomials.

Keywords Collocation · Two-step collocation · Runge–Kutta methods · Two-step Runge–Kutta methods · Mixed collocation Mathematics Subject Classification (2000) 65L05 · 65L60 · 65R20

3.1 Introduction Collocation is a widely applied and powerful technique in the construction of numerical methods for ODEs and VIEs. As it is well known, a collocation method is based on the idea of approximating the exact solution of a given functional equation with a suitable approximant belonging to a chosen finite dimensional space, usually a piecewise algebraic polynomial, which exactly satisfies the equation on a certain subset of the integration interval (i.e. the set of the so-called collocation points). D. Conte · R. D’Ambrosio · B. Paternoster () Dipartimento di Matematica e Informatica, Università di Salerno, Fisciano, Italy e-mail: [email protected] D. Conte e-mail: [email protected] R. D’Ambrosio e-mail: [email protected] T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_3, © Springer Science+Business Media B.V. 2011

41

42

D. Conte et al.

This technique, when applied to problems based on functional equations, allows the derivation of methods having many desirable properties. In fact, collocation methods provide an approximation over the entire integration interval to the solution of the equation. Moreover, the collocation function can be expressed as a linear combination of functions ad hoc for the problem we are integrating, in order to better reproduce the qualitative behavior of the solution. The systematic study of collocation methods for initial value problems in ODEs, VIEs, and Volterra integro-differential equations (VIDEs) has its origin, respectively, in the late ’60, the early ’70 and the early ’80s. The idea of multistep collocation was first introduced by Lie and Norsett in [58], and further extended and investigated by several authors [12, 24–26, 28, 31, 34–36, 38, 42, 57, 62]. Multistep collocation methods depend on more parameters than classical ones, without any significant increase in the computational cost, by regarding them as special case of multistep Runge–Kutta methods: therefore, there are much more degrees of freedom to be spent in order to obtain strong stability properties and an higher order and stage order of convergence. As a direct consequence the effective order of multistep collocation methods is generally higher with respect to one step collocation methods with the same number of stages. Moreover, as they generally have high stage order, they do not suffer from the order reduction phenomenon (see [11, 43]), which occurs in the integration of stiff systems. The purpose of this paper is to present a review of recently introduced families of collocation and modified collocation methods for ODEs and VIEs. In particular we aim to present the main results obtained in the context of multistep collocation and almost collocation methods, i.e. methods obtained by relaxing some collocation and/or interpolation conditions in order to obtain desirable stability properties. The paper is organized as follows: Sect. 3.2 reviews the main results concerning classical one-step and multistep collocation methods for ODEs and their recent extensions and modifications; Sect. 3.3 is dedicated to collocation methods for second order initial value problems and also collocation methods based on functional basis other than polynomials; in Sect. 3.4 we consider the evolution of the collocation technique for Volterra integral equations.

3.2 Collocation Based Methods for First Order ODEs In this section we focus our attention on the hystorical background and more recent results concerning the collocation technique, its modifications and extensions for the derivation of highly stable continuous methods for the numerical solution of initial value problems based on first order ODEs   y (x) = f (x, y(x)), x ∈ [x0 , X], (3.2.1) y(x0 ) = y0 ∈ Rd , with f : [x0 , X] × Rd → Rd . It is assumed that the function f is sufficiently smooth, in such a way that the problem (3.2.1) is well-posed.

3 Advances on Collocation Based Numerical Methods for Ordinary

43

3.2.1 Classical One-Step Collocation Methods Let us suppose that the integration interval [x0 , X] is discretized in an uniform grid x0 < x1 < · · · < xN = X. Classical collocation methods (see [6, 10, 11, 44, 45, 55, 78]) are determined by means of a continuous approximant, generally an algebraic polynomial P (x), satisfying some opportune conditions: in order to advance from xn to xn+1 , the polynomial P (x) interpolates the numerical solution in xn , and exactly satisfies the ODE (3.2.1)—i.e. co-locates—in the set of points {xn + ci h, i = 1, 2, . . . , m}, where c1 , c2 , . . . , cm are m real numbers (named collocation nodes), that is  P (xn ) = yn , (3.2.2) P  (xn + ci h) = f (xn + ci h, P (xn + ci h)), i = 1, 2, . . . , m. The solution in xn+1 can then be computed from the function evaluation yn+1 = P (xn+1 ).

(3.2.3)

The classical framework in which collocation methods must be placed is certainly constituted by implicit Runge–Kutta methods (IRK). In fact, Guillou and Soule in [42] and Wright in [78] independently proved that one step collocation methods form a subset of implicit Runge–Kutta methods yn+1 = yn + h

m 

bi f (xn + ci h, Yi ),

(3.2.4)

i=1

Yi = yn + h

m 

aij f (xn + cj h, Yj ),

i = 1, 2, . . . , m,

(3.2.5)

i, j = 1, 2, . . . , m

(3.2.6)

j =1

where  aij =



ci

Lj (s)ds, 0

bj =

1

Lj (s)ds, 0

and Lj (s), j = 1, . . . , m, are fundamental Lagrange polynomials. The maximum attainable order of such methods is at most 2m, and it is obtained by using Gaussian collocation points [45, 55]. Anyway, unfortunately, the order 2m is gained only at the mesh points: the uniform order of convergence over the entire integration interval is only m. As a consequence, they suffer from order reduction showing effective order equal to m (see [10, 11, 43, 45]). Butcher (see [10] and references therein) gave an interesting characterization of collocation methods in terms of easy algebraic conditions, and analogous results are also reported in [45, 55]. This characterization, together with many other several results regarding the main properties of collocation methods, comes out as natural consequence of an interesting interpretation of collocation methods in terms of

44

D. Conte et al.

quadrature formulae. In fact, if f (x, y(x)) = f (x), (3.2.4)–(3.2.5) can be respec x +c h  x +h tively interpreted as quadrature formulae for xnn f (x)dx and xnn i f (x)dx, for i = 1, 2, . . . , m. We next consider the following linear systems A(q) :

m  j =1

B(p) :

m  i=1

aij cjk−1 =

cik , k

1 bi cik−1 = , k

k = 1, 2, . . . , q, i = 1, 2, . . . , m,

k = 1, 2, . . . , p.

(3.2.7)

(3.2.8)

Then, the following result holds (see [44, 55]): Theorem 3.1 If the condition B(p) holds for some p ≥ m, then the collocation method (3.2.2) has order p. As a consequence, a collocation method has the same order of the underlying quadrature formula (see [44], p. 28). Finally, the following result characterizing classical collocation methods arises (see [10, 44, 45, 55]). Theorem 3.2 An implicit m-stage Runge–Kutta method, satisfying B(m) and having distinct collocation abscissae, is a collocation method if and only if conditions A(m) holds. The most used collocation methods are those based on the zeros of some orthogonal polynomials, that is Gauss, Radau, Lobatto [10, 11, 43, 45, 55], having respectively order of convergence 2m, 2m − 1, 2m − 2, where m is the number of collocation points (or the number of stages, regarding the collocation method as an implicit Runge–Kutta). Concerning their stability properties, it is known that Runge–Kutta methods based on Gaussian collocation points are A-stable, while the ones based on Radau IIA points are L-stable and, moreover, they are also both algebraically stable (see [11, 43, 49] and references therein contained); Runge–Kutta methods based on Lobatto IIIA collocation points, instead, are A-stable but they are not algebraically stable (see [10, 44, 45, 55]).

3.2.2 Perturbed Collocation As remarked by Hairer and Wanner in [43], only some IRK methods are of collocation type, i.e. Gauss, Radau IIA, and Lobatto IIIA methods. An extension of the collocation idea, the so-called perturbed collocation is due to Norsett and Wanner (see [63, 64]), which applies to all IRK methods.

3 Advances on Collocation Based Numerical Methods for Ordinary

45

We denote by m the linear space of polynomials of degree at most m and consider the polynomial Nj ∈ m defined by 1  (pij − δij )x i , j! m

Nj (x) =

j = 1, 2, . . . , m,

i=0

where δij is the usual Kronecker delta. We next define the perturbation operator Px0 ,h : m → m by (Px0 ,h u)(x) = u(x) +

n  j =1

 Nj

 x − x0 (j ) u (x0 )hj . h

Next, the following definition is given (see [63, 64]). Definition 3.3 Let c1 , . . . , cm be given distinct collocation points. Then the corresponding perturbed collocation method is defined by u(x0 ) = y0 ,

u ∈ m ,

u (x0 + ci h) = f (x0 + ci h, (Px0 ,h u)(x0 + ci h)),

i = 1, 2, . . . , m,

y1 = u(x0 + h). As the authors remark in [64], if all Nj ’s are identically zero, then Px0 ,h is the identical map and the definition coincides with classical collocation. In the same paper the authors provide the equivalence result between the family of perturbed collocation methods and Runge–Kutta methods (see [64]). The interest of this results, as again is stated in [64], is that the properties of collocation methods, especially in terms of order, linear and nonlinear stability, can be derived in a reasonable short, natural and very elegant way, while it is known that, in general, these properties are very difficult to handle and investigate outside collocation.

3.2.3 Discontinuous Collocation In the literature, perturbed collocation has been considered as a modification of the classical collocation technique, in such a way that much more Runge–Kutta methods could be regarded as perturbed collocation based methods, rather than classically collocation based. There are other possible extensions of the collocation idea, which apply to wider classes of Runge–Kutta methods, such as the so-called discontinuous collocation (see [44]). Definition 3.4 Let c2 , . . . , cm−1 be distinct real numbers (usually between 0 and 1), and let b1 , bm be two arbitrary real numbers. The corresponding discontinuous method is then defined via a polynomial of degree m − 2 satisfying u(x0 ) = y0 − hb1 (u(x ˙ 0 ) − f (x0 , u(x0 )),

46

D. Conte et al.

u(x ˙ 0 + ci h) = f (x0 + ci h, u(x0 + ci h)),

i = 2, 3, . . . , m − 1,

y1 = u(x1 ) − hbs (u(x ˙ 1 ) − f (x1 , u(x1 )). Discontinuous collocation methods fall inside a large class of implicit Runge– Kutta methods, as stated by the following result (see [44]). Theorem 3.5 The discontinuous collocation method given in Definition 3.4 is equivalent to an m-stage Runge–Kutta method with coefficients determined by c1 = 0, cm = 1 and ai1 = b1 ,

aim = 0,

i = 1, 2, . . . , m,

while the other coefficients result as solutions of the linear systems A(m − 2) and B(m − 2) defined in (3.2.7) and (3.2.8). As a consequence of this result, if b1 = 0 and bm = 0, then the discontinuous collocation method in Definition 3.4 is equivalent to the (m − 2)-collocation method based on c2 , . . . , cm−1 . An interesting example of implicit Runge–Kutta method which is not collocation based but is of discontinuous collocation type is the Lobatto IIIB method (see [10, 44, 45, 55]), which plays an important role in the context of geometric numerical integration, together with Lobatto IIIA method (see [44], p. 33). They are both nonsymplectic methods (see Theorem 4.3 in [44]) but, considered as a pair, the resulting method is symplectic. This is a nice example of methods which possess very strong properties, but are difficult to investigate as discrete scheme (they cannot be studied as collocation methods, because they are not both collocation based); however, re-casted as discontinuous collocation based methods, their analysis is reasonably simplified and very elegant [44].

3.2.4 Multistep Collocation The successive results which appeared in literature (see [22, 42, 43, 57, 58]) have been devoted to the construction of multistep collocation methods. Guillou and Soulé introduced multistep collocation methods [42], by adding interpolation conditions in the previous k step points, so that the collocation polynomial is defined by  i = 0, 1, . . . , k − 1, P (xn−i ) = yn−i , (3.2.9) P  (xn + cj h) = f (xn + cj h, P (xn + cj h)), j = 1, 2, . . . , m. The numerical solution is given, as usual by yn+1 = P (xn+1 ).

(3.2.10)

3 Advances on Collocation Based Numerical Methods for Ordinary

47

Hairer–Wanner [43] and Lie–Norsett [58] derived different strategies to obtain multistep collocation methods. In [43] the Hermite problem with incomplete data (3.2.9) is solved by means of the introduction of a generalized Lagrange basis {ϕi (s), ψj (s), i = 1, 2, . . . , k, j = 1, 2, . . . , m} and, correspondingly, the collocation polynomial is expressed as linear combination of this set of functions, i.e. P (xn + sh) =

k 

ϕi (s)yn−k+i + h

i=1

s 

ψi (s)P  (xn + ci h),

i=1

n where s = x−x h . Therefore, the problem (3.2.9) is transformed in the problem of deriving {ϕi , ψj , i = 1, 2, . . . , k, j = 1, 2, . . . , m} in such a way that the corresponding polynomial P (xn + sh) satisfies the conditions (3.2.9). Lie–Norsett in [58] completely characterized multistep collocation methods, giving the expressions of the coefficients of collocation based multistep Runge–Kutta methods in closed form, as stated by the following

Theorem 3.6 The multistep collocation method (3.2.9)–(3.2.10) is equivalent to a multistep Runge–Kutta method Yj =

k−1 

ϕi (cj )yn+k−1−i

i=0

+h

m 

ψi (cj )f (xn+k−1 + ci h, Yi ),

j = 1, 2, . . . , m,

i=1

yn+k =

k−1  i=0

ϕi (1)yn+k−1−i + h

m 

ψi (1)f (xn+k−1 + ci h, Yi ),

i=1

where the expression of the polynomials ϕi (s), ψi (s) are provided in Lemma 1 of [58]. Lie and Norsett in [58] also provided a complete study of the order of the resulting methods, stating order conditions by means of the study of variational matrices, and showing that the maximum attainable order of a k-step m-stage collocation

nodes that allow method is 2m + k − 1. They also proved that there exist m+k−1 k−1 superconvergence and, in analogy with Runge–Kutta methods, they are named multistep Gaussian collocation points. As Hairer–Wanner stated in [43], these methods are not stiffly stable and, therefore, they are not suited for stiff problems: in order to obtain better stability properties, they derived methods of highest order 2m + k − 2, imposing cm = 1 and deriving the other collocation abscissa in a suited way to achieve this highest order and named the corresponding methods of “Radau”-type, studied their stability properties, deriving also many A-stable methods.

48

D. Conte et al.

3.2.5 Two-Step Collocation and Almost Collocation Methods In more recent times, our strenghts have been devoted to extend the multistep collocation technique to the class of two-step Runge–Kutta methods (TSRK)   [n−1] yn+1 = θyn−1 + θ˜ yn + h m ) + wj f (Yj[n] )), j =1 (vj f (Yj (3.2.11)  [n−1] ) + bij f (Yj[n] )), Yi[n] = ui yn−1 + u˜ i yn + h m j =1 (aij f (Yj introduced by Jackiewicz and Tracogna [50] and further investigated by several authors (see [49] and references therein contained). Two-step Runge–Kutta methods (3.2.11) differ from the multistep Runge–Kutta methods above described, because they also depend on the stage derivatives at two consecutive step points: as a consequence, “we gain extra degrees of freedom associated with a two-step scheme without the need for extra function evaluations” (see [50]), because the function evaluations f (Yj[n−1] ) are completely inherited from the previous step. Therefore, the computational cost of these formulae only depends on the structure of the matrix B. Different approaches to the construction of continuous TSRK methods outside collocation are presented in [1, 2] and [51]. The continuous approximant ⎧ m ⎨ P (xn + sh) = ϕ0 (s)yn−1 + ϕ1 (s)yn + h j =1 (χj (s)f (P (xn−1 + cj h)) +ψj (s)f (P (xn + cj h))), ⎩ yn+1 = P (xn+1 ), (3.2.12) expressed as linear combination of the basis functions {ϕ0 (s), ϕ1 (s), χj (s), ψj (s), j = 1, 2, . . . , m}, is an algebraic polynomial which is derived in order to satisfy some interpolation and collocation conditions, i.e. P (xn−1 ) = yn−1 ,

P (xn ) = yn ,

P  (xn−1 + ci h) = f (xn−1 + ci h, P (xn−1 + ci h)), P  (x

n

+ ci h) = f (xn + ci h, P (xn + ci h)),

i = 1, 2, . . . , m,

(3.2.13)

i = 1, 2, . . . , m.

As a first attempt, we have generalized in [34, 38] the techniques introduced by Guillou–Soulé [42], Hairer–Wanner [43] and Lie–Norsett [58], adapting and extending this technique to TSRK methods. Using the techniques introduced in [58], we have derived in [38] the coefficients of (3.2.12) in closed form: the corresponding results are reported in the following theorem (see [38]). Theorem 3.7 The method (3.2.12) is equivalent to a TSRK method in the form Yj[n] = ϕ0 (cj )yn−1 + ϕ1 (cj )yn + h

m  i=1

[χj (ci )f (xn−1 + ci h, Yi[n−1] )

3 Advances on Collocation Based Numerical Methods for Ordinary

+ ψj (ci )f (xn + ci h, Yi[n] )], yn+1 = ϕ0 (1)yn−1 + ϕ1 (1)yn + h

49

j = 1, 2, . . . , m,

m  [χj (1)f (xn−1 + cj h, Yj[n−1] ) j =1

+ ψj (1)f (xn + cj h, Yj[n] )], where  ψj (s) =

s

0

lj (τ )dτ −

0

 χj (s) =

s

0

−1 lj (τ )dτ 0 −1 M(τ )dτ

0

l˜j (τ )dτ −  −1 0

s ϕ0 (s) = −  00



M(τ )dτ,

j = 1, 2, . . . , m,

M(τ )dτ,

j = 1, 2, . . . , m,

0

 l˜j (τ )dτ

−1 M(τ )dτ

M(τ )dτ

s

s 0

,

−1 M(τ )dτ s M(τ )dτ ϕ1 (s) = 1 +  00 −1 M(τ )dτ

.

with li (s) =

2m  j =1,j =i

l˜j (s) =

2m  i=1,i=j

s − dj , d i − dj s − ei , e j − ei

M(s) =

2m 

 (s − dj ),

j =1



ei = ci − 1, em+i = ci ,

di = ci , dm+i = ci − 1, i = 1, 2, . . . , m,

i = 1, 2, . . . , m.

We proved in [38] that the resulting methods have uniform order 2m + 1 but such an high order enforces these methods to have bounded stability regions only. For this reason, in order to derive highly stable methods (i.e. A-stable and L-stable), we have introduced in [27, 28, 36] the class of almost collocation methods, which are obtained in such a way that only some of the above interpolation and collocation conditions are satisfied. Relaxing the above conditions, we obtain more degrees of freedom, which have been used in order to derive many A-stable and L-stable methods of order m + r, r = 0, 1, . . . , m. Therefore the highest attainable order is 2m which, in principle, can seem the same of standard Runge–Kutta methods. As a matter of fact, this is not true: in fact, Runge–Kutta–Gauss methods have order 2m in the grid points, while the stage order is equal to m, therefore they suffer from order reduction in the integration of stiff problems (see [10, 11, 43]), i.e. the effective order of convergence in presence of stiffness is only m. Our methods, instead, do not suffer from order reduction, i.e. the effective order of convergence in the integration of stiff problems is 2m, because they have high stage order. In [36] we have studied the existence of such methods, derived continuous order conditions, provided

50

D. Conte et al.

characterization results and studied their stability properties. A complete analysis of m-stage two-step continuous methods, with m = 1, 2, 3, 4, has been provided in [32], while the analysis of the implementation issues for two-step collocation methods has been provided in [33]. The construction of algebraically stable two-step collocation methods is object of a current research project.

3.3 Collocation Methods for Second Order ODEs of Special Type We now concentrate our attention on the hystorical evolution of the collocation technique for the numerical solution of initial value problems based on second order ODEs with periodic and oscillating solution ⎧  ⎨ y (x) = f (x, y(x)), x ∈ [x0 , X], (3.3.1) y  (x0 ) = y0 ∈ Rd , ⎩ y(x0 ) = y0 , where f : [x0 , X] × Rd → Rd is assumed to be a is sufficiently smooth function, in order to ensure the existence and the uniqueness of the solution.

3.3.1 Direct and Indirect Collocation Methods In the context of collocation methods for second order ODEs, two possibilities have been taken into account in the literature, i.e. methods based on indirect or direct collocation [77]. Indirect collocation methods are generated by applying a collocation based Runge–Kutta method to the first order representation of (3.3.1), which has doubled dimension. If c

A bT

is the Butcher array of a collocation Runge–Kutta method, the tableau of the corresponding indirect collocation method is c

A2 AT b bT

which results in a Runge–Kutta–Nyström method [45]. The theory of indirect collocation methods completely parallels the well-known theory of collocation methods for first order equations (see [77]) and, therefore, the properties of a collocation method are totally inherited by the corresponding indirect collocation method. Thus, the maximum attainable order is 2m, where m is the number of stages, and

3 Advances on Collocation Based Numerical Methods for Ordinary

51

it is achieved by Gauss-type methods, which are also A-stable, while L-stability is achieved by Radau IIA-type methods, of order 2m − 1. In the case of direct collocation methods, the collocation polynomial is derived directly for the second order problem. Van der Houwen et al. in [77] studied the order, stage order of direct collocation methods and also provided their stability analysis, extending the results of Kramarz [54]. Concerning order and stage order, the following result holds (see [77]): Theorem 3.8 Direct and indirect collocation methods with the same collocation nodes have the same order. The stage order of direct collocation methods is one higher whenever  1 m (s − ci )ds = 0. 0 i=1

Therefore, while indirect and direct collocation methods have the same order, their stage order is different and, in particular, direct methods have higher stage order. However, they are not competitive in terms of stability. Van der Houwen et al. in [77] clearly state that “From a practical point of view, direct collocation methods based on Gauss, Radau and Lobatto collocation points are of limited value, because the rather small stability or periodicity boundaries make them unsuitable for stiff problems. The A-stable indirect analogues are clearly more suitable for integrating stiff problems”. Moreover, Coleman [17] proved that no P -stable one step symmetric collocation methods exist. P -stability (see Lambert-Watson paper [56]) is a very relevant property for the numerical treatment of a second order system whose theoretical solution is periodic with a moderate frequency and a high frequency oscillation of small amplitude superimposed. This phenomenon is known in literature as periodic stiffness [73], which can be reasonably faced using P -stable methods, exactly as A-stable methods are suitable for stiff problems. In other terms, P -stability ensures that the choice of the stepsize is independent from the values of the frequencies, but it only depends on the desired accuracy [21, 68]. In [56], the authors proved that P -stable linear multistep methods p 

αj yn+j = h2

j =0

p 

βj fn+j

j =0

can achieve maximum order 2. In the context of Runge–Kutta–Nyström methods yn+1 = yn + hyn + h2

m 

b¯i f (xn + ci h, Yi ),

i=1  yn+1 = yn + h

m  i=1

bi f (xn + ci h, Yi ),

52

D. Conte et al.

Yi = yn + ci hyn + h2

m 

aij f (xn + cj h, Yj ),

i = 1, 2, . . . , m,

j =1

many A-stable and P -stable methods exist, but the ones falling in the subclass of collocation methods, whose coefficients (see [45]) are of the form  ci Lj (s)ds, aij = 

0



0

1

bi = b¯i =

Li (s)ds, 1

(1 − s)Li (s)ds,

0

have only bounded stability intervals and are not P -stable [68].

3.3.2 Two-Step Runge–Kutta–Nyström Methods We have observed in the previous paragraph that P -stability is a desirable property that only few methods in the context of linear multistep methods and Runge–Kutta– Nyström methods possess. In order to create a good balance between high order and strong stability properties, further steps in the literature have been devoted to the development of multistep Runge–Kutta–Nystrom methods for second order problems. Much of this work has been done by Paternoster (see [61, 68–72]). In particular, the author proved that no P -stable methods can be found in the class of indirect collocation TSRK methods, while it is possible to find P -stable methods in the context of two-step Runge–Kutta–Nyström methods  Yj[n−1] = yn−1 + cj hyn−1 + h2

m 

aj k f (xn−1 + ck h, Yk[n−1] ),

j = 1, 2, . . . , m,

k=1

Yj[n] = yn + cj hyn + h2

m 

aj k f (xn + ck h, Yk[n] ),

j = 1, 2, . . . , m,

k=1 m   (vj yn−1 + wj yn ) yn+1 = (1 − θ )yn + θyn−1 + h j =1

+ h2

m 

v¯j f (xn−1 + cj h, Yj[n−1] ) + w¯ j f (xn + cj h, Yj[n] ),

j =1   yn+1 = (1 − θ )yn + θyn−1 +h

m  (vj f (xn−1 + cj h, Yj[n−1] ) j =1

+ wj f (xn + cj h, Yj[n] )),

3 Advances on Collocation Based Numerical Methods for Ordinary

53

which represent the extension to second order problems of the two-step Runge– Kutta methods introduced in [52] for first order ODEs.

3.3.3 Collocation Based Two-Step Hybrid Methods In the numerical integration of second order ODEs through collocation, many possibilities can be taken into account: for example, Runge–Kutta–Nyström methods provide an approximation to the solution and its first derivative at each step point. However, as Henrici observed in [46], “If one is not particularly interested in the values of the first derivatives, it seems unnatural to introduce them artificially”. For this reason, other types of methods have been taken into account in the literature, i.e. methods which provide an approximation to the solution without computing any approximation to the first derivative. Coleman introduced in [19] the following class of two-step hybrid methods for second order equations: Yi[n] = ui yn−1 + (1 − ui )yn + h2

m 

aij f (xn + cj h, Yj[n] ),

j =1

i = 1, 2, . . . , m, yn+1 = θyn−1 + (1 − θ )yn + h2

(3.3.2) m 

wj f (xn + cj h, Yj[n] ).

(3.3.3)

j =1

This class of methods has been further investigated in [16, 37, 40, 75, 76]. In more recent times, we derived in [35] collocation based methods belonging to the class of Coleman hybrid methods (3.3.2)–(3.3.3), extending the technique introduced by Hairer and Wanner in [43] for first order problems. The collocation polynomial takes the form P (xn + sh) = ϕ1 (s)yn−1 + ϕ2 (s)yn + h

2

m 

χj (s)P  (xn + cj h),

j =1

where s =

x−xn h

∈ [0, 1], and the unknown basis functions {ϕ1 (s), ϕ2 (s), χj (s), j = 1, 2, . . . , m},

are derived imposing the following m + 2 conditions P (xn−1 ) = yn−1 , P (xn ) = yn , 

P (xn + cj h) = f (xn + cj h, P (xn + cj h)),

j = 1, 2, . . . , m.

(3.3.4)

54

D. Conte et al.

After computing the basis functions as solutions of m + 2 linear systems (see [62]), the resulting class of methods takes the following form Yi[n] = ϕ1 (ci )yn−1 + ϕ2 (ci )yn + h2

m 

χj (ci )P  (xn + cj h),

(3.3.5)

j =1

yn+1 = ϕ1 (1)yn−1 + ϕ2 (1)yn + h2

m 

χj (1)P  (xn + cj h).

(3.3.6)

j =1

In [35] we have provided the study of stability and periodicity properties and derived continuous order conditions for (3.3.6)–(3.3.5), which are object of the following result. Theorem 3.9 Assume that the function f is sufficiently smooth. The collocation method associated to (3.3.4) has uniform order p if the following conditions are satisfied: 1 − ϕ1 (s) − ϕ2 (s) = 0,

s + ϕ1 (s) = 0,

s k + (−1)k+1 ϕ1 (s) − k(k − 1)

m 

χj (s)cjk−2 = 0,

k = 2, 3, . . . , p, s ∈ [0, 1].

j =1

Theorem 3.9 allows us to prove that every two-step collocation method associated to (3.3.4), has order p = m on the whole integration interval, and this result is in keeping with [19].

3.3.4 Mixed Collocation Methods The development of classical collocation methods (i.e. methods based on algebraic polynomials), even if it is not the most suitable choice for second order problems that do not possess solutions with polynomial behavior, it is the first necessary step in order to construct collocation methods whose collocation function is expressed as linear combination of different functions, e.g. trigonometric polynomials, mixed or exponential basis (see, for instance, [20, 48]), which can better follow the qualitative behavior of the solution. It is indeed more realistic to choose basis functions which are not polynomials. Many authors have considered in literature different functional basis, instead of the polynomial one, e.g. [8, 18, 21, 37, 39, 41, 48, 53, 65, 67, 69, 71, 74]. In particular we mention here the work by Coleman and Duxbury [20], where the authors introduced mixed collocation methods applied to the Runge–Kutta–Nyström scheme, where the collocation function is expressed as linear combination of trigonometric

3 Advances on Collocation Based Numerical Methods for Ordinary

55

functions and powers, in order to provide better approximations for oscillatory solutions. The methods are derived in order to exactly integrate the armonic oscillator y  = −k 2 y, where k is a constant, a feature which is not achievable by algebraic polynomial collocation. The term mixed interpolation appeared for the first time in [39] to describe interpolation by a linear combination of a sine and cosine of a given frequency, and powers of the relevant variable, and later used by Brunner et al. in [8] in the context of Volterra integral equations. The solution on the generic integration interval [xn , xn+1 ] is approximated by the collocating function u(xn + sh) = a cos θ s + b sin θ s +

m−1 

i s i ,

(3.3.7)

i=0

which satisfies the following collocation and interpolation conditions u(xn ) = yn ,

u (xn ) = yn ,

u (xn + cj h) = f (xn + cj h, u(xn + cj h)),

j = 1, 2, . . . , m.

Integrating (3.3.7) twice, we obtain the Runge–Kutta–Nyström formulation of the methods, i.e. u (xn + sh) = yn + h

m 

αi (s)f (xn + ci h),

i=1

u(xn + sh) =

yn + shyn

+h

2

m 

βi (s)f (xn + ci h),

i=1

where





s

αi (s) =

Li (τ )dτ,

s

βi (s) =

0

(s − τ )Li (τ )dτ.

0

Outside collocation, many authors derived methods having frequency dependent parameters (see, for instance, [48, 53, 66, 74] and references therein contained). The linear stability analysis of these methods is carried out in [21]. In [37] also a method with parameters depending on two frequencies is presented, and the modification in the stability analysis is performed, leading to a three dimensional region.

3.4 Collocation Methods for VIEs Piecewise polynomial collocation methods for Volterra Integral Equations introduce a number of aspects not present when solving ODEs. In this section we will present the main results in the context of collocation and almost collocation methods for

56

D. Conte et al.

VIEs of the form



x

y(x) = g(x) +

k(x, τ, y(τ ))dτ,

x ∈ I := [0, X],

(3.4.1)

0

where k ∈ C(D × R), with D := {(x, τ ) : 0 ≤ τ ≤ x ≤ X}, and g ∈ C(I ), also underlying connections and differences with the case of ODEs.

3.4.1 Classical One-Step Collocation Methods Let us discretize the interval I by introducing a uniform mesh Ih = {xn := nh, n = 0, . . . , N, h ≥ 0, Nh = X}. Equation (3.4.1) can be rewritten, by relating it to this mesh, as y(x) = Fn (x) + n (x), where



x ∈ [xn , xn+1 ],

xn

Fn (x) := g(x) +

k(x, τ, y(τ ))dτ 0

and

 n (x) :=

x

k(x, τ, y(τ ))dτ xn

represent respectively the lag term and the increment function. Let us fix m collocation parameters 0 ≤ c1 < · · · < cm ≤ 1 and denote by xnj = xn + cj h the collocation points. The collocation polynomial, restricted to the interval [xn , xn+1 ], is of the form: un (xn + sh) =

m 

Lj (s)Unj ,

s ∈ [0, 1], n = 0, 1, . . . , N − 1,

(3.4.2)

j =1

where Lj (s) is the j -th Lagrange fundamental polynomial with respect to the collocation parameters and Unj := un (xnj ). Exact collocation methods are obtained by imposing that the collocation polynomial (3.4.2) exactly satisfies the VIE (3.4.1) in the collocation points xni and by computing yn+1 = un (xn+1 ):  Uni = F ni + ni , (3.4.3) yn+1 = m j =1 Lj (1)Unj , where Fni = g(xni ) + h

n−1  

1

k(xni , xν + sh, uν (xν + sh))ds,

i = 1, 2, . . . , m,

ν=0 0

(3.4.4)

3 Advances on Collocation Based Numerical Methods for Ordinary

 ni = h

ci

k(xni , xn + sh, un (xn + sh))ds,

i = 1, 2, . . . , m.

57

(3.4.5)

0

Note that the first equation in (3.4.3) represents a system of m nonlinear equations in the m unknowns Uni . We obtain an approximation u(x) of the solution y(x) of the integral equation (3.4.1) in [0, X], by considering u(x)|(xn ,xn+1 ] = un (x),

(3.4.6)

where un (x) given by (3.4.2). We recall that, in contrast with what happens in the case of ODEs, generally u(x) is not continuous in the mesh points, as (−1) u(x) ∈ Sm−1 (Ih ),

where

(3.4.7)

  Sμ(d) (Ih ) = v ∈ C d (I ) : v|(xn ,xn+1 ] ∈ μ (0 ≤ n ≤ N − 1) .

Here, μ denotes the space of (real) polynomials of degree not exceeding μ. A complete analysis of collocation methods for linear and nonlinear Volterra integral and integro–differential equations, with smooth and weakly singular kernels is given in [6]. In particular, as shown in [6, 7], the classical one–step collocation methods for a second-kind VIE do no longer exhibit O(h2m ) superconvergence at the mesh points if collocation is at the Gauss points, in fact they have uniform order m for any choice of the collocation parameters and local superconvergence order in the mesh points of 2m − 2 (m Lobatto points or m − 1 Gauss points with cm = 1) or 2m − 1 (m Radau II points). The optimal order is recovered only in the iterated collocation solution. We observe that, differently from the case of ODEs, the collocation equations are in general not yet in a form amenable to numerical computation, due to the presence of the memory term given by the Volterra integral operator. Thus, another ¯ ni  ni for apdiscretization step, based on quadrature formulas F¯ni  Fni and proximating the lag term (3.4.4) and the increment function (3.4.5), is necessary to obtain the fully discretized collocation scheme, thus leading to discretized collocation methods. Such methods preserve, under suitable hypothesis on the quadrature formulas, the same order of the exact collocation methods [7]. The connection between collocation and implicit Runge–Kutta methods for VIEs (the so called VRK methods) is not immediate: a collocation method for VIEs is equivalent to a VRK method if and only if cm = 1 (see Theorem 5.2.2 in [7]). Some other continuous extensions of Runge–Kutta methods for VIEs, which do not necessarily lead to collocation methods, have been introduced in [3]. Many efforts have been made in the literature with the aim of obtaining fast collocation and more general Runge–Kutta methods for the numerical solution of VIEs. It is known that the numerical treatment of VIEs is very expensive from computational point of view because of presence of the lag-term, which contains the entire history of the phenomenon. To this cost, it has also to be added the one due

58

D. Conte et al.

to the increment term which leads, for implicit methods (generally possessing the best stability properties), to the resolution of a system of nonlinear equations at each step of integration. In order to reduce the computational effort in the lag–term computation, fast collocation and Runge–Kutta methods have been constructed for convolution VIEs of Hammerstein type, see [13, 23, 59, 60]. The stability analysis of collocation and Runge–Kutta methods for VIEs can be found in [4, 7, 14, 30] and the related bibliography. In particular a collocation method for VIEs is A-stable if the corresponding method for ODEs is A-stable.

3.4.2 Multistep Collocation Multistep collocation and Runge–Kutta methods for VIEs, have been introduced in order to bring down the computational cost related to the resolution of non-linear systems for the computation of the increment term. As a matter of fact such methods, showing a dependence on stages and steps in more consecutive grid points, permit to raise the order of convergence of the classical methods, without inflating the computational cost or, equivalently, having the same order at a lower computational cost. A first analysis of multistep collocation methods for VIEs appeared in [24, 25], where the methods are obtained by introducing in the collocation polynomial the dependence from r previous time steps; namely we seek for a collocation polynomial, whose restriction to the interval [xn , xn+1 ] takes the form Pn (xn + sh) =

r−1 

ϕk (s)yn−k +

m 

ψj (s)Ynj ,

j =1

k=0

s ∈ [0, 1], n = 0, 1, . . . , N − 1,

(3.4.8)

where Ynj := Pn (xnj )

(3.4.9)

and ϕk (s), ψj (s) are polynomials of degree m + r − 1 to be determined by imposing the interpolation conditions at the points xn−k , that is un (xn−k ) = yn−k , and by satisfying (3.4.9). It is proved in [26, 29] that, assuming ci = cj and c1 = 0, the polynomials ϕk (s), ψj (s) have the form: ϕk (s) =

m r−1  s − ci  s + i · , −k − ci −k + i i=1

ψj (s) =

r−1  i=0

i=0 i=k

m s + i  s − ci . · cj + i c j − ci i=1 i=j

(3.4.10)

3 Advances on Collocation Based Numerical Methods for Ordinary

59

Then the discretized multistep collocation method assumes the form: 

¯ ni , Yni = F¯ni + r−1  yn+1 = k=0 ϕk (1)yn−k + m j =1 ψj (1)Ynj .

(3.4.11)

The lag-term and increment–term approximations F¯ni = g(xni ) + h

μ1 n−1  

bl k(xni , xν + ξl h, Pν (xν + ξl h)),

ν=0 l=0

i = 1, 2, . . . , m ¯ ni = h

μ0 

(3.4.12)

wil k(xni , xn + dil h, Pn (xn + dil h)),

i = 1, 2, . . . , m

(3.4.13)

l=0

are obtained by using quadrature formulas of the form μ

1 (ξl , bl )l=1 ,

μ

0 (dil , wil )l=1 ,

i = 1, 2, . . . , m,

(3.4.14)

where the quadrature nodes ξl and dil satisfy 0 ≤ ξ1 < · · · < ξμ1 ≤ 1 and 0 ≤ di1 < · · · < diμ0 ≤ 1, μ0 and μ1 are positive integers and wil , bl are suitable weights. The discretized multistep collocation method (3.4.8)–(3.4.11) provides a continuous approximation P (x) of the solution y(x) of the integral equation (3.4.1) in [0, X], by considering P (x)|(xn ,xn+1 ] = Pn (x),

(3.4.15)

where Pn (x) is given by (3.4.8). We note that usually the polynomial constructed in the collocation methods for VIEs doesn’t interpolate the numerical solution in the previous step points, resulting a discontinuous approximation of the solution (3.4.7). In this multistep extension, the collocation polynomial is instead a continuous ap(0) proximation to the solution, i.e. u(x) ∈ Sm+r−1 (Ih ). The discretized multistep collocation method (3.4.8)–(3.4.11) can be regarded as a multistep Runge–Kutta method for VIEs: ⎧ ⎪ ni ) ⎨ Yni = F¯n (x

  μ0 wil k xn + eil h, xn + dil h, r−1 γilk yn−k + m βilj Ynj , + h l=1 j =1 k=0 r−1 m ⎪ ⎩y n+1 = j =1 λj Ynj , k=0 θk yn−k + (3.4.16) where   μ1 n−1  r−1 m    bl k x, xν + ξl h, δlk yν−k + ηlj Yν,j (3.4.17) F¯n (x) = g(x) + h ν=0 l=1

k=0

j =1

60

D. Conte et al.

and eil = ci ,

γilk = ϕk (dil ),

θk = ϕk (1), δlk = ϕk (ξl ),

βilj = ψj (dil ),

λj = ψj (1), ηlj = ψj (ξl ).

The reason of interest of the multistep collocation methods lies in the fact that they increase the order of convergence of collocation methods without increasing the computational cost, except for the cost due to the starting procedure. As a matter of fact, in advancing from xn to xn+1 , we make use of the approximations yn−k , k = 0, 1, . . . , r − 1, which have already been evaluated at the previous steps. This permits to increase the order, by maintaining in (3.4.11) the same dimension m of the nonlinear system (3.4.3). The r-steps m-points collocation methods have uniform order m + r, and order of local superconvergence 2m + r − 1. The knowledge of the collocation polynomial, which provides a continuous approximation of uniform order to the solution, will allow a cheap variable stepsize implementation. Indeed, when the stepsize changes, the new approximation values can be computed by simply evaluating the collocation polynomial, without running into problems of order reduction, as a consequence of the uniform order.

3.4.3 Two-Step Collocation and Almost Collocation Methods Unfortunately multistep methods of the form (3.4.8)–(3.4.11) do not lead to a good balance between high order and strong stability properties, infact, altough methods with unbounded stability regions exist, no A-stable methods have been found. Therefore in [25] a modification in the technique has been introduced, thus obtaining two-step almost collocation methods, also for systems of VIEs, by relaxing some of the collocation conditions and by introducing some previous stage values, in order to further increase the order and to have free parameters in the method, to be used to get A-stability. The methods are defined by  ⎧ (s)yn−1 + ϕ1 (s)yn + m ⎨ P (xn + sh) = ϕ0 j =1 χj (s)P (xn−1,j ) ¯ ¯ (3.4.18) ψ (s)( F + ), + m j nj nj j =1 ⎩ yn+1 = P (xn+1 ), s ∈ (0, 1], n = 1, 2, . . . , N − 1. If the polynomials ϕ0 (s), ϕ1 (s), χj (s) and ψj (s), j = 1, 2, . . . , m satisfy the interpolation conditions ϕ0 (0) = 0, ϕ0 (−1) = 1,

ϕ1 (0) = 1, ϕ1 (−1) = 0,

χj (0) = 0, ψj (0) = 0, χj (−1) = 0, ψj (−1) = 0,

3 Advances on Collocation Based Numerical Methods for Ordinary

61

and the collocation conditions ϕ0 (ci ) = 0, ϕ0 (ci − 1) = 0,

ϕ1 (ci ) = 0, χj (ci ) = 0, ψj (ci ) = δij , ϕ1 (ci − 1) = 0, χj (ci − 1) = δij , ψj (ci − 1) = 0,

i = 1, 2, . . . , m, then we obtain order p = 2m + 1. In our search for A-stable methods we will have been mainly concerned with methods of order p = 2m + 1 − r, where r = 1 or r = 2 is the number of relaxed conditions. Namely we have chosen ϕ0 (s) as a polynomial of degree ≤ 2m + 1 − r, which satisfies the collocation conditions ϕ0 (ci ) = 0,

i = 1, 2, . . . , m.

(3.4.19)

This leads to the polynomial ϕ0 (s) of the form ϕ0 (s) = (q0 + q1 s + · · · + qm+1−r s m+1−r )

m 

(s − ci ),

(3.4.20)

i=1

where q0 , q1 , . . . , qm+1−r are free parameters. Moreover, for p = 2m − 1 we have chosen ϕ1 (s) as a polynomial of degree ≤ 2m − 1 which satisfies the collocation conditions ϕ1 (ci ) = 0,

i = 1, 2, . . . , m.

(3.4.21)

This leads to the polynomial ϕ1 (s) of the form ϕ1 (s) = (p0 + p1 s + · · · + pm−1 s m−1 )

m 

(s − ci ),

(3.4.22)

i=1

where p0 , p1 , . . . , pm−1 are free parameters. The methods have uniform order of convergence p = 2m + 1 − r, and are therefore suitable for an efficient variable stepsize implementation. Moreover methods which are A-stable with respect to the basic test equation and have unbounded stability regions with respect to the convolution test equation have been provided.

3.4.4 Mixed Collocation In the case of VIEs with periodic highly oscillatory solutions, traditional methods may be inefficient, as they may require the use of a small stepsize in order to follow accurately the oscillations of high frequency. As in the case of ODEs “ad hoc” numerical methods have been constructed, incorporating the a priori knowledge of the behavior of the solution, in order to use wider stepsizes with respect to classical methods and simultaneously to simulate with high accuracy the oscillations of the solution.

62

D. Conte et al.

A first work on the numerical treatment of VIEs with periodic solution is [5], where numerical methods were constructed by means of mixed interpolation. Recently, mixed collocation methods have been introduced in [8, 9] for VIEs and VIDEs. In particular in [8], mixed collocation methods have been introduced for linear convolution VIEs of the form  x k(x − τ )y(τ )dτ, x ∈ [0, X], (3.4.23) y(x) = g(x) + −∞

with y(x) = ψ(x),

x ∈ [−∞, 0],

∈ L1 (0, ∞),

g is a continuous periodic function and ψ is a given bounded where k and continuous function. The collocation polynomial is taken in the form m 

Pn (xn + sh) =

Bk (s)Yn,k ,

k=0

where the Bk (s) are combinations of trigonometric functions and algebraic polynomials given in [8]. The numerical method is of the form  ¯ Yni = F¯ni + ni , (3.4.24) yn+1 = m k=0 Bk (1)Yn,k , where the lag-term and increment term approximations are given by F¯ni = g(xni ) + ¯ ni = hci

m 



0

−∞

k(xni − τ )ψ(τ )dτ + h 

wl (1)k(xni − xn − hci cl )

l=0

m n−1  

wl (1)k(xnj − xν,l Pν (xν,l ),

ν=0 l=0 m 



Bk (ci cl )Yn,k

k=0

with



s

wl (s) =

Bl (τ )dτ. 0

With some suitable choices for collocation parameters such methods accurately integrate systems for which the period of oscillation of the solution is known. In the paper [15] the authors introduce a family of linear methods, namely Direct Quadrature (DQ) methods, specially tuned on the specific feature of the problem, based on the exponential fitting [47, 48], which is extremely flexible when periodic functions are treated. Such methods are based on a three-term quadrature formula, that is of the same form as the usual Simpson rule, but specially tuned on integrands of the form k(s)y(s) where k and y are of type k(x) = eαx ,

y(x) = a + b cos(ωx) + c sin(ωx),

(3.4.25)

where α, ω, a, b, c ∈ R. The coefficients of the new quadrature rule depend on the parameters of the integrand, i.e. α and ω. It has been shown as the use of exponentially fitted based three-point quadrature rules produces a definite improvement in

3 Advances on Collocation Based Numerical Methods for Ordinary

63

the accuracy when compared with the results from the classical Simpson rule, and that the magnitude of the gain depends on how good is the knowledge of the true frequencies. The results also indicate that, as a rule, if the input accuracy is up to 10 percent, then the accuracy gain in the output is substantial.

3.5 Conclusions and Future Perspectives In this paper we have described, at the best of our knowledge, some of the collocation methods appeared in the literature for ODEs and VIEs. Some interesting properties of collocation-based methods are, in our opinion, still to be exploited. For instance, the knowledge of the collocation function on the whole interval of integration might allow cheap and realiable error estimators, to be used in a variable stepsize-variable order environment, also for problems with delay. Therefore, although collocation technique is an old idea in Numerical Analysis, we strongly believe that it will constitute building blocks for the development of modern software for an efficient and accurate integration of evolutionary problems.

References 1. Bartoszewski, Z., Jackiewicz, Z.: Derivation of continuous explicit two-step Runge–Kutta methods of order three. J. Comput. Appl. Math. 205, 764–776 (2007) 2. Bartoszewski, Z., Podhaisky, H., Weiner, R.: Construction of stiffly accurate two-step Runge– Kutta methods of order three and their continuous extensions using Nordsieck representation. Report No. 01, Martin-Luther-Universität Halle-Wittenberg, Fachbereich Mathematik und Informatik (2007) 3. Bellen, A., Jackiewicz, Z., Vermiglio, R., Zennaro, M.: Natural continuous extensions of Runge–Kutta methods for Volterra integral equations of the second kind and their applications. Math. Comput. 52(185), 49–63 (1989) 4. Bellen, A., Jackiewicz, Z., Vermiglio, R., Zennaro, M.: Stability analysis of Runge–Kutta methods for Volterra integral equations of the second kind. IMA J. Numer. Anal. 10(1), 103– 118 (1990) 5. Bother, P., De Meyer, H., Vanden Berghe, G.: Numerical solution of Volterra equations based on mixed interpolation. Comput. Math. Appl. 27, 1–11 (1994) 6. Brunner, H.: Collocation Methods for Volterra Integral and Related Functional Equations. Cambridge University Press, Cambridge (2004) 7. Brunner, H., van der Houwen, P.J.: The Numerical Solution of Volterra Equations. CWI Monographs, vol. 3. North-Holland, Amsterdam (1986) 8. Brunner, H., Makroglou, A., Miller, R.K.: On mixed collocation methods for Volterra integral equations with periodic solution. Appl. Numer. Math. 24, 115–130 (1997) 9. Brunner, H., Makroglou, A., Miller, R.K.: Mixed interpolation collocation methods for first and second order Volterra integro-differential equations with periodic solution. Appl. Numer. Math. 23, 381–402 (1997) 10. Butcher, J.C.: The Numerical Analysis of Ordinary Differential Equations. Runge–Kutta and General Linear Methods. Wiley, Chichester/New York (1987) 11. Butcher, J.C.: Numerical Methods for Ordinary Differential Equations, 2nd edn. Wiley, Chichester (2008)

64

D. Conte et al.

12. Butcher, J.C., Tracogna, S.: Order conditions for two-step Runge–Kutta methods. Appl. Numer. Math. 24, 351–364 (1997) 13. Capobianco, G., Conte, D., Del Prete, I., Russo, E.: Fast Runge–Kutta methods for nonlinear convolution systems of Volterra integral equations. BIT 47(2), 259–275 (2007) 14. Capobianco, G., Conte, D., Del Prete, I., Russo, E.: Stability analysis of fast numerical methods for Volterra integral equations. Electron. Trans. Numer. Anal. 30, 305–322 (2008) 15. Cardone, A., Ixaru, L.Gr., Paternoster, B.: Exponential fitting Direct Quadrature methods for Volterra integral equations. Numer. Algorithms (2010). doi:10.1007/s11075-010-9365-1 16. Chan, R.P.K., Leone, P., Tsai, A.: Order conditions and symmetry for two-step hybrid methods. Int. J. Comput. Math. 81(12), 1519–1536 (2004) 17. Coleman, J.P.: Rational approximations for the cosine function; P-acceptability and order. Numer. Algorithms 3(1–4), 143–158 (1992) 18. Coleman, J.P.: Mixed interpolation methods with arbitrary nodes. J. Comput. Appl. Math. 92, 69–83 (1998) 19. Coleman, J.P.: Order conditions for a class of two-step methods for y  = f (x, y). IMA J. Numer. Anal. 23, 197–220 (2003) 20. Coleman, J.P., Duxbury, S.C.: Mixed collocation methods for y  = f (x, y). J. Comput. Appl. Math. 126(1–2), 47–75 (2000) 21. Coleman, J.P., Ixaru, L.Gr.: P -stability and exponential-fitting methods for y  = f (x, y). IMA J. Numer. Anal. 16(2), 179–199 (1996) 22. Cong, N.H., Mitsui, T.: Collocation-based two-step Runge–Kutta methods. Jpn. J. Ind. Appl. Math. 13(1), 171–183 (1996) 23. Conte, D., Del Prete, I.: Fast collocation methods for Volterra integral equations of convolution type. J. Comput. Appl. Math. 196(2), 652–663 (2006) 24. Conte, D., Paternoster, B.: A family of multistep collocation methods for Volterra integral equations. In: Simos, T.E., Psihoyios, G., Tsitouras, Ch. (eds.) Numerical Analysis and Applied Mathematics. AIP Conference Proceedings, vol. 936, pp. 128–131. Springer, Berlin (2007) 25. Conte, D., Paternoster, B.: Multistep collocation methods for Volterra integral equations. Appl. Numer. Math. 59, 1721–1736 (2009) 26. Conte, D., Jackiewicz, Z., Paternoster, B.: Two-step almost collocation methods for Volterra integral equations. Appl. Math. Comput. 204, 839–853 (2008) 27. Conte, D., D’Ambrosio, R., Ferro, M., Paternoster, B.: Piecewise-polynomial approximants for solutions of functional equations, in press on Collana Scientifica di Ateneo, Univ. degli Studi di Salerno 28. Conte, D., D’Ambrosio, R., Ferro, M., Paternoster, B.: Practical construction of two-step collocation Runge–Kutta methods for ordinary differential equations. In: Applied and Industrial Mathematics in Italy III. Series on Advances in Mathematics for Applied Sciences, pp. 278– 288. World Scientific, Singapore (2009) 29. Conte, D., D’Ambrosio, R., Ferro, M., Paternoster, B.: Modified collocation techniques for Volterra integral equations. In: Applied and Industrial Mathematics in Italy III. Series on Advances in Mathematics for Applied Sciences, pp. 268–277. World Scientific, Singapore (2009) 30. Crisci, M.R., Russo, E., Vecchio, A.: Stability results for one-step discretized collocation methods in the numerical treatment of Volterra integral equations. Math. Comput. 58(197), 119–134 (1992) 31. D’Ambrosio, R.: Highly stable multistage numerical methods for functional equations: theory and implementation issues. Bi-Nationally supervised Ph.D. Thesis in Mathematics, University of Salerno, Arizona State University (2010) 32. D’Ambrosio, R., Jackiewicz, Z.: Continuous two-step Runge–Kutta methods for ordinary differential equations. Numer. Algorithms 54(2), 169–193 (2010) 33. D’Ambrosio, R., Jackiewicz, Z.: Construction and implementation of highly stable two-step collocation methods, submitted 34. D’Ambrosio, R., Ferro, M., Paternoster, B.: A general family of two step collocation methods for ordinary differential equations. In: Simos, T.E., Psihoyios, G., Tsitouras, Ch. (eds.)

3 Advances on Collocation Based Numerical Methods for Ordinary

35. 36. 37.

38.

39. 40. 41. 42. 43.

44.

45. 46. 47. 48. 49. 50. 51. 52. 53.

54. 55. 56. 57. 58. 59.

65

Numerical Analysis and Applied Mathematics. AIP Conference Proceedings, vol. 936, pp. 45–48. Springer, Berlin (2007) D’Ambrosio, R., Ferro, M., Paternoster, B.: Two-step hybrid collocation methods for y  = f (x, y). Appl. Math. Lett. 22, 1076–1080 (2009) D’Ambrosio, R., Ferro, M., Jackiewicz, Z., Paternoster, B.: Two-step almost collocation methods for ordinary differential equations. Numer. Algorithms 53(2–3), 195–217 (2010) D’Ambrosio, R., Ferro, M., Paternoster, B.: Trigonometrically fitted two-step hybrid methods for second order ordinary differential equations with one or two frequencies, to appear on Math. Comput. Simul. D’Ambrosio, R., Ferro, M., Paternoster, B.: Collocation based two step Runge–Kutta methods for ordinary differential equations. In: Gervasi, O., et al. (eds.) ICCSA 2008. Lecture Notes in Comput. Sci., Part II, vol. 5073, pp. 736–751. Springer, New York (2008) De Meyer, H., Vanthournout, J., Vanden Berghe, G.: On a new type of mixed interpolation. J. Comput. Appl. Math. 30, 55–69 (1990) Franco, J.M.: A class of explicit two-step hybrid methods for second-order IVPs. J. Comput. Appl. Math. 187(1), 41–57 (2006) Gautschi, W.: Numerical integration of ordinary differential equations based on trigonometric polynomials. Numer. Math. 3, 381–397 (1961) Guillou, A., Soulé, F.L.: La résolution numérique des problèmes differentiels aux conditions par des méthodes de collocation. RAIRO Anal. Numér. Ser. Rouge R-3, 17–44 (1969) Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II—Stiff and Differential– Algebraic Problems. Springer Series in Computational Mathematics, vol. 14. Springer, Berlin (2002) Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration—Structure-Preserving Algorithms for Ordinary Differential Equations. Springer Series in Computational Mathematics. Springer, Berlin (2000) Hairer, E., Norsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I—Nonstiff Problems. Springer Series in Computational Mathematics, vol. 8. Springer, Berlin (2000) Henrici, P.: Discrete Variable Methods in Ordinary Differential Equations. Wiley, New York (1962) Ixaru, L.Gr.: Operations on oscillatory functions. Comput. Phys. Commun. 105, 1–19 (1997) Ixaru, L.Gr., Vanden Berghe, G.: Exponential Fitting. Kluwer Academic, Dordrecht (2004) Jackiewicz, Z.: General Linear Methods for Ordinary Differential Equations. Wiley, Hoboken (2009) Jackiewicz, Z., Tracogna, S.: A general class of two-step Runge–Kutta methods for ordinary differential equations. SIAM J. Numer. Anal. 32, 1390–1427 (1995) Jackiewicz, Z., Tracogna, S.: Variable stepsize continuous two-step Runge–Kutta methods for ordinary differential equations. Numer. Algorithms 12, 347–368 (1996) Jackiewicz, Z., Renaut, R., Feldstein, A.: Two-step Runge–Kutta methods. SIAM J. Numer. Anal. 28(4), 1165–1182 (1991) Konguetsof, A., Simos, T.E.: An exponentially-fitted and trigonometrically-fitted method for the numerical solution of periodic initial-value problems. Numerical methods in physics, chemistry, and engineering. Comput. Math. Appl. 45(1–3), 547–554 (2003) Kramarz, L.: Stability of collocation methods for the numerical solution of y  = f (t, y). BIT 20(2), 215–222 (1980) Lambert, J.D.: Numerical Methods for Ordinary Differential Systems: The Initial Value Problem. Wiley, Chichester (1991) Lambert, J.D., Watson, I.A.: Symmetric multistep methods for periodic initial value problems. J. Inst. Math. Appl. 18, 189–202 (1976) Lie, I.: The stability function for multistep collocation methods. Numer. Math. 57(8), 779–787 (1990) Lie, I., Norsett, S.P.: Superconvergence for multistep collocation. Math. Comput. 52(185), 65–79 (1989) Lopez-Fernandez, M., Lubich, C., Schädle, A.: Fast and oblivious convolution quadrature. SIAM J. Sci. Comput. 28, 421–438 (2006)

66

D. Conte et al.

60. Lopez-Fernandez, M., Lubich, C., Schädle, A.: Adaptive, fast, and oblivious convolution in evolution equations with memory. SIAM J. Sci. Comput. 30, 1015–1037 (2008) 61. Martucci, S., Paternoster, B.: General two step collocation methods for special second order ordinary differential equations. Paper Proceedings of the 17th IMACS World Congress Scientific Computation, Applied Mathematics and Simulation, Paris, July 11–15 (2005) 62. Martucci, S., Paternoster, B.: Vandermonde-type matrices in two step collocation methods for special second order ordinary differential equations. In: Bubak, M., et al. (eds.) Computational Science, ICCS 2004. Lecture Notes in Comput. Sci., Part IV, vol. 3039, pp. 418–425. Springer, Berlin (2004) 63. Norsett, S.P.: Collocation and perturbed collocation methods. In: Numerical Analysis, Proc. 8th Biennial Conf., Univ. Dundee, Dundee, 1979. Lecture Notes in Math., vol. 773, pp. 119– 132. Springer, Berlin (1980) 64. Norsett, S.P., Wanner, G.: Perturbed collocation and Runge Kutta methods. Numer. Math. 38(2), 193–208 (1981) 65. Paternoster, B.: Runge–Kutta(–Nyström) methods for ODEs with periodic solutions based on trigonometric polynomials. Appl. Numer. Math. 28, 401–412 (1998) 66. Paternoster, B.: A phase-fitted collocation-based Runge–Kutta–Nyström methods. Appl. Numer. Math. 35(4), 339–355 (2000) 67. Paternoster, B.: General two-step Runge–Kutta methods based on algebraic and trigonometric polynomials. Int. J. Appl. Math. 6(4), 347–362 (2001) 68. Paternoster, B.: Two step Runge–Kutta–Nystrom methods for y  = f (x, y) and P-stability. In: Sloot, P.M.A., Tan, C.J.K., Dongarra, J.J., Hoekstra, A.G. (eds.) Computational Science, ICCS 2002. Lecture Notes in Computer Science, Part III, vol. 2331, pp. 459–466. Springer, Amsterdam (2002) 69. Paternoster, B.: Two step Runge–Kutta–Nystrom methods for oscillatory problems based on mixed polynomials. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Dongarra, J.J., Zomaya, A.Y., Gorbachev, Y.E. (eds.) Computational Science, ICCS 2003. Lecture Notes in Computer Science, Part II, vol. 2658, pp. 131–138. Springer, Berlin/Heidelberg (2003) 70. Paternoster, B.: Two step Runge–Kutta–Nystrom methods based on algebraic polynomials. Rend. Mat. Appl., Ser. VII 23, 277–288 (2003) 71. Paternoster, B.: Two step Runge–Kutta–Nystrom methods for oscillatory problems based on mixed polynomials. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Dongarra, J.J., Zomaya, A.Y., Gorbachev, Y.E. (eds.) Computational Science, ICCS 2003. Lecture Notes in Computer Science, Part II, vol. 2658, pp. 131–138. Springer, Berlin/Heidelberg (2003) 72. Paternoster, B.: A general family of two step Runge–Kutta–Nyström methods for y  = f (x, y) based on algebraic polynomials. In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J.J. (eds.) Computational Science, ICCS 2003. Lecture Notes in Computer Science, Part IV, vol. 3994, pp. 700–707. Springer, Berlin (2006) 73. Petzold, L.R., Jay, L.O., Yen, J.: Numerical solution of highly oscillatory ordinary differential equations. Acta Numer. 6, 437–483 (1997) 74. Simos, T.E.: Dissipative trigonometrically-fitted methods for linear second-order IVPs with oscillating solution. Appl. Math. Lett. 17(5), 601–607 (2004) 75. Van de Vyver, H.: A phase-fitted and amplification-fitted explicit two-step hybrid method for second-order periodic initial value problems. Int. J. Mod. Phys. C 17(5), 663–675 (2006) 76. Van de Vyver, H.: Phase-fitted and amplification-fitted two-step hybrid methods for y  = f (x, y). J. Comput. Appl. Math. 209(1), 33–53 (2007) 77. Van den Houwen, P.J., Sommeijer, B.P., Nguyen, H.C.: Stability of collocation-based Runge– Kutta–Nyström methods. BIT 31(3), 469–481 (1991) 78. Wright, K.: Some relationships between implicit Runge–Kutta, collocation and Lanczos τ -methods, and their stability properties. BIT 10, 217–227 (1970)

Chapter 4

Basic Methods for Computing Special Functions Amparo Gil, Javier Segura, and Nico M. Temme

Abstract This paper gives an overview of methods for the numerical evaluation of special functions, that is, the functions that arise in many problems from mathematical physics, engineering, probability theory, and other applied sciences. We consider in detail a selection of basic methods which are frequently used in the numerical evaluation of special functions: converging and asymptotic series, including Chebyshev expansions, linear recurrence relations, and numerical quadrature. Several other methods are available and some of these will be discussed in less detail. We give examples of recent software for special functions where these methods are used. We mention a list of new publications on computational aspects of special functions available on our website.

Keywords Numerical evaluation of special functions · Chebyshev expansions · Quadrature methods · Transformation of series · Continued fractions · Asymptotic analysis Mathematics Subject Classification (2000) 65D20 · 41A60 · 33C05 · 33C10 · 33C15 A. Gil Departamento de Matemática Aplicada y CC. de la Computación, ETSI Caminos, Universidad de Cantabria, 39005 Santander, Spain e-mail: [email protected] J. Segura () Departamento de Matemáticas, Estadística y Computación, Universidad de Cantabria, 39005 Santander, Spain e-mail: [email protected] N.M. Temme CWI, Science Park 123, 1098 XG Amsterdam, The Netherlands e-mail: [email protected] T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_4, © Springer Science+Business Media B.V. 2011

67

68

A. Gil et al.

4.1 Introduction For workers in the applied sciences the Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables [1], edited by Milton Abramowitz and Irene Stegun, and published in 1964 is usually the first source of information about the properties of special functions. It may be the most cited book in mathematics. These days the Handbook is being updated as a Digital Library of Mathematical Functions (DLMF), and will be freely accessible in a Web version. Other sources for collections of formulas for special functions on the web are Wolfram MathWorld1 and Wikipedia2 . These sources give many properties of special functions of which a number can be used for their numerical evaluation, sometimes with references to suitable algorithms. However, it is not always clear how to write efficient and reliable algorithms. The present paper gives an overview on numerical methods for special functions. It is based on our recent book [54] in which we consider four Basic Methods, namely 1. 2. 3. 4.

Convergent and divergent series. Chebyshev expansions. Linear recurrence relations and associated continued fractions. Quadrature methods.

In addition we give a selection of published algorithms for special functions. There are many other methods, which are also discussed in our book, and some of them will be discussed in this overview as well. For example, the use of differential equations will be discussed in connection with the Taylor expansion method for initial boundary problems. Our general point of view in connection with special functions is to remain modest in the number of parameters. It is possible to design straightforward algorithms for the generalized hypergeometric function  p Fq

  ∞ (a1 )n · · · (ap )n zn a 1 , . . . , ap ;z = , b 1 , . . . , bq (b1 )n · · · (bq )n n!

(4.1.1)

n=0

where p ≤ q + 1 and (a)n is the Pochhammer symbol, also called the shifted factorial, defined by (a + n) . (a) (4.1.2) Many special functions can be written in terms of this function, with the main cases given by p = 1, q = 1 (Kummer functions, with convergence of the series for all complex z) and p = 2, q = 1 (Gauss hypergeometric functions, with convergence if (a)0 = 1,

(a)n = a(a + 1) · · · (a + n − 1) (n ≥ 1),

1 http://mathworld.wolfram.com/. 2 http://en.wikipedia.org/.

(a)n =

4 Basic Methods for Computing Special Functions

69

|z| < 1). For efficient and reliable algorithms the series in (4.1.1) is only of limited use. Also, differential equations, especially those that arise in the physical sciences, are the reservoir that generates many special functions. One may define a general second order equation, make an algorithm for this equation, and expect that solutions of simpler equations follow from the general solution. However, very difficult problems may arise then. Consider as an example the equation d2 w(z) = (pz2 + qz + r)w(z), dz2

(4.1.3)

the solutions of which can be expressed in terms of the parabolic cylinder functions U and V , see [1, Chap. 19], which are solutions of the equation   1 2 d2 w(z) = + a w(z). (4.1.4) z 4 dz2 When p = r = 0 in (4.1.3) that equation reduces to the Airy equation, whereas the Airy functions are not special cases of the parabolic cylinder functions U and V (in the sense that Airy functions are Bessel functions for certain values of the order of these functions). A nontrivial limiting process with p → 0 and r → 0 is needed to get Airy functions from a linear combination of the solutions of (4.1.3). In its turn, the parabolic cylinder function U (a, z) is a special case of the confluent hypergeometric function (also called Kummer functions) U (a, c, z). We have two forms [1, p. 691]:   1 1 1 2 1 1 1 1 a + , , z2 U (a, z) = 2− 4 − 2 a e− 4 z U 2 4 2 2   1 3 3 1 2 − 34 − 12 a − 14 z2 =2 ze U a+ , , z . (4.1.5) 2 4 2 2 The first form suggests that the function U (a, z) is an even function of z, the second one that it is odd. The point is that this Kummer function is multi-valued, and the representation   √ − 1 − 1 a − 1 z2 1 a + 14 1 2 π2 4 2 e 4 2 ; 2z U (a, ±z) = 1 F1 1 ( 34 + 12 a) 2   √ 1 − 1 a − 1 z2 1 a + 34 1 2 π2 4 2 ze 4 2 ; 2z (4.1.6) ∓ 1 F1 3 ( 14 + 12 a) 2 gives a better insight. However, this form is extremely unstable for intermediate or large values of z. In our opinion it is important to have codes that can be used for a limited class of functions. In this sense we have written algorithms for conical functions μ P−1/2+iτ (x) [55] for real x, τ and μ, and not for Legendre functions of general

70

A. Gil et al.

complex degree. Also, we have written codes [51] for modified Bessel functions of purely imaginary order, that is for Kia (x) and a related function, and not a general code for Bessel functions with general complex order.

4.2 Convergent and Divergent Series Convergent series for special functions usually arise in the form of hypergeometric series, with as general form the one shown in (4.1.1). The series is easy to evaluate because of the recursion (a)n+1 = (a + n)(a)n , n ≥ 0, of the Pochhammer symbols in (4.1.2). For certain special function, for example for the modified Bessel function  Iν (z) =

1 z 2

ν  ∞ n=0

( 14 z2 )n = (ν + n + 1) n!



1 z 2

ν

 0 F1

− 1 2 ; z ν +1 4

 (4.2.1)

it gives a stable representation when z > 0 and ν ≥ 0 and it is an efficient representation when z is not large compared with ν. However, when we use this expansion in the representation of the other modified Bessel function 1 I−ν (z) − Iν (z) Kν (z) = π , 2 sin πν

(4.2.2)

it can be used only for small values of z. This is because of the cancellation of numerical digits, which can be seen from the asymptotic estimates  π −z ez , Kν (z) ∼ (4.2.3) e , z → ∞, Iν (z) ∼ √ 2z 2πz which is valid for fixed values of ν. There is another phenomenon when using combinations of hypergeometric functions. When ν is an integer, the form in (4.2.2) is well defined by a limiting process, but for numerical computations a special algorithm is needed. See [109], where it is shown that it is sufficient to treat the case ν ∼ 0 in detail and that the remaining integer values follow from recursion relations. The case for confluent hypergeometric functions is more complicated. We have for the function U (a, c, z) the representation   a 1+a−c 1 F1 ( 2−c ; z) 1 F1 ( c ; z ) π 1−c U (a, c, z) = −z , sin πc (1 + a − c)(c) (a)(2 − c)

(4.2.4)

and it is useful for small values of z. Consider c ∼ 0. We have

lim

c→0

a 1 F1 ( c ; z )

(1 + a − c)(c)

=

z 1 F1 ( a+1 ; z) 2 (a)

.

(4.2.5)

4 Basic Methods for Computing Special Functions

71

So, apart from this limit, another (simultaneous) limiting process for c → 0 needs to be controlled, and also the extra parameter a makes it more difficult to write a stable algorithm. A published algorithm seems not to be available for this case. For Gauss hypergeometric functions similar problems arise in the connection formulas, say the one writing a function with argument z as a linear combination of two functions with argument 1 − z. See [36] for numerical algorithms. Other instabilities occur when the parameters of the hypergeometric function become large and/or complex. For Gauss and Kummer hypergeometric functions many other convergent expansions are available, for example in terms of Chebyshev polynomials and of Bessel functions; see [75, Sects. 9.3.4, 9.4.1, 9.4.3]. For a different type of expansion in terms of Bessel functions, with an application to the parabolic cylinder functions, see [78].

4.2.1 Divergent Expansions With this we mean asymptotic expansions of the form F (z) ∼

∞  cn n=0

zn

,

z → ∞.

(4.2.6)

The series usually diverges, but it has the property F (z) =

N−1  n=0

cn + RN (z), zn

 RN (z) = O z−N , z → ∞,

(4.2.7)

for N = 0, 1, 2, . . . , and the order estimate holds for fixed N . This is the Poincarétype expansion and for special functions like the gamma and Bessel functions they are crucial for evaluating these functions. Other variants of the expansion are also important, in particular expansions that hold for a certain range of additional parameters (this leads to the uniform asymptotic expansions in terms of other special functions like Airy functions, which are useful in turning point problems). Usually the optimal choice of N with a given (large) z occurs for the N that makes cN /zN the smallest term. And usually the error estimate in (4.2.7) may be exponentially small for this N . Say, with z > 0, the smallest term is achieved when N ∼ z, then it may happen that RN (N ) = O(exp(−N )). Many asymptotic expansions for special functions share this property, and it makes asymptotic expansions very useful for numerical computations. Convergent power series may be very unstable in certain parts of their convergence region, as the expansion of e−z for z > 0. In a similar way, asymptotic expansions may be very useful in a certain sector of the complex z-plane, but may become useless in other sectors. Other expansions may be available in these sectors.

72

A. Gil et al.

For example, consider the compound expansion of the Kummer function 

R−1    −R 1 ez za−c  (c − a)n (1 − a)n a + O |z| ;z = 1 F1 c (c) (a) n! zn n=0



S−1  −S z−a e±iπa  (a)n (1 + a − c)n + , + O |z| (c − a) n! (−z)n

(4.2.8)

n=0

the upper sign being taken if − 12 π < ph z < 32 π , the lower sign if − 32 π < ph z < 1 z 2 π . When z > 0 the second term can be neglected because of e in front of the 1 first term. We see that within a sector properly inside the sector − 2 π < ph z < 12 π we can work with one expansion, and in a sector containing the negative z-axis with another one. In sectors containing the imaginary axis we need both expansions. The fact that an entire function, as is the Kummer function, does not have a unique asymptotic expansion valid for all phases of z will be explained in Sect. 4.6, where we discuss elements of the Stokes phenomenon. A remarkable point is in this example that we have, say for − 12 π + δ < ph z < 1 2 π − δ (δ a small positive number), not only one expansion, but also an expansion that gives an exponentially small correction to the main expansion. For computations (and also for applications in physics) this may give interesting information. The role of exponentially small terms in asymptotics has been discussed in great detail the last twenty years. For many aspects from a physicists’ point of view, we refer to The Devil’s Invention: Asymptotic, Superasymptotic and Hyperasymptotic Series [13] for a lively introduction to this topic.3 In Sect. 4.6 we also discuss aspects of the role of exponentially small terms.

4.3 Linear Recurrence Relations Many special functions of mathematical physics satisfy three-term recurrence relations. We first give a simple relation and discuss stability and direction of recursion, which elements are important in the general theory.

4.3.1 A Simple Recurrence Relation The recurrence relations fn = fn−1 − 3 The

xn , n!

gn = gn−1 +

xn , n!

n = 1, 2, . . . ,

(4.3.1)

“Devil’s invention” refers to a quote from Niels Hendrik Abel (1828), who claimed “Divergent series are the invention of the devil, and it is shameful to base on them any demonstration whatsoever.”

4 Basic Methods for Computing Special Functions

73

with initial values f0 = ex − 1, g0 = 1 have solutions ∞  xm , fn = m!

n  xm g n = e − fn = , m! x

m=n+1

(4.3.2)

m=0

which are in fact special cases of the incomplete gamma functions: fn = e x

γ (n + 1, x) , n!

gn = ex

(n + 1, x) , n!

n = 0, 1, 2, . . . .

(4.3.3)

Assume that x > 0. Then, following our intuition, the recursion for gn will not cause any problem, since two positive numbers are always added during the recursion. For the recurrence relation of fn it is not clear, but there is a potential danger owing to the subtraction of two positive quantities. Note that the computation of the initial value f0 , for small values of x, may produce a large relative error, when the quantities ex and 1 are simply subtracted. This problem repeats itself for each subsequent fn that is computed by using the recurrence relation: in each step the next term of the Taylor series is subtracted from the exponential function. Apparently, this is a hopeless procedure for computing successive fn (even when x is not small). On the other hand, the computation of successive gn does not show any problem. In the study of recurrence relations it may make sense to change the direction of the recursion. Writing the recursion for fn and gn in the backward direction: fn−1 = fn +

xn , n!

gn−1 = gn −

xn n!

(4.3.4)

then we note that for both solutions the roles are reversed: gn is obtained by subtraction, whereas fn is obtained by addition of positive numbers. In addition, limn→∞ fn = 0. It can be easily verified that both fn and gn satisfy the recurrence relation (n + 1)yn+1 − (x + n + 1)yn + xyn−1 = 0.

(4.3.5)

Again, this relation is stable for the computation of gn in the forward direction; it is stable for fn in the backward direction. Note that the solutions of this recursion satisfy fn → 0, gn → ex as n → ∞. Apparently, the solution which becomes ultimately small in the forward direction (small compared to the other solution), is the victim. A similar phenomenon occurs in the backward direction. This phenomenon will be explained and put in a general framework in the following section.

4.3.2 Some Elements of the General Theory For details on the theory of this topic we refer to [54, Chap. 4].

74

A. Gil et al.

Consider the recurrence relation yn+1 + bn yn + an yn−1 = 0,

n = 1, 2, 3, . . . ,

(4.3.6)

where an and bn are given, with an = 0. Equation (4.3.6) is also called a linear homogeneous difference equation of the second order. In analogy with the theory of differential equations, two linearly independent solutions fn , gn exist in general, with the property that any solution yn of (4.3.6) can be written in the form yn = Afn + Bgn ,

(4.3.7)

where A and B do not depend on n. We are interested in the special case that the pair {fn , gn } satisfies lim

n→∞

fn = 0. gn

(4.3.8)

Then, for any solution (4.3.7) with B = 0, we have fn /yn → 0 as n → ∞. When B = 0 in (4.3.7), we call yn a minimal solution; when B = 0, we call yn a dominant solution. When we have two initial values y0 , y1 , assuming that f0 , f1 , g0 , g1 are known as well, then we can compute A and B. That is, A=

g1 y0 − g0 y1 , f 0 g1 − f1 g0

B=

y 0 f 1 − y1 f 0 . g0 f1 − g1 f0

(4.3.9)

The denominators are different from 0 when the solutions fn , gn are linearly independent. When we assume that the initial values y0 , y1 are to be used for generating a dominant solution, then A may, or may not, vanish; B should not vanish: y0 f1 = y1 f0 . When however the initial values are to be used for the computation of a minimal solution, then the much stronger condition y0 f1 = y1 f0 should hold. It follows that, in this case, one and only one initial value can be prescribed and the other one follows from the relation y0 f1 = y1 f0 ; in other words, the minimal solutions, if it exists, is unique up to a constant multiplicative factor. In the numerical approach this leads to the well-known instability phenomena for the computation of minimal solutions. The fact is that, when our initial values y0 , y1 are not specified to an infinite precision,—and consequently B does not vanish exactly—the computed solution (4.3.7) always contains a fraction of a dominant solution gn . Hence, in the long run, our solution yn does not behave as a minimal solution, although we assumed that we were computing a minimal solution. This happens even if all further computations are done exactly. In applications it is important to know whether a given equation (4.3.6) has dominant and minimal solutions. Often this can be easily concluded from the asymptotic behavior of the coefficients an and bn . Assume that for large values of n the coefficients an , bn behave as follows: an ∼ anα ,

bn ∼ bnβ ,

ab = 0

(4.3.10)

4 Basic Methods for Computing Special Functions

75

with α and β real; assume that t1 , t2 are the zeros of the characteristic polynomial

(t) = t 2 + bt + a with |t1 | ≥ |t2 |. Then it follows from Perron’s theorem [54, p. 93] that we have the following results. 1. If β > 12 α then the difference equation (4.3.6) has two linearly independent solutions yn,1 and yn,2 , with the property yn+1,1 ∼ −bnβ , yn,1

yn+1,2 a ∼ − nα−β , yn,2 b

n → ∞.

(4.3.11)

2. If β = 12 α and |t1 | > |t2 |, then the difference equation (4.3.6) has two linear independent solutions yn,1 and yn,2 , with the property yn+1,1 ∼ t1 nβ , yn,1

yn+1,2 ∼ t2 nβ , yn,2

n → ∞.

(4.3.12)

3. If β = 12 α and |t1 | = |t2 |, or if β < 12 α, then some information is still available, but the theorem is inconclusive with respect to the existence of minimal and dominant solutions.

4.3.3 Miller’s Algorithm This algorithm can be used for calculating a sequence f0 ,

f1 ,

...,

fN

(4.3.13)

of values of a minimal solution that satisfies (4.3.6); N is a non-negative integer. Such sequences frequently occur in expansions of special functions; see for example the expansions in terms of Chebyshev polynomials in (4.4.16). When we use (4.3.6) in the backward direction we may start with two initial values fN and fN−1 . But these are perhaps difficult to obtain. Miller’s algorithm does not need these values, and uses a smart idea for the computation of the required sequence (4.3.13). The algorithm works for many interesting cases and gives an efficient method for computing such sequences. Assume we have a relation of the form S=

∞ 

λn f n ,

S = 0.

(4.3.14)

n=0

The series should be convergent and λn and S should be known. The series in (4.3.14) plays a role in normalizing the required minimal solution. The series may be finite; we only require that at least one coefficient λn with n ≤ N is different from zero. When just one coefficient, say λn , is different from zero, we assume that the value fn is available.

76

A. Gil et al.

Table 4.1 Computing the modified Bessel functions In (x) for x = 1 by using (4.3.17) in the backward direction. The underlined digits in the third column are correct

n

yn before normalization

. yn = In (1) after normalization

0

2.2879 49300 10+8

1.26606 587801 10−0

1

1.0213 17610 10+8

5.65159 104106 10−1

2

2.4531

1.35747 669794 10−1

3

4.0061

4

4.9434

5

4.9057

6

4.0640

7

2.8900

8

1.8000

9

1.0000

10

0.0000

40800 10+7 29000 10+6 00000 10+5 00000 10+4 00000 10+3 00000 10+2 00000 10+1 00000 10+0 00000 10+0

2.21684 249288 10−2 2.73712 022160 10−3 2.71463 156012 10−4 2.24886 614761 10−5 1.59921 829887 10−6 9.96052 919710 10−8 5.53362 733172 10−9 0.00000 000000 10−0

(ν)

In Miller’s algorithm a starting value ν is chosen, ν > N , and a solution {yn } of (4.3.6) is computed with the false initial values (ν)

yν+1 = 0,

yν(ν) = 1.

(4.3.15)

The right-hand sides may be replaced by other values; at least one value should be different from zero. In some cases a judicious choice of these values may improve the convergence of the algorithm. The computed solution yn , with (4.3.15) as initial values, is a linear combination of the solutions fn and gn , gn being a dominant solution. When we choose ν large . enough it follows that the wanted solution fn satisfies fn = ρyn , n = 0, 1, . . . , N , because the dominant solution gn can be neglected in the backward direction. For details and proofs we refer to [37] and [54, Sect. 4.6]. The number ρ then follows from the normalizing sum (4.3.14). That is, ν . 1 λn yn . ρ= S

(4.3.16)

n=0

In [11] the above method was introduced for computing the modified Bessel functions In (x). The recurrence relation for these functions reads In+1 (x) +

2n In (x) − In−1 (x) = 0. x

(4.3.17)

A normalizing condition (4.3.14) is ex = I0 (x) + 2I1 (x) + 2I2 (x) + 2I3 (x) + · · · .

(4.3.18)

That is, S = ex , λ0 = 1, λn = 2 (n ≥ 1). We take x = 1 and initial values (4.3.15) with ν = 9 and obtain the results given in Table 4.1.

4 Basic Methods for Computing Special Functions

77

The rightmost column in Table 4.1 is obtained by dividing the results of the middle column by . 1 ρ= λn yn(9) = 1.807132898610+8 . e 9

(4.3.19)

n=0

When we take N = 5, which means we want to compute the sequence I0 (1), I1 (1), . . . , I5 (1), we see that these quantities are computed with at least 10 correct decimal digits.

4.3.4 Examples of Hypergeometric Functions and Recursions We mention classes of functions of hypergeometric type that are of interest for applications and give a few details about their recursions. 4.3.4.1 Bessel Functions In the case of ordinary Bessel functions, we have the recurrence relation yn+1 −

2n yn + yn−1 = 0, z

z = 0,

(4.3.20)

with solutions fn = Jn (z),

gn = Yn (z).

(4.3.21)

This is covered by (4.3.11), with a = 1,

α = 0,

2 b=− , z

β = 1.

(4.3.22)

In this case fn+1 z ∼ , fn 2n

gn+1 2n ∼ . gn z

The known asymptotic behavior of the Bessel functions reads     1 z n (n − 1)! 2 n , gn ∼ − , n → ∞. fn ∼ n! 2 π z

(4.3.23)

(4.3.24)

Similar results hold for the modified Bessel functions, with recurrence relation yn+1 +

2n yn − yn−1 = 0, z

z = 0,

(4.3.25)

with solutions In (z) (minimal) and Kn (z) (dominant). There is an extensive literature on the use of recursion for evaluating Bessel functions, with [37] as pioneering paper; see also [5, 70, 101].

78

A. Gil et al.

4.3.4.2 Kummer Functions The Kummer functions (or confluent hypergeometric functions) 1 F1 and U do not satisfy the same recurrence relations, but by multiplying them with certain gamma functions they do. We assume z > 0. An overview of the relevant recurrence relations can be found in [1, Chap. 13]. Recursion with respect to a. The functions   (a + n) a+n ;z and 1 F1 c (a + n + 1 − c)

(a + n) U (a + n, c, z) (a)

are respectively dominant and minimal. Recursion with respect to c. The functions   (c − a + n) a ;z and U (a, c + n, z) 1 F1 c+n (c + n)

(4.3.26)

(4.3.27)

are respectively minimal and dominant. There are other interesting cases: recursion with respect to both a and c, and recursion with respect to negative n. All the possible cases are analyzed in [103], where it is shown that the Kummer recurrences always have a minimal solution except for the case of recursion over a when z is real and positive (for a → −∞) or negative real (for a → +∞). See also [30] and [54, Sect. 4.5.1].

4.3.4.3 Gauss Hypergeometric Functions The recursions for the functions   a + 1 n, b + 2 n ;z , 2 F1 c + 3 n

(4.3.28)

where j = 0, ±1, not all equal to zero, and z is complex are analyzed in [53, 59]. Of the 27 nontrivial cases, only a limited set of these recursions need to be considered. This is because of several relations between contiguous Gauss functions. Among other results, in [59] it is shown that the function (4.3.28) is minimal around z = 0 when 3 > 0. An overview of the relevant recurrence relations can be found in [1, Chap. 15].

4.3.4.4 Legendre Functions For definitions and properties, see [1, Chap. 8] and [111, Chap. 8]. Legendre functions are special cases of Gauss hypergeometric functions, but the recursions need μ special attention. When z > 0, z ∈ / (0, 1], Pν (z) is the minimal solution of the μ recursion with respect to positive order μ; Qν (z) is dominant. Particular cases

4 Basic Methods for Computing Special Functions

79 μ

are toroidal functions and conical functions. The latter have the form P−1/2+iτ (z), μ Q−1/2+iτ (z), which are real for z > −1 and real τ and μ. μ For recursion with respect to the degree ν, Qν (z) is a minimal solution and μ Pν (z) is dominant. For further details on numerical aspects and algorithms we refer to [41–44, 46, 55] and [54, Sect. 12.3].

4.3.4.5 Coulomb Wave Functions Information on these functions can be found in [1, Chap. 14]. Coulomb wave functions are special cases of the Kummer functions, and they can also be viewed as generalizations of Bessel functions. The regular function Fλ (η, ρ) is the minimal solution with respect to increasing λ, while the irregular Gλ (η, ρ) function is a dominant one. Algorithms based on recursions are discussed in [85]; in [96, 97] several types of series expansions are considered, with references to earlier algorithms.

4.3.4.6 Parabolic Cylinder Functions For definitions and properties, see [1, Chap. 19]. The standard forms are U (a, z) and V (a, z), and, again, special cases of the Kummer functions. The function U (a, x) is minimal in the forward a-recursion. For negative values of a the situation is quite different, and for |a| large enough (a −z2 /4), the solutions are neither minimal nor dominant. See [54, p. 102]. Algorithms using recursion can be found in [57, 58, 95, 102].

4.4 Chebyshev Expansions Chebyshev expansions are examples of convergent expansions, considered earlier, but because of their special properties they deserve a separate discussion. Chebyshev polynomials of the first kind Tn (x) have the nice property Tn (cos θ ) = cos(nθ), giving an equal ripple in the θ -interval [0, π] and in the x-interval [−1, 1]. Because of their excellent convergence properties, Chebyshev expansions may replace convergent power series and divergent asymptotic expansions, or they may be used for filling the gap between the domains where convergent and asymptotic expansions can be used. The standard Chebyshev expansion is of the form f (x) =

∞  n=0

cn Tn (x),

−1 ≤ x ≤ 1,

(4.4.1)

80

A. Gil et al.

where the prime means that the first term is to be halved. Provided that the coefficients ck decrease in magnitude sufficiently rapidly, the error made by truncating the Chebyshev expansion after the terms k = n, that is, En (x) =

∞ 

ck Tk (x),

(4.4.2)

. En (x) = cn+1 Tn+1 (x),

(4.4.3)

k=n+1

will be given approximately by

that is, the error approximately satisfies the equioscillation property, which is happening in best-approximation (mini-max) methods.

4.4.1 Clenshaw’s Summation Method There is a very simple algorithm due to Clenshaw [24] for evaluating the finite sum  1 Sn (x) = c0 + ck Tk (x), 2 n

(4.4.4)

k=1

which is based on the recurrence relation xTk (x) =

1 Tk+1 (x) + T|k−1| (x) . 2

(4.4.5)

The algorithm computes a sequence b1 , b2 , . . . , bn+1 and starts with putting bn+1 = 0 and bn = cn . Next, bk = 2xbk+1 − bk+2 + ck ,

k = n − 1, n − 2, . . . , 1.

(4.4.6)

Then, Sn (x) = xb1 − b2 + 12 c0 .

4.4.2 Methods for Obtaining the Coefficients The coefficients cn in the expansion in (4.4.1) can be obtained in several ways, and we mention a few elements of the main methods. For details we refer to [54, Chap. 3].

4 Basic Methods for Computing Special Functions

81

4.4.2.1 Tabled Coefficients In the case that the function f is an elementary or a one-variable special function, such as the error function erf x, the Bessel function J0 (x), and so on, the coefficients can be obtained from tables, see [75]. Usually 20D accuracy of the coefficients is given. In [93] 30D coefficients are given for the error function and the complementary error function. Nowadays, computer algebra systems can be used to obtain tables of high precision coefficients.

4.4.2.2 Discretizing the Integral A numerical method uses discretization of the integral representation. That is, ck =

2 π



1 −1

f (x)Tk (x) 2 dx = √ 2 π 1−x



π

f (cos θ ) cos(kθ ) dθ,

(4.4.7)

0

and discretization gives   n πj πkj . 2 

f cos cos , ck = n n n

(4.4.8)

j =0

where the primes mean that the first and last terms is are to be halved. This is a discrete cosine transform, which can be computed by methods based on the fast Fourier transform [116].

4.4.2.3 Clenshaw’s Method This method can be used for functions satisfying linear ordinary differential equations with polynomial coefficients of the form m 

pk (x)f (k) (x) = h(x),

(4.4.9)

k=0

with pk polynomials and where the coefficients of the Chebyshev expansion of the function h are known. The idea is as follows. Next to the expansion in (4.4.1), we introduce expansions for the relevant derivatives: f (s) (x) =

∞ 

(s) cn Tn (x),

s = 0, 1, 2, . . . ,

(4.4.10)

n=0

and from known properties of the Chebyshev polynomials we have (s+1)

(s+1)

2rcr(s) = cr−1 − cr+1 ,

r ≥ 1.

(4.4.11)

82

A. Gil et al.

A next element in Clenshaw’s method is to handle the powers of x occurring in the differential equation satisfied by f . For this we need the relation (4.4.5) and formulas for higher powers of x in the left-hand side. These can easily be obtained from the above one. By substituting the expansion in (4.4.1) in the equation in (4.4.9) and using the (s) formulas for the powers of x, a set of recursions for the coefficients cr can be obtained. Together with boundary values (or other known relations) this set of recursions can be solved numerically, and Clenshaw [25, 26] explained how this can be done by using a backward recursion method. For example, the exponential function y(x) = eax satisfies the differential equa(1) tion y = ay. Substituting expansions following from (4.4.1) we obtain cr = acr . (1) Using this in the form cr+2 = acr+2 and using (4.4.11) we have the recursion for the coefficients: 2(r + 1)rcr+1 = a(cr − cr+2 ),

r ≥ 0.

(4.4.12)

This is the recursion relation for the modified Bessel function Ir (a), and so cr is multiple of this function (the other modified Bessel function Kr (a) being excluded because of its behavior for large r). The value y(0) = 1 gives cr = Ir (a). This result is known because of the expansion ea cos θ =

∞ 

Ir (a) cos(rθ ).

(4.4.13)

r=0

It is not at all needed that we know the solution in terms of a known function; for numerical purposes it is enough to have (4.4.12), and to use a backward recursion scheme, the Miller algorithm, as explained in Sect. 4.3.3. However, several questions arise in this successful method. The recursion given in (4.4.12) is very simple, and we can find its exact solution. In more complicated (s) recursion schemes obtained for the coefficients cr this information is not available. The scheme may be of large order and may have several solutions of which the asymptotic behavior is unknown. So, in general, we don’t know if Clenshaw’s method for differential equations computes the solution that we want, and if for the wanted solution the scheme is stable in the backward direction. Clenshaw’s method goes wrong in another simple example. Consider y(x) = 2 eax+bx with differential equation y = (a + 2bx)y. It is again easy to give a recur(s) sion scheme for the coefficients cr . It reads (we use also (4.4.5))  cr(1) = acr + b cr+1 + c|r−1| ,

r ≥ 0.

(4.4.14)

(1)

The coefficient cr can be eliminated simply by using this relation with r replaced with r + 2, and invoking (4.4.11). This gives 2(r + 1)cr+1 = a(cr − cr+2 ) + b(cr+1 − cr+3 ),

r = 0, 1, 2, . . . .

(4.4.15)

4 Basic Methods for Computing Special Functions

83

When applying a backward recursion algorithm for computing cr it appears that the solution does not give the requested function y(x). The problem is that the recurrence is a third order difference equation, with three independent solutions that can be chosen in such a way that two of them are minimal solutions while the third one is dominant. Straightforward application of Miller’s backward algorithm explained in Sect. 4.3.3 gives a linear combination of such two minimal solutions. There are modifications of the Miller algorithm, which can be used for obtaining the requested minimal solution, eliminating the contamination introduced by the unwanted minimal solution. In [56] a similar phenomenon has been discussed for the computation of the Weber function W (a, x), solution of the equation y

+ (x 2 /4 − a)y = 0. In that paper we describe a modification of Miller’s algorithm in detail. See also [77] for an instability problem in Chebyshev expansions for special functions.

4.4.2.4 Known Coefficients in Terms of Special Functions As we have seen in (4.4.13), the coefficients in a Chebyshev expansion for the exponential function are known in terms of special functions. There are many other examples, also for higher transcendental functions. For example, we have (see [75, p. 37]) J0 (ax) =

∞ 

n (−1)n Jn2 (a/2) T2n (x),

n=0 ∞  J1 (ax) = 2 (−1)n Jn (a/2)Jn+1 (a/2) T2n+1 (x),

(4.4.16)

n=0

where −1 ≤ x ≤ 1 and 0 = 1, n = 2 if n > 0. The parameter a can be any complex number. Similar expansions are available for J -Bessel functions of any complex order, in which the coefficients are 1 F2 -hypergeometric functions, and explicit recursion relations are available for computing the coefficients. For general integer order, the coefficients are products of two J -Bessel functions, as in (4.4.16). See again [75]. Another example is the expansion for the error function, e

a2 x 2

erf(ax) =

√

πe

1 2 2a

∞  n=0

 In+ 1

2

 1 2 a T2n+1 (x), 2

−1 ≤ x ≤ 1,

(4.4.17)

in which the modified Bessel function is used. Again, a can be any complex number. The complexity of computing the coefficients of the expansions in (4.4.16) seems to be greater than the computation of the function that has been expanded. In some sense this is true, but the coefficients in (4.4.16), and those of many other examples for special functions, satisfy linear recurrence relations, and the coefficients satisfying such relations can usually be computed very efficiently by the backward recursion algorithm; see Sect. 4.3.3.

84

A. Gil et al.

The expansions in (4.4.16) and (4.4.17) can be viewed as expansions near the origin. Other expansions are available that can be viewed as expansions at infinity, and these may be considered as alternatives for asymptotic expansions of special functions. For example, for the confluent hypergeometric U -functions we have the convergent expansion in terms of shifted Chebyshev polynomials Tn∗ (x) = Tn (2x − 1): (ωz) U (a, c, ωz) = a

∞ 

Cn (z)Tn∗ (1/ω),

(4.4.18)

n=0

where 3 |ph z| < π, 1 ≤ ω ≤ ∞. (4.4.19) 2 Furthermore, a, 1 + a − c = 0, −1, −2, . . . . When equalities hold for these values of a and c, the Kummer U -function reduces to a Laguerre polynomial. This follows from z = 0,

U (a, c, z) = z1−c U (1 + a − c, 2 − c, z)

(4.4.20)

and U (−n, α + 1, z) = (−1)n n!Lαn (z),

n = 0, 1, 2, . . . .

(4.4.21)

The expansion (4.4.18) is given in [75, p. 25]. The coefficients can be represented in terms of generalized hypergeometric functions, in fact, Meijer G-functions, and they can be computed from the recurrence relation 2Cn (z) = 2(n + 1)A1 Cn+1 (z) + A2 Cn+2 (z) + A3 Cn+3 (z),

n

(4.4.22)

where b = a + 1 − c, 0 = 12 , n = 1 (n ≥ 1), and A1 = 1 −

(2n + 3)(n + a + 1)(n + b + 1) 2z − , 2(n + 2)(n + a)(n + b) (n + a)(n + b)

A2 = 1 −

2(n + 1)(2n + 3 − z) , (n + a)(n + b)

A3 = −

(4.4.23)

(n + 1)(n + 3 − a)(n + 3 − b) . (n + 2)(n + a)(n + b)

For applying the backward recursion algorithm it is important to know that ∞  (−1)n Cn (z) = 1, n=0

3 |ph z| < π. 2

(4.4.24)

This follows from lim (ωz)a U (a, c, ωz) = 1 and Tn∗ (0) = (−1)n .

ω→∞

(4.4.25)

4 Basic Methods for Computing Special Functions

85

The standard backward recursion scheme (see Sect. 4.3) for computing the coefficients Cn (z) works only for |ph z| < π , and for ph z = ±π a modification seems to be possible; see [75, p. 26]. Although the expansion in (4.4.18) converges for all z = 0 in the indicated sector, it is better to avoid small values of the argument of the U -function. Luke gives an estimate of the coefficients Cn (z) of which the dominant factor that determines the speed of convergence is given by

2 1 3 3 Cn (z) = O n2(2a−c−1)/3 e−3n z ,

n → ∞,

(4.4.26)

and we see that large values of z1/3 improve the convergence. The expansion in (4.4.18) can be used for all special cases of the Kummer U -function, that is, for Bessel functions (Hankel functions and K-modified Bessel function), for the incomplete gamma function (a, z), with special cases the complementary error function and exponential integrals. In [54, Sect. 3.10] numerical coefficients are derived for expansions of the Airy function Ai(x) for x ≥ 7 and for its derivative by using the expansion in (4.4.18).

4.5 Quadrature Methods We start with a simple example in which an oscillatory integral can be transformed into a stable representation. Consider the integral ∞ √ 2 2 e−t +2iλt dt = π e−λ . (4.5.1) F (λ) = −∞

Taking λ = 10 we get . F (λ) = 0.6593662990 10−43 .

(4.5.2)

When we ask a well-known computer algebra system to do a numerical evaluation of the integral, without using the exact answer in (4.5.1) and with standard 10 digits accuracy, we obtain . F (λ) = 0.24 10−12 . (4.5.3) We see that in this way this simple integral, with strong oscillations, cannot be evaluated correctly. Increasing the accuracy from 10 to 50 digits we obtain the answer . F (λ) = 0.65936629906 10−43 , (4.5.4) the first 8 digits being correct. Observe that we can shift the path of integration upwards until we reach the point t = iλ, the saddle point, and we write ∞ ∞ 2 2 2 2 F (λ) = e−(t−iλ) −λ dt = e−λ e−s ds. (4.5.5) −∞

−∞

86

A. Gil et al.

Now the saddle point is at s = 0, we integrate through this point along a path where 2 no oscillations occur, a steepest descent path. Moreover the small factor e−λ that causes the main problems in the standard quadrature method, is now in front of the s-integral. Similar methods can be applied to more complicated functions, in particular to a wide class of special functions from mathematical physics. Much software has been developed for many of these functions, but for large parameters the software is not at all complete and reliable, in particular when the parameters are large and complex. We have come to the conclusion that methods based on asymptotic analysis are important for evaluating integrals by quadrature. Choosing suitable paths of integration and scaling the functions by separating dominant factors are important steps in these methods. In this section we discuss the application of a simple quadrature rule, namely, the trapezoidal rule. For integral representations of special functions it may perform very well. Not always the standard integral representations should be taken, but modifications obtained by transformations or by choosing contours in the complex plane.

4.5.1 The Trapezoidal Rule Gauss quadrature is a well-known quadrature method for evaluating integrals. It has a very good performance for various types of integrals over real intervals, given that the quadrature has maximal degree of exactness. However, one of the drawbacks is that it is not very flexible in algorithms when we want adjustable precision or when additional parameters are present. Also, we need zeros and weights of a certain class of orthogonal polynomials. For high precision algorithms computing these numbers in advance may be time consuming and/or not reliable. The n-point extended trapezoidal rule a

b

 1 f (t) dt = h[f (a) + f (b)] + h f (h j ) + Rn , 2 n−1 j =1

h=

b−a , n

(4.5.6)

is more flexible, because we don’t need precomputed zeros and weights; for this rule these numbers are trivial. The error term has the form Rn = −

1 (b − a) h2 f

(ξ ), 12

(4.5.7)

for some point ξ ∈ (a, b), and for functions with continuous second derivative. More insight in the error term follows from the Euler-Maclaurin summation rule [54, p. 131]. This rule gives the representation (for functions f having 2m + 2

4 Basic Methods for Computing Special Functions Table 4.2 The remainder Rn of the rule in (4.5.9) for several choices of n

87

n

Rn

4

−0.12 10−0

8

−0.48 10−6

16

−0.11 10−21

32

−0.13 10−62

64

−0.13 10−163

128

−0.53 10−404

continuous derivatives in [a, b]): Rn =

m  B2j 2j  (2j −1) (a) − f (2j −1) (b) h f (2j )! j =0

− (b − a)h2m+2

B2m+2 f (2m+2) (ξ ), (2m + 2)!

(4.5.8)

for some point ξ ∈ (a, b). Bm are the Bernoulli numbers. The first numbers with 1 even index are B0 = 1, B2 = 16 , B4 = − 30 . We take as an example the Bessel function π J0 (x) = 0

π

cos(x sin t) dt = h + h

n−1 

  cos x sin(h j ) + Rn ,

(4.5.9)

j =1

where h = π/n, and use this rule for x = 5. The results of computations are shown in Table 4.2. We observe that the error Rn is much smaller than the upper bound that can be obtained from (4.5.7). The explanation comes from the periodicity in the integral for the Bessel function. Hence, all terms of the sum in (4.5.8) vanish, and we infer that now the error is O(h2m+2 ). And because for this integral this is true for any positive m, we conclude that the error is exponentially small as a function of h. Another example is the Bessel function integral for general order 1 ex sinh t−νt dt, (4.5.10) Jν (x) = 2πi C where C starts at ∞ − iπ and terminates at ∞ + iπ ; see [111, p. 222] and [118, p. 176]. Without further specifications, on such contour oscillations will occur, but we will select a special contour that is free of oscillations for the case 0 < x ≤ ν. This contour will run through a saddle point of the integrand. In particular when the parameters are large, strong oscillations occur on a general path, and numerical quadrature will be very unstable. When we select the contour that is free of oscillations we are also able to pick up the most relevant part in the asymptotic representation of this Bessel function.

88

A. Gil et al.

We write ν = x cosh μ, μ ≥ 0. The real saddle points of x sinh t − νt = x(sinh t − t cosh μ) occur at t = ±μ, and at these saddle points the imaginary part of x sinh t − νt equals zero. It is possible to select a path free of oscillations (a steepest descent path) through the saddle point at t = μ (this is not possible when we would have selected the saddle point at t = −μ). This path can be described by the equation (x sinh t − νt) = 0. Writing t = σ + iτ we obtain for the path the equation τ , sin τ

μ ≤ σ, −π < τ < π.

(4.5.11)

(x sinh t − νt) = x(sinh σ cos τ − σ cosh μ).

(4.5.12)

cosh σ = cosh μ On this path we have

Integrating with respect to τ , using dt/dτ = (dσ/dτ + i) (where dσ/dτ is an odd function of τ , and does not contribute), we obtain π 1 ex(sinh σ cos τ −σ cosh μ) dτ, 0 < x ≤ ν. (4.5.13) Jν (x) = 2π −π The integrand is analytic and vanishes with all its derivatives at the points τ = ±π . We can interpret the integrand as being a periodic C ∞ function with period 2π , and consider representation (4.5.8) of the remainder. Again, the error in the trapezoidal rule is exponentially small. When ν  x the Bessel function becomes very small and we can take the dominant part ex(sinh μ−μ cosh μ) in front of the integral. When x ≥ ν (the oscillatory case), the Bessel function can be represented in a similar way, now by using two integrals (coming from the Hankel functions). Integral representations in terms of non-oscillating integrals (starting from complex contours) can be obtained for many of other special functions with large parameters. In other cases it may be difficult to obtain a suitable parametrization of the path, in which case it still may be possible to choose a path through a relevant saddle point and running into the valleys of the saddle point. In that case the oscillations will be less harmful compared with a path not going through a saddle point. For more details we refer to [54, Sect. 5.5]. Our main conclusion of this section is that the trapezoidal rule may be very efficient and accurate when dealing with a certain class of integrals. Smoothness and periodicity of the integrand are the key properties here. 4.5.1.1 The Trapezoidal Rule on R In the previous section we considered integrals over finite intervals. For integrals over R the trapezoidal rule may again be very efficient and accurate. We consider ∞ ∞  f (t) dt = h f (hj + d) + Rd (h), (4.5.14) −∞

j =−∞

4 Basic Methods for Computing Special Functions Table 4.3 The remainder R0 of the rule in (4.5.19) for several choices of h

89

h

j0

R0 (h)

1

2

−0.18 10−1

1/2

5

−0.24 10−6

1/4

12

−0.65 10−15

1/8

29

−0.44 10−32

1/16

67

−0.19 10−66

1/32

156

−0.55 10−136

1/64

355

−0.17 10−272

where h > 0 and 0 ≤ d < h. We apply this rule with even functions f analytic in a strip Ga of width 2a > 0 around R: Ga = {z = x + iy | x ∈ R, −a < y < a},

(4.5.15)

which are bounded in Ga and for which limx→±∞ f (x + iy) = 0 (uniformly in |y| ≤ a) and ∞ |f (x + ia)| dx < ∞. (4.5.16) Ma (f ) = −∞

Then, for real functions f , the remainder Rd (h) of (4.5.14) satisfies |Rd (h)| ≤

e−πa/ h Ma (f ). sinh(πa/ h)

(4.5.17)

The proof is based on residue calculus; see [54, p. 151]. As an example we consider the modified Bessel function 1 K0 (x) = 2



∞

−∞

e−x cosh t dt.

(4.5.18)

We have, with d = 0, ∞

 1 ex K0 (x) = h + h e−x(cosh(hj )−1) + R0 (h). 2

(4.5.19)

j =1

For x = 5 and several values of h we obtain the results given in Table 4.3 (j0 denotes the number of terms used in the series in (4.5.19)). We see in this example that halving the value of h gives a doubling of the number of correct significant digits (and, roughly speaking, a doubling of the number of terms needed in the series). When programming this method, observe that when halving h, previous function values can be used.

90

A. Gil et al.

4.5.2 Complex Contours In Sect. 4.5.1 we have transformed the complex contour integral (4.5.10) for the Bessel function Jν (x) into a more suitable integral (4.5.13) by using saddle point methods. Here we explain how this works for the Airy function for applying the trapezoidal rule on the real line of Sect. 4.5.1.1. This gives a very flexible and efficient algorithm with adjustable precision. We consider 1 3 1 e 3 w −zw dw, (4.5.20) Ai(z) = 2πi C where ph z ∈ [0, 23 π] and C is a contour starting at ∞e−iπ/3 and terminating at ∞e+iπ/3 (in the valleys of the integrand). Let 1 (4.5.21) φ(w) = w 3 − zw. 3 √ The saddle points are w0 = z and −w0 and follow from solving φ (w) = w 2 − z = 0. The saddle point contour (the path of steepest descent) that runs through the saddle point w0 is defined by [φ(w)] = [φ(w0 )].

(4.5.22)

We write w = u + iv,

z = x + iy = reiθ ,

w0 = u0 + iv0 .

(4.5.23)

Then u0 =

√ 1 r cos θ, 2

v0 =

√

1 r sin θ, 2

x = u20 − v02 ,

y = 2u0 v0 . (4.5.24)

The path of steepest descent through w0 is given by the equation u = u0 +

(v − v0 )(v + 2v0 )  , 3[u0 + 13 (v 2 + 2v0 v + 3u20 )]

−∞ < v < ∞.

(4.5.25)

Examples for r = 5 and a few θ -values are √ shown in Fig. 4.1. The relevant saddle points are located on the circle with radius r and are indicated by small dots. The saddle point on the positive real axis corresponds with the case θ = 0 and the two saddles on the imaginary axis with the case θ = π . This is out of the range of present interest, but it is instructive to see that the contour may split up and run through both saddle points ±w0 . When θ = 23 π both saddle points are on one path, and the half-line in the z-plane corresponding with this θ is called a Stokes line (see Sect. 4.6).

4 Basic Methods for Computing Special Functions

91

Fig. 4.1 Saddle point contours for θ = 0, 13 π, 23 π, π and r = 5

Integrating with respect to τ = u − u0 (and writing σ = v − v0 ) we obtain e−ζ Ai(z) = 2πi





∞

−∞

e

ψr (σ,τ )

 dσ + i dτ, dτ

(4.5.26)

3

where ζ = 23 z 2 , and σ=

τ (τ + 3v0 )  , 3[u0 + 13 (τ 2 + 4v0 τ + 3r)]

−∞ < τ < ∞,

1 ψr (σ, τ ) = [φ(w) − φ(w0 )] = u0 (σ 2 − τ 2 ) − 2v0 σ τ + σ 3 − σ τ 2 . 3

(4.5.27)

(4.5.28)

Quadrature methods for evaluating complex Airy functions can be found in [39, 47, 48, 50, 94].

4.6 The Stokes Phenomenon The Stokes phenomenon concerns the abrupt change across certain rays in the complex plane, known as Stokes lines, exhibited by the coefficients multiplying exponentially subdominant terms in compound asymptotic expansions. There is much recent interest in the Stokes phenomenon, and it fits in the present paper because it has to do with sudden changes in approximations when a certain parameter (in this case the phase of the large parameter) passes critical values.

92

A. Gil et al.

4.6.1 The Airy Function First we explain this phenomenon by using a simple example from differential equations. Consider Airy’s equation d 2w = z w, dz2

(4.6.1)

the solutions of which are entire functions. When |z| is large the solutions of (4.6.1) are approximated by linear combinations of 2 ξ = z3/2 . 3

1

w± = z− 4 e±ξ ,

(4.6.2)

Obviously, w± are multivalued functions of the complex variable z with a branch point at z = 0. Therefore, as we go once around the origin, the solutions of (4.6.1) will return to their original values, but w± will not. It follows that the constants c± in the linear combination w(z) ∼ c− w− (z) + c+ w+ (z),

z → ∞,

(4.6.3)

are domain-dependent. The constants change when we cross certain lines, the boundaries of certain sectors in the z-plane. In the above example one of the terms eξ , e−ξ maximally dominates the other one at the rays ph z = 0, ph z = ±2π/3. In this example these 3 rays are the Stokes lines. At the rays ph z = ± 13 π and the negative z-axis the quantity ξ is purely imaginary, and, hence, the terms eξ , e−ξ are equal in magnitude. These three rays are called the anti-Stokes lines4 . For the Airy function Ai(z) we have the full asymptotic expansion (see [1, Chap. 10]) 1

Ai(z) ∼ c− z− 4 e−ξ

∞ 

(−1)n cn ξ −n ,

c− =

n=0

1 −1 π 2 , |ph z| < π, 2

(4.6.4)

with coefficients cn =

(3n + 12 ) 54n n!(n + 12 )

,

n = 0, 1, 2, . . . .

On the other hand, in another sector of the z-plane, we have

 ∞ ∞   − 14 −ξ n −n ξ −n (−1) cn ξ + ie cn ξ e , Ai(−z) ∼ c− z n=0 4 This

(4.6.5)

(4.6.6)

n=0

terminology is not the same in all branches of applied mathematics and mathematical physics: sometimes one sees a complete interchange of the names ‘Stokes line’ and ‘anti-Stokes line’.

4 Basic Methods for Computing Special Functions

93

in which exactly the same term (with the same constant c− ) is involved as in (4.6.4), and there is another term corresponding with w+ . We can rewrite this in a more familiar expansion    ∞ c2n 1 − 12 − 14 (−1)n 2n Ai(−z) ∼ π z sin ξ + π 4 ξ n=0    ∞ 1 n c2n+1 − cos ξ + π (−1) 2n+1 , (4.6.7) 4 ξ n=0

valid in the sector |ph z| < 23 π . In the overlapping domain of expansions (4.6.4) and (4.6.7), that is, when 13 π < |ph z| < π , the term with w+ is asymptotically small compared with w− , and it suddenly appears in the asymptotic approximation when we cross with increasing values of |ph z| the Stokes lines at ph z = ± 23 π . It seems that, when going from (4.6.4) to (4.6.6), the constant multiplying w+ changes discontinuously from zero values (when |ph z| < 23 π ) to a non-zero value when we cross the Stokes line. This sudden appearance of the term w+ does not have much influence on the asymptotic behavior near the Stokes lines at |ph z| = 23 π , because w+ is dominated maximally by w− at these rays. However, see Sect. 4.6.3 below. Observe also the special value θ = 23 π in Sect. 4.5.2.

4.6.2 The Recent Interest in the Stokes Phenomenon This phenomenon of the discontinuity of the constants was discovered by Stokes and was discussed by him in a series of papers (on Airy functions in 1857, on Bessel functions in 1868). It is a phenomenon which is not confined to Airy or Bessel functions. The discovery by Stokes was, as Watson says, apparently one of those which are made at three o’clock in the morning. Stokes wrote in a 1902 retrospective paper: “The inferior term enters as it were into a mist, is hidden for a little from view, and comes out with its coefficients changed.” In 1989 the mathematical physicist Michael Berry provided a deeper explanation. He suggested that the coefficients of the subdominant expansion should be regarded not as a discontinuous constant but, for fixed |z|, as a continuous function of ph z. Berry’s innovative and insightful approach was followed by a series of papers by himself and other writers. In particular, Olver put the formal approach by Berry on a rigorous footing in papers with applications to confluent hypergeometric functions (including Airy functions, Bessel functions, and Weber parabolic functions). At the same time interest arose in earlier work by Stieltjes [106], Airey [2], Dingle [32], and others, to expand remainders of asymptotic expansions at optimal values of the summation variable. This resulted in exponentially-improved asymptotic expansions, a method of improving asymptotic approximations by including small terms in the expansion that are in fact negligible compared with other terms in the expansion.

94

A. Gil et al.

4.6.3 Exponentially Small Terms in the Airy Expansions We conclude this discussion by pointing out the relation between the Stokes phenomenon and the exponentially small terms in the asymptotic expansion of the Airy function. Consider the terms in the expansions in (4.6.4)–(4.6.7). They have the asymptotic form   cn ξ −n = O (n) (2ξ )−n , n → ∞. (4.6.8) When z is large the terms decrease at first and then increase. The least term of the first series of (4.6.6) is near n = n∗ = [|2ξ |] and its size is of order e−2|ξ | . At the Stokes lines at |ph z| = 23 π the quantity ξ is negative and the exponential term in front of the first series in (4.6.6) equals e|ξ | . Hence the order of magnitude of ∗ e−ξ cn∗ ξ −n is roughly of the same size as the second part in (4.6.7), that is, of the size of eξ that is present in front of the second series. It follows that near the Stokes lines (and of course when z turns to the negative axis) the second series in (4.6.7) is not at all negligible when we truncate the first series at the least term with index n∗ . At present we know, after Berry’s observations, that near the Stokes lines one of the constants c± in the asymptotic representation in (4.6.2) in fact is a rapidly changing function of z. In the case of (4.6.6) we can write

 ∞ ∞   − 14 −ξ n −n ξ −n Ai(z) ∼ c− z (−1) cn ξ + iS(z) e cn ξ e , (4.6.9) n=0

n=0

where S(z) switches rapidly but smoothly from 0 to 1 across the Stokes line at ph z = 2 3 π . A good approximation to S(z) involves the error function, which function can describe the fast transition in this asymptotic problem. Many writers have contributed recently in this field, both for the Stokes phenomenon of integrals and that of differential equations. For more details see the survey paper [87].

4.7 A Selection of Other Methods Many other methods are available for computing special functions. In this section we mention a selection. For all these topics more details can be found in [54].

4.7.1 Continued Fractions For many elementary and special functions representations as continued fractions exist. We give examples for incomplete gamma functions and incomplete beta functions that are useful for numerical computations.

4 Basic Methods for Computing Special Functions

95

∞ We introduce the following notation. Let {an }∞ n=1 and {bn }n=0 be two sequences of real or complex numbers. With these numbers we construct a continued fraction of the form a1 b0 + (4.7.1) a2 b1 + a3 b2 + a4 b3 + b4 + . .. A more convenient notation is

b0 +

a1 a2 a3 a4 . b1 + b2 + b3 + b4 + · · ·

(4.7.2)

For convergence aspects, contractions, even and odd parts, equivalence transformations, and so on, we refer to the literature; see [54, Chap. 6] and a recent handbook with many details for special functions [29]. To evaluate the finite part (also called the convergent) Cn = b 0 +

a1 a 2 a 3 an ··· , b1 + b2 + b3 + bn

(4.7.3)

we can use recursion. Let A−1 = 1,

A0 = b0 ,

B−1 = 0,

B0 = 1.

(4.7.4)

We compute An and Bn by using the following recursion An = bn An−1 + an An−2 ,

Bn = bn Bn−1 + an Bn−2 ,

n ≥ 1.

(4.7.5)

Then Cn = An /Bn . Several other algorithms are available; see [54, Sect. 6.6]. The recursion for An and Bn may produce large numbers of these quantities, causing overflow. However, because only the ratio An /Bn is needed to compute the convergent Cn , scaling can be used to keep control.

4.7.1.1 Incomplete Gamma Functions The incomplete gamma functions are defined by z γ (a, z) = t a−1 e−t dt, (a, z) = 0

∞

t a−1 e−t dt,

(4.7.6)

z

where for the first form we require a > 0 and for the second one |ph z| < π . We have a1 a 2 a 3 a4 , (4.7.7) z−a ez γ (a, z) = b0 + b1 + b2 + b3 + b4 + · · ·

96

A. Gil et al.

where z and a are complex, a = 0, −1, −2, . . . , and 1 , a−z

b0 =

am = mz,

bm = a + m − z,

m ≥ 1.

(4.7.8)

This fraction corresponds to the power series az

−a z

e γ (a, z) =

∞  k=0

zk . (1 + a)k

(4.7.9)

For (a, z) we have (z + 1 − a)z−a ez (a, z) =

1 α1 α2 α3 , 1+ 1+ 1+ 1 + · · ·

(4.7.10)

where αn =

n(a − n) , (z + 2n − 1 − a)(z + 2n + 1 − a)

n = 1, 2, 3, . . . .

(4.7.11)

This form is used in [38] for computing the function (a, x), for x > 1.5 and −∞ < a < α ∗ (z), where α ∗ (x) ∼ x for large x. The fraction in (4.7.10) is convergent for all z = 0 in the sector |ph z| < π , and for computations it is an excellent alternative for the corresponding asymptotic expansion z1−a ez (a, z) ∼

∞  (1 − a)k (−1)k , zk

(4.7.12)

k=0

valid for a ∈ C, z → ∞, |ph z| < 32 π . 4.7.1.2 Incomplete Beta Function This function is defined by x Bx (p, q) = t p−1 (1 − t)q−1 dt,

p > 0, q > 0,

(4.7.13)

0

and usually 0 ≤ x ≤ 1; when x < 1 the condition on q can be omitted. The beta integral is obtained when we take x = 1, that is,

1

B(p, q) =

t p−1 (1 − t)q−1 dt =

0

(p)(q) , (p + q)

p > 0, q > 0.

(4.7.14)

We have the continued fraction px −p (1 − x)−q Bx (p, q) =

1 d1 d2 d3 , 1+ 1+ 1+ 1 + · · ·

(4.7.15)

4 Basic Methods for Computing Special Functions

97

where, for n = 0, 1, 2, . . . , d2n+1 = −

(p + n)(p + q + n) x, (p + 2n)(p + 2n + 1)

(n + 1)(q − n − 1) d2n+2 = x. (p + 2n + 1)(p + 2n + 2)

(4.7.16)

When p > 1 and q > 1, the maximum of the integrand in (4.7.13) occurs at x0 = (p − 1)/(p + q − 2), and the best numerical results are obtained when x ≤ x0 . When x0 < x ≤ 1, we use the reflection relation with the beta integral (see (4.7.14)) Bx (p, q) = B(p, q) − B1−x (q, p).

(4.7.17)

From a numerical point of view the continued fraction (4.7.15) has an interesting property of the convergents: C4n and C4n+1 are less than this value of the continued fraction and C4n+2 , C4n+3 are greater than this value. This gives excellent control of the convergence of an algorithm that uses (4.7.15).

4.7.2 Sequence Transformations When applying numerical techniques to physical problems, results are usually produced in the form of sequences. Examples are iterative methods, discretization methods, perturbation methods, and—most important in the context of special functions—series expansions. Often, the sequences that are produced in this way converge too slowly to be numerically useful. When dealing with asymptotic series, summation of the sequences may also be difficult. Sequence transformations are tools to overcome convergence problems of that kind. A slowly convergent (or even divergent in the asymptotic sense) sequence {sn }∞ n=0 , whose elements may be the partial sums sn =

n 

ak

(4.7.18)

k=0

of a convergent or formal infinite series, is converted into a new sequence {sn }∞ n=0 with hopefully better numerical properties. We discuss sequence transformations that are useful in the context of special functions. For many special functions convergent and divergent (asymptotic) power series are available. Consequently, the emphasis in this section will be on sequence transformations that are able either to accelerate the convergence of slowly convergent power series effectively or to sum divergent asymptotic series.

98

A. Gil et al.

4.7.2.1 Padé Approximations From a numerical point of view, the Padé approximants are important for computing functions outside the disk of convergence of the power series of the function, as well as inside the disk (for example, near the boundary of the disk). The Padé method can also be successfully applied for locating zeros and poles of the function. Consider the power series f (z) = c0 + c1 z + c2 z2 + · · · ,

(4.7.19)

with c0 = 0. This series may be convergent or just a formal power series. We intron (z), where N n (z) and D n (z) are polynomials of duce a rational function Nmn (z)/Dm m m maximal degree n and m, respectively. That is, Nmn (z) = a0 + a1 z + · · · + an zn ,

n Dm (z) = b0 + b1 z + · · · + bm zm . (4.7.20)

We choose these polynomials such that the power series expansion of Nmn (z) − n (z) starts with a term A n+m+1 . The ratio N n (z)/D n (z), of which the f (z)Dm n,m z m m n n polynomials Nm (z) and Dm (z) satisfy the conditions degree Nmn (z) ≤ n,

n degree Dm (z) ≤ m,

n Nmn (z) − f (z)Dm (z) = An,m zn+m+1 + · · · ,

(4.7.21)

is called a Padé approximant of type (n, m) to the power series (4.7.19) (the funcn (z) is denoted by [n/m] . tion f ). The ratio Nmn (z)/Dm f For each pair (n, m) at least one rational function exists that satisfies the conditions in (4.7.21), and this function can be found by solving the equations ⎧ a 0 = c0 b0 , ⎪ ⎪ ⎪ ⎨ a 1 = c 1 b 0 + c0 b 1 , .. ⎪ . ⎪ ⎪ ⎩ an = cn b0 + cn−1 b1 + · · · + cn−m bm , (4.7.22) ⎧ ⎪ ⎨ 0 = cn+1 b0 + · · · + cn−m+1 bm , .. . ⎪ ⎩ 0 = cn+m b0 + · · · + cn bm , where cj = 0 if j < 0. When m = 0 the system of equations at the right is empty. In this case aj = cj (j = 0, 1, . . . , n) and b0 = 1, and the partial sums of (4.7.19) yield the Padé approximants of type (n, 0). In general, first the set at the right-hand side of (4.7.22) is solved (a homogeneous set of m equations for the m + 1 values bj ), which has at least one nontrivial solution. We take a normalization, for example, by taking b0 = 1 (see also the discussion in [8, p. 18]), and with this choice the last m equations give b1 , . . . , bm as the solution of a system of m linear equations. The set on the left-hand side in (4.7.22) then yields a0 , . . . , an .

4 Basic Methods for Computing Special Functions

99

The array of Padé approximants [0/0]f [0/1]f [0/2]f · · · [1/0]f [1/1]f [1/2]f · · · [2/0]f [2/1]f [2/2]f · · · .. .. .. .. . . . .

(4.7.23)

is called a Padé table. It is arranged here so that approximants with the same denominator degree are located in the same column. As remarked earlier, the first column corresponds to the partial sums of the power series in (4.7.19). The elements of the first row correspond to the partial sums of the power series of 1/f . In the literature special attention is paid to the diagonal elements [n, n]f of the table, with applications to orthogonal polynomials, quadrature formulas, moment problems, and other problems of classical analysis. In applied mathematics and in theoretical physics, Padé approximants have become a useful tool for overcoming convergence problems with power series. The popularity of Padé approximants in theoretical physics is due to Baker [6], who also wrote a monograph on Padé approximants [7]. Of interest also is the monograph by Baker and Graves-Morris [8]. An extended bibliography on Padé approximants and related matters containing several thousand references was published by Brezinski in [14]. For an annotated bibliography focusing on computational aspects, see [126]. Luke gives many rational approximations of special functions, and usually these are Padé approximants; see [75, 76].

4.7.2.2 How to Compute the Padé Approximants The approximants can be computed by Wynn’s cross rule. Any five Padé approximants arranged in the Padé table as N W

C

E

S satisfy Wynn’s cross rule (see [128]) (N − C)−1 + (S − C)−1 = (W − C)−1 + (E − C)−1 .

(4.7.24)

Starting with the first column [n/0]f , n = 0, 1, 2, . . . , initializing the preceding column by [n/ − 1]f = ∞, n = 1, 2, . . . , (4.7.24) enables us to compute the lower triangular part of the table. Likewise, the upper triangular part follows from the first row [0/n]f , n = 0, 1, 2, . . . , by initializing [−1/n]f = 0, n = 1, 2, . . . .

100

A. Gil et al.

The elements of the Padé table can also be computed by the epsilon algorithm of Wynn [127]. We consider the recursions (n)

ε−1 = 0, (n)

(n)

ε0 = sn , 1

(n+1)

εm+1 = εm−1 +

n = 0, 1, 2, . . . ,

(n+1)

εm

(4.7.25)

n, m = 0, 1, 2, . . . .

, (n)

− εm

(n)

If sn is the nth partial sum of a power series f , then ε2k is the Padé approximant (n) [n + k/k]f (cf. (4.7.23)). The elements ε2k+1 are only auxiliary quantities which diverge if the whole transformation process converges and shouldn’t be used for convergence tests or output. A recent review of the applications of the epsilon algorithm can be found in [63]. In applications one usually concentrates on obtaining diagonal elements [n/n]f and elements not far away from the diagonal; see [121], which also has an efficient modified algorithm for these elements.

4.7.2.3 Nonlinear Sequence Transformations We discuss a few other sequence transformations that, in the case of power series, produce different rational approximants, and they can also be applied to other convergence acceleration problems. Details on the history of sequence transformations and related topics, starting from the 17th century, can be found in [15]; see also [16]. For review papers, with many references to monographs devoted to this topic, we refer the reader to [65, 121]. See also Appendix A in [12], written by Dirk Laurie, with interesting observations and opinions about sequence transformations. First we mention Levin’s sequence transformation [71], which is defined by k (n) Lk (sn , ωn ) =

j =0 (−1)

k

 k−1 s n+j j k (n+j +1) j (ζ +n+k)k−1 ωn+j

j j =0 (−1)

k n+j +1)k−1 j

(ζ +n+k)k−1

1

,

(4.7.26)

ωn+j

where sn are the partial sums of (4.7.18) and the quantities ωn are remainder estimates. For example, we can simply take ζ = 1 and ωn = sn+1 − sn = an+1 ,

(4.7.27)

but more explicit remainder estimates can be used. Another transformation is due to Weniger [121], who replaced the powers (n + j + 1)k−1 in Levin’s transformation by Pochhammer symbols (n + j + 1)k−1 . That is, Weniger’s transformation reads  +n+j )k−1 sn+j k (−1)j jk (ζ j =0 (ζ +n+k)k−1 ωn+j (n) Sk (sn , ωn ) = k (4.7.28) k (ζ +n+j )k−1 1 . j j =0 (−1) j (ζ +n+k)k−1 ωn+j

4 Basic Methods for Computing Special Functions

101

Other sequence transformations can be found in [121] or in [17, Sect. 2.7]. The sequence transformations (4.7.26) and (4.7.28) differ from other sequence transformations because not only the elements of a sequence {sn } are required, but also explicit remainder estimates {ωn }. For special functions this information is usually available when divergent asymptotic expansions are considered. It was shown in several articles that the transformation (4.7.28) is apparently very effective, in particular if divergent asymptotic series are to be summed; see [10, 123, 125]. For transforming partial sums fn (z) = nk=0 γk zk of a formal power series f (z) =

∞ 

γk z k ,

(4.7.29)

ωn = γn+1 zn+1 ,

(4.7.30)

k=0

we can take the remainder estimates

and we replace z by 1/z in the case of an asymptotic series. With these modifications the transformations (4.7.26) and (4.7.28) become rational functions of the variable z. If the coefficients γn in (4.7.29) are all different from zero, these rational functions satisfy the asymptotic error estimates [124, Eqs. (4.28)–(4.29)] (n)

z → 0,

(n)

z → 0.

f (z) − Lk (fn (z), γn+1 zn+1 ) = O(zk+n+2 ), f (z) − Sk (fn (z), γn+1 zn+1 ) = O(zk+n+2 ),

(4.7.31)

These estimates imply that all terms of the formal power series, which were used for construction of the rational approximants in this way, are reproduced exactly by a (n) Taylor expansion around z = 0. Thus, the transformations Lk (fn (z), γn+1 zn+1 ) (n) and Sk (fn (z), γn+1 zn+1 ) are formally very similar to the analogous estimate n (z). (4.7.21) satisfied by the Padé approximants [n/m]f (z) = Nmn (z)/Dm

4.7.2.4 Numerical Examples Simple test problems, which nevertheless demonstrate convincingly the power of sequence transformations using explicit remainder estimates, are the integrals

∞

E(ν) (z) = 0

e−t dt 1 + zt ν

(4.7.32)

and their associated divergent asymptotic expansions E(ν) (z) ∼

∞  (νk)!(−z)k , k=0

z → 0.

(4.7.33)

102

A. Gil et al.

For ν = 1, E(ν) (z) is the exponential integral E1 with argument 1/z according to E(1) (z) = e1/z E1 (1/z)/z. For ν = 2 or ν = 3, E(ν) (z) cannot be expressed in terms of known special functions. In order to demonstrate the use of sequence transformations with explicit remain(n) der estimates, both Sk (fn (z), γn+1 zn+1 ) and Padé approximants are applied to the partial sums En(ν) (z) =

n  (νk)!(−z)k ,

0 ≤ n ≤ 50,

(4.7.34)

k=0

of the asymptotic series (4.7.33) for ν = 1, 2, 3. The Padé approximants were computed with the help of Wynn’s epsilon algorithm (see Sect. 4.7.2.2). All calculations were done in Maple, and the integrals E(ν) (z) were computed to the desired precision with the help of numerical quadrature. For the remainder estimates we took ωn = (ν(n + 1))!(−z)n+1 . The results for E(1) (z) with z = 1 are E(1) (1) = 0.59634 73623 23194 07434 10785, (50)

(1)

L0 (E0 (1), ω50 ) = 0.59634 73623 23194 07434 10759, (50)

S0

(1)

(E0 (1), ω50 ) = 0.59634 73623 23194 07434 10785,

(4.7.35)

[25/24] = 0.59634 7322, [25/25] = 0.59634 7387. The Padé approximants are not very efficient. Nevertheless, it seems that they are able to sum the divergent series (4.7.33) for ν = 1. The results for E(2) (z) with z = 1/10 are E(2) (1/10) = 0.88425 13061 26979, (50)

(2)

L0 (E0 (1/10), ω50 ) = 0.88425 13061 26980, (50)

S0

(2)

(E0 (1/10), ω50 ) = 0.88425 13061 26985,

(4.7.36)

[25/24] = 0.88409, [25/25] = 0.88437. Here, the Padé approximants are certainly not very useful since they can only extract an accuracy of three places.

4 Basic Methods for Computing Special Functions

103

The results for E(3) (z) with z = 1/100 are E(3) (1/100) = 0.96206 71061, S0(50) (E0(3) (1/100), ω50 ) = 0.96206 71055, (50)

(3)

L0 (E0 (1/100), ω50 ) = 0.96206 71057,

(4.7.37)

[25/24] = 0.960, [25/25] = 0.964. In [62] it is shown that an asymptotic series, whose coefficients grow more rapidly than (2n)!, is not Padé summable since subsequences [n + j/n] in the Padé table converge to different, j -dependent limits as n → ∞. The Levin and Weniger transformations are apparently able to sum the asymptotic series (4.7.33) even for ν = 3. Other numerical examples of sequence transformations using explicit remainder estimates can be found in [10, 122, 123]. For a recent paper on transformations of hypergeometric series, see [22].

4.7.3 Other Quadrature Methods In Sect. 4.5 we have described the use of the trapezoidal rule in the evaluation of special functions. In [54, Sects. 5.3, 9.6] several other methods are discussed, with as the main one Gaussian quadrature, and the relation with orthogonal polynomials. Other methods are Romberg quadrature, which provides a scheme for computing successively refined rules with a higher degree of exactness. Fejér and Clenshaw– Curtis quadratures are interpolatory rules which behave quite similarly to Gauss– Legendre rules, but which are easier to compute and provide nested rules. Other nested rules, related to Gauss quadrature but harder to compute than the Clenshaw– Curtis rule, are Kronrod and Patterson quadratures. Specific methods for oscillatory integrands are also described, with special attention to Filon’s method.

4.7.4 Numerical Inversion of Laplace Transforms We consider the pair of Laplace transforms

∞

F (s) = 0

e−st f (t) dt,

f (t) =

1 2πi



c+i∞

est F (s) ds,

(4.7.38)

c−i∞

where f should be absolutely integrable on any finite interval [0, a] and the number c is chosen such that all singularities of F (s) are at the left of the vertical line s = c.

104

A. Gil et al.

The inversion problem is to find f (t) when F (s) is given. To solve this problem numerically, an essential condition is whether function values of F (s) are only available for real s or for complex values of s. The first case is quite difficult and requires completely different techniques compared with those for the second case. In this section we consider a method for the case that F (s) is available as an analytic function in part of the complex s-plane. We describe a method based on the deformation of the contour of integration. We give an example in which an optimal representation of the integral is obtained by deforming the contour of integration and by using a proper value of c in the complex integral in (4.7.38). After selecting this new contour, the trapezoidal rule can be used for numerical quadrature. As explained in Sect. 4.5 this method may be very efficient for evaluating a class of integrals with analytic integrands. We use the Laplace transform pair (see [1, Eq. (29.3.83)]) √ 1 F (s) = e−k s = s



∞ 0

1 k erfc √ = 2πi 2 t



k e−st erfc √ dt, 2 t

c+i∞

est−k

c−i∞

√ s

ds , s

(4.7.39)

where in this case c > 0. We take k = 2λ and t = 1, which gives 1 erfc λ = 2πi



c+i∞

√

es−2λ

s

c−i∞

ds , s

(4.7.40)

and we assume that λ > 0. When λ is large the integral becomes exponentially small, and straightforward application of a quadrature rule is useless. With the transformation s = λ2 t , (4.7.40) becomes 1 erfc λ = 2πi



c+i∞

√

2 (t−2

eλ

t)

c−i∞

dt . t

(4.7.41)

When we take c = 1 the path runs through the saddle point at t = 1, where the 2 exponential function of the integrand has the value e−λ , which corresponds to the main term in the asymptotic estimate e−λ erfc λ ∼ √ , πλ 2

λ → ∞.

(4.7.42)

Because the convergence at ±i∞ along the vertical through t = 1 is rather poor, the next step is to deform the contour into a new contour that terminates in the left half-plane, with t → −∞. In fact many contours are suitable, but there is only one contour through t = 1 on which √ no oscillations occur. That contour, the steepest descent path, is given by (t − 2 t) = 0, or in polar coordinates t = reiθ we have r = sec2 ( 12 θ ). See Fig. 4.2.

4 Basic Methods for Computing Special Functions

105

Fig. 4.2 The new contour of integration for (4.7.41) has the shape of a parabola

Table 4.4 Composite trapezoidal rule for the integral in (4.7.43) with λ = 10

h

erfc λ

n

0.25

0.20949 49432 96679 10−44

5

0.20

0.20886 11645 34559 10−44

6

0.15

0.20884 87588 72946 10−44

8

0.10

0.20884 87583 76254 10−44

11

This gives, by integrating with respect to θ ∈ [−π, π], e−λ 2π

2

erfc λ =



π

−π

2 tan2 ( 1 θ) 2

e−λ

dθ.

(4.7.43)

As discussed in Sect. 4.5.1 the trapezoidal rule is exceptionally accurate in this case. Table 4.4 gives the results of applying the composite trapezoidal rule with step size h; n indicates the number of function values in the rule that are larger than 10−15 (we exploit the fact that the integrand is even). All digits shown in the approximation in the final row are correct. When F (s) in (4.7.38) has singularities or poles, a straightforward and optimal choice of the path of integration, as in the above example, might not be easy to find. In these cases, or when less information is available on the function F (s), a less optimal contour may be chosen. For example, we can take a parabola or a hyperbola that terminates in the left half-plane at −∞. When we write s = u + iv, and consider the parabola defined by u = p − qv 2 , and integrate with respect to v. When we choose p and q properly all singularities of F (s) may remain inside the contour (unless F (s) has an infinite number of singularities up to ±i∞). Further details can be found in [84, 91, 107]. Recent investigations are discussed in [92, 114, 119, 120].

106

A. Gil et al.

4.7.5 Computing Zeros of Special Functions The zeros of special functions appear in a great number of applications in mathematics, physics, and engineering, from the computation of Gauss quadrature rules [60] in the case of orthogonal polynomials to many applications in which boundary value problems for second order ordinary differential equations arise. In some sense, the computation of the zeros of special functions has nothing special: to compute the roots of an equation y(x) = 0, with y(x) special or not, we can apply well-known methods (bisection, secant, Newton–Raphson, or whatever) once we know how to compute the function y(x) (and in the case of Newton–Raphson also its derivative) accurately enough. However, as is generally the case with nonlinear equations, some information on the location of the zeros is desirable, particularly when applying rapidly convergent (but often unpredictable) methods like Newton–Raphson or higher order methods. The zeros of special functions usually appear nicely arranged, forming clear patterns from which a priori information can be found. For instance, the zeros of Ja(α,β) (x) are all real and in the interval (−1, 1) for α > −1 and cobi polynomials Pn β > −1, and they satisfy other regularity properties (such as, for instance, interlacing with the zeros of the derivative and with the zeros of contiguous orders). As α and/or β become smaller than −1, some or all of the n zeros escape from the real axis, forming a regular pattern in the complex plane (see Fig. 4.3). These regular patterns formed by the zeros is a common feature of “classical” special functions and beyond [23]. The regularity in the distribution of zeros helps in the design of specific algorithms with good convergence properties. In addition, for many special functions, accurate a priori approximations are available. This a priori information, when wisely applied, will save computation time and avoid divergent (or even chaotic) algorithms. In a fortunate situation, as in the case of Bessel functions, asymptotic approximations provide accurate enough starting values for higher order Newton–Raphson methods; see [110]. In [54, Chap. 7], a variety of methods for computing zeros of special functions is discussed, starting with bisection and the fixed point method, including NewtonRaphson. In general, for computing zeros of special functions it is a wise idea to use some of their properties and to design specific methods. Next to methods that take advantage of information about asymptotic approximations for the zeros of special functions, there are methods for which it is not necessary to compute values of these functions themselves in order to obtain their zeros. This is the case for the classical orthogonal polynomials, the zeros of which are the exact eigenvalues of real tridiagonal symmetric matrices with very simple entries; this method is usually named the Golub-Welsch algorithm [60]. The recurrence relation of the special functions plays a crucial role because the matrix is built from the coefficients of the recursion. Also, there are other functions, minimal solutions of three-term recurrence relations (the Bessel function Jν (x) is among them) for which the problem of computing zeros is not exactly an eigenvalue problem for a (finite) matrix, but it can be approximated by it [61, 67].

4 Basic Methods for Computing Special Functions

Fig. 4.3 Zeros of the Jacobi (2,−42.5) polynomials P50 (x) (2,−52) (x) (center), (left), P50 (2,−63.5) (x) (right) in and P50 the complex plane

107

108

A. Gil et al.

Another type of methods, which are global in the sense that, similarly as matrix methods, don’t require a priori estimations of the zeros for ensuring convergence, are the fixed point methods of [45, 100]; these methods use first order differential systems, which are satisfied by a large number of special functions (hypergeometric functions among them). More recently, a fixed point method of fourth order was obtained in [99], which can be used to compute the zeros of any solution of any second order equation y

(x) + A(x)y(x) = 0 in an interval where A(x) is continuous. It is shown that, when A(x) > 0, the fixed point iteration T (x) = x − 

1

 arctan A(x)y(x)/y (x)

(4.7.44)

A(x)

has order four and the sequence xn+1 = T (xn ) converges for any x0 under mild conditions on A(x). A scheme is given which guarantees convergence for any continuous A(x) and allows for computing all the zeros in an interval. Typically 3 or 4 iterations per zero are required for 100 digits accuracy. Additional methods and details on obtaining information on the zeros by using asymptotic expansions of the special functions, with examples such as Airy functions, Scorer functions, error functions, parabolic cylinder functions, Bessel func(α) tions, and Laguerre polynomials Ln (x) with large values of α are given in [54, Chap. 7].

4.7.6 Uniform Asymptotic Expansions The asymptotic expansions considered in Sect. 4.2.1 are simple in the sense that they hold for large values of one variable, in fact the argument of the special function. There are many powerful expansions available that hold also for other large parameters. For example, the expansion for the incomplete gamma function in (4.7.12) holds when z is large, but it becomes useless when a is also large. Also, the expansion in (4.7.9) converges for all a and z with trivial exceptions: a = 0, −1, −2, . . . . But for computations it becomes useless when z is much larger than a. There is a nice alternate asymptotic representations for these functions which can be used when a and/or z are large, and which in particular holds when a ∼ z, a transition point in the behavior for large a and z. In this representation the complementary error function ∞ 2 2 e−t dt (4.7.45) erfc z = √ π z plays the role of the transition form very small to very large values, or for the scaled functions (a, z) γ (a, z) , Q(a, z) = (4.7.46) P (a, z) = (a) (a) from values close to 0 to values close to 1.

4 Basic Methods for Computing Special Functions

109

We have the following representations:  1 erfc(η a/2) + Ra (η), 2  1 P (a, z) = erfc(−η a/2) − Ra (η), 2

Q(a, z) =

(4.7.47)

where 1 2 η = λ − 1 − ln λ, 2

z λ= , a

(4.7.48)

and 1

e− 2 aη Ra (η) = √ Sa (η), 2πa 2

Sa (η) ∼

∞  Cn (η) n=0

an

,

(4.7.49)

as a → ∞. The relation between η and λ in (4.7.48) becomes clear when we expand 1 1 1 λ − 1 − ln λ = (λ − 1)2 − (λ − 1)3 + (λ − 1)4 + · · · , 2 3 4 and in fact the relation in (4.7.48) can also be written as  2(λ − 1 − ln λ) η = (λ − 1) , (λ − 1)2

(4.7.50)

(4.7.51)

where the sign of the square root is positive for λ > 0. For complex values we use analytic continuation. An expansion for small values of |λ − 1| reads 1 7 η = (λ − 1) − (λ − 1)2 + (λ − 1)3 + · · · , 3 36

(4.7.52)

and, upon inverting this expansion, 1 1 λ = 1 + η + η2 + η3 + · · · . 3 36

(4.7.53)

The asymptotic expansion for Sa (η) in (4.7.49) holds uniformly with respect to z ≥ 0. Both a and z may be complex. Note that the symmetry relation P (a, z) + Q(a, z) = 1 is preserved in the representations in (4.7.47) because erfc z + erfc(−z) = 2. The first coefficients for Sa (η) are C0 =

1 1 − , λ−1 η

C1 (η) =

1 1 1 1 − − − . (4.7.54) 3 3 2 12(λ − 1) η (λ − 1) (λ − 1)

These coefficients, and all higher ones, are regular at the transition point a = z, or λ = 1, or η = 0. For numerical applications Taylor expansions can be used, as explained in [54, Sect. 8.3].

110

A. Gil et al.

For Bessel functions we have a similar problem in the design of efficient algorithms. All Bessel functions can be expanded in terms of Airy functions, and these expansions are in particular useful in the neighborhood of the turning point z = ν. For example, the Bessel function Jν (z) is oscillatory for z > ν and monotonic for z < ν. Airy functions have a similar turning point behavior, as follows from their differential equation w

− zw = 0. The coefficients in the asymptotic series are regular at the turning point z = ν, but for numerical evaluations we need expansions of the used coefficients in the neighborhood of the turning point. For more details we refer to [54, Sect. 8.4] and [112]. Airy-type expansions are used in software for Bessel functions in [5, 51, 70], and for parabolic cylinder functions in [56, 57, 95].

4.7.7 Taylor Expansion Methods for Ordinary Differential Equations The special functions of mathematical physics usually arise as special solutions of ordinary linear differential equations, which follow from certain forms of the wave equation. Separation of the variables and the use of domains such as spheres, circles, cylinders, and so on, are the standard ways of introducing Bessel functions, Legendre functions, and confluent hypergeometric functions (also called Whittaker functions). For an introduction to this topic, see [111, Chap. 10]. In numerical mathematics, computing solutions of ordinary linear differential equations is a vast research area, with popular methods such as, for example, Runge– Kutta methods. These techniques are usually not used for computing special functions, mainly because so many other efficient methods are available for these functions. However, when the differential equation has coefficients in terms of analytic functions, as is the case for the equations of special functions, a method based on Taylor expansions may be considered as an alternative method, in particular for solving the equation in the complex plane. In [54, Sect. 9.5] the basic steps are given for the Taylor expansion method, in particular for linear second order equations of the form dw d 2w + f (z) + g(z)w = h(z), dz dz2

(4.7.55)

where f , g, and h are analytic functions in a domain D ⊆ C. For applications to special functions f , g, and h are often simple rational functions, and usually the equation is homogeneous. For further information and examples, see [86] and [73]. For an application to compute solutions in the complex plane of the Airy differential equation, see [35] with a Fortran computer program in [34]. In [56] Taylor methods are used for computing the parabolic cylinder function W (a, x).

4 Basic Methods for Computing Special Functions

111

4.7.8 Computing Symmetric Elliptic Integrals Legendre’s standard elliptic integrals are the incomplete elliptic integral of the first kind, φ dθ  K(k) = , (4.7.56) 0 1 − k 2 sin2 θ the incomplete integral of the second kind, φ 1 − k 2 sin2 θ dθ, E(k) =

(4.7.57)

0

and the incomplete elliptic integral of the third kind, φ 1 dθ  (n; φ, k) = . 2 0 1 − n sin θ 1 − k 2 sin2 θ

(4.7.58)

It is assumed here that k ∈ [0, 1], φ ∈ [0, 12 π], although the functions can also be defined for complex values of the parameters. Also, n is real, and if n > 1, the integral of the third kind should be interpreted as a Cauchy principal value integral. When φ = 12 π the integrals are called complete elliptic integrals. The computational problem for the elliptic integrals has received much attention in the literature, and the algorithms are usually based on successive Landen transformations or Gauss transformations, or by infinite series. By considering a new set of integrals it is possible to compute the elliptic integrals, also by using successive transformations, by very efficient algorithms. The integrals are introduced in [18]. For example we have dt 1 ∞ . (4.7.59) RF (x, y, z) = √ 2 0 (t + x)(t + y)(t + z) This function is symmetric and homogeneous of degree − 12 in x, y, z and is nor1

malized so that RF (x, x, x) = x − 2 . Many elementary functions can be expressed in terms of these integrals. For example, dt 1 ∞ , (4.7.60) RF (x, y, y) = √ 2 0 (t + x) (t + y) which is a logarithm if 0 < y < x and an inverse circular function if 0 ≤ x < y. The three standard elliptic integrals in (4.7.56)–(4.7.58) can be written in terms of symmetric integrals. For example, we have  (4.7.61) F (φ, k) = sin φRF cos2 φ, 1 − k 2 sin2 φ, 1 . For further details we refer to [54, Sect. 11.4] and to the work of B.C. Carlson, who wrote very efficient algorithms for the computation of Legendre’s standard elliptic integrals, also for complex parameters [19–21].

112

A. Gil et al.

Other methods [82], based on series expansions [31], can be also considered and may be faster for some parameter values.

4.7.9 Best Rational Approximations In the theory of best rational approximation (which includes the best polynomial of approximation) the goal is to find a rational function that approximates a function f on a finite real interval as best as possible. The rational approximations can be written in the form Nmn (x) a 0 + a1 x + · · · + a n x n . (4.7.62) = n (x) Dm b 0 + b1 x + · · · + b m x m The characterization of the best approximation to a function f may be given in terms of oscillations of the error curve. Let R = N/D be an irreducible rational function of the form (4.7.62). A necessary and sufficient condition that R be the best approximation to f is that the error function R(x) − f (x) exhibits at least 2 + max{m + ∂N, n + ∂D} points of alternation. (Here ∂P denotes the degree of the polynomial P .) For the proof see [79]. For the elementary and well-known higher transcendental functions the polynomials in the best rational approximations are not explicitly known, and the coefficients of these polynomials should be computed by an algorithm. This algorithm is not as simple as the one for computing coefficients in Chebyshev series (see Sect. 4.4) or Padé approximants (which can be based on solving a set of linear equations). For best approximation the second algorithm of Remes can be used [90, p. 176], and for a Fortran program see [69]. For many elementary and special functions best rational approximations have been computed. See [64] for many tables (and an explanation of the Remes algorithm). For several other special functions we refer to the survey [27]. Computer algebra packages, such as Maple, also have programs for computing best rational approximants. For flexible algorithms for special functions we prefer the method based on Chebyshev polynomials. Chebyshev series (4.4.1) usually converge rapidly (for example, for functions in C ∞ on [−1, 1]), we obtain a very good first approximation to the polynomial pn (x) of best approximation for [−1, 1] if we truncate (4.4.1) at its (n + 1)th term. This is because f (x) −

n 

. ck Tk (x) = cn+1 Tn+1 (x),

(4.7.63)

k=0

and the right-hand side enjoys exactly those properties concerning its maxima and minima that are required for the polynomial of best approximation. In practice the gain in replacing a truncated Chebyshev series expansion by the corresponding minimax polynomial approximation is hardly worthwhile; see [88].

4 Basic Methods for Computing Special Functions

113

Inspection of the size of the coefficients ck gives a good idea about the applicability for a certain choice of n, and for a new choice of n the computations are easy to modify. In best approximations for each choice of n (or of n and m in rational approximations), new coefficients have to be computed by using a complicated algorithm. In addition, representations of the polynomials in best approximations may be quite unstable.

4.8 Recent Software and Publications on Methods for Computing Special Functions 4.8.1 A Selection of Recent Software for Special Functions Software packages such as Mathematica, Maple, and Matlab have many excellent algorithms in multi-length arithmetic. For large scale and high performance computing these packages are not the optimal platforms. Also, there are many published books with software for special functions, some with supplied sources of the algorithms. We mention [9, 83, 89, 113, 117, 129]. Many software collections of special functions are available on the web, for instance the Cephes math library5 and in more general repositories6,7 . For an extensive survey of the available software for special functions we refer to [74]. The latest update of this project appeared in December 2000. In this section we give a selection of software for special functions published in the period 2000–2009 (disclaimer: we provide references, but we don’t claim that all the software listed is equally reliable). Books describing published software 1. Cuyt et al. [29], a Handbook describing numerical methods for evaluating special functions by using continued fractions. Many tables are given based on Maple programs published elsewhere by the group. All kinds of functions are considered: from gamma to Gauss, confluent, generalized and basic hypergeometric functions. 2. Gautschi [40] describes routines for generating recursion coefficients of orthogonal polynomials as well as routines dealing with applications. 3. Gil et al. [54] describes software for Airy and Scorer functions, associated Legendre functions of integer and half-integer degrees (including toroidal harmonics), Bessel functions (including modified Bessel functions with purely imaginary orders), parabolic cylinder functions, and a module for computing zeros of Bessel functions. 5 http://www.moshier.net. 6 http://www.netlib.org/. 7 http://gams.nist.gov/Classes.html.

114

A. Gil et al.

Gamma, error, Fadeeva, Voigt and related functions 1. Smith [105]: Fortran 90 software for floating-point multiple precision arithmetic, gamma and related functions. 2. Linhart et al. [72]: the logarithm of the normal distribution. 3. Shippony and Read [104]: Voigt function (a special case of the plasma dispersion function or the complex error function). Bessel functions 1. Kodama [70]: all kinds of cylindrical functions of complex order and nonnegative argument. 2. Gil et al. [52]: modified Bessel functions Iia (x) and Kia (x) for real a and positive x. 3. Van Deun and Cools [115]: infinite range integrals of an arbitrary product of Bessel functions. 4. Talman [108]: spherical Bessel transforms. Airy and related functions 1. Gil et al. [48, 49]: complex Airy and Scorer functions. 2. Fabijonas [34]: complex Airy functions. Parabolic cylinder functions 1. Gil et al. [57]: functions U (a, x) and V (a, x) for real a and x. Coulomb wave functions 1. 2. 3. 4.

Michel [80]: functions F (η, ρ) and G (η, ρ) for complex parameters Seaton [97]: functions F (η, ρ) and G (η, ρ). Seaton [98]: Numerov integrations of Coulomb functions. Noble [85]: negative energy Coulomb (Whittaker) functions.

Legendre functions 1. Gil and Segura [43, 44]: toroidal harmonics. 2. Inghoff et al. [68]: Maple procedures for the coupling of angular momenta. Hypergeometric functions 1. Michel and Stoitsov [81]: Gauss hypergeometric function with all its parameters complex. 2. Colavecchia and Gasaneo [28]: Appell’s F1 function. 3. Huber and Maître [66]: expanding hypergeometric functions about half-integer parameters. Mathieu functions 1. Alhargan [3, 4]: Mathieu functions and characteristic numbers. 2. Erricolo [33]: expansion coefficients of Mathieu functions using Blanch’s algorithm.

4 Basic Methods for Computing Special Functions

115

4.8.2 Recent Literature on the Computation of Special Functions From our website8 a list will be soon available with recent literature from the last ten years (2000–2009). The list can be viewed as an addition to the bibliography of Lozier and Olver [74], and contains references to software and papers describing methods for computing special functions. Some of the references are mentioned in earlier sections. The following topics can be found in the list. 1. General aspects in books and papers: continued fractions, recurrence relations, Hadamard-type expansions, infinite products. 2. Gamma function, Barnes multiple gamma function, incomplete gamma functions, beta distribution, error functions, exponential integrals. 3. Bessel functions, integrals of Bessel functions, series of K-Bessel function. 4. Airy functions, Airy-type integrals: oscillatory cuspoid integrals with odd and even polynomial phase functions, Pearcy integral. 5. Hypergeometric functions: Gauss, confluent, Coulomb, Weber parabolic, Appell. 6. Legendre functions: toroidal, conical, spherical harmonics. 7. Orthogonal polynomials, Gauss quadrature. 8. q-functions. 9. Mathieu functions. 10. Spheroidal wave functions. 11. Polylogarithms. 12. Mittag-Leffler function, Wright function. 13. Elliptic integrals, elliptic functions. 14. Riemann zeta function, Riemann theta function. 15. Bose–Einstein, Fermi–Dirac integrals. 16. Hubbell rectangular source integrals, Lambert’s W -function, leaky aquifer function. 17. Multicenter integrals, Slater orbitals, other integrals from physics. 18. Zeros of special functions. 19. Multiprecision implementation of elementary and special functions. Acknowledgements We thank the referees for their helpful comments and Dr. Ernst Joachim Weniger for providing us with notes that we used for writing Sect. 4.7.2. We acknowledge financial support from Ministerio de Educación y Ciencia, project MTM2006–09050. NMT acknowledges financial support from Gobierno of Navarra, Res. 07/05/2008.

References 1. Abramowitz, M., Stegun, I.A.: Handbook of mathematical functions with formulas, graphs, and mathematical tables. National Bureau of Standards Applied Mathematics Series, vol. 55. US Printing Office (1964) 8 http://functions.unican.es.

116

A. Gil et al.

2. Airey, J.R.: The “converging factor” in asymptotic series and the calculation of Bessel, Laguerre and other functions. Philos. Mag. 24, 521–552 (1937) 3. Alhargan, F.A.: Algorithm 804: subroutines for the computation of Mathieu functions of integer orders. ACM Trans. Math. Softw. 26(3), 408–414 (2000) 4. Alhargan, F.A.: Algorithm 855: subroutines for the computation of Mathieu characteristic numbers and their general orders. ACM Trans. Math. Softw. 32(3), 472–484 (2006) 5. Amos, D.E.: Algorithm 644: a portable package for Bessel functions of a complex argument and nonnegative order. ACM Trans. Math. Softw. 12(3), 265–273 (1986) 6. Baker, G.A. Jr.: The theory and application of the Padé approximant method. In: Advances in Theoretical Physics, vol. 1, pp. 1–58. Academic Press, New York (1965) 7. Baker, G.A. Jr.: Essentials of Padé Approximants. Academic Press, New York/London (1975). [A subsidiary of Harcourt Brace Jovanovich, Publishers] 8. Baker, G.A. Jr., Graves-Morris, P.: Padé Approximants, 2nd edn. Encyclopedia of Mathematics and Its Applications, vol. 59. Cambridge University Press, Cambridge (1996) 9. Baker, L.: C Mathematical Function Handbook. Programming Tools For Engineers and Scientists. McGraw-Hill, New York (1992) 10. Bhattacharya, R., Roy, D., Bhowmick, S.: Rational interpolation using Levin-Weniger transforms. Comput. Phys. Commun. 101(3), 213–222 (1997) 11. Bickley, W.G., Comrie, L.J., Miller, J.C.P., Sadler, D.H., Thompson, A.J.: Bessel Functions. Part II. Functions of Positive Integer Order. British Association for the Advancement of Science, Mathematical Tables, vol. X. University Press, Cambridge (1952) 12. Bornemann, F., Laurie, D., Wagon, S., Waldvogel, J.: The SIAM 100-Digit Challenge. SIAM, Philadelphia (2004). A study in high-accuracy numerical computing, With a foreword by David H. Bailey 13. Boyd, J.P.: The devil’s invention: Asymptotic, superasymptotic and hyperasymptotic series. Acta Appl. Math. 56(1), 1–98 (1999) 14. Brezinski, C.: A Bibliography on Continued Fractions, Padé Approximation, Sequence Transformation and Related Subjects. Prensas Universitarias de Zaragoza, Zaragoza (1991) 15. Brezinski, C.: History of Continued Fractions and Padé Approximants. Springer, Berlin (1991) 16. Brezinski, C.: Convergence acceleration during the 20th century. J. Comput. Appl. Math. 122(1–2), 1–21 (2000). Numerical analysis 2000, Vol. II: Interpolation and extrapolation 17. Brezinski, C., Redivo-Zaglia, M.: Extrapolation Methods. Theory and Practice. Studies in Computational Mathematics, vol. 2. North-Holland, Amsterdam (1991) 18. Carlson, B.C.: Special Functions of Applied Mathematics. Academic Press, New York (1977). [Harcourt Brace Jovanovich Publishers] 19. Carlson, B.C.: Computing elliptic integrals by duplication. Numer. Math. 33(1), 1–16 (1979) 20. Carlson, B.C.: Numerical computation of real or complex elliptic integrals. Numer. Algorithms 10(1–2), 13–26 (1995). Special functions (Torino, 1993) 21. Carlson, B.C., FitzSimons, J.: Reduction theorems for elliptic integrals with the square root of two quadratic factors. J. Comput. Appl. Math. 118(1–2), 71–85 (2000) 22. Chatterjee, S., Roy, D.: A class of new transforms tailored for the hypergeometric series. Comput. Phys. Commun. 179(8), 555–561 (2008) 23. Clarkson, P.A., Mansfield, E.L.: The second Painlevé equation, its hierarchy and associated special polynomials. Nonlinearity 16(3), R1–R26 (2003) 24. Clenshaw, C.W.: A note on the summation of Chebyshev series. Math. Tables Aids Comput. 9(51), 118–120 (1955) 25. Clenshaw, C.W.: The numerical solution of linear differential equations in Chebyshev series. Proc. Camb. Philos. Soc. 53, 134–149 (1957) 26. Clenshaw, C.W.: Chebyshev Series for Mathematical Functions. National Physical Laboratory Mathematical Tables, vol. 5. Her Majesty’s Stationery Office, London (1962). Department of Scientific and Industrial Research 27. Cody, W.J.: A survey of practical rational and polynomial approximation of functions. SIAM Rev. 12(3), 400–423 (1970)

4 Basic Methods for Computing Special Functions

117

28. Colavecchia, F.D., Gasaneo, G.: f1: a code to compute Appell’s F1 hypergeometric function. Comput. Phys. Commun. 157(1), 32–38 (2004) 29. Cuyt, A., Petersen, V.B., Verdonk, B., Waadeland, H., Jones, W.B.: Handbook of Continued Fractions for Special Functions. Springer, New York (2008). With contributions by Franky Backeljauw and Catherine Bonan-Hamada, Verified numerical output by Stefan Becuwe and Cuyt 30. Deaño, A., Segura, J., Temme, N.M.: Identifying minimal and dominant solutions for Kummer recursions. Math. Comput. 77(264), 2277–2293 (2008) 31. DiDonato, A.R., Hershey, A.V.: New formulas for computing incomplete elliptic integrals of the first and second kind. J. Assoc. Comput. Mach. 6, 515–526 (1959) 32. Dingle, R.B.: Asymptotic expansions and converging factors. I. General theory and basic converging factors. Proc. R. Soc. Lond. Ser. A 244, 456–475 (1958) 33. Erricolo, D.: Algorithm 861: Fortran 90 subroutines for computing the expansion coefficients of Mathieu functions using Blanch’s algorithm. ACM Trans. Math. Softw. 32(4), 622–634 (2006) 34. Fabijonas, B.R.: Algorithm 838: Airy functions. ACM Trans. Math. Softw. 30(4), 491–501 (2004) 35. Fabijonas, B.R., Lozier, D.W., Olver, F.W.J.: Computation of complex Airy functions and their zeros using asymptotics and the differential equation. ACM Trans. Math. Softw. 30(4), 471–490 (2004) 36. Forrey, R.C.: Computing the hypergeometric function. J. Comput. Phys. 137(1), 79–100 (1997) 37. Gautschi, W.: Computational aspects of three-term recurrence relations. SIAM Rev. 9(1), 24–82 (1967) 38. Gautschi, W.: A computational procedure for incomplete gamma functions. ACM Trans. Math. Softw. 5(4), 466–481 (1979) 39. Gautschi, W.: Computation of Bessel and Airy functions and of related Gaussian quadrature formulae. BIT 42(1), 110–118 (2002) 40. Gautschi, W.: Orthogonal Polynomials: Computation and Approximation. Numerical Mathematics and Scientific Computation. Oxford University Press, New York (2004) 41. Gil, A., Segura, J.: Evaluation of Legendre functions of argument greater than one. Comput. Phys. Commun. 105(2–3), 273–283 (1997) 42. Gil, A., Segura, J.: A code to evaluate prolate and oblate spheroidal harmonics. Comput. Phys. Commun. 108(2–3), 267–278 (1998) 43. Gil, A., Segura, J.: Evaluation of toroidal harmonics. Comput. Phys. Commun. 124, 104–122 (2000) 44. Gil, A., Segura, J.: DTORH3 2.0: A new version of a computer program for the evaluation of toroidal harmonics. Comput. Phys. Commun. 139(2), 186–191 (2001) 45. Gil, A., Segura, J.: Computing the zeros and turning points of solutions of second order homogeneous linear ODEs. SIAM J. Numer. Anal. 41(3), 827–855 (2003) 46. Gil, A., Segura, J., Temme, N.M.: Computing toroidal functions for wide ranges of the parameters. J. Comput. Phys. 161(1), 204–217 (2000) 47. Gil, A., Segura, J., Temme, N.M.: On nonoscillating integrals for computing inhomogeneous Airy functions. Math. Comput. 70(235), 1183–1194 (2001) 48. Gil, A., Segura, J., Temme, N.M.: Algorithm 819: AIZ, BIZ: two Fortran 77 routines for the computation of complex Airy functions. ACM Trans. Math. Softw. 28(3), 325–336 (2002) 49. Gil, A., Segura, J., Temme, N.M.: Algorithm 822: GIZ, HIZ: two Fortran 77 routines for the computation of complex Scorer functions. ACM Trans. Math. Softw. 28(4), 436–447 (2002) 50. Gil, A., Segura, J., Temme, N.M.: Computing complex Airy functions by numerical quadrature. Numer. Algorithms 30(1), 11–23 (2002) 51. Gil, A., Segura, J., Temme, N.M.: Algorithm 831: Modified Bessel functions of imaginary order and positive argument. ACM Trans. Math. Softw. 30(2), 159–164 (2004) 52. Gil, A., Segura, J., Temme, N.M.: Computing solutions of the modified Bessel differential equation for imaginary orders and positive arguments. ACM Trans. Math. Softw. 30(2), 145– 158 (2004)

118

A. Gil et al.

53. Gil, A., Segura, J., Temme, N.M.: The ABC of hyper recursions. J. Comput. Appl. Math. 190(1–2), 270–286 (2006) 54. Gil, A., Segura, J., Temme, N.M.: Numerical Methods for Special Functions. SIAM, Philadelphia (2007) μ 55. Gil, A., Segura, J., Temme, N.M.: Computing the conical function P−1/2+iτ (x). SIAM J. Sci. Comput. 31(3), 1716–1741 (2009) 56. Gil, A., Segura, J., Temme, N.M.: Fast and accurate computation of the Weber parabolic cylinder function w(a, x) (2009). Submitted to IMA J. Numer. Anal. 57. Gil, A., Segura, J., Temme, N.M.: Algorithm 850: Real parabolic cylinder functions U (a, x), V (a, x). ACM Trans. Math. Softw. 32(1), 102–112 (2006) 58. Gil, A., Segura, J., Temme, N.M.: Computing the real parabolic cylinder functions U (a, x), V (a, x). ACM Trans. Math. Softw. 32(1), 70–101 (2006) 59. Gil, A., Segura, J., Temme, N.M.: Numerically satisfactory solutions of hypergeometric recursions. Math. Comput. 76(259), 1449–1468 (2007) 60. Golub, G.H., Welsch, J.H.: Calculation of Gauss quadrature rules. Math. Comput. 23(106), 221–230 (1969). Loose microfiche suppl. A1–A10 61. Grad, J., Zakrajšek, E.: Method for evaluation of zeros of Bessel functions. J. Inst. Math. Appl. 11, 57–72 (1973) 62. Graffi, S., Grecchi, V.: Borel summability and indeterminacy of the Stieltjes moment problem: Application to the anharmonic oscillators. J. Math. Phys. 19(5), 1002–1006 (1978) 63. Graves-Morris, P.R., Roberts, D.E., Salam, A.: The epsilon algorithm and related topics. J. Comput. Appl. Math. 122(1–2), 51–80 (2000). Numerical analysis 2000, vol. II: Interpolation and extrapolation 64. Hart, J.F., Cheney, E.W., Lawson, C.L., Maehly, H.J., Mesztenyi, C.K., Rice, J.R., Thacher, H.C. Jr., Witzgall, C.: Computer Approximations. SIAM Ser. in Appl. Math. Wiley, New York (1968) 65. Homeier, H.H.H.: Scalar Levin-type sequence transformations. J. Comput. Appl. Math. 122(1–2), 81–147 (2000). Numerical analysis 2000, Vol. II: Interpolation and extrapolation 66. Huber, T., Maître, D.: HypExp 2, expanding hypergeometric functions about half-integer parameters. Comput. Phys. Commun. 178(10), 755–776 (2008) 67. Ikebe, Y.: The zeros of regular Coulomb wave functions and of their derivatives. Math. Comput. 29, 878–887 (1975) 68. Inghoff, T., Fritzsche, S., Fricke, B.: Maple procedures for the coupling of angular momenta. IV: Spherical harmonics. Comput. Phys. Commun. 139(3), 297–313 (2001) 69. Johnson, J.H., Blair, J.M.: REMES2—a Fortran program to calculate rational minimax approximations to a given function. Technical Report AECL-4210, Atomic Energy of Canada Limited. Chalk River Nuclear Laboratories, Chalk River, Ontario (1973) 70. Kodama, M.: Algorithm 877: A subroutine package for cylindrical functions of complex order and nonnegative argument. ACM Trans. Math. Softw. 34(4), Art. 22, 21 (2008) 71. Levin, D.: Development of non-linear transformations of improving convergence of sequences. Int. J. Comput. Math. 3, 371–388 (1973) 72. Linhart, J.M.: Algorithm 885: Computing the logarithm of the normal distribution. ACM Trans. Math. Softw. 35(3), Art. 20 (2008) 73. Lozier, D.W., Olver, F.W.J.: Airy and Bessel functions by parallel integration of ODEs. In: Sincovec, R.F., Keyes, D.E., Leuze, M.R., Petzold, L.R., Reed, D.A. (eds.) Parallel Processing for Scientific Computing. Proceedings of the Sixth SIAM Conference, vol. II, pp. 530–538. SIAM, Philadelphia (1993) 74. Lozier, D.W., Olver, F.W.J.: Numerical evaluation of special functions. In: Mathematics of Computation 1943–1993: A Half-century of Computational Mathematics, Vancouver, BC, 1993. Proc. Sympos. Appl. Math., vol. 48, pp. 79–125. Am. Math. Soc., Providence (1994). Updates are available at http://math.nist.gov/mcsd/Reports/2001/nesf/ 75. Luke, Y.L.: The Special Functions and Their Approximations II. Mathematics in Science and Engineering, vol. 53. Academic Press, New York (1969) 76. Luke, Y.L.: Mathematical Functions and Their Approximations. Academic Press, New York (1975)

4 Basic Methods for Computing Special Functions

119

77. MacLeod, A.J.: An instability problem in Chebyshev expansions for special functions. ACM SigNum Newslett. 28(2), 2–7 (1993) 78. Maino, G., Menapace, E., Ventura, A.: Computation of parabolic cylinder functions by means of a Tricomi expansion. J. Comput. Phys. 40(2), 294–304 (1981) 79. Meinardus, G.: Approximation of Functions: Theory and Numerical Methods. Springer Tracts in Natural Philosophy, vol. 13. Springer, New York (1967). Expanded translation from the German edition. Translated by Larry L. Schumaker 80. Michel, N.: Precise Coulomb wave functions for a wide range of complex l, η and z. Comput. Phys. Commun. 176, 232–249 (2007) 81. Michel, N., Stoitsov, M.V.: Fast computation of the Gauss hypergeometric function with all its parameters complex with application to the Pöschl-Teller-Ginocchio potential wave functions. Comput. Phys. Commun. 178, 535–551 (2008) 82. Morris, A.H. Jr.: NSWC library of mathematical subroutines. Naval Surface Warfare Center, Dahlgren Division, Dahlgren, VA (1993) 83. Lloyd Baluk Moshier, S.: Methods and Programs for Mathematical Functions. Ellis Horwood Series: Mathematics and Its Applications. Ellis Horwood, Chichester (1989) 84. Murli, A., Rizzardi, M.: Algorithm 682: Talbot’s method for the Laplace inversion problem. ACM Trans. Math. Softw. 16(2), 158–168 (1990) 85. Noble, C.J.: Evaluation of negative energy Coulomb (Whittaker) functions. Comput. Phys. Commun. 159(1), 55–62 (2004) 86. Olde Daalhuis, A.B., Olver, F.W.J.: On the asymptotic and numerical solution of linear ordinary differential equations. SIAM Rev. 40(3), 463–495 (1998) 87. Paris, R.B., Wood, A.D.: Stokes phenomenon demystified. Bull. Inst. Math. Appl. 31(1–2), 21–28 (1995) 88. Powell, M.J.D.: On the maximum errors of polynomial approximations defined by interpolation and by least squares criteria. Comput. J. 9(4), 404–407 (1967) 89. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C++. Cambridge University Press, Cambridge (2002). The art of scientific computing, 2nd edition, updated for C++ 90. Rice, J.R.: The Approximation of Functions. Vol. I: Linear Theory. Addison-Wesley, Reading (1964) 91. Rizzardi, M.: A modification of Talbot’s method for the simultaneous approximation of several values of the inverse Laplace transform. ACM Trans. Math. Softw. 21(4), 347–371 (1995) 92. Schmelzer, T., Trefethen, L.N.: Computing the gamma function using contour integrals and rational approximations. SIAM J. Numer. Anal. 45(2), 558–571 (2007) (electronic) 93. Schonfelder, J.L.: Chebyshev expansions for the error and related functions. Math. Comput. 32(144), 1232–1240 (1978) 94. Schulten, Z., Anderson, D.G.M., Gordon, R.G.: An algorithm for the evaluation of the complex Airy functions. J. Comput. Phys. 31(1), 60–75 (1979) 95. Schulten, Z., Gordon, R.G., Anderson, D.G.M.: A numerical algorithm for the evaluation of Weber parabolic cylinder functions U (a, x), V (a, x), and W (a, ±x). J. Comput. Phys. 42(2), 213–237 (1981) 96. Seaton, M.J.: Coulomb functions for attractive and repulsive potentials and for positive and negative energies. Comput. Phys. Commun. 146(2), 225–249 (2002) 97. Seaton, M.J.: FGH, a code for the calculation of Coulomb radial wave functions from series expansions. Comput. Phys. Commun. 146(2), 250–253 (2002) 98. Seaton, M.J.: NUMER, a code for Numerov integrations of Coulomb functions. Comput. Phys. Commun. 146(2), 254–260 (2002) 99. Segura, J.: Reliable computation of the zeros of solutions of second order linear ODEs with a fourth order method. SIAM J. Numer. Anal. 48(2), 452–469 (2010) 100. Segura, J.: The zeros of special functions from a fixed point method. SIAM J. Numer. Anal. 40(1), 114–133 (2002) 101. Segura, J., de Córdoba, P.F., Ratis, Yu.L.: A code to evaluate modified Bessel functions based on the continued fraction method. Comput. Phys. Commun. 105(2–3), 263–272 (1997)

120

A. Gil et al.

102. Segura, J., Gil, A.: Parabolic cylinder functions of integer and half-integer orders for nonnegative arguments. Comput. Phys. Commun. 115(1), 69–86 (1998) 103. Segura, J., Temme, N.M.: Numerically satisfactory solutions of Kummer recurrence relations. Numer. Math. 111(1), 109–119 (2008) 104. Shippony, Z., Read, W.G.: A correction to a highly accurate Voigt function algorithm. JQSRT 78(2), 255 (2003) 105. Smith, D.M.: Algorithm 814: Fortran 90 software for floating-point multiple precision arithmetic, gamma and related functions. ACM Trans. Math. Softw. 27(4), 377–387 (2001) 106. Stieltjes, T.-J.: Recherches sur quelques séries semi-convergentes. Ann. Sci. École Norm. Sup. (3) 3, 201–258 (1886) 107. Talbot, A.: The accurate numerical inversion of Laplace transforms. J. Inst. Math. Appl. 23(1), 97–120 (1979) 108. Talman, J.D.: NumSBT: A subroutine for calculating spherical Bessel transforms numerically. Comput. Phys. Commun. 180(2), 332–338 (2009) 109. Temme, N.M.: On the numerical evaluation of the modified Bessel function of the third kind. J. Comput. Phys. 19(3), 324–337 (1975) 110. Temme, N.M.: An algorithm with ALGOL 60 program for the computation of the zeros of ordinary Bessel functions and those of their derivatives. J. Comput. Phys. 32, 270–279 (1979) 111. Temme, N.M.: Special Functions. Wiley, New York (1996). An introduction to the classical functions of mathematical physics 112. Temme, N.M.: Numerical algorithms for uniform Airy-type asymptotic expansions. Numer. Algorithms 15(2), 207–225 (1997) 113. Thompson, W.J.: An Atlas for Computing Mathematical Functions: An Illustrated Guide for Practitioners, with Programs in Fortran 90 and Mathematica. Wiley, New York (1997) 114. Trefethen, L.N., Weideman, J.A.C., Schmelzer, T.: Talbot quadrature and rational approximations. Technical report, Oxford University Computing Laboratory Numerical Analysis Group (2005) 115. Van Deun, J., Cools, R.: Algorithm 858: Computing infinite range integrals of an arbitrary product of Bessel functions. ACM Trans. Math. Softw. 32(4), 580–596 (2006) 116. Van Loan, C.: Computational Frameworks for the Fast Fourier Transform. Frontiers in Applied Mathematics, vol. 10. SIAM, Philadelphia (1992) 117. Wang, Z.X., Guo, D.R.: Special Functions. World Scientific, Teaneck (1989). Translated from the Chinese by Guo and X.J. Xia 118. Watson, G.N.: A Treatise on the Theory of Bessel Functions, 2nd edn. Cambridge Mathematical Library. Cambridge University Press, Cambridge (1944) 119. Weideman, J.A.C.: Optimizing Talbot’s contours for the inversion of the Laplace transform. Technical Report NA 05/05, Oxford U. Computing Lab. (2005) 120. Weideman, J.A.C., Trefethen, L.N.: Parabolic and hyperbolic contours for computing the Bromwich integral. Math. Comput. 76(259), 1341–1356 (2007) 121. Weniger, E.J.: Nonlinear sequence transformations for the acceleration of convergence and the summation of divergent series. Comput. Phys. Rep. 10(5,6), 189–371 (1989) 122. Weniger, E.J.: On the summation of some divergent hypergeometric series and related perturbation expansions. J. Comput. Appl. Math. 32(1–2), 291–300 (1990). Extrapolation and rational approximation (Luminy, 1989) ˇ 123. Weniger, E.J., Cížek, J.: Rational approximations for the modified Bessel function of the second kind. Comput. Phys. Commun. 59(3), 471–493 (1990) ˇ 124. Weniger, E.J., Cížek, J., Vinette, F.: The summation of the ordinary and renormalized perturbation series for the ground state energy of the quartic, sextic, and octic anharmonic oscillators using nonlinear sequence transformations. J. Math. Phys. 34(2), 571–609 (1993) 125. Weniger, E.J.: Computation of the Whittaker function of the second kind by summing its divergent asymptotic series with the help of nonlinear sequence transformations. Comput. Phys. 10, 496–503 (1996)

4 Basic Methods for Computing Special Functions

121

126. Wuytack, L.: Commented bibliography on techniques for computing Padé approximants. In: Padé approximation and its applications, Proc. Conf., Univ. Antwerp, Antwerp, 1979. Lecture Notes in Math., vol. 765, pp. 375–392. Springer, Berlin (1979) 127. Wynn, P.: On a device for computing the em (Sn ) tranformation. Math. Tables Aids Comput. 10, 91–96 (1956) 128. Wynn, P.: Upon systems of recursions which obtain among the quotients of the Padé table. Numer. Math. 8(3), 264–269 (1966) 129. Zhang, S., Jin, J.: Computation of Special Functions. Wiley, New York (1996)

Chapter 5

Melt Spinning: Optimal Control and Stability Issues Thomas Götz and Shyam S.N. Perera

Abstract A mathematical model describing the melt spinning process of polymer fibers is considered. Newtonian and non-Newtonian models are used to describe the rheology of the polymeric material. Two key questions related to the industrial application of melt spinning are considered: the optimization and the stability of the process. Concerning the optimization question, the extrusion velocity of the polymer at the spinneret as well as the velocity and temperature of the quench air serve as control variables. A constrained optimization problem is derived and the first-order optimality system is set up to obtain the adjoint equations. Numerical solutions are carried out using a steepest descent algorithm. Concerning the stability with respect to variations of the velocity and temperature of the quench air, a linear stability analysis is carried out. The critical draw ratio, indicating the onset of instabilities, is computed numerically solving the eigenvalue problem for the linearized equations.

Keywords Melt spinning · Non-Newtonian fluids · Optimal control · First-order optimality system · Linear stability analysis Mathematics Subject Classification (2000) 49K15 · 93C15

5.1 Introduction Many kinds of synthetic textile fibers, like Nylon, Polyester, etc. are manufactured by a so-called melt spinning process. In this process, the molten polymer is extruded T. Götz () Department of Mathematics, TU Kaiserslautern, 67663 Kaiserslautern, Germany e-mail: [email protected] S.S.N. Perera Department of Mathematics, University of Colombo, Colombo 03, Sri Lanka e-mail: [email protected] T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_5, © Springer Science+Business Media B.V. 2011

123

124

T. Götz and S.S.N. Perera

Fig. 5.1 Sketch of the melt spinning process

through a die, called the spinneret, to create a slender, cylindrical jet of viscous polymer, the fiber. Far away from the spinneret, the fiber is wrapped around a drum, which pulls it away at a pre-determined take-up speed, see Fig. 5.1. The take-up speed is much higher than the extrusion speed; in industrial processes the take-up speed is about 50 m/s and the extrusion speed is about 10 m/s, see [2, 10]. The ratio between the take-up speed vL and the extrusion speed v0 is called draw-ratio d = vL /v0 > 1. Hence the filament is stretched considerably in length and therefore it decreases in diameter. The ambient atmosphere temperature is below the polymer solidification temperature such that the polymer is cooled and solidifies before the take-up. In industrial processes a whole bundle of hundreds of single filaments is extruded and spun in parallel, however for the analysis we consider a single filament. The dynamics of melt spinning processes has been studied by many research groups throughout the world during the last decades starting with early works of Kase and Matsuo [8] and Ziabicki [18]. In later works more and more sophisticated descriptions including crystallization kinetics and viscoelastic behavior were developed by several authors in order to achieve a better understanding of the fiber formation process. Up to now it is possible to use the basic models with more or less modifications in different technological aspects of the melt spinning process. The outcome of the melt spinning process depends significantly on the boundary conditions, e.g. the draw ratio, the ambient temperature, the quench air velocity and temperature. The questions of optimizing the fiber production or investigating the stability of the melt spinning process with respect to these external variables has not yet been treated in the literature.

5 Melt Spinning: Optimal Control and Stability Issues

125

The aim of this paper is twofold. First, we want to optimize the melt spinning process with respect to the final temperature, the quench air velocity and temperature. To model the fiber spinning process, we consider both a Newtonian model for the viscosity and a non-Newtonian model, where the viscosity is temperaturedependent. We formulate the optimal control problem as a constrained minimization problem, see [6], and derive formally the corresponding first-order optimality system via the Lagrange functional. For the numerical computation of the optimal control variables we present a steepest descent algorithm using the adjoint variables. Second, we want to investigate the stability of the process. The onset of the socalled draw resonance instability occurring in the melt spinning process as well as related processes like fiber spinning, film casting and tubular film blowing, have been investigated by many research groups theoretically as well as experimentally during the last four decades, see [3–5]. The draw resonance phenomenon was first experimentally observed and named as such in the early 1960s. Draw resonance is of importance, both theoretically as well as practically, since it is closely related to the quality of the final product. Its theoretical analysis involves a fundamental understanding of the nonlinear dynamics of the fiber formation process. Earlier studies [3–5, 11] aimed to understand the physics behind draw resonance. In this paper, we try to analyse the stability of the melt spinning process with respect to the quench air velocity and temperature. The stability analysis of the melt spinning process is based on a linear approach. The paper is organized as follows. In Sect. 5.2, we shortly derived the underlying stationary model. In Sect. 5.3 we introduce the constrained optimization problem and derive the according first-order optimality system. A steepest descent algorithm is proposed to solve the optimization problem numerically. Numerical results of the optimization are shown in Sect. 5.4. Section 5.5 deals with the linear stability analysis for the isothermal Newtonian and the temperature-dependent non-Newtonian case. The numerical results are presented in Sect. 5.6 and concluding remarks can be found in Sect. 5.7.

5.2 Governing Equations for Melt Spinning 5.2.1 Melt Spinning Model Considering the conservation laws for mass, momentum and energy of a viscous polymer jet, one can obtain by averaging over the cross-section of the slender fiber, the following set of equations, see [10–12]: ∂A ∂ + (Av) = 0, ∂t ∂z   √ ∂v ∂v ∂ ρA +v = (Aτ ) − πA Cd ρair v 2 + ρ A g, ∂t ∂z ∂z

(5.2.1a) (5.2.1b)

126

T. Götz and S.S.N. Perera

 ρCp

∂T ∂T +v ∂t ∂z



√ 2α π = − √ (T − T∞ ). A

(5.2.1c)

In the mass balance (5.2.1a), z denotes the coordinate along the spinline, t is the time, A denotes the cross-sectional area of the fiber and v is the velocity of the fiber along the spinline. The density ρ of the polymer is assumed to be constant. In the momentum balance (5.2.1b), the axial stress τ is related via the constitutive equation τ = 3η

dv dz

to the viscosity η [1]. By g we denote the gravitational acceleration and Cd is the air drag coefficient. In the energy equation (5.2.1c), T and Cp denote the temperature and the heat capacity of the polymer, T∞ is the temperature of the quench air and α denotes the heat transfer coefficient between the fiber and the quench air. According to [10], we assume the following relations for the air drag and the heat transfer coefficient Cd = 0.37 Re−0.61 air and  1 1 0.21 64vc2 6 3 α = √ κ Reair 1 + 2 v A0 depending on the Reynolds-number of the quench air flow  2vρair A . Reair = ηair π Here A0 = πR02 denotes the cross-sectional area of the spinneret, ρair , ηair and κ represent the density, viscosity and heat conductivity of the air and vc is the velocity of the quench air. In the Newtonian model, the viscosity η = η0 of the polymer is constant, whereas in the non-Newtonian case, we consider an Arrhenius-type temperature dependence [10]    Ea 1 1 η = η0 exp − , RG T T0 where η0 > 0 is the zero shear viscosity at the initial temperature T0 , Ea denotes the activation energy and RG equals to the gas constant. The system (5.2.1a)–(5.2.1c) is subject to the boundary conditions A = A0 , v = vL

v = v0

and T = T0

at z = L for all t,

where L denotes the length of the spinline.

at z = 0 for all t,

(5.2.1d) (5.2.1e)

5 Melt Spinning: Optimal Control and Stability Issues

127

In the momentum balance (5.2.1b), the air drag and gravity effects are neglected for the sake of brevity and clarity. However, including these effects into the model would not change the general outline of our study.

5.2.2 Dimensionless Form Introducing the dimensionless quantities t∗ =

tv0 , L

v∗ =

v , v0

z∗ =

z , L

T∗ =

T , T0

A∗ =

A A0

and

τ∗ =

τL , η0 v 0

and dropping the star, the system (5.2.1a)–(5.2.1e) can be re-formulated in dimensionless form ∂A ∂ + (Av) = 0, ∂t ∂z

(5.2.2a)

  1 ∂v 3 ∂ ∂v v2 ∂v +v = ηA −C√ + , ∂t ∂z Re A ∂z ∂z Fr A

∂T T − T∞ ∂T , +v = −γ √ ∂t ∂z A where Re = πC √d ρair L A0 ρ

ρLv0 η0

is the Reynolds number, Fr =

(5.2.2b) (5.2.2c)

v02 gL

stands for the Froude number,

and γ = are the scaled air drag and heat transfer coefficients. C= The viscosity is given by  1 for the Newtonian model, 

 η= (5.2.2d) exp REGaT0 T1 − 1 for the non-Newtonian model. 2αL ρCp v0 R0

The boundary conditions read as A(0) = 1, v(1) = d

v(0) = 1

and T (0) = 1 for all t,

for all t,

(5.2.2e) (5.2.2f)

where d = vL /v0 > 1 denotes the draw-ratio. Now, we can easily read off the steady state equations of melt spinning. The continuity equation (5.2.2a) reduces to A = 1/v and writing the second order momentum equation (5.2.2b) as two first order equations, we finally arrive at dv τ = , dz 3η

(5.2.3a)

dτ vτ τ2 1 = Re + − + Cv 3/2 , dz 3η 3ηv Fr

(5.2.3b)

128

T. Götz and S.S.N. Perera

dT T − T∞ , = −γ √ dz v

(5.2.3c)

subject to the boundary conditions v(0) = 1 and T (0) = 1,

(5.2.3d)

v(1) = d.

(5.2.3e)

In the sequel we will consider two main questions related to the model (5.2.3a)– (5.2.3e) (1) How to choose the process conditions, i.e. the external air velocity vc and temperature T∞ , such that the fiber is solid at the take-up? Moreover, we wish to maximize the mass flux, i.e. the inflow velocity v0 . In Sects. 5.3 and 5.4 this optimization problem is treated using optimal control techniques. Some numerical simulations illustrate our findings. (2) Is the system (5.2.3a)–(5.2.3e) stable with respect to small perturbations of the process parameters? In Sect. 5.5 we apply a linear stability analysis to investigate this question. Again, some numerical computations highlight the results, see Sect. 5.6.

5.3 Optimal Control of the Melt Spinning Process We want to control the temperature profile of the fiber, such that the final temperature T (1) is below the solidification point Ts∗ = Ts /T0 . On the other hand we want to maximize the outflow, i.e. maximize v0 . The air temperature T∞ and the air velocity vc can be influenced and hence serve as control variables. Therefore, we consider the following cost functional J = J (y, u) = J1 + J2 + J3 + J4 = − ω1 u3 + ω2 (y3 (1) − Ts∗ ) ω3 1 ω4 1 2 + (u2 (z) − Tref ) dz + u1 (z)2 dz, 2 0 2 0

(5.3.1)

where y = (v, τ, T ) ∈ Y denotes the vector of state variables and u = (vc , T∞ , v0 ) ∈ U are the controls. The weighting coefficients ωi > 0, i = 1, . . . , 4 allow to adjust the cost functional to different scenarios. The larger the coefficient ω1 , the more importance is given to the maximization of the outflow; ω2 and ω3 measure the influence of the temperature constraints and the larger ω4 , the more weight is given to a low and hence “cheap” cooling air velocity. The actual choice of the numerical values of the weighting coefficients ω1 , . . . , ω4 depends of the demands of the applicant. Summarizing, we consider the following constrained optimization problem minimize J (y, u) with respect to u, subject to (5.2.3a)–(5.2.3e).

(5.3.2)

In the sequel, we will address this problem using the calculus of adjoint variables.

5 Melt Spinning: Optimal Control and Stability Issues

129

5.3.1 First-Order Optimality System In this section we introduce the Lagrangian associated to the constrained minimization problem (5.3.2) and derive the system of first-order optimality conditions. Let Y = C 1 ([0, 1]; R3 ) be the state space consisting of triples of differentiable functions y = (v, τ, T ) denoting velocity, stress and temperature of the fiber. Further, let U = C 1 ([0, 1]; R2 ) × R be the control space consisting of a pair (u1 , u2 ) = (vc , T∞ ) of differentiable functions, i.e. air velocity and temperature, and a scalar u3 = v0 interpreted as the inflow velocity. We define the operator e = (ev , eτ , eT ) : Y × U → Y ∗ via the weak formulation of the state system (5.2.3a)–(5.2.3e): e(y, u), ξ Y,Y ∗ = 0,

∀ξ ∈ Y ∗ ,

where ·, ·Y,Y ∗ denotes the duality pairing between Y and its dual space Y ∗ . Now, the minimization problem (5.3.2) reads as minimize J (y, u) with respect to u ∈ U ,

subject to e(y, u) = 0.

Introducing the Lagrangian L : Y × U × Y ∗ → R defined as L(y, u, ξ ) = J (y, u) + e(y, u), ξ Y,Y ∗ , the first-order optimality system reads as ∇y,u,ξ L(y, u, ξ ) = 0. Considering the variation of L with respect to the adjoint variable ξ , we recover the state system e(y, u) = 0 or in the classical form dy = f (y, u), dz where

with v(0) = 1, v(1) = d, T (0) = 1, ⎛

⎜ vτ f (y, u) = ⎜ ⎝Re 3η +

τ 3η τ2 3η

−γ

(5.3.3)

⎞ ⎟

1 3/2 ⎟ Fr + Cv ⎠. T −T ∞ √ v

−

Second, taking variations of L with respect to the state variable y we get the adjoint system Jy (y, u) + ey∗ (y, u)ξ = 0

130

T. Götz and S.S.N. Perera

or in classical form −

dξ = F (y, u, ξ ), dz

with ξq (0) = 0, ξq (1) = 0, ξT (1) = −ω2 ,

where

 F (y, u, ξ ) =

∂f ∂y

(5.3.4)

 ξ.

Finally, considering variations of L with respect to the control variable u in a direction of δu we get the optimality condition Ju (y, u), δu + eu (y, u)δu, ξ  = 0.

(5.3.5)

In the optimum, this holds for all δu ∈ U .

5.3.2 Decent Algorithm To solve the nonlinear first-order optimality system consisting of (5.3.3), (5.3.4) and (5.3.5), we propose an iterative steepest-descent method. 1. Set k = 0 and choose initial control u(0) ∈ U . 2. Given the control u(k) . Solve the state system (5.3.3) with a shooting method to obtain y (k+1) . 3. Solve the adjoint system (5.3.4) with a shooting method to obtain ξ (k+1) . 4. Compute the gradient g (k+1) of the cost functional. 5. Update the control u(k+1) = u(k) − βg (k+1) for a step size β > 0. 6. Compute the cost functional J (k+1) = J (y (k+1) , u(k+1) ). 7. If |g (k+1) | ≥ δ, goto 2. Here, δ > 0 is some prescribed relative tolerance for the termination of the optimization procedure. In each iteration step, we need to solve two boundary value problems, i.e. the state system (5.3.3) and the adjoint system (5.3.4) in the steps 2 and 3 of the algorithm. Both systems are solved using a shooting method based on a Newton-iteration. The main steps of the shooting method for solving, e.g. the state system (5.3.3) are the following. Choose some initial guess s for y2 (0) and denote by y(z; s) the solution of the initial value problem dy = f (y, u), dz

with y1 (0) = 1, y2 (0) = s, y3 (0) = 1.

Now we introduce new dependent variables x(z; s) =

∂y ∂s

(5.3.6)

5 Melt Spinning: Optimal Control and Stability Issues

131

and define a second system as follows ∂x = ∂z



 ∂f x, ∂y

with x1 (0; s) = 0, x2 (0; s) = 1, x3 (0; s) = 0.

(5.3.7)

The solution of y(z; s) of the initial value problem (5.3.6) coincides with the solution y(z) of the boundary value state system (5.3.3) provided that the value s can be found such that φ(s) = y1 (1; s) − d = 0. Using the system (5.3.7), φ (s) can be computed as φ (s) = x1 (1; s). Applying Newton’s method generates a sequence (sn )n∈N sn+1 = sn −

φ(sn ) φ (sn )

for a given initial guess s0 .

If the initial guess s0 is a sufficiently good approximation to the required root of φ(s) = 0, the theory of the Newton-iteration method ensures that the sequence (sn )n∈N converges to the desired root s. A crucial ingredient for the convergence of the decent algorithm is the choice of the step size β in the direction of the gradient, see step 5 of the algorithm. Clearly, the best choice would be the result of a line search β ∗ = argmin J (uk − βgk ). β>0

However this is numerically quite expensive although it is a one dimensional minimization problem. Instead of the exact line search method, we use an approximation based on a quadratic polynomial method [9] in order to find β ∗ such that we minimize J (uk − βgk ). We construct a quadratic polynomial p(β) to approximate J (uk − βgk ) using following data points p(0) = J (uk ),

p(1) = J (uk − gk ),

p (0) = −∇J (uk )T gk < 0.

Now, the quadratic approximation to our cost functional reads as p(β) = p(0) + p (0)β + (p(1) − p(0) − p (0))β 2 and its global minimum is located at β∗ =

−p (0) ∈ (0, 1). 2(p(1) − p(0) − p (0))

132 Table 5.1 Processing conditions and parameter values

T. Götz and S.S.N. Perera Parameter

Value

Unit

ρ

0.98

g/cm3

Cp

0.46

cal/(g°C) cm

R0

0.01

T0

300

°C

v0

16.667

m/s

vL

50

m/s

L

1

m

T∞

24

°C

Tref

20

°C

vc

0.4

m/s

ρair

1.189

kg/m3

ηair

1.819 × 10−5

pa s

RG

1.9859

cal/(K mol)

Ea

13500

cal/mol

κ

0.0257

W/(m K)

5.4 Optimal Control Results Both the state and the adjoint system (5.3.3) and (5.3.4) were solved using the M ATLAB routine ode23tb. This routine uses an implicit method with backward differentiation to solve stiff differential equations. As weighting coefficients for the cost functional (5.3.1) we use (ω1 , . . . , ω4 ) = (1, 1, 1.5, 2.5). The relevant simulation conditions and parameter values are shown in Table 5.1. These values are typical for Nylon-66.

5.4.1 Newtonian Model Figure 5.2 shows the spinline velocity and temperature as well as the cooling air velocity and temperature profiles before and after optimization for the Newtonian model. Some of the intermediate profiles are also included in the graphs. The corresponding cost functional is shown on the left in Fig. 5.3. It can be seen that in the optimum, the final temperature is below 50°C. The extrusion velocity drops from 16.67 m/s to 12.65 m/s. The optimal air temperature is more or less close to 20°C, which we considered as a reference temperature. In this case, the optimal air velocity is quite large near the spinneret and drops after the first 30 cm almost to zero. Figures 5.4 and 5.5 visualize the optimal air velocity and air temperature profiles in Newtonian model for different weighting coefficients.

5 Melt Spinning: Optimal Control and Stability Issues

133

Fig. 5.2 Graphs for the spinline velocity (top-left) and temperature (top-right) as well as for the air velocity (bottom-left) and temperature (bottom-right). The dashed curve is the initial profile, while dotted curves correspond to intermediate profiles. The final, optimized profile is shown with the solid line

Fig. 5.3 Reduction of the cost functional for the Newtonian (left) and non-Newtonian model (right)

5.4.2 Non-Newtonian Model Figure 5.6 visualizes the spinline velocity and temperature as well as the cooling air velocity and temperature before and after optimization for the non-Newtonian model. The reduction of the cost functional is shown on the right in Fig. 5.3.

134

T. Götz and S.S.N. Perera

Fig. 5.4 Optimal air velocity in the Newtonian case for different weighting coefficients

Fig. 5.5 Optimal air temperature in the Newtonian case for different weighting coefficients

As for the Newtonian model, the optimized final temperature is well below 50°C and the optimal air temperature profile is more or less close to 20°C. The extrusion velocity drops from 16.67 m/s to 10.42 m/s. In the optimal state, the air velocity reaches a peak value near the spinneret exit point and just after this point it almost close to zero. Figures 5.7 and 5.8 show the optimized air velocity and temperature profiles in the non-Newtonian case for different cost coefficients.

5.5 Linear Stability Analysis Now, we turn our attention the second question treated in this paper. Are the solutions of the steady state system (5.2.3a)–(5.2.3e) stable with respect to small pertur-

5 Melt Spinning: Optimal Control and Stability Issues

135

Fig. 5.6 Graphs for the spinline velocity (top-left) and temperature (top-right) as well as for the air velocity (bottom-left) and temperature (bottom-right). The dashed curve is the initial profile, while dotted curves correspond to intermediate profiles. The final, optimized profile is shown with the solid line Fig. 5.7 Optimal air velocity in the non-Newtonian model for different weighting coefficients

bations of the process conditions? For the sake of simplicity of notation, we assume Fr = 0 and C = 0, i.e.—we neglect the influence of gravity and air drag. Including those effects in the analysis, however, is straightforward.

136

T. Götz and S.S.N. Perera

Fig. 5.8 Optimal air temperature in the non-Newtonian model for different weighting coefficients

Let vs (z) and Ts (z) denote the solutions of the steady state system (5.2.3a)– (5.2.3e). Then As (z) = 1/vs (z) solves the stationary version of the mass balance (5.2.2a). Now, we consider small perturbations φ, ϕ and θ of the state variables A, v and T around their steady states. A linearization leads to A(t, z) = As (z) + φ(z) exp(t), v(t, z) = vs (z) + ϕ(z) exp(t),

(5.5.1)

T (t, z) = Ts (z) + θ (z) exp(t), where  ∈ C is a complex eigenvalue that accounts for the growth rate of the perturbation. Substituting (5.5.1) into the transient system (5.2.2a)–(5.2.2f) and neglecting higher order terms yields the following linearized equations. In the case of a isothermal Newtonian flow, i.e. η ≡ 1, we get     ϕ

φ = − (vs φ) + , vs     3 ϕ

ϕ = −(vs ϕ) + vs (vs φ) + vs , Re vs

(5.5.2a) (5.5.2b)

subject to the boundary conditions 0 = φ(0) = ϕ(0) = ϕ(1).

(5.5.2c)

In the non-Newtonian case, we get     ϕ , φ = − (vs φ) + vs

(5.5.3a)

5 Melt Spinning: Optimal Control and Stability Issues

137

        3η ϕ 3Ea ηθ vs

+ T ϕ vs (vs φ) + vs − v s s Re vs vs RG Re T0 Ts2     3Ea η Ea 2

+ vs Ts θ + − θ , (5.5.3b) RG Re T0 Ts RG Re T0 Ts3 Ts2

ϕ = −(vs ϕ) +

11 5 1 1 1 θ = C0 (Ts − T∞ )vs6  6 φ − C0 vs6  6 θ − vs θ 2   13 1 64 1 − 16 2 −6 − 56

+ C0 vc vs (Ts − T∞ ) − C0 vs (Ts − T∞ ) 6 − Ts ϕ, 3 3 (5.5.3c)

subject to the boundary conditions 0 = φ(0) = ϕ(0) = θ (0) = ϕ(1),

(5.5.3d)

where   2  vc and  = 1 + 64 vs

C0 =

 1 0.42Lκ 2R0 v0 ρair 3 . ηair ρCp v0 R02

5.5.1 Numerical Solution of the Eigenvalue Problem Now we discretize the linearized system (5.5.2a)–(5.5.2c) or (5.5.3a)–(5.5.3d) respectively, using a finite-difference scheme on a uniform grid zi = ih, i = 0, . . . , n with grid size h = 1/n. The derivatives are approximated using centered differences dy yi+1 − yi−1 ≈ dz 2h

and

d 2 y yi+1 − 2yi + yi−1 ≈ , dz2 h2

where i = 1, . . . , n − 1 ranges through the interior points. At the endpoint i = n we apply a backward difference formula dy yn − yn−1 ≈ , dz h In each point zi we have two unknowns, namely φi , ϕi , in the isothermal Newtonian case and three unknowns φi , ϕi and θi in the non-Newtonian case. Due to the boundary conditions φ0 = ϕ0 = ϕn = 0 and θ0 = 0, we end up with m = 2n − 1 degrees of freedom in the isothermal Newtonian model and m = 3n − 1 in the nonNewtonian model. Plugging the difference approximations into the linearized systems (5.5.2a)– (5.5.2c) or (5.5.3a)–(5.5.3d) and re-arranging the terms, we arrive at the algebraic eigenvalue problem Y = MY,

(5.5.4)

138 Table 5.2 Critical draw ratio in the isothermal Newtonian case

T. Götz and S.S.N. Perera dc

Literature source

20.218

This study

20.20

Pearson & Shah [14]

20.22

van der Hout [17]

20.218

Jung, Song, Hyun [7]

where Y = [φ1 , ϕ1 , . . . φn−1 , ϕn−1 , φn ]T ∈ Rm for the isothermal Newtonian case and Y = [φ1 , ϕ1 , θ1 , . . . , φn−1 , ϕn−1 , θn−1 , φn , θn ]T in the non-Newtonian case and M ∈ Rm×m arises from the discretization. The stability is determined by the real parts of the eigenvalues . Positive real parts indicate instability due to unbounded growth of the state variables with time. The critical draw-ratio dc is defined to be the minimal draw-ratio, where instability, i.e. an eigenvalue  with Re() > 0 occurs. In order to determine the critical drawratio dc , we have to compute the eigenvalues of the system (5.5.4). To do so, we first solve the stationary state system (5.2.3a)–(5.2.3e) for a given draw ratio d and given parameters vc and T∞ . In this step, we use the M ATLAB-routine ode23tb. This routine uses an implicit method with backward differentiation to solve stiff differential equations. It is an implementation of TR-BDF2 [15], an implicit two stage Runge–Kutta formula where the first stage is a trapezoidal rule step and the second stage is a backward differentiation formula of order two. The computed stationary solution As , vs and Ts is then plugged into the eigenvalue problem (5.5.4). The eigenvalue problem is solved using the M ATLAB-routine eigs based on the ARPACK-library for large sparse matrices. This algorithm is based upon an algorithmic variant of the Arnoldi process called the implicitly restarted Arnoldi method, see [13, 16]. As a result, we finally obtain the eigenvalues  as functions of d.

5.6 Stability Results Table 5.2 compares the computed critical draw ratio dc in the isothermal Newtonian case with values reported in literature. An excellent agreement is found. Table 5.3 reports the critical draw ratio for the non-Newtonian model depending on the velocity of the quench air. These simulations are carried out using a fixed air temperature of 24°C. With increasing air velocity, the onset of instability is shifted to higher draw ratios. Hence, the stability of the spinning process can be improved by increasing the quench air velocity. Figure 5.9 visualizes the relation between the air velocity vc and the critical draw ratio in the non-Newtonian case. The simulation results suggest an almost linear relation between the air velocity vc and the critical draw ratio. A regression analysis yields—as a rule of thumb— dc = 3.4vc + 27, where the air velocity vc is measured in m/s.

(5.6.1)

5 Melt Spinning: Optimal Control and Stability Issues Table 5.3 Critical draw ratio for the non-Newtonian case depending on the air velocity (air temperature 24°C fixed)

Air velocity [m/s]

139 Critical draw ratio

0

30.50

2.5

36.12

5.0

45.27

7.7

52.49

10.0

60.04

15.0

76.94

20.0

93.31

25.0

111.44

30.0

135.17

Fig. 5.9 Plot of the critical draw ratio vs. the quench air velocity for the non-Newtonian model

Figure 5.10 shows the critical draw ratio vs. the temperature of the quench air at a fixed air velocity of 0 m/s. The critical draw ratio decreases with a cooler quench air; the according numerical values are reported in Table 5.4. Again a linear regression was carried out. Measuring the air temperature T∞ in °C, one finds dc = 0.21T∞ + 25 indicating a weak dependence of the critical draw ratio on the air temperature.

5.7 Conclusions We considered two key questions related to the industrial application of the melt spinning process: the optimization and the stability of the process. The mathematical model describing the melt spinning process of polymer fibers uses Newtonian and non-Newtonian models for the rheology of the polymeric material.

140 Table 5.4 Critical draw ratio depending on the air temperature (air velocity 0 m/s)

T. Götz and S.S.N. Perera Air temperature [°C]

Critical draw ratio

10

27.40

15

28.20

20

29.40

24

30.50

30

31.60

35

32.50

Fig. 5.10 Critical draw ratio vs. air temperature

Concerning the optimization question, the aim was to maximize the outflow, minimize the air velocity and air temperature and to get the final spinline temperature below the fiber solidification temperature. Defining an appropriate cost functional, we converted the problem into a constrained optimization problem and derived the first-order optimality system. Based on the adjoint variables, we propose a steepest descent algorithm with suitable step size control to solve the problem numerically. Simulations show the applicability of our approach and yield the desired results. After the optimization, the final fiber temperature is well below the solidification temperature. The velocity of the quench air was reduced as much as possible. Regarding the stability of the process, instabilities may arise, if some process variables exceeded a certain critical value. Furthermore, instabilities lead to irregular fibers or induces breakage of the individual filaments along the spinline. Hence, a stability analysis of the process with respect to process parameters is needed. We analysed the stability of the melt spinning process with respect to the quench air velocity and the temperature. From this results the following points can be concluded. In the Newtonian regime, the stability domain is independent of the quench air velocity and air temperature. The critical draw ratio is found to be dc ≈ 20.218; in accordance with the results of other research groups.

5 Melt Spinning: Optimal Control and Stability Issues

141

In the non-Newtonian case, the stability domain strongly depends on the quench air velocity. A linear regression reveals the approximate relation dc = 3.4vc + 27, hence the process stability can be improved by increasing the air velocity. The quench air temperature only has a minor influence on the process stability. However, there is a weak trend towards improved stability with higher air temperature. Acknowledgements The research of S.S.N. Perera was supported by a grant from the DAAD (German Academic Exchange Services).

References 1. Bird, R.B., Amstrong, R.C., Hassager, O.: Dynamics of Polymeric Liquids, 2nd edn. Fluid Mechanics, vol. 1. Wiley, New York (1987) 2. Brünig, H., Roland, H., Blechschmidt, D.: High filament velocities in the underpressure spunbonding nonwoven process. In: IFJ, pp. 129–134, December (1997) 3. Gelder, D.: The stability of fiber drawing processes. Ind. Eng. Chem. Fundam. 10, 534–543 (1978) 4. Hyun, J.C.: Theory of draw resonance: Part I. Newtonian fluids. AIChE J. 24(3), 418–422 (1978) 5. Hyun, J.C.: Theory of draw resonance: Part II. Power-law and Maxwell fluids. AIChE J. 24(3), 423–426 (1978) 6. Ito, K., Ravindran, S.S.: Optimal control of thermally convected fluid flows. SIAM J. Sci. Comput. 19(6), 1847–1869 (1998) 7. Jung, H.W., Song, H.S., Hyun, J.C.: Draw resonance and kinematic waves in viscoelastic isothermal spinning. AIChE J. 46(10), 2106–2110 (2000) 8. Kase, S., Matsuo, T.: Studies on melt spinning, fundamental equations on the dynamics of melt spinning. J. Polym. Sci., Part A 3, 2541–2554 (1965) 9. Kelley, C.T.: Iterative Methods for Optimization. SIAM, Philadelphia (1999) 10. Langtangen, H.P.: Derivation of a mathematical model for fiber spinning. Department of Mathematics, Mechanics Division, University of Oslo, December (1997) 11. Lee, J.S., Jung, H.W., Hyun, J.C., Seriven, LE: Simple indicator of draw resonance instability in melt spinning processes. AIChE J. 51(10), 2869–2874 (2005) 12. Lee, J.S., Shin, D.M., Jung, H.W., Hyun, J.C.: Transient solution of the dynamics in low-speed fiber spinning process accompanied by flow-induced crystallization. J. Non-Newtonian Fluid Mech. 130, 110–116 (2005) 13. Lehoucq, R.B., Sorensen, D.C., Yang, C.: ARPACK users’ guide: Solution of large scale eigenvalue problems with implicitly restarted Arnoldi methods. Department of computational and Applied mathematics. Rice University, Houston, Texas (1997) 14. Pearson, J.R.A., Shah, Y.T.: On the stability of isothermal and nonisothermal fiber spinning of power law fluids. Ind. Eng. Chem. Fund. 13(2), 134–138 (1974) 15. Shampine, L.F., Reichelt, M.W.: The MATLAB ODE Suite. SIAM J. Sci. Comput. 18, 1–22 (1997) 16. Sorensen, D.C.: Implicitly restarted Arnoldi/Lanczos methods for large scale eigenvalue calculations. Department of Computational and Applied Mathematics, Rice University, Houston, Texas (1995) 17. van der Hout, R.: Draw resonance in isothermal fibre spinning of Newtonian and power-law fluids. Eur. J. Appl. Math. 11, 129–136 (2000) 18. Ziabicki, A.: Fundamentals of Fiber Formation. Wiley, New York (1976)

Chapter 6

On Orthonormal Polynomial Solutions of the Riesz System in R3 K. Gürlebeck and J. Morais

Abstract The main goal of the paper is to deal with a special orthogonal system of polynomial solutions of the Riesz system in R3 . The restriction to the sphere of this system is analogous to the complex case of the Fourier exponential functions {einθ }n≥0 on the unit circle and has the additional property that also the scalar parts of the polynomials form an orthogonal system. The properties of the system will be applied to the explicit calculation of conjugate harmonic functions with a certain regularity.

Keywords Quaternionic analysis · Homogeneous monogenic polynomials · Riesz system · Harmonic conjugates Mathematics Subject Classification (2000) 30G35 · 31B05 · 31B35

6.1 Introduction Quaternionic analysis is nowadays a comfortable tool to solve boundary value problems of mathematical physics given in three or four dimensions. The reader who wishes a comprehensive account of such theory and its more recent achievements can profitably consult [28–31, 33, 34, 43–45] and elsewhere. Related to concrete boundary value problems, we need special systems of functions (well-adapted to series expansions and special geometries) that can be used theoretically and/or numerically in stable procedures. A starting point for these considerations relies on K. Gürlebeck () · J. Morais Institut für Mathematik/Physik, Bauhaus-Universität Weimar, Coudraystr. 13B, 99421 Weimar, Germany e-mail: [email protected] J. Morais e-mail: [email protected] T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_6, © Springer Science+Business Media B.V. 2011

143

144

K. Gürlebeck, J. Morais

the concept of null solutions of higher-dimensional Cauchy-Riemann or Dirac systems, known as monogenic functions. The present work is intended as a study on the best approximation of a given class of monogenic functions in the quaternionic analysis setting. For a general orientation the reader is suggested to read some of the works [9, 11, 12] and [39], of which the first three deal entirely with the fourdimensional case and the last the three-dimensional case. In view of many applications to boundary value problems and for simplicity one is mainly interested in the approximation of functions defined in domains of R3 and with values again in R3 . One possible approach is then, roughly speaking, the identification of vectors from R3 with the so-called reduced quaternions. Unfortunately, such a structure is not closed under the quaternionic multiplication. For those reasons, Sect. 6.3 studies the approximation of monogenic functions in the linear space of square integrable functions over R. This is also supported by the fact that the operators of some boundary value problems are not H-linear but are nevertheless efficiently treated by means of quaternionic analysis tools (e.g., Lamé system [30], Stokes system [29]). Therefore, we aim to approximate reduced quaternion-valued monogenic functions in terms of well-defined homogeneous monogenic polynomials. To pave the way, we choose a convenient system of homogeneous monogenic polynomials that should replace the holomorphic function systems (polynomials) used in the complex case. We wish to deal with a system that has a simple structure, in the sense that the underlying functions can be explicitly calculated and the numerical costs are only slightly growing. To define some differential operators on the basis functions and to extend them to more general functions via Fourier expansions, one has forcibly to consider complete orthonormal systems. This gives then via continuous extension a basis to approximate monogenic functions or solutions of more general differential equations by series expansions in terms of homogeneous monogenic polynomials. The latter is also necessary to ensure the numerical stability of the considered approximations. The remarks above cover only a part of the problem. Analogously to the onedimensional case, the following facts have also to be considered: 1. Homogeneous monogenic polynomials of different order are orthogonal; 2. The scalar parts of the basis polynomials are orthogonal to each other; 3. All (hypercomplex-)derivatives of the basis elements deliver again basis elements, one degree lower. The general problem of approximating a monogenic function by monogenic polynomials started basically with early works of Fueter [21, 22]. This was managed by means of the notion of hypercomplex variables. Later, in [6] and [36] it is shown that a monogenic function can be developed locally as a Taylor series in terms of homogeneous monogenic polynomials based on those variables. The resulting homogeneous monogenic polynomials are the so-called Fueter polynomials. In this line of reasoning Leutwiler in 2001 (see [35]), based on these polynomials, constructed a basis in the real-linear Hilbert space of reduced quaternion-valued homogeneous monogenic polynomials in R3 . His results were recently generalized to arbitrary dimensions in the framework of a Clifford algebra by Delanghe in 2007 [16]. The

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

145

approach followed from both authors relies on the notion of harmonic conjugates. However, a drawback remains in the fact that in general the Fueter polynomials and their associated scalar parts are not orthogonal with respect to the real-inner product (see [39] Chap. 2 for a special approach). A naive approach is to apply the GramSchmidt procedure for normalization of these polynomials. Unfortunately, this orthonormalization process is not easy to handle and it is numerically highly unstable. For this reason, research has been directed to the construction of a more suitable basis. A different effort in this direction was done by Ryan in 1986 [41]. Therein, the author constructed a complete orthonormal system of homogeneous monogenic polynomials for even dimensions. However, his system is not appropriate for our case since we consider only functions defined in domains of the Euclidean space R3 , of odd dimension. In this context we also mention the works by Brackx, Delanghe and Sommen in [6] and the first author in [23]. The authors have constructed shifted systems of Cauchy kernels as rational basis systems to approximate a monogenic function. Although the constructed systems are complete and give a simple structured basis, they are not orthogonal and do not have the property of having (hypercomplex-)derivatives within the same basis. Also, in these systems the construction of a series expansion is not possible and derivatives of the basis functions are not even finite linear combinations of the original basis functions. Other intensive works in the 90-ties were done by Abul-Ez and Constales (see e.g. [1–3]). The authors have constructed so-called special monogenic polynomials in the framework of Clifford analysis, as an extension of the basic sets of polynomials of one complex variable that appeared in the thirties in the work of Whittaker and resumed later in his book [49]. The completeness of the set in the space of homogeneous monogenic polynomials is not considered. Recently, Falcão and Malonek [18–20] have also constructed a set of special homogeneous monogenic polynomials involving only products of a hypercomplex variable and its hypercomplex conjugate. It is proved that the obtained set is an Appell set of monogenic polynomials with respect to the (hypercomplex-)derivative. Since the authors have just constructed one polynomial for each degree, the latter is not enough to form a basis for the space of square integrable monogenic functions. In a different setting, there are several other works as for example those by Cnops [13], Brackx, De Schepper and Sommen [8], De Bie and Sommen [14], related to Clifford-Hermite and Gegenbauer polynomials. An important effort was done recently by Cação, worked out in her thesis [9] and in follow-up papers [11, 12]. Cação et al. have constructed a complete orthonormal system of homogeneous monogenic polynomials in the unit ball of R3 and taking values in the full quaternions (R3 → R4 case). In [10] it is proved that the rate of convergence for a monogenic function approximated by this system of homogeneous monogenic polynomials has the same quality as in the similar case of a harmonic function approximated by homogeneous harmonic polynomials (for comparison see [32]). In particular one part of the resulting polynomials, contrary to the sets described above, carry the property of having monogenic (hypercomplex-)derivatives within the same basis one degree lower, like in the complex case. Partially motivated by the strategy adopted by the previous authors,

146

K. Gürlebeck, J. Morais

in [39] the second author constructed a basis (R3 → R3 case) such that property 2 of the previous list has been also regarded. In this paper we mainly follow the notations as in [39]; cf. also [27]. The paper is organized as follows. After some preliminaries, in Sect. 6.3 we start by presenting the orthogonal polynomial system exploited in [39]. The usage of a system of spherical harmonics in R3 (considered e.g. in [42]) for the construction of our system allows to use some well-known results like its orthogonality on the unit sphere. As a consequence of the interrelation between spherical harmonics and Legendre polynomials, the constructed homogeneous monogenic polynomials are related to the Legendre polynomials as well. Surprisingly, the set formed by the scalar parts of the basis elements are multiples of the used spherical harmonics. This reproducing property is in no way self-evident and it establishes a special relationship between quaternionic analysis and harmonic analysis. By the nature of the given approach, which is genuinely constructive, this relation is also fundamental in the study of the notion of conjugate harmonicity. As an illustration, we are able to set up under some particular asymptotic conditions on the Fourier coefficients of a scalar-valued function U , a (sufficient) condition that ensures the existence of a vector-valued function V conjugate to U such that U + V is square integrable and reduced quaternion-valued monogenic. It is also described an explicit formula for the construction of V .

6.2 Basic Notions and Terminology Consider a holomorphic function f (z) = u(x, y) + iv(x, y) defined in a domain  ⊂ C. As is well-known, its real and imaginary parts are real-valued harmonic functions in , satisfying the so-called Cauchy-Riemann system  ∂u ∂v ∂x = ∂y , ∂u ∂y

∂v = − ∂x .

As in the case of two variables, we may characterize the analogue of the Cauchy-Riemann system in an open domain of the Euclidean space R3 . More precisely, consider a vector-valued function f ∗ = (u0 , u1 , u2 ) whose components ui = ui (x0 , x1 , x2 ) are real functions of real variables x0 , x1 , x2 for which  2 i=0 ∂xi ui = 0, ∂xj ui − ∂xi uj = 0

(i = j, 0 ≤ i, j ≤ 2)

or, equivalently, in a more compact form:  div f ∗ = 0, rot f ∗ = 0.

(6.2.1)

This 3-tuple f ∗ is said to be a system of conjugate harmonic functions in the sense of Stein-Weiß [46, 47] and system (6.2.1) is called the Riesz system [40].

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

147

The system (6.2.1) can be obtained naturally using the quaternionic algebra. Let H := {a = a0 + a1 e1 + a2 e2 + a3 e3 , ai ∈ R, i = 0, 1, 2, 3} be the algebra of the real quaternions, where the imaginary units ei (i = 1, 2, 3) are subject to the multiplication rules e21 = e22 = e23 = −1, e1 e2 = e3 = −e2 e1 ,

e2 e3 = e1 = −e3 e2 ,

e3 e1 = e2 = −e1 e3 .

For the standard basis system of the Hamiltonian quaternions one uses the original notation {1, i, j, k}. In this paper we use a more general notation, {1, e1 , e2 , e3 }, more suited for a future extension. The real number Sc(a) := a0 and Vec(a) := a1 e1 + a2 e2 + a3 e3 are the scalar and vector parts of a, respectively. Analogously to the complex case, the conjugate√of a is the quaternion a = a0 − a1 e1 − a2 e2 − a3 e3 . The norm of a is given by |a| = aa, and coincides with its corresponding Euclidean norm, as a vector in R4 . We consider the subset A := spanR {1, e1 , e2 } of H. Its elements are usually called reduced quaternions. The real vector space R3 is to be embedded in A via the identification x := (x0 , x1 , x2 ) ∈ R3 ↔ x := x0 + x1 e1 + x2 e2 ∈ A. We denote by x the vectorial part of the reduced quaternion x, that is, x := x1 e1 + x2 e2 . Also, we emphasize that A is a real vectorial subspace, but not a sub-algebra, of H. Let  be an open subset of R3 with a piecewise smooth boundary. A reduced quaternion-valued function or, briefly, an A-valued function is a mapping f :  −→ A such that f(x) = [f(x)]0 +

2 

[f(x)]i ei ,

x ∈ ,

i=1

where the coordinate-functions [f]i (i = 0, 1, 2) are real-valued functions defined in . Properties such as continuity, differentiability or integrability are ascribed coordinate-wise. We will work with the real-linear Hilbert space of square integrable A-valued functions defined in , that we denote by L2 (; A; R). The real-valued inner product is defined by  f, g L2 (;A;R) =

Sc(fg) dV , 

where dV denotes the Lebesgue measure on .

(6.2.2)

148

K. Gürlebeck, J. Morais

One possibility to generalize complex holomorphy is offered by following the Riemann approach. In this context for continuously real-differentiable functions f :  −→ A, we consider the (reduced) quaternionic operator D=

∂ ∂ ∂ + e1 + e2 ∂x0 ∂x1 ∂x2

(6.2.3)

thet is called generalized Cauchy-Riemann operator on R3 , as it corresponds to the 3-dimensional extension of the classical Cauchy-Riemann operator ∂z . In the same way, we define the conjugate quaternionic Cauchy-Riemann operator D=

∂ ∂ ∂ − e1 − e2 , ∂x0 ∂x1 ∂x2

(6.2.4)

which is a generalization of the operator ∂z . Definition 6.1 (Monogenicity) A continuously real-differentiable A-valued function f is called monogenic in  if Df = 0 in . Remark 6.1 It is not necessary to distinguish between left and right monogenic in the case of A-valued functions because Df = 0 implies fD = 0 and vice versa. The generalized Cauchy-Riemann operator (6.2.3) and its conjugate (6.2.4) factorize the 3-dimensional Laplace operator, in the sense that 3 f = DDf = DDf, whenever f ∈ C 2 , which implies that any monogenic function is also a harmonic function. This means that quaternionic analysis can be seen as a refinement of harmonic analysis. According to [24, 37, 48], we use ( 12 D)f as the hypercomplex derivative of a monogenic function f. An A-valued monogenic function with an identically vanishing hypercomplex derivative is called hyperholomorphic constant (see again [24]). It is immediately clear that such a function depends only on the variables x1 and x2 . Reconsider now a function f :  ⊂ R3 −→ A and write the Riesz system (6.2.1) explicitly as ⎧ ∂[f] ∂[f]1 ∂[f]2 0 ⎪ ⎪ ∂x0 − ∂x1 − ∂x2 = 0, ⎪ ⎪ ⎪ ⎪ ⎨ ∂[f]0 + ∂[f]1 = 0, ∂x1 ∂x0 (R) ∂[f]0 ∂[f]2 ⎪ ⎪ ⎪ ∂x2 + ∂x0 = 0, ⎪ ⎪ ⎪ ⎩ ∂[f]1 − ∂[f]2 = 0. ∂x2

∂x1

Using the generalized Cauchy-Riemann operator this reads as Df = fD = 0. Following [35], the solutions of the system (R) are called (R)-solutions. The subspace of polynomial (R)- solutions of degree n is denoted by M+ (; A; n). In [35], it is shown that the space M+ (; A; n) has dimension 2n + 3. Later, this result was generalized to arbitrary higher dimensions in the framework of a Clifford algebra by Delanghe in [17]. We denote further by M+ (; A) := L2 (; A; R) ∩ ker D the

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

149

space of square integrable A-valued monogenic functions defined in . A basis of M+ (R3 ; A; n), based on the well-known Fueter polynomials, was constructed by Leutwiler in [35]. His results were recently generalized to arbitrary dimensions by Delanghe in [16]. Both papers rely on the notion of harmonic conjugates. The main difficulty in carrying out such constructions is that only a certain number of these polynomials and their associated scalar parts are orthogonal to each other with respect to the real-inner product (6.2.2). Also, their norms are not easy to handle and the calculations are numerically highly unstable. For an account of such arguments, see [39], Chap. 2.

6.3 A Basis of M+ (B; A; n) In ([9], Chap. 3) and [11], special R-linear and H-linear complete orthonormal systems of H-valued homogeneous monogenic polynomials defined in the unit ball B of R3 are explicitly constructed (R3 → R4 case). The main idea of such constructions is the already referred factorization of the Laplace operator. Partially motivated by these results (see [39], Chap. 1) we deal from now on with constructive aspects of approximation theory by presenting a convenient system of polynomial solutions of the Riesz system (R3 → R3 case). The restriction to the sphere of this system can be viewed as analogue to the complex case of the Fourier exponential functions {einθ }n≥0 on the unit circle and constitutes a refinement of the well-known spherical harmonics. The strategy adopted (the idea is taken from [9]) is the following: we start by considering the set of homogeneous harmonic polynomials  n+1 0 m m Un+1 , r n+1 Un+1 , r n+1 Vn+1 , m = 1, . . . , n + 1 n∈N , (6.3.1) r 0

formed by the extensions in the ball of an orthogonal basis of spherical harmonics in R3 considered e.g. in [42]. The application of the operator 12 D to the homogeneous harmonic polynomials in (6.3.1) leads to the following set of homogeneous monogenic polynomials:  0,† m,† m,† (6.3.2) Xn , Xn , Yn : m = 1, . . . , n + 1 , with the notation Xn0,† := r n X0n ,

n m Xm,† n := r Xn ,

n m Ym,† n := r Yn .

In [9] it is proved the following result: Lemma 6.1 For each n, the set  0,† m,† m,† Xn , Xn , Yn : m = 1, . . . , n + 1 is orthogonal with respect to the real-inner product (6.2.2).

150

K. Gürlebeck, J. Morais

The consideration of the orthogonal system of harmonic spherical functions {Un0 , Unm , Vnm : m = 1, . . . , n} for the construction of our system allows to use properly some well-known results like its orthogonality on the unit sphere. Of course, as a consequence of the interrelation between spherical harmonics and Legendre polynomials (resp. Legendre functions), the constructed homogeneous monogenic polynomials are related to the Legendre polynomials (resp. Legendre functions) as well. A detailed study of Legendre polynomials and associated Legendre functions together with their relations with the polynomials (6.3.2) shows more interesting properties of the basis polynomials but they are not needed in the present article and so will not be discussed. We now come to the aim of this section. In order to establish our main results, we first state the following properties of the basis polynomials. Theorem 6.1 For a fixed n, the homogeneous monogenic polynomials 

m,† n+1,† Xl,† , Ynn+1,† : l = 0, . . . , n, m = 1, . . . , n , n , Yn , Xn

satisfy the properties: 1. The following relations hold: (n + l + 1) l,† Un , 2 (n + m + 1) m,† Vn ; Sc(Ym,† n ) := 2 Sc(Xl,† n ) :=

2. The polynomials Xnn+1,† and Ynn+1,† are hyperholomorphic constants. Proof The proof of the Affirmation 1 may be found in [25] and [39] by direct inspection of the scalar parts of the basis polynomials and their relations with the Legendre polynomials and associated functions. Affirmation 2 is already known from [9] because our constructed system is a subsystem of the polynomials which were constructed in [9].  The surprising observation in Affirmation 1 is that the scalar parts of the A-valued homogeneous monogenic polynomials, which were obtained by applying the D operator to scalar-valued harmonic polynomials, are strongly related to the original polynomials. This means nothing else than the scalar parts being orthogonal to each other. In this sense, it reflects one of the most noteworthy properties of the system and it shows an immediate relationship with the classical complex function theory in the plane, where the real parts of the complex variables zn are also orthogonal to each other. We remark that such a behavior is very different in the case of the Fueter polynomials (see [39], Chap. 2 for a special approach).

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

151

Now we want to orthonormalize the basis polynomials (6.3.2), and for that we proceed by presenting explicit formulae of their norms (see [9]) and of their corresponding scalar parts (see [25] and [26]). Proposition 6.1 For n ∈ N0 , the norms of the homogeneous monogenic polynomials m,† Xn0,† , Xm,† n and Yn (m = 1, . . . , n) and their associated scalar parts are given by

π(n + 1) 0,† , Xn L2 (B;A;R) = 2n + 3  π (n + 1) (n + 1 + m)! m,† m,† Xn L2 (B;A;R) = Yn L2 (B;A;R) = , 2 2n + 3 (n + 1 − m)!

π (n + 1)(2n + 2)! n+1,† n+1,† L2 (B;A;R) = Yn L2 (B;A;R) = , Xn 2 2n + 3

π (n + 1) Sc(Xn0,† )L2 (B) = √ , 2n + 3 2n + 1  (n + 1 + m) π 1 (n + m)! m,† m,† . Sc(Xn )L2 (B) = Sc(Yn )L2 (B) = √ 2 (2n + 1) (n − m)! 2n + 3 From now on we shall denote by Xn0,†,∗ , Xnm,†,∗ , Ynm,†,∗ (m = 1, . . . , n + 1) the m,† new normalized basis functions Xn0,† , Xm,† n , Yn in L2 (B; A; R). We begin by formulating a theorem from [39], p. 96: Theorem 6.2 For each n, the normalized set of 2n + 3 homogeneous monogenic polynomials  0,†,∗ m,†,∗ m,†,∗ Xn , Xn , Yn : m = 1, . . . , n + 1 (6.3.3) forms an orthonormal basis in the subspace M+ (B; A; n) with respect to the realinner product (6.2.2). Consequently,  0,†,∗ m,†,∗ m,†,∗ Xn , Xn , Yn , m = 1, . . . , n + 1; n = 0, 1, . . . is an orthonormal basis in M+ (B; A). The above consideration makes it possible to define the Fourier expansion of a square integrable A-valued monogenic function. Next we formulate the result: Theorem 6.3 Let f be a square integrable A-valued monogenic function. The function f can then be represented with the orthonormal system (6.3.3): f=

∞  n=0

 Xn0,†,∗ an0

+



n+1  

Xnm,†,∗ anm

m=1

+ Ym,†,∗ bnm n



,

(6.3.4)

152

K. Gürlebeck, J. Morais

where for each n ∈ N0 , an0 , anm , bnm ∈ R (m = 1, . . . , n + 1) are the associated Fourier coefficients. Remark 6.2 Since by construction the system (6.3.3) forms an orthonormal basis with respect to the real-inner product (6.2.2), we stress that the coefficients an0 , anm and bnm (m = 1, . . . , n + 1) are real constants. Based on representation (6.3.4), in [26] we have proved that each A-valued monogenic function can be decomposed in an orthogonal sum of a monogenic “main part” of the function (g) and a hyperholomorphic constant (h). Lemma 6.2 (Decomposition theorem) A function f ∈ M+ (B; A) can be decomposed into f := f(0) + g + h,

(6.3.5)

where the functions g and h have the Fourier series   ∞ n     Xnm,†,∗ (x)anm + Ynm,†,∗ (x)bnm , g(x) = Xn0,†,∗ (x)an0 + n=1

h(x) =

m=1

∞  

 Xnn+1,†,∗ (x)ann+1 + Ynn+1,†,∗ (x)bnn+1 .

n=1

Obviously, using Parseval’s identity, f may be characterized by its coefficients in the following way: Theorem 6.4 The function f is a square integrable A-valued monogenic function iff   ∞ n+1    m 2  0 2 m 2 (an ) + (bn ) < ∞. (6.3.6) (an ) + n=0

m=1

6.4 Generation of A-Valued Monogenic Functions Despite the fact that Quaternionic analysis offers a possibility to generalize some of the most important features of classical complex analysis, the monogenic functions do not enjoy all the properties of the holomorphic functions in one complex variable. In fact, because of the non-commutativity of the quaternionic multiplication the product of two monogenic functions is seldom a monogenic function. Therefore, the construction of elementary monogenic functions is a challenging problem. It is already known that there are a lot of techniques, which generate monogenic functions (a list of those techniques can be found e.g. in [15]). In this section we consider the problem of deriving A-valued monogenic functions with prescribed asymptotic

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

153

properties, in particular, asymptotic properties related to the Fourier coefficients of a scalar-valued harmonic function. This approach is related to the generation of monogenic functions by means of conjugate harmonic functions. Given a harmonic function U in a domain  of R4 with a specific geometric property, the problem of finding a harmonic conjugate V , generalizing the wellknown case of the complex plane to the case of quaternion-valued monogenic functions, was introduced by Sudbery in [48]. The author proposed an algorithm for the calculation of quaternion-valued monogenic functions. However, since our particular interest in this contribution is the study of monogenic functions with values in the reduced quaternions, the described procedure is not well-adapted to our case. Later and independently, Xu in [50] considered the problem of conjugate harmonics in the framework of Clifford analysis. In [52] and [51] the construction of conjugate harmonics to the Poisson kernel in the open unit ball and the upper half space respectively are obtained in this setting. The extension and completeness of these results were obtained in [7] and [5] by Brackx, Delanghe and Sommen. Therein an algorithm is constructed for the calculation of a harmonic conjugate V to a given harmonic function U in . By the nature of the given construction, the authors observed that such function V is not unique. While these results establish a general form of V , its precise description is given only up to a solution of a Poisson equation. Also, it is not discussed if the functions U and V belong to certain function spaces. For this reason, our research has been focused on the construction of an explicit algorithm respecting the underlying function spaces. Next, we briefly introduce the notion of harmonic conjugates: Definition 6.2 (Conjugate harmonic functions, see [7]) Let U be a harmonic function defined in an open subset  of R3 . A vector-valued harmonic function V in  is called conjugate harmonic to U if f := U + V is monogenic in . The pair (U ; V ) is called a pair of conjugate harmonic functions in . In the sequel, assume U be a square integrable harmonic function defined in an open subset  of R3 . In contrast to the approach presented in [5, 7], the general theory developed in the previous section gives the possibility to express explicitly the general form of a square integrable vector-valued harmonic function V conjugate to the scalar-valued U . This special case of conjugate harmonicity was first introduced by Moisil in [38] and taken up again by Stein and Weiß in [47]. First result in our line of consideration was already proved in [39]. Theorem 6.5 Let U be harmonic and square integrable in B ⊂ R3 with respect to the orthonormal system 

Sc(Xn0,† )

Sc(Xm,† n )

Sc(Ym,† n )

, , : m = 1, . . . , n m,† Sc(Xn0,† )L2 (B) Sc(Xm,† n )L2 (B) Sc(Yn )L2 (B)

 (6.4.1)

154

K. Gürlebeck, J. Morais

given by U=

∞  

Sc(Xn0,† )

an0 0,† n=0 Sc(Xn )L2 (B) +

n   m=1

Sc(Xm,† n ) Sc(Xm,† n )L2 (B)

anm

+

Sc(Ym,† n ) Sc(Ym,† n )L2 (B)

 bnm

,

(6.4.2)

where for each n ∈ N0 , an0 , anm , bnm ∈ R (m = 1, . . . , n) are the associated Fourier coefficients. If the series ∞   ([Xn0,† ]1 e1 + [Xn0,† ]2 e2 ) n=0

+

Sc(Xn0,† )L2 (B)

an0

n  m,†  ([Xm,† n ]1 e1 + [Xn ]2 e2 )

Sc(Xm,† n )L2 (B)

m=1

anm

+

m,† ([Ym,† n ]1 e1 + [Yn ]2 e2 )

Sc(Ym,† n )L2 (B)

 bnm

(6.4.3)

is convergent, then it defines a square integrable vector-valued harmonic function V conjugate to U . Proof Let U ∈ L2 (B) be a harmonic function. Consider the Fourier series of U with respect to the orthonormal system (6.4.1). Since the series (6.4.2) is convergent in L2 , it converges uniformly to U in each compact subset of B. Now, replacing the scalar part of each polynomial by the full polynomial we get formally f=

∞  

Xn0,†

an0 0,† Sc(X ) L2 (B) n n=0   n  Ym,† Xm,† n n m m + a + b n n m,† Sc(Ym,† n )L2 (B) m=1 Sc(Xn )L2 (B)

:= U + V .

(6.4.4) 

Remark 6.3 If the previous series are finite sums then the functions U and V are polynomials. Then, it is clear that the partial expansion (6.4.3) makes always sense. In this special case our approach covers the results obtained in [7] and [5]. Remark 6.4 The vector-valued function V conjugate to the scalar-valued U is not unique. This is due to the fact that by adding any hyperholomorphic constant ϕ to V the resulting function V˜ := V + ϕ is also harmonic conjugate to U . Remark 6.5 By the direct construction of formula (6.4.4), we get only 2n + 1 homogeneous monogenic polynomials (i.e., the monogenic “main part” of f). How-

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

155

ever, since dim M+ (R3 ; A; n) = 2n + 3, adding two hyperholomorphic constants the necessary number of independent polynomials is achieved. Remark 6.6 Originally the Fourier coefficients an0 , anm , bnm (m = 1, . . . , n) are defined as inner products of U and elements of the space Hn (R3 ). The new result is that one obtains, up to a factor, the same coefficients but now as inner products between f and elements of the space M+ (R3 ; A; n). The main point in the approach presented in [5, 7] as well as Sudbery’s formula [48] is the construction of harmonic conjugates “function by function”. No attention was paid to the question to which function spaces these conjugate harmonics and the whole monogenic function belong. In [4] this question was studied for conjugate harmonics via Sudbery’s formula in the scale of Bergman spaces. As already mentioned this result is not applicable for A-valued functions. If we want to make the previous theorem more precise then we need an a-priori criterion that ensures the convergence of the constructed series for mononogenic functions in L2 (B; A; R). Theorem 6.6 Let U be harmonic and square integrable in B ⊂ R3 with respect to the orthonormal system (6.4.1). Additionally, we assume that the Fourier coefficients satisfy the condition   ∞ n  2n + 1 0 2  (n + 1)(2n + 1) m 2 m 2 [(an ) + (bn ) ] < ∞. (a ) + n+1 n (n + 1)2 − m2 n=0

m=1

Then, the series in Theorem 6.5 converges in L2 (B). The proof runs analogously to the previous theorem. Of course, this criterion is not well-applicable in practice and it is still open to characterize which function space (for the functions U ) is described by the condition of the theorem. If we suppose for the moment more smoothness of the given function U , then we can count with an exponential decay of the Fourier coefficients and we can formulate a simple sufficient condition to guarantee the convergence of the series expansion for V . Theorem 6.7 (See [39]) Let U be harmonic and square integrable in B ⊂ R3 . If the absolute values of its Fourier coefficients an0 , anm and bnm (m = 1, . . . , n), with c respect to the orthonormal system (6.4.1) in L2 (B), are less than (n+1) 1+α (α > 1/2) + with a positive constant c, then f := U + V ∈ M (B; A). Proof Let U ∈ L2 (B) be a harmonic function. Consider the Fourier series of U with respect to the orthonormal system (6.4.1). Now, as before, we replace the scalar part of each polynomial by the full polynomial and by introducing suitable correction factors we can rewrite the obtained series as a series expansion with respect to the

156

K. Gürlebeck, J. Morais

normalized full polynomials. We get formally   

∞ n    2n + 1 (n + 1)(2n + 1)  m,†,∗ m 0,†,∗ 0 m,†,∗ m f= Xn an + Yn bn . a + Xn n+1 n (n + 1)2 − m2 n=0

m=1

On the right-hand side of the previous equality we recognize the Fourier expansion of the function f with respect to the orthonormal system (6.3.3) in M+ (B; A). Having in mind the conditions of the L2 -convergence of (6.4.2), our task now is to find out if the series   ∞ n   (2n + 1) 0 2  (n + 1)(2n + 1)  m 2 (an ) + (bnm )2 (6.4.5) (an ) + n+1 (n + 1)2 − m2 n=0

m=1

is convergent. By assumption, there exists a constant c such that the Fourier coefficients an0 , anm , bnm (n ∈ N0 , m = 1, . . . , n) satisfy |an0 |, |anm |, |bnm | <

c , (n + 1)1+α

α > 1/2,

m = 1, . . . , n.

Substituting in the expression (6.4.5) we get f2L2 (B;A;R)

<

∞  n=0

c2 (n + 1)2(1+α)



  ∞ 2c2 2n + 1 . + 2(n + 1)n ≤ n+1 (n + 1)2α n=0

The series on the right-hand side is convergent, because by assumption α > 12 . Consequently, the series (6.4.5) is convergent. 

References 1. Abul-Ez, M.A., Constales, D.: Basic sets of polynomials in Clifford analysis. Complex Var. 14(1–4), 177–185 (1990) 2. Abul-Ez, M.A., Constales, D.: Linear substitution for basic sets of polynomials in Clifford analysis. Portugaliae Math. 48(2), 143–154 (1991) 3. Abul-Ez, M.A., Constales, D.: The square root base of polynomials in Clifford analysis. Arch. Math. 80(5), 486–495 (2003) 4. Avetisyan, K., Gürlebeck, K., Sprößig, W.: Harmonic conjugates in weighted Bergman spaces of quaternion-valued functions. Comput. Methods Funct. Theory 9(2), 593–608 (2009) 5. Brackx, F., Delanghe, R.: On harmonic potential fields and the structure of monogenic functions. Z. Anal. Anwend. 22(2), 261–273 (2003) 6. Brackx, F., Delanghe, R., Sommen, F.: Clifford Analysis. Pitman, Boston/London/Melbourne (1982) 7. Brackx, F., Delanghe, R., Sommen, F.: On conjugate harmonic functions in Euclidean space. Math. Methods Appl. Sci. 25(16–18), 1553–1562 (2002) 8. Brackx, F., De Schepper, N., Sommen, F.: Clifford algebra-valued orthogonal polynomials in Euclidean space. J. Approx. 137(1), 108–122 (2005) 9. Cação, I.: Constructive Approximation by Monogenic Polynomials. Ph.D. thesis, Universidade de Aveiro, Departamento de Matemática, Dissertation (2004)

6 On Orthonormal Polynomial Solutions of the Riesz System in R3

157

10. Cação, I., Gürlebeck, K., Malonek, H.: Special monogenic polynomials and L2 approximation. Adv. Appl. Clifford Algebras 11(S2), 47–60 (2001) 11. Cação, I., Gürlebeck, K., Bock, S.: On derivatives of spherical monogenics. Complex Var. Elliptic Equ. 51(8–11), 847–869 (2006) 12. Cação, I., Gürlebeck, K., Bock, S.: Complete orthonormal systems of spherical monogenics— a constructive approach. In: Son, L.H., et al. (eds.) Methods of Complex and Clifford Analysis, pp. 241–260. SAS International, Delhi (2005) 13. Cnops, J.: Orthogonal functions associated with the Dirac operator. Ph.D. thesis, Ghent university (1989) 14. De Bie, H., Sommen, F.: Hermite and Gegenbauer polynomials in superspace using Clifford analysis. J. Phys. A, Math. Theor. 40(34), 10441–10456 (2007) 15. Delanghe, R.: Clifford analysis: history and perspective. Comput. Methods Funct. Theory 1(1), 107–153 (2001) 16. Delanghe, R.: On homogeneous polynomial solutions of the Riesz system and their harmonic potentials. Complex Var. Elliptic Equ. 52(10–11), 1047–1062 (2007) 17. Delanghe, R.: On a class of inner spherical monogenics and their primitives. Adv. Appl. Clifford Algebras 18(3–4), 557–566 (2008) 18. Falcão, M.I., Malonek, H.: Generalized exponentials through Appell sets in Rn+1 and Bessel functions. In: AIP-Proceedings, pp. 738–741 (2007) 19. Falcão, M.I., Malonek, H.: Special monogenic polynomials—properties and applications. In: AlP-Proceedings, pp. 764–767 (2007) 20. Falcão, M., Cruz, J., Malonek, H.: Remarks on the generation of monogenic functions. In: Gürlebeck, K., Könke, C. (eds.) Proceedings 17th International Conference on the Applications of Computer Science and Mathematics in Architecture and Civil Engineering. Weimar (2006) 21. Fueter, R.: Analytische Funktionen einer Quaternionenvariablen. Comment. Math. Helv. 4, 9–20 (1932) 22. Fueter, R.: Functions of a Hyper Complex Variable. Lecture notes written and supplemented by E. Bareiss, Math. Inst. Univ. Zürich, Fall Semester (1949) 23. Gürlebeck, K.: Interpolation and best approximation in spaces of monogenic functions. Wiss. Z. TU Karl-Marx-Stadt 30, 38–40 (1988) 24. Gürlebeck, K., Malonek, H.: A hypercomplex derivative of monogenic functions in Rn+1 and its applications. Complex Var. Elliptic Equ. 39(3), 199–228 (1999) 25. Gürlebeck, K., Morais, J.: On mapping properties of monogenic functions. CUBO A Math. J. 11(1), 73–100 (2009) 26. Gürlebeck, K., Morais, J.: Bohr type theorems for monogenic power series. Comput. Methods Funct. Theory 9(2), 633–651 (2009) 27. Gürlebeck, K., Morais, J.: On local mapping properties of monogenic functions. In: Gürlebeck, K., Könke, C. (eds.) Proceedings 18th International Conference on the Applications of Computer Science and Mathematics in Architecture and Civil Engineering. Weimar (2009) 28. Gürlebeck, K., Sprössig, W.: Quaternionic Analysis and Elliptic Boundary Value Problems. Akademie Verlag, Berlin (1989) 29. Gürlebeck, K., Sprössig, W.: On the treatment of fluid problems by methods of Clifford analysis. Math. Comput. Simul. 44(4), 401–413 (1997) 30. Gürlebeck, K., Sprössig, W.: Quaternionic Calculus for Engineers and Physicists. Wiley, Chichester (1997) 31. Gürlebeck, K., Sprössig, W.: On eigenvalue estimates of nonlinear Stokes eigenvalue problems. In: Micali, A., et al. (eds.) Clifford Algebras and Their Applications in Mathematical Physics, pp. 327–333. Kluwer Academic, Amsterdam (1992) 32. Kamzolov, A.: The best approximation of the classes of functions Wpα (S n ) by polynomials in spherical harmonics. Math. Not. 32(3), 622–626 (1982) 33. Kravchenko, V.: Applied Quaternionic Analysis. Research and Exposition in Mathematics, vol. 28. Heldermann, Lemgo (2003)

158

K. Gürlebeck, J. Morais

34. Kravchenko, V., Shapiro, M.: Integral Representations for Spatial Models of Mathematical Physics. Research Notes in Mathematics. Pitman Advanced Publishing Program, London (1996) 35. Leutwiler, H.: Quaternionic analysis in R3 versus its hyperbolic modification. In: Brackx, F., Chisholm, J.S.R., Soucek, V. (eds.) NATO Science Series II. Mathematics, Physics and Chemistry, vol. 25. Kluwer Academic, Dordrecht/Boston/London (2001) 36. Malonek, H.: Power series representation for monogenic functions in Rm+1 based on a permutational product. Complex Var. Elliptic Equ. 15(3), 181–191 (1990) 37. Mitelman, I., Shapiro, M.: Differentiation of the Martinelli-Bochner integrals and the notion of hyperderivability. Math. Nachr. 172(1), 211–238 (1995) 38. Moisil, G.: Sur la généralisation des fonctions conjuguées. Rend. Acad. Naz. Lincei 14, 401– 408 (1931) 39. Morais, J.: Approximation by homogeneous polynomial solutions of the Riesz system in R3 . Ph.D. thesis, Bauhaus-Universität Weimar (2009) 40. Riesz, M.: Clifford numbers and spinors. Inst. Phys. Sci. Techn. Lect. Ser., vol. 38. Maryland (1958) 41. Ryan, J.: Left regular polynomials in even dimensions, and tensor products of Clifford algebras. In: Chisholm, J.S.R., Common, A.K. (eds.) Clifford Algebras and Their Applications in Mathematical Physics, pp. 133–147. Reidel, Dordrecht (1986) 42. Sansone, G.: Orthogonal Functions. Pure and Applied Mathematics, vol. IX. Interscience Publishers, New York (1959) 43. Shapiro, M., Vasilevski, N.L.: Quaternionic ψ -hyperholomorphic functions, singular operators and boundary value problems I. Complex Var. Theory Appl. (1995) 44. Shapiro, M., Vasilevski, N.L.: Quaternionic ψ -hyperholomorphic functions, singular operators and boundary value problems II. Complex Var. Theory Appl. (1995) 45. Sprössig, W.: Boundary value problems treated with methods of Clifford analysis. Contemp. Math. 212, 255–268 (1998) 46. Stein, E.M.: Singular Integrals and Differentiability Properties of Functions. Princeton University Press, Princeton (1970) 47. Stein, E.M., Weiß, G.: On the theory of harmonic functions of several variables. Part I: the theory of H p spaces. Acta Math. 103, 25–62 (1960) 48. Sudbery, A.: Quaternionic analysis. Math. Proc. Camb. Philos. Soc. 85, 199–225 (1979) 49. Whittaker, J.M.: Sur les séries de base de polynômes quelconques. Avec la col laboration de C. Gattegno (Collection de monographies sur la theorie des fonctions). Paris: Gauthier-Villars (1949) 50. Xu, Z.: Boundary value problems and function theory for spin-invariant differential operators. Ph.D thesis. Gent (1989) 51. Xu, Z., Zhou, C.: On boundary value problems of Riemann-Hilbert type for monogenic functions in a half space of Rm (m ≥ 2). Complex Var. Theory Appl. 22, 181–194 (1993) 52. Xu, Z., Chen, J., Zhang, W.: A harmonic conjugate of the Poisson kernel and a boundary value problem for monogenic functions in the unit ball of Rn (n ≥ 2). Simon Stevin 64, 187–201 (1990)

Chapter 7

Brief Survey on the CP Methods for the Schrödinger Equation L.Gr. Ixaru

Abstract The CP methods have some salient advantages over other methods, viz.: (i) the accuracy is uniform with respect to the energy E; (ii) there is an easy control of the error; (iii) the step widths are unusually big and the computation is fast; (iv) the form of the algorithm allows a direct evaluation of collateral quantities such as normalisation constant, Prüfer phase, or the derivative of the solution with respect to E; (v) the algorithm is of a form which allows using parallel computation.

Keywords Schrödinger equation · CP methods · ηm set of functions Mathematics Subject Classification (2000) 81Q05

7.1 Introduction The piecewise perturbation methods (PPM) are a class of numerical methods specially devised for the solution of the Schrödinger equation, and the CP methods form a subclass of these. The main element of attractivity for the methods in this subclass is that their algorithms are the easiest to construct and the fastest in runs. Given the one-dimensional Schrödinger equation d 2y + [E − V (x)]y = 0, dx 2

x ∈ [a, b],

(7.1.1)

L.Gr. Ixaru () Department of Theoretical Physics, “Horia Hulubei” National Institute of Physics and Nuclear Engineering, P.O. Box MG-6, Bucharest, Romania e-mail: [email protected] L.Gr. Ixaru Academy of Romanian Scientists, 54 Splaiul Independentei, 050094, Bucharest, Romania T.E. Simos (ed.), Recent Advances in Computational and Applied Mathematics, DOI 10.1007/978-90-481-9981-5_7, © Springer Science+Business Media B.V. 2011

159

160

L.Gr. Ixaru

where the potential V (x) is a well behaved function and the energy E is a free parameter, one can formulate either an initial value (IV) or a boundary value (BV) problem. The form of V (x) is determined by the physical phenomenon under investigation, and a large variety of shapes are encountered. However, analytic solutions of (7.1.1), either for the IV or for the BV problem, are known only for a small number of expressions for the function V (x), let such functions be denoted by V¯ (x), such that any attempt to obtain an approximate solution in analytic form for a realistic problem must have the selection of V¯ (x) which best fits the shape of the original V (x) as its first step. In fact, this was the standard way of doing the things decades ago, when computers were at their infancy. In the early stages the fit was global (that is a single fitted V¯ (x) was used in place of V (x) over the whole equation interval) but quite soon it became clear that the quality of the approximation increases if this fit is made piecewise, i.e., when the interval Ia,b = [a, b] is first partitioned, x0 = a, x1 , x2 , . . . , xkmax = b, and a suitable V¯ (x) is introduced on each step Ik = [xk−1 , xk ]. If so is done, the solution on the whole Ia,b requires an extra algebraic manipulation of the piecewise analytic solutions, a task which can be accomplished efficiently only on a computer, and this explains why the first systematic investigations along these lines are in the 60’s. In [4, 6, 7] the piecewise V¯ (x) was a constant, that is the average value of V (x) over Ik , while in [5] it was a straightline segment which is tangent to V (x). The versions corresponding to the two options are called CP (short for constant-based potential) and LP (line-based potential) methods, respectively. The two linear independent solutions are expressed by trigonometric or hyperbolic functions for the CP methods, and by the Airy functions for the LP methods.

7.2 The Algorithm of a CP Method To fix the ideas we concentrate on a single step Ik denoted generically as [X, X + h], and show how the solution can be propagated from one end to the other end. As said before, the first versions [4–7] contained only the solution from the reference potential V¯ (X + δ), δ ∈ [0, h] and therefore their order was low, namely 2, see [8]. It has also became clear that the order can be increased only if corrections from the perturbation V (δ) = V (X + δ) − V¯ (X + δ) can be added into the algorithm but this raised another problem: are the formulae of these corrections simple enough for not charging the run time too much? The answer was given by Ixaru in [9]: if V¯ (X + δ) is a constant and V (δ) is a polynomial then a set of special functions can be introduced such that the perturbation corrections have simple analytic forms. In fact, all CP versions built up later on were based on this. The starting point then technically consists in approximating the potential function by a polynomial, and a first problem is how the polynomial coefficients have to be chosen in order to ensure the best fit. The most important result is due to Pruess [24] and it says in essence that the best fit consists in taking a finite number of

7 Brief Survey on the CP Methods for the Schrödinger Equation

161

terms in the expansion of V (x) over shifted Legendre polynomials of argument δ/ h, V (X + δ) ≈ V (X + δ) = p

N 

n

Vn h

n=0

Pn∗

  δ . h

(7.2.1)

In [9] V p is called a pilot potential and all subsequent operations use only this. As a matter of fact, taking V p for V induces a residual error of O(h2N+2 ). The expressions of several Pn∗ (γ ) polynomials,γ ∈ [0, 1], are as follows (see [1]): P0∗ (γ ) = 1,

P1∗ (γ ) = −1 + 2γ ,

P2∗ (γ ) = 1 − 6γ + 6γ 2 ,

P3∗ (γ ) = −1 + 12γ − 30γ 2 + 20γ 3 .

The form (7.2.1) is separated into the constant reference potential V¯ (X + δ) = V0 and the perturbation V (δ) = V p (X +δ)−V0 such that the one step solution results by taking the exact solution from the reference potential and by adding to this as many corrections as possible from the perturbation V . Each member of the CPM family is identified by the degree N of the polynomial V (δ) and by the number of perturbation corrections Q retained in the algorithm, CPM[N, Q] for short. The simplest version, that is when no correction is introduced, is identified either as CPM[0, 0] or directly as CPM(0). How do N and Q influence the quality of the method? The main parameter for the quality of a numerical method is its order. Roughly speaking, if (h) and (h/2) are the errors produced by using the sufficiently small steps h and h/2 then the order p is that number such that (h/2) = (h)/2p . Example For p = 4 the error at h/2 is by a factor 24 = 16 smaller than that at h, but for p = 6 it is smaller by a factor 26 = 64. The Numerov and standard Runge-Kutta methods are of order p = 4. Intuition says that the order p must increase with N and Q and indeed this is confirmed by a theorem proved in [16]. In particular, for fixed N the maximal p is 2N + 2 because the residual error acts as a barrier. The dependence of p with respect to N and Q is listed in Table 7.1 for a number of cases. In that table the value of the minimal Q is given, with the meaning that any greater number of corrections does not influence anymore the value of the order. For example, versions CPM[4, 3] and CPM[4, 4] have the same order p = 10. As for the versions for solving the one-channel equation (7.1.1) published in the literature, we find versions of orders between 2 (when no correction is added) and 6 in [9], up to 12 in [16], and 18 in [19]. A Fortran code for the CP version of [16] is in [17]. In all these the independent variable x, potential function V (x) and energy E are assumed real. CPM versions when these are complex are also available, see [15].

162

L.Gr. Ixaru

Table 7.1 Link between order p and parameters N and Q of the version CPM[N, Q]

N

minimal Q

2

0

0

4

1

1

6

2

2

8

3

3

10

4

3

12

5

4

p

The case of systems of coupled-channel equations also enjoyed much attention. Again, the need of first having each element of the potential matrix approximated by a polynomial plays a central role in making the expressions of the corrections attractive from computational point of view. However the orders of the existing versions is comparatively lower than before. In fact, the orders are between 2 and 6 in [9, 11], and up to 10 in [21]. Other numerical approaches pertinent to the Schrödinger equation can be found in [25]. What about the computational effort? For the one-channel case the CPU time/step increases with p but moderately: for p = 12 it is only 2–3 times bigger than for p = 2. For the coupled-channel case the rate of increase depends on the number of channels. As said, we look for a procedure which gives the values of y and y  at X + h when the values at X are given (forward propagation) or, conversely, yields the values at X when the values at X + h are given (backward propagation). The values of y and y  at X and X +h are connected via the so-called propagation (or transfer) matrix   u(h) v(h) , (7.2.2) P=  u (h) v  (h) where u(δ) and v(δ), δ = x − X ∈ [0, h] are two linear independent solutions of (7.1.1) on [X, X + h] which satisfy the following initial conditions u(0) = 1,

u (0) = 0,

v(0) = 0,

v  (0) = 1.

(7.2.3)

Indeed, with the column vector y(x) = [y(x), y  (x)]T

(7.2.4)

we have y(X + h) = Py(X),

y(X) = P−1 y(X + h)

(7.2.5)

and then the propagation of the solution in either of the two directions requires the generation of the elements of matrix P, i.e. the values at δ = h of the two independent solutions u(δ) and v(δ) and of their first derivatives.

7 Brief Survey on the CP Methods for the Schrödinger Equation

163

Functions u(δ) and v(δ) are constructed by perturbation. The procedure consists in taking the constant V¯ = V0 as the reference potential and the polynomial V (δ) as a perturbation. Each of the two propagators, denoted generically as p(δ), is written as a perturbation series, p(δ) = p0 (δ) + p1 (δ) + p2 (δ) + p3 (δ) + · · · ,

(7.2.6)

where the zeroth-order term p0 (δ) is the solution of p0 = [V0 − E]p0

(7.2.7)

with p0 (0) = 1, p0 (0) = 0 for u0 and p0 (0) = 0, p0 (0) = 1 for v0 . The correction pq , q = 1, 2, . . . obeys the equation pq = [V0 − E]pq + V (δ)pq−1 ,

pq (0) = pq (0) = 0.

(7.2.8)

With Z(δ) = (V0 − E)δ 2 and functions ξ(Z), η0 (Z), η1 (Z), . . . , defined in the Appendix, the zeroth order propagators are u0 (δ) = ξ(Z(δ)),

v0 (δ) = δη0 (Z(δ))

(7.2.9)

and the following iteration procedure exists to construct the corrections. Correction pq−1 is assumed as known and of such a form that the product V (δ)pq−1 reads V (δ)pq−1 (δ) = Q(δ)ξ(Z(δ)) +

∞ 

Rm (δ)δ 2m+1 ηm (Z(δ)).

(7.2.10)

m=0

Then pq (δ) and pq (δ) are of the form pq (δ) =

∞ 

Cm (δ)δ 2m+1 ηm (Z(δ)),

(7.2.11)

m=0

pq (δ) = C0 (δ)ξ(Z(δ)) +

∞ 

(Cm (δ) + δCm+1 (δ))δ 2m+1 ηm (Z(δ)),

(7.2.12)

m=0

where C0 (δ), C1 (δ), . . . are given by quadrature (see again [9]): C0 (δ) =

1 2



δ

(7.2.13)

Q(δ1 )dδ1 , 0

1 Cm (δ) = δ −m 2



δ 0

δ1m−1 [Rm−1 (δ1 )

 − Cm−1 (δ1 )]dδ1 ,

m = 1, 2, . . . .

(7.2.14)

164

L.Gr. Ixaru

To calculate successive corrections for u, the starting functions in V (δ)p0 (δ) are Q(δ) = V (δ), R0 (δ) = R1 (δ) = · · · = 0, while for v they are Q(δ) = 0, R0 (δ) = V (δ), R1 (δ) = R2 (δ) = · · · = 0. Integrals (7.2.13) and (7.2.14) have analytic forms. Each Cm (δ) is a polynomial and the series (7.2.11) and (7.2.12) are finite. We give below the expression of u(h) for the CPM version of order 12, [16]; the expressions of the other three matrix elements of the propagation matrix can be found in the same paper. With V0 , V1 , V2 , . . . , V10 defined in (7.2.1) and V¯i = Vi hi+2 , i = 1, 2, . . . , 10, Z = (V0 − E)h2 and functions ξ(Z), ηm (Z), m = 0, 1, 2, . . . as in the Appendix, this is: u(h) = ξ + V¯1 [−576η1 − 48η2 V¯1 + 24η4 V¯12 + η5 V¯13 ]/1152 + V¯2 [280η3 V¯1 + 14η4 V¯12 − 14(η2 + 3η3 )V¯2 + 7(η4 − 31η5 )V¯1 V¯2 + 2(η4 + 10η5 )V¯22 ]/560 + V¯3 [−840η1 + 4200η2 + 420η3 V¯1 + 35(η4 − 23η5 )V¯12 + 420(2η3 − 15η4 )V¯2 + 6(9η4 − 50η5 )V¯1 V¯2 − 30(η2 − 4η3 + 15η4 )V¯3 ]/1680 + V¯4 [18(2η3 − 21η4 )V¯1 − 18η5 V¯12 + 54η4 V¯2 + 36(η3 − 18η4 + 105η5 )V¯3 − (η2 + 3η3 − 75η4 + 105η5 )V¯4 ]/72 + V¯5 [−2η1 + 28η2 − 126η3 + (η3 − 9η4 )V¯1 + 2(η3 − 21η4 + 126η5 )V¯2 + (η3 − 9η4 + 15η5 )V¯3 ]/4 + V¯6 [2(η3 − 27η4 + 198η5 )V¯1 + 3(η4 − 11η5 )V¯2 ]/4 + V¯7 [−2η1 + 54η2 − 594η3 + 2574η4 + (η3 − 22η4 + 143η5 )V¯1 ]/4 + V¯9 [−η1 + 44η2 − 858η3 + 8580η4 − 36465η5 ]/2.

(7.2.15)

7.3 Advantages of the CPM Versions A list of advantages is largely publicised in the literature (see, e.g., [9, 10, 22]), to mention only: (i) the accuracy is uniform with respect to E, a feature unparallelled by any other numerical method; (ii) there is an easy control of the error; (iii) the step widths are unusually big and the computation is fast; (iv) the form of the algorithm allows a direct evaluation of the normalisation constant, of the Prüfer phase and of the partial derivative with respect to E of the solution. Finally, (v), the algorithms are of a form which allows using parallel computation. Related to point (i) above, it is instructive mentioning that another class of numerical methods is also often presented as much suited for application at big values of the energy. This is the class of methods whose algorithms are built up by exponential fitting, e.g., [2, 3, 13, 14, 18, 26–28] and references therein. Exponential fitting (ef)

7 Brief Survey on the CP Methods for the Schrödinger Equation

165

is a procedure for building up approximation formulae which are appropriate for linear combinations of trigonometric or hyperbolic functions sin, cos, with coefficients that are smooth enough to be well approximated by low degree polynomials. The dependence of the error with respect to E has been examined in detail for various versions of the Numerov method, see [13, 14]. Four existing degrees of eftuning are available for this method, that is 0-classical version, 1, 2, and 3, and it has been proved that the error increases asymptotically as E s where s = 3, 2, 3/2 and 1, respectively. As a matter of fact, there are reasons to believe that the same behaviour holds true for all other ef-based methods: each new higher degree of eftuning results in a reduction of the