Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis, and J. van Leeuwen
2907
Springer Berlin Heidelberg New York Hong Kong London Milan Paris Tokyo
Ivan Lirkov Svetozar Margenov Plamen Yalamov
Large-Scale Scientific Computing 4th International Conference, LSSC 2003 Sozopol, Bulgaria, June 4-8, 2003 Revised Papers
Springer
eBook ISBN: Print ISBN:
3-540-24588-X 3-540-21090-3
©2005 Springer Science + Business Media, Inc. Print ©2004 Springer-Verlag Berlin Heidelberg Dordrecht All rights reserved No part of this eBook may be reproduced or transmitted in any form or by any means, electronic, mechanical, recording, or otherwise, without written consent from the Publisher Created in the United States of America
Visit Springer's eBookstore at: and the Springer Global Website Online at:
http://ebooks.kluweronline.com http://www.springeronline.com
Preface
The papers in this volume were presented at the 4th International Conference on Large-Scale Scientific Computations ICLSSC 2003. It was held in Sozopol, Bulgaria, June 4–8, 2003. The conference was organized and sponsored by the Central Laboratory for Parallel Processing at the Bulgarian Academy of Sciences. Support was also provided from the Center of Excellence “BIS 21” (funded by the European Commission), SI AM and GAMM. A co-organizer of this traditional scientific meeting was the Division of Numerical Analysis and Statistics of the University of Rousse. The success of the conference and the present volume in particular are the outcome of the joint efforts of many colleagues from various institutions and organizations. First thanks to all the members of the Scientific Committee for their valuable contribution to forming the scientific face of the conference, as well as for their help in reviewing contributed papers. We would like to specially thank the organizers of the special sessions: R. Blaheta, N. Dimitrova, A. Ebel, K. Georgiev, O. Iliev, A. Karaivanova, H. Kosina, M. Krastanov, U. Langer, P. Minev, M. Neytcheva, M. Schäfer, V. Veliov, and Z. Zlatev. We are also grateful to the staff involved in the local organization. Special Events: The conference was devoted to the 60th anniversary of Raytcho Lazarov. During the conference, the nomination for the World Level of the Hall of Fame for Engineering, Science and Technology, HOFEST, was officially awarded to Owe Axelsson. Traditionally, the purpose of the conference is to bring together scientists working with large-scale computational models of environmental and industrial problems, and specialists in the field of numerical methods and algorithms for modern high-speed computers. The key lectures reviewed some of the advanced achievements in the field of numerical methods and their efficient applications. The ICLSSC 2003 talks were presented by university researchers and practical industry engineers, including applied mathematicians, numerical analysts and computer experts. The general theme for ICLSSC 2003 was Large-Scale Scientific Computing, focusing on: Recent achievements in preconditioning techniques; Monte Carlo and quasi-Monte Carlo methods; set-valued numerics and reliable computing; environmental modelling; large-scale computations for engineering problems. More than 90 participants from all over the world attended the conference, representing some of the strongest research groups in the field of advanced large-
VI
Preface
scale scientific computing. This volume contains 55 papers submitted by authors from over 20 countries, out of which 5 were invited and 50 were contributed. The 5th International Conference LSSC 2005 is planned for June 2005.
November 2003
Ivan Lirkov Svetozar Margenov Plamen Yalamov
Table of Contents
I
Invited Papers
Eigenvalue Estimates for Preconditioned Saddle Point Matrices Owe Axelsson A 3D Projection Scheme for Incompressible Multiphase Flows Using Dynamic Front Refinement and Reconnection Tong Chen, Peter Dimitrov Minev, and Krishnaswamy Nandakumar Schwarz-Like Methods for Approximate Solving Cooperative Systems Ivo Marek
3
17 25
Computational Infrastructure for Parallel, Distributed, and Grid-Based Monte Carlo Computations Michael Mascagni and Yaohang Li
39
Parallel Solution of Very Large Sparse Systems of Linear Algebraic Equations Zahari Zlatev
53
II
Recent Achievements in Preconditioning
Solution of Robust Linear Regression Problems by Krylov Subspace Methods Venansius Baryamureeba
67
Two-Level Preconditioning of Crouzeix-Raviart Anisotropic FEM Systems Gergana Bencheva, Ivan Georgiev, and Svetozar Margenov
76
Parallel High Performance Computing on Composite Grids Roman Kohut
85
Parallel Preconditioning for Sedimentary Basin Simulations Roland Masson, Philippe Quandalle, Stéphane Requena, and Robert Scheichl
93
III
Monte Carlo and Quasi-Monte Carlo Methods
Modeling of Narrow-Width SOI Devices: The Role of Quantum Mechanical Narrow Channel Effects on Device Performance Shaikh S. Ahmed and Dragica Vasileska
105
VIII
Table of Contents
Monte Carlo Algorithm for Ballistic Object Tracking with Uncertain Drag Parameter Donka Angelova, Iliyana Simeonova, and Tzvetan Semerdjiev
112
Efficient CPU-Specific Algorithm for Generating the Generalized Faure Sequences Emanouil I. Atanassov
121
A New Quasi-Monte Carlo Algorithm for Numerical Integration of Smooth Functions Emanouil I. Atanassov, Ivan T. Dimov, and Mariya K. Durchova
128
Monte Carlo Method for Multiple Knapsack Problem Stefka Fidanova
136
Importance Separation for Solving Integral Equations Rayna Georgieva and Sofiya Ivanovska
144
A Parallel Monte Carlo Method for Electron Quantum Kinetic Equation Todor V. Gurov and Ivan T. Dimov
153
Solving BVPs Using Quasirandom Walks on the Boundary Aneta Karaivanova, Michael Mascagni, and Nikolai A. Simonov
162
A Stable Backward Monte Carlo Method for the Solution of the Boltzmann Equation Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr
170
A Weight Decomposition Approach to the Sign Problem in Wigner Transport Simulations Mihail Nedjalkov, Hans Kosina, and Siegfried Selberherr
178
A Zero Field Monte Carlo Algorithm Accounting for the Pauli Exclusion Principle Sergey Smirnov, Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr
IV
185
Set-Valued Numerics and Reliable Computing
Solution Methods for Age-Structured Optimal Control Models with Feedback Christian Almeder Multivariate Rational Interpolation of Scattered Data Stefan Becuwe, Annie Cuyt, and Brigitte Verdonk Approximation Methods for Nonconvex Parabolic Optimal Control Problems Using Relaxed Controls Ion Chryssoverghi
197 204
214
Table of Contents
Stabilizing Feedback of a Nonlinear Biological Wastewater Treatment Plants Model Neli Dimitrova and Mikhail Krastanov Higher Order Approximations of Affinely Controlled Nonlinear Systems Nikolay Kirov and Mikhail Krastanov Outlier Detection under Interval Uncertainty: Algorithmic Solvability and Computational Complexity Vladik Kreinovich, Luc Longpré, Praveen Patangay, Scott Ferson, and Lev Ginzburg
IX
222 230
238
On the Approximation of Centered Zonotopes in the Plane Svetoslav Markov and Dalcidio Claudio
246
On Range Evaluation of Polynomials by Applying Interval Arithmetic Shinya Miyajima and Masahide Kashiwagi
254
262 Sharp Bounds for Strains and Stresses in Uncertain Mechanical Models Evgenia D. Popova, Maria Datcheva, Roumen Iankov, and Tom Schanz
One Approximation to the Reachable Sets of Strongly Convex Differential Inclusions Nedka Pulova and Vladimir Pulov Robust Methodology for Characterizing System Response to Damage: Approach Based on Partial Order Paul J. Tanenbaum, Carlos de la Mora, Piotr Wojciechowski, Olga Kosheleva, Vladik Kreinovich, Scott A. Starks, and Alexandr V. Kuzminykh
V
270
276
Environmental Modelling
Comparison of Two Local Refinement Methods for Large Scale Air Pollution Simulations Anton Antonov, Krassimir Georgiev, Emilia Komsalova, and Zahari Zlatev Testing Weighted Splitting Schemes on a One-Column Transport-Chemistry Model Mike Botchev, István Faragó, and Ágnes Havasi Long-Range Transport of Dust to the Baltic Sea Region Marke Hongisto and Mikhail Sofiev Atmospheric Dispersion and Pollutant Chemical Transformation Simulated with Parallel Calculation Using a PC Cluster Christelle Philippe, Alexis Coppalle
287
295 303
312
X
Table of Contents
A Mesoscale Study of Large Industrial Emission Impact over Madrid Mesoscale Domain by Using MM5-CMAQ Modelling System Roberto San José García, Juan Luis Pérez Camaño, Raul Priego, and Rosa M. González Barras
320
Study of the Pollution Exchange between Bulgaria and Northern Greece 328 Christos Zerefos, Dimiter Syrakov, Kostadin Ganev, Alexandros Vasaras, Kostas Kourtidis, Maria Tzortziou, Maria Prodanova, Reneta Dimitrova, Dimiter Yordanov, and Nikolai Miloshev Studying High Ozine Levels in Bulgaria and Europe Zahari Zlatev and Dimiter Syrakov
VI
337
Large Scale Computation for Engineering Problems
Collocation Methods for Boundary Value Problems with an Essential Singularity Winfried Auzinger, Othmar Koch, and Ewa Weinmüller
347
Boundary Integral Method for Deformable Interfaces in the Presence of Insoluble Surfactants Ivan B. Bazhlekov, Patrick D. Anderson, and Han E.H. Meijer
355
Large Eddy Simulation of Turbulent Square Channel Flow Using a PC-Cluster Architecture Jordan Denev, Thomas Frank, and Klaus Pachler
363
Adaptive Grid Refinement for Computation of the Homogenized Elasticity Tensor Ronald H. W. Hoppe, Svetozara I. Petrova, and Yuri V. Vassilevski
371
Multigrid Preconditioned Solvers for Some Elastoplastic Problems Johanna Kienesberger
379
The Free Transmission Problem for a Water Droplet Dirk Langemann
387
Numerical Simulation of the Flow in Magnetic Fluid Rotary Shaft Seals Teodora Mitkova and Lutz Tobiska
396
Phase-Field Method for 2D Dendritic Growth Vladimir Slavov, Stefka Dimova, and Oleg Iliev
404
Design of 2-D FIR Digital Filters Using a Parallel Machine Felicja Wysocka-Schillak
412
Table of Contents
VII
XI
Contributed Talks
Parallel Performance Comparison of Three Direct Separable Elliptic Solvers Gergana Bencheva
421
Finite Volume Difference Methods for Convection-Dominated Problems with Interface Iliya A. Brayanov and Lubin G. Vulkov
429
Parameter Estimations in Nonlinear Parabolic Systems with Time Delay Gabriel Dimitriu and
438
Systolic Architecture of Adaptive Post Detection Integration CFAR Processor in Binomial Distribution Pulse Jamming Ivan Garvanov, Christo Kabakchiev, and Plamen Daskalov
448
Immersed-Boundary Level Set Approach for Numerical Solution of Elliptic Interface Problems Juri D. Kandilarov
456
Generalized Nonstandard Numerical Methods for Nonlinear Advection-Diffusion-Reaction Equations Hristo V. Kojouharov and Bruno D. Welfert
465
On the Computation of Blow-Up Solutions of Elliptic Equations with Semilinear Dynamical Boundary Conditions Miglena N. Koleva
473
A Method for Solving Hermitian Pentadiagonal Block Circulant Systems of Linear Equations 481 Borislav V. Minchev and Ivan G. Ivanov
Author Index
489
This page intentionally left blank
Part I
Invited Papers
This page intentionally left blank
Eigenvalue Estimates for Preconditioned Saddle Point Matrices Owe Axelsson University of Nijmegen, Nijmegen, The Netherlands
[email protected]
Abstract. Eigenvalue bounds for saddle point matrices on symmetric or, more generally, nonsymmetric form are derived and applied for preconditioned versions of the matrices. The preconditioners enable efficient iterative solution of the corresponding linear systems.
1
Introduction
Matrices on saddle point form arise in constrained problems which in various forms occur in applications such as in constrained optimization, flow problems for incompressible materials, and domain decomposition methods, to name a few. For large scale such problems iterative methods must be used and require then efficient preconditioners. We derive first bounds on the eigenvalues of symmetric forms of saddle point matrices, showing how they depend on the top matrix block and the Schur complement matrix. The bounds are then improved using a congruence transformation of the matrix, which makes the off-diagonal matrices small. The transformation is followed by a block-diagonal preconditioner. It is seen that the off-diagonal blocks have only a second order influence on the resulting eigenvalue bounds, and the eigenvalues depend mainly only on the preconditioned top matrix and negative Schur complement matrix, which, by assumption, both have positive eigenvalues. The eigenvalues of the resulting preconditioned matrix are real and cluster around – 1 and +1. A corresponding form of transformation can be applied for nonsymmetric matrices. One source of nonsymmetry can be due to a shift of sign of the constrained equations and we show in this case how the eigenvalues of the preconditioned matrix cluster around the unit number in the complex plane. The preconditioned matrix problems can be solved using a generalized minimal residual form of the conjugate gradient method. For intervals symmetrically located around the origin, the effective condition number equals the square of the condition number for each of the two intervals, showing the importance of having preconditioned the matrices to have intervals with sufficiently small condition numbers. For problems where the matrix defining the constraint is (nearly) rank deficient or, equally, the corresponding term in the Schur complement matrix has a zero or small eigenvalues, one can apply a regularization technique to stabilize the smallest eigenvalue. Such methods have been used in [1,3], for instance. I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 3–16, 2004. © Springer-Verlag Berlin Heidelberg 2004
4
Owe Axelsson
The present method is applicable also for nonsymmetric problems on saddle point form. Here the top matrix block is strongly nonsymmetric when it originates from a convection dominated convection diffusion problem, for instance. For such problems it is important to use a discretization method which leads to nearly M-matrices enabling a efficient preconditioning, in particular if a proper node ordering based on the directions of the flow field has been used. Examples of such methods can be found in [4] and [5], for instance. Some of the above estimates have previously been derived in [6]. See also other references cited there. In the present paper the estimates of eigenvalue bounds for the preconditioned matrices are derived from the general estimates for matrices on saddle point form. In particular, they generalize the methods presented in [6, 7] and elsewhere, where the preconditioning matrix itself has a saddle point form but is factorized in block triangular factors.
2
Eigenvalues of Matrices on Saddle Point Form
Given
where M of order
is symmetric and positive defi-
nite, C of order is symmetric and it is assumed that the negative Schur complement matrix is positive definite. Note that if C = 0, the latter assumption implies that and B has full
Let be the eigenvalues of M be the eigenvalues of can be factored as
where
are corresponding unit matrices, that is, it can be written as a con-
gruence transformation of
Since, by assumption, the latter matrix is
indefinite and since, by Sylvester’s theorem, congruence transformations preserve the signs of the eigenvalues it follows that is indefinite and also nonsingular, because both M and S are nonsingular. We shall compute intervals that contain the eigenvalues of Being symmetric and indefinite, the eigenvalues of are real and located on both sides of the origin. The next theorem shows bounds for the eigenvalue intervals. Theorem 1. Let
where M and
metric and positive definite. Let be the eigenvalues of M and the spectral radius. Then
are symrespectively, and let
Eigenvalue Estimates for Preconditioned Saddle Point Matrices
(a) The eigenvalues
of
5
are located in the two intervals
(b) If C is positive semidefinite then the upper bound can be replaced by the more accurate bound
(c) If C = 0 and B has full row rank, then
Proof. Let be an eigenpair of rewrite this eigenvalue problem in the form
or, with
i.e.,
in the form
This is further rewritten in the form
or
Since is nonsingular, it holds that Given an eigenpair, we consider the two cases, (i) (ii) separately. (i) In this case is nonsingular and (2) reduces to
We
6
Owe Axelsson
or where Hence
Since
(ii)
it follows that
In this case
is nonsingular and (2) reduces to
or
so Since now
it follows that
that is
Here
where Therefore, by (3)
This proves part (a). For parts (b) and (c) we use (1), which for to
or Let
Since
it holds that
and it follows that
reduces
Eigenvalue Estimates for Preconditioned Saddle Point Matrices
7
For C = 0, we have It holds
or
Therefore, since
where if C = 0, otherwise It only remains to prove the bounds for the negative eigenvalues for the case C = 0. It follows from (4) and (5),
where For
we get the solution
which, since
shows the stated bounds for C = 0.
Corollary 1. The eigenvalues of the block-diagonal preconditioned matrix are contained in the intervals where definite. Proof. Replace S with
Here and M with
if C is positive semi-
in Theorem 1.
Remark 1. The above bounds are more accurate than previously given bounds in [8] and [9]. In addition, they hold even if C is not positive semi-definite. An important observation from Theorem 1 is that the negative eigenvalues depend essentially only on S while the positive eigenvalues depend essentially only on M. If M is well-conditioned, the eigenvalue bounds are most sensitive to the values of and Frequently is bounded by a not very big number but can take small values if B is nearly rank-deficient and C does not compensate for this.
8
2.1
Owe Axelsson
Improvement of Eigenvalue Bounds by Congruence Transformation
We show now how the eigenvalue bounds of a given matrix on saddle point form can be improved by a congruence transformation for a proper block triangular matrix Z. Since, by Sylvester’s theorem, a congruence transformation preserves the signs of the eigenvalues, as is seen from Corollary 1, the aim is here to cluster the eigenvalues around the points –1 and +1. Let then be an approximate inverse of M and let
Then a computation shows that
Here –S, where is the Schur complement of It is readily seen that this also equals the Schur complement of Therefore, applying Theorem 1 for with B replaced by shows that the eigenvalues of are contained in the intervals
where accurate preconditioner to approximately in the intervals
When is a sufficiently it follows that the eigenvalues are contained
where the perturbations are of second order. Applying now proper symmetric and positive definite preconditioners and of S will shrink the intervals to (approximately)
of M
where are the extreme eigenvalues of and where The matrix can be a sparse matrix, possibly just diagonal, if it suffices to make small. We can also apply a more accurate preconditioner, such as The most sensitive part in the preconditioning is determined by which is small if B is nearly rank deficient and the matrix C does not stabilize, i.e. compensate for this defect. See further Section 4 for comments on this.
Eigenvalue Estimates for Preconditioned Saddle Point Matrices
9
Block-Diagonal Preconditioner; Computational Complexity To compute the action on a vector
of the preconditioned matrix two actions of
and one of
where and
take
place, in addition to some matrix vector multiplications and vector additions. For the choice it suffices with two actions of the following algorithm is used:
(i) (ii) (iii) (iv) (v)
2.2
and one of
if
Solve Solve Compute Compute Solve
Iteration Number Bounds
The preconditioned saddle-point system can be solved by a generalized conjugate gradient-minimal residual method (see e.g. [2] ), which are based on the best approximations in the Krylov subspace
where is the preconditioner and for a given initial vector For symmetric matrices, the rate of convergence of such methods can be estimated by a best polynomial approximation, namely for some norm it holds
where is the set of eigenvalues of B and denotes the set of polynomials of degree normalized at the origin. For matrices with eigenvalues located symmetrically about the origin in intervals of equal length, it follows that the best polynomial approximation takes the form
which corresponds to a condition number that is, the square of the condition number for a single interval. For non-equal length intervals better bounds can be derived, but the above shows that it is important to precondition both S and M so that the condition number of each interval is not too large. It also shows that there are good reasons to look for preconditioning methods which transforms the eigenvalues to the positive halfplane, which we now consider.
10
Owe Axelsson
3 Clustering the Eigenvalues around the Unit Number in the Complex Plane Instead of using a preconditioner which clusters the eigenvalue around the points –1,+1 on the real axis, it can be more efficient to precondition the matrix to cluster the eigenvalues around the unit number in the complex plane. Applying the congruence transformation in (6), we change then the sign in the lower block part of the matrix and precondition it with a block diagonal matrix,
where
are symmetric and positive definite. The
corresponding eigenvalue problem takes then the form
which can be rewritten in the form
where To find the relative perturbations of the eigenvalues from those of matrix in (7) is rewritten in the form
and
the
where Here (8) shows that there is a perturbation of the real part with and of the imaginary part, arising from the skew symmetric part of the matrix, of It can be seen that if the eigenvalues are contained in an ellipse symmetrically located on the positive real axis, with eccentricity (that is, ratio of the semi axes), then the convergence factor which holds for a real interval is multiplied by the factor
(see e.g. [2]). The importance of having a narrow ellipse,
where are the foci of the ellipse, is seen. For applications in partial differential equation problems, where M is nonsymmetric (such as arising from a convection-diffusion problem) using discretization methods which leads to matrices M which are nearly M-matrices, enables the construction of accurate preconditioners to M. We show now how such preconditioners cluster the eigenvalues near the eigenvalues of The method is applicable also for nonsymmetric matrices.
Eigenvalue Estimates for Preconditioned Saddle Point Matrices
3.1
11
A General Preconditioning Technique
Consider a nonsingular and in general nonsymmetric matrix, partitioned in twoby-two block form,
where has order We assume that is diagonalizable, i.e., has a complete eigenvector space and that is nonsingular. The most important application we have in mind is when is a matrix of regularized saddle point form, in which case is zero or negative semidefinite. When the given matrix is indefinite, we may transform it by multiplying the matrix rows for the bottom blocks by (–1). In this case the form is typically
where now the block matrix consists of a block diagonal part and a skewsymmetric off-diagonal part. The matrix M may be nonsymmetric. We shall present a general block incomplete factorization method for which enables clustering of eigenvalues of the preconditioning matrix around the unit number or around +1 and – 1, depending on the signs of certain matrices. Let then be nonsingular matrices of orders consistent with this partitioning and let
Let further
Here is a preconditioner to for instance an incomplete factorization, and is a preconditioner to the Schur complement matrix (see below for an explanation of these choices). will be used as a preconditioner to either as a left preconditioner or in the form which latter matrix is similarly equivalent to A computation shows that
The matrix
can be rewritten in the form
12
Owe Axelsson
where It follows from (10) that in order for to be an accurate preconditioner of must be an accurate preconditioner of and an accurate preconditioner of The accuracies of these approximations will be made precise in the next theorem. We need then the following (elementary) result. Lemma 1. It holds
where Proof. Let
There holds
where
and a computation shows that Theorem 2. Let
be preconditioned by
in (9), where
is a nonsingular preconditioner to and is a nonsingular preconditioner to Then, for a sufficiently accurate preconditioner the eigenvalues of the block preconditioned matrix cluster around unity and around the eigenvalues of where Namely, by a proper ordering of the eigenvalues it holds where
and Proof. Take norms in the second term of (11) and use Lemma 1. The theorem shows that we can control the clustering of the eigenvalues to be sufficiently close to unity and to the eigenvalues for by applying a sufficiently accurate preconditioner to These two approximations can be controlled, essentially independently of each other, the only dependence is that involves the inverse of the first preconditioner. Note that for saddle point matrices we may have eigenvalues with positive or negative real parts, depending on if the initial the transformation to the form has taken place, or not.
Eigenvalue Estimates for Preconditioned Saddle Point Matrices
3.2
13
One-Sided versus Two-Sided Preconditioners
When dealing with iterative methods for nonsymmetric matrices an additional complication arises in that the matrix may not be normal, i.e., not symmetrizable. This means appearance of Jordan blocks, which can delay the rate of convergence substantially, or cause convergence stagnation. We discuss this issue in connection with one-sided versus two-sided preconditioners. Consider then the factorization of a nonsingular matrix
i.e.
Let be a preconditioner to and to for a left-sided, i.e. block Gauss-Seidel preconditioner it holds
Then
Now, even if is an accurate preconditioner to or, even if the presence of the upper off-diagonal matrix in the second factor in (12) may cause a delay or stagnation in the convergence of an iterative solution method. If
since then
it follows that the
minimal polynomial to is just i.e. has degree 2 and the convergence bound in section 2.2 shows that the GCG – MR method converges in just two iterations. However, in practice we have only approximations of and and the degree of the minimal polynomial is the equal or close to the order of the whole system rather than 2. In addition, the norms of the matrix powers
which arise in a Krylov method, can grow
rapidly with increasing number of iterations, causing perturbations and loss of orthogonality among the direction vectors in the minimal residual method. Numerical test show also a strong dependence on the preconditioning accuracy, in particular of to M. For reasons of robustness of an iterative method, it is therefore advisable to use a two-sided preconditioner, where the powers of the corresponding factors grow little. The one-sided preconditioner still involves actions of matrices and which is normally the computationally most expensive part in each iteration step, and little is saved compared to using a two-sided preconditioner, while the latter normally gives much fewer iterations.
14
4
Owe Axelsson
Regularization of (Nearly) Rank Deficient Problems
In finite element analysis the so-called Babuška-Brezzi (BB) condition, which is an inf sup condition, is commonly used to analyse the stability of constrained boundary value problems. This condition can be presented in algebraic form as follows. As we shall see, it is related to the smallest eigenvalues of the Schur complement matrix. Let B be of order where and rank First recall that the MoorePenrose generalized inverse of B equals The algebraic form of the BB condition is
Here, given
the sup is taken for
(or its scalar multiple), for which
We have
Hence, it follows that
Similarly, we obtain for the M-norm of
where M is spd,
Here if B has full rank but if B is rank deficient. If B is (nearly) rank deficient, one must regularize the problem, writing it in the form
Here the right-hand side term can sometimes be computed from the given problem as for Stokes problems or, otherwise, it is treated in a defect-correction way, i.e. with some initial approximation one computes a correction from
which may be repeated ones a number of times using the updated solutions.
Eigenvalue Estimates for Preconditioned Saddle Point Matrices
15
The matrix C is symmetric and positive semidefinite and positive definite on the nullspace of For the corresponding Schur complement it holds then
For the BB-condition to hold in finite element problems one must use stable element pairs for the two variables (such as velocity and pressure). This means that the space V for (velocity) must be sufficiently richer than that (W) for (pressure). For the simplest piecewise linear finite elements one can add bubble functions to V to form W. On the other hand, when finite element methods are implemented on parallel computer platforms, the data communication is simplified if one uses equal order finite elements for V and W. To stabilize for this, or to increase the stabilization even when one uses some stable element pairs, but where the constant is too small, one must use a stabilization term for some proper scalar The stabilization term should be active for the functions which are missing in an equal order finite element method compared to that for a BB-stable method. As such functions are of bubble type, they correspond to oscillating functions for which the choice the Laplacian operator is active and gives large perturbations On the other hand, the size of the coefficient must be sufficiently small in order not to perturb the discretization order of the method. This indicates that is a proper choice. This has been shown for the Stokes problem in [3]. Here, taking the divergence of the first equation using the incompressibility
results in the equation,
which, multiplied by a constant
gives
and corresponds to the regularization term
Similar choices can hold for more general problems, see e.g. [6]. However, the form of the regularization term is problem dependent.
5
Concluding Remarks
It has been shown how saddle point problems can be solved efficiently by iterative solution methods using particular preconditioners. Thereby it is important to precondition both the top-block matrix and the Schur complement matrix
16
Owe Axelsson
accurately, to achieve a method requiring few iterations. Stabilization is necessary to increase the smallest eigenvalue of the Schur complement matrix, when the constraint condition matrix is singular or nearly singular. In particular, it enables the use of simpler (equal order) finite elements in certain incompressible material problems (see [3] ). A way to make it possible to accurately precondition the top matrix block when it arises from convection diffusion problems is to use a discretization method for which the arising matrix ( M ) is an M-matrix. Here incomplete factorization methods results in accurate factorizations if a node ordering based on the local flow directions is used, see [5] for an example where block matrix factorizations have been used.
References 1. Axelsson, O.: Preconditioning of indefinite problems by regularization. SIAM Journal on Numerical Analysis 16 (1979) 58–69. 2. Axelsson, O.: A generalized conjugate gradient, least squares method. Numerische Mathematik 51 (1987) 209–227. 3. Axelsson, O., Barker, V.A., Neytcheva, M., Polman, B.: Solving the Stokes problem on a massively parallel computer. Mathematical Modelling and Analysis 4 (2000) 1–22. 4. Axelsson, O., Marinova, R.: A hybrid method of characteristics and central difference method for convection-diffusion problems. Computational and Applied Mathematics 21 (2002) 631–659. 5. Axelsson, O., Eijkhout, V., Polman, B., Vassilevski, P.: Iterative solution of singular perturbation 2nd order boundary value problems by use of incomplete block-matrix factorization methods. BIT 29 (1989) 867–889. 6. Axelsson, O., Neytcheva, M.: Preconditioning methods for linear systems arising in constrained optimization problems. Numerical Linear Algebra with Applications 10 (2003) 3–31. 7. Peruggia, I., Simoncini, V.: Block-diagonal and indefinite symmetric preconditioners for mixed finite element formulations. Preconditioning techniques for large sparse matrix problems in industrial application. Numerical Linear Algebra with Applications 7 (2000) 585–616. 8. Rusten, T., Winther, R.: A preconditioned iterative method for saddle point problems. SIAM J. Matrix. Anal. Appl. 13 (1992) 887–904. 9. Wathen, A., Silvester, D.: Fast Iterative Solution of Stabilized Stokes Systems Part II: Using General Block Preconditioners. SIAM Journal on Numerical Analysis 31 (1994) 1352–1367.
A 3D Projection Scheme for Incompressible Multiphase Flows Using Dynamic Front Refinement and Reconnection Tong Chen1, Peter Dimitrov Minev2, and Krishnaswamy Nandakumar1 1
Department of Chemical and Materials Engineering University of Alberta, Edmonton, Alberta, Canada T6G 2G6 2 Department of Mathematical and Statistical Sciences University of Alberta, Edmonton, Alberta, Canada T6G 2G1 {tc8,minev,kumar.nandakumar}@ualberta.ca
Abstract. This paper presents a 3D finite element method for incompressible multiphase flows with capillary interfaces based on a (formally) second-order projection scheme. The discretization is on a fixed reference (Eulerian) grid with an edge-based local The fluid phases are identified and advected using a level set function. The reference grid is then temporarily reconnected around the interfaces in order to maintain optimal interpolations accounting for the singularities. This method is simple and efficient, as demonstrated by the numerical examples on several free-surface problems.
1
Introduction
Multiphase flows, including the free-surface flows and bubbly flows, occur in a wide range of geophysical, chemical and industrial processes. Among them are oil transportation, mixing in chemical reactors, cooling of nuclear reactors, aeration processes, film coating, plastic molding and biomechanics. In general, it seems that it is more efficient to solve such problems in an Eulerian reference frame rather than in a Lagrangian or Eulerian-Lagrangian framework. There are numerous techniques based on the Eulerian approach and the reader is referred to the recent and comprehensive reviews by Scardovelli and Zaleski (1999) for the volume-of-fluid method, Osher and Fedkiw (2001) for the level-set method, and Tryggvason et al. (2001) for the front-tracking method. The difficulties with multiphase flow simulations are mostly due to the singularities (discontinuous pressure and normal derivatives of velocity) across the time-dependent free boundaries. At present, the most popular way to deal with the singularities at the interface, particularly in the finite difference context, is an approach based on a regularization of the using trigonometric approximation. However, such methods are at best only first-order accurate in space. The finite element method (FEM) for free-boundary problems has also undergone rapid development. Minev at el. (2003) have presented a 3D finite element technique based on a dynamic basis enrichment for pressure and velocity in Taylor-Hood I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 17–24, 2004. © Springer-Verlag Berlin Heidelberg 2004
18
Tong Chen, Peter Dimitrov Minev, and Krishnaswamy Nandakumar
elements. In 2D, a simple but efficient projection scheme has been proposed by the authors (Chen et al., 2003). In the present paper, we extend this scheme to 3D and demonstrate its capabilities on several difficult numerical examples.
2
Mathematical Formulation
We consider a 3D flow domain that contains two different Newtonian fluids and with constant densities and viscosities The fluids are assumed to be homogeneous, immiscible and incompressible, and separated by an interface In each single-phase domain the fluid motion is governed by the Navier-Stokes equations which, in a Cartesian coordinate system with the pointing opposite to gravity, read (in a stress-divergence form)
where the hydrodynamic pressure is the total pressure minus the hydrostatic pressure are the velocities and is the gravity acceleration, is the deviatoric stress with being the rate-of-strain tensor. Without the loss of generality, we assume Dirichlet boundary conditions on the external boundary Based on the classical hypothesis that the surface tension is proportional to the mean curvature of the balance of the total stress and the domain continuity results in the well-known interfacial conditions: Here, I is the identity tensor. [·] represents the jump at between the limiting values from the two sides of is the unit vector normal to pointing towards from and is the coefficient of surface tension. The mean curvature is The fluid properties are identified using a continuous indicator function that satisfies It is convenient though if the fluid interface
The normal direction to
3
is given by
corresponds to
so that
at
Discretization
With the time discretization, the Navier-Stokes equations in each of the singlephase domain can be split into a pure convection problem and a generalized
A 3D Projection Scheme for Incompressible Multiphase Flows
19
Stokes problem. In the present study, the former is solved using the AdamsBashforth method and the latter is tackled with the second-order projection scheme of Kim and Moin (1985). Starting at time level where the linearized fractional substeps are summarized as
Here, is the convected velocity. pressure by
is an auxiliary variable related to the
which is never explicitly computed. The free-surface boundary condition (2.2a) is split into and as discussed by Chen et al. (2003). The weak formulation of (3.1a) is trivial and therefore we present only the formulations of the split generalized Stokes problem (3.1b-d), with the special treatment for the interfacial boundary conditions. Let us choose, in each of the single-phase domain the test functions for the velocity and the test functions for the pressure Then the Galerkin formulations for (3.1b,c) and the for (3.1d) yield, respectively
The projection step (3.3c) makes the scheme equivalent to one of the stabilized schemes (at least in case of a single-phase flow) discussed by Codina (2001) and therefore allows for the use of an equal-order approximation for the pressure and velocity (see also Chen et al., 2003). The spatial discretization is performed on a fixed, Eulerian finite element grid (reference grid) which, however, is locally adapted (working grid) at each time step so that the interface is always aligned with element faces i.e. it never crosses the interior of a finite element of the working grid at any given discrete time instant An element on the reference grid that intersects with the interface and is to be reconnected to construct the working grid is called a front element. Furthermore, the basic reference grid can be refined prior to the adjustment of element faces with the interface. The pressure is double valued
20
Tong Chen, Peter Dimitrov Minev, and Krishnaswamy Nandakumar
in the nodes of the interface to account for its jump. We use approximation for the velocity space on the working grid, which automatically accounts for the discontinuity of the first derivatives of velocity. Equation (3.1a) as well as the equation (2.5) for the indicator are discretized on the reference grid while (3.1b-d) are discretized on the working grid. Let be the shape function for the velocity, and for the pressure, corresponding to a point The subscript denotes a discrete quantity and by definition
Imposing the condition
the unified weak formulation (3.3a) becomes
Here, is subject to the external boundary condition With which is a direct consequence of the weak formulation (3.3b) reads
and
Equation (3.5) contains two degrees of freedom for the pressure at each node that belongs to One of them can be condensed using the boundary condition
which appears to be a periodic-like boundary condition corresponding to the two fluids in and Here, is given by
due to (3.2). We can compute pointwise as explained later. Let us consider one interface node, J, and denote by are all the nodes linked to J through the elements containing J, including J itself. Let also be the subset of belonging to from Substituting (3.6) into (3.5) for node J so as to eliminate (condense) the values from we end up with
Likewise, we can obtain the discrete formulation for (3.3c) with the help of
A 3D Projection Scheme for Incompressible Multiphase Flows
21
of which the right-hand side can also be rewritten similar to (3.7):
In summary, equations (3.4), (3.7) and (3.8) form the unified algebraic system for the generalized Stokes problem on Obviously, it reduces to a conventional single-phase flow solver on either the reference grid or the working grid when For the purpose of pressure condensation, we need to evaluate the jump pointwise in all the nodes J on This is obtained by a localized least squares method as explained in Chen et al. (2003). Local refinement of mesh is a powerful tool for reducing the computational cost, especially in three dimensional and multiphase flow simulations where most effects are concentrated around the free interface. We applied the local technique (Ruprecht and Heinrich, 1998) to the front elements and their immediate neighbouring elements of the basic reference mesh. It is an edgebased subdivision of tetrahedron, very fast for time-dependant problems if the permanent edge information is stored prior to the time marching. The refinement is conformal in the sense that all hanging nodes are avoided according to the built-in templates. The values of a mid-edge (inserted) node will be inherited from the previous time level if it is refined previously; otherwise, linear averaging is applied. The identification of the front elements on the reference mesh is straightforward because in such elements the level set function would change sign in two vertices of an edge. Any variable to be interpolated at including the position of intersection can be determined using the linear finite element interpolant of along the edge (say, AB):
where It introduces dynamically new degrees of freedom to the system for both the velocity and pressure along the interface Generally, the interface (a plane in a linear tetrahedron) will cut the element, as shown in Fig. 1, in two configurations. In case (a) the tetrahedron is cut into one tetrahedron and one prism, while in case (b) it is cut into two prisms. Each of the prisms is further subdivided into three tetrahedra by trying to use the better diagonal combinations of the quadrilateral faces without violating the connectivity rules. Since these are the most common situations, the algorithm is dramatically simplified if it is not allowed that the intersection coincides with a vertex of the front element and then these would be the only two possible situations left. This restriction is enforced in the following manner. If the distance between on edge AB and its closest vertex (A/B) is smaller than a preset minimum allowable distance, then this point is fictitiously moved away from this vertex (without resetting on this distance along the edge. This is done by limiting where is a small positive number less than one half. This deviation introduces an error in the approximation of the zero-th level set,
22
Tong Chen, Peter Dimitrov Minev, and Krishnaswamy Nandakumar
Fig. 1. Reconnection of the front elements (pressure is double valued at intersection points 5, 6, 7, 8). The prism is further subdivided into three tetrahedra by linking the shorter diagonals of the quadrilateral faces. which is consistent with the overall accuracy of the method. for all the examples presented here.The choice of can affect the quality of the grid and therefore different values may be recommendable for other physical situations.
4
Numerical Examples
The present technique was quantitatively verified using the problem for bubble oscillation in a low-viscosity and initially quiescent liquid at zero gravity. We performed three tests starting with an initially ellipsoidal bubble whose volume is equal to that of a sphere of radius R = 0.5. The problem was solved in 1/8 of the domain by imposing symmetric conditions on the axis planes. The basic reference grid used for these simulations was a uniform grid containing uniform cubes, each subdivided into 6 tetrahedra. The dynamic gives an effective resolution of nodes. The densities were chosen to be and the viscosities, and The surface tension coefficients in the three cases were 0.5 and 0.2 correspondingly. The computed periods of oscillation are 0.81, 1.15 and 1.83 correspondingly, and they agree well with the classical potential flow results of Lamb (1932): 0.79, 1.11 and 1.76. The next testcase is about the Rayleigh-Taylor instability in a three dimensional rectangular box with a square horizontal cross section. The height-width aspect ratio was fixed at H = 4W. In order to compare our results with those of He et al. (1999), the surface tension was neglected and the viscosities were chosen to be the same for both the heavy and the light fluids. We solve the 1/4 domain symmetric with respect to the and planes. Periodic bound-
A 3D Projection Scheme for Incompressible Multiphase Flows
23
ary conditions were applied at the four vertical sides, while no-slip boundary conditions were applied at the top and bottom walls. The instability develops from the imposed single-mode initial perturbation
at 3/4 of the height above the bottom. The parameter setting is and which gives effectively an Atwood number a Reynolds number and a Froude number Fr = 1. The domain was resolved with the basic reference grid 16 × 16 × 128 nodes in right-angle tetrahedra. The effective local (around the interface) resolution is doubled after the one-level The interface evolution as the fluids penetrate each other is presented in Fig. 2. It is seen that the heavy fluid falls down in a relatively narrow “spike” forming a mushroomshaped end while the lighter fluid rises upward as a large “bubble”. The existence of four “saddle points” at the middle of the vertical sides distinguishes it from a two-dimensional Rayleigh-Taylor instability. In Fig. 3 we present the time evolutions of the spike tip, bubble front and the saddle point. The data correlate favourably with the higher-resolution (128 × 128 × 512, full domain) parallel computations of He et al. (1999) using the lattice Boltzmann method.
Fig. 2. Evolution of the fluid interface due to the Rayleigh-Taylor instability from a single-mode perturbation (At = 0.5, Re = 1000, Fr = 1).
24
Tong Chen, Peter Dimitrov Minev, and Krishnaswamy Nandakumar
Fig. 3. Positions of the bubble front, spike tip and saddle point versus time (At = 0.5, Re = 1000 ,Fr= 1).
References 1. Chen, T., Minev, P.D. and Nandakumar, K.: A projection scheme for incompressible multiphase flow using adaptive Eulerian grids. Int. J. Numer. Meth. Fluids (to appear 2003). 2. He, X.Y., Zhang, R.Y., Chen, S.Y. and Doolen, G.D.: On the three dimensional Rayleigh-Taylor instability. Phys. Fluids 11(5) (1999) 1143–1152. 3. Kim, J. and Moin, P.: Application of a fractional-step method to incompressible Navier-Stokes equations. J. Comput. Phys. 59 (1985) 308–323. 4. Lamb, H.: Hydrodynamics. Dover, New York (1932) p. 473. 5. Minev, P.D., Chen, T. and Nandakumar, K.: A finite element technique for multifluid incompressible flow using Eulerian grids. J. Comput. Phys. 187 (2003) 255–273. 6. Osher, S. and Fedkiw, R.P.: Level set methods: an overview and some recent results. J. Comput. Phys. 169 (2001) 463–502. 7. Ruprecht, D. and Heinrich, M.: A scheme for edge-based adaptive tetrahedron subdivision. Mathematical Visualization, (Ed.) Hege, H.C. and Polthier, K., Springer Verlag, Heidelberg (1998) pp. 61–70. 8. Scardovelli, R. and Zaleski, S.: Direct numerical simulation of free-surface and interfacial flow. Annu. Rev. Fluid Mech. 31 (1999) 567–603. 9. Tryggvason, G., Bunner, B., Esmaeeli, A., Juric, D., Al-Rawahi, N., Tauber, W., Han, J., Nas, S. and Jan, Y.J.: A front-tracking method for the computations of multiphase flow. J. Comput. Phys. 169 (2001) 708–759.
Schwarz-Like Methods for Approximate Solving Cooperative Systems* Ivo Marek Faculty of Civil Engineering, Czech Insititute of Technology, Prague Thákurova 7, 166 29 Praha, Czech Republic
[email protected]
Abstract. The aim of this contribution is to propose and analyze some computational means to approximate solving mathematical problems appearing in some recent studies devoted to biological and chemical networks.
Basic mathematical tools for investigating some biological and chemical networks as presented in [7,8] are recalled in Section 1. In Section 2 some variants of iterative Schwarz-like methods studied in [1,11] are described and applied to problems discussed in Section 1. An analysis and comparison of two particular Schwarz-like methods is presented in Section 4.
1
Cooperative Systems
In [7] a theory for linear problems of the type
where T is a given infinitesimal generator of a semigroup of operators is developed. Several examples mainly from biology and chemistry where the state vectors of the underlying chemical network follow an evolution (1) are presented there. In these cases the conservation of matter requires the existence of an element such that the pairing is constant during all times of the evolution so that we have
More complicated networks (and some examples are shown [8]) are described by a state vector which is formed by finitely many substates in the fashion
*
Research has been supported by grant No.201/02/0595 of the Grant Agency of the Czech Republic and grant No. MSM 210000010 of the Ministry of Education of the Czech Republic.
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 25–38, 2004. © Springer-Verlag Berlin Heidelberg 2004
26
Ivo Marek
following an evolution
Hence, the subsystems evolve like (1) and (2) for which we developed the theory in [7]. The dependence of the operators on the data is typically on the complete state (3) rather than only on some substates. If for the states of the subsystems, the product can be formed and the “block”-diagonal operator
can be defned on X. Now (3) evolves according to
where and B is the infinitesimal generator of a semigroup of operators of class [9, p.321]. Note that structurally (6) is very similar to (1), (2). Only the infinitesimal generator itself depends on the total state (3) so that our problem becomes nonlinear. In [8] the evolution problem was assumed in the form (6) under very general conditions and an existence theorem (concerning mild solutions which in our applications become classical solutions) was proven for all times and the question of its long run behaviour settled: In fact, any solution of (6) settles in the long run at a steady state. Defining equations to determine this steady state are given there. This theorem is important not only in its own right. It also provides the basis for singular perturbation techniques on such systems to obtain analytic expressions which characterize the speed of reaction systems, and we refer to [4], [17] for the pseudo-steady state process leading to the definition of the speed of the underlying chemical networks. Note that the theory does not need the diagonal form of (5). However, it relies heavily on (2). Relation (6) is only an easy way to fit (3), (4) under the general pattern (10). The original form of the examples is (4). As in the earlier papers [7], [8] we need the theoretical basis for infinitesimal generators with monotonicity properties. The corresponding basic definitions are summarized in this section. Let be a Banach space over the field of real numbers. Let denote the dual space of Let be the corresponding complex extensions of respectively and let and be the spaces of all bounded linear operators mapping into and into respectively. In fact, we are going to provide our investigations in a Hilbert space equipped with an inner product [., .].
Schwarz-Like Methods for Approximate Solving Cooperative Systems
27
Let be a closed normal and generating cone, i.e. let (i) (ii) (iii) (iv) where denotes the norm-closure of (v) and whenever such that (vi) there exists a Property (vi) is called normality of
We let and there exist (vii) For every pair as elements of A cone satisfying condition (vii) is called a lattice cone and the partial order on a lattice order. In the terminology of H.H. Schaefer [14] is called a Banach lattice. Our theory is free of hypothesis (vii). Let and We call the dual cone of and the dual interior of respectively. If happens to be a Hilbert space then is replaced by a representation of in the sense of natural isomorphism of the dual with In the following analysis we assume that the dual interior is nonempty. A set is called if the following implication holds
A linear form
is called strictly positive, if
for all
We write in place of where and respectively. If happens to be a Hilbert space then denotes the appropriate inner product. A bounded linear operator is called if We write in this case and equivalently If T and S both in satisfy we write or equivalently If then denotes its dual and hence, In case is a Hilbetrt space, the dual operator is to be replaced by the adjoint operator defined via relations valid for all in the domain of T and in the domain of
28
Ivo Marek
Definition 1. Operator is called if there exists a vector such that for the dual map the following relation
holds. We also say that T is a transition operator of a Markov chain or process and that operator T corrersponds to vector If is a Hilbert space the dual is to be replaced by its adjoint
Definition 2. A bounded linear operator B is if for every pair of elements there is a positive integer such that This implies that in the Markov chain each state has access to every other state, i.e., the chain is ergodic [15]. The Perron-Frobenius theorem states that for irreducible, is an isolated eigenvalue, and the corresponding eigenvector is positive; see, e.g., [3]. Let and let denote its spectrum. Further, let We introduce the operator by setting where and call it complex extension of T. By definition, we let Similarly, we let where denotes the spectral radius of In order to simplify notation we will identify T and its complex extension and will thus omit the tilde sign denoting the complex extension. The set is called peripheral spectrum of T. Note that If is an isolated singularity of Laurent expansion of around
where
and
where and is such that Furthermore,
is never empty. we have the following
[16], [12]
belong to
Moreover, it holds [16]
Schwarz-Like Methods for Approximate Solving Cooperative Systems If there is a positive integer
29
such that
then is called a pole of the resolvent operator and We define the symbol
its multiplicity.
and call it the index of T at In particular, we call ind(T) the index of T instead of index of T at 0. The motivating examples are particular cases of Problem (P) defined as follows: Problem (P) To find
solutions
where B is generally an unbounded linear densely defined operator and for every is a bounded linear map on where denotes the underlying space to be specified in each particular situation. We assume that we can identify situations in which Problem (P) as formulated above possesses solutions and we are aware of conditions guaranteeing the existence of them as well as some of their properties such as uniqueness, asymptotic behaviour etc. A rather typical representative of such problem is described in our study [8]. Since our aim in the present contribution is to propose some algorithms of computational nature and analyse their properties we do not go into much details referring the reader to [8] to consulting general aspects. All properties needed for a good understanding of the numerical processes studied will be presented here.
2
Schwarz Iterative Methods
In this section we present some notations, definitions, and preliminaries. Analogous concepts on nonnegative matrices (defined here for generally infinite dimensional spaces) can be found in the standard reference [3]. By we denote the spectrum of C and by its spectral radius. By and we denote the range and null space of C, respectively. Let be a pole of the resolvent operator The multiplicity of as a pole of is called the index of C with respect to and denoted Equivalently, if it is the smallest integer for which This happens if and only if Definition 3. Let A be a densely defined being its domain of definition) and spectrally bounded from above linear operator and let be its bound, i.e., implies Assume that for there exists a system of operators such that relation
30
Ivo Marek
holds for every Operator A is called approximate if each of the operators in the collection mentioned has the form with and each being A pair of operators ( M , W) is called a splitting of A if A = M – W and exists as a bounded linear operator on A splitting of an operator A is called of type if the operator is [10]. If, in particular, both operators and W are the splitting is called regular [18]. If and are nonnegative, the splitting is called weak regular [13]. Note a weak regular splitting does require explicitely no conditions upon part W of the splitting of A. Let T be a bounded linear operator. T is called convergent if and zero-convergent, if moreover
exists
Standard stationary iterations of
the form converge if and only if either T is zero-convergent or, if T is convergent. A bounded linear operator T with unit spectral radius being an isolated pole of the resolvent operator is convergent if the following two conditions hold: (i) if and then (ii) Equivalent conditions for (ii) can be found in [3]. It is useful to write T = Q + S, where Q is the first term of the Laurent expansion of T, i.e., the eigenprojection onto the invariant subspace corresponding to see, e.g., [16]. Then QS = SQ = O, and This is called the spectral decomposition of T. The condition (i) above is equivalent to having We state a very useful Lemma; its proof can be found, e.g., in [5]. We note that when this lemma can be used to show condition (ii) above. To prove convergence one needs to show in addition that condition (i) also holds. Lemma 1. Let T be a with Then operator then
3
bounded linear operator such that If furthermore is a pole of the resolvent
Algebraic Formulation of Schwarz Methods
In this Section we want to generalize convergence of some procedures well known as algebraic Schwarz iteration techniques [1], [11]; our aim is however to analyze solving equations in infinite dimensional spaces as well. Given an initial approximation to the solution of
the (one-level) multiplicative Schwarz method can be written as the stationary iteration
Schwarz-Like Methods for Approximate Solving Cooperative Systems
31
where
and
is a certain vector in
Here
where is a suitable linear operator and its adjoint with respect to the inner product in the Hilbert space Note that each and hence each is a projection operator; i.e., Each naturally has spectral radius equal to 1. The additive Schwarz method for the solution of (13) is of the form (14), where
is a damping parameter. where The operator corresponds to the restriction operator from the whole space to a subset of the state space (usually of finite dimension the dimension of the range is infinite in general) in the domain decomposition setting, and the operator is the restriction of A to that subset. A solution using is called a local solver as in the domain decomposition method as well as in the algebraic case. We assume that our standard choice of operators follows the same idea as does the choice of the rows of as rows of the identity matrix I in case of e.g.,
Let us assume that the Hilbert space with which we provide our analyses is partially ordered by a closed normal cone generating i.e. Moreover, let subsets exist such that
is and holds if and only if there exists an element such that In order to allow overlaps of individual reduction maps we do not assume that and Obviously, if and only if there are and such that and if and only if whenever and Formally, space can be considered as a direct sum
32
Ivo Marek
where is the “complementary” subspace to decomposition of operator A given by formula
where maps into We assume that the maps
and
We also have a corresponding
maps
into
are diagonal operators, i.e. for some real In fact, we expect that similarly as in the finite dimensional case operators will signal whether the method chosen does possess some overlaps and how extensive they are. It is easy to see that as in case of finite dimensional situation both and are [3]. For each we construct diagonal operators associated with chosen
If A is an collection of operators
for each associated with
we construct a second as follows
where is invertible diagonal operator representing a “diagonal” of A. The following result comes from [1]. Proposition 1. Let A be a nonsingular Let be defined as in (20). Then the splittings are regular (and thus weak regular and of nonnegative type). In the cases considered in this paper, we always have that defined in (20) are nonsingular. With the definitions (19) and (20) we obtain the following equality We can thus rewrite (15) as
Similarly, (17) can be rewritten as
Schwarz-Like Methods for Approximate Solving Cooperative Systems
33
This is how we interpret the multiplicative and additive Schwarz iterations. In [1] it was shown that when A is nonsingular, and thus, the method (12) is convergent. Furthermore, there exists a unique splitting A = M – W such that This splitting is a weak regular splitting. In this paper we want to explore the convergence of (12), using the iterations defined by (23), (24), when A is singular.
4
Two Approaches to Approximate Solving Stationary Equation
We are going to describe two ways for constructing solutions to stationary equations. The first one is based on a application of one of Schwarz algorithms directly to the stationary equation, the second one utilizes the asymptotic behaviour of solutions of the appropriate evolution. The second alternative is suitable in particular for problems when no additional information is available such as irreducibility etc. As classical properties of M-matrices suggest strict positive diagonals may strongly influence convergence and the speed of convergence of the investigated iterative proccesses. We recall a useful Lemma 2. Let Then
satisfy is convergent.
and
with some real
Proof. Obviously, and hence, For completing the proof we need to show that 1 is a unique spectral point with modulus Let By hypothesis, and can be written as Hence, Let with reals and It follows that
only if or, equivalently, if is real and hence positive. This completes the proof of the Lemma. In [8] it has been shown that a solution to Problem (P) gets stationary and is a solution to
where and are suitable nonnegative reals and Since, by hypothesis,
the initial condition in (10).
34
Ivo Marek
where and denote the appropriate discretizations of C and spectively, it is an easy matter to show that
re-
satisfies Thus, in view of Definition 3,
is
corresponding to vector
Theorem 1. Let be a Hilbert space with an inner product [.,.]. Let A = I –B, where is a operator such that with Let be a positive integer and be splittings of nonnegative type such that the diagonals of are positive. Then
and are convergent operators. Furthermore, there is a splitting of nonnegative type such that erties:
and the iteration operator T possesses the following prop-
and The existence of a splitting of nonnegative type, and properties (28) and (29) also hold for Proof. We begin with the operator Let be such that i.e., For each splittings of we then have that This implies that and by Lemma 1 we have that and that the index is 1. To show that is convergent, we show that for some real This follows from the fact that each of the operators satisfies relation with positive reals We follow a similar logic for the multiplicative Schwarz iteration operator (23). Since and thus and Each factor in (23) can be written as
and since and each factor is nonnegative. For a row in which is zero, the diagonal entry in this factor has value one. For a row in which has value one, the diagonal entry in this factor is the positive diagonal entry of Thus, again, we have a finite product of operators, each dominanting a positive multiple of the identity operator, implying
Schwarz-Like Methods for Approximate Solving Cooperative Systems
35
that the product T does dominate a positive multiple of the identity operator too, and therefore it is convergent. The rest of the proof applies equally to T and we only detail it for T. The matrix T being convergent implies the spectral decomposition (28), where Q is the spectral projection onto the eigenspace of T corresponding to Furthermore since We show now that According to construction of T, the null spaces satisfy Any element of which does not belong to has to have a form for some and Since we have that On the other hand a contradiction. Since we then have that the existence of a splitting of the form (27) follows similarly as does the finite dimensional analog of Theorem 1 from Theorem 2.1 of [2]. The fact that indicates that this splitting is of nonnegative type. With this splitting, using (28) the following identity holds AQ = M(I – T)Q = O, so we also have (29). An example of splittings that lead to iteration matrices satisfying the hypotheses of Theorem 1 is described in the following proposition requiring no proof. It provides a possible modification to the local solvers, when the iteration operator defined by (20) does not dominate a positive multiple of the identity operator. Proposition 2. Let numbers. Let
Let
be any positive real be defined by
and where are defined in (20)-(21). Then, the splittings are regular, and iteration operator dominate a positive multiple of the identity operator. Let us recall that we want to compute the stationary state, i.e. a solution of the system (25) This problem can be reformulated in terms of a new iterative process with the generating nonnegative operators C and
An alternative to the method of approximate solving stationary equation of Problem (P) as described in Theorem 1 is to compute the limit using process (31). From a variety of possible choices of rational approximations of the exponential we choose one from the class of limited Padé approximations, say
36
Ivo Marek
with appropriate real
and polynomial
of degree
Denoting
we want to compute according to (31)
The above process can be implemented as follows. Let us omit the discretization parameter index and set and
Then
Convergence of the method just described is an easy consequence of the fact that the operators are nonsingular in the spirit of Definition 3 and the convergence results of [1]. Actually, we have Theorem 2. Let be a Hilbert space over the reals generated by a closed normal cone Assume B is a generally unbounded linear operator defined on a dense domain and its adjoint satisfies relation We assume further that B generates a semigroup of operators of class such that Finally assume that for any operator is an satisfying Then the iteration process (32) returns a sequence of approximations to a unique solution to Problem (P) such that
5
Conclusions
Nowadays it is accepted by the community of numerical analysts that two- and generally multi-level iterative methods offer an essentially broader variety of tools to solve large scale computational problems. In this context Schwarz and Schwarz-like methods play a quite important role. Many contributions of many authors document this statement as a rule by investigating problems characterized by nonsingular operators. An emphasis of our approach is just the opposite that is to problems with singular operators. Another goal of our analyses is that we consider the computational problems in their original form, i.e. we work with
Schwarz-Like Methods for Approximate Solving Cooperative Systems
37
generally infinite dimensional objects and let discretizations to be made at an appropriate moment, e.g. at each iteration step. We apply our Schwarz-like methods to a problem coming from stochastic modeling in biology and chemistry. We can thus profit from having nicely structured operators but suffer of approaching problems with hardly accessible data. We thus propose two methods each suitable in the corresponding situation. The first method assumes all data accessible and the second just the opposite. In particular, the method using auxilliary time evolution does require no apriori knowledge of location as well as access to each single data. As example let us mention irreducibility of the operators of the model and access to matrix elements of appropriate discretizations. The latter is compensated by ability of our method of an easy computation of corresponding matrix actions.
References l. Benzi M., Frommer A., Nabben R., Szyld D. Algebraic theory of multiplicative Schwarz methods. Numer. Math. 89, 605-639 (2002). 2. Benzi M., Szyld D. B. Existence and uniqueness of splittings for stationary iterative methods with applications to alterning methods. Numer. Math. 76, 309-321 (1997). 3. Berman A., Plemmons R. Non-negative Matrices in the Mathematical Sciences. Academic Press, 1979. 4. Bohl E., Boos W. Quantitative analysis of binding protein-mediated ABC transport system, J. Theor. Biology 186, 65-74 (1997). 5. Bohl E., Marek I. A model of amplification, J. Comput. Appl. Math. 63 , 27-47, 1995. 6. Bohl E., Marek I. A nonlinear model involving M - operators. An amplification effect measured in the cascade of vision. J. Comput. Appl. Math. 60, 13-28, (1994). 7. Bohl E., Marek I. A stability theorem for a class of linear evolution problems, Integral Equations Operator Theory. 34, 251-269, (1999). 8. Bohl E., Marek I. Existence and uniqueness results for nonlinear cooperative systems. Operator Theory : Advances and Applications, Vol. 130, 153-170 (2001). 9. Hille E., Phillips R.S. Functional Analysis and Semigroups. Amer. Math. Socitey Coll. Publ. Vol XXXI, Third printing of Revised Edition Providence, Rhode Island, 1968. 10. Marek I. Frobenius theory of positive operators. Comparison theorems and applications. SIAM Journal on Applied Mathematics 19, 608–628, (1970). 11. Marek I., Szyld D. Algebraic Schwarz methods for the numerical solution of Markov chains. Linear Algebra Appl. Submitted 2003. 12. Marek I., Žitný K. Analytic Theory of Matrices for Applied Sciences, Vol 1., Teubner Texte zur Mathematik Band 60, Leipzig, 1983. 13. J. M. Ortega and W. Rheinboldt. Iterative Solution of Nonlinear Equations In Several Variables. Academic Press, New York, San Francisco, London, 1970. 14. Schaefer H.H. Banach Lattices and Positive Operators. Springer-Verlag BerlinHeidelberg-New York, 1974. 15. Stewart W.J. Introduction to the Numerical Solution of Markov Chains. Princeton University Press. Princeton NJ, 1994. 16. Taylor A. E., Lay D.C. Introduction to Functional Analysis. Second edition, J. Wiley Publ. New York, 1980.
38
Ivo Marek
17. Tralau C., Greller G., Pajatsch M., Boos W., Bohl E. Mathematical treatment of transport data of bacterial transport system to estimate limitation in diffusion through the outer membrane, J. Theor. Biol. 207, 1-14, (2000). 18. R. S. Varga. Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs, New Jersey, 1962. Second Edition, revised and expanded, Springer, Berlin, Heidelberg, New York, 2000.
Computational Infrastructure for Parallel, Distributed, and Grid-Based Monte Carlo Computations Michael Mascagni1 and Yaohang Li2 1
Department of Computer Science and School of Computational Science and Information Technology, Florida State University, Tallahassee, FL 32306-4530, USA
[email protected] http://www.cs.fsu.edu/~mascagni 2
North Carolina A&T State University, Greensboro, NC 27411, USA
[email protected]
Abstract. Monte Carlo applications are widely perceived as computationally intensive but naturally parallel. Therefore, they can be effectively executed using the dynamic bag-of-work model which is well suited to parallel, distributed, and grid-based architectures. This paper concentrates on providing computational infrastructure for Monte Carlo applications on such architectures. This is accomplished by analyzing the characteristics of large-scale Monte Carlo computations, and leveraging the existing Scalable Parallel Random Number Generators (SPRNG) library. Based on these analyses, we improve the efficiency of subtaskscheduling by implementing and analyzing the “N-out-of-M” strategy, and develop a Monte Carlo-specific lightweight checkpointing technique, which leads to a performance improvement. Also, we enhance the trustworthiness of Monte Carlo applications on these architectures by utilizing the statistical nature of Monte Carlo and by cryptographically validating intermediate results utilizing the random number generator already in use in the Monte Carlo application. All these techniques lead to a highperformance grid-computing infrastructure that is capable of providing trustworthy Monte Carlo computation services.
1 Introduction Readers are most likely already familiar with parallel and distributed computing architectures, so we choose not to elaborate on them further. Thus, grid computing is characterized by the large-scale sharing and cooperation of dynamically distributed resources, such as CPU cycles, communication bandwidth, and data, to constitute a computational environment [1]. In the grid’s dynamic environment, from the application point-of-view, two issues are of prime importance: performance – how quickly the grid-computing system can complete the submitted tasks, and trustworthiness – that the results obtained are, in fact, due to the computation requested. To meet these two requirements, many grid-computing or distributed-computing systems, such as Condor [2], HARNESS [3], Javelin [4], I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 39–52, 2004. © Springer-Verlag Berlin Heidelberg 2004
40
Michael Mascagni and Yaohang Li
Globus [5], and Entropia [7], concentrate on developing high-performance and trust-computing facilities through system-level approaches. In this paper, we are going to analyze the characteristics of Monte Carlo applications, which are a potentially large computational category of parallel, distributed, and grid applications, to develop approaches to address performance and trustworthiness issues at the application level. The remainder of this paper is organized as follows. In §2, we analyze the characteristics of Monte Carlo applications. We also quickly describe the random number generation requirements for these applications on parallel, distributed, and grid-based architectures. We then develop a generic grid-computing paradigm for Monte Carlo computations. We discuss how to take advantage of the characteristics of Monte Carlo applications to improve the performance and trustworthiness of Monte Carlo grid computing in §3 and §4, respectively. Finally, §5 summarizes our conclusions and future research directions.
2
Grid-Based Monte Carlo Applications
Among parallel, distributed, and grid applications, those using Monte Carlo methods, which are widely used in scientific computing and simulation, have been considered too simplistic for consideration due to their natural parallelism. However, below we will show that many aspects of Monte Carlo applications can be exploited to provide much higher levels of performance and trustworthiness for computations on these architectures. According to word of mouth, about 50% of the CPU time used on supercomputers at the U. S. Department of Energy National Labs is spent on Monte Carlo computations. Unlike data-intensive applications, Monte Carlo applications are usually computation intensive [6], and they tend to work on relatively small data sets. Parallelism is a way to accelerate the convergence of a Monte Carlo computation. If N processors execute N independent copies of a Monte Carlo computation, the accumulated result will have a variance N time smaller than that of a single copy. In a distributed Monte Carlo application, once a distributed task starts, it can usually be executed independently with almost no interprocess communication. Therefore, Monte Carlo applications are perceived as naturally parallel, and they can usually be programmed via the so-called dynamic bag-of-work model. Here a large task is split into smaller independent subtasks and each are then executed separately. Effectively using the dynamic bag-of-work model for Monte Carlo requires that the underlying random number streams in each subtask be independent in a statistical sense. The SPRNG (Scalable Parallel Random Number Generators) library [11] was designed to use parameterized pseudorandom number generators to provide independent random number streams to parallel processes. Some generators in SPRNG can generate up to independent random number streams with sufficiently long period and good quality [13]. These generators meet the random number requirements of most Monte Carlo applications for these types of architectures.
Computational Infrastructure
41
SPRNG was originally designed to provide independent and dynamic streams of random numbers for massively parallel supercomputers. The design of SPRNG was based on the needs of Monte Carlo applications, like neutronics. In neutronics, it is commonplace to associate a single random number stream with each neutron. When a neutron splits into more neutrons, each child neutron is then associated with a new (and hopefully independent) random number stream. Given such a computation, SPRNG ensures that, when desired, such a computation performed on a parallel or distributed machine can be entirely reproducible. The details of how this is achieved is beyond the current scope [11]. However, these capabilities of SPRNG are extensible to distributed and grid-based architectures, and form the underpinnings for the grid services we will discuss. In the remainder of this paper we thus focus exclusively on grid computing issues for Monte Carlo applications. The intrinsically parallel aspect of Monte Carlo applications makes them an ideal fit for the grid-computing paradigm. In general, grid-based Monte Carlo applications can divide the Monte Carlo task into a number of subtasks by the task-split service and utilize the grid’s schedule service to dispatch these independent subtasks to different nodes [15]. The connectivity services provide communication facilities among nodes providing computational services. The execution of a subtask takes advantage of the storage service of the grid to store intermediate results and to store each subtask’s final (partial) result. When the subtasks are done, the collection service can be used to gather the results and generate the final result of the entire computation. The inherent characteristics of Monte Carlo applications motivate the use of grid computing to effectively perform large-scale Monte Carlo computations. Furthermore, within this Monte Carlo grid-computing paradigm, we can use the statistical nature of Monte Carlo computations and the cryptographic aspects of random numbers to reduce the wallclock time and to enforce the trustworthiness of the computation.
3 3.1
Improving the Performance of Grid-Based Monte Carlo Computing The N-out-of-M Strategy
Subtask-Scheduling Using the N-out-of-M Strategy. The nodes that provide CPU cycles in a grid system will most likely have computational capabilities that vary greatly. A node might be a high-end supercomputer, or a low-end personal computer, even just an intelligent widget. In addition, these nodes are geographically widely distributed and not centrally manageable. A node may go down or become inaccessible without notice while it is working on its task. Therefore, a slow node might become the bottleneck of the whole computation if the assembly of the final result must wait for the partial result generated on this slow node. A delayed subtask might delay the accomplishment of the whole task while a halted subtask might prevent the whole task from ever finishing. To address this problem, system-level methods are used in many grid systems. For
42
Michael Mascagni and Yaohang Li
example, Entropia [7] tracks the execution of each subtask to make sure none of the subtasks are halted or delayed. However, the statistical nature of Monte Carlo applications provides a shortcut to solve this problem at the application level. Suppose we are going to execute a Monte Carlo computation on a grid system. We split it into N subtasks, with each subtask based on its unique independent random number stream. We then schedule each subtask onto the nodes in the grid system. In this case, the assembly of the final result requires all the N partial results generated from the N subtasks. Each subtask is a “key” subtask, since the suspension or delay of any one of these subtasks will have a direct effect on the completion time of the whole task. When we are running Monte Carlo applications, what we really care about is how many random samples (random trajectories) we must obtain to achieve a certain, predetermined, accuracy. We do not much care which random sample set is estimated, provided that all the random samples are independent in a statistical sense. The statistical nature of Monte Carlo applications allows us to enlarge the actual size of the computation by increasing the number of subtasks from N to M, where M > N. Each of these M subtasks uses its unique independent random number set, and we submit M instead of N subtasks to the grid system. Therefore, M bags of computation will be carried out and M partial results may be eventually generated. However, it is not necessary to wait for all M subtasks to finish. When N partial results are ready, we consider the whole task for the grid system to be completed. The application then collects the N partial results and produces the final result. At this point, the grid-computing system may broadcast abort signals to the nodes that are still computing the remaining subtasks. We call this scheduling strategy the N-out-of-M strategy. In the N-out-of-M strategy more subtasks than are needed are actually scheduled, therefore, none of these subtasks will become a “key” subtask and we can tolerate at most M – N delayed or halted subtasks. Also notice that the Monte Carlo computation using the N-out-of-M strategy is reproducible, because we know exactly which N out of M subtasks are actually involved and which random numbers were used. Thus each of these N subtasks can be reproduced later. However, if we want to reproduce all of these N subtasks at a later time on the grid system, the N-out-of-N strategy must be used. One drawback of the N-out-of-M strategy is we must execute more subtasks than actually needed and will therefore increase the computational workload on the grid system. However, our experience with distributed computing systems such as Condor and Javelin shows that most of the time there are more nodes providing computing services available in the grid system than subtasks. Therefore, properly increasing the computational workload to achieve a shorter completion time for a computational task should be an acceptable tradeoff in a grid system. Analysis of the N-out-of-M Strategy. In Monte Carlo applications, N is determined by the application and depends on the number of random samples
Computational Infrastructure
43
or random trajectories needed to obtain a predetermined accuracy. The problem is thus how to choose the value M properly. A good choice of M can prevent a few subtasks from delaying or even halting the whole computation. However, if M is chosen too large, there may be little benefit to the computation at the cost of significantly increasing the workload of the grid system. In order to determine a proper value of M to achieve a specific performance requirement, we study the grid behavior and consider some system parameters. In the Nout-of-M strategy, the completion time of a Monte Carlo computational task depends on the performance of each individual node that is assigned a subtask, the node failure rate, and also the interconnection network failure rate. We make the following assumptions to set up our model:
1. The execution of a task completely occupies a node on the grid, and no other jobs can be executed on the same node concurrently. 2. Compared to the execution time, the tasks’ scheduling time and result collection time is short enough to be ignored. 3. Each node works on its task independently. 4. Each node has an equal probability of obtaining a task from the schedule service. The tasks are scheduled without noticing the performance of each node. To analyze this we establish a Petri Net (PN) model of the N-out-of-M strategy. This PN model has M nodes in total. A node, alternates between an up state (place and a down state (place Transition represents node unavailability (with unavailability rate and transition node back to service (with availability rate Transition is assigned the task progress threshold W (usually 100%) so that the subtask completion condition (token in is reached when W is hit. When gathers N tokens, transition enables firing and a token in indicates the completion of the Monte Carlo task. We also establish a simpler binomial model for the subtask-scheduling scheme using the N-out-of-M strategy based on the above PN model. Assume that the probability of a subtask completing by time is given by The function describes the aggregate probability over the pool of nodes in the grid. In a reallife grid system, could be measured by computing the empirical frequencies of completion times over the pool. In this paper, we model based on an analytic probability distribution function. Let S be the total number of nodes available in the grid system, be the probability of node participating in the computations is up, where be the service rate of node which can be measured as the number of tasks that can be finished within a specific period of time without interruption. Considering node availability, the actual service rate, in node is At time the probability that a Monte Carlo subtask will be done on node is Since each node has equal probability to be scheduled a subtask, can be represented as
44
Michael Mascagni and Yaohang Li
If the service rates, density function
conform to a distribution with probability can thus be written as
Here L is the maximum value of in the computation. Typically, if all of the nodes have the same service rate fied to
can be simpli-
Then, the probability that exactly N out of M subtasks are complete at time is given by
We can approximate Then,
using a Poisson distribution with can be approximated as
The probability that at least N subtasks are complete is thus given by
The old strategy can be thought of as “N-out-of-N” which has probability given by
Now the question is to decide on a reasonable value for M to satisfy a required task completion probability (when N subtasks are complete on the grid). Unfortunately, it is hard to explicitly represent M in analytic form. However, we use a numerical method, which gradually increases M by 1 to evaluate until the value of is greater than This empirically gives us the minimum value of M. An alternative approach to estimate M/N is to use a normal distribution to approximate the underlying binomial. When and the binomial distribution can be approximated by a normal curve with mean and standard deviation Then, we can find the minimum value M that satisfies
Computational Infrastructure
45
Fig. 1. Simulations and Model Prediction of the N-out-of-M Scheduling Strategy for Grid Monte Carlo Applications.
where is the normal cumulative distribution function. In a grid system, nodes providing computational services join and leave dynamically. Some nodes are considered to be “transient” nodes, which provide computational services temporarily and may depart from the system permanently. A subtask submitted to a “transient” node may have no chance of being finished. Suppose the fraction of “transient” nodes in a grid is then, we need to enlarge M to to tolerate these never-finished subtasks. Simulation of the N-out-of-M Strategy. In our simulation program of the N-out-of-M strategy, we simulated a 1,000-node computational grid. Nodes join and leave the system with a specified probability. Also, nodes have a variety of computational capabilities. Each simulation is run for 1,000 time steps. (A task running on a node with service rate will take time steps, e.g., a fast node with service rate 0.01 will take 100 time steps to complete the task while a slow one with service rate 0.001 will take 1.000) At each time step, a certain number of nodes go down while a certain number of nodes become available for computation. We built our simulations in order to
1. evaluate the validity of our model, and to 2. compare the performance of the N-out-of-M strategy in grid systems with different configurations. Figure 1 shows our simulation results and model prediction of the N-out-of-M strategy for grid Monte Carlo applications. Our analytical model matches the simulation results quite well. Also, we can find that with the proper choice of M (20 in the graph), the Monte Carlo task completion time can be improved
46
Michael Mascagni and Yaohang Li
Fig. 2. Simulations of the N-out-of-M Strategy on a Grid System with Nodes Service Rates Normally Distributed (Mean=0.005, Variance=0.001).
significantly over the N-out-of-N strategy. However, if we enlarge M too much, the workload of the system increases without significantly reducing the Monte Carlo task completion time. Also, we notice that as time goes on, the N-out-ofM strategy always has a higher probability of completion than the N-out-of-N strategy, although they all converge to probability one at large times. Figures 2 and 3 show the simulation results of the N-out-of-M strategy in different grid systems. Both simulated grid systems assume that the service rates of nodes are normally distributed with the same means (0.005) but different variances (0.001 in Figure 2 and 0.003 in Figure 3). Figure 2 simulates a grid comprised of nodes with similar performance characteristics. This can be a grid constructed from computers in a computer lab that have similar performance parameters. On the other hand, Figure 3 is the simulation of a grid whose nodes have computational capabilities in a wide range. In practice, this grid can be a system with geographically widely distributed nodes like SETI@home [9], where a node might be a high-end supercomputer, or a low-end personal computer. From the graphs, we see that the N-out-of-M scheduling strategy improves the Monte Carlo task completion time in both grid systems; however, we gain more significant improvement in the system comprised of nodes with service rates having a large variance. This experimental result indicates that the N-out-of-M strategy is more effective in a grid system where an individual node’s performance varies greatly. More interestingly, the simulation results also show that, in both grid systems, with a sufficiently large value of M, the time values after which the Monte Carlo task is complete with a high probability is close to 200 time steps, which is exactly the subtask completion time for a single node with the mean (0.005) service rate. Therefore, we can expect that, with a proper number of sub-
Computational Infrastructure
47
Fig. 3. Simulations of the N-out-of-M Strategy on a Grid System with Nodes Service Rates Normally Distributed (Mean=0.005, Variance=0.003).
tasks scheduled using the N-out-of-M strategy, the Monte Carlo task completion time on a grid can be made to be almost the same as the subtask completion time in a node with average computational capability. Lightweight Checkpointing. A subtask running on a node of a grid system may take a very long time to finish. The N-out-of-M strategy is an attempt to mitigate the effect of this on the overall running time. However, if checkpointing is incorporated, one can directly attack reducing the completion time of the subtasks. Some grid computing systems implement a process-level checkpoint. Condor, for example, takes a snapshot of the process’s current state, including stack and data segments, shared library code, process address space, all CPU states, states of all open files, all signal handlers, and pending signals [12]. On recovery, the process reads the checkpoint file and then restores its state. Since the process state contains a large amount of data, processing such a checkpoint is quite costly. Also, process-level checkpointing is very platform-dependent, which limits the possibility of migrating the process-level checkpoint to another node in a heterogeneous grid-computing environment. Fortunately, Monte Carlo applications have a structure highly amenable to application-based checkpointing. Typically, a Monte Carlo application starts in an initial configuration, evaluates a random sample or a random trajectory, estimates a result, accumulates means and variances with previous results, and repeats this process until some termination condition is met. Although different Monte Carlo applications may have very different implementations, many of them can be developed or adjusted in a typical programming structure that consists of initialization, followed by a loop for generating statistical samples, and ending with the summation of the overall statistics.
48
Michael Mascagni and Yaohang Li
Thus, to recover an interrupted computation, a Monte Carlo application needs to save only a relatively small amount of information. The necessary information to reconstruct a Monte Carlo computation image at checkpoint time will be the current results based on the estimates obtained so far, the current status and parameters of the random number generators, and other relevant program information like the current iteration number. This allows one to make a smart and quick application checkpoint in most Monte Carlo applications. Using XML [8] to record the checkpointing information, we can make this checkpoint platform-independent. More importantly, compared to a process checkpoint, the application-level checkpoint is much smaller in size and much quicker to generate. Therefore, it should be relatively easy to migrate a Monte Carlo computation from one node to another in a grid system. With the application-level checkpointing and recovery facilities, the typical Monte Carlo application’s programming structure can be amended as described above. However, the implementation of application level checkpointing will somewhat increase the complexity of developing new Monte Carlo grid applications.
4 4.1
Enhancing the Trustworthiness of Grid-Based Monte Carlo Computing Distributed Monte Carlo Partial Result Validation
The correctness and accuracy of grid-based computations are vitally important to an application. In a grid-computing environment, the service providers of the grid are often geographically separated with no central management. Faults may hurt the integrity of a computation. These might include faults arising from the network, system software or node hardware. A node providing CPU cycles might not be trustworthy. A user might provide a system to the grid without the intent of faithfully executing the applications obtained. Experience with SETIhome has shown that users often fake computations and return wrong or inaccurate results. The resources in a grid system are so widely distributed that it appears difficult for a grid-computing system to completely prevent all “bad” nodes from participating in a grid computation. Unfortunately, Monte Carlo applications are very sensitive to each partial result generated from each subtask. An erroneous partial result will most likely lead to the corruption of the whole grid computation and thus render it useless. To enforce the correctness of the computation, many distributed computing or grid systems adapt fault-tolerant methods, like duplicate checking [10] and majority vote [16]. In these approaches, subtasks are duplicated and carried out on different nodes. Erroneous partial results can be found by comparing the partial results of the same subtask executed on different nodes. Duplicated checking requires doubling computations to discover an erroneous partial result. Majority vote requires at least three times more computation to identify an erroneous partial result. Using duplicate checking or majority vote will significantly increase the workload of a grid system.
Computational Infrastructure
49
In the dynamic bag-of-work model as applied to Monte Carlo applications, each subtask works on the same description of the problem but estimates based on different random samples. Since the mean in a Monte Carlo computation is accumulated from many samples, its distribution will be approximately normal based in the Central Limit Theorem. Suppose are the partial results generated from individual nodes on a grid system. The mean of these partial results is
and we can estimate its standard error,
via the following formula
Specifically, the Central Limit Theorem states that should be distributed approximately as a random variable with mean standard deviation and degrees-of-freedom. However, since the number of subtasks, is often large, we may instead approximate the distribution with the normal. Standard normal confidence interval theory states that with 68% confidence that the exact mean is within 1 standard deviation of with 95% confidence within 2 standard deviations, and 99% confidence within 3 standard deviations. This statistical property of Monte Carlo computation can be used to develop an approach for validating the partial results of a large grid-based Monte Carlo computation. Here is the proposed method for distributed Monte Carlo partial result validation. Suppose we are running Monte Carlo subtasks on the grid, and the subtask returns partial result, We anticipate that the are approximately normally distributed with mean, and standard deviation, We expect that about one of the in this group of to lie outside a normal confidence interval with confidence In order to choose a confidence level that permits events we expect to see, statistically, yet flags events as outliers requires us to choose a multiplier, so that we flag events that should only occur once in a group of size The choice of is rather subjective, but implies that in only 1 in 10 runs of size we should expect to find an outlier with confidence With a given choice of one computes the symmetric normal confidence interval based on a confidence of Thus the confidence interval is where is unit normal value such that If
is in this confidence interval, we can consider this
partial result as trustworthy. However, if falls out of the interval, which may happen merely by chance with a very small probability, this particular partial result is suspected. There are two possibilities for a partial result, to fall out of the confidence interval. These are
50
Michael Mascagni and Yaohang Li
1. errors occur during the computation of this subtask, or 2. a rare event with very low probability is captured.
In former case, this partial result is erroneous and should be discarded, whereas in the latter case, we need to take it into consideration. To identify these two cases, we can rerun the particular subtask that generated the suspicious partial result on a trusted node for further validation. This Monte Carlo partial result validation method supplies us with a way to identify suspicious results without running more subtasks. This method assumes that the majority of the nodes in grid system are “good” service providers, which can correctly and faithfully execute their assigned task and transfer the result. If most of the nodes are malicious, this validation method may not be effective. However, experience has shown that the fraction of “bad” nodes in volunteered computation is very small.
4.2
Intermediate Value Checking
Usually, a grid-computing system compensates the service providers to encourage computer owners to supply resources. Many Internet-wide grid-computing projects, such asSETI@home [9], have the experience that some service providers don’t faithfully execute their assigned subtasks. Instead they attempt to provide bogus partial results at a much lower personal computational cost in order to obtain more benefits. Checking whether the assigned subtask from a service provider is faithfully carried out and accurately executed is a critical issue that must be addressed by a grid-computing system. One approach to check the validity of a subtask computation is to validate intermediate values within the computation. Intermediate values are quantities generated within the execution of the subtask. To the node that runs the subtask, these values will be unknown until the subtask is actually executed and reaches a specific point within the program. On the other hand, to the clever application owner, certain intermediate values are either pre-known and secret or are very easy to generate. Therefore, by comparing the intermediate values and the pre-known values, we can control whether the subtask is actually faithfully carried out or not. Monte Carlo applications consume pseudorandom numbers, which are generated deterministically from a pseudorandom number generator. If this pseudorandom number generator has a cheap algorithm for computing arbitrarily within the period, the random numbers are perfect candidates to be these cleverly chosen intermediate values. Thus, we have a very simple strategy to validate a result from subtasks by tracing certain predetermined random numbers in Monte Carlo applications. For example, in a grid Monte Carlo application, we might force each subtask to save the value of the current pseudorandom number after every N (e.g., N = 100,000) pseudorandom numbers are generated. Therefore, we can keep a record of the Nth, 2Nth, ..., random numbers used in the subtask. To validate the actual execution of a subtask on the server side, we can just recompute the Nth, 2Nth, . . . , random numbers applying the specific generator with
Computational Infrastructure
51
the same seed and parameters as used in this subtask. We then simply match them. A mismatch indicates problems during the execution of the task. Also, we can use intermediate values of the computation along with random numbers to create a cryptographic digest of the computation in order to make it even harder to fake a computational result. Given our list of random numbers, or a deterministic way to produce such a list, when those random numbers are computed, we can save some piece of program data current at that time into an array. At the same time we can use that random number to encrypt the saved data and incorporate these encrypted values in a cryptographic digest of the entire computation. At the end of the computation the digest and the saved values are then both returned to the server. The server, through cryptographic exchange, can recover the list of encrypted program data and quickly compute the random numbers used to encrypt them. Thus, the server can decrypt the list and compare it to the “plaintext” versions of the same transmitted from the application. Any discrepancies would flag either an erroneous or faked result. While this technique is certainly not a perfect way to ensure correctness and trustworthiness, a user determined on faking results would have to scrupulously analyze the code to determine the technique being used, and would have to know enough about the mathematics of the random number generator to leap ahead as required. In our estimation, surmounting these difficulties would far surpass the amount of work saved by gaining the ability to pass off faked results as genuine.
5
Conclusions
Monte Carlo applications generically exhibit naturally parallel and computationally intensive characteristics. Moreover, we can easily fit the dynamic bag-of-work model, which works so well for Monte Carlo applications, onto a grid system to implement large-scale grid-based Monte Carlo computing. Furthermore, based on the analysis of grid-based Monte Carlo applications, we may take advantage of the statistical nature of Monte Carlo calculations and the cryptographic nature of random numbers to enhance the performance and trustworthiness of this Monte Carlo grid-computing infrastructure at the application level. We are developing a Grid-Computing Infrastructure for Monte Carlo Applications (GCIMCA) based on the Globus toolkit [5] and the SPRNG library [11], using the techniques described in this paper. The infrastructure software aims to provide grid services to facilitate the development of grid-based Monte Carlo applications and the execution of large-scale Monte Carlo computations in a grid-computing environment. At the same time, we are also trying to execute some real-life large-scale Monte Carlo applications, such as the Monte Carlo simulation of Ligand-Receptor interaction in structured protein systems [17] and the Monte Carlo molecular modeling applications, on our developing grid-computing infrastructure.
52
Michael Mascagni and Yaohang Li
References 1. I. Foster, C. Kesselman, and S. Tueske, The Anatomy of the Grid, International Journal of Supercomputer Applications 15 3 (2001) 1–25. 2. M. Litzkow, M. Livny, and M. Mutka, Condor–A Hunter of Idle Workstations, Proceedings of the 8th International Conference of Distributed Computing Systems, (1998) 104–111. 3. M. Beck, J. Dongarra, G. Fagg, A. Geist, P. Gray, J. Kohl, M. Migliardi, K. Moore, T. Moore, P. Papadopoulous, S. Scott, and V. Sunderam, HARNESS: A Next Generation Distributed Virtual Machine, Journal of Future Generation Computer Systems, 15 5/6 (1999) 571–582. 4. B. O.Christiansen, P. Cappello, M. F. Ionescu, M. O. Neary, K. E. Schauser, and D. Wu, Javelin: Internet-Based Parallel Computing Using Java, Concurrency: Practice and Experience, 9 11 (1997) 1139–1160. 5. I. Foster and C. Kesselman, Globus: A Mmetacomputing Infrastructure Toolkit, International Journal of Supercomputer Applications, 11 2 (1997) 115–128. 6. A. Srinivasan, D. M. Ceperley, and M. Mascagni, Random Number Generators for Parallel Applications, Monte Carlo Methods in Chemical Physics, 105 (1997) 13–36. 7. Entropia website: http://www.entropia.com. 8. XML website: http://www.xml.org. 9. E. Korpela, D. Werthimer, D. Anderson, J. Cobb, and M. Lebofsky, SETI@homeMassively Distributed Computing for SETI, Computing in Science and Engineering, 3 1 (2001) 78–83. 10. C. Aktouf, O. Benkahla, C. Robach, and A. Guran, Basic Concepts & Advances in Fault-Tolerant Computing Design, World Scientific Publishing Company, 1998. 11. M. Mascagni and A. Srinivasan, Algorithm 806: SPRNG: A Scalable Library for Pseudorandom Number Generation, ACM Transactions on Mathematical Software, 26 (2000) 436–461. 12. M. Livny, J. Basney, R. Raman, and T. Tannenbaum, Mechanisms for High Throughput Computing, SPEEDUP Journal, 11 1 (1997) 13. SPRNG website: http://sprng.cs.fsu.edu. 14. Condor website: http://www.cs.wisc.edu/condor. 15. R. Buyya, S. Chapin, and D. DiNucci, Architectural Models for Resource Management in the Grid, In: R. Buyya, M. Baker (Eds.): Grid Computing — GRID 2000, Lecture Notes in Computer Science, 1971, Springer Verlag (2000) 18–35. 16. L. F. G. Sarmenta, Sabotage-Tolerance Mechanisms for Volunteer Computing Systems, Proceedings of ACM/IEEE International Symposium on Cluster Computing and the Grid (CCGrid’01), (2001) 561–572. 17. Y. Li, M. Mascagni, and M. H. Peters, Grid-based Nonequilibrium Multiple-Time Scale Molecular Dynamics/Brownian Dynamics Simulations of Ligand-Receptor Interactions in Structured Protein Systems, Proceedings of the First BioGrid Workshop at the 3rd IEEE/ACM Symposium Cluster Computing and the Grid, (2003) 568–573.
Parallel Solution of Very Large Sparse Systems of Linear Algebraic Equations Zahari Zlatev National Environmental Research Institute Frederiksborgvej 399, P. O. Box 358, DK-4000 Roskilde, Denmark
[email protected] http://www.dmu.dk/AtmosphericEnvironment/staff/zlatev.htm
Abstract. Systems of linear algebraic equations occur very often when large-scale mathematical models are treated. The solution of these systems is as a rule the most time-consuming part of the computational work when large-scale mathematical models are handled on computers. Therefore, it is important to be able to solve such problems efficiently. It is assumed that the systems which must be solved many times during the treatment of the models, are (i) very large (containing more than equations) and (ii) general sparse. Moreover, it is also assumed that parallel computers with shared memory are available. An efficient algorithm for the solution of such large systems under the above assumptions is described. Numerical examples are given to demonstrate the ability of the algorithm to handle very large systems of linear algebraic equations. The algorithm can be applied in the treatment of some large-scale air pollution models without using splitting procedures.
1 Motivation Large-scale air pollution models can successfully be used to resolve many tasks connected with the selection of appropriate measures that could be applied in the efforts to avoid possible damages caused by harmful pollutants. Some of these tasks are legislated in official documents of the Parliament of the European Union (see, for example, [3]). Most of the large-scale air pollution models are described by systems of partial differential equations (the number of equations being equal to the number of chemical species studied by the model); see [1, 8, 18, 19]. The direct application of either finite elements or finite differences during the numerical treatment of such models leads to the solution of very large systems of linear algebraic equations. Assume that the numbers of grid points along the coordinate axes is and respectively. Assume also that the number of chemical species is Several systems containing equations have to be treated at each time-step. If and then N = 80640000. Not only can N be very large (as shown in the above example), but also many time-steps are to be carried out. The number of time-steps is I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 53–64, 2004. © Springer-Verlag Berlin Heidelberg 2004
54
Zahari Zlatev
normally in the range from to Thus the computational tasks are very difficult both because of the computing time requirements and because of the storage needed. Most of the difficulties can be overcome by applying some splitting procedure. The large problem is split into a sequence of small problems, which can be treated on the available computers. The splitting procedures simplify very much the computational tasks. However, they are also causing splitting errors. It is difficult to control the size of these errors. This is why it is desirable to avoid (if possible) the application of splitting procedures. This can be done only if new and very efficient methods for the solution of very large systems of linear algebraic equations are developed and used. Some preliminary results obtained in the efforts to develop such methods will be discussed in this paper. The results are rather general and it will be possible to apply the new methods for the treatment of very large systems of linear algebraic equations also in some other fields of science and engineering.
2
Major Assumptions
Since the direct discretization of the systems of partial differential equations arising in air pollution modelling leads to the solution of large systems of linear algebraic equations, we shall assume that systems of the type
are to be solved. Matrix A is (i) very large (in this paper “very large” means that (ii) non-symmetric and (iii) very badly scaled, because some of the non-zero elements of A depend on concentrations of the chemical species, which vary in the range from to because these quantities are measured by using the number of molecules in cubic centimeter. Therefore, the direct use of iterative methods is not advisable. Matrix A is in general banded. However, when A is very large, then the bandwidth is also large and the use of direct methods for banded matrices is prohibitive (in spite of the fact that very efficient software for banded matrices is available; see [2]). Indeed, the bandwidth is about 168 000 when an air pollution model involving 35 species is discretized on a (480 × 480 × 10) grid. Matrix A is sparse. However, it is also difficult to exploit the sparsity directly, because of the great number of fill-ins which will be produced during the factorization even if a good pivotal strategy (as, for example, some of the pivotal strategies described in [15] and [17]) is used in the efforts to preserve better the original sparsity pattern of matrix A. This short analysis indicates that it is worthwhile to try the following approach. Apply some good method for factorization of sparse matrices combined with dropping of all non-zero elements which are smaller than a given droptolerance parameter multiplied by the maximal in absolute value element in the row to which the candidate for dropping belongs. An approximate LUfactorization will be calculated in this way (the method is fully described in [7, 16, 17]). The approximate LU-factorization of matrix A can then be used as a
Parallel Solution of Very Large Sparse Systems
55
preconditioner for some iterative method (as, for example, some of the methods discussed in [4, 5, 9–14, 16, 17]). The calculation of the preconditioner remains the most time-consuming part in the solution of (1) also when dropping is used during the factorization in order to preserve better the sparsity pattern of matrix A. Therefore, it is important to develop a parallel algorithm for the computation of the preconditioner in which the ideas sketched in the previous paragraph can efficiently be applied. The development of such a parallel algorithm is discussed in this paper.
3
Description of the Algorithm
The algorithm used to obtain an approximate LU-factorization of matrix A (that can be used as a preconditioner in the solution of the system of linear algebraic equations by some iterative method), consists of the following five steps.
3.1
Step 1 – Obtaining an Upper Block-Triangular Matrix
A simple reordering is performed in an attempt to push the non-zeros in the lower triangular part of matrix A as close to the main diagonal as possible. First the columns of A are ordered. Let be the number of non-zeros in column Column interchanges are applied to obtain a permutation of A such that
After that the rows are ordered in a similar way. Assume that is the number of non-zeros in row Row interchanges are applied to obtain a permutation (of the matrix obtained after the column interchanges) such that
The result of the column and row interchanges performed as described above is a matrix in which many non-zero elements are moved to the lower left-hand corner of the matrix and, thus, the matrix obtained after these interchanges can be considered as an upper block-triangular matrix. Some other reordering algorithms can also be used. For example, the algorithm LORA from [6] can be applied. The advantage of the algorithm sketched above is that it is very cheap; its computational cost is O ( N Z ) , where NZ is the number of non-zeros in A. The computational cost of LORA is O(NZ log N) which is a considerable increase when N is very large. On the other hand, a more expensive computationally, but also more advanced, algorithm (such as LORA) may lead to a better reordering of the matrix and, thus, to a better performance in the remaining steps of the algorithm.
3.2
Step 2 – Dividing the Reordered Matrix into Block-Rows
The reordered, as described in the previous sub-section, matrix can be divided into several block-rows the diagonal blocks of which are rectangular matrices.
56
Zahari Zlatev
Denote the reordered matrix by B (i.e. B = PAQ, where P and Q the permutation matrices that are induced by the row and column interchanges). It is desirable to divide matrix B into block-rows which can be processed in parallel. A simple example for such a division with is given in Fig. 1. While
Fig. 1. Division of matrix B when the special case is used both in Fig. 1 and in the other figures (in order to facilitate the understanding of the ideas), all remarks given below are for the general case (i.e. for an arbitrary Note that all the blocks located under the diagonal blocks where contain only zero elements. This is essential in the attempts to achieve parallel computations when the different block-rows are processed. It should also be noted that the diagonal blocks are rectangular. The number of rows is greater than or equal to the number of columns for the first diagonal blocks. The situation is changed for the last diagonal block i.e. the number of columns is less than or equal to the number of rows. The computations in each block-row are independent of the computations in the remaining block-rows. This means that there are parallel tasks related to the computations in the next step (Step 3). It is important to emphasize here that an attempt to produce block-rows with approximately equal numbers of rows is carried out in order to achieve a better loading balance. The fact that rectangular diagonal blocks with are allowed facilitates this task. However, a perfect loading balance will in general not be achieved even if the numbers of the rows in all block-rows are the same (because the numbers of non-zero elements may vary from one block-row to another). Therefore, (where is the number of processors which are to be assigned to the job) is used in the experiments in Section 4 in order to increase the possibility of achieving a better loading balance.
3.3
Step 3 – Factorizing the Block-Rows
Gaussian transformations are carried out in order to produce zero elements under the main diagonal of each diagonal block These calculations can be performed in parallel. The result of performing Gaussian transformations for the matrix given in Fig. 1. is shown in Fig. 2. While the special case is used in Fig. 2 in order to facilitate the understanding of the ideas, the remarks given below are for the general case (i.e. for an arbitrary It is seen (compare Fig. 2 with Fig. 1) that each of the first block-rows of matrix B is divided into two block-rows. Consider the diagonal row-block
Parallel Solution of Very Large Sparse Systems
57
Assume that this diagonal block contains rows and columns. Then, as a results of the Gaussian transformations in row-block the diagonal block is split to an upper triangular matrix and a zero matrix of order Consider block and assume that it contains rows and columns with The results of performing Gaussian transformation in row-block will lead to an upper triangular matrix
Fig. 2. The result of the factorization of the block-rows of the matrix given in Fig. 1. The dimensions of the remaining blocks in Fig. 2 are listed below:
and and
with with with with
The computations related to the blocks and as well as the computations related to the blocks are finished at the end of Step 3. The further computations are carried out in the remaining blocks. Therefore, it is desirable that these blocks are small (the reordering procedure from Step 1 is performed in attempt to reduce as much as possible the size of these blocks).
3.4
Step 4 – Producing Zeros in Blocks
The computations should be continued by producing zeros in the blocks with and It is appropriate to carry out the calculations by “diagonals”. During the calculations along the “first diagonal”, the pivotal elements of block are used to produce non-zeros in the pivotal elements of are used to produce non-zeros in and so on. The number of tasks along the “first diagonal” is and these tasks can be performed in parallel. When the calculations along the “first diagonal” are finished, the calculations along the “second diagonal” can be started. The pivotal elements of block are used to produce non-zeros in the pivotal elements
58
Zahari Zlatev
of are used to produce non-zeros in and so on. The number of parallel tasks is now The calculations are carried out in a similar manner along the “remaining diagonals”. It is important to emphasize that the number of parallel tasks is reduced by one when the calculations along the next diagonal are to be carried out. This is not a big problem when the blocks are small. In order to improve the performance it is worthwhile to choose the number of block-rows considerably larger than the number of processors that will be used (this is another reason for choosing the first one was discussed in the end of When the calculations in Step 4 are completed, the matrix shown in Fig. 2 will be transformed so that the structure given in Fig. 3 will be obtained. As
Fig. 3. The result of producing zeros in the blocks
of the matrix given in Fig. 2.
mentioned above, the number of parallel tasks is gradually reduced when the calculations are carried out by “diagonals”. Parallel computations can also be achieved in the following way. Consider any block and The production of zeros in any row of this block (by using the pivotal elements of block is a parallel task. This means that the computations in each row of block can be carried in parallel. The parallel tasks are not so large as in the previous case (where zeros in whole blocks are produced). However, the tasks are still sufficiently large when matrix A is very large. Moreover the possibility of achieving a better loading balance is in general increased, because the number of parallel tasks is very large. The second approach is used to obtain the results shown in Section 4.
3.5
Step 5 – Reordering and Finishing the Computations
The matrix, which structure is shown in Fig. 3, can be reordered (by row permutations) to the form given in Fig. 4. It should be emphasized here that it is not necessary to perform the actual row interchanges, it is quite sufficient to renumber the rows. The large block formed by the blocks and the appropriate zero blocks is an upper triangular matrix. The order of this large block is
Parallel Solution of Very Large Sparse Systems
59
Fig. 4. The result of reordering the blocks in the matrix given in Fig. 3.
The column-block formed by the blocks is a rectangular matrix. It is clear that the first dimension of this block is the same as that of the square matrix formed by the blocks which is given in (4). The second dimension of this matrix is Simple calculations show that the matrix formed by the blocks is square. Its order is Furthermore, this matrix is normally not very large. Furthermore, it should be expected that this matrix is relatively dense, because a lot of computations involving the blocks have already been carried out in the previous steps. Therefore, it is worthwhile to switch to dense matrix technique in Step 5 and to apply subroutines from, for example, [2].
4
Numerical Results
The numerical algorithm discussed in the previous sections has been tested by using different matrices produced by the matrix generators described in [17]. There are several advantages when such generators are used: (i) one can produce arbitrarily many matrices (while the number of test-matrices in the available data-bases is limited), (ii) one can vary the size of the matrices, (iii) one can vary the sparsity pattern, (iv) one can vary the number of non-zero elements and (v) one can vary the condition number of the matrices tested. A few experiments will be given in this section to demonstrate the performance of the algorithm in the following situations. The behaviour of the numerical algorithm when the number of block-rows is varied for very large matrices is different from the corresponding behaviour for small matrices. It should be noted that matrices of order are called here small. Such matrices are considered as large in many papers on the solution of sparse systems of linear algebraic equations. The parallel properties of the algorithm. The conclusions drawn in this section and in the next one are based on results obtained in much more experiments, than those used in the preparation of Table 1–Table 3.
60
Zahari Zlatev
Some attempts to simulate matrices which arise after the discretization of air pollution models were made: (i) the average number of non-zero elements per row was no less than 10, (ii) the non-zero elements were widely spread and (iii) the matrices chosen for experiments were badly scaled. All experiments, which are reported here were carried out on SUN computers at DCSC (the Danish Center for Scientific Computing). Some experiments on IBM computers at the University of Aarhus gave quite similar results.
4.1
Varying the Number of Block-Rows for Small Matrices
Systems containing linear algebraic equations are used in this sub-section. The number of the non-zero elements is 1 000 110 (i.e. the average number of non-zero elements per row is about 10). The number of block-rows is varied in the range from 8 to 512. The results are shown in Table 1 (the computing times are given in seconds). Computing times measured in seconds are also used in the remaining tables (Table 2 and Table 3). The notation that is applied in all the tables can be explained as follows: ORD time is the sum of the time needed for the reordering of the matrix (described in Step 1; see Section 3) and the time needed to divide the matrix into block-rows (Step 2). FACT time is the time needed to obtain the approximate LU-factorization, which is used as a preconditioner in the solution part (the preconditioner is calculated in Step 3 - Step 5 in Section 3). SOLV time is the time needed to obtain a starting approximation for the iterative procedure (the back substitution step) and, after that, to carry out iterations until a prescribed accuracy (an accuracy requirement of was actually used in all experiments) is achieved by the preconditioned iterative method chosen (the modified Orthomin method from [17] was actually used in all runs). TOTAL time is the total computing time spent in the run. Several conclusions can be drawn by studying the results in Table 1 (similar conclusions can be drawn by using the results of many other runs which were performed). 1. The ORD times and the SOLV times do not depend too much on the parameter (on the number of block-rows). 2. The FACT times (and, therefore, also the TOTAL times) depend on the choice of For this matrix, the best result is obtained with the choice 3. When the FACT time is the best one (i.e. when the SOLV time are comparable with the FACT time.
4.2
the ORD time and
Varying the Number of Block-Rows for Large Matrices
The number of equations in the system of linear algebraic equations is increased 40 times, i.e. from 100000 to 4000000. The number of non-zero elements in the
Parallel Solution of Very Large Sparse Systems
61
matrix is increased from 1000110 to 40000110, i.e. the average number of nonzero elements per row is again about 10. This very large system is solved by using different values of Results are shown in Table 2. Similar conclusions, as those drawn in the previous sub-section, can be drawn by studying the results in Table 2 (it should again be emphasized that the results obtained in many other runs show the same trends).
1. It is clearly seen that both the ORD times and the SOLV times practically do not depend on the parameter (on the number of block-rows). 2. The FACT times (and, therefore, also the TOTAL times) depend on the choice of For this matrix, the best result is obtained with the choice 3. The FACT time is the largest part of the TOTAL time also with the best choice of (i.e. when
4.3
Parallel Runs
The system used in the previous section (i.e. the system with 4000000 equations and with 40000110 non-zero elements in its coefficient matrix) was also run on four processors. Results are shown in Table 3. Not only are the computing times
62
Zahari Zlatev
shown in Table 3, but also the speed-up factors are given there (in brackets). The speed-up factors are calculated by taken the ratios of the computing times obtained in the runs on one processor (these times are given in Table 2) and the computing times obtained on four processors.
The results that are given in Table 3 (as well as the results obtained in many other runs) indicate that the following conclusions can be drawn.
1. The ORD times and the SOLV times remain practically the same when parameter (i.e. the number of block-rows) is varied. 2. The FACT times (and, therefore, also the TOTAL times) depend again on the choice of For this matrix, the best result is obtained with the choice 3. The ORD time and the SOLV time remain considerably smaller than the FACT time also when the FACT time is the smallest one (i.e. for the best choice of 4. The speed up factors for the ORD times remain practically the same for all values of These factors are also very small. This shows that the ordering of matrix and the division to row-block are two operations which are in fact run sequentially. 5. Also the speed up factors for the SOLV times remain practically the same for all values of These factors are greater than the corresponding factors for the ORD times, but still not very large. This shows that the back substitution and the iterations during the preconditioned modified Orthomin algorithm do not parallelize very well. 6. The speed up factors for the FACT times tend to decrease when the value of is increased (i.e. when the matrix is divided into more block-rows). These factors are greater than the corresponding factors for the ORD times and the SOLV times. The efficiency achieved (i.e. the speed-up factor divided by the number of processors and multiplied by 100) is very often over 90% (about 94% for the best choice
Parallel Solution of Very Large Sparse Systems
63
7. Also the speed up factors for the TOTAL times tend to decrease when the value of is increased (i.e. when the matrix is divided into more block-rows). These factors are smaller than the corresponding factors for the FACT times, but considerably greater than the correspond factors for the ORD times and the SOLV times. The efficiency achieved is often over 90%. However, for the important case where the best choice is made the efficiency is considerably smaller; about 71%.
5
Conclusions
An algorithm for the solution of very large sparse systems of linear algebraic equations was discussed and tested. Several conclusions about the performance of the algorithm (including the performance on a SUN parallel computer) were drawn in the previous section. Some additional improvements are needed in order to apply this algorithm in the treatment of large-scale mathematical models for studying air pollution phenomena. It is necessary to improve the performance of the following parts of the algorithm: the reordering part, the division into block-rows the preconditioned iterative methods. It might be worthwhile to try to exploit some special properties of the systems of linear algebraic equations arising in large scale air pollution models in order to find a simpler and more efficient preconditioner.
Acknowledgements Two grants (CPU-1002-27 and CPU-1101-17) from the Danish Centre for Scientific Computing (DCSC) gave us access to the IBM computers at the Århus University and the SUN computers at the Technical University of Denmark. The members of the staff of DCSC helped us to resolve some difficult problems related to the efficient exploitation of the grids of IBM and SUN computers.
References l. Alexandrov, V., Sameh, A., Siddique, Y., and Zlatev, Z.: Numerical integration of chemical ODE problems arising in air pollution models. Environmental Modelling and Assessment, Vol. 2 (1997), 365-377. 2. Anderson, E., Bai, Z., Bischof, C., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Ostrouchov, S., and Sorenson, D., LAPACK: Users’ guide, SIAM, Philadelphia, 1992. 3. Directive 2002/3EC of the European Parliament and of the Council of 12 February 2002 relating to ozone in ambient air. Official Journal of the European Communities, Vol. L 67 (2002), pp. 14-30.
64
Zahari Zlatev
4. Eirola, T. and Nevanlinna, O.: Accelerating with rank-one updates. Lin. Alg. Appl., Vol. 121 (1989), pp. 511-520. 5. Freund, R. W.: A transpose-free quasi-minimal residual algorithm for nonHermitian linear systems. SIAM J. Sci. Stat. Comput., Vol. 14 (1993), pp. 470-482. 6. Gallivan, K., Hansen, P. C., Ostromsky, Tz., and Zlatev, Z.: A locally optimized reordering algorithm and its application to a parallel sparse linear system solver. Computing, Vol. 54 (1995), pp. 39-67. 7. Gallivan, K. A., Sameh, A. H., and Zlatev, Z.: A parallel hybrid sparse linear system solver. Computing Systems in Engineering, Vol. 1 (1990), pp. 183-195. 8. Havasi, A. and Zlatev, Z.: Trends of Hungarian air pollution levels on a long timescale. Atmospheric Environment, Vol 36 (2002), 4145-4156. 9. Saad, Y. and Schultz, M. H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., Vol. 7 (1986), pp. 856-869. 10. Sonneveld, P.: CGS, a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., Vol. 10 (1989), pp. 36-52. 11. Vinsome, P.: ORTHOMIN, an iterative method for solving sparse sets of simultaneous linear equations. Proc. Fourth Sympos. on Reservoir Simulation, Society of Petr. Eng. of AIME (1976) 12. Vorst, H. A. van der: BI-CGSTAB: A fast and smoothly converging variant of BICG for the solution of nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., Vol. 13 (1992), pp. 631-644. 13. Vuik, C. and Vorst, H. A. van der: A comparison of some GMRES-like methods. Lin. Alg. Appl., Vol. 160 (1992), pp. 131-160. 14. Yang, U. M. and K. Gallivan, K.: A Family of Preconditioned Iterative Solvers for Sparse Linear Systems. Applied Numerical Mathematics, Vol. 30 (1999), pp. 155-173. 15. Zlatev, Z.: On some pivotal strategies in Gaussian elimination by sparse technique. SIAM J. Numer. Anal., Vol. 17 (1980), pp. 18-30. 16. Zlatev, Z.: Use of iterative refinement in the solution of sparse linear systems. SIAM J. Numer. Anal, Vol. 19 (1982), pp. 381-399. 17. Zlatev, Z.: Computational methods for general sparse matrices. Kluwer Academic Publishers, Dordrecht-Boston-London (1991) 18. Zlatev, Z.: Computer treatment of large air pollution models. Kluwer Academic Publishers, Dordrecht-Boston-London (1995) 19. Zlatev, Z., Dimov, I. Ostromsky, Tz. Geernaert, G., Tzvetanov, I. and BastrupBirk, A.: Calculating losses of crops in Denmark caused by high ozone levels. Environmental Modelling and Assessment, Vol. 6 (2001), 35-55.
Part II
Recent Achievements in Preconditioning
This page intentionally left blank
Solution of Robust Linear Regression Problems by Krylov Subspace Methods Venansius Baryamureeba Institute of Computer Science, Makerere University, P.O. Box 7062, Uganda
[email protected],
[email protected] http://www.ii.uib.no/~barya, http://www.makerere.ac.ug/ics
Abstract. Several methods have been suggested for obtaining the estimates of the solution vector for the robust linear regression model using an iteratively reweighted criterion based on weighting functions, which tend to diminish the influence of outliers. We consider a combination of Newton method (or iteratively reweighted least squares method) with a Krylov subspace method. We show that we can avoid preconditioning with A preconditioner by merely transforming the sequence of linear systems to be solved. Appropriate sequences of linear systems for sparse and dense matrices are given. By employing efficient sparse QR factorization methods, we show that it is possible to solve efficiently large sparse linear systems with an unpreconditioned Krylov subspace method.
1 Introduction Consider the standard linear regression model where is a vector of observations, is the data or design matrix of rank is the vector of unknown parameters, and is the unknown vector of measurement errors. Let the residual vector be given by
where
denotes the
row of A. In this paper we need to solve
where is a given weighting function, and is the scale factor (connected to the data). The functions we are interested in are those that are less sensitive to large absolute residuals. The functions we will consider are twice continuously differentiable almost everywhere with Baryamureeba [1] recently suggested the Barya function
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 67–75, 2004. © Springer-Verlag Berlin Heidelberg 2004
68
where
Venansius Baryamureeba
and
are problem-dependent parameters. Huber [8] sug-
gested the Huber function
where
dependent parameter. The Logistic function is proposed the Fair function posed the function
is a problemFair [6]
Huber [8] also prowhich is given the name Talwar
[7]. The Barya function is a linear combination of the Talwar and LSQ weighting functions. In particular, let and denote the Barya, LSQ and Talwar functions respectively. Then The parameter must be chosen to satisfy two conflicting goals: (i) to guarantee a positive definite weight matrix and (ii) to minimize the influence of large absolute residuals. Let where and are the piece-wise sub-domains over which the Barya function is defined. It is clear that the Barya function is strictly convex in any sub-domain It is also continuous over the entire domain Thus we refer to the Barya function as a piece-wise convex function.
1.1
Problem Formulation
For a known
the first order conditions for (2) are:
where is an with elements Newton’s Method Approach. One of the methods for solving the nonlinear problem (4) is Newton’s method. Given an initial guess we compute a sequence of search directions and iterates as follows:
where is a diagonal matrix with diagonal elements and is a step-length parameter. We remark that when is positive definite, can be formulated as a weighted least squares problem Iteratively Reweighted Least Square (IRLS) Method. The IRLS linear system corresponding to (4) differs from the Newton equation (5) in the weight matrix G. For IRLS method Approach. On the other hand, Scales et al. [11] suggest another formulation of IRLS algorithm based on the optimization problem: By setting the derivative of this problem to zero, we get the generalized normal equation
Solution of Robust Linear Regression Problems by Krylov Subspace Methods
for
69
is defined in (1) and G is the diagonal matrix defined by
where we choose Active Set Method. Let
sufficiently small. be defined by
where is given by (1). For the Talwar function, the influence of large absolute residuals is reduced drastically from both the right hand side and the coefficient matrix since and for all Observe that IRLS method and Newton method are equivalent for this function since for all Hence, Newton’s method based on the Talwar function does not require a line-search when a direct method is used. Do we need a line-search when we carry out inexact computation? Interestingly, it has been shown [1] that the Talwar approach is an active set method (c.f. simplex method) where elements of corresponding to small absolute residuals form the basic variables and those corresponding to large absolute residuals form the nonbasic variables.
2
IRLS Method Based Comparative Study
The Barya and Huber Functions. Recall that IRLS method based on the Huber function is globally convergent [9] and no line-search is necessary with exact computation (direct method). For the Barya function, IRLS method is equivalent to Newton’s method. Here we compare the Barya and Huber functions at a given iteration of IRLS method. The purpose of this comparison is to determine the choice of the parameter in (3). Let defined by (8) be generated by IRLS method based on the Huber function at a given iteration. Let be such that
Let
be a diagonal matrix with diagonal elements
70
Then
Venansius Baryamureeba
Let
be diagonal matrices with diagonal elements
Then H = KG. Thus we have:
Let
Then the weighted problems corresponding to I and II are:
since G and K are positive definite diagonal matrices. To be able to reduce the influence of large absolute residuals it is clear from comparing (9) and (10), (11) and (12) that for all Thus we suggest to choose in (3) by
A natural choice of is 1. In this paper we set It is clear from (13) and (14) that the IRLS method based on the Barya function reduces the influence of large absolute residuals more than IRLS method based on the Huber function. Since IRLS method and Newton method are equivalent for the Barya function, IRLS method based on the Barya function exhibits quadratic convergence [5] near the solution. This is supported by the numerical tests in [3]. Recall that the Barya function is a linear combination of the Talwar and least squares functions. The parameter must be kept small (usually because of the need to minimize the bound on the spectral condition number of the (preconditioned) coefficient matrix and also to keep iterates away from the boundary of the feasible region. A line-search procedure is not necessary for IRLS method (or Newtons’s method) based on the Barya function. For inexact Newton method based on the Barya function, if the search directions are computed to low accuracy then we need to do a line-search so that is decreased substantially. Since the Barya function is a piece-wise convex function the line-search based on decreasing in (2) will always lead to a local minimum in the desired region
Solution of Robust Linear Regression Problems by Krylov Subspace Methods
71
Block Form of the Barya Function. Matrix K in (14) is a scale matrix. The IRLS method based on the Barya function may be interpreted as applying IRLS method based on the Huber function to the scaled problem where the scale matrix is carefully chosen to minimize the influence of large absolute residuals. Results (12) and (14) show that it is possible for weight matrix H (corresponding to the Barya function) to have at least two distinct diagonal elements. We only need to ensure that for all Let be given by (8) and Define for and Then we define the block form of the Barya weighting function by
In practice we require that the weight matrix H in (12) to have few distinct diagonal elements. Thus we choose in (15) to be small. On the other hand it is easy to show that
Since
we can see that Thus Newton’s method and IRLS method are equivalent for this block form of the Barya function.
3
Solving the Sequence of Linear Systems
Any robust linear regression problem can be posed as a sequence of weighted linear systems and solved by either a direct method or iterative method or both.
3.1
Iterative Method Approach
Lemma 1. Let Let a diagonal matrix with each diagonal element equal to 1 or for Then For the proof see [1]. Unpreconditioned Linear System. Consider equation (5). Let
be
72
where
Let
Venansius Baryamureeba
is such that (Q consists of orthogonal columns) and is a nonsingular upper triangular matrix. Given (16), (5) simplifies to
Instead of solving (17) we solve the symmetric system
and then compute by Preconditioning with Matrix. Here we consider preconditioner [4]. Considering system (5), the preconditioned linear system can be written as
Next we consider the preconditioned matrix factorization (16)
Given the QR
By similarity transformations the matrices and are similar and thus We note that if G is a positive definite diagonal matrix then see [1]. Thus from a theoretical point of view it is evident that preconditioning with is unnecessary. In other words, instead of solving (5) with preconditioner one should solve (18) with an unpreconditioned iterative method, for instance unpreconditioned conjugate gradient method. The major draw back of solving (18) is that for a sparse matrix A, Q is often nearly full (dense). The general results on the upper bound of the spectral condition of the preconditioned matrix are given in [3] and [4]. Similar results for can be derived by simply observing that Least Squares Approach. When G is positive definite then (18) corresponds to the weighted least squares problem
A good and generally available reliable solver like LSQR can applied to (21). Linear systems arising from Newton method based on the Barya, Fair, and Logistic functions can be formulated in the form (21). Recall that for the Talwar function and for all Thus, for the Talwar function, (18) can be formulated as a least squares problem of the form (21) if and only if is positive definite, whereas it is not possible to formulate (5) in the form (21) for the Huber function since and for all Iteratively Reweighted Least Squares (IRLS) Method. Since G is positive definite for IRLS method, (5) & (6) can be formulated in the form (21). Related Work. Let G be a diagonal weight matrix and AQ be given by (16). Scales et al. [11] used Conjugate Gradient (CG) method to solve linear systems of the form
Solution of Robust Linear Regression Problems by Krylov Subspace Methods
73
O’Leary [10] used CG method to solve linear systems of the form
In [3] and [4] preconditioned CG method was used to solve linear systems of the form (22) with as preconditioner. In [4] and [10], a line-search procedure was used to determine the step-length when Newton’s method based on the Talwar function was used. The following observations are now in order:
1. For a dense data matrix A, if unpreconditioned CG method is used, it should be used on system (23) instead of system (22). When the weight matrix G is positive definite, the alternative approach is to solve the corresponding weighted least squares problem. Furthermore, for a dense data matrix A , instead of solving linear systems of the form (22) with as preconditioner, we suggest solving linear systems of the form (23) with unpreconditioned CG method. 2. For sparse data matrix A, we note that the factor Q may not be as sparse as A. Thus there is need to compute the density For an matrix X we define the density At the beginning of the iteration process we need to compute the densities of A, Q and R using (24). We then estimate the cost per iteration for solving (23) with unpreconditioned CG method and the cost for solving (22) with preconditioner. We use these relative costs when determining the linear system to be solved. Alternatively, we can compute the cost of one iteration for (22) with preconditioner and for (23) with unpreconditioned iterative method and then continue with the linear system that is less costly. We may add that with emerging efficient sparse QR factorization methods, the choice between solving (23) with unpreconditioned CG method and (22) with preconditioned CG method depends heavily on numerical results other than theoretical results. 3. Newton’s method based on the Talwar function should be used without linesearch and the solution to the linear system should be computed to high accuracy. We remark that for a positive definite weight matrix G solving the least squares problem corresponding to (23) with for example LSQR may be worthwhile.
3.2
Mixed Method Approach
We consider solving the sequence of linear systems by combining a direct method and an iterative method in some manner, see [2]. The idea is to utilize the factorization from the previous iteration in constructing the preconditioner for the current linear system. There are situations when preconditioning is necessary. For instance when is very large for G positive definite or when is very large for G positive semi-definite. For a discussion on low-rank correction preconditioners see[1].
74
Venansius Baryamureeba
4
Numerical Comparison
The data matrices are sparse random matrices as described in Table 1. We use the matlab sparse function to compute the QR factorization A = QR. We observe that where is a nonsingular upper triangular matrix. Table 1 shows that for A sparse, Q may be either sparse or dense. Since the problems we are interested in are large and sparse, iterative methods may be efficient if we solve the sparse linear system. Thus, it is necessary that we compute the densities for A and Q at the beginning of the iterative process. Otherwise, we need to compute the cost of one iteration for the linear system in A and also for the linear system in Q and then compare the costs.
5
Concluding Remarks
(Un)preconditioned Krylov subspace methods have been combined with Inexact Newton Method to solve robust linear regression problems. This paper sheds more light on when a line-search with Inexact Newton Method is (not) necessary. We have suggested new approaches that can be used to efficiently solve robust linear regression problems.
References 1. Baryamureeba, V.: The impact of equal-weighting of both low-confidence and highconfidence observations on robust linear regression computation. BIT. 41 (2001) 847–855. 2. Baryamureeba, V.: Solution of large-scale weighted least squares problems. Numerical Linear Algebra with Applications, 9 (2002) 93–106. 3. Baryamureeba, V.: Solution of robust linear regression problems by preconditioned conjugate gradient type methods. Technical Report No. 187, Department of Informatics, University of Bergen, 5020 Bergen, Norway, March 22, 2000. 4. Baryamureeba, V., and Steihaug, T: On the properties of preconditioners for robust linear regression. Technical Report No. 184, March 2000, Department of Informatics, University of Bergen, Norway. Journal of Opt. Theory and Appl. (to appear) 5. Dembo, R.S., Eisenstat, S.C., and Steihaug, T.: Inexact Newton methods. SIAM Journal of Numerical Analysis. 19 (1982) 400–407. 6. Fair, R.C.: On the robust estimation of econometric models. Ann. Econ. Social Measurement. 3 (1974) 667–678.
Solution of Robust Linear Regression Problems by Krylov Subspace Methods
75
7. Hinich, M.J., and Talwar, P.P.: A simple method for robust regression. J. Amer. Statist. Assoc. 70 (1975) 113–119. 8. Huber, P.J.: Robust estimation of a location parameter. Annals Math. Statist. 35 (1964) 73 –101. 9. Huber, P.J.: Robust statistics. J. Wiley, New York. (1981). 10. O’Leary, D.P.: Robust regression computation using iteratively reweighted least squares. SIAM Journal of Matrix Analysis and Applications. 11 (1990) 466–480. 11. Scales, J.A., Gersztenkorn, A., and Treitel, S.: Fast solution of large, sparse, linear systems: Application to seismic travel time tomography. Journal of Computation Physics. 75 (1988) 314–333.
Two-Level Preconditioning of Crouzeix-Raviart Anisotropic FEM Systems Gergana Bencheva, Ivan Georgiev, and Svetozar Margenov Central Laboratory for Parallel Processing, Bulgarian Academy of Sciences Acad. G. Bontchev, Bl. 25A, 1113 Sofia, Bulgaria {gery,john,margenov}@parallel.bas.bg
Abstract. The studied large-scale linear problems arise from CrouzeixRaviart non-conforming FEM approximation of second order elliptic boundary value problems. A two-level preconditioner for the case of coefficient anisotropy is analyzed. A special attention is given to the potential of the method for a parallel implementation. Keywords: Non-conforming FEM, preconditioning, parallel algorithms.
AMS Subject Classifications: 65F10, 65N30.
1 Introduction We consider the elliptic boundary value problem
where
is a convex polygonal domain in is a given function in is a symmetric and uniformly positive definite matrix in is the outward unit vector normal to the boundary and We assume that the entries are piece-wise smooth functions on In the paper we use the terminology of the flow in porous media and we refer to as a pressure and as a velocity vector. The problem (1) can be discretized by the finite volume method, the Galerkin finite element method (FEM), conforming or non-conforming, or the mixed FEM. Each of these methods has its advantages and disadvantages when the problem (1) is used in a particular application. In [2] Arnold and Brezzi have demonstrated that after the elimination of the velocity and pressure unknowns the obtained Schur system for the Lagrange multipliers is equivalent to a discretization of (1) by Galerkin method using linear non-conforming finite elements. Namely, in [2] it is shown that the lowest-order Raviart-Thomas mixed finite element approximations are equivalent to the usual Crouzeix-Raviart non-conforming linear finite element approximations when the non-conforming space is augmented with cubic bubbles. Further, such a relationship between the mixed and nonconforming finite element methods has been studied and simplified for various I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 76–84, 2004. © Springer-Verlag Berlin Heidelberg 2004
Two-Level Preconditioning of Crouzeix-Raviart Anisotropic FEM Systems
77
finite element spaces (see, e.g. [1,5]). The work in this direction resulted also in construction of efficient iterative methods for solving mixed FE systems (see, e.g., [6–8]). Furthermore, the stiffness matrix has a regular sparsity structure such that in each row the number of non-zero entries is at most five. This paper provides a continuation of the study recently published in [12]. Here, the sparsity pattern of the approximation algorithm is changed to handle the case of coefficient anisotropy. The study is also related to the parallel techniques from [4, 10], where MIC(0) preconditioning of linear conforming and rotated bilinear non-conforming finite elements is considered. The rest of the paper is organized as follows. In Sections 2 and 3 we introduce the finite element approximation and the two-level algorithm. In Section 4 we propose a locally optimized anisotropic sparse approximation B of the Schur complement S. The condition number is uniformly estimated. Next, in Section 5 we derive expression for the execution time on a multiprocessor computer system which shows a good parallel efficiency for large scale problems. Numerical tests of the PCG convergence rate are presented in Section 6 where the system size and the anisotropy ratio are varied. At the end, short concluding remarks are given.
2
Finite Element Discretization
The domain is discretized using triangular elements. The partition is denoted by and is assumed to be quasi-uniform with a characteristic mesh-size It is aligned with the discontinuities of the coefficient matrix, so that over each element the function is smooth. Further, we assume that is generated by first partitioning into quadrilaterals Q and then splitting each quadrilateral into two triangles by one of its diagonals, see Figure 1. To simplify our considerations we assume that the splitting into quadrilaterals is topologically equivalent to a square mesh. We approximate the weak formulation of (1) using Crouzeix-Raviart non-conforming linear triangular finite elements. The FEM space consists of piece wise linear functions over determined by their values in the midpoints of the edges. The nodal basis functions of have a support on not more than two neighboring triangles where the corresponding node is the midpoint of their common side. Then the finite element formulation is: find satisfying
Here is defined as the integral averaged value of over each Now, the standard computational procedure leads to the linear system of equations
where A is the corresponding global stiffness matrix and is the vector of the unknown nodal values of The matrix A is sparse, symmetric and positive
78
Gergana Bencheva, Ivan Georgiev, and Svetozar Margenov
definite. For large scale problems, the preconditioned conjugate gradient (PCG) method is known to be the best solution method. The goal of this study is to present a robust and parallelizable preconditioner for the system (3).
3 The Two-Level Algorithm Since the triangulation is obtained by diagonal-wise splitting of each cell Q into two triangles, see Figure 1-a, we can partition the grid nodes into two groups. The first group contains the centers of the quadrilateral super-elements and the second group contains the rest of the nodes. With respect to this splitting, A admits the following two-by-two block partitioning that can be written also in a block-factored form
where S stands for the related Schur complement. Obviously, is a diagonal matrix so that the Schur complement S can be assembled from the corresponding super-element Schur complements i.e.
where stands for the restriction mapping of the global vector of unknowns to the local one corresponding to a macroelement Q containing two triangles. Such a procedure is called static condensation. We now introduce the local Schur complement and its approximation
Here and In a very general setting, S and B are spectrally equivalent, if and have equal rowsums. The case corresponds to the isotropic approximation (see Figure 1-c). A particular case of this kind was proposed and analyzed in [12]. The dash lines represent the connectivity pattern of the Schur complement (see Figure 1-b) and its locally modified sparse approximations and (see Figure 1-c,d). Assembling the local matrices we get
After the static condensation step, the initial problem (3) is reduced to the solution of a system with the Schur complement matrix S. At this point we apply the PCG method with a preconditioner defined as a MIC(0) factorization (see, e.g., [3, 9]) of B, that is, This needs of course B to allow a stable MIC(0) factorization, which will be shown in Section 4.
Two-Level Preconditioning of Crouzeix-Raviart Anisotropic FEM Systems
79
Fig. 1. (a) Node numbering of the super-element Q; (b) Connectivity pattern of (c) Connectivity pattern of (d) Connectivity pattern of
4
Locally Optimized Approximation for the Model Anisotropic Problem
The model anisotropic problem analyzed in this section is set on a uniform square mesh i.e. The coefficient matrix is macroelement-wise constant, and the anisotropy ratio is fixed, i.e. Then, the macroelement stiffness matrix and the related Schur complement are as follows:
Taking into account the symmetry arguments, we look for a locally optimized matrix (see (5)) in the form:
Consider now the local eigenvalue problem: find
The optimal value of the parameter condition number
such that
will minimize the local spectral
Gergana Bencheva, Ivan Georgiev, and Svetozar Margenov
80
Obviously where Thus, (10) is reduced to a 3 × 3 eigenvalue problem and the related eigenvalues are:
Three cases are considered depending on the value of
namely:
The global condition number estimate follows directly from local one. The result of the presented analysis is summarized in the next theorem. Theorem 1. Consider the non-conforming FEM problem (2) on a square mesh. Let B is defined by (6) where are determined by (9) and the parameter satisfies the local optimality condition
Then:
(i) the sparse approximation B of the Schur complement S satisfies the conditions for a stable MIC(0) factorization; (ii) the matrices B and S are spectrally equivalent, where the following estimate for the relative condition number holds uniformly with respect to any possible coefficients’ jumps
5
Parallel Preconditioning Algorithm
The first step, i.e. the static condensation, is local and therefore can be performed fully in parallel. This is the reason to focus our attention on the PCG solution of the reduced system with the Schur complement S. Let us recall that the preconditioner was introduced as where D is the diagonal and L is the strictly lower triangular part of B. The computational complexity of one PCG iteration is given by
One can find some more details in [12] where a similar analysis for the isotropic case is made.
Two-Level Preconditioning of Crouzeix-Raviart Anisotropic FEM Systems
81
MIC(0) is an inherently sequential algorithm where the solution of the arising triangular systems is typically recursive. We overcome this disadvantage by the proposed construction of the matrix B. For simplicity of the presentation we consider the model problem in on a square mesh with a mesh size (subsequently each cell is split into two triangles to get The structure of the matrices S and B is illustrated in Figure 2 where a column-wise numbering of the unknowns is used. A serious advantage of the introduced matrix B is that all of its diagonal blocks are diagonal. In this case, the implementation of the PCG solution step is fully parallel within each of these blocks.
Fig. 2. Sparsity pattern of the matrices S and B,
A simple model for the arithmetic and the communication times is applied (see, e.g., [13]) in the analysis of the parallel algorithm. We assume that: a) the execution of M arithmetic operations on one processor takes time where is the average unit time to perform one arithmetic operation on one processor (no vectorization), and b) the time to transfer M data elements from one processor to another can be approximated by where is the start-up time and is the incremental time necessary for each of M elements to be sent, and is the graph distance between the processors. Further, we consider a distributed memory parallel computer where the number of processors is and for some natural number The domain is split into equally sized strips where the processor is responsible for data and computations related to the strip. Then, we get the following expressions for the communication times related to and
The above communications are completely local and do not depend on the number of processors assuming that and are neighbors (see Figure 3). With are denoted the components which sends to and with - those
82
Gergana Bencheva, Ivan Georgiev, and Svetozar Margenov
to The sign × indicates the place where the transferred components are stored. Although it is not mapped in the figure, the processor also have to receive the corresponding components from processors and This setting leads to the following expression for the parallel time per one PCG iteration
Fig. 3. Communication scheme for matrix-vector multiplication of systems with lower triangular and upper triangular
and for solution matrices.
Remark 1. From (15) we conclude that the parallel algorithm is asymptotically optimal. A more realistic analysis of the parallel performance needs more information about the behavior of the introduced average timing parameters and The key point here is that a good parallel scalability could be expected if
6
Numerical Tests
The numerical tests presented below illustrate the PCG convergence rate when the size of the discrete problem and the anisotropy ratio are varied in the test problem
A relative stopping criterion
is used
in the PCG algorithm, where stands for the residual at the iteration step, is the standard Euclidean inner product, and The computational domain is the unit square where homogeneous Dirichlet boundary conditions are assumed at the bottom side. A uniform mesh is used, where and the size of the discrete problem is
Two-Level Preconditioning of Crouzeix-Raviart Anisotropic FEM Systems
83
The qualitative analysis of the results given in Table 1 confirms that the number of iterations in all cases is namely, it grows proportionally to when is large enough. The influence of the anisotropy is in a full agreement with the theoretical estimate The last column is included in the table to show how important is to have a proper node-numbering following the direction of dominating anisotropy.
7
Concluding Remarks
In this paper we have proposed an anisotropic modification of the recently introduced two-level preconditioner for isotropic Crouzeix-Raviart FEM elliptic systems. The new construction allows efficient PCG solution of strongly anisotropic problems with fixed direction of dominating anisotropy. The new sparsity structure of the local matrices does not change the structure of the diagonal blocks of the global matrix B. The latter preserves the potential for efficient parallel implementation. One of the contributions of this paper is the locally optimized construction of where the minimal value of is realized in the interval given by Theorem 1. Our further plans include a generalization to 3-D problems on tetrahedral meshes. This is a much more complicated problem where symbolic methods could be needed to optimize the parametrized presentation of In this context we would refer to the results from [11] where an element preconditioning technique is reported for conforming linear FEM in 3-D. The structure of the Schur complement matrix S is the same as the element matrix for the rotated bilinear elements introduced by Rannacher and Turek. As a result, the parallel properties of the related algorithms are similar. Some representative results in this context can be found in [4] where the parallel times are analyzed in some more details and a set of numerical tests on distributed memory machines are presented.
Acknowledgements This work is supported by Center of Excellence BIS-21 grant ICA1-2000-70016, and the Bulgarian NSF Grants I-1001 and MU-I-901.
84
Gergana Bencheva, Ivan Georgiev, and Svetozar Margenov
References 1. Arbogast, T., Chen, Z.: On the implementation of mixed methods as nonconforming methods for second order elliptic problems. Math. Comp., 64 (1995) 943–972. 2. Arnold, D. N., Brezzi, F.: Mixed and nonconforming finite element methods: implementation, postprocessing and error estimates. RAIRO, Model. Math. Anal. Numer., 19 (1985) 7–32. 3. Blaheta, R.: Displacement decomposition — incomplete factorization preconditioning techniques for linear elasticity problems. Numer. Lin. Alg. Appl., 1 (1994) 107–126. 4. Bencheva, G., Margenov, S.: Performance analysis of parallel MIC(0) preconditioning of rotated bilinear nonconforming FEM systems. Mathematica Balkanica, 18 (2004) to appear. 5. Chen, Z.: Analysis of mixed methods using conforming and nonconforming finite element methods. RAIRO, Math. Model. Numer. Anal., 27 (1993) 9–34. 6. Chen, Z.: Equivalence between multigrid algorithms for mixed and nonconformning methods for second order elliptic problems. East-West J. Numer. Math., 4 (1996) 1–33. 7. Chen, Z., Ewing, R.E., Kuznetsov, Yu.A., Lazarov, R.D., Maliassov, S.: Multilevel preconditioners for mixed methods for second order elliptic problems. Numer. Lin. Alg. with Appl., 3 5 (1996) 427-453. 8. Chen, Z., Ewing, R.E., Lazarov, R.D.: Domain decomposition algorithms for mixed methods for second order elliptic problems. Math. Comp., 65 214 (1996) 467-490. 9. Gustafsson, I.: Modified incomplete Cholesky (MIC) factorization. In: D.J. Evans, ed., Preconditioning Methods; Theory and Applications, Gordon and Breach, (1984) 265–293. 10. Gustafsson, I., Lindskog, G.: On parallel solution of linear elasticity problems: Part I: Theory. Numer. Lin. Alg. Appl., 5 (1998) 123–139. 11. Langer, U.,Reitzinger, S.,Schicho, J.: Symbolic methods for the element preconditioning technique. Johanes Kepler University of Linz, SFB-Report No. 02-03 (2002) 12. Lazarov, R.D.,Margenov, S.D.: On a two-level parallel MIC(0) preconditioning of Crouzeix-Raviart non-conforming FEM systems. In: I. Dimov, I. Lirkov, S. Margenov, Z. Zlatev Eds., Numerical Methods and Applications. Lecture Notes in Computer Science, 2542, Springer Verlag, (2003) 192–201. 13. Saad, Y.,Schultz, M.H.: Data Communication in Parallel Architectures. Parallel Comput., 11 (1989) 131–150.
Parallel High Performance Computing on Composite Grids Roman Kohut Institute of Geonics, Academy of Sciences of the Czech Republic Studentská 1768, 70800 Ostrava-Poruba, Czech Republic
[email protected]
Abstract. The paper is concerned with the FE solution of the linear elastic BV problems on composite grids. Special interest is given to the implementation and numerical tests of both sequential and parallel codes.
1 Introduction This paper deals with the numerical solution of linear elasticity boundary value problems, which can be written in a weak form as follows:
Here is a Hilbert space of vector functions (see e.g. [4]). For the numerical solution of the problem (1) we use the finite element (FE) method on composite grids, which arise as a composition of (regular, structured) global coarse grid and (regular) local fine grid(s). The composite grid FE method starts with a global (coarse, regular) division of the domain For simplicity, we will consider a division of the domain into tetrahedrons. The division then enables us to define the standard FE space
where is in our case the set of linear polynomials. Further, let be a part of where the previous discretization should be refined and let be a finer discretization in which enables to define another FE space
Now, the composite grid FE space can be defined as
The composite grid FE space composite grid solution
can be then used for finding an approximate
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 85–92, 2004. © Springer-Verlag Berlin Heidelberg 2004
86
Roman Kohut
Note that, in this construction, can consist of several separate parts. Moreover, the finer grid in can be further refined and so on, which gives the multilevel construction resulting in the composite grid space The paper completes results published in [1, 2] with numerical tests performed for 3D case. Tests are performed using both sequential and parallel codes.
2
The Iterative Solution of Composite Grid Problems
The decomposition (4) of the composite grid FE space can be used to construct iterative methods for solving the composite grid problem (5). Such a method was first introduced in [3]. The algorithm of this method, called FAC, is as follows: FAC method
Further, we will also use a symmetric version of FAC, which is obtained by adding another correction from the subspace to the beginning of each FAC iteration. This variant will be referred to as SFAC. The additive variant of FAC called JFAC (as the Jacobi-type method with damping), takes the form: JFAC method
These methods can be rewritten into an operator form which is more convenient for computer implementation. The composite grid FE problem can be then rewritten as the equation
with a symmetric positive definite operator A. The iterative methods FAC, JFAC and SFAC can be written as preconditioned Richardson’s iterations
Parallel High Performance Computing on Composite Grids
87
with
where and are the restriction and inclusion operators, respectively Since A is symmetric positive definite, the system (6) can be solved by the conjugate gradient method with preconditioning matrices or This preconditioning is implemented as the approximate solution of the system using one FAC (JFAC, SFAC) iteration starting from the zero initial guess.
3
Implementation
In this section, we focus on the implementation of the two–level composite grid FE method arising from the spaces and We shall start by introducing the standard FE bases in and respectively. Then we can introduce the subproblem matrices and right-hand side vectors as
For the implementation of the composite grid FE method, we must introduce some parameterization of the functions from the composite grid FE space V. In our case we use the nodal basis implementation for fully compatible global and local grids where is represented in the form
Here the first sum is taken over those coarse grid basis functions which have their support in The second sum is over all basis functions from The third sum is over the new basis functions corresponding to the coarse grid nodes lying on the interface The basis functions have zero values in all coarse and fine grid nodes with the exception of value 1 in one coarse grid node on and the values given by the interpolation in the corresponding fine grid nodes on This interpolation guarantees the continuity of the basis function See also Fig.1. Note that the nodal basis implementation in principle requires compatibility of the global and local grids only on the interface In the case of not fully compatible global and local grids, we seek the composite grid solution in the space where is the space of the functions, that have the form
88
Roman Kohut
Fig. 1. Nodal basis of the composite grid FE space
(13). Functions from V having a support in are used only for solving the subproblem on the coarse grid which is a part of the preconditioning matrix. For the implementation of the conjugate gradient method on composite grids and for the implementation of the FAC and JFAC iterative methods as preconditioners, we need procedures for a prolongation of the displacement which is realized by a linear interpolation, restrictions of the residual given by the transformation of nodal forces from fine grid nodes to coarse grid ones a computation of the residual implemented using elementby-element multiplications for the nodes lying on the interface and subproblem matrix and vector multiplications for the nodes outside the interface. a solution of the subproblems obtained by the conjugate gradient method with a preconditioning given by the displacementdecomposition (DiD) or the domain-decomposition (DD). Remark: In some real-life problems the coarse grid cannot resolve the fine distribution of materials. It means that
It was proved ([1]) that even in this case the FAC, SFAC and JFAC methods converge if we use a suitable damping. The numerical tests show ( see next section) that if we use these methods as preconditioning methods, the CG on composite grids may not converge.
Parallel High Performance Computing on Composite Grids
4
89
The Numerical Tests
Numerical tests were performed on two benchmark problems referred to as “MOST” problem and “Dolní Rožínka” problem.
I: The “MOST” Problem – A Bridge Pile under Loading This problem, which is illustrated in Fig.2, is formulated in a domain of size 37.2 × 37.2 × 31 metres. This domain contains a concrete pile of size 1.2 × 1.2 × 15 metres with elastic modulus E = 31.5 GPa, Poisson’s ratio and unit weight The pile is surrounded by an elastic clay with elastic modulus E = 19.88 MPa, Poisson’s ratio and unit weight The pile is loaded by a pressure V = 1.5 MPa. On the other sides of we assume zero normal displacements and zero tangential stresses. The global coarse grid has 32 × 32 × 32 nodes (98304 unknowns), the local fine grid uses halved grid sizes, the fine mesh has 23 × 23 × 41 nodes (65067 unknowns).
Fig. 2. “Most” problem
II: The “Dolní Rožínka” Problem – A Mining at the Uranium Ore Deposit This problem, which is illustrated in Fig.3, is formulated in a domain of size 1430 × 550 × 600 metres. The two basic materials in the problem domain are the uranium ore and the surrounding rock. These materials are assumed to be elastic with the same material constants – elasticity modulus E=2000 MPa, Poisson ratio and the unit mass The extracted volumes and their vicinity are filled by a goaf material. For the goaf material, we again assume a linear elastic behaviour with elastic constant E=2500 MPa, and
90
Roman Kohut
We use pure Dirichlet boundary conditions given by the solution of a pure Neumann problem modelling the pre-mining stress state. The global coarse grid has 124 × 137 × 76 nodes (3873264 unknowns), the local fine grid uses quater grid sizes, the fine mesh has 49 × 113 × 65 nodes (1079715 unknowns).
Fig. 3. “Dolní Rožínka” problem The numerical tests were done using both sequential and parallel codes. The sequential solution was performed on a workstation IBM RS/6000 43P-260 (the preconditioning for subproblems given by DiD), the parallel solution on a cluster of 9 PC’s with AMD Athlon 1.4 GHz 266MHz FSB processors (the preconditioning for subproblems given by DD). On the benchmark problems the following tests were performed: a) Tests on the preconditioners The tests were performed with five FAC-like preconditioners: 1. the nonsymmetric FAC starting from a fine grid 2. the symmetric FAC starting from a fine grid 3. the nonsymmetric FAC starting from a coarse grid 4. the symmetric FAC starting from a coarse grid 5. the additive FAC (JFAC) To test the parallel code, tests were done only on Benchmark II. The accuracy for CG on the composite grid was (see Table 1). The tests show that we can use succesfully the nonsymmetric preconditioners and that orthogonalization of the search directions is not necessary if we solve the subproblems more accurately The results for the JFAC preconditioner show that the “vertical” parallelization (the subproblems are solved in parallel) does not give any significantly improvement of the CPU time.
Parallel High Performance Computing on Composite Grids
91
b) Influence of the material inconsistency on the efficiency of the preconditioners. In some problems the coarse grid is not able to model small nonhomogeneous details and subproblems are material inconsistent. If we use for solving such problems the FAC–like techniques, the algorithm converges for suitable damping (see [1]). But if we use these techniques as preconditioners this material inconsistency can cause non-efficiency of these preconditioners. We tested the influence of material inconsistency on Benchmark I for three cases which differ by the value of Young modulus for the pillar in the coarse and fine grids: A0: E=31500 MPa for the coarse g., E=31500 MPa for the fine g. (mat. consist.) A1: E=1000 MPa for the coarse grid, E=31500 MPa for the fine grid A2: E=1 MPa for the coarse grid, E=31500 MPa for the fine grid. The results are shown in Table 2. The tests confirmed that strong material inconsistency can degrade the convergence of CG method with such a “preconditioner”. The reason is that the solution on a coarse grid gives in the case of
92
Roman Kohut
material inconsistency incorrect Dirichlet boundary conditions for the solution on a fine grid and a preconditioning provides pseudoresidual which does not correspond to the composite problem error. Tests show that in the case of strong inconsistency we must use the preconditioner 2 - the symmetric FAC method starting on a fine grid and we must solve subproblems more accurately. c) CPU time for various parts of the algorithm We solved Benchmark II with the sequential and parallel solvers to show how much time consuming various parts of the algorithm are when using the sequential solver and how it can be improved using the parallel solver. The results are shown in Table 3. Note that the parallel an sequential codes run on different computers. We see that the most time consuming part of the sequential code is the solution of linear systems for the subproblems. Further CPU time reduction will be possible after introducing a parallel access to the composite matrix–vector multiplication.
References 1. Blaheta, R.;Byczanski, P.; Kohut, R.: Composite grid finite element method: Implementation and iterative solution with inexact subproblems, Applications of Mathematics 47(2002),83–100 2. Kohut, R.: The FE on composite grids. The numerical tests, proceedings of the conference Contemporary mathematical methods in engineering, J. Doležalová, ed. TU Ostrava, 2002, pp. 99–104. 3. McCormick, S. F.: Fast adaptive grid (FAC) methods: theory for variational case, in Defect Correction Methods: Theory and Applications, K. Böhmer and H.J. Stebter, eds. Computing Supplementum, 5, Springer–Verlag, Wien, 1984, pp. 115–122. 4. Mathematical theory of elastic and elasto–plastic bodies: An introduction, Elsevier, Amsterdam 1981
Parallel Preconditioning for Sedimentary Basin Simulations Roland Masson1, Philippe Quandalle1, Stéphane Requena1, and Robert Scheichl2 1
2
Institut Français du Pétrole, Rueil-Malmaison, France
[email protected] Deptartment of Mathematical Sciences, University of Bath, UK
[email protected]
Abstract. The simulation of sedimentary basins aims at reconstructing its historical evolution in order to provide quantitative predictions about phenomena leading to hydrocarbon accumulations. The kernel of this simulation is the numerical solution of a complex system of non-linear partial differential equations (PDE) of mixed parabolic-hyperbolic type in 3D. A discretisation and linearisation of this system leads to very large, ill-conditioned, non-symmetric systems of linear equations with three unknowns per mesh cell, i.e. pressure, geostatic load, and oil saturation. This article describes the parallel version of a preconditioner for these systems, presented in its sequential form in [7]. It consists of three steps: in the first step a local decoupling of the pressure and saturation unknowns aims at concentrating in the “pressure block” the elliptic part of the system which is then, in the second step, preconditioned by AMG. The third step finally consists in recoupling the equations. Each step is efficiently parallelised using a partitioning of the domain into vertical layers along the and a distributed memory model within the PETSc library (Argonne National Laboratory, IL). The main new ingredient in the parallel version is a parallel AMG preconditioner for the pressure block, for which we use the BoomerAMG implementation in the hypre library [4]. Numerical results on real case studies, exhibit (i) a significant reduction of CPU times, up to a factor 5 with respect to a block Jacobi preconditioner with an ILU(0) factorisation of each block, (ii) robustness with respect to heterogeneities, anisotropies and high migration ratios, and (iii) a speedup of up to 4 on 8 processors.
1 Introduction In recent years the 3D modelling and simulation of sedimentary basins has become an integral part in the exploration of present and future reservoirs for almost all major oil companies. It aims at reconstructing the historical evolution of a sedimentary basin in order to provide quantitative predictions about phenomena leading to hydrocarbon accumulations. A simplified kinematic model of the geometry of the basin is first computed from the geology of the basin using a back-stripping algorithm. The full model I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 93–102, 2004. © Springer-Verlag Berlin Heidelberg 2004
94
Roland Masson et al.
comprising hydrocarbon generation, heat transfer, compaction of the porous media, and multi-phase Darcy flow is then calculated forward in time adding layer after layer using the geometric history calculated during the back-stripping process. The most computing intensive part of the simulation consists in solving numerically a complex system of non-linear partial differential equations (PDEs) modelling the compaction of the porous media and multi-phase Darcy flow in 3D. These equations are discretized using a cell-centred finite volume method in space and an implicit Euler scheme in time, and linearised using Newton’s method. In the case of two-phase flow (water – oil), we will therefore have to solve at each time step and in each Newton iteration, a system of coupled linear equations with three unknowns per element (pressure, geostatic load, oil saturation), very large, non-symmetric, and ill-conditioned, in particular due to the strong heterogeneities and anisotropies of the porous media. The idea for our preconditioning strategy is based on the methods developed in [2,3,5,9] for the related problem of oil reservoir simulation and is described in detail in [7]. It consists of three steps. First of all the equations for pressure and saturation are locally decoupled by taking linear combinations of the two equations on each element. This decoupling aims not only at reducing the coupling, but also at concentrating in the pressure block the elliptic part of the system which requires a good global preconditioning. The second step consists then in preconditioning the pressure block by an algebraic multi-grid method. The recoupling of the pressure with the other unknowns is then in a final step achieved by applying a block Gauss-Seidel strategy or the combinative technique defined in [2] for the reservoir case. Each step of the preconditioner is efficiently parallelised using a partitioning of the domain into vertical layers along the and a distributed memory model within the PETSc library (Argonne National Laboratory, IL). Only slight modifications are necessary for most of the components of the sequential preconditioner in [7]. The main new ingredient which is necessary however, is a parallel preconditioner for the pressure block for which we use BoomerAMG [4] from the hypre package (Lawrence Livermore National Laboratory, CA). We present results with the sequential preconditioner for a new test case with very high migration ratios which was not available for [7]. For this test case, the Quasi-IMPES decoupling together with a recoupling using the combinative technique outperforms all other combinations suggested in [7] significantly. This suggests that in the case of high migration ratios a local decoupling of the two mass conservation equations does not reduce the global coupling of these two equations significantly. Moreover, Gauss decoupling and Householder decoupling destroy the ellipticity of the pressure block and thus AMG does not work very well anymore. All the performance tests (sequential and parallel) were therefore carried out with Quasi-IMPES decoupling. In a series of numerical tests, the performance of the preconditioner is compared to block Jacobi with an incomplete LU factorisation, ILU(0), of each block, previously the only way to precondition these linear systems efficiently in paral-
Parallel Preconditioning for Sedimentary Basin Simulations
95
lel. We observe a very significant reduction of the CPU-time for the linear solver on one processor, up to a factor 5 with respect to ILU(0). The performance of the preconditioner shows no degradation with respect to the number of elements or to strong heterogeneities and anisotropies in the porous media, and depends only very mildly on the migration ratios of oil. Furthermore, the parallel solver exhibits good speedup on an SGI Origin2000. In all the test cases we obtain speedups between 3 and 4. The paper is organised as follows. In Section 2 we will give a detailed description of the preconditioning strategy, followed by a series of numerical tests in Section 3.
2
Preconditioning Strategy
The mathematical model for the forward problem is formulated as a system of 4 coupled non-linear PDEs and accounts for the conservation equations of two phase Darcy flow (water and oil), as well as the conservation equation of vertical momentum describing the compaction of the porous media together with a rheology law relating the porosity to the geostatic load and to the average fluid pressure. We refer to [7] and [8] and to the references therein for a detailed description of the model. The equations are discretized using a finite volume scheme in space and a fully implicit time integration. After a Newton linearisation and an elimination of the rheology law we end up at each time step and in each Newton iteration with a system of linear equations of the form
where each one of the blocks in A is an N × N matrix with N being the number of mesh cells. The block rows correspond to the discretisations of the water conservation equation, the vertical compaction equation and the oil conservation equation, respectively. The block columns correspond to the derivatives with respect to the primary variables (i.e. average pore pressure, geostatic load and oil saturation), respectively. The details of the discretisation and the linearisation can be found in [7]. System (1) is very large, non-symmetric and ill-conditioned, in particular due to the strong heterogeneities and anisotropies of the porous media. For an efficient and robust iterative solution of this system, it is therefore crucial to find good preconditioners. The idea for our preconditioning strategy stems from the related field of oil reservoir simulation, and was first presented for sedimentary basin simulations in [7]. In reservoir simulations, the geological time spans that are simulated are much shorter in comparison to the time spans considered here (i.e. years instead of million years), so that the porosity and the effective stress can be kept constant, thus leading to systems similar to (1), but in pressure and
96
Roland Masson et al.
saturation only. In [2, 3, 5] a preconditioning strategy has been developed for these problems decoupling the treatment of pressure and saturation. In [7] we have used a similar strategy for (1) which consists of three steps: (i) local decoupling of the two mass conservation equations on each cell of the mesh (or in other words, choosing an appropriate equation for the average pore pressure in (ii) preconditioning of the pressure unknowns, (iii) preconditioning of the remaining unknowns (Block Gauss-Seidel), or if this is not sufficient, feedbackcoupling with the pressure equation (two-stage preconditioners). We will describe these three steps briefly and explain in particular how they are parallelised. In order to do this we need to first describe the parallelisation strategy.
2.1
Parallelisation Strategy
At the moment, the parallelisation uses a partitioning of the domain into vertical layers along the and a distributed memory model. However, this onedimensional topology for the parallel model is inherited from the parallelisation of the entire sedimentary basin simulation code and the preconditioner itself is not restricted to a partitioning into vertical layers. Communications between the processors and synchronisation are handled using MPI (Message Passing Interface) within the PETSc library (Argonne National Laboratory, IL) on the basis of a ghost point strategy (see [1] for details). The parallelisation of each individual component of the preconditioner presented in [7] is discussed in the following.
2.2
First Step: Local Decoupling – Choice of the Pressure Equation
The decoupling step corresponds to multiplying system (1) by
and leads to the transformed system
with of (1) by
and The matrix
This corresponds to a left preconditioning is chosen such that
(a) concentrates the elliptic part of the system, (b) the block (c) the norm of is O(1).
Parallel Preconditioning for Sedimentary Basin Simulations
97
In [7] we proposed three different choices for based on Gaussian elimination, on Householder reflection, and on a quasi-explicit treatment of the saturation (Quasi-IMPES), respectively. Unfortunately none of these choices satisfies all the three criteria above. The first choice satisfies (a), but does not necessarily satisfy (b) and (c) very well; the second choice satisfies (a) and (c), but again does not satisfy (b) very well; the third choice on the other hand does not satisfy (a), but satisfies (b) and (c). We will analyse the importance of these three criteria more carefully in Section 3.1. No additional work is necessary to parallelise this step, since all the operations are local to each mesh cell and thus local to each processor.
2.3
Second Step: Pressure Block Preconditioner
From the decoupling step, the block should concentrate the ellipticity, and thus the ill-conditioning of the system and basically correspond to a discrete diffusion operator with a strongly heterogeneous and anisotropic diffusivity tensor. We propose to use algebraic multi-grid (AMG) which is known to be efficient and robust for such stiff elliptic linear systems, even in the presence of strong heterogeneities and anisotropies. More precisely, we shall use BoomerAMG by Henson and Yang [4], a very efficient and robust parallel implementation of AMG within the hypre software package (Lawrence Livermore National Lab, CA). To test whether this implementation is competitive in the sequential case, we will also include some results in Section 3 comparing BoomerAMG on one processor with the implementation AMG1R5 by Ruge and Stüben [6] which was used in [7],
2.4
Third Step: Recoupling
The decoupling step is aimed at reducing the dependency of the pressure equations on the saturation. Thus, provided the decoupling is effective, we can assume that the block is small compared to Furthermore, physical considerations also allow us to make the same assumption about the block (see [7] for details). This leads to the idea for the following Block Gauss-Seidel (BGS) preconditioner for (3):
Here, is the AMG preconditioner for the pressure block discussed above, and is a preconditioner for the saturation block Since the dependency on saturation is hyperbolic or transport dominated parabolic, it is sufficient to use a simple relaxation scheme, like Jacobi or Gauss-Seidel, for In Section 3 we will use one iteration of a hybrid version of Jacobi and Gauss-Seidel for which involves no communication between the different processors and reduces to Gauss-Seidel on each processor. Finally, since the compaction equation is one dimensional and can be solved in each column independently, the block can
Roland Masson et al.
98
be cheaply inverted by back substitutions. This is also the case in the parallel version, provided each vertical mesh column is assigned to one processor. The assumption that the blocks and can be neglected may be too strong. It is necessary in such a case that the preconditioner also provides some feedback from the second and the third block row to the first. A way of providing such a feedback-coupling is to combine the Block Gauss-Seidel preconditioner in a multiplicative way with a second preconditioner that provides (at least locally) such a coupling and that is cheap to invert. A good candidate for such a preconditioner is zero fill-in incomplete LU factorisation, ILU(0), of It is cheap and provides a good local coupling of the different physical unknowns. Again to reduce communication in the parallel version we carry out the incomplete LU factorisation only locally on each processor. In other words, the second preconditioner is chosen to be a block Jacobi preconditioner with an ILU(0) factorisation of each block, denoted The combined preconditioner can then be written as
Similar techniques in reservoir simulations, notably in [2] and [5], are usually referred to as combinative two-stage preconditioners (C2S) and we will adopt the same name. The recoupling preconditioners and are used as right preconditioners for (3). And so in summary (putting all three steps together) we obtain the following preconditioned system:
where
3
is either
and
is either
or
Numerical Results
In this section we will discuss the results of the numerical tests which we effectuated to evaluate the performance of the preconditioner in terms of efficiency and robustness on some test cases from real case studies. The five test cases are subsections of sedimentary basins taken from real case studies of major oil companies. These studies are confidential and so we will refer to the test cases as Problems A – E. Problems A and D are problems with strong heterogeneities and anisotropies in the permeability tensors of the porous media but with low migration ratios, and thus represent almost single-phase flow. Problems B and C are problems with higher migration ratios, but without any strong heterogeneities in the permeability tensors. Problem E, on the other hand, comprises both these difficulties and is thus the hardest test case (not yet available in [7]). We will only regard a set of time steps towards the end of the simulation, when most the are deposited and the linear systems are the largest. The number of unknowns ranges from 16974 for Problem A to 129756 for Problem E. Unfortunately, larger problems of entire sedimentary basins were not available for reasons of confidentiality.
Parallel Preconditioning for Sedimentary Basin Simulations
99
As the iterative solution method for the preconditioned systems (6) we use the Bi-CGStab method by Van der Vorst [10] which is particularly suited for nonsymmetric linear systems. The stopping criterion is the reduction of the residual by a factor of The CPU-times are those obtained on an SGI Origin2000 using between 1 and 8 processors. We will compare the results obtained with our preconditioner with the results using block Jacobi preconditioning with an incomplete LU factorisation ILU(0) of each block, previously the only way to precondition these linear systems efficiently in parallel.
As we can see in Table 1, the performance of ILU(0) deteriorates rather strongly as the mesh size gets larger even in the sequential case, in particular for problems with strong heterogeneities (i.e. A, D, and E). Moreover, the block Jacobi parallelisation of it provides no global preconditioning at all and will eventually be not sufficient anymore for convergence (when the number of processors gets too large). Increasing the fill-in for the incomplete LU decomposition does not cure these problems, and the reduction in the number of iterations is in all cases weighed off by the extra cost of each iteration in terms of efficiency.
3.1
Comparison of the Different Decoupling/Recoupling Strategies
To compare the robustness of the different decoupling and recoupling strategies which have been proposed in Section 2, we will look at the average number of BiCGStab iterations per Newton iteration in the sequential case. These are given Based on the results for Problems A – D, we concluded in [7] that the choice of decoupling has no strong influence on the robustness of the preconditioner, but that for problems with high migration ratios the block Gauss-Seidel recoupling was not sufficient to guarantee the robustness of the preconditioner anymore. The combinative two-stage preconditioner, on the other hand, proved to be very robust in all cases. The new test case E however, which combines both strong heterogeneities and high migration ratios, gives us some very valuable new insight into the robustness of our preconditioner. In particular, we can observe that Gauss decoupling, (as well as Householder decoupling lead to a much worse
100
Roland Masson et al.
performance than Quasi-IMPES decoupling i.e. 55.4 iterations for in comparison to 8.4 iterations for using two-stage recoupling. This is probably due to the loss of ellipticity in the pressure block. The results also underline (even more dramatically) that block Gauss-Seidel recoupling is not sufficient for problems with high migration ratios, i.e. 88.7 iterations for block Gauss-Seidel in comparison to 8.4 iterations for the two-stage recoupling (using Thus, we can conclude that in the case of high migration ratios a local decoupling of the two mass conservation equations does not reduce the global coupling of the two equations significantly. However, the combinative two-stage preconditioner provides the necessary recoupling and is very robust in combination with Quasi-IMPES decoupling (see the final column in Table 1).
3.2
Comparison of the Pressure Preconditioners
In order to allow for a parallelisation of AMG the BoomerAMG implementation by Henson and Yang [4] differs from the original AMG code AMG1R5 [6] in its coarsening strategy. (Here we use Falgout coarsening within BoomerAMG.) To test whether BoomerAMG is competitive as the pressure preconditioner in the sequential case, we compare in Table 2 the results for Problems D and E using one V-cycle of BoomerAMG on one processor with the results using one V-cycle of AMG1R5 [6]. We see that apart form slight variations they perform almost identically.
We also include (in Table 2) the number of iterations for our preconditioner when the pressure block is inverted exactly using an LU factorisation of The number of iterations decreases at most by a factor of 2 which is astonishing.
3.3
Comparison with Block Jacobi – Parallel Speedup
Finally and most importantly we look at the parallel robustness and efficiency of our preconditioner and compare it to block Jacobi preconditioning with ILU(0) per block, The results (with Quasi-IMPES decoupling) for Problems D and E are given in Tables 3 and 4, respectively. First of all, observe the extremely good parallel robustness of our preconditioner for all of the recoupling strategies. The number of iterations hardly grows at all with the number of processors. In terms of CPU-time, our preconditioner
Parallel Preconditioning for Sedimentary Basin Simulations
101
Fig. 1. Parallel Speedup: Problems D (left) and E (right)
is more than 3 times more efficient than on one processor and the parallel speedup is very good: between 3 and 4 on 8 processors (see Figure 1). This is in fact about the same speedup than the one reported in [4] for BoomerAMG on more simple test problems. Finally, it is interesting to note that the modification of the combinative two-stage recoupling seems to lead to a better efficiency of the preconditioner both in terms of iterations and CPU-time and that this method therefore is our method of choice.
102
Roland Masson et al.
References 1. S. Balay, W. Gropp, L.C. McInnes, and B. Smith, PETSc Users Manual, Technical Report ANL-95/11 – Revision 2.1.0, Argonne, IL (2001). 2. A. Behie and P.K.W. Vinsome, Block iterative methods for fully implicit reservoir simulation, Soc. Petroleum Eng. J 22 (1982), 658–668. 3. H.C. Edwards, A parallel multilevel-preconditioned GMRES solver for multiphase flow models in the Implicit Parallel Accurate Reservoir Simulator (IPARS), TICAM Report 98-04, The University of Texas at Austin (1998). 4. V.E. Henson and U.M. Yang, BoomerAMG: A parallel algebraic multigrid solver and preconditioner, Applied Numerical Mathematics 41 (2002), 155–177. 5. S. Lacroix, Y.V. Vassilevski, and M.F. Wheeler, Decoupling preconditioners in the Implicit Parallel Accurate Reservoir Simulator (IPARS), Numer. Lin. Alg. Appl. 8 (2001), 537–549. 6. J.W. Ruge and K. Stüben, Algebraic Multigrid (AMG), In Multigrid Methods (S.F. McCormick, ed.), Frontiers in Applied Mathematics, vol. 5, SIAM, Philadelphia, 1986. 7. R. Scheichl, R. Masson, and J. Wendebourg, Decoupling and block preconditioning for sedimentary basin simulations in TEMIS3D, submitted to Computational Geosciences, Bath Mathematics Preprint 02/05, University of Bath (2002). 8. F. Schneider, S. Wolf, I. Faille, and D. Pot, A 3D basin model for hydrocarbon potential evaluation: Application to Congo Offshore, Oil and Gas Science and Technology - Rev. IFP 55, 1 (2000), 3–13. 9. Y.V. Vassilevski, Iterative Solvers for the Implicit Parallel Accurate Reservoir Simulator (IPARS), II: Parallelization Issues, TICAM Report 00-33, The University of Texas at Austin (2000). 10. H.A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM J. Sci. Comput. 12 (1992), 631–644.
Part III
Monte Carlo and Quasi-Monte Carlo Methods
This page intentionally left blank
Modeling of Narrow-Width SOI Devices: The Role of Quantum Mechanical Narrow Channel Effects on Device Performance Shaikh S. Ahmed and Dragica Vasileska Department of Electrical Engineering and Center for Solid State Electronics Research Arizona State University, Box 875706, Tempe, AZ 85287-5706, USA {shaikh.ahmed,vasileska}@asu.edu
Abstract. The impact of quantum mechanical space-quantization effects on the operation of a narrow-width SOI device structure has been investigated. The presence of a two-dimensional carrier confinement (both vertical and along the width direction) gives rise to larger average displacement of the carriers from the interface and lower sheet electron density in the channel region. This, in turn, results not only in a significant increase in the threshold voltage but also in pronounced channel width dependency of the drain current. In these simulations we have used classical 3D Monte Carlo particle-based simulations. Quantum mechanical space-quantization effects have been accounted for via an effective potential scheme that has been quite successful in describing bandgap widening effect and charge set back from the interface in conventional MOSFET devices.
1 Introduction For quite some time, the dimensions of semiconductor devices have been scaled aggressively in order to meet the demands of reduced cost per function on a chip used in modern integrated circuits. There are some problems associated with device scaling, however. Critical dimensions, such as transistor gate length and oxide thickness, are reaching physical limitations. Considering the manufacturing issues, photolithography becomes difficult as the feature sizes approach the wavelength of ultraviolet light. In addition, it is difficult to control the oxide thickness when the oxide is made up of just a few monolayers. In addition to the processing issues, there are also some fundamental device issues. As the oxide thickness becomes very thin, the gate leakage current due to tunneling increases drastically. This significantly affects the power requirements of the chip and the oxide reliability. Short-channel effects (SCEs), such as drain-induced barrier lowering (DIBL) and the Early effect in bipolar junction transistors (BJTs), degrade the device performance. Hot carriers also degrade device reliability. The control of the density and location of the dopant atoms in the MOSFET channel and source/drain region to provide a high on-off-current ratio becomes a problem in the 50 nm node devices where there are no more than 100 impurity atoms in the device active region. Yet another issue that becomes prominent in nano-scale I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 105–111, 2004. © Springer-Verlag Berlin Heidelberg 2004
106
Shaikh S. Ahmed and Dragica Vasileska
MOSFET devices is the quantum-mechanical nature of the charge description which, in turn, gives rise to inversion layer capacitance comparable to the oxide capacitance. This, in turn, degrades the total gate capacitance and, therefore, the device transconductance. A solution to some of the above mentioned problem, in order to achieving enhanced device performance, is to use silicon-on-insulator (SOI) materials. Over the last three decades, SOI CMOS has been identified as one possible method for increasing the performance of CMOS over that offered by simple scaling. However, prior to 1990’s, SOI had not been suitable as a substrate for mainstream applications. The barriers to its widespread usage were many, the main ones being SOI material quality, device design, and the steady progress in bulk CMOS performance through scaling. In view of these problems, IBM launched the first fully functional SOI mainstream microprocessor in 1999 marking that SOI technology was becoming the state-of the art technology for future lowpower ICs. An SOI SIMOX (separation by implanted oxygen) substrate with partially depleted epitaxial films (greater than 0.15 micron) was used for this purpose following a 0.22-micron technology. With this effort, the IBM specifications predict a 25 – 35% improvement over bulk CMOS counterpart, which is equivalent to about two years of progress in bulk CMOS design and fabrication process [14]. The above described SOI devices are not without problems, however. The thin gate oxide gives rise to gate leakage and the quantum-mechanical charge description due to the vertical confinement leads to device transconductance degradation. The quantization of the channel in the FD device along the growth direction gives rise to threshold voltage increase. This is a consequence of the fact that there is a reduction in the sheet electron density due to the smaller density of states function of the quasi-two-dimensional electron gas (Q2DEG) and the vanishing of the wavefunctions at the interface because of the large potential barriers (3.24eV). In a recent work by Majima et al. [13], it has been shown that, if the channel width is on the order of 10 nm or so, quantization occurs in the lateral direction as well. Even though this device structure has top oxide thickness of 34 nm, it serves as a proof of concept that a two-dimensional quantization effects can give rise to significant threshold voltage shift in nanoscale SOI device structures. It is the purpose of this paper to investigate the observed threshold voltage shifts in the device structure fabricated by Majima et al. [13] due to quantum-mechanical space-quantization effects using the effective potential scheme proposed by Ferry et al. [1]. It is worth noting that in some previous investigations we have shown that the effective potential approach can be successfully used for the case of two-dimensional carrier confinement effects in a narrow width wires, and the simulation results for the line electron density obtained by using the effective potential approach and the self-consistent solution of the 2D Schrödinger-Poisson problem can be found in [1]. The incorporation of the effective potential approach into a Monte Carlo particle-based simulator allows us, for the first time, to investigate the device transfer and output characteristics with proper treatment of two-dimensional quantization
Modeling of Narrow-Width SOI Devices
107
effects, velocity overshoot and carrier heating on an equal footing. Previously, this scheme has been applied to a conventional MOSFET device structure in which only one-dimensional quantum-mechanical size-quantization effects have been considered [7].
2
Theoretical Model
The inclusion of quantum effects in the description of the inversion layer at the semi-conductor/oxide interface of a SOI device involves solving the Schrödinger equation for the carriers in an approximately rectangular potential well. As a result, one obtains bound states, which give rise to two major features: reduced sheet density and charge set-back. Since the lowest bound state can be regarded as the new bottom of the conduction band, the spacing between the Fermi level and the conduction band edge is effectively increased, which results in the reduced sheet charge density with respect to the case in which quantum effects are excluded. Moreover, the probability density in the lowest bound state now has a maximum away from the top semiconductor/oxide interface, resulting in charge displacement from the oxide, which accounts for an effective increase in the oxide thickness. An alternative to solving the Schrödinger wave equation is to use quantum potentials. The idea of quantum potentials originates from the hydrodynamic formulation of quantum mechanics, first introduced by de Broglie and Madelung [4,5,12], and later developed by Bohm [2,3]. In this picture, the wave function is written in complex form in terms of its amplitude and phase. When substituted back into the Schrödinger equation one arrives at coupled equations of motion for the density and phase. The equations arising from this so-called Madelung transformation to the Schrödinger equation have the form of classical hydrodynamic equations with the addition of an extra potential, often referred to as the quantum or Bohm potential. The Bohm potential essentially represents a field through which the particle interacts with itself. It has been used, for example, in the study of wave packet tunneling through barriers [6], where the effect of the quantum potential is shown to lower or smoothen barriers, and hence allow for the particles to leak through. In analogy to the smoothed potential representation discussed above, Ferry et al. [7] suggested an effective potential that emerges from a wave packet description of particle motion, where the extent of the wave packet spread is obtained from the range of wavevectors in the thermal-distribution function (characterized by an electron temperature). The effective potential, is related to the self-consistent Hartree potential, V, obtained from the Poisson equation, through an integral smoothing relation
where G is a Gaussian with the standard deviation The effective potential is then used to calculate the electric field that accelerates the carriers in the transport kernel of the simulator described below.
108
Shaikh S. Ahmed and Dragica Vasileska
Regarding the Monte Carlo model, used in the transport portion of the simulator, it is based on the usual Si band-structure for three-dimensional electrons in a set of non-parabolic with energy-dependent effective masses. The six conduction band valleys are included through three pairs: valley pair 1 pointing in the (100) direction, valley pair 2 in the (010) direction, and valley pair 3 in the (001) direction. The explicit inclusion of the longitudinal and transverse masses is important and this is done in the program using the Herring-Vogt transformation [9]. Intravalley scattering is limited to acoustic phonons. For the intervalley scattering, we include both and processes. At present, impact ionization, surface-roughness and Coulomb scattering are not included in the model. They are omitted as they tend to mask the role of the space-quantization effects on the overall device performance. In solving Poisson’s equation, the Monte Carlo simulation is used to obtain the charge distribution in the device. We use the Incomplete Lower-Upper decomposition method for the solution of the 3D Poisson equation. After solving for the potential, the electric fields are calculated and used in the free-flight portion of the Monte Carlo transport kernel. However, the charges obtained from the EMC simulation are usually distributed within the continuous mesh cell instead of on the discrete grid points. The particle mesh method (PM) is used to perform the switch between the continuum in a cell and discrete grid points at the corners of the cell. The charge assignment to each mesh-point depends on the particular scheme that is used. A proper scheme must ensure proper coupling between the charged particles and the Coulomb forces acting on the particles. Therefore, the charge assignment scheme must maintain zero self-forces and a good spatial accuracy of the forces. To achieve this, two major methods have been implemented in the present version of the code: the Nearest-Grid-Point (NGP) scheme and the Cloud-in-Cell (CIC) scheme. The cloud-in-cell scheme (CIC) produces a smoother force interpolation, but introduces self-forces on non-uniform meshes. These issues have been dealt with extensively by Hockney and Eastwood [10], and quite recently by Laux [11]. Note that the electric field used in the free-flight portion of the simulator is calculated using the effective potential approach due to Ferry. Because of the presence of the semiconductor-oxide barrier, the convolution of the Hartree potential, obtained from solving the Poisson equation, and a Gaussian function with smoothing parameter of 0.63nm along the growth direction and 1nm along the channel (from source to drain) gives rise to electric fields that repel the carriers in the vicinity of the top semiconductor/oxide interface. This gives rise to larger average displacement of the carriers from the interface and degradation of the total gate capacitance due to the increase of the effective oxide thickness.
3
Simulation Results
The device structure we simulate consists of a thick silicon substrate, on top of which is grown 400nm of buried oxide. The thickness of the silicon on insulator layer is 7nm, with width of 10nm. The channel length is 50nm and the
Modeling of Narrow-Width SOI Devices
109
doping of the is On top of the SOI layer sits a gate-oxide layer, the thickness of which is 34nm. This is a rather thick gate oxide, but we have used it to compare our simulation results with the experimental data of Majima et al. [13]. The doping of the source/drain junctions equals and the gate is assumed to be a metal gate with workfunction equal to the semiconductor affinity.
Fig. 1. Width dependence of the threshold voltage for
The use of the effective potential shifts the conduction band edge upwards along the width and the depth of the device. It, therefore, accounts for the socalled band-gap widening effect, which leads to a reduction of the carrier density at the interface. Also, the electrons are moved away from the interface because of the additional perpendicular electric field in the vicinity of the interface. The quantization of charge in the inversion layer produces an expected increase in the threshold voltage of the device. The increase in the threshold voltage in the structure being investigated is due to both lateral and vertical confinement. The width dependence of the threshold voltage is shown in Figure 1. The sharp decrease in the threshold voltage with increasing channel width can be explained by increase in the channel conductance due to less quantized carriers. Note that similar trends have been observed in the experimental device structure from [13]. The slight discrepancy between the theory and the experiment can be due to two factors. First, the experimental device is grown on a (110) substrate and the theory is performed for a (100) substrates. Second, the threshold voltage in [13] is extracted using the constant current technique, whereas we have used the normalized mutual integral difference method due to He and coworkers [8]. The normalized mutual integral difference method utilizes the exponential-linear characteristic of the drain current versus gate voltage to obtain the drain voltage. As such, it is much better than any of the conventionally used methods including constant current technique, the linear extrapolation technique, the second derivative technique, the quasi-constant current technique, etc.
110
4
Shaikh S. Ahmed and Dragica Vasileska
Conclusions
In this work, we have utilized the effective potential approach to successfully simulate two dimensional space quantization effects in a model of a narrowchannel SOI device structure. The effective potential provides a set back of the charge from the interface, and a quantization of the energy within the channel. Both of these effects lead to an increase in the threshold voltage. We find a threshold voltage shift of about 250mV when the effective potential is included in the model for a device with 10nm channel width. Note that all simulations have been performed on a dual processor PC using Fortran 90. Also note that the scheme proposed by Ferry for incorporation of the quantum-mechanical spacequantization effects leads to only 10% increase of the overall CPU time. The solution of the 2D Schrödinger equation along slices of the device would be much more computationally costly.
Acknowledgements The authors would like to thank Prof. Stephen M. Goodnick, and D. K. Ferry for valuable discussions during the preparation of this manuscript. The financial support from the Office of Naval Research under Contract No. N000149910318 and the National Science Foundation under Contract Nos. NSF-ECS-9976484 and NSF-ECS-0214867 are also acknowledged.
References 1. R. Akis, S. Milicic, D. K. Ferry, and D. Vasileska, An effective potential method for including quantum effects into the simulation of ultra-short and ultra-narrow channel MOSFETs, Proceedings of the 4th International Conference on Modeling and Simulation of Microsystems, Hilton Head Island, SC, March 19-21, (2001) 550–3. 2. D. Bohm, A suggested interpretation of the quantum theory in terms of hidden variables. I., Phys. Rev. 85 (1952) 166. 3. D. Bohm, A suggested interpretation of the quantum theory in terms of hidden variables. II, Phys. Rev. 85 (1952) 180. 4. L. de Broglie, Sur la possibilité de relier les phénomènes d’interference et de diffraction a la théorie des quanta de luminère, C. R. Acad. Sci. Paris 183 (1926) 447. 5. L. de Broglie, La structure atomique de la matière et du rayonnement et la Mècanique ondulatoire, C. R. Acad. Sci. Paris 184 (1927) 273. 6. C. Dewdney and B. J. Hiley, A quantum potential description of one-dimensional time-dependent scattering from square barriers and square wells, Found. Phys. 12 (1982) 27. 7. D. K. Ferry, R. Akis, and D. Vasileska, Quantum Effects in MOSFETs: Use of an Effective Potential in 3D Monte Carlo Simulation of Ultra-Short Channel Devices, IEDM Tech. Dig. (IEEE Press, New York, 2000) 287–290. 8. J. He, X. Xi, M. Chan, K. Cao, C. Hu, Y. Li, X. Zhang, R. Huang and Y. Wang, Normalized mutual integral difference method to extract threshold voltage of MOSFETs, IEEE Electron Dev. Lett. 23 (2002) 428–430.
Modeling of Narrow-Width SOI Devices
111
9. C. Herring and E. Vogt, Transport and Deformation-Potential Theory for ManyValley Semiconductors with Anisotropic Scattering, Phys. Rev. 101 (1956) 944. 10. R. W. Hockney and J. W. Eastwood, Computer Simulation Using Particles, (Maidenhead: McGraw-Hill, 1981). 11. S. E. Laux, On particle-mesh coupling in Monte Carlo semiconductor device simultion, IEEE Trans. CAD Integr. Circ. Syst. 15 (1996) 1266. 12. E. Madelung, Quantatheorie in hydrodynamischer form, Z. Phys. 40 (1926) 322. 13. H. Majima, H. Ishikuro, and T. Hiramoto, Experimental evidence for quantum mechanical narrow channel effect in ultra-narrow MOSFET’ s, IEEE Electron Dev. Lett. 21 (2000) 396. 14. G. G. Shahidi, SOI Technology for the GHz era, IBM J. Res. & Dev. 46 (2002) 121.
Monte Carlo Algorithm for Ballistic Object Tracking with Uncertain Drag Parameter* Donka Angelova1, Iliyana Simeonova2, and Tzvetan Semerdjiev1 1
Central Laboratory for Parallel Processing, Bulgarian Academy of Sciences 25A Acad. G. Bonchev, 1113 Sofia, Bulgaria
[email protected] 2 Center for Systems Engineering and Applied Mechanics (CESAME) Batiment Euler, 4, Avenue G. Lemaitre, B-1348 LOUVAIN-LA-NEUVE, Belgium
[email protected]
Abstract. The problem of tracking a reentry ballistic object by processing radar measurements is considered in the paper. Sequential Monte Carlo-based filter is proposed for dealing with high nonlinearity of the object dynamics. A multiple model configuration is incorporated into the algorithm for overcoming the uncertainty about the object ballistic characteristics. The performance of the suggested multiple model particle filter (PF) is evaluated by Monte Carlo simulation.
1 Introduction The flight of a ballistic object (BO) consists of three phases: boost, ballistic (free flight) and terminal (reentry) phase. The topic of investigation in the present paper is the object reentry trajectory, when the Earth’s atmosphere is reached and the atmospheric drag becomes considerable. This phase finishes with the landing point on Earth [11]. The precise estimation of BO motion parameters and the accurate prediction of its impact point are of great importance for defense and for safety against reentry of old satellites and falling debris [6]. There exist a lot of extensive studies of BO tracking by processing radar measurements or using electro optical sensors. The challenges that must be overcome are connected primarily with high nonlinearity of the dynamic system, formed by BO and sensor. In addition, difficulties arise due to the uncertainty of the ballistic object parameters and noisy observations, received by the sensor. There are two major approaches to nonlinear recursive estimation of the object dynamics [13]. The nonlinear state and measurement equations in the first one are approximated by Taylor series expansions. The linearized (about the current state estimate) functions are applied directly to the linear Kalman filter to produce Extended Kalman filter (EKF). Various algorithms based on EKF for * Research Supported in part by Center of Excellence BIS21 grant ICA1-2000-70016, an European Community Marie Curie Fellowship and in the framework of the Control Training Site, and by Bulgarian Foundation for Scientific Investigations grants I-1202/02 and I-1205/02. I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 112–120, 2004. © Springer-Verlag Berlin Heidelberg 2004
Monte Carlo Algorithm for Ballistic Object Tracking
113
the reentry phase are proposed in the literature, resulting in different trade-offs between accuracy and computational costs. However, when the filtering problem is highly nonlinear, the local linearity assumption breaks down and the EKF may introduce large estimation errors. The second approach is based on the approximation of the underlying density functions of the state vector. Recursive algorithms on the densities can be derived from Bayes’formula. These densitybased algorithms are preferable in the cases of both nonlinear equations and non-Gaussian error terms. The unscented Kalman filter and PF belong to this class. The variants of these filters are suggested in [6] for the purposes of reentry vehicle tracking. BO uncertainty during the reentry phase is concentrated on the drag characteristics [11]. In general, the ballistic profile of the foreign vehicle under surveillance is not known and must be estimated for precise tracking. A natural technique is to include drag (or another suitable quantity) into the state vector as a state to be evaluated [3,4,7]. The authors of [12] adopt an uncertain interval for a key system parameter (the atmospheric density gradient). Then the entire tracking system is modeled as an interval nonlinear system and analyzed via an extended interval Kalman filter. A Monte Carlo filter is proposed in the work for BO radar tracking. Since BO dynamics are governed by a stochastic differential equation (SDE), the time evolution of the BO state probability density follows a Fokker-Plank equation (FPE). A numerical solution of SDE is realized in order to approximate the probability density of the predicted state. The objective is to refine the proposal distribution and consequently to improve the overall estimation accuracy. A multiple model approach is proposed also for identifying the unknown drag parameter (DP). It is assumed that the ballistic coefficient (the inverse of DP) belongs to a discrete set of possible values. The rest of the paper is organized as follows. The problem under consideration is set up in Section 2. Section 3 presents the features of BO SDE and its approximate solution. The suggested PF is described in Section 4. Computer simulation results are shown in Section 5, with conclusions given in Section 6.
2
Problem Formulation
BO and Sensor Models. The starting point for modeling time evolution of BO state is the Ito SDE [8,9]
where is an vector, f is an vector valued function, g is an matrix with the system noise covariance matrix and is a Wiener process. The function of tracking filter is to estimate at time given a set of measurements made at discrete times
114
Donka Angelova, Iliyana Simeonova, and Tzvetan Semerdjiev
The measurement model is given by the equation
where is a measurement vector, h is a measurement function and is a Gaussian white noise sequence with zero mean and covariance R. Density-based Filtering. The objective of density-based filtering methods is to construct the state posterior probability density function (PDF) based on all available information. According to the Bayesian philosophy, may be obtained recursively within the time interval in two stages – prediction and update [5]: Prediction. PDF evolution is connected directly with the Ito equation (1). It is given by the FPE [8]
where the initial condition is available from the previous time step It is assumed that the initial PDF is known. Information Update. When the measurement is realized:
arrives, the update stage
The normalizing constant in the denominator depends on the likelihood function determined by the parameters of the measurement model (2). The measurements are assumed conditionally independent in time. All information about the state contained in the measurements is captured in the conditional posterior PDF (4). An optimal (with respect to any criterion) estimate of the state and a measure of its accuracy can be obtained from [1,5]. In particular, the mean can be BO state estimate solving in this way the filtering problem. The recursive propagation (3) and (4) of the PDF is only a conceptual solution and can be implemented exactly for a restrictive set of cases [1]. Whereas the simulation-based methods offer efficient approximations to this optimal Bayesian solution. Multiple Model (MM) Approach. One of the problems connected with BO tracking is that the unknown DP reflects on the uncertainty about the state transition function in equation (1). However, in practice the drag values are positioned in an a priori known fixed interval. MM approach is a widely used methodology for overcoming this problem. It is assumed that the object state belongs to one of models corresponding to one of discrete values in the drag interval. The prior probability that is correct is assumed known and
Monte Carlo Algorithm for Ballistic Object Tracking
115
independent model-matched filters are run in parallel to produce model-conditioned state estimates Given the measurement data the posterior probability of model being correct can be calculated using Bayes’ formula [2]:
The output estimate is the weighted sum of the individual estimates (combined estimate) or the output from the “best” filter (with maximum posterior probability).
3
Numerical Solution of BO SDE
During the reentry phase, two significant forces act on the object [11]: the Earth’s gravity and the atmospheric drag. The drag force acts opposite to the velocity vector with an intensity, proportional to the air density and the square of the object speed The drag induced acceleration is given by [11]
where denotes the object altitude, and is the drag parameter, depending on object mass, body shape, cross-sectional object area perpendicular to the velocity, Mach number, drag coefficient and other unknown parameters. Therefore, for the reentry tracking, all uncertainties associated with the drag are collected in the DP [11]. The air density is usually approximated as a locally exponential function of the object altitude where and are known constants for a particular layer of the atmosphere. In the simulation below are accepted the following constant values for and for according to [6]. Assuming a flat, nonrotating Earth model, BO motion model has the following state-space form in the sensor East-North-Up coordinate system:
where is the gravity acceleration and is the process noise intensity. In the terminology of SDE (1), the state vector includes BO positions and velocities in the radar-centered Cartesian coordinate system; f(x) and g(x) have the following form:
116
Donka Angelova, Iliyana Simeonova, and Tzvetan Semerdjiev
Euler approximation is used to generate approximate sample paths of a solution of BO SDE. For a given equidistant discretization of the time interval the Euler approximation realizes the following iterative scheme [10]:
where the components of are distributed increments of the standard Wiener process, L is some a priori chosen integer.
4
Simulation-Based Suboptimal MM Filter
The key idea of Monte Carlo methods is to represent the required posterior density functions by a set of random samples with associated weights and to compute estimates based on these samples and weights [5]. Respectively, the evolving PDFs in eqs. (3) and (4) are replaced by a set of random samples which are predicted and updated by the particle algorithm for independent PFs are run in parallel
bank of
where is the common number of particles. The weights of the particular filters are used to obtain model-conditioned estimates. The weights of all N particles take part in the calculation of the output combined estimate and posterior model probabilities. The algorithm is described as follows:
1. Initialization, k = 0. * For sample and set 2. For (possibly in parallel) execute * Prediction step for generate samples according to the Euler scheme. * Measurement processing step on receipt of a new measurement for evaluate the normalized weights
* Selection step resample with replacement from the set
particles according to the importance weights
Monte Carlo Algorithm for Ballistic Object Tracking
117
* Compute model-conditioned estimates
End for 3. Output: Compute combined output estimate and model posterior probabilities
4. Set
and go to step 2.
The suboptimality of the procedure is due to the finite number of samples used. As the sample size tends to infinity, the optimal results can be achieved.
5
Simulation Results
BO scenario. BO trajectories for two drag parameters are shown in Fig. 1. The true initial parameters
and
are chosen like [6]. Measurement model. Tracking radar usually provides measurements in a spherical coordinate system. The measurement vector comprising range and elevation measurements, has a nonlinear relationship with the object state variables. The form of the measurement function in eq. (2) is as follows: The error standard deviations of range and elevation angle measurements are and respectively. A measurement conversion is implemented from spherical to Cartesian coordinates: Thus the measurement function becomes linear. The components of the corresponding covariance matrix R are as follows [2]:
The time interval between measurements is Parameters of MM scheme. A bank of filters is implemented for DPs and identical with the scenario parameters. The prior probability of each model being correct is selected as The equal number of particles is chosen for each filter. Filter design parameters. The parameters of state vector initial distribution are selected as follows: contains the exact initial BO parameters and The following parameters of the Euler scheme are accepted: prediction integration step is L = 10; process noise variance: and The sample size is N = 10000. Measures of performance. The state estimation accuracy is evaluated in terms of mean absolute error (MAE) – the absolute value of the actual error, averaged over
118
Donka Angelova, Iliyana Simeonova, and Tzvetan Semerdjiev
runs. Position MAE (both coordinates combined) and speed (magnitude of the velocity vector) MAE are calculated in the simulation experiments. The average probability of correct model identification is a measure for correct recognition of the DP.
Fig. 1. BO Scenario: Trajectory (a)
Fig. 2. Position MAE (a)
and
and
(b) Speed versus time
(b) Speed MAE versus time
Simulation results. As it can be seen from Fig.1, trajectories with different drag parameters are close to each other in a rather long time interval. The differences become obvious after the 50-th scan, when the speeds rapidly distinguish one from the other. This fact complicates the task of tracking. The tracking filter performance is evaluated based on a trajectory with The results are obtained by averaging over 50 Monte Carlo runs. Position and speed MAEs of the combined estimate and the “best” filter (filter which parameter is tuned to estimate are shown in Fig.2. The “best” filter
Monte Carlo Algorithm for Ballistic Object Tracking
119
Fig. 3. Posterior model probabilities (a) and (b) PDF histograms
estimate is better during the transitional time interval of identifying DP (between 50-th and 70-th scans), which is due to the fairly large posterior probability of the “wrong” filter. The averaged (over the runs) error standard deviation of the measurements (also shown in Fig.2(a)) can serve as a quality measure of the accuracy, achieved by the filter. The curve of the position MAE is below in the whole tracking interval. Plots of the posterior model probabilities are given in Fig.3(a). As it can be seen the delay in correct model detection is no more than 15 or 20 scans. The DP identification is accomplished with a high degree of confidence. PDF histograms for are presented in Fig.3(b). The posterior PDF of the appropriate model is located above the true object position, while the PDF of the “wrong” model is far from the desired place, determined mainly from the solution of the SDE with
6
Conclusion
Monte Carlo-based filter is suggested in the paper for tracking a reentry ballistic object by a radar. The objective is to explore the capability of particle algorithms for precise and reliable estimation of such highly nonlinear systems. The feature of the proposed version is connected with the SDE, describing object dynamics. A numerical solution of SDE is realized in order to refine the probability density of the predicted state and consequently to improve the overall estimation accuracy. MM approach is applied also for identifying the unknown drag parameter. The filter performance is examined by simulation over a typical BO scenario. The results show a reliable tracking and correct drag parameter determination. It gives an alternative solution to the difficult and important problem of BO tracking.
120
Donka Angelova, Iliyana Simeonova, and Tzvetan Semerdjiev
References 1. Arulampalam, S., Maskell, S., Gordon, N., Clapp, T.: A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking. IEEE Trans. Signal Processing, Vol. 50 (2002) 174–188. 2. Bar-Shalom, Y., Li, X.R.: Multitarget–Multisensor Tracking: Principles and Techniques. YBS Publishing, (1995) 3. Cardillo, G., Mrstik, A., Plambeck, T.: A Track Filter for Reentry Objects with Uncertain Drag. IEEE Trans. AES Vol. 35 (1999) 394–409. 4. Costa, P.: Adaptive Model Architecture and Extended Kalman-Bucy Filters. IEEE Trans. AES Vol. 30 (1994) 525–533. 5. Doucet, A., de Frietas, N., Gordon, N. (eds.): Sequential Monte Carlo Methods in Practice. Statistics for Engineering and Information Science. Springer-Verlag, New York (2001) 6. Farina, A., Ristic, B., Benvenuti, D.: Tracking a Ballistic Target: Comparison of Several Nonlinear Filters. IEEE Trans. AES Vol. 38 (2002) 854–867. 7. Farina, A., Benvenuti, D., Ristic, B.: Estimation accuracy of a landing point of a ballistic target. Proc. FUSION 2002 Conf., Annapolis, USA, (2002) 2–9. 8. Jilkov, V., Semerdjiev, Tz., Angelova, D: Monte Carlo Filtering Algorithm for Nonlinear Stochastic Environmental Model. Recent Advances in Numerical Methods and Applications II. World Scientific, London (1998) 266–274. 9. Kastella, K.: Finite Difference Methods for Nonlinear Filtering and Automatic Target Recognition. In: Bar-Shalom, Y., Blair, W. (eds.): Multitargrt-Multisensor Tracking. Applications and Advances, Vol. 3. Artech House, Boston (2000) 233– 258. 10. Kloeden, P., Platen, E., Schurz, H.: Numerical Solution of SDE Through Computer Experiments. Springer-Verlag, New York (1994). 11. Li, X., Jilkov V.: A Survey of Maneuvering Target Tracking - Part II:Ballistic Target Models. Proc. SPIE Conf. Signal and Data Processing of Small Targets, Vol. 4473-63, San Diego, CA, USA (2001) 12. Siouris, G., Chen, G., Wang, J.: Tracking an Incoming Ballistic Missile Using an Extended Interval Kalman Filter. IEEE Trans. AES Vol. 33 (1997) 232–240. 13. Tanizaki, H.: Nonlinear Filters. Estimation and Applications. Springer-Verlag (1996)
Efficient CPU-Specific Algorithm for Generating the Generalized Faure Sequences* Emanouil I. Atanassov Central Laboratory for Parallel Processing, Bulgarian Academy of Sciences Acad. G. Bontchev, Bl. 25A, 1113 Sofia, Bulgaria
[email protected]
Abstract. The Faure sequences are a popular class of low-discrepancy sequences. Their generalized variants, with better equi-distribution properties, are extensively used in quasi-Monte Carlo methods, especially for very high dimensional problems. The task of generating these sequences can take substantial part of the overall CPU time of a quasiMonte Carlo computation. We present an efficient algorithm for generating these sequences, and demonstrate how it may be tuned to use the extended instruction sets, available on many modern CPUs, to reduce drastically the CPU-time, spent for generating these sequences.
1 Introduction 1.1
Ideas of Monte Carlo and Quasi-Monte Carlo Methods
The main idea in the Monte Carlo methods is to represent the unknown quantity, as the mathematical expectation of some random variable so that by averaging over N independent samples of one can obtain an estimate of Usually the random variable is expressed as a function of independent random variables, uniformly distributed in [0,1]. Thus the function is defined in the unit cube and the Monte Carlo algorithm for approximating can be thought of as an algorithm for approximate calculation of the integral
by means of the formula:
where the values are computed using a pseudo-random number generator. The integer becomes the constructive dimension of the Monte Carlo algorithm *
Supported by the project of European Commission - BIS 21 under contract ICA1CT-2000-70016 and by the Ministry of Education and Science of Bulgaria under contract NSF I-1201/02 and NSF MM-902/99.
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 121–127, 2004. © Springer-Verlag Berlin Heidelberg 2004
122
Emanouil I. Atanassov
(see, e.g, [10], p. 254). Such an algorithm has an order of convergence of the probable error In order to obtain faster convergence for suitable classes of functions, the idea of the quasi-Monte Carlo methods is introduced. Instead of random numbers, one can use a deterministic uniformly distributed sequence so that the average is computed using the values of at the first N terms of the sequence The quality of distribution of the sequence is usually measured by its discrepancy (for a definition see, e.g., [5]). It is believed that the best order of the discrepancy of an infinite sequence is (see the results of W.M. Schmid [9], K.F. Roth [8], Baker [1]), which yields an asymptotic order of the integration error, better than that of Monte Carlo. Infinite sequences, which achieve such order of convergence to zero of the discrepancy, are called “low-discrepancy” sequences. Various theoretical and practical results show that even for vary large the low-discrepancy sequences may outperform Monte Carlo on classes of functions, important for the applications (see, e.g., the work of Paskov and Traub [7], and of A. Papageorgiou and J.Traub [6]). The question of finding the best low-discrepancy sequence remains unsolved. Many classes of low-discrepancy sequences are known, and for different problems different classes perform better. One important class of lowdiscrepancy sequences are the sequences introduced by Faure. The generalized Faure sequences, proposed by Tezuka [11–13] and sometimes called GFaure, are widely used in practice. The difference from point of view of generation algorithms between the general and the particular case is small, and we decided to deal only with the general case.
1.2
Definitions of the Faure and Generalized Faure Sequences
Tezuka [11–13] introduced a generalization of the Faure sequences, adding more degrees of freedom, which permit the construction of better distributed sequences. We use the following definition (Faure [4]): Definition 1. Let Let matrices, and let
be a fixed integer, greater than or equal to the dimension be given infinite non-singular lower triangular be the powers of the Pascal matrix, so that
The term of the expanding in
generalized Faure sequence number system:
and then setting
is obtained by
Efficient CPU-Specific Algorithm
123
where by mod we denote the remainder of modulo If the matrices are all equal to the identity matrix, we obtain the original Faure sequences. These sequences are of low-discrepancy, and moreover, they are the first class of sequences, for which an estimate of this kind was obtained (Faure [3]):
with a constant tending to zeros as tends to infinity. This result suggests that these sequences should be good for practical purposes, and in the area of financial mathematics they become very popular for estimating some very high-dimensional integrals. They are used for instance in the so-called FinDer program [2], which is polupar in Financial Mathematics. Various constructions for the matrices are known. We assume that they are given to us as input.
1.3
Importance of CPU-Specific Generation Algorithms
Because of their similarity from algorithmic point of view, the quasi-Monte Carlo and classical Monte Carlo methods compete directly both in terms of precision and speed of computation. The main difference in these setting is that in the case of classical Monte Carlo we use a pseudo-random number generator, and in quasi-Monte Carlo methods we need a suitable generator function for the chosen low-discrepancy sequence. Obviously, for some applications the function will be complicated and expensive to evaluate in terms of computational time, while for others the main part of the CPU time will be spent in the generation. In most quasi-Monte Carlo algorithms one can use almost any of the popular low-discrepancy sequences and compare the results, analogously to the situation in Monte Carlo, where one is advised to test different pseudo-random number generators. This suggests that it is preferable to have generation functions with similar calling conventions for every popular class of low-discrepancy sequences. However, we aim to develop the most efficient implementation possible for a given computer architecture. Since there is a certain degree of fine-grained parallelism in the operations, required for generating the terms of the sequence, an important speed-up can be obtained by using the special instructions for processing multi-media data, which are available on most modern CPU. In this paper we concentrate on the most popular such instruction sets. The MMX instruction set was the first extension to the 386 instruction set, introduced by Intel. It was later implemented on the newer AMD processors like Athlon and Duron, and therefore it can be considered as standard for the newer Pentium-compatible processors. The SSE1 and especially the SSE2 instruction set, introduced by Intel, are more powerful, but are available only on the newest processors. We show how our algorithm can be implemented using the MMX and the SSE2 instruction set. Since the Pentium and Athlon processors are very widespread at both workstations and clusters, the speed-up obtained by using our algorithm will be available to a large part of the scientific community. Writing a version of the generation code for another instruction set is just a matter of implementing some simple operations on large vectors of numbers.
124
2
Emanouil I. Atanassov
Description of the New Generation Algorithm
We developed our algorithm under the assumption, that the terms of the sequence need to be generated consecutively. In such case it is justified to have a preprocessing phase, so that each term of the sequence could be easily generated using the preprocessed data and some data from the previous invocation of the generation function. The data is organized so as to make maximal use of the extended instructions available, which means the array describe below should be at 8-byte aligned memory locations. At the preprocessing stage we perform the following operations:
1. Input the dimensionality, of the sequence, the prime number the coefficients of the matrices and the index of the first term to be generated, 2. Compute the coefficients of the matrices 3. Compute the “skewed” coefficients, by the formula
4. Compute the digits system. 5. Fill the table of relative positions:
of
in
positional number
6. Compute the table of partial sums, defined as follows:
Note that we need at most the first 64 rows and columns of the matrices and since we work in double precision. The partial sums are, in fact, the coordinates of the term of the sequence. Observe that all these steps have complexity comparable to that of step 2. When the generation function is invoked for generating the Nth term of the sequence, we first compute the digits of N + 1 and copy the partial sums to the buffer, supplied by the user, since they represent the Nth term of the sequence. Then we proceed in two possible ways, depending on the index, of the first least-significant non-zero digit of N. If we perform the following operations:
1. For add to the partial sum greater than 1, subtract 1 from it; add to every relative position 2. For greater than subtract from it.
and if the result is and if it becomes
Efficient CPU-Specific Algorithm
125
The key point in the algorithm is to perform these operations using the extended instruction sets available, without any conditional jumps, which would degrade the performance. This is explained in more detail in the next section. If we perform the following operations:
1. For to 0 with step –1, do the following: (a) For every set to and if it becomes greater than subtract from it. (b) For every store in Obviously, the probability that will be greater than 1 is and therefore the way we handle this second case is not as crucial for the performance of the algorithm as the case We can vectorize the operations, that are performed here, in a similar manner to the previous case, but in this case the speed-up is not as dramatic. In the next section we give more details about the generation code that we wrote based on this algorithm. Note that we do obtain the sequence in its natural order, without any Gray code reordering.
3
Implementation Details
As it can be seen from the description of the algorithm, the most crucial part in terms of CPU time is the way we handle the case when the index, N, of the term of the sequence, is not divisible by the prime That is why when we implement this algorithm on a given computer architecture, we need to write CPU-specific code only for this part, even though the case can be handled in a similar way. Since the operations that need to be vectorized are relatively simple, we successfully wrote variants of the generation code for the MMX and the SSE2 instruction sets. We always worked in double precision, but it should not be a problem to write faster single precision versions. Note that the MMX instruction set is less powerful and does not allow the floating point operations in double precision to be vectorized. Yet we obtained good speed-up, as it can be seen in Table 1.
The vectorization is straight-forward, basically equivalent to manual loopunrolling. The code is available under the GNU Public license at: http://parallel.bas.bg/~emanouil/sequences.html
126
Emanouil I. Atanassov
Here we only give the main idea. We wish to perform efficiently the operations, described above, without using conditional jumps, because they significantly degrade the performance. We use the fact, that if we need to compute the expression
we can use the formula
where the plus sign denotes integer or floating point summation, and by we denote a sequence of bits, with values equal to the value of the predicate and with width in bytes equal to the width of and in bytes. Since our predicates are of comparison type, they are vectorized naturally and efficiently. As an example, we show how step 1 of the algorithm in the case is vectorized using the MMX instruction set (only the innermost loop is shown):
4 Timing Results We present timing results, obtained by compiling our code with the gcc compiler. The non-standard instructions are coded using inline assembly statements. In our tests we measured the time, required for producing ten million terms of the sequence in various dimensions. The generation speed is measured in millions of coordinates, generated per one second. We observed that the generation speed falls somewhere between that obtained with our generators for the Sobol’ and
Efficient CPU-Specific Algorithm
127
Halton sequences, which are also available at the above web address. Note that an average pseudo-random number generator runs at speed of around 10 millions numbers per second, and therefore the use of low-discrepancy sequences offers significant performance advantage. We also show the generation time and speed for the portable version of our algorithm, which is not restricted to the x86 architecture, so that the significant speed-up, obtained trough vectorization, can be observed.
Conclusions and Directions for Future Work
5
With the new vectorized algorithm we are able to achieve significant speed-up with respect to regular C or FORTRAN code implementation. At the same time porting the algorithm to another architecture would require only trivial changes. Our plans are to implement generation codes for some other popular classes of low-discrepancy sequences under the same paradigm.
References 1. R. C. Baker. On Irregularities of Distribution II. J. London Math, Soc., 2(59)
(1999) 50–64.
2. The FinDer Program, http://www1.cs.columbia.edu/~ap/html/finder.html. 3. H. Faure. Discrépance de suites associées à un système de numération (en dimen-
sion
Acta. Arith., 41 (1982)
4. H. Faure. Monte-Carlo and quasi-Monte-Carlo methods for numerical integration,
5. 6. 7. 8. 9. 10. 11. 12. 13.
Combinatorial & computational mathematics, (Pohang, 2000) 1–12, World Sci. Publishing, River Edge, NJ, 2001. H. Niederreiter, L. Kuipers. Uniform Distribution of Sequences. John Wiley & sons, New York, 1974. A. Papageorgiou and J. Traub. Beating Monte Carlo. Risk, June (1996) 63-65. S. Paskov and J. Traub. Faster Valuation of Financial Derivatives. Journal of Portfolio Management, Vol. 22:1, (1995) 113–120. K. F. Roth. On Irregularities of Distribution. Mathematika, Vol 1 (1954) 73–79. W. M. Schmidt. Irregularities of Distribution VII. Acta Arith., Vol. 21 (1972) 45–50. I. M. Sobol. Monte Carlo Methods. Nauka, Moscow, 1973. S. Tezuka. Polynomial Arithmetic Analogue of Halton Sequences. TOMACS, 3:2, (1993) 99–107. S. Tezuka , T. Tokuyama. A Note on Polynomial Arithmetic Analogue of Halton Sequences. TOMACS, 4:3 (1994) 279–284. S. Tezuka. Uniform Random Numbers: Theory and Practice. Kluwer Academic Publishers, Dordrecht, (1995).
A New Quasi-Monte Carlo Algorithm for Numerical Integration of Smooth Functions Emanouil I. Atanassov, Ivan T. Dimov, and Mariya K. Durchova Central Laboratory for Parallel Processing, Bulgarian Academy of Sciences Acad. G. Bontchev, Bl. 25A, 1113 Sofia, Bulgaria
[email protected],
[email protected],
[email protected]
Abstract. Bachvalov proved that the optimal order of convergence of a Monte Carlo method for numerical integration of functions with bounded order derivatives is w h where i s the dimension. W e construct a new Monte Carlo algorithm with such rate of convergence, which adapts to the variations of the sub-integral function and gains substantially in accuracy, when a low-discrepancy sequence is used instead of pseudo-random numbers. Theoretical estimates of the worst-case error of the method are obtained. Experimental results, showing the excellent parallelization properties of the algorithm and its applicability to problems of moderately high dimension, are also presented.
1 Introduction The paper proposes new quasi-Monte Carlo algorithm for integrating smooth functions. We consider integration over the unit cube The class of functions under consideration is the following: Definition 1. For given integers and consists of all real functions defined in
exist for all
and the class such that all the derivatives
and there absolute values are bounded by M.
The results of Bachvalov [2,3] establish lower bounds on the integration error of both deterministic and stochastic or Monte Carlo methods. Various Monte Carlo methods for approximate integration of such functions with the optimal order of convergence are known (see, e.g. [1,5,10,20]). In this paper we investigate from theoretical and practical view point a quasi-Monte Carlo algorithm, based loosely on the same ideas. The idea of quasi-Monte Carlo algorithms is to replace the pseudo-random numbers, used in Monte Carlo, with a deterministic sequence of points, uniformly distributed in some high-dimensional unit cube The quality of distribution of these sequences is measured through I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 128–135, 2004. © Springer-Verlag Berlin Heidelberg 2004
A New Quasi-Monte Carlo Algorithm
129
their discrepancy. Infinite sequences with order of the discrepancy which is believed to be the optimal, are called low-discrepancy sequences. Note that in Monte Carlo methods one can frequently obtain a statistical error estimate along with the result, which is necessary in many areas. In quasi-Monte Carlo methods a similar error estimate of statistical nature can be obtained using the so-called scrambling of sequences, which is a way of adding some randomness to otherwise entirely deterministic sequences. The most popular quasi-Monte Carlo integration method is based on the simple formula:
where is a uniformly distributed sequence. For a good introduction in the theory of uniform distribution modulo 1 see, e.g., the books [9] and [7]. More complex methods are developed and tested by many authors (see, e.g., [18, 16]). A method, based on scrambled nets, can be seen in [12]. In quasi-Monte Carlo methods it is important to achieve low constructive dimensionality (for a definition, see [17], p. 255). Since the first few coordinates of a low-discrepancy sequence are better distributed, various ways to adjust the integration procedure to the properties of the integrand are proposed (see, e.g., the Brownian bridge construction in [4]). The performance of the quasi-Monte Carlo method depends on the quality of the distribution of the underlying low-discrepancy sequence. Various families of such sequences are known, and we tested some of the most popular ones in our algorithm. The algorithm is described in Sec. 2. Numerical results, showing the applicability of the method and its excellent parallelization properties, are given in Sec. 3.
2
Description of the Algorithm
Since we are going to describe our algorithm using the term pseudo-inverse matrix , we provide the following Definition 2. (see e.g. [11], p. 257) Let A be an real matrix with rank Suppose that is the SVD of A. Then the pseudo inverse matrix of A is defined as where
is referred to as the pseudo inverse of A. If Definition 3. Let be an integer, Let for some M. The
and let
then be fixed points in with
130
Emanouil I. Atanassov, Ivan T. Dimov, and Mariya K. Durchova
can be ordered lexicographically and there are exactly of them. Consider the matrix B with rows and columns, such that the column of B corresponds to the tuple and contains all the products
Suppose that and that B has rank pseudo-inverse of B. C has rows and polynomial is defined by
Let the matrix C be the columns. Now the interpolation
Observe that if the points are in general position, the matrix B has rank Since in our definition B has always full rank, the pseudo-inverse C is equal to The reason for using formulae with for numerical integration is that the coefficients of the matrix C are much smaller in this case. Of all the matrices satisfying BAB = B the pseudo-inverse is chosen because its Frobenius norm is the smallest. Now let us fix some integers and greater than 1 and consider functions defined over the unit cube We also fix the number and the points such that an interpolation formula as described in Definition 3 is defined. Observe that by integrating over we get a quadrature formula
where
Definition 4. Consider a rectangular area and an interpolation formula as in definition 3. Let T be a linear transformation such that For a given function consider the function such that The interpolation formula is defined by Now we define a Monte Carlo integration formula, which we are going to investigate in the sequel. Definition 5. Let be an integer. Consider a representation of the unit cube as a union of N rectangles
A New Quasi-Monte Carlo Algorithm
For the rectangle
131
we consider the simplest linear transfor-
mation T such that
By
the points
are transformed into points
Let be a given integer, and let be N independent random variables, uniformly distributed in the respective rectangles For every we use samples of and define the Monte Carlo estimate of to be
The Monte Carlo estimate for the integral
is obtained by summing
all the estimates for the integrals Such integration formulae were considered in [1], but only when has the smallest possible value This additional degree of freedom results in faster convergence of the resulting formulae.
3
Numerical Results
In this section we present numerical results that are obtained for calculating the value of the integral
The functions, that are considered here, are with different peculiarities. Often they are used for benchmarking of Monte Carlo and quasi-Monte Carlo algorithms. They are given as follows:
132
Emanouil I. Atanassov, Ivan T. Dimov, and Mariya K. Durchova
The first function has been used in [8] and [14] as a test in a context of rating point sets for quasi-Monte Carlo and Monte Carlo integration. The last three functions are a part of the test functions package proposed by Genz or in [13,15,19]. The parameters are divided into unaffective and affective parameters. The vector of affective parameters is scaled so that it satisfies the requirement
for the functions respectively, that depends only on the space dimensionality. In Table 1 we present numerical results from the approximate integration of the integral using the Monte Carlo and quasi-Monte Carlo version of the algorithm. The algorithms where tested for dimension 4, 5, 6, and 9. We did not compare our algorithm with crude Monte Carlo as in [6], because of its much slower convergence. From this table we made the conclusion, that the quasiMonte Carlo variant of the algorithm shows approximately the same accuracy as the Monte Carlo algorithm, i.e., for smooth functions.
A New Quasi-Monte Carlo Algorithm
133
In Table 2 and 3 we show numerical results for the other integrals in 5 dimensions. We again observe roughly the same probable error of the Monte Carlo and the quasi-Monte Carlo algorithm. All these results are obtained with the parameter (the number of quasi-random points in every small cube) equal to 40. In Table 4 we show the parallel efficiency of the algorithm. The computations for every small cube can be done in parallel, and thus the algorithm possesses a significant amount of natural parallelism. The results are achieved on a cluster of Intel Xeon 2.2 processors, using MPI.
4
Conclusions
We have developed a quasi-Monte Carlo method, which achieves order of convergence on smooth functions. The proposed quasi-Monte Carlo
134
Emanouil I. Atanassov, Ivan T. Dimov, and Mariya K. Durchova
method shows roughly the same accuracy as the Monte Carlo version, and also exhibits good parallel efficiency. However, the statistical aposteriory error estimation, based on the use of scrambled low-discrepancy sequences, is slightly more complicated and less reliable, than Monte Carlo methods. In our experiments we obtained good results by using the Sobol sequences. Note that the constructive dimensionality of the algorithm is relatively high - sm, and thus the integration method can be used for assessing the quality of distribution of other families of low-discrepancy sequences, i.e., for benchmarking purposes.
Acknowledgements Supported by the project of European Commission - BIS 21 under contract ICA1-CT-2000-70016 and by the Ministry of Education and Science of Bulgaria under contract NSF I-1201/02 and NSF MM-902/99.
References 1. E. Atanassov, I. Dimov. A new optimal Monte Carlo method for calculating in-
2. 3. 4.
5. 6.
7. 8. 9. 10.
11. 12. 13.
tegrals of smooth functions. Monte Carlo Methods and Applications, 5 2 (1999), 149–167. N.S. Bachvalov. On the approximate computation of multiple integrals. Vestnik Moscow State University, Ser. Mat., Mech., 4 (1959) 3–18. N. S. Bachvalov. Average Estimation of the Remainder Term of Quadrature Formulas. USSR Comput. Math. and Math. Phys., 1 1 (1961) 64–77. R. E Caflisch, W. Morokoff, A. B. Owen. Valuations of mortgage backed securities using Brownian bridges to reduce effective dimension. Journal of Computational Finance, 1 (1997) 27–46. S. Capstick, B. D. Keister. Multi dimensional Quadrature Algorithms at Higher Degree and/or Dimension, www.scri.fsu.edu/ ~capstick /papers/quad.ps. I. Dimov, A. Karaivanova, R. Georgieva, S. Ivanovska. Parallel Importance Separation and Adaptive Monte Carlo Algorithms for Multiple Integrals, In: I. Dimov, I. Lirkov, S. Margenov, Z. Zlatev Eds., Numerical Methods and Applications, Lecture Notes in Computer Science, 2542, Springer Verlag (2003) 99–107. M. Drmota, R. F. Tichy. Sequences, Discrepancies and Applications, Lecture Notes on Mathematics, Springer, Berlin, 1997, N 1651. K. Entacher, A. Uhl, S. Wegenkittl. Linear congruential generators for parallel Monte Carlo: the leap-frog case, Monte Carlo Meth. Appl., 4 (1998) 1–16. L. Kuipers, H. Niederreiter. Uniform distribution of sequences, John Wiley & sons, New York, 1974. N. Kjurkchiev, Bl. Sendov, A. Andreev. Numerical Solution of Polynomial Equations, P.G. Ciarlet and J. L. Lions (Eds.), Handbook of Numerical Analysis, Solution of Equations in (Part 2), 3, North-Holland, Amsterdam, NY, 1994. C. Van Loan, G. Golub. Matrix Computations, The John Hopkins University Press, Baltimore and London, third edition, 1996. A. B. Owen. Scrambled Net Variance for Integrals of Smooth Functions, Annals of Statistics, 25 4 (1997) 1541–1562. A. B. Owen. The dimension distribution and quadrature test functions, wwwstat.stanford.edu/ ~owen/reports, November, 2001.
A New Quasi-Monte Carlo Algorithm
135
14. W. Ch. Schmid, A. Uhl. Techniques for parallel quasi-Monte Carlo integration with digital sequences and associated problems, Math. and Comp. in Sim., 55 (2001) 249–257. 15. R. Schuerer. A comparison between (quasi-)Monte Carlo and cubature rule based method for solving high-dimensional integration problems, Math. and Comp. in Sim., 62 3–6 (2003) 509–517. 16. I. H. Sloan, H. Wozniakowski. When are quasi-Monte Carlo Algorithms efficient in high-dimensional integration, Journal of Complexity, 14 (1998) 1–33. 17. I. M. Sobol' . Monte Carlo Methods, Nauka, Moscow, 1973. 18. I. M. Sobol' , D. I. Asotsky. One more experiment on estimating high-dimensional integrals by quasi-Monte Carlo methods, Math. and Comp. in Sim., 62 3–6 (2003) 255–275. 19. http://www.math.wsu.edu /math / faculty /genz / homepage, Software / MULTST. 20. M. D. Takev. On Probable Error of the Monte Carlo Method for Numerical Integration, Mathematica Balkanica (New Series), 6 (1992) 231–235.
Monte Carlo Method for Multiple Knapsack Problem Stefka Fidanova CLPP – BAS, Acad. G. Bonchev, bl.25A, 1113 Sofia, Bulgaria
[email protected] Abstract. This paper describes Monte Carlo (MC) method for Multiple Knapsack Problem (MKP). The MKP can be defined as economical problem like resource allocation and capital budgeting problems. The Ant Colony Optimization (ACO) is a MC method, created to solve Combinatorial Optimization Problems (COPs). The paper proposes a Local Search (LC) procedure which can be coupled with the ACO algorithm to improve the efficiency of the solving of the MKP. This will provide optimal or near optimal solutions for large problems with an acceptable amount of computational effort. Computational results have been presented to assess the performance of the proposed technique.
1 Introduction There are many NP-hard combinatorial optimization problems for which it is impractical to find an optimal solution. Among them is the MKP. For such problems the reasonable way is to look for algorithms that quickly produce near-optimal solutions. ACO [1, 3, 2] is a MC method, created to solve COPs. It is a metaheuristic procedure for quickly and efficiently obtaining high quality solutions to complex optimization problems [7]. ACO algorithm can be interpreted as parallel replicated Monte Carlo systems [10]. MC systems [8] are general stochastic simulation systems, that is, techniques performing repeated sampling experiments on the model of the system under consideration by making use of a stochastic component in the state sampling and/or transition rules. Experimental results are used to update some statistical knowledge about the problem, as well as the estimate of the variables the researcher is interested in. In turn, this knowledge can be also iteratively used to reduce the variance in the estimation of the described variables, directing the simulation process toward the most interesting state space regions. Analogously, in ACO algorithms the ants sample the problem’s solution space by repeatedly applying a stochastic decision policy until a feasible solution of the considered problem is built. The sampling is realized concurrently by a collection of differently instantiated replicas of the same ant type. Each ant “experiment” allows to adaptively modify the local statistical knowledge on the problem structure. The recursive retransmission of such knowledge by means of stigmergy determines a reduction in the variance of the whole search process the so far most interesting explored transitions probabilistically bias future search, preventing ants to waste resources in not promising regions of the search. I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 136–143, 2004. © Springer-Verlag Berlin Heidelberg 2004
Monte Carlo Method for Multiple Knapsack Problem
137
The ACO algorithms were inspired by the observation of real ant colonies. Ants are social insects, that is, insects that live in colonies and whose behavior is directed more to the survival of the colony as a whole then to that of a single individual component of the colony. An important and interesting of ant colonies is how ants can find the shortest path between food sources and their nest. ACO is the recently developed, population-based approach which has been successfully applied to several NP-hard COPs [4]. One of its main ideas is the indirect communication among the individuals of a colony of agents, called “artificial” ants, based on an analogy with trails of a chemical substance, called pheromone which real ants use for communication. The “artificial” pheromone trails are a kind of distributed numerical information which is modified by the ants to reflect their experience accumulated while solving a particular problem. When construct a solution, at each step ants compute a set of feasible moves and select the best according to some probabilistic rules. The transition probability is based on the heuristic information and pheromone trail level of the move (usability of the move in the past). The rest of the paper is organized as follows: Section 2 describes the general framework for MKP as a COP. The search strategy of the local search method is explained in Section 3. Section 4 outlines the implemented ACO applied to MKP. In Section 5 experimental results over test problems are shown. Finally some conclusions are done.
2
Formulation of the Problem
The MKP has received wide attention from operation research community, because it embraces many practical problems. Applications include resource allocation in distributed systems, capital budgeting and cutting stock problems. In addition, MKP can be seen as a general model for any kind of binary problems with positive coefficients [6, 5]. The MKP can be thought as a resource allocation problem, where we have resources (the knapsacks) and objects. The object has a profit Each resource has its own budget (knapsack capacity) and consumption of resource by object We are interested in maximizing the profit, while working with a limited budget. The MKP can be formulated as follows:
There are constraints in this problem, so MKP is also called knapsack problem. Let and with for all A well-stated MKP assumes that and for all and Note that the matrix and vector are both non-negative.
138
Stefka Fidanova
In the MKP we are not interested in solutions giving a particular order. Therefore a partial solution is represented by and the most recent elements incorporated to S, need not to be involved in the process for selecting the next element. Moreover, solutions for ordering problems have a fixed length as we search for a permutation of a known number of elements. Solutions for MKP, however, do not have a fixed length. We define the graph of the problem as follows: the nodes correspond to the items, the arcs fully connect nodes.
3
The Local Search Strategy
The LS method (move-by-move method) [9] perturbs a given solution to generate different neighborhoods using a move generation mechanism. In general, neighborhoods for large-sized problems can be much complicated to search. Therefore, LS attempts to improve an initial solution by a series of local improving changes. A move-generation is a transition from a solution S to another one in one step. These solutions are selected and accepted according to some predefined criteria. The returned solution may not be optimal, but it is the best solution in its local neighborhood V(S). A local optimal solution is a solution with the local maximal possible cost value. Knowledge of a solution space is the essential key to more efficient search strategies. These strategies are designed to use this prior knowledge and to overcome the complexity of an exhaustive search by organizing searches across the alternatives offered by a particular representation of the solution. The main purpose of LS implementation is to speed up and improve the solutions constructed by the ACO. A practical LS strategy that satisfies the MKP requirements has been developed and coupled with ACO algorithm. The MKP solution can be represented by string with 0 for not chosen items and 1 for chosen items. In our LS procedure we will replace one of the 1 with 0 and one of the 0 with 1. The change in cost is completed and the swap is accepted or rejected according to the acceptance strategy.
4 ACO Algorithm for MKP Real ants foraging for food lay down quantities of pheromone (chemical cues) marking the path that they follow. An isolated ant moves essentially at random but an ant encountering a previously laid pheromone will detect it and decide to follow it with high probability and thereby reinforce it with a further quantity of pheromone. The repetition of the above mechanism represents the auto-catalytic behavior of real ant colony where the more the ants follow a trail, the more attractive that trail becomes. The ACO algorithm uses a colony of artificial ants that behave as co-operative agents in a mathematic space were they are allowed to search and reinforce pathways (solutions) in order to find the optimal ones. After initialization of the pheromone trails, ants construct feasible solutions, starting from random
Monte Carlo Method for Multiple Knapsack Problem
139
nodes, then the pheromone trails are updated. At each step ants compute a set of feasible moves and select the best one (according to some probabilistic rules) to carry out the rest of the tour. The transition probability is based on the heuristic information and pheromone trail level of the move. The higher value of the pheromone and the heuristic information, the more profitable is to select this move and resume the search. In the beginning, the ant do feasible moves and select the best one (according to some probabilistic rules) to carry out the rest of the tour. In the beginning, the initial pheromone level is set to a small positive constant value and then ants update this value after completing the construction stage. ACO algorithms adopt different criteria to update the pheromone level. In our implementation we use Ant Colony System (ACS) [3] approach. In ACS the pheromone updating stage consists of:
4.1
Local Update Stage
While ants build their solution, at the same time they locally update the pheromone level of the visited paths by applying the local update rule as follows:
Where is an amount of the pheromone on the arc of the graph of the problem, is a persistence of the trail and term can be interpreted as trail evaporation. The aim of the local updating rule is to make better use of the pheromone information by dynamically changing the desirability of edges. Using this rule, ants will search in wide neighborhood of the best previous solution. As shown in the formula, the pheromone level on the paths is highly related to the value of evaporation parameter The pheromone level will be reduced and this will reduce the chance that the other ants will select the same solution and consequently the search will be more diversified.
4.2
Global Updating Stage
When all ants have completed their solution, the pheromone level is updated by applying the global updating rule only on the paths that belong to the best solution since the beginning of the trail as follows:
where
is the cost (profit) of the best solution from the beginning. This global updating rule is intended to provide a greater amount of pheromone on the
140
Stefka Fidanova
paths of the best solution, thus intensifying the search around this solution. The transition probability to select the next item is given as:
where is a pheromone level on the arc is the heuristic and is the set of remaining feasible items. Thus the higher the value of the more profitable it is to include item in the partial solution. Let For heuristic information we use:
and
where and are parameters. Hence the objects with greater profit and less average expenses will be more desirable.
5
Experimental Results
This section reports on the computational experience of the ACS coupled with local search procedures using large MKP from “OR-Library” available within WWW access at http://mscmga.ms.ic.ac.uk/jeb/orlib. To provide a fair comparison for the above implemented ACS algorithm, a predefined number of iterations, K=500, is fixed for all the runs. The developed technique has been coded on C + + language and implemented on a Pentium 3 (450 MHz). Because of the random start of the ants we can use less ants then the number of the nodes (objects). After the tests we found that 10 ants are enough to achieve good results. Thus we decrease the running time of the program. The results on the Figures show the advantage to use local search procedures. The reported results are an average over 10 runs of the program. The chosen LS strategy appears to be very effective for MKP. For all tested problems our algorithm achieved, in some of the runs, the best found results in the literature. In all tested problems achieved results using LS procedure are better than without local search.
6
Conclusion
In this paper a local search procedure have been developed. The comparison of the performance of the ACS coupled with this procedure applied to different MKP problems are reported. The results obtained are encouraging and the ability of the developed technique to generate rapidly high-quality solutions for MKP can be seen. For future work another important direction for current research is to try different strategies to explore the search space more effectively and provide good results.
Monte Carlo Method for Multiple Knapsack Problem
141
Fig. 1. The figures show the average solution quality per iteration over 10 runs
Acknowledgments Stefka Fidanova was supported by the European Community program “Center of Excellence” under contract number No ICA1-2000-70016.
References 1. Dorigo, M., Di Caro, G.: The Ant Colony Optimization metaheuristic. In: Corne,
D., Dorigo, M., Glover, F.(eds.): New Idea in Optimization, McGrow-Hill (1999) 11–32. 2. Dorigo, M., Di Caro, G., Gambardella, L. M.: Ant Algorithms for Distributed Discrete Optimization. J of Artificial Life 5 (1999) 137–172.
142
Stefka Fidanova
Fig. 2. The figures show the average solution quality per iteration over 10 runs
3. Dorigo, M., Gambardella, L.M.: Ant Colony System: A cooperative Learning Ap-
4. 5. 6.
7.
proach to the Traveling Salesman Problem. IEEE Transaction on Evolutionary Computation 1 (1999) 53–66. Gambardella, M. L., Taillard, E. D., Dorigo, M.: Ant Colonies for the QAP. J of Oper. Res. Soc. 50 (1999) 167–176. Kochenberger, G., McCarl, G., Wymann, F.: A Heuristic for General Integer Programming. J of Decision Sciences 5 (1974) 34–44. Leguizamon, G., Michalevich, Z.: A New Version of Ant System for Subset Problems In: Proceedings of Int. Conf. on Evolutionary Computations, Washington (1999). Osman, I. H., Kelley, J. P.: Metaheuristic: An Overview. In: Osman, I. H., Kelley, J.P. (eds.), Metaheuristic: Theory and Applications, Kluwer Academic Publishers (1996).
Monte Carlo Method for Multiple Knapsack Problem
143
8. Rubinstein, R. Y.: Simulation and the Monte Carlo Method. John Wiley & Sons
(1981). 9. Schaffer, A. A., Yannakakis, M.: Simple Local Search Problems that are Hard to
Solve. Society for Industrial Applied Mathematics Journal on Computing, Vol 20 (1991) 56–87. 10. Strelsov, S., Vakili, P.: Variance Reduction Algorithms for Parallel Replicated Simulation of Uniformized Markov chains. J. of Discrete Event Dynamic Systems: Theory and Applications, 6(1996) 159–180.
Importance Separation for Solving Integral Equations Rayna Georgieva and Sofiya Ivanovska CLPP - Bulgarian Academy of Sciences Acad. G. Bonchev, Bl.25A, 1113 Sofia, Bulgaria {rayna,sofia}@copern.bas.bg
Abstract. In this paper we describe and study importance separation Monte Carlo method for integral equations. Based on known results for integrals, we extend this method for solving integral equations. The method combines the idea of separation of the domain into uniformly small subdomains (adaptive technique) with the Kahn approach of importance sampling. We analyze the error and compare the results with the crude Monte Carlo method.
1 Introduction An important advantage of Monte Carlo methods (MCMs) is that they allow to find directly the unknown functional of the solution of integral equations with a number of operations necessary to calculate the solution of an integral equation only in one point of the domain [11]. Much of the efforts to improve MCM are in construction of variance reduction methods which speed up the computation (see [2]). In [6, 8, 9] such method for numerical integration called importance separation (IS) is presented and studied. This method has the best possible rate of convergence for a certain class of functions (see [1]), but its disadvantage is that it gives better accuracy only for low dimensions. The problem for calculation of a linear functional of the solution of an integral equation can be transformed into approximate evaluation of a finite number of integrals (linear functionals of iterative functions) [6]. We apply the IS method for each of the integrals and extend the study of its properties. We also study the conditions under which the IS method has an advantage over other methods. We compare these results with Crude Monte Carlo method (CrMCM) for integrals and Monte Carlo method for integral equations (MCM-int.eq.) (see [11]).
2
Formulation of the Problem
Consider the Fredholm integral equation of the second kind:
or I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 144–152, 2004. © Springer-Verlag Berlin Heidelberg 2004
Importance Separation for Solving Integral Equations
145
are given functions and is an unknown function, is a bounded domain). We are interested in Monte Carlo method for evaluation of linear functionals of the solution of the following type:
It is assumed that for solving integral equations:
We can apply successive approximation method
where It is known that the condition is a sufficient condition for convergence of the Neumann series. Thus, when this condition is satisfied, the following statement holds:
Therefore,
An approximation of the unknown value Neumann series (2) for sufficiently large
can be obtained using a truncated
So, we transform the problem for solving integral equations into a problem for approximate evaluation of a finite number of multidimensional integrals. We will use the following denotation where is a value, obtained after integration over It is obvious that the calculation of the estimate can be replaced by evaluation of a sum of linear functionals of iterative functions of the following type which can be presented as:
where function
If we denote by
the integrand
146
Rayna Georgieva and Sofiya Ivanovska
then we will obtain the following expression for (3):
So, from now on we will consider the problem for approximate calculation of multiple integrals of the type (4). We will first review briefly the most widely used Monte Carlo methods for integrals and integral equations. It is well-known that Monte Carlo methods reduce the problem to the approximate calculation of mathematical expectation which coincides with the unknown functional defined by (1).
3 3.1
Monte Carlo Methods Crude Monte Carlo Method for Integrals
The Monte Carlo quadrature formula is based on the probabilistic interpretation of an integral. If is an uniformly distributed sequence in G, then the Monte Carlo approximation to the integral is
with integration error volume of G.
3.2
where V(G) is the
Monte Carlo Method for Integral Equations
It is known (see [11]) that the approximate calculation of linear functionals of the solution of an integral equation brings to the calculation of a finite sum of linear functionals of iterative functions First, in we construct a random trajectory (Markov chain) of length starting in state
according to the initial and transition probabilities. The functions and satisfy the requirements for non-negativeness, to be acceptable to function and the kernel respectively and for any From the above supposition on the kernel and the well-known fact that
Importance Separation for Solving Integral Equations
it follows that the corresponding Monte Carlo estimation of
147
is:
Therefore, the random variable can be considered as an estimate of the desired value for sufficiently large with a statistical error of order where N is the number of chains and is the value of taken over the n-th chain. It is very important that the same trajectories of the type can be used for approximate evaluation of for various functions Furthermore, they can be used for various integral equations with the same kernel but with different right-hand side
3.3
Importance Separation for Integrals
The importance separation is a Monte Carlo method which combines the idea of separation of the domain of integration into uniformly small subdomains (stratification, [5]) and the Kahn approach to implement more samples in those subdomains where the integrand is large (importance sampling for integrals, [7], and for integrals equations, [4,10]). This method has the best rate of convergence for the class of functions with bounded derivatives. One approach how to make a partition of the given domain into subdomains was studied in [9] where the problem for evaluation of the integral is considered. The suggested there partition scheme of the domain into M subintervals (one-dimensional case) is the following one:
It is known (see [11]) that where
and
is a random point in the
subdomain of G.
148
Rayna Georgieva and Sofiya Ivanovska
In the general case of multidimensional integrals integration error (the probable error) holds [9]:
the following
where is the dimension of the domain of integration, M is the number of subdomains, the integrand is a positive function which belongs to This means that is continuous on G with partially continuous first derivatives and
The constants and the vectors of constants are determined from the requirement the subdomains have to be uniformly small in probability and in geometrical size, and it is also assumed that From (6) it is clear that the error of the importance separation method which has the order asymptotically goes to for large dimensions This estimation of integration error shows that importance separation can be considered as a good method for approximate calculation of integrals only if is not very large. Therefore when we translate this conclusion in the terms of integral equation, it means that the von Neumann series has to converge quickly.
4
Description of the Algorithm
In the application of importance separation for multidimensional integrals various approaches can be used. If we use the following simplified scheme: the number of subintervals and the points of partition along all axis directions are the same; all components of any point of the partition are identical by reason of using and the formula
for each of components of the next point An essential step of the importance separation realization is the partition into subdomains. The idea, subdomains to be of identical measure (volume), is used as a base for obtaining the points of partition. We will demonstrate some requirements imposed on the integrand for one-dimensional case. The first and the last point of the partition are fixed - they coincide with the bounds of the
Importance Separation for Solving Integral Equations
149
considered domain (interval). The following equation is used to obtain the point
The trapezoid rule is applied to the integral at the left-hand side of the above equation and the left rectangle rule at the right-hand side. This approach of partition cannot be used for functions which vanish in some regions because of using the value of integrand at the left endpoint of the interval. On the other hand, the formula (5) will be applicable if the following inequality is fulfilled:
This inequality can be transformed to a condition for the number of subintervals M:
It is obvious that M will become too large when is close to zero for some Like it is mentioned before, the importance separation method contains the idea to generate more samples in the regions where the integrand has larger values. So, IS is appropriate to integrands that have regions of different significance. On the other hand the formula (7) shows that if the ratio of the integrand values at the right endpoint and at some interior point of the interval is large, M will increase too. For example, if in multidimensional case the number of subintervals on each axis direction is 10 and (the dimension of the domain of integration) one has to perform realizations of the random variable. The method cannot be realized in fewer samples because it is necessary to pick up at least one point in every subdomain.
5
Error Analysis
We consider the problem for approximate calculation of linear functional of the solution of an integral equation Let us denote the iterative approximation of by and the Monte Carlo approximation by It is known that two kinds of error, systematic (a truncation error) and stochastic (a probabilistic one), are characteristic for iterative Monte Carlo methods. If is a given sufficiently small positive parameter then
where is the truncation error, is the probabilistic error. We can obtain a lower bound for the number of iterations using [3]:
150
Rayna Georgieva and Sofiya Ivanovska
where is the initial approximation of inequality it is easy to show that
Using the Cauchy-Bunyakowski
The second multiplier can be estimated:
Taking the limit for
It holds if the condition
6
one can obtain
is fulfilled.
Numerical Experiments
We present the numerical results (accuracy and CPU-time) for the considered algorithm, importance separation, applied to calculate a linear functional of the solution of an integral equation. We use the following integral equation as a test example:
This kind of equation describes some neuron networks procedures. We are interested in an approximate calculation of where The results are presented as a function of the number of iterations (length of Markov chain) and N, the number of samples (only one sample of N random points is used for the presented results). The importance separation algorithm is constructed so that only one sample of the random variable is chosen in every subdomain. The number of iterations is fixed, but it has been chosen in advance according to the of the kernel (8). For the approximate computation of any integral different number of samples are used in order to have error balancing. The results for importance separation are compared with CrMCM and MCM-int.eq. and they are presented in Table 2 and Table 1. It is known that the importance separation is a variance reduction method. Table 2 illustrates this property with the fact that the importance separation method leads to smaller relative error in comparison with CrMCM for fixed number of samples. From the estimation (6) it is seen that the acceleration of the convergence for large is insignificant. The CPU-time (see Table 1) for importance
Importance Separation for Solving Integral Equations
151
separation exceeds slightly the CPU-time for crude MCM because of the additional number of calculations necessary for the partition although the number of samples is the same for both methods. The CPU-time of the MCM-int.eq. is several times larger than the corresponding CPU-time of the IS method. The reason for that is the use of the acceptance-rejection method for modeling using initial probability density (in general case MCM-int.eq. needs modeling with transition probability densities too). However, when the inverse function method is used for simulation, the CPU-time of IS can exceed several times the CPUtime of MCM-int.eq. The reason is that one has to use a larger number of calls of pseudorandom generator for IS method (the random points belong to than for MCM-int.eq. (the random points belong to
7
Conclusion
Importance separation gives better accuracy than CrMCM and MCM-int.eq. when the von Neumann series converges rapidly, i.e. when the number of iterations and the dimension of the integrals are not
152
Rayna Georgieva and Sofiya Ivanovska
very large. CrMCM can have priority in accuracy over IS when the dimension increases. This can be explained by the fact that for large dimension the error of IS method asymptotically goes to which corresponds to the error of CrMC method.
Acknowledgments The authors would like to express their gratitude to Prof. Ivan Dimov and Dr. Aneta Karaivanova for the helpful comments and suggestions. This work is supported by Center of Excellence BIS-21 Grant ICA1-200070016 and by the Ministry of Education and Science of Bulgaria under Grants # I 1201/02 and # MM 902/99.
References 1. N. S. Bahvalov. On the optimal estimations of convergence of the quadrature processes and integration methods. Numerical Methods for Solving Differential and Integral Equations, Nauka, Moskow, 5-63, 1964 (in Russian). 2. R. E. Caflisch. Monte Carlo and quasi-Monte Carlo methods. Acta Numerica, 7, 1-49, 1998. 3. J. H. Curtiss. Monte Carlo methods for the iteration of linear operators, J. Math Phys., 32, 209-232, 1954. 4. I. Dimov. Minimization of the probable error for some Monte Carlo methods, Mathematical Modelling and Scientific Computations, Andreev, Dimov, Markov, Ulrich (Eds.), Bulgarian Academy of Sciences, Sofia, 159-170, 1991. 5. V. Dupach. Stochasticke pocetni metody, Cas. pro pest. mat. 81(1), 55-68, 1956. 6. I. Dimov. Efficient and overconvergent Monte Carlo methods. Parallel algorithms, Advances in Parallel Algorithms, I.Dimov, O.Tonev (Eds.), IOS Press, Amsterdam, 100–111, 1994. 7. H. Kahn. Random sampling (Monte Carlo) techniques in neutron attenuation problems, Nucleonics 6(5), 27-33, 1950; 6(6), 60-65, 1950. 8. A. Karaivanova. Adaptive Monte Carlo methods for numerical integration, Mathematica Balkanica, 11, 391–406, 1997. 9. A. Karaivanova and I. Dimov. Error analysis of an adaptive Monte Carlo method for numerical integration, Mathematics and Computers in Simulation, 47, 201–213, 1998. 10. G. A. Mikhailov. Optimization of the “weight” Monte Carlo methods, Nauka, Moskow, 1987. 11. I. M. Sobol. Monte Carlo Numerical Methods, Nauka, Moscow, 1973 (in Russian).
A Parallel Monte Carlo Method for Electron Quantum Kinetic Equation* Todor V. Gurov and Ivan T. Dimov CLPP – BAS, Acad. G. Bonchev, bl. 25A, 1113 Sofia, Bulgaria
[email protected],
[email protected]
Abstract. We study a parallel Monte Carlo (MC) method for investigation of a quantum kinetic equation which accounts for the action of the electric field during the process of electron-phonon interaction. Optimization of the presented parallel algorithm is done using variance reduction techniques and parallel random sequences from the Scalable Parallel Random Number Generator (SPRNG) library. The developed code written in C is parallelized with MPI and OpenMP codes. Numerical results for the parallel efficiency of the algorithm are obtained. The dependence of the electron energy distribution on applied electric field is investigated for long evolution times. The distribution function is computed for GaAs material parameters.
1 Introduction The development and application of the MC methods for quantum transport in semiconductors and semiconductor devices has been initiated during the last decade [1,2]. The stochastic approach relies on the numerical MC theory as applied to the integral form of the generalized electron-phonon Wigner equation. An equation for reduced electron-phonon Wigner function which accounts for electron-phonon interaction has been recently derived [3]. For a bulk semiconductor with an applied electric field the equation resembles the Levinson equation [4], or equivalently the Barker-Ferry (B-F) equation [5] with infinite electron lifetime. A crude MC method has been proposed to find quantum solutions up to 200 femtoseconds (fs) evolution times of the B-F equation at zero temperature [6]. It is proved [7] that stochastic error has order where is the evolution time, N is the number of samples of the MC estimator, and is a constant depending on the kernel of the quantum kinetic equation. This estimate shows that when is fixed and the error decreases, but for N fixed and large the factor for the error looks ominous. Therefore, the problem of estimating the electron energy distribution function for long evolution times with small stochastic error requires combining both MC variance reduction techniques and distributed or parallel computations. *
Supported by the European Commission through grant number HPRI-CT-1999-
00026 and by Center of Excellence BIS-21 grant ICA1-2000-70016, as well as by the NSF of Bulgaria through grant number I-1201/02. I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 153–161, 2004. © Springer-Verlag Berlin Heidelberg 2004
154
Todor V. Gurov and Ivan T. Dimov
In this paper a parallel MC method for solving the B-F equation is studied. The transition density function for the Markov chain is chosen to be proportional to the contribution from the kernels. The integral parts of the kernels are estimated using MC integration. An importance sampling technique is introduced to reduce the variance in the MC quadrature. A new rule for sampling the transition density function of the Markov chain is used to construct the MC estimator. In this way, we avoid the acceptance-rejection techniques used [6]. The parallelisation of the MC algorithm is done using MPI and OpenMP codes. All these improvements in the MC approach lead to a decrease of the computational complexity of the algorithm and allow to estimate the electron energy distribution function for long evolution times.
2
The Quantum Kinetic Equation
The quantum kinetic equation accounting for the electron-phonon interaction in presence of applied electric field can be written in the following integral form [6]:
where the kernel is separated in two terms:
Here, k and are the momentum and the evolution time, respectively, is the distribution function. is the initial electron distribution function. where E is the applied electric field. is the Bose function, where is the Boltzmann constant and T is the temperature of the crystal, corresponds to an equilibrium distributed phonon bath. is the phonon energy which generally depends on and is the electron energy. A Fröhlich coupling is considered
where and are the optical and static dielectric constants. The damping factor is considered independent of the electron states k and This is
A Parallel Monte Carlo Method for Electron Quantum Kinetic Equation
155
reasonable since weakly depends on k and for states in the energy region above the phonon threshold, where the majority of the electrons reside due to the action of the electric field.
3
Monte Carlo Approach
Consider the problem for evaluating the following functional
by a MC method. Suppose the distribution function and the arbitrary function belong to any Banach space X and the adjoint space X*, respectively. The wave vector k belongs to a finite domain and The value of at a fixed point is provided by the special case Since the Neumann series of the integral equation (3) converges [7] the solution can be evaluated by a MC method. Define a terminated Markov chain such that as is the truncation parameter). All points are sampled using an arbitrary transition density function which is tolerant1 of both kernels in equation (1). The biased backward MC estimator for the solution of (1) at the fixed point has the following form:
where
Here is a MC estimator for in the transition at the Markov chain. The probabilities chosen to be proportional to the absolute values of the kernels. Now we can define a Monte Carlo method
are
where means stochastic convergence as is the iterative solution obtained by the Neumann series of (1). The relation (5) still does not determine 1
is tolerant of
if
when
and
when
156
Todor V. Gurov and Ivan T. Dimov
the computational algorithm. The sampling rule, which compute the next point at the Markov chain, has to be specified by using random number generators. In order to avoid the singularity in (2) the following transition density function is suggested: where In spherical coordinates the function is chosen in the following way: where and is distance in the direction of the unit vector from k to the boundary of the domain G. If G is a sphere with radius Q, the function satisfies the condition for a transition density. Indeed,
Suppose the direction of the field E parallel to the Then the unit vector can be sampled in the xz-plane (or because of the symmetry of the task around the direction of the field. Thus, if we know the wave vector k the next state can be computed by the following sample rule: Algorithm 1: as and 1. Sample a random unit vector where is an uniformly distributed number in (0, 1); 2. Calculate where means a scalar product between two vectors; 3. Sample where is an uniformly distributed number in (0, 1); 4. Calculate In order to evaluate in the fixed point N random walks of the MC estimator can be computed by the following algorithm: Algorithm 2: 1. Choose a positive small number
and set initial values
2. Sample a value with a density function 3. Sample the next state using Algorithm 1; 4. Sample independent random values of with a density function
5. Calculate
A Parallel Monte Carlo Method for Electron Quantum Kinetic Equation
157
uniformly distributed random variable in (0,1); 6. Choose a value If then
else
and repeat from step 2 until 7. Set 8. Repeat N times steps 1 – 7 and estimate the electron energy distribution function by Eq.(5).
4
Parallel Implementation and Numerical Results
The computational complexity of the obtained Algorithm 2 can be measured by the quantity The number of the random walks, N, and the average number of transitions in the Markov chain, are connected with stochastic and systematic errors [7]. The mean time for modeling one transition, depends on the complexity of the transition density functions and on the sampling rule, as well as on the choice of the random number generator (rng). It is well known that the MC algorithms are very convenient for parallel implementations on parallel computer systems [8], because every realization of the MC estimator can be done independently and simultaneously. Although MC algorithms are well suited to parallel computation, there are a number of potential problems. The available computers can run at different speeds; they can have different user loads on them; one or more of them can be down; the rng’s that they use can run at different speeds; ets. On the other hand, these rng’s must produce independent and non-overlapping random sequences. Thus, the parallel realization of the MC algorithms is not a trivial process on different parallel computer systems. In our research, the Algorithm 2 has been implemented in C and has been parallelized using an MPI code. The numerical tests have been performed on Sunfire 6800 SMP system with twenty-four 750 MHz UltraSPARC-Ill processors located at the Edinburgh Parallel Computer Centre (EPCC). The SPRNG library has been used to produce independent and non-overlapping random sequences [9]. Our aim is to estimate the electron energy distribution function for evolution times greater than 200 fs. That is way, our parallel implementation with processors includes the following strategy. A master-slave model is used where the master processor delegates work random walks) to the other processors. The slave processors complete the required work and obtain “local” MC estimates. After that they return the results back to the master processor which produces the “global” MC estimate. Such a parallel strategy is expected to give linear speed-up and high parallel efficiency. The results in Table 1 confirm this assumption in general. In addition, an MPI/OpenMP mixed code has
158
Todor V. Gurov and Ivan T. Dimov
been developed for parallel implementation of the Algorithm 2. Such mixed mode code has been proposed in [10] as a more efficient parallelisation strategy to perform diffusion Monte Carlo calculations for an SMP (Share Memory Programming) cluster.
The results in Table 1 demonstrate that this style of programming is not always the most efficient mechanism on SMP systems and cannot be regarded as the ideal programming model for all codes. The numerical results discussed in Figures 1–4 are obtained for zero temperature and GaAs material parameters: the electron effective mass is 0.063, the optimal phonon energy is 36meV, the static and optical dielectric constants are and The initial condition at is given by a function which is Gaussian in energy, and scaled in a way to ensure, that the peak value is equal to unity. A value is chosen for a radius of integration domain G. The solution is estimated in 2 × 96 points that are symmetrically located on the direction of applied field. The truncation parameter The quantity presented on the in all figures is i.e., it is proportional to the distribution function multiplied by the density of states. It is given in arbitrary units. The quantity given on the in units of is proportional to the electron energy. The results for the computational cost of the Algorithm 2 are obtained and are compared with the algorithm from [6]. To obtain a smooth solution in the case when the Algorithm 2 needs approximately 9 minutes per point on one processor while the algorithm presented in [6] needs approximately 30 minutes. These results and the parallelisation of the Algorithm 2 allow to estimate the solution of (1) with high accuracy for 300 f s evolution time. The results for the electron energy distribution are presented on Figures 1-4 for 200 f s and 300 f s evolution times. The relaxation leads to a time-dependent
A Parallel Monte Carlo Method for Electron Quantum Kinetic Equation
159
Fig. 1. Solutions versus at positive direction on the and The electric field is 0, 6 kV/cm, and 12 kV/cm and the number of random walks per point is 1 million.
Fig. 2. Solutions versus at positive direction on the and The electric field is 0, 6kV/cm, and 12 kV/cm and the number of random walks per point is 24 millions.
160
Todor V. Gurov and Ivan T. Dimov
Fig. 3. Solutions versus at negative direction on the and The electric field is 0, 6 kV/cm, and 12 kV/cm and the number of random walks per point is 1 million.
Fig. 4. Solutions versus at negative direction on the and The electric field is 0, 6 kV/cm, and 12 kV/cm and the number of random walks per point is 24 millions.
A Parallel Monte Carlo Method for Electron Quantum Kinetic Equation
161
broadening of the replicas. The presented solutions on Figures 1 and 2 are along the electric field and the replicas are shifted to the right by the increasing electric field. Figures 3 and 4 show the solutions in direction opposite to the field and the replicas are shifted to the left. Also, we see on Figures 2 and 4 that second peak is appeared on the left of the initial condition and the replicas begin to shift at the same way with the increase of the electric field. The solution in the classically forbidden region (see Figures 3 and 4), on the right of the initial condition, demonstrates enhancement of the electron population with the growth of electric field. The numerical results show that the intra-collisional field effect is well demonstrated for 300 f s evolution time of the electron-phonon relaxation. Conclusions. A parallel MC algorithm for solving the B-F equation in presence of applied electric field is presented. A new transition density for the Markov chain and an algorithm described the sample rule are suggested. The MC algorithm has low complexity in comparison with the algorithm from [6]. MPI/OpenMP mixed mode code is developed and is compared with pure MPI performance. The numerical results show that the pure MPI performance is preferable for large-scale MC simulations in order to investigate the quantum kinetic equation under consideration.
References 1. R. Rossi, C. Jacoboni, M. Nedjalkov, A Monte Carlo Solution of the Wigner trans-
port Equation, Semiconductor Science Technology. 9, p.934, 1994. 2. M. Nedjalkov, I. Dimov, F. Rossi, C. Jacoboni, J. Math. Comp. Mod. 23 (8/9),
159-166, 1996. 3. M. Nedjalkov, H. Kosina, R. Kosik, and S. Selberherr, A Wigner Equation with
Quantum Electron-Phonon Interaction, J. of Microel. Eng.63 (1/3), 199-203, 2002. 4. J. Rammer, Quantum transport theory of electrons in solids: A single-particle
approach, Reviews of Modern Physics. 63 (4), 781-817, 1991. 5. J. Barker, D. Ferry, Self-scattering path-variable formulation of high field time-
6.
7.
8. 9. 10.
dependent quantum kinetic equations for semiconductor transport in the finitecollision-duration regime. Physical Review Letters. 42 (26), 1779-1781, 1979. N. Nedjalkov , T.V. Gurov, H. Kosina, and P.A. Whitlock, Statistical Algorithms for Simulation of Electron Quantum Kinetics in Semiconductors - Part II, Lect. Notes in Comp. Sci. 2179, Springer, 183-190, 2001. T.V. Gurov, P.A. Whitlock, An efficient backward Monte Carlo estimator for solving of a quantum kinetic equation with memory kernel, Math. and Comp. in Simulation. 60, 85-105, 2002. I. Dimov, O. Tonev, Monte Carlo Algorithms: Performance Analysis for Some Computer Architectures, J. of Comp. and Appl. Mathematics. 48, 253-277, 1993. Scalable Parallel Random Number Generators Library for Parallel Monte Carlo Computations, SPRNG 1.0 and SPRNG 2.0 – http://sprng.cs.fsu.edu. L. Smith and M. Bull, Development of mixed mode MPI/OpenMP applications, Scientific Programming. 9 (2/3), 83-98, 2001.
Solving BVPs Using Quasirandom Walks on the Boundary Aneta Karaivanova1,2, Michael Mascagni1, and Nikolai A. Simonov1,3 1
Florida State University Department of Computer Science, 253 Love Bldg., and School of Comp. Science and Inf. Technology, 400 DSL Tallahassee, FL 32306, USA
[email protected] http://www.cs.fsu.edu/~mascagni/ 2
CLPP - Bulgarian Academy of Sciences Acad. G. Bonchev, Bl.25A, 1113 Sofia, Bulgaria
[email protected],
[email protected] http://parallel.bas.bg/~anet/ 3
ICM&MG, Russian Academy of Sciences, Novosibirsk, Russia
[email protected],
[email protected] http://osmf.sscc.ru/LabSab/nas
Abstract. The “random walk on the boundary” Monte Carlo method has been successfully used for solving boundary-value problems. This method has significant advantages when compared to random walks on spheres, balls or a grid, when solving exterior problems, or when solving a problem at an arbitrary number of points using a single random walk. In this paper we study the properties of the method when we use quasirandom sequences instead of pseudorandom numbers to construct the walks on the boundary. Theoretical estimates of the convergence rate are given and numerical experiments are presented in an attempt to confirm the convergence results. The numerical results show that for “walk on the boundary” quasirandom sequences provide a slight improvement over ordinary Monte Carlo.
1 Introduction The “random walk on the boundary” Monte Carlo method has been successfully used for solving various boundary-value problems of mathematical physics. This method is based on classical potential theory which makes it possible to convert the original problem (boundary-value problem) into an equivalent problem (boundary integral equation). The Monte Carlo technique is then used to solve numericaly the integral equation. This method was first published in [13], and then applied to solving various boundary-value problems. The major drawback of the conventional Monte Carlo approach is the statistical rate of convergence: computational error behaves as where N is the number of random walks. Computer simulation of randomness is usually based on I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 162–169, 2004. © Springer-Verlag Berlin Heidelberg 2004
Solving BVPs Using Quasirandom Walks on the Boundary
163
the generation of a sequence of standard pseudorandom numbers that mimic the theoretical behavior of “real” random numbers. One possible way to improve the convergence is to change the type of random numbers used. Quasi-Monte Carlo methods (QMCMs) use quasirandom (also known as low-discrepancy) sequences instead of pseudorandom numbers. The quasi-Monte Carlo method for integration in has a convergence rate of approximately While numerical integration is the major application area of quasirandom sequences, there are many other problems where quasi-Monte Carlo methods also give better results than the Monte Carlo methods. There are many papers on using quasirandom numbers on differential equations. The curious reader should consult, for example, [10,3,12,8,9]. Here, we recall some basic concepts of QMCMs, [1]. First, for a sequence of N points in the half-open unit cube define
where J is a rectangular set and cies
is its volume. Then define two discrepan-
where E is the set of all rectangular subsets in and rectangular subsets in with one vertex at the origin.
is the set of all
The basis for analyzing QMC quadrature error is the Koksma-Hlawka inequality: Theorem (Koksma-Hlawka, [1]): For any sequence and any function of bounded variation (in the Hardy-Krause sense), the integration error is bounded as follows
The star discrepancy of a point set of N truly random numbers in one dimension is while the discrepancy of N quasirandom numbers in dimensions can be as low as (see, e.g. [1,11]). Most notably there are the constructions of Hammersley, Halton, Faure, and Niederreiter for producing quasirandom numbers. Description of these can be found, for example, in Niederreiter’s monograph [11]. In this paper, we present quasirandom walks on the boundary for solving some boundary-value problems. The paper is organized as follows. The formulation of the problems are given in §2. The random walk on the boundary method is described in §3. In §4 the use of quasirandom sequences is discussed, and some numerical results are presented.
Aneta Karaivanova, Michael Mascagni, and Nikolai A. Simonov
164
2
Formulation of the Problem
Consider the problem of computing the electrostatic potential, in the interior of a compact dielectric G surrounded by another dielectric media. Let be the density of this charge distribution. Then we have
and
satisfies the continuity conditions on the boundary of the domain:
Here, and are the limit values of the solution from inside and outside respectively, and are the corresponding constant dielectric permittivities. With the assumption that is smooth enough and there are only point charges inside G, it is possible to represent the solution in the form [5,4]
where
and
are the positions of the point charges.
Taking into account boundary conditions (3) and discontinuity properties of the single-layer potential’s normal derivative [5], we arrive at the integral equation for the unknown density,
which is valid almost everywhere on
Here,
and the kernel
of the integral operator
where
is the angle between the
external normal vector
is and
The free term of this equation equals
and it can be computed analytically. Since
the Neumann series
for (5) converges (see, e.g. [5,4]), and it is possible to calculate the solution as
where
Usually, however, Here,
and, hence,
is the smallest characteristic value of the operator
This means that convergence in (6) is rather slow.
Solving BVPs Using Quasirandom Walks on the Boundary
165
The situation is even worse when we consider a grounded body, G. In this case the original problem reduces to an internal Dirichlet problem for the Laplace equation:
The double-layer potential representation of its solution to the following integral equation for the unknown density,
Here,
the operator
is the adjoint of
leads
and has the same
characteristic values (see [5,4]). This means that and the Neumann series for solving (8) does not converge. To speed up the convergence in (6), and to calculate the solution of (8), we apply the method of spectral parameter substitution (see, e.g., [7], and [14,15] for Monte Carlo algorithms based on this method). This means that we consider the parameterized equation and analytically continue its solution given by the Neumann series for This goal can be achieved by substituting in its analytical expression in terms of another complex parameter, and representing as a series in powers of In this particular case, it is possible to use the substitution and hence
where
and
The rate,
of geometric
convergence of the transformed series in powers of at the point is determined by the ratio of and Here, are characteristic values of (and ) and L = 1 [5,7]. Given a desired computational accuracy, we can calculate the number of terms needed in (9). Thus, the problem reduces to computing a finite number of multidimensional integrals. The same representation is valid for the solution of the Dirichlet problem (7):
Here,
and
(see [15]).
Aneta Karaivanova, Michael Mascagni, and Nikolai A. Simonov
166
3
Random Walks on the Boundary
To construct Monte Carlo estimates for it is sufficient to calculate the integral functionals and of iterations of the reciprocally adjoint integral operators. Here, the domain of integration is Let G be a convex domain. In this case, the kernel, of corresponds to the uniform in a solid angle distribution of the point as viewed from the point This means that the most convenient way to implement the random walk on the boundary algorithm here is to use direct estimates for and adjoint estimates for [15]. Therefore
where ary
is the Markov chain of random points on the boundwith the initial density and transition density and random weights are Hence, biased estimators for
are
and
(for
the Dirichlet problem). Construction of the Markov chain, Y, is based on its geometrical interpretation. Given a point, we simulate a random isotropic direction and find the next point as the intersection of this direction with the boundary surface It is well known that different procedures can be used to choose We consider the procedure based on the direct simulation of the longitudiual angle. Normally, an acceptance-rejection method would be used. But since we plan to use quasirandom numbers, this is inadvisable (see, e.g. [6]). So we use the following algorithm: where the uniform pseudorandom numbers in the unit interval.
4
are standard
Quasirandom Walks on the Boundary
In this section we discuss how to use quasirandom numbers for solving the boundary-value problems (2), (3) and (7). To construct Monte Carlo estimates, in §2, we reformulated the original problem into the problem of solving integral equations (5) and (8). So, in order to make use of these representations when constructing quasirandom estimates, we have to refer to a Koksma-Hlawka type inequality for integral equations, [2]:
Solving BVPs Using Quasirandom Walks on the Boundary
167
where Q is a sequence of quasirandom vectors in and is the number of QRNs in one step of a random walk, T is the maximal number of steps in a single random walk, and corresponds to an estimate based on the random walk Y* generated from Q by a one-to-one map. Space precludes more discussion of the work of Chelson, but the reader is referred to the original for clarification, [2]. This inequality ensures convergence when is of bounded variation in the Hardy-Krause sense, which is a very serious limitation. But even when this condition is satisfied, the predicted rate of convergence is very pessimistic due to the high (and, strictly speaking, possibly unbounded) dimension of the quasirandom sequence in the general Monte Carlo method for solving these integral equations, (e.g. (6)). To avoid this limitation, we consider variants of the method with each random walk having fixed length. Clearly, the smaller the dimension of Q, the better the rate estimate in (12). Guided by this reasoning, we used the representations (9) and (10), and the correspondent random estimators and to construct quasirandom solutions to the original boundary-value problems. It is essential to note that despite the improved rate of convergence of our quasirandom-based calculations, constants in the error estimates are hard to calculate. On the contrary, the statistical nature of Monte Carlo solutions makes it possible to determine confidence intervals almost exactly.
5
Numerical Tests
We performed numerical tests with two problems. The first one is the Dirichlet problem for the Laplace equation inside a sphere with an exact analytic solution. To solve this problem with quasirandom sequences, we fix the length of series to be That provides a 1% bias, and use Sobol, Halton and Faure sequences. We compared the approximate value of the solution at different points computed using MCM and QMCM. The QMCM solution shows a slightly better rate of convergence for Sobol and Halton sequences.
168
Aneta Karaivanova, Michael Mascagni, and Nikolai A. Simonov
The second problem is (2), (3) with G being the unit cube with the only one unit point charge inside, To calculate defined by (4), we use the estimator and compute point values for the same order of bias (number of terms, ) and different sample number. Both tables show that we achieve a better acuracy when we replace pseudorandom numbers with quasirandom sequences. We performed numerical tests with Sobol, Halton and Faure sequences.
6
Conclusions
In this paper we presented a successful application of quasirandom sequences to the walks on the boundary method for solving boundary-value problems. The success is due to the following reason: instead of solving the original integral equation arising from integral representation of the solution, we parameterized the equation, analytically continued the solution, and used a special substitution to accelerate the convergence significantly. In this way, the problem was reduced to solving a small number of multidimensional integrals. This is the key point for the successive use of quasirandom sequences - they are designed to solve multidimensional integrals with a better rate of convergence than arises from pseudorandom numbers. We tested our approach by solving two problems using pseudorandom numbers and the Sobol, Halton and Faure sequences. The accuracy of the quasirandom walks on the boundary method is better and the advantage of this method is significant for the second test problem.
Acknowledgments This work is supported, in part, by the U.S. Army Research Office under Contract # DAAD19-01-1-0675.
Solving BVPs Using Quasirandom Walks on the Boundary
169
References 1. Caflisch, R.E.: Monte Carlo and quasi-Monte Carlo methods. Acta Numerica, 7 (1998) 1–49. 2. Chelson, P.: Quasi-Random Techniques for Monte Carlo Methods. Ph.D. dissertation, The Claremont Graduate School (1976). 3. Coulibaly, I., Lecot, C.: A quasi-randomized Runge-Kutta method. Math Comput., 68 (1999) 651–659. 4. Goursat, E.: A course in mathematical analysis. Volume III, part 2. New York, Dover Publications (1964). 5. Günter, N.M.: La theorie du potentiel et ses applications aux problemes fondamentaux de la physique mathematique. Gauthier-Villars, Paris (1934). 6. Jäckel, P.: Monte Carlo Methods in Finance. John Wiley & Sons (2002). 7. Kantorovich, L.W., Krylov, V.I.: Approximate Methods of Higher Analysis. Interscience, New York (1964). 8. Mascagni, M., Karaivanova, A.: Matrix computations using quasirandom sequences. In: Vulkov, L., Wasniewski, J., Yalamov, P., eds.: Numerical Analysis and Its Applications. Lecture Notes in Computer Science, 1988. Springer-Verlag, Berlin (2001) 552–559. 9. Mascagni, M., Karaivanova, A., Hwang, C.-O.: Quasi-Monte Carlo methods for elliptic boundary value problems. In: Proceedings of the 5th Int. Conf. on MCQMC, Singapore, November 2002 (submitted). 10. Morokoff, W., Caflisch, R.E.: A quasi-Monte Carlo approach to particle simulation of the heat equation. SIAM J. Numer. Anal., 30 (1993) 1558–1573. 11. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo methods. SIAM, Philadelphia (1992). 12. Ogawa, S., Lecot, C.: A quasi-random walk method for one-dimensional reactiondiffusion equations. Math. Comput. Simulation, 62 3-6 (2003) 487–494. 13. Sabelfeld, K.K.: Vector algorithms of the Monte Carlo method for solving systems of 2-nd order elliptic equations and Lame equation. Doklady USSR Acad.Sci. 262 (1982) 1076–1080 (Russian). 14. Sabelfeld, K.K.: Monte Carlo methods in boundary value problems. Springer Verlag, New York Heidelberg Berlin (1991). 15. Sabelfeld, K.K., Simonov, N.A.: Random walks on boundary for solving PDEs. VSP, Utrecht, The Netherlands (1994).
A Stable Backward Monte Carlo Method for the Solution of the Boltzmann Equation Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr Institute for Microelectronics, TU-Vienna Gusshausstrasse 27-29/E360, A-1040 Wien, Austria
[email protected]
Abstract. Backward Monte Carlo methods for solving the Boltzmann equation are investigated. A stable estimator is proposed since a previously published estimator was found to be numerically unstable. The principle of detailed balance, which is obeyed by state transitions of a physical system and ensures existence of a stable equilibrium solution, is violated by the transition probability of the unstable method, and is satisfied by construction with the proposed backward transition probability.
1 Introduction For the numerical study of non-equilibrium charge carrier transport in semiconductors the Monte Carlo (MC) method has found wide spread application. In particular the physically transparent forward MC method is commonly employed, which evaluates functionals of the distribution function. The more abstract backward MC method, however, has found virtually no application in semi-classical transport calculations. This method follows the particle history back in time and allows the distribution function to be evaluated at given points with desired accuracy. The method is particularly appealing in cases where the solution is sought in sparsely populated regions of the phase space only. In the field of semi-classical transport the backward MC method has been proposed end of the 1980’s [4,8]. One of the roots of this method is in quantum transport [1], a field where also various applications are reported [7,3].
2
Boltzmann Equation
On a semi-classical level the transport of charge carriers in semiconductors is described by the Boltzmann Equation (BE). For the time- and position-dependent transport problem the BE reads
This equation is posed in the simulation domain D and has to be supplemented by boundary and initial conditions. The distribution function is commonly normalized as with denoting the number of carriers contained in the semiconductor domain of volume I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 170–177, 2004. © Springer-Verlag Berlin Heidelberg 2004
A Stable Backward Monte Carlo Method
171
In (1) the carrier’s group velocity v is related to the band energy by The force field F takes into account electric and magnetic fields. If only an electric field E is present, the force field is given by where is the charge of the carrier. The scattering operator consists of a gain and a loss term, respectively. If many-body effects such as carrier-carrier scattering and degeneracy are neglected, the scattering operator will be linear, an assumption that is crucial for the presented approach. The two components of Q are
with
2.1
denoting the total scattering rate.
Integral Form of the Boltzmann Equation
The BE is now transformed to integral form by a formal integration over a phase space trajectory.
This trajectory has the initial condition and and solves the equations of motion in phase space, given by Newton’s law and the carrier’s group velocity. The following integral form of the BE is obtained [5]:
This equation represents the generalized form of Chamber’s path integral [2]. The source term contains a given initial distribution. Augmenting the kernel by a delta-function of the real space coordinate and a unit step function of the time coordinate allows transformation of (5) into an integral equation of the second kind:
172
2.2
Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr
The Neumann Series
Substituting (5) recursively into itself gives the Neumann series expansion of the solution in a given phase space point k, r at time Convergence of the series has been proven in [6]. MC algorithms for solving an integral equation can be derived by evaluating the terms of the iteration series by MC integration. Each term of the iteration series has the same structure. In the term of order the integral operator is applied to the source term As an example we write the term of second order explicitly.
Final conditions for the k-space trajectories are given first by and then by the before-scattering states and (See Fig. 1). The real space trajectory ends at final time in the given point and is continuous at the time of scattering: The iteration term (9) describes the contribution of all second order trajectories to the solution. On such a trajectory a particle undergoes two scattering events during propagating from time 0 to and is found on its third free-flight path at time
3
Backward Monte Carlo Methods
Backward MC algorithms for the solution of the Boltzmann equation have been proposed in [4, 8]. Given an integral equation the backward estimator of the iteration term is constructed as
A Stable Backward Monte Carlo Method
Fig. 1. Sketch of a backward trajectory starting at time three free flights. The symbols used in (9) are shown.
173
and reaching time 0 after
where denotes a transition probability density. The set of points is referred to as a numerical trajectory. After generation of N numerical trajectories the iteration term is estimated by the sample mean
For the Boltzmann equation considered here the variable denotes In this work we discuss two specific choices of the transition probability
3.1
Probability Density Functions
The components of the kernel (6) are used to construct probability density functions (pdf). From the scattering rate S one can define a pdf of the after-scattering states
with the total scattering rate Conversely, the pdf of the before-scattering states
as a normalization factor. is defined as
The normalization factor is given by the backward scattering rate, The pdf of the backward free-flight time
is given by
174
Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr
and satisfies the normalization jectory is
3.2
Final condition of the tra-
The Source Term
In case of the Boltzmann equation the source term is treated in a specific way.
The exponential function represents the probability that a particle moves without scattering from 0 to This probability is now used as an acceptance probability. The probability is expressed as an integral over the respective pdf, given by (14).
The acceptance probability is checked as follows. For a particle in state k at time the backward free flight time is generated from If is negative, the estimator is nonzero, otherwise, the estimator evaluates to zero. To obtain a nonzero estimator of the iteration term all the generated times must be positive, whereas the next time generated, must be negative. For a trajectory of order all other estimators of order evaluate to zero. In this way an estimator for the distribution function is obtained as
Here denotes the order of the now defined as
numerical trajectory. The estimator is
Note that this estimator samples the initial distribution source term is sampled.
3.3
whereas in (10) the
Transition Probability Densities
In the original work [8] is interpreted as the unnormalized distribution of the before-scattering states and consequently the normalized pdf (13) is employed. Using the transition density
A Stable Backward Monte Carlo Method
175
the estimator (18) becomes
Although the MC algorithm based on the estimator (20) is consistently derived from the integral form of the BE, computer experiments reveal a stability problem. The particle energy becomes very large when the trajectory is followed backward in time. The initial distribution takes on very small values at high energies, such that many realizations of the estimator will be very small. With small probability, the particle energy will stay low, where the initial distribution is large. These rare events give large contributions to the estimator, resulting in a large variance. The computer experiments show that the variance increases rapidly with time. However, for a given time the variance of the estimator is finite. The time evolution of the particle energy can be understood from a property of the scattering rate known as the principle of detailed balance. This property ensures that in any system particles scatter preferably to lower energies. If for trajectory construction the backward transition rate (13) is employed, the principle of detailed balance is inverted in the simulation and scattering to higher energies is preferred. The principle of detailed balance is reflected by the following symmetry property of the scattering rate:
where and denotes the carrier energy. The scattering rate of carriers in a semiconductor contains contributions from various scattering sources and is thus represented by the sum of the corresponding rates.
The total scattering rate is given by A scattering mechanism is either elastic, that is or inelastic, For each inelastic process the sum contains two entries, where one entry describes the inverse process of the other. In the case of phonon scattering these partial processes are caused by absorption and emission of a phonon, respectively. The scattering rates are commonly derived from Fermi’s golden rule,
where M denotes the interaction matrix element, the Bose-Einstein statistics and the phonon energy. Interchanging and and taking into account the relation gives the following symmetry property
Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr
176
Fig. 2. Electron energy distribution functions obtained by backward and forward MC algorithms. This formulation shows that the absorption rate in backward direction is proportional to the emission rate in forward direction, and vice versa. From (25) and (26) it follows then that has the symmetry property (21). The stability problem can be solved using the forward scattering rate also for the construction of the backward trajectory and changing the estimator accordingly. In the transition density the forward pdf (12) is employed.
The estimator (18) becomes
The event.
4
denote the difference in particle energy introduced by the
scattering
Results
MC calculations of electron transport in silicon have been performed. Conditions assumed are E = 10 kV/cm and at T = 300 K. Fig. 2 compares the electron energy distributions as computed by the backward MC method and a forward MC method employing statistical enhancement through event biasing. As initial distribution a Maxwellian distribution is assumed. The backward method is used to evaluate the energy distribution at discrete points above 800 meV. The statistical uncertainty of the result is controlled by the number of numerical trajectories starting from each point. In the simulation backward trajectories are computed for each point. The backward method resolves the high energy tail with high precision as shown in Fig. 2. The depicted range of 30 decades is out of reach for the here considered variant of the forward MC method.
A Stable Backward Monte Carlo Method
177
Acknowledgment This work has been partly supported by the European Commission, project NANOTCAD, IST-1999-10828.
References 1. R. Brunetti, C. Jacoboni, and F. Rossi, Quantum Theory of Transient Transport in Semiconductors: A Monte Carlo Approach, Physical Review B, 39 (1989) 10781– 10790. 2. R. Chambers, The Kinetic Formulation of Conduction Problems, Proc. Phys. Soc. (London), A65 (1952) 458–459. 3. T. Gurov and P. Whitlock, An Efficient Backward Monte Carlo Estimator for Solving of a Quantum-Kinetic Equation with Memory Kernel, Mathematics and Computers in Simulation, 60 (2002) 85–105. 4. C. Jacoboni, P. Poli, and L. Rota, A new Monte Carlo Technique for the Solution of the Boltzmann Transport Equation, Solid-State Electron., 31 3/4 (1988) 523–526. 5. H. Kosina, M. Nedjalkov, and S. Selberherr, Theory of the Monte Carlo Method for Semiconductor Device Simulation, IEEE Trans.Electron Devices, 47 10 (2000) 1898–1908. 6. M. Nedjalkov and I. Dimov, Convergency of the Monte Carlo Algorithms for Linear Transport Modeling, Mathematics and Computers in Simulations, 47 (1998) 383– 390. 7. M. Nedjalkov, H. Kosina, S. Selberherr, and I. Dimov, A Backward Monte Carlo Method for Simulation of the Electron Quantum Kinetics in Semiconductors, VLSI Design, 13 1-4 (2001) 405–411. 8. M. Nedjalkov and P. Vitanov, Iteration Approach for Solving the Boltzmann Equation with the Monte Carlo Method, Solid-State Electron., 32 10 (1989) 893–896.
A Weight Decomposition Approach to the Sign Problem in Wigner Transport Simulations Mihail Nedjalkov, Hans Kosina, and Siegfried Selberherr Institute for Microelectronics, TU-Vienna Gusshausstrasse 27-29/E360, A-1040 Wien, Austria
[email protected]
Abstract. A stochastic method for simulation of carrier transport in semiconductor nanostructures is presented. The Wigner formulation of quantum mechanics is used. The method which aims at evaluation of mean values of physical quantities, is obtained by following the rules of the Monte Carlo theory. A particular numerical feature of the method are statistical weights with inverse signs which can achieve large absolute values. They give rise to high variance of the simulation results, the so called sign problem in quantum Monte Carlo simulations. A weight decomposition approach is proposed which limits the value of a weight by storing part of it on a phase space grid. Annihilation of positive and negative stored weights, which occurs during the simulation significantly improves the variance of the method.
1 Introduction The Wigner-function formalism has been recognized as a convenient approach to describe electron transport in mesoscopic systems. The formalism combines a rigorous quantum-kinetic level of treatment with the classical concepts for phase space and open boundary conditions. Moreover dissipation processes introduced by phonons can be taken into account by adding the classical Boltzmann collision operator to the Wigner operator The basic similarity with the classical transport picture motivates a Monte Carlo (MC) method for solving the Wigner equation. For the sake of transparency the method is described for the case of stationary, coherent one-dimensional transport. A generalization for phonon scattering follows the same approach. The corresponding Wigner equation reads:
where V is the device potential, is the position, and is the momentum. The proposed method aims at evaluation of the mean value of a given physical quantity The basic expression is obtained by a series expansion of derived in the next section. I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 178–184, 2004. © Springer-Verlag Berlin Heidelberg 2004
A Weight Decomposition Approach to the Sign Problem
2
179
The Series Expansion of
As a first step an integral form of the Wigner equation is obtained. A nonnegative function is introduced, which will be determined later, and is added to both sides of (1). The characteristics of the differential operator are classical Newton’s trajectories
A trajectory is initialized by the phase space point at time 0. is the electron velocity and the parameterization is backward in time, Newton’s trajectories are used to transform (1) into the following integral equation:
Here is the time of the trajectory crossing point with the device boundary, where the Wigner function values are known. The mean value is defined by the inner product of A and
In this formulation is obtained from the solution which can be evaluated by a backward MC method. An alternative formulation leads to a forward MC method which directly evaluates the mean value Considered is the conjugate equation of (2) with a free term A and solution The following relation can be established:
Equation (3) shows that the mean value of A is determined by the inner product of the free term of (2) with the solution of the conjugate Wigner equation. The latter is obtained from (2) by applying the same steps used to derive the conjugate Boltzmann equation [2].
Here the indicator of the device domain D takes values 1 if and 0 otherwise. The trajectories are in a forward parameterization, The last term in (3) is also expressed in a forward parameterization [2]. Then the space
180
Mihail Nedjalkov, Hans Kosina, and Siegfried Selberherr
integral gives rise to a time integral and a sum over the left and right boundaries of the one-dimensional device:
The index denotes the subspace of wave vectors pointing inside the device. The iterative expansion of obtained from (4), is replaced in (5) which leads to the series expansion:
In this way the mean value of any physical quantity is determined by the velocity weighted boundary conditions and the consecutive iterations of the conjugate kernel K on the free term A. The expression for is shown as an instructive example:
Due to only that part of a Newton’s trajectory which belongs to D contributes to The series (6) is the main entity of the stochastic approach. Following the MC theory, numerical trajectories are constructed which evaluate the consecutive terms of the series. The numerical trajectories are constructed by an initial P and a transition PP probability density chosen for this purpose. P is used to choose the initial points of the numerical trajectories on the device boundary. The initial density P is adopted from the classical single-particle MC method [2], since in (5) resembles the boundary term of the Boltzmann transport case. The trajectories are built up by consecutive applications of the transition density PP. At each iteration of the kernel a quantity called statistical weight is multiplied by the weight factor The random variable whose realizations sample is evaluated from the product of The random variable corresponding to is given by The choice of the transition density plays an important role for the numerical properties of the MC algorithm. The kernel can be expressed as a product of a weight factor and conditional probability densities which comprise the transition density PP: In the following two possible formulations are discussed. In the first case the numerical trajectories are associated with particles which evolve on parts of Newton’s trajectories, linked by scattering processes. The kernel is interpreted as a scattering operator. In the second case a weight decomposition is proposed.
A Weight Decomposition Approach to the Sign Problem
181
This is achieved by splitting the evolving trajectory in phase space. The kernel is interpreted as an operator which generates particles during the process of evolution.
3
Scattering Interpretation of the Kernel
The kernel of (4) is augmented by extra factors, introduced in a way to conserve the value of the integrand. They ensure the normalization of the conditional probability densities which are enclosed in curly brackets.
Here
Short notations are used for The subscripts denote the order of application of the conditional probabilities. generates a value of associated with a free flight time of a particle which drifts over a piece of Newton’s trajectory between an initial state at time 0, and the final state, The final state is used in the next probability density to select the value The transition between and is interpreted as scattering of the particle. is the probability to use the first kernel component for selection of the after-scattering value of Since the second component is selected according Thus is chosen either with the probability density or with the probability density The normalization of the latter is ensured by the function The after-scattering state is the initial state of a free flight for the next iteration of the kernel. The weight factor where the sign is given by the sign of multiplies the weight accumulated by the trajectory. It can be shown that the mean accumulated weight is evaluated by where T is the dwelling time of the particle in the device. Since the weight does not depend on the latter can be chosen according to some criteria of convenience. A choice gives rise to a weight factor For typical nanoscale devices so that and thus the realizations of which sample become huge positive and negative numbers. The sample variance is orders of magnitude larger than the sample mean, where positive and negative terms cancel each other. This problem, known as the sign problem, exists also in stochastic approaches to alternative formulations of quantum mechanics. For instance MC evaluations of Feynman path integrals demonstrate an exponential growth of the variance with the evolution time [4]. Due to the sign problem, the application of the derived method is restricted to single barrier tunneling and small barrier heights.
182
Mihail Nedjalkov, Hans Kosina, and Siegfried Selberherr
4 Weight Decomposition Approach The following modification of the method is proposed to overcome the problem of the growing variance. The idea is to decompose the weight into a part which continues with the trajectory and a part which is left for future processing. The antisymmetric function is decomposed into two positive functions: With these functions can be expressed as The kernel is written as:
The three kernel components simultaneously create three after-scattering states. Each state gives rise of a trajectory which must be simulated until it leaves the simulation domain. Two of the trajectories carry the weight of the trajectory before the scattering event, the third one changes its sign. Since the initial weight at the boundary is 1, the absolute value of the weight of each trajectory remains 1. The following picture can be associated to the transport process. With each iteration of the kernel a positive (negative) particle undergoes a free flight and scattering event. After the scattering event the particle survives in the same state with the same weight due to the delta function. Additionally a positive and a negative particle are created by A phase space grid is introduced, where all three particles are stored. The simulated trajectory continues from the grid cell with the highest number of stored particles. It is selected among all cells with a position index where is the location of the scattering event. Positive and negative particles have opposite contribution to the statistics. They have the same probabilistic future if located close together in the phase space and thus can be canceled. The active cancellation reduces the simulation time leading to a variance reduction. If phonon scattering is included the picture remains similar. The free flights are additionally interrupted by the phonon scattering events, which as in the classical case, only change the particle momentum.
5
Results and Discussions
The method has been applied for simulation of a resonant tunneling diode. A double barrier structure from the literature [5] is used as a benchmark device. Physical parameters of GaAs with a uniform 0.067 effective mass and a temperature T= 77K is assumed. The potential drop is linear across the central device, the barriers have a thickness of 2.825nm and a height 0.27eV, the well is 4.52nm wide. The device is shown schematically in Fig.1.
A Weight Decomposition Approach to the Sign Problem
183
Fig. 1. Electron concentration distribution in the central part of the resonant tunneling diode for three different bias points. Numerical results for the electron concentration, are presented for three different values of the applied bias. The values correspond to bias points near equilibrium, and the peak and valley current of the currentvoltage characteristics. The accumulation of electrons in the quantum well at the resonance bias 0.13V respectively is well demonstrated. The physical quantity which gives the current is where is the electron charge and is the length of the device. The current-voltage characteristics are shown on Fig. 2. The peak of the current at 0.13V is followed by a negative differential mobility region, which is a typical feature of the resonant tunneling diodes. The current-voltage characteristics are in good quantitative agreement with the results obtained by other methods [1]. Fig. 3 shows the current for the chosen three bias points as a function of the number of scattering events. The latter quantity is an empirical measure of the elapsed simulation time which is independent of the computer platform. scattering events correspond to a 24 hours simulation time on a 1GHz CPU. The three curves illustrate the variations of the during the simulation. Above scattering events the variance of the corresponding values becomes negligible.
6
Conclusions
The proposed weight decomposition method significantly improves the numerical properties of the stochastic approach to the Wigner transport equation. Annihilation of positive and negative weights reduces the computational effort of the task. The method has proven suitable for the simulation of nanoelectronic devices.
184
Mihail Nedjalkov, Hans Kosina, and Siegfried Selberherr
Fig. 2. Current-voltage characteristics of the resonant tunneling diode. The peak is at resonance bias 0.13V.
Fig. 3. Electron current for the chosen bias points as a function of the number of scattering events.
Acknowledgment This work has been supported by the European Commission Project NANOTCAD, IST-1999-10828.
References 1. Klimeck G., JPL, California Institute of Technology. Personal Communications. 2. Kosina H., Nedjalkov M., Selberherr S, The Stationary Monte Carlo Method for Device Simulation - Part I: Theory. Journal of Applied Physics, 93 (2003) 3553– 3563. 3. Nedjalkov M., Kosik R., Kosina H., Selberherr S., A Wigner Equation for Nanometer and Femtosecond Transport Regime. Proceedings of the 2001 First IEEE Conference on Nanotechnology, (2001), 277–281. 4. Schmidt T., Moehring K., Stochastic Path-Integral Simulation of Quantum Scattering. Physical Review A, 48 (1993) R3418–R3420. 5. Tsuchiya, H., Ogawa, M., Miyoshi, T., Simulation of Quantum Transport in Quantum Devices with Spatially Varying Effective Mass, IEEE Trans. Electron Devices, (1991) 1246–1252.
A Zero Field Monte Carlo Algorithm Accounting for the Pauli Exclusion Principle Sergey Smirnov, Hans Kosina, Mihail Nedjalkov, and Siegfried Selberherr Institute for Microelectronics, TU-Vienna, Gusshausstrasse 27-29/E360, A-1040 Wien, Austria
[email protected]
Abstract. A Monte Carlo method for calculation of the carrier mobility in degenerate bulk semiconductors at zero electric field is presented. The method is obtained as a limiting case of an existing small-signal approach replacing the distribution function by the Fermi-Dirac distribution which is valid at zero electric field. The general form of the Boltzmann equation which takes into account the Pauli exclusion principle in the scattering term is used to derive the integral representation of a Boltzmann-like equation for a small perturbation of the distribution function. The method allows calculation of the whole mobility tensor in comparison with the one particle Monte Carlo algorithm which is traditionally used to compute low field carrier mobility.
1 Introduction The low field carrier mobility of a bulk semiconductor is an important kinetic property of a semiconductor. It is used to analyze the carrier transport in semiconductor devices at low applied voltages and enters expressions for high field mobility models as an additional parameter. Thus the knowledge of the low field carrier mobility and its correct dependence on the material properties such as the doping concentration are necessary to adequately simulate carrier transport in semiconductor devices. The standard approach for obtaining the low field carrier mobility is a single particle Monte Carlo method. Within this method in order to calculate the low field mobility along the direction of the electric field one should carefully choose the magnitude of the applied electric field. On the one hand side, the magnitude of the electric field must be as low as possible. In principle it is desirable to have zero electric field. However, there exist limitations related to the increase of the variance of standard Monte Carlo methods. On the other hand side, the field must not be too high to avoid a mobility reduction due to carrier heating. In addition to these disadvantages, the standard approaches only give one component of the carrier mobility namely the component along the direction of the electric field. For isotropic conditions it does not make any difference since the mobility tensor is diagonal and all diagonal values are equal. However when anisotropy is present, for example in strained semiconductors, the mobility tensor I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 185–193, 2004. © Springer-Verlag Berlin Heidelberg 2004
186
Sergey Smirnov et al.
elements may be different and several Monte Carlo simulations are required to obtain all the components of the tensor. To overcome these problems associated with standard Monte Carlo methods a new Monte Carlo algorithm has been suggested recently [8], which solves the Boltzmann equation at exact zero field and represents a limiting case of a small signal algorithm obtained in [6]. One of the most remarkable properties of the algorithm is the absence of self-scattering that allows to significantly reduce calculation time and achieve very good accuracy of the results. This method is restricted to the simulation of low doped semiconductors. The quantum mechanical Pauli exclusion principle is not included in the scattering term of the Boltzmann equation used for the derivation of the algorithm. As a result there are limitations on the doping level of materials analyzed by this technique. It allows to obtain excellent results at low and intermediate doping levels while results obtained for higher doping levels where the effects of degenerate statistics are more pronounced are incorrect. As the standard Monte Carlo methods exhibit a very high variance especially in degenerate case, it is thus desirable to have a powerful technique to analyze the carrier mobility at high doping levels. In this work we present a zero field algorithm to account for degenerate statistics. The Pauli exclusion principle is taken into consideration in the scattering term of the Boltzmann equation. As a result the Boltzmann equation becomes nonlinear. Using this nonlinear equation we derive a generalized zero field algorithm applicable for the analysis of highly doped materials.
2
Nonlinear Boltzmann Equation and Its Linearized Form
Let us consider a bulk homogenous semiconductor. Then we can neglect the space dependence of the distribution function and the differential scattering rate. We also assume the differential scattering rate to be time invariant. With these conditions the time dependent Boltzmann equation taking into account the Pauli exclusion principle takes the following form:
where is an electric field and is the particle charge. the scattering operator which is given by the following expression:
represents
where stands for the differential scattering rate. Thus is the scattering rate from a state with wave vector to states in around is the distribution function and the factors mean that the final state
A Zero Field Monte Carlo Algorithm Accounting
187
must not be occupied according to the Pauli exclusion principle. As can be seen from (2), there are terms which render the equation nonlinear. Only when the condition is valid the factors can be replaced by unity and the equation takes the usual linear form. To linearize (1) we write the electric field in the form:
where stands for a stationary field and denotes a small perturbation which is superimposed on a stationary field. It is assumed that this small perturbation of the electric field causes a small perturbation of the distribution function which can be written as follows:
where is a stationary distribution function and is a small deviation from a stationary distribution. Substituting (4) into (2) the scattering operator takes the form:
It should be noted that in spite of the fact that one should take care when linearizing terms such as as especially in the degenerate case it may happen that the inequality becomes valid because of Neglecting terms of second order we derive the zeroth order equation:
and the first order equation:
where we introduced the notation erator which has the form:
for the first order scattering op-
188
Sergey Smirnov et al.
Equation (7) is linear with respect to it is a kinetic equation which differs from the usual form of the Boltzmann equation. The first difference is related to the presence of the additional term on the right hand side being the term proportional to which additionally depends on the stationary distribution. The second difference is in the expression for the scattering operator which now has a more complex form and also depends on the stationary distribution.
3
Integral Form of the First Order Equation
To construct a new Monte Carlo algorithm we reformulate the Boltzmann equation of the first order into integral form. For this purpose we introduce a new differential scattering rate and new total scattering rate defined by the following expressions:
It is worth noting that the similarity with the standard Boltzmann equation is only formal as both, differential scattering rate and total scattering rate, are now functionals of the stationary distribution function which is the solution of the equation of zero order (6). With these definitions the scattering operator of the first order formally takes the conventional form:
and the Boltzmann-like equation can be rewritten as follows:
We derive the integral form of this equation using techniques described in [7]. Introducing a phase space trajectory which is the solution of Newton’s equation, and taking into account that for we obtain the following integral form:
A Zero Field Monte Carlo Algorithm Accounting
189
Finally we assume an impulse like excitation of the electric field, and obtain:
where The essential difference of this integral representation from the one of the nondegenerate approach consists in the appearance of the new differential scattering rate and of the total scattering rate Another difference which is common for both approaches is the additional free term on the right hand side which in general cannot be treated as an initial distribution because it also takes negative values.
4
Zero Field Approach
When the electric field tends to zero, the distribution function approaches its equilibrium which is in case of particles with fractional spin represented by the Fermi-Dirac distribution function:
where denotes the Fermi energy, stands for an electron energy and is an equilibrium temperature equal to the lattice temperature. Since the stationary distribution is known, there is no necessity to solve the zeroth order equation (6). As can be seen from (16), in equilibrium the distribution function depends only on the carrier energy and not on the wave vector. This fact allows one to significantly simplify (10) using the Fermi golden rule [2]:
Making use of the delta function in the last expression we can rewrite (10) in the following manner:
where denotes the final carrier energy and we introduced the backward scattering rate and (18) represents a linear combination of the forward and backward total scattering rates. In the
190
Sergey Smirnov et al.
Fig. 1. Schematic illustration of the scattering processes at high degeneracy. non-degenerate case, we obtain that means that scattering processes are mostly determined by the forward scattering rate and thus the algorithm developed in [8] for nondegenerate statistics is restored. On the other hand side for highly doped semiconductors, scattering processes are dominantly backward In case of intermediate doping levels both forward and backward scattering contribute to the kinetics. It should be also mentioned that as at high doping levels the backward scattering rate is dominant, the probability to scatter to higher energy states is larger than to lower energy states as schematically shown in Fig.1(a). This means that lower energy levels are already occupied by particles i.e. (see Fig.1(b)) and, due to the Pauli exclusion principle, scattering to these energy levels is quantum mechanically forbidden. The additional free term in (14) cannot be considered as an initial distribution because function may take negative values. However, in case of zero electric field the stationary distribution is known analytically and we can calculate explicitly:
where denotes the group velocity. This expression can be rewritten in the following manner:
where the term in curly brackets represents the normalized distribution function of the before-scattering states. The Monte Carlo algorithm contains the same steps as those in [8] except that the whole kinetics must be now considered in terms of and instead of and It may be seen by derivation of the second iteration term of the Neumann series for (14) using the forward formulation to obtain the ensemble Monte Carlo algorithm:
A Zero Field Monte Carlo Algorithm Accounting
where
stands for an after-scattering wave vector and
191
denotes an initial wave
vector. The quantity represents a normalized after-scattering distribution. As can be seen from (9) and (10) it is normalized to unity. As it follows from (21), during Monte Carlo simulation a particle trajectory is constructed in terms of new quantities and Another difference from the non-degenerate zero field algorithm is that the weight coefficient must be multiplied by the factor With these modifications the steps of the algorithm are as follows:
1. 2. 3. 4.
Set Select initial state arbitrarily. Compute a sum of weights: Select a free-flight time and add time integral to estimator: or use the expected value of the time integral:
5. Perform scattering. If mechanism was isotropic, reset weight: 6. Continue with step 3 until N have been generated. 7. Calculate component of zero field mobility tensor as
5
Results and Discussion
As the first example we calculate the doping dependence of the zero field mobility in silicon. The analytical band structure reported in [3] is adopted. The scattering processes included are acoustic deformation potential scattering, ionized impurity scattering [5] and plasmon scattering [1]. Ionized impurity scattering is treated as an isotropic process [4] which effectively reduces small-angle scattering. Fig.2 shows two electron mobility curves obtained by the new zero field algorithm and the one particle Monte Carlo algorithm, respectively. The Pauli exclusion principle at low electric field has been included in the one particle Monte Carlo method using the Fermi-Dirac distribution. This leads to a prefactor for all scattering processes within the low field approach. The value of the electric field used for the one particle approach has been chosen to be 1 kV/cm. The other physical parameters of both algorithms are the same.
192
Sergey Smirnov et al.
Fig. 2. Electron zero field and low field mobilities in Si.
As can be seen from Figure 2, the curve obtained with the one particle Monte Carlo method has some variance at low and high doping levels while the zero field curve appears rather smooth. Moreover the calculation time for the new zero field algorithm is about 20% of the one particle Monte Carlo method. Fig.3 shows the electron mobility as a function of doping concentration for GaAs obtained by both algorithms. The degeneracy effects are more pronounced in this semiconductor because of the smaller effective mass of electrons in the valley. The absolute value of the electric field is the same as for silicon. As is seen from this figure, in addition to a high variance at low doping levels the one particle Monte Carlo gives an incorrect behavior of the mobility at high doping levels. This is related to the fact that the value 1 kV/cm of the electric field is still high from the viewpoint of using the Fermi-Dirac distribution with lattice temperature within the one particle algorithm. In order to obtain correct results at high doping levels by the one particle Monte Carlo method it is necessary to reduce further the magnitude of the electric field. However in this case the variance would increase considerably leading to an extremely long computation time.
6
Conclusion
A zero field Monte Carlo algorithm accounting for the quantum mechanical Pauli exclusion principle has been presented. The method has been derived from the integral representation of a linearized Boltzmann-like equation. It has been shown that particle trajectories are constructed in terms of a new scattering rate which in general represents a linear combination of the forward and backward scattering rates. It has been also pointed out that for energies below the Fermi
A Zero Field Monte Carlo Algorithm Accounting
193
Fig. 3. Electron zero field and low field mobilities in GaAs.
level kinetic properties are predominantly determined by the backward scattering rate while for energy levels above the Fermi level the forward scattering rate is dominant. In the latter case the non-degenerate zero field algorithm is recovered.
References 1. M. V. Fischetti. Effect of the electron-plasmon interaction on the electron mobility in silicon, Phys.Rev.B, 44 11 (1991) 5527–5534. 2. C. Jacoboni and P. Lugli. The Monte Carlo Method for Semiconductor Device Simulation, Wien-New York, Springer, 1989. 3. C. Jacoboni and L. Reggiani. The Monte Carlo Method for the Solution of Charge Transport in Semiconductors with Applications to Covalent Materials, Rev.Mod.Phys., 55 3 (1983) 645–705. 4. H. Kosina. A Method to Reduce Small-Angle Scattering in Monte Carlo Device Analysis, IEEE Trans. Electron Devices, 46 6 (1999) 1196–1200. 5. H. Kosina and G. Kaiblinger-Grujin. Ionized-Impurity Scattering of Majority Electrons in Silicon, Solid-State Electron., 42 3 (1998) 331–338. 6. H. Kosina, M. Nedjalkov, and S. Selberherr. A Monte Carlo Method for Small Signal Analysis of the Boltzmann equation, J.Appl.Phys., 87 9 (2000) 4308–4314. 7. H. Kosina, M. Nedjalkov, and S. Selberherr. Theory of the Monte Carlo Method for Semiconductor Device Simulation, IEEE Trans. Electron Devices, 47 10 (2000) 1898–1908. 8. H. Kosina, M. Nedjalkov, and S. Selberherr. Monte Carlo Analysis of the SmallSignal Response of Charge Carriers, In: S. Margenov, J. Wasniewski, P. Yalamov, eds.: Large-Scale Scientific Computing. Lecture Notes in Computer Science, 2179, Springer Verlag, (2001) 175–182.
This page intentionally left blank
Part IV
Set-Valued Numerics and Reliable Computing
This page intentionally left blank
Solution Methods for Age-Structured Optimal Control Models with Feedback* Christian Almeder
Department of Business Studies University of Vienna, Bruenner Str. 72, A-1210 Vienna, Austria
[email protected]
Abstract. Modeling the dynamics of infectious diseases, illicit drug initiation, or other systems, where the age of an individual or the duration of being in a specific state is essential, leads to a generalized form of the McKendrick equations, i.e., a system of first-order partial differential equations in the time-age space. Typically such models include agespecific feedback components which are represented by integral terms in the right hand side of the equation. This paper presents the general framework of such optimal control models, the corresponding necessary optimality conditions and two different approaches for numerical solution methods.
1 Introduction The classical McKendrick equation is the basic instrument for modeling agestructured population dynamics (see [1, 5–7]). It is a first-order partial differential equation of the form
where denotes the time, is the age, is the state variable (e.g., the number of people of age at time and is an age-dependent death rate. This equation can be solved easily by transforming it into a set of ordinary differential equations along the characteristics
where represents the time of birth and When modeling concrete problems, it is often the case that depends also on aggregated state variables (e.g., the total population size). This fact leads to an integral term (integration w.r.t. age) in the right hand side of equation (1). This integral term couples the infinite set of ODEs in (2). * This work was partly financed by the Austrian Science Foundation under contract
No. 14060-OEK and by the Austrian Science and Research Liaison Office Sofia. I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 197–203, 2004. © Springer-Verlag Berlin Heidelberg 2004
198
Christian Almeder
Furthermore, if optimal control models with distributed (age-specific) control variables are considered, an analytical solution is hardly feasible. But it is possible to formulate the necessary optimality conditions and based on them, two different approaches for solving those problems numerically are presented in this paper. These two approaches are the following: Direct Method: This method is based on the discretization of the age parameter. It leads to an ordinary boundary value problem which can be solved by a collocation method. Gradient Method: Another possibility is to apply an iterative method. Based on an initial estimate of the control variable the primal system is solved in the forward direction and using the results the dual system is solved in the backwards direction. The approximated solutions of the primal and the dual systems allow to calculate the anti-gradient of the Hamiltonian, which is used to improve the approximation of the control.
2
Model Framework
We consider a system of different states with depending on the time and a second parameter which can either be the age or the duration in this state. In both cases has the form where indicates the time of birth (in the age case) or the time of state entrance (in the duration case). To distinguish between an age specific or a duration specific system, we introduce the age-duration-index
There are different parameters describing the transition between the states, and into and out of the system: denotes the transition from state to depending on time the parameter the state of the system S, the overall feedback F of the system, and the distributed control represents the outflow from the state out of the system. is the inflow into the state from outside of the system. The dynamic of the system is represented by a McKendrick-like equation
with boundary conditions
Methods for Age-Structured Optimal Control Models with Feedback
199
and initial conditions The difference between an age- and a duration-specific system is the way of entering a new state. In the age case the transitions into a new state are included in the partial differential equation (4), because the age is a preserved property. But in the duration case the transitions occur in the boundary condition (5), because the duration has to be reset, if the state changes. The feedback component F describes the current status of the system with respect to a certain aspect. The general form is:
As indicated, F is a vector function, so that it is possible to include different types of feedback. A simple example is that the feedback function consists only of the total amount in each state at time (so that f = S). The restriction that is chosen due to a numerical implementation and easier analysis. Furthermore, in real applications it is very unlikely to have an infinite age or duration range, (e.g., epidemiological models are often described using an infinite age range, but it is no restriction, if the age range is cut off at 130 years for humans.) The control shall be optimized with respect to an objective function of the general form
where U is the set of feasible solutions.
3
Necessary Optimality Conditions
In [4] a maximum principle is described for a more general case than this type of models is. Applying this method leads to the following maximum principle (To increase the readability arguments of functions are not written. The missing arguments are all set along the optimal solution.): Maximum Principle 1 If described through (3) - (8), then defined through
is the optimal control for the dynamic system defined through (3) - (7), and
200
Christian Almeder
with boundary and terminal conditions
together with
satisfy
where the Hamiltonian is
This maximum principle can be derived easily by application of theorem 1 in section 4 of [4].
4
Numerical Solution Algorithms
The model equations (3)-(8) together with the dynamics of the adjoint system (9)-(13) form a system of first-order partial differential equations defined on a rectangle in the The primal system has initial and boundary conditions at and whereas the dual system must fulfill terminal and boundary conditions at and Furthermore, the primal and the dual system have conflictive dynamics in the sense that the (numerical) stable direction (increasing time) for solving the primal system is unstable for the dual system. In the following two different methods are described which consider those circumstances.
Methods for Age-Structured Optimal Control Models with Feedback
201
Direct Method
In this context direct method means that an approximation of the solution of the problem is calculated in one step. The main idea is to transform the system of partial differential equations into a system of ordinary differential equations by applying the method of lines. In detail this means that the above equations are discretized with respect to age (duration) by introducing an appropriate grid For each grid point a separate state vector as function of time is specified. The derivatives w.r.t. are approximated using a finite difference formula of appropriate order, and the integration w.r.t. is substituted using a quadrature formula (for details see [8]). At each grid point the function describes the state at this point. The state vector extends to a state matrix, one element for each state and each age:
According to that all functions are transformed into time-dependent grid functions. The ODE system corresponding to (4) can be formulated as follows:
with boundary conditions
and initial conditions where denotes an appropriate finite difference operator w.r.t. and is the quadrature operator. In the same way the feedback function F, the objective function J, and the adjoint system can be transformed. This procedure leads to an ordinary boundary value problem, where one half of the state variables has initial conditions and the other half has terminal conditions. The number of equations depends on the number of grid points chosen. As explained above, the primal and the dual system have a complementary stability behavior. Therefore, it is not possible to apply a shooting method for the solution of the problem. But a finite difference or a collocation method can be used to solve the boundary value problem, since these methods solve the problem over the whole range in one step (see [2]). This is done by discretizing the problem and getting a non-linear equation system. Unfortunately this nonlinear equation system can become extremely large. Its size depends on the
202
Christian Almeder
number of states, on the number of grid points and on the discretization of the time Hence, this method can only be applied to small and medium-sized problems. Gradient Method
Another approach for a solution algorithm is to approximate the optimal control variable iteratively using the anti-gradient of the Hamiltonian function. The steps of this iteration are as follows:
1. Choose a first estimate for the control Set 2. Get the solution of the primal system using For solving the primal system the equations are transformed into an ordinary differential equation system either by discretizing or by reformulating the problem along the characteristics (see equation (2)). 3. Get the solution of the dual system using and The same method as in the previous step can be applied here, but the equations must be solved backwards (due to the terminal conditions). 4. Now a direction for improving the control variable can be calculated using the anti-gradient of the Hamiltonian
If the (anti-)gradient is sufficiently small, the iteration stops and is the approximate solution. 5. The new estimate can be calculated by performing a line search and minimizing the objective along the line For this step it is again necessary to solve the primal system for different control values 6. Change to and proceed with step 2. For the gradient method it is only necessary to solve initial value problems, because the adjoint system is solved in the backwards direction and the terminal conditions turn into initial conditions. The effort for solving initial value problems is far less than solving boundary value problems, especially if the problems are large. Therefore, this method is preferable for models with many states and a long time horizon.
5
Discussion and Conclusions
Both methods were implemented and several different models were solved (see [1,3]). In general the gradient method seems preferable, because it is faster and less memory consuming. The usual disadvantages are that the iteration can easily be caught in a local optimum and that the convergence speed depends strongly on the problem and it can be very low. On the other hand the direct method is more time-consuming, but usually more robust concerning the convergence of the collocation algorithm. Discontinuities of functions or high variations need a special handling, which means that appropriate fixed grid points have to be chosen in order to get convergence.
Methods for Age-Structured Optimal Control Models with Feedback
203
A combination of both algorithms may improve the results, if the direct method is used to calculate a quick first approximation and the gradient method is used for the fine tuning. But we did not test that possibility. Since a very general formulation of the model is used, a wide range of applications can be realized. Also further extensions of the model framework are possible and partly already included in our programs: Different Types of Feedback: It is possible to implement a simple feedback of the state values (i.e., the transition functions depend on an aggregated but age-independent feedback (i.e., using a function which does not depend on or an aggregated age-dependent feedback All three types can be used in one model. Different Types of Control: The model can include distributed, boundary, and non-distributed controls. With different types of controls, we also get different Hamiltonian functions, and it is only a question of calculating the controls (for the direct method) or the anti-gradients (for the gradient method) based on that functions. The solution methods presented in this paper are formulated for a very general case. For concrete problems it is useful and sometimes necessary to adapt the methods and consider the special structure of the problem in order to improve the quality of the results and to reduce the computational time.
References 1. Ch. Almeder, J.P. Caulkins, G. Feichtinger, and G. Tragler, An age-structured single-state drug initiation model — cycles of drug epidemics and optimal prevention programs, Research Report 249, Department of Operations Research and Systems Theory, Vienna University of Technology, Vienna, (2000). (forthcoming in Social and Economic Planning Sciences) 2. U. Ascher, J. Christiansen, and R.D. Russel, Collocation software for boundaryvalue odes, ACM Trans. Math. Software, 7 (1981), 209–222. 3. G. Feichtinger, R.F. Hartl, P.M. Kort, and V.M. Veliov, Dynamic investment behavior taking into account aging of the capital, CentER Discussion Paper, No. 2001-13, (2001), Tilburg University. 4. G. Feichtinger, G. Tragler, and Veliov V.M, Optimality conditions for age-structured control systems, J. Math. Anal. Appl., forthcoming. 5. A. Haurie, S. Sethi, and R. Hartl, Optimal control of an age-structured population model with applications to social services planning, Large Scale Systems, 6 (1984), 133–158. 6. M. Iannelli, F.A. Milner, and A. Pugliese, Analytical and numerical results for the age-structured S-I-S epidemic model with mixed inter-intracohort transmission, SIAM J. Math. Anal, 23(3) (1992), 662–688. 7. B.L. Keyfitz and N. Keyfitz, The McKendrick partial differential equation and its uses in epidomology and population studies, Math. Comput. Modelling, 26(6) (1997), 1–9. 8. W.E. Schiesser, The Numerical Method of Lines, Academic Press (1991).
Multivariate Rational Interpolation of Scattered Data Stefan Becuwe, Annie Cuyt, and Brigitte Verdonk
University of Antwerp Department of Mathematics and Computer Science Middelheimlaan 1, B-2020 Antwerpen, Belgium {stefan.becuwe,annie.cuyt,brigitte.verdonk}@ua.ac.be
Abstract. Rational data fitting has proved extremely useful in a number of scientific applications. We refer among others to its use in some network problems [6, 7,15,16], to the modelling of electro-magnetic components [20,13], to model reduction of linear shift-invariant systems [2, 3,8] and so on. When computing a rational interpolant in one variable, all existing techniques deliver the same rational function, because all rational functions that satisfy the interpolation conditions reduce to the same unique irreducible form. When switching from one to many variables, the situation is entirely different. Not only does one have a large choice of multivariate rational functions, but moreover, different algorithms yield different rational interpolants and apply to different situations. The rational interpolation of function values that are given at a set of points lying on a multidimensional grid, has extensively been dealt with in [11,10, 5]. The case where the interpolation data are scattered in the multivariate space, is far less discussed and is the subject of this paper. We present a fast solver for the linear block Cauchy-Vandermonde system that translates the interpolation conditions, and combine it with an interval arithmetic verification step.
1 Introduction In one variable the rational interpolation problem can be solved using the recursive technique developed by Bulirsch-Stoer, or a fast solver for the solution of the structured linear system defining the coefficients. The rational interpolant can also be obtained as the convergent of a Thiele interpolating continued fraction with inverse differences in the partial denominators. For multivariate generalizations of the Thiele interpolating continued fraction approach we refer to [21,22, 19,12]. For a multivariate generalization of the Bulirsch-Stoer algorithm we refer to [9]. The structured linear system solver, which is developed here, delivers the coefficients of the rational interpolant, and hence an explicit representation of the rational function. Let the value of the univariate function be given in the interpolation points which are non-coinciding. The rational interpolation problem of order for consists in finding polynomials I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 204–213, 2004. © Springer-Verlag Berlin Heidelberg 2004
Multivariate Rational Interpolation of Scattered Data
with
205
irreducible and such that
In order to solve (1) we rewrite it as
Condition (2) is a homogeneous system of linear equations in the unknown coefficients and of and and hence it has at least one nontrivial solution. It is well-known that all the solutions of (2) have the same irreducible form and we shall therefore denote by
the irreducible form of with and satisfying (2) where is normalized according to a chosen normalization. We say that “interpolates” the given function and by this we mean that and satisfy some of the interpolation conditions (1). This does not imply that actually interpolates the given function at all the data because, by constructing the irreducible form, a common factor and hence some interpolation conditions may be cancelled in the polynomials and that provide Since is the irreducible form, the rational functions with and satisfying (2) are called “equivalent”. If the rank of the linear system (2) is maximal then Let us take a closer look at the linear system of equations (2), defining the numerator and denominator coefficients and In the sequel we assume, for simplicity but without loss of generality, that this homogeneous linear system of equations can be solved for the choice The concept of displacement rank was first introduced in [18]. We use the definition given in [14] where the displacement rank of an matrix A is defined as the rank of the matrix LA – AR with L and R being so-called left and right displacement operators. If A is a Cauchy-Vandermonde matrix, as in (2) after choosing and if all and all then suitable displacement operators are given by and with
206
Stefan Becuwe, Annie Cuyt, and Brigitte Verdonk
The resulting matrix LA – AR then takes the form
Hence the displacement rank
of A equals
When a factorization [14]
is known, then an LU factorization of the Cauchy-like matrix (the denotes complex conjugation and transposition) where the columns of the matrices and contain the eigenvectors of for which explicit formulas are known, can be obtained from [14] with order of complexity
2
Multivariate Rational Interpolation
Although the situation between one and more variables is substantially different, there is no loss in generality by describing the bivariate case instead of the general higher-dimensional case. Let the bivariate function be given in the set of points and let us assume that none of the points coincide. Let N and D be two finite subsets of with which we associate the bivariate polynomials
The multivariate rational interpolation problem consists in finding polynomials and with irreducible such that
In applications where adaptive sampling is used and data points are placed at optimally located positions, it is an exception rather than the rule that some data points have the same or Hence techniques available for a grid-like set of data points, such as in [21, 5,10, 22, 19, 4] cannot be used. In the sequel we shall deal with the more general and less-studied multivariate situation where the dataset is not necessarily grid-structured. We do however require that the sets N and D satisfy the inclusion property, which is not a serious restriction. The problem of interpolating the data by is reformulated as
The set of polynomial tuples
satisfying (3) is denoted by
Multivariate Rational Interpolation of Scattered Data
3
207
Fast Block Cauchy-Vandermonde Solver
Let us introduce the notations
Let and let us introduce the shorthand notations and Since both N and D satisfy the inclusion property, we can decompose the sets as follows:
If
we proceed analogously. If is attained in the instead of in the then the sets N and D are decomposed horizontally instead of vertically. We point out to the reader at this point that the sequel does not apply if both the sets N and D are not decomposed in the same way, either both horizontally or both vertically.
Fig. 1. Breaking up N and D Using these notations, we arrange the unknown coefficients
and
as
208
Stefan Becuwe, Annie Cuyt, and Brigitte Verdonk
Introducing the matrices of dimension
of size
and the matrices respectively given by
the system of interpolation conditions (3) looks like
Let all Again, without loss of generality, we solve this system with normalized to 1. We denote by A the coefficient matrix of the square inhomogeneous linear system, which results from removing the first unknown from and the first column from In order to generalize the fast CauchyVandermonde solver, we choose a if any of the is equal to one in case is either or or any of the is equal to one in case is either or and introduce
With
it is easy to see that the resulting can be factored as where the are given by
and
matrix
Submatrices
and
and the matrix B consists of zeroes with the exception of the following entries which equal one:
Multivariate Rational Interpolation of Scattered Data
When decomposed horizontally, then the factorization (4) for
where the eigenvalues of the unitary matrix
the factorization (the tion)
209
is either or and N and D are being is replaced by From and from the factorizations
are the complex zeroes of are the eigenvectors of
and the columns we obtain with
denotes transposition and complex conjuga-
Since the matrix is a Cauchy-like matrix, the technique proposed in [14] can be applied. It incorporates partial pivoting while its complexity of is noticeably smaller than that of the classical Gaussian elimination. The latter is achieved because the technique exploits the structure of the coefficient matrix by constructing the LU factorization of from the knowledge of the matrix factors G and B in the factorization (5).
4
Complexity, Stability and Reliability
The procedure to compute the LU factorization of directly from the matrix factors G and B is detailed in [14]. Roughly speaking all entries in the LU
210
Stefan Becuwe, Annie Cuyt, and Brigitte Verdonk
factors can be computed from the scalar products of the rows in G and the columns in B, and the differences of the entries in the left and right displacement operators and Having the LU decomposition of at our disposal, an approximate inverse W of A can be computed (remember that and the following interval arithmetic verification step can easily be performed. We again for simplicity assume that the system of interpolation conditions (3) can be solved with the choice We denote the coefficient matrix resulting from the choice by A, the righthand side of the square inhomogeneous linear system by and the computed floating-point solution of by means of the fast technique described in [14] by The fixpoint of the iteration function is the defect vector where is the exact solution of denotes its interval extension and if for some interval E,
[17]. If
where denotes the interior of the interval E, then the linear system has one and only one solution in the interval For classical Gaussian elimination with partial pivoting performed on the full matrix A instead of on the factors G and B of the error in say the width of is typically of the order of the product of the condition number of A and the machine epsilon where and respectively denote the radix and precision of the floating-point system in use. In Table 1 we illustrate that the fast Gaussian elimination with partial pivoting performed on the factors G and B enjoys the same property, under the condition that (6) is not too small. This is in fact an optimal result for a fast linear system solver. In Table 1 the value diam(E) with is defined by
Let us denote the matrix elements in the factors G and B of (5) by and According to [1], instabilities can occur if the size of the matrix elements
is small compared to that of the elements Therefore, in the Tables 3 and 4, the value
is tabulated together with the evaluations of each scattered rational interpolant, the norms of the residue and normalized residue and the condition number of the square matrix A.
Multivariate Rational Interpolation of Scattered Data
211
212
Stefan Becuwe, Annie Cuyt, and Brigitte Verdonk
As an illustration of the technique, we apply the structured Cauchy-Vandermonde solver to the computation of some bivariate rational interpolants from scattered data obtained from the Beta-function
where denotes the Gamma function. The Beta function is an interesting example because of its meromorphy. By means of the recurrence formulas
for the Gamma function, we can write
If we approximate the function by a rational interpolant and plug this approximant into (7), we obtain a rational approximant for Because of the location of the poles of at and for and its zeroes at for we choose
With this choice of index sets, and For the interpolation points indexed by we choose randomly generated tuples in the domain [–1,1] × [–1,1]. The displacement rank of equals Since the technique described in Sect. 3 significantly reduces the complexity when is larger. All rational interpolants are evaluated in the 16 points
The exact value of the Beta function in these 16 points can be found in Table 2.
References 1. Brent, R.P.: Stability of fast algorithms for structured linear systems. In Kailath, T., Sayed, A.H., eds.: Fast reliable algorithms for matrices with structure. SIAM, Philadelphia, PA (1999) 103–116 2. Bultheel, A.: Algorithms to compute the reflection coefficients of digital filters. In Collatz, L., et al., eds.: Numer. Math. Approx. Th. 7. (1984) 33–50 3. Bultheel, A., Van Barel, M.: Padé techniques for model reduction in linear system theory : a survey. J. Comput. Appl. Math. 14 (1986) 401–438
Multivariate Rational Interpolation of Scattered Data
213
4. Chaffy, C.: Interpolation polynomiale et rationnelle d’une fonction de plusieurs variables complexes. Thèse, Institut National Polytechnique de Grenoble (1984) 5. Cuyt, A.: A recursive computation scheme for multivariate rational interpolants. SIAM J. Numer. Anal. 24 (1987) 228–239 6. Cuyt, A., Lenin, R.B.: Computing packet loss probabilities in multiplexer models using adaptive rational interpolation with optimal pole placement. (IEEE Transactions on Computers) Submitted for publication. 7. Cuyt, A., Lenin, R.B.: Multivariate rational approximants for multiclass closed queuing networks. IEEE Transactions on Computers 50 (2001) 1279–1288 8. Cuyt, A., Ogawa, S., Verdonk, B.: Model reduction of multidimensional linear shift-in variant recursive systems using Padé techniques. Multidimensional systems and Signal processing 3 (1992) 309–321 9. Cuyt, A., Verdonk, B.: Different techniques for the construction of multivariate rational interpolants. In Cuyt, A., ed.: Nonlinear numerical methods and rational approximation (Wilrijk, 1987). Reidel, Dordrecht (1988) 167–190 10. Cuyt, A.: A multivariate qd-like algorithm. BIT 28 (1988) 98–112 11. Cuyt, A., Verdonk, B.: General order Newton-Padé approximants for multivariate functions. Numer. Math. 43 (1984) 293–307 12. Cuyt, A., Verdonk, B.: A review of branched continued fraction theory for the construction of multivariate rational approximants. Appl. Numer. Math. 4 (1988) 263–271 13. De Geest, J., Dhaene, T., Faché, N., De Zutter, D.: Adaptive CAD-model building algorithm for general planar microwave structures. IEEE Transactions on microwave theory and techniques 47 (1999) 1801–1809 14. Gohberg, I., Kailath, T., Olshevsky, V.: Fast Gaussian elimination with partial pivoting for matrices with displacement structure. Math. Comp. 64 (1995) 1557– 1576 15. Gong, W.B., Nananukul, S.: Rational interpolation for rare event probabilities. In: Stochastic Networks: Lecture Notes in Statistics. (1996) 139–168 16. Gong, W.B., Yang, H.: Rational approximants for some performance analysis problems. IEEE Trans. Comput. 44 (1995) 1394–1404 17. Hammer, R., Hocks, M., Kulisch, U., Ratz, D.: Numerical Toolbox for verified computing I. Springer-Verlag (1993) 18. Kailath, T., Kung, S.Y., Morf, M.: Displacement ranks of matrices and linear equations. J. Math. Anal. Appl. 68 (1979) 395–407 19. Kuchminskaya, K.I.: On approximation of functions by two-dimensional continued fractions. In Gilewicz, J., Pindor, M., Siemaszko, W., eds.: Rational approximation and applications in mathematics and physics ( 1985). Volume 1237 of Lecture Notes in Mathematics. Springer, Berlin (1987) 207–216 20. Lehmensiek, R., Meyer, P.: An efficient adaptive frequency sampling algorithm for model-based parameter estimation as applied to aggressive space mapping. Microwave and Optical Technology Letters 24 (2000) 71–78 21. O’Donohoe, M.: Applications of continued fractions in one and more variables. PhD thesis, Brunel University (1974) 22. Siemaszko, W.: Thiele-type branched continued fractions for two-variable functions. J. Comput. Appl. Math. 9 (1983) 137–153
Approximation Methods for Nonconvex Parabolic Optimal Control Problems Using Relaxed Controls Ion Chryssoverghi
Department of Mathematics, National Technical University of Athens Zografou Campus, 15780 Athens, Greece
[email protected]
Abstract. We consider an optimal distributed control problem involving semilinear parabolic partial differential equations, with control and state constraints. Since no convexity assumptions are made, the problem is reformulated in relaxed form. The state equation is discretized using a finite element method in space and a in time, while the controls are approximated by blockwise constant relaxed controls. The first result is that, under appropriate assumptions, the properties of optimality, and of extremality and admissibility, carry over in the limit to the corresponding properties for the relaxed continuous problem. We also propose progressively refining discrete conditional gradient and gradient-penalty methods, which generate relaxed controls, for solving the continuous relaxed problem, thus reducing computations and memory. Numerical examples are given.
1 Introduction Optimal control problems, without strong and often unrealistic convexity assumptions, have no classical solutions in general. For this reason they are reformulated in the so-called relaxed form, which in turn has a solution under mild assumptions. Relaxation theory has been extensively used to develop not only existence theory, but also necessary conditions for optimality and optimization methods and approximation methods (see [1–10], and references in [8]). Here we consider an optimal distributed control problem involving semilinear parabolic PDE’s, with state constraints. Since no convexity assumptions are made, the problem is reformulated in relaxed form. The state equation is then discretized using a finite element method in space and a in time (extending the method in [2] and [5]), while the controls are approximated by blockwise constant relaxed controls. The first result is that, under appropriate assumptions, the properties of optimality, and extremality and admissibility, carry over in the limit to the corresponding properties for the relaxed continuous problem (extending the results in [2]). In addition, we propose progressively refining discrete conditional gradient and gradient-penalty methods, which generate relaxed controls, for solving the continuous relaxed problem (extending [5]). The refining I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 214–221, 2004. © Springer-Verlag Berlin Heidelberg 2004
Approximation Methods for Nonconvex Parabolic Optimal Control Problems
215
procedure has the advantage of reducing computing time and memory. On the other hand, the use of relaxed controls exploits the nonconvex structure of the problem. Finally, two numerical examples are given.
2
The Continuous Optimal Control Problems
Let be a bounded domain in with a Lipschitz boundary (0,T). Consider the following semilinear parabolic state equation in in and in where is the second order elliptic differential operator
and let I =
The constraints on the control variable are in Q, where U is a compact, not necessarily convex, subset of The constraints on the state and the control variables are
and the cost functional to be minimized Define the set of classical controls measurable from Q to U}, and the set of relaxed controls (Young measures, for relevant theory, see [8, 9]) by weakly measurable } where is the set of probability measures on U. The set W (resp. R) is endowed with the relative strong (resp. weak star) topology, and R is convex, metrizable and compact. If we identify each classical control with its associated Dirac relaxed control then W may be considered as a subset of R, and W is thus dense in R. For given and we write for simplicity The relaxed formulation of the above control problem is the following for every
a.e. in I, and in where is the usual bilinear form associated with < .,. > the duality between V and with control constraint state constraints
in
and cost to be minimized
Ion Chryssoverghi
216
We suppose in the sequel that functions are subquadratic w.r.t.
is Lipschitz and sublinear w.r.t. and sublinear w.r.t.
Theorem 1. The mappings
and
the
are continuous.
Theorem 2. if the relaxed problem is feasible, then it has a solution. Since
we have in general
and if there are no state constraints, since W is dense in R It can be shown (see [2]) that, for given controls the directional derivative of the functional J (we drop the index ) is given by where the Hamiltonian is and the general adjoint
satisfies the equation in Q, in and in We have the following relaxed necessary conditions for optimality.
Theorem 3.
If is optimal, then it is extremal, i.e. there exist multipliers not all zero, such that for every (Transversality conditions),
where
is replaced by
in the definitions of a.e. in Q
Theorem 4. The mappings continuous.
3
and
H, or equivalently
(Minimum principle),
are
Discretization
The set is supposed here to be a polyhedron, for simplicity. For each we choose a finite element discretization, with piecewise affine basis functions of a subspace of V, w.r.t. an admissible and regular triangulation of into simplices of maximum diameter and a time partition of I into equal subintervals of length The set of discrete relaxed controls is the set of relaxed controls that are equal to a constant measure in on each block Consider the following discrete problems, with state equation defined by the implicit for every given,
Approximation Methods for Nonconvex Parabolic Optimal Control Problems
control constraint
217
discrete functionals
either of the two cases of perturbed discrete equality constraints case (i) or case (ii) perturbed discrete inequality constraints where
and cost functional to be minimized
Theorem 5. The mappings
and
are continuous.
Theorem 6. If the discrete problem (case (i) or (ii)) is feasible, then it has a solution. Dropping
the general discrete adjoint equation is here
for every and the directional derivative of
Theorem 7. (Constraint case (ii)). If is optimal, then it is extremal, i.e. there exist multipliers, similar to the continuous case, such that
where
is replaced by
or equivalently
Theorem 8. The mappings tinuous.
4
are con-
Behavior in the Limit
Theorem 9. (Control approximation) For each that converges to in R (the controls blockwise constant classical ones).
there exists a sequence can even be chosen to be
We suppose that in V strongly. In addition, if that for an appropriate Theorem 10. (Consistency) If
in R, then
we suppose
218
Ion Chryssoverghi
The two following results concern the behavior in the limit of the properties of optimality, and of extremality and admissibility. Theorem. 11. We suppose that the continuous relaxed problem is feasible. Let be some optimal control for the continuous relaxed problem, and any sequence converging to Let be a sequence of optimal controls for the discrete problems (constraint case (i)), where the are such that as and satisfy the admissibility conditions and If is sufficiently small, then every accumulation point of is optimal for the continuous relaxed problem Theorem 12. Let be a sequence of extremal and admissible controls for the discrete problems (constraint case (ii)), where as and the perturbations satisfy a (constructive) minimum feasibility condition (see [4]). If is sufficiently small, then every accumulation point of is extremal and admissible for the continuous relaxed problem.
5
Discrete Optimization Methods
Using the above Theorem 12, one can directly apply some optimization method to the discrete problem (case (ii)), for some fixed, sufficiently large, Here, we propose relaxed discrete, progressively refining, conditional gradient and conditional gradient-penalty methods, where the discretization is refined according to some convergence criterion. This refining procedure has the advantage of reducing computations and memory and completely avoiding the consideration of discrete problems and the computation of their perturbations We suppose that, for every either and or each simplex is a subset of some and each interval a subset of some Let be nonnegative increasing sequences, such that as and define the penalized discrete functionals (we use here the common index for discretization and penalization)
Let with positive decreasing sequences converging to zero, and consider the following algorithm. Algorithm: Relaxed conditional gradient/gradient-penalty method Step 1. Set Step 2. Find
and choose such that
Step 3. If Step 4. Either (a) (Optimal step option) Find
and go to Step 2. such that
Approximation Methods for Nonconvex Parabolic Optimal Control Problems
or (b) (Armijo step option) Find the smallest positive integer satisfies the inequality Step 5. Choose any Set
219
such that
such that and go to Step 2.
The Armijo step option is a finite procedure and is practically faster than the optimal step option (use of golden section search). If there are no state constraints, we set and the above method reduces to a relaxed conditional gradient method. In the state constrained case (conditional gradientpenalty method), using the sequence generated in Step 3 of the algorithm, we define the sequences of multipliers
Theorem 13. (A) (Conditional Gradient Method) Every accumulation point of the sequence generated by the algorithm in Step 3 is extremal for the continuous relaxed problem. (B) (Conditional Gradient-Penalty Method) Let be a subsequence of the sequence generated by the algorithm in Step 3, which converges to some control (i) If the sequences are bounded, then is admissible and extremal for the continuous relaxed problem. (ii) Suppose that the continuous relaxed problem has no admissible abnormal extremal controls (see [9]). If is admissible, then the sequences are bounded and is extremal for the continuous relaxed problem. Proof. (For the optimal step option) Suppose that remains constant after some iterations. Then the penalty factors remain also constant. By construction, Let and be convergent subsequences and suppose that Using the Mean Value Theorem, we have for
where
for
for the convergence 5). Therefore
as
and
Hence
Hence, by Step 4 (a), we have
This implies that
which contradicts due to the continuity of (Theorem and we must have
Ion Chryssoverghi
220
Now, let any If each sequence
i.e. 10)
and
By Steps 2 and 4
is bounded (and we can suppose that
), we have
is admissible. Passing to the limit in inequality (1), we obtain (Theorem
If for some Therefore, is extremal. If is admissible, suppose that setting
and we easily see that
then clearly
for large
hence
for some Dividing inequality (1) by and passing to the limit in (1), we find
is abnormal extremal, which contradicts our assumption.
For the implementation of algorithms generating relaxed controls, see [5]. Approximate discrete relaxed controls computed by the above algorithm can then be simulated by piecewise constant classical controls using a simple approximation procedure (see [2]).
6
Numerical Examples
a) Let
and define the state
and the control
Consider the following optimal control problem, with state equation
(nonconvex) control constraints and (nonconvex) cost functional It can be verified that the optimal relaxed control is given by (non-classical part), (classical part), with optimal state relaxed cost value
and optimal cost Since the optimal cannot be attained by classical controls due to the con-
Approximation Methods for Nonconvex Parabolic Optimal Control Problems
221
straints, but can be approximated as close as possible since W is dense in R, the classical problem cannot have a solution. After 100 iterations of the relaxed conditional gradient method, with optimal step option, and with three successive pairs of step sizes we obtained b) Modifying the main state equation and imposing the additional equality state constraint the relaxed conditional gradient-penalty method with optimal step yielded, after 100 iterations totally (i.e. in the results
References l. Chryssoverghi, I., Bacopoulos, A.: Discrete approximation of relaxed optimal control problems. J. Optim. Theory and Appl., 65 (1990) 395-407 2. Chryssoverghi, I., Bacopoulos, A.: Approximation of relaxed nonlinear parabolic optimal control problems. J. Optim. Theory and Appl., 77 (1993) 31-50 3. Chryssoverghi, I., Bacopoulos, A., Kokkinis, B., Coletsos, J.: Mixed Frank-Wolfe penalty method with applications to nonconvex optimal control problems. J. Optim. Theory and Appl., 94 (1997) 311-334 4. Chryssoverghi, I., Bacopoulos, A., Coletsos, J., Kokkinis, B.: Discrete approximation of nonconvex hyperbolic optimal control problems with state constraints. Control & Cybernetics. 27 (1998) 29-50 5. Chryssoverghi, I., Coletsos, J., Kokkinis, B.: Discrete relaxed method for semilinear parabolic optimal control problems. Control & Cybernetics. 28 (1999) 157-176 6. Chryssoverghi, I., Coletsos, J., Kokkinis, B.: Approximate relaxed descent method for optimal control problems. Control & Cybernetics. 30 (2001) 385-404 7. A convergent computational method for constrained optimal relaxed control problems. J. Optim. Theory and Appl., 69 (1991) 589-603 8. Relaxation in Optimization Theory and Variational Calculus. Walter de Gruyter, Berlin. (1997) 9. Warga, J.: Optimal Control of Differential and Functional Equations. Academic Press, New York. (1972) 10. Warga, J.: Steepest descent with relaxed controls. SIAM J. on Control. 15 (1977) 674-682
Stabilizing Feedback of a Nonlinear Biological Wastewater Treatment Plants Model Neli Dimitrova and Mikhail Krastanov Institute of Mathematics and Informatics, Bulgarian Academy of Sciences Acad. G. Bonchev str. bl. 8, 1113 Sofia
[email protected],
[email protected]
Abstract. The present paper deals with a nonlinear anaerobic digester model of wastewater treatment plants. The asymptotic stabilizability of this control system is studied under suitable assumptions and a smooth stabilizing feedback is proposed.
1 Introduction The last decades have been marked by a rapid interaction between mathematics and biology, forcing the implementation of new mathematical methods in the biological research. Typically, problems arising in biological application involve uncertain data in the form of intervals [5,6]. The development and implemetation of the new methodology for treatment of problems, which data are “unknown but bounded” is one of the S. Markov’s many contributions to modern mathematical biology. The authors greatfully acknowledge S. Markov’s generous support in their efforts to promote advanced methods in biological problems. In this paper we consider an anaerobic digester model of biological wastewater treatment plants (WWTPs), described by the following control system [3]
where
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 222–229, 2004. © Springer-Verlag Berlin Heidelberg 2004
Stabilizing Feedback
223
The definition of the model variables and parameters is listed in Table 1. It is assumed that the dilution rate is a control input in (1)–(6). The numerical values for and Q are taken from the graphical outputs in [3]. This model has been experimentally validated for an anaerobic up-flow fixed bed reactor used for the treatment of industrial wine distillery wastewater. More details about that can be found in [3] and in the references there. The paper is organized as follows. Section 2 presents steady state analysis of the system. Some considerations regarding the linearized system are presented in Section 3. A continuous feedback stabilizing asymptotically the dynamics is proposed in Section 4. Computer simulations are reported in Section 5.
2
Steady States Analysis
The steady states are obtained as solutions of a nonlinear algebraic system, obtained from (1)–(6) by setting
224
Neli Dimitrova and Mikhail Krastanov
Excluding the trivial solutions (which are called wash-out steady states and are not of practical interest), it is straightforward to see that for any where
the steady states (chosen for biological reasons as one of the two possible nontrivial solutions) are
Let us fix a point
Denote
The point is called rest point of (1)–(6). Let be a compact neighborhood of In the following we shall assume that is the set of the admissible values of the control. We shall study the stabilizability of the system (l)–(6) in a neighborhood of the point where the “size” of depends on the choice of
3
Stability of the Linearization of the Model
In this section we consider the system consisting of the first five ODEs (l)–(5) of the model with
and make the following change:
Denote Using the Taylor expansion around the origin, the system (in the new coordinates) can be written in the form
where C is the Jacobian of the right-hand side functions with respect to at the point is the vector of the derivatives of the right-hand side functions with respect to at the point is the residual, which can be easily computed.
Stabilizing Feedback
225
A standard approach in the nonlinear control theory is to try to stabilize the linear approximation of the system (7) at the origin. Using the numerical values for the model parameters in Table 1, one can check that the matrix is regular. Therefore, according to the Kalman controllability criterion, the above linear system is controllable at the origin. We decided to apply an approach based on the Ackermann formula (cf. [1,2]). Let be arbitrary negative real numbers. Denote
Then the linear approximation takes the form
It can be directly checked that the eigenvalues of the matrix are Since are negative, the linear system (8) is asymptotically stable. Unfortunately the computer simulation showed very bad convergence of the solution of (8) to the rest point Therefore we decided to investigate the Lyapunov function of (8). Using Claim 5.35 ([7], p. 211), a symmetric positive matrix P is found by solving the linear algebraic system where E is the identity matrix. Then is the Lyapunov function of the linear system (8). Using the numerical values in Table 1 (with and choosing the matrix P and its eigenvalues can be explicitely computed; the latter are
Since the difference
between the maximal and minimal eigenvalue of P,
is too large, the computer simulations show very bad convergence of the solution of (8) to the rest point As a next step we tried to minimize by solving a nonlinear optimization problem. Unfortunately, all efforts to stabilize the linear approximation (8) of the nonlinear system (7) were useless – in all cases the convergence to the rest point was very slow and practically unusable.
4
Stabilization of the Nonlinear System
We consider the nonlinear dynamic system (1)–(6) with This value of corresponds to the ideal continuous stirred tank reactor [3]. For all considerations can be done in a similar way.
226
Neli Dimitrova and Mikhail Krastanov
Let be the rest point corresponding to the fixed (see Section 2) from the interior int of the set of admissible values for the control. It can be directly verified that the following subset of
is invariant with respect to the system (1)–(6) for every admissible control (i. e. for every integrable function with values from ). This means that the trajectory of (1)–(6) remains in H for every choice of the admissible control whenever the starting point belongs to H. Moreover, starting from a point outside the set H, the trajectory of (1)–(6) tends to H for every choice of the control function This observation is the main motivation to define the feedback
where
is a parameter that will be chosen later in a suitable way.
Definition 1. Let be a neighborhood of the rest point P*. Any locally Lipschitz continuous function is called admissible feedback. It is said that the admissible feedback stabilizes asymptotically the control system (1)–(6) if: i) for every point the trajectory of (1)–(6) starting from and corresponding to the control is defined for every ii) for every iii) Theorem 1. There exists a neighborhood of P* and a number such that the feedback (9) is admissible and stabilizes asymptotically the control system (1)–(6) to the point P*. Proof. Since
the positive real number
belongs to int Hence, there exist such that the feedback (9) is admissible on the neighborhood where
Let
and of the point P*,
be an arbitrary point and let be the solution of (1)–(6) starting from and corresponding to the feedback It can be directly checked that is well defined on and for every (according to Theorem 2.4 ([4], p. 194)). Denote further
Stabilizing Feedback
There exist constants
and
227
such that for every
hold true. Moreover,
The above relations and (10) imply Since and it follows that Both relations
and
whenever
Further the definition of and lead to
implies
Taking into account the equalities
we obtain further
Denote
Using the expressions for
and
one gets
By means of the numerical values of the model parameters (see Table 1) it can be verified that in the neighborhood with (and Hence, there exists a constant M, such that for every for Therefore,
Neli Dimitrova and Mikhail Krastanov
228
where Taking into account that whenever we obtain The Cauchy formula for solution of linear ODEs and the Gronwall inequality imply
which means that Further the relations
lead to
Finally, (5), (10) and the equality imply
Using again the Cauchy formula and the relations and we obtain as above that This completes the proof.
5
Numerical Simulation
All computations and graphical outputs are carried out in the computer algebra system Maple 7. Using the numerical values from Table 1, the admissible interval for the control is U = (0, 0.5359]. With the rest point P* is
Let the initial point at
be
Using the feedback (9) with the system (1)–(6) is solved numerically on a uniform mesh with The numerical outputs are visualized in different phase planes (see Figures 1 and 2). In all plots the symbol circle means the projection of the starting point and the symbol diamond denotes the projection of the rest point in the corresponding coordinates. Figure 1 (left) visualizes the projection of the neighborhood in the at all points below the curve the feedback law is admissible.
Stabilizing Feedback
Fig. 1. Plots of the trajectories in
Fig. 2. Plots of the trajectories in
(left) and
(left) and
229
(right) coordinates
(right) coordinates
Acknowledgments This research has been partially supported by the Swiss NSF Project No. 7 IP 65642 (Program SCOPES) and by the Bulgarian National Science Fund under contract No. MM-807/98 and MM-1104/01.
References 1. Ackermann, J.: Sampled-Data Control Systems. Springer-Verlag, Berlin (1985). 2. Ackermann, J., Utkin, V.: Sliding Mode Control Design Based on Ackermann’s Formula. IEEE Transactions on Automatic Control 43, No. 2 (1998) 234–237. 3. Alcaraz-González, V., Harmand, J., Rapaport, A., Steyer, J. P., González–Alvarez, V., Pelayo–Ortiz, C.: Software sensors for highly uncertain WWTPs: a new apprach based on interval observers. Water Research, 36 (2002) 2515–2524. 4. Clarke, F., Ledyaev, Yu., Stern, R., Wolenski, P.: Nonsmooth analysis and control theory. Graduate text in Mathematics 178, Springer (1998). 5. Markov, S.: Mathematical modelling of dynamical processes under interval experimental data. IIASA WP-91-004 (1991) 1–29. 6. Markov, S.: Some problems of mathematical modelling in ecology involving uncertainties. Phytologia Balcanica, 3/2–3 (1997) 155–165. 7. Sastry, Sh.: Nonlinear Systems: Analysis, stability and control. Springer-Verlag, Berlin (1999).
Higher Order Approximations of Affinely Controlled Nonlinear Systems Nikolay Kirov and Mikhail Krastanov Institute of Mathematics and Informatics, Bulgarian Academy of Sciences Acad. G. Bonchev bl.8, 1113 Sofia {nkirov,krast}@math.bas.bg
Abstract. An numerical approach for numerical approximation of trajectories of a smooth affine control system is proposed under suitable assumptions. This approach is based on expansion of solutions of systems of ordinary differential equations (ODE) by Volterra series and allows to estimate the distance between the obtained approximation and the true trajectory.
1 Introduction To apply the traditional numerical schemes (such as Runge-Kutta schemes) of higher order for approximation trajectories of nonlinear control systems is a nontrivial task (cf. for example [4,11] etc.) due to the nonsmoothness of the control functions. For the case of linear differential inclusions, this problem is studied in [9]. There is proposed a numerical procedure based on a suitable approximation of integrals of multivalued mappings (cf. for example [10]) and on the ideology of algorithms with result verification (cf. for example [2]). Another approach for approximating trajectories of affinelly controlled systems is proposed in [5] and [6]. This approach is based on the expansion of the solution of the systems of ordinary differential equations by Volterra series (cf. for example [7]). In this note, combining this approach with the ideas developed in [9], we propose a method for approximation of trajectories of analytic control systems with guaranteed accuracy. We would like to point out that our approach can be applied for general control systems that are smooth with respect to the phase variables and continuous with respect to the control.
2
Systems of ODE and Volterra Series
First, we introduce briefly some notations and notions: For every point from we set and let B be the unit ball in (according to this norm) centered at the origin. Let C denote the set of all complex numbers, and be a convex compact neighbourhood of the point If then by Re and Im we denote its norm, its real and imaginary parts, respectively. Let We set
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 230–237, 2004. © Springer-Verlag Berlin Heidelberg 2004
Higher Order Approximations of Affinely Controlled Nonlinear Systems
231
By we denote the set of all real analytic functions defined on such that every has a bounded analytic extension on We define a norm in the set as follows:
Let usually, we identify
Let
be a vector field defined on with the corresponding differential operator
As
be the set of all real analytic vector fields defined on such that every belongs to We define the following norm in
An integrable analytic vector field (parameterized by is a map such that: i) for every the functions are measurable; ii) for every and where is an integrable (on every compact interval) function. Further we shall consider only uniformly integrable vector fields i.e.
Let M be a compact set contained in the interior of and containing the point and let be an integrable analytic vector field defined on Then there exists a real number such that for every point of M the solution of the differential equation
is defined on the interval In this case we denote by morphism defined by
and
for every T from the diffeo-
According to proposition 2.1 from [1], can be chosen in such a way that for every T, for every point from M and for every function from the following expansion of holds true:
232
Nikolay Kirov and Mikhail Krastanov
where
and the series is absolutely convergent. The proof of these identities is based on the following estimate: for every positive real numbers for every point of M, for every function from and for every points from the following inequality holds true
This estimate implies the following technical lemma: Lemma 1. Let M be a convex compact subset of We set
Then for every inequality holds true:
and for every two points
and
of M the following
Let the functions be defined as follows: We set Applying Lemma 1 we obtain that for every two points and from M the following estimate holds true
We use the estimate (4) to prove an existence criterion for where belongs to some compact set M: Proposition 1. Let M and be convex compact subsets of and be a positive integer such that for every point the following relations hold true:
and for every
Higher Order Approximations of Affinely Controlled Nonlinear Systems
Then
is well defined for every point for every point and for every
233
Moreover, from
Proof. Let be an arbitrary point from M. By we denote the set of all continuous functions defined on with values from the set We can define the following operator as follows:
Let
denote the usual uniform norm in
i.e.
where
This operator is well defined on Let and be arbitrary elements of Applying the estimate (4), we obtain that
where
Since the differential operator F is contractive. Applying the Banach fixed point theorem we obtain that there exists an unique function from such that The last relation means that This completes the proof.
234
Nikolay Kirov and Mikhail Krastanov
The next proposition helps us to estimate the global approximation error using the local approximation error and motivates our approximation procedure. Proposition 2. Let
is well defined and
Let
for every
Then there exist compact subsets to and
of
such that
belongs
where Proof. We set and define the sets If is already defined, we set
We set verified that
Let us denote
as follows: where
It can be directly and
by
Clearly, where
Let us assume that Then
Higher Order Approximations of Affinely Controlled Nonlinear Systems
235
Hence, we have proved that
and
where
Applying theorem 1.7 (p. 182, [3]), we obtain that for every
Substituting
3
and
we complete the proof.
A Computational Procedure
Let us consider the following control system:
where the state variable belongs to are real analytic vector functions and the admissible controls are the Lebesgue integrable functions. Let and be an admissible control. First, using Proposition 1 we can find a positive real T (not greater than and a compact subset of containing the point such that the corresponding trajectory is well defined on and for every from Next, we show how we can calculate a point such that We assume that for some and for We set and
Let For every positive integer we set define the points where sufficiently large, so that the following relation holds true:
We choose
and to be
236
Nikolay Kirov and Mikhail Krastanov
Next we choose a value for
such that the following inequality holds true:
At the end we determine the accuracy of the desirable local approximation by choosing the positive integer to be so large that
Since the trajectory is well defined on [0, T] and belongs to we obtain according to Proposition 2 that where and for every
In [8] is studied the following control systems:
All time-optimal controls are piecewise constant with at most two pieces. The constant controls and generate the lower boundaries of the reachable sets and respectively. The controls with for and for steer to the curves of endpoints and respectively. Figure 1 shows the approximate reachable set for the system at the moment using first and second order terms in Volterra series. The timestep is
Fig. 1. First and second order terms The exact reachable set is presented on Fig. 2, using also third order terms in Volterra series. It is enough to take timestep equal to 1 in this case.
Higher Order Approximations of Affinely Controlled Nonlinear Systems
237
Fig. 2. The exact reachable set
Acknowledgments This research was partially supported by Swiss NSF No. 7 IP 65642 (Program SCOPES) and by the Ministry of Science and Higher Education - National Fund for Science Research under contracts MM-807/98 and MM-1104/01.
References 1. A. Agrachev, R. Gamkrelidze, The exponential representation of flows and the chronological calculus, Math. USSR Sbornik 107 (1978), 467–532. 2. P. Bochev, S. Markov, A self-validating method for matrix exponential, Computing 43 (1989), 59–72. 3. F. Clarke, Yu. Ledyaev, R. Stern, P. Wolenski: Nonsmooth analysis and control theory, Graduate text ion Mathematics 178, Springer, 1998. 4. A. Dontchev, F. Lempio, Difference methods for differential inclusions, SIAM J. Review 34 (1992), 263–294. 5. L. Grüne, P. Kloeden, Higher order numerical schemes for affinely controlled nonlinear systems, Numer. Math. 89 (2001), 669–690. 6. L. Grüne und P. E. Kloeden, Numerical schemes of higher order for a class of nonlinear control systems, In: I. Dimov, I. Lirkov, S. Margenov, Z. Zlatev, eds.: Numerical Methods and Applications, Lecture Notes in Computer Science, 2542, Springer Verlag, (2003), 213–220. 7. A. Isidori, Nonlinear control systems. An introduction, Springer, 1995. 8. M. Kawski, High-order Maximal Principle, preprint, 2003. 9. M. Krastanov, N. Kirov, Dynamic Interactive System for Analysis of Linear Differential Inclusions, “Modeling techniques for uncertain systems”, A. Kurzhanski, V. Veliov (Eds.), Birkhauser, 123–130, 1994. 10. V. Veliov, Discrete approximations to integrals of multivalued inclusions, Compt. Rend. Acad. Bulg. Sci. 42 (1989), 51–54. 11. V. Veliov, On the time discretization of control systems, SIAM J. Control Optim. 35 (1997), 1470–1486.
Outlier Detection under Interval Uncertainty: Algorithmic Solvability and Computational Complexity Vladik Kreinovich 1, Luc Longpré 1 , Praveen Patangay1, Scott Ferson2, and Lev Ginzburg 2 1
Computer Science Department, University of Texas at El Paso El Paso, TX 79968, USA {vladik,longpre,praveen}@cs.utep.edu 2
Applied Biomathematics, 100 North Country Road Setauket, NY 11733, USA {scott,lev}@ramas.com
Abstract. In many application areas, it is important to detect outliers. Traditional engineering approach to outlier detection is that we start with some “normal” values compute the sample average E, the sample standard variation and then mark a value as an outlier if is outside the interval (for some preselected parameter In real life, we often have only interval ranges for the normal values In this case, we only have intervals of possible values for the bounds and We can therefore identify outliers as values that are outside all intervals. In this paper, we analyze the computational complexity of these outlier detection problems, and provide efficient algorithms that solve some of these problems (under reasonable conditions).
1 Introduction In many application areas, it is important to detect outliers, i.e., unusual, abnormal values. In medicine, unusual values may indicate disease (see, e.g., [7]); in geophysics, abnormal values may indicate a mineral deposit or an erroneous measurement result (see, e.g., [5,9,13,16]); in structural integrity testing, abnormal values may indicate faults in a structure (see, e.g., [2,6,7,10,11,17]), etc. Traditional engineering approach to outlier detection (see, e.g., [1,12,15]) is as follows: first, we collect measurement results uations; then, we compute the sample average
corresponding to normal sit-
mal values and the (sample) standard deviation
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 238–245, 2004. © Springer-Verlag Berlin Heidelberg 2004
of these norwhere
Outlier Detection under Interval Uncertainty
239
finally, a new measurement result is classified as an outlier if it is outside the interval [L,U] (i.e., if either or where and is some pre-selected value (most frequently, 3, or 6). In some practical situations, we only have intervals of possible values of This happens, for example, if instead of observing the actual value of the random variable, we observe the value measured by an instrument with a known upper bound on the measurement error; then, the actual (unknown) value is within the interval For different values we get different bounds L and U. Possible values of L form an interval – we will denote it by possible values of U form an interval How do we now detect outliers? There are two possible approaches to this question: we can detect possible outliers and we can detect guaranteed outliers: a value
is a possible outlier if it is located outside one of the possible intervals [L, U] (but is may be inside some other possible interval
[L,U]); a value is a guaranteed outlier if it is located outside all possible intervals [L, U]. Which approach is more reasonable depends on a possible situation: if our main objective is not to miss an outlier, e.g., in structural integrity tests, when we do not want to risk launching a spaceship with a faulty part, it is reasonable to look for possible outliers; if we want to make sure that the value is an outlier, e.g., if we are planning a surgery and we want to make sure that there is a micro-calcification before we start cutting the patient, then we would rather look for guaranteed outliers. The two approaches can be described in terms of the endpoints of the intervals L and U: A value guaranteed to be normal – i.e., it is not a possible outlier – if belongs to the intersection of all possible intervals [L, U]; the intersection corresponds to the case when L is the largest and U is the smallest, i.e., this intersection is the interval So, if or then is a possible outlier, else it is guaranteed to be a normal value. If a value is inside one of the possible intervals [L, U], then it can still be normal; the only case when we are sure that the value is an outlier is when is outside all possible intervals [L, U], i.e., is the value does not belong to the union of all possible intervals [L, U] of normal values; this union is equal to the interval So, if or then is a guaranteed outlier, else it can be a normal value. In real life, the situation may be slightly more complicated because, as we have mentioned, measurements often come with interval inaccuracy; so, instead of the exact value of the measured quantity, we get an interval of possible values of this quantity.
Vladik Kreinovich et al.
240
In this case, we have a slightly more complex criterion for outlier detection: the actual (unknown) value of the measured quantity is a possible outlier if some value from the interval is a possible outlier, i.e., is outside the intersection thus, the value is a possible outlier if one of the two inequalities hold: or the actual (unknown) value of the measured quantity is guaranteed to be an outlier if all possible values from the interval are guaranteed to be outliers (i.e., are outside the union thus, the value is a guaranteed outlier if one of the two inequalities hold: or Thus: to detect possible outliers, we must be able to compute the values and to detect guaranteed outliers, we must be able to compute the values and In this paper, we consider the problem of computing these bounds.
2
What Was Known before
As we discussed in the introduction, to detect outliers under interval uncertainty, we must be able to compute the range of possible values of and the range of possible values of In [3,4], we have shown how to compute the intervals and of possible values for E and In principle, we can use the general ideas of interval computations to combine these intervals and conclude, e.g., that L always belongs to the interval However, as often happens in interval computations, the resulting interval for L is wider than the actual range – wider because the values E and are computed based on the same inputs and cannot, therefore, change independently. We mark a value as an outlier if it is outside the interval [L,U]. Thus, if, instead of the actual ranges for L and U, we use wider intervals, we may miss some outliers. It is therefore important to compute the exact ranges for L and U. In this paper, we show how to compute these exact ranges.
3
Detecting Possible Outliers
To find possible outliers, we must know the values and In this section, we design feasible algorithms for computing the exact lower bound of the function U and the exact upper bound of the function L. Specifically, our algorithms are quadratic-time, i.e., require computational steps (arithmetic operations or comparisons) for interval data points The algorithms for computing and for computing are as follows: In both algorithms, first, we sort all take line is divided into zones
values and
into a sequence Thus, the real
Outlier Detection under Interval Uncertainty
For each of these zones
and equation
241
we compute the values
total number of such
and
Then, we solve the quadratic
where
For computing and
we select only those solutions for which and for computing we select only those solutions for which For each selected solution, we compute the
values of
and, correspondingly,
Finally, if we are computing we return the smallest of the values if we are computing we return the smallest of the values Theorem 1. The algorithms time.
and
always compute
and
in quadratic
Comment. The main idea of this proof is given in the last (Proofs) section. The detailed proofs are given in http://www.cs.utep.edu/vladik/2003/tr03-10c.ps.gz and in http://www.cs.utep.edu/vladik/2003/tr03-10c.pdf
4
In General, Detecting Guaranteed Outliers Is NP-Hard
As we have mentioned in Section 1, to be able to detect guaranteed outliers, we must be able to compute the values and In general, this is an NP-hard problem: Theorem 2. For every of possible values of
computing the upper endpoint is NP-hard.
of the interval
Theorem 3. For every of possible values of
computing the lower endpoint is NP-hard.
of the interval
242
Vladik Kreinovich et al.
Comment. For interval data, the NP-hardness of computing the upper bound for was proven in [3] and [4]. The general overview of NP-hardness of computational problems in interval context is given in [8].
5
How Can We Actually Detect Guaranteed Outliers?
How can we actually compute these values? First, we will show that if (which is true, e.g., if and then the maximum of U (correspondingly, the minimum of L) is always attained at some combination of endpoints of the intervals thus, in principle, to determine the values and it is sufficient to try all combinations of values and Theorem 4. If then the maximum of the function U and the minimum of the function L on the box are attained at its vertices, i.e., when for every either or NP-hard means, crudely speaking, that there are no general ways for solving all particular cases of this problem (i.e., computing in reasonable time. However, we show that there are algorithms for computing and for many reasonable situations. Namely, we propose efficient algorithms that compute and for the case when all the interval midpoints (“measured values”) are definitely different from each other, in the sense that the “narrowed” intervals
where and intersect with each other. The algorithms and
is the interval’s half-width – do not are as follows:
In both algorithms, first, we sort all endpoints of the narrowed intervals and into a sequence This enables us to divide the real line into segments (“small intervals”) where we denoted and For each of small intervals we do the following: for each from 1 to we pick the following value of
if
then we pick
if then we pick for all other we consider both possible values and As a result, we get one or several sequences of for each small interval. To compute for each of the sequences we check whether, for the selected values the value of is indeed within the corresponding small interval, and if it is, compute the value Finally, we return the largest of the computed values U as
Outlier Detection under Interval Uncertainty
243
To compute for each of the sequences we check whether, for the selected values the value of is indeed within the corresponding small interval, and if it is, compute the value Finally, we return the smallest of the computed values L as Theorem 5. Let The algorithms and compute and in quadratic time for all the cases in which the “narrowed” intervals do not intersect with each other. These algorithms also work when, for some fixed C, no more than C “narrowed” intervals can have a common point: Theorem 6. Let For every positive integer C, the algorithms and compute and in quadratic time for all the cases in which no more than C “narrowed” intervals can have a common point. The corresponding computation times are quadratic in but grow exponentially with C. So, when C grows, this algorithm requires more and more computation time. It is worth mentioning that the examples on which we prove NP-hardness correspond to the case when out of narrowed intervals have a common point.
6
Proofs: Main Idea
Our proof of Theorem 2.1 is based on the fact that when the function attains its smallest possible value at some point then, for every the corresponding function of one variable
the function that is obtained from by fixing the values of all the variables except for – also attains its minimum at the value A differentiable function of one variable attains its minimum on a closed interval either at one of its endpoints or at an internal point in which its first derivative is equal to 0. This first derivative is equal to 0 when i.e., when where Thus, for the optimal values for which U attains its minimum, for every we have either or or We then show that if the open interval contains the value then the minimum of the function cannot be attained at points or therefore, has to be attained at the value We also show that: when the minimum cannot be attained for therefore, it is attained when when the minimum cannot be attained for therefore, it is attained when
and
and and
244
Vladik Kreinovich et al.
Due to what we have proven, once we know how the value is located with respect to all the intervals we can find the optimal values of Hence, to find the minimum, we need to analyze how the endpoints and divide the real line, and consider all the resulting sub-intervals.
Conclusions In many application areas, it is important to detect outliers. Traditional engineering approach to outlier detection is that we start with some “normal” values compute the sample average E, the sample standard variation and then mark a value as an outlier if is outside the interval (for some pre-selected parameter In real life, we often have only interval ranges for the normal values For different values we get different values of and – and thus, different intervals [L,U]. We can therefore identify guaranteed outliers as values that are outside all intervals, and possible outliers as values that are outside some intervals. To detect guaranteed and possible outliers, we must therefore be able to compute the range of possible values of L and the range of possible values of U. In our previous papers [3,4], we have shown how to compute the intervals and of possible values for E and In principle, we can combine these intervals and conclude, e.g., that L always belongs to the interval However, the resulting interval for L is wider than the actual range – wider because the values E and are computed based on the same inputs and are, therefore, not independent from each other. If, instead of the actual ranges for L and U, we use wider intervals, we may miss some outliers. It is therefore important to compute the exact ranges for L and U. In this paper, we showed that computing these ranges is, in general, NP-hard, and we provided efficient algorithms that compute these ranges under reasonable conditions.
Acknowledgments This work was supported in part by NASA grants NCC5-209 and grant NCC21232, by the Air Force Office of Scientific Research grant F49620-00-1-0365, by NSF grants CDA-9522207, EAR-0112968, EAR-0225670, and 9710940 Mexico/Conacyt, by IEEE/ACM SC2001 and SC2002 Minority Serving Institutions Participation Grants, by Small Business Innovation Research grant 9R44CA81741 to Applied Biomathematics from the National Cancer Institute (NCI), a component of the National Institutes of Health (NIH), and by a grant from Sandia National Laboratories as part of the Department of Energy Accelerated Strategic Computing Initiative (ASCI).
Outlier Detection under Interval Uncertainty
245
References 1. Devore, J., Peck, R.: Statistics: the Exploration and Analysis of Data, Duxbury, Pacific Grove, California (1999) 2. Ferregut, C., Osegueda, R.A., and Nuñez, A. (eds.): Proceedings of the International Workshop on Intelligent NDE Sciences for Aging and Futuristic Aircraft, El Paso, TX, September 30–October 2, 1997 3. Ferson, S., Ginzburg, L., Kreinovich, V., Longpré, L., Aviles, M.: Computing Variance for Interval Data is NP-Hard. ACM SIGACT News 33, No. 2 (2002) 108–118 4. Ferson, S., Ginzburg, L., Kreinovich, V., Aviles, M.: Exact Bounds on Sample Variance of Interval Data. In: Extended Abstracts of the 2002 SIAM Workshop on Validated Computing, Toronto, Canada (2002) 67–69 5. Goodchild, M., Gopal, S.: Accuracy of Spatial Databases, Taylor & Francis, London (1989) 6. Gros, X.E.: NDT Data Fusion, J. Wiley, London (1997) 7. Kosheleva, O., Cabrera, S., Osegueda, R., Nazarian, S., George, D.L., George, M.J., Kreinovich, V., Worden, K.: Case study of non-linear inverse problems: mammography and non-destructive evaluation. In: Mohamad-Djafari, A. (ed.): Bayesian Inference for Inverse Problems, Proceedings of the SPIE/International Society for Optical Engineering, vol. 3459, San Diego, CA (1998) 128–135. 8. Kreinovich, V., Lakeyev, A., Rohn, J., Kahl, P.: Computational complexity and feasibility of data processing and interval computations, Kluwer, Dordrecht (1997) 9. McCain, M., William, C.: Integrating Quality Assurance into the GIS Project Life Cycle. Proceedings of the 1998 ESRI Users Conference http://www.dogcreek.com/html/documents.html
10. Osegueda, R., Kreinovich, V., Potluri, L., Aló, R.: Non-Destructive Testing of Aerospace Structures: Granularity and Data Mining Approach. Proceedings of FUZZ-IEEE’2002, Honolulu, Hawaii, May 12–17 (2002) vol. 1, 685–689 11. Osegueda, R.A., Seelam, S.R., Holguin, A.C., Kreinovich, V., Tao, C.-W.: Statistical and Dempster-Shafer Techniques in Testing Structural Integrity of Aerospace Structures. International Journal of Uncertainty, Fuzziness, Knowledge-Based Systems (IJUFKS) 9 (2001) 749–758 12. Rabinovich, S.: Measurement Errors: Theory and Practice, American Institute of Physics, New York (1993) 13. Scott, L.: Identification of GIS Attribute Error Using Exploratory Data Analysis. Professional Geographer 46 (1994) 378–386 14. Vavasis, S.A.: Nonlinear optimization: complexity issues, Oxford University Press, N.Y. (1991) 15. Wadsworth, H.M. Jr. (ed.): Handbook of statistical methods for engineers and scientists, McGraw-Hill Publishing Co., N.Y. (1990) 16. Wen, Q., Gates, A.Q., Beck, J., Kreinovich, V., Keller, G.R.: Towards automatic detection of erroneous measurement results in a gravity database. Proceedings of the 2001 IEEE Systems, Man, and Cybernetics Conference, Tucson, Arizona, October 7–10 (2001) 2170–2175 17. Worden, K., Osegueda, R., Ferregut, C., Nazarian, S., George, D.L., George, M.J., Kreinovich, V., Kosheleva, O., and Cabrera, C.: Interval Methods in NonDestructive Testing of Material Structures. Reliable Computing 7 (2001) 341–352
On the Approximation of Centered Zonotopes in the Plane Svetoslav Markov 1 and Dalcidio Claudio2 1
Institute of Mathematics and Informatics Bulgarian Academy of Sciences, “Acad. G. Bonchev” Str., block 8 BG-1113 Sofia, Bulgaria
[email protected]
2
PUCRS, Faculdade de Informatica/matematica Av. Ipiranda, 6681 Predio 16 90619-900 Porto Alegre - RS, Brasil
[email protected]
Abstract. The present work is devoted to computation with zonotopes in the plane. Using ideas from the theory of quasivector spaces we formulate an approximation problem for zonotopes and propose an algorithm for its solution.
1 Introduction Zonotopes axe convex bodies with simple algebraic presentation: they are Minkowski sums of segments. By means of a translation (addition of a vector) any zonotope can be centered at the origin, therefore without loss of generality we can restrict our considerations to centered (origin symmetric) zonotopes. The latter are positive combinations of centered unit segments [7, 8]. A fixed system of centered unit segments generates a class of zonotopes consisting of all positive combinations of the unit segments. This class is closed under addition and multiplication by scalar and forms a quasilinear space [2, 5, 6]. A quasilinear space over the field of reals is an additive abelian monoid with cancellation law endowed with multiplication by scalars. Any quasilinear space can be embedded in a group; in addition a natural isomorphic extension of the multiplication by scalars leads to quasilinear spaces with group structure called quasivector spaces [4]. Quasivector spaces obey all axioms of vector spaces, but in the place of the second distributive law we have: if If, in addition, the elements satisfy the relation: then the space is called symmetric quasilinear space. Every quasivector space is a direct sum of a vector space and a symmetric quasivector space [3,4]. On the other side, the algebraic operations in vector and symmetric quasivector spaces are mutually representable. This enables us to transfer basic vector space concepts (such as linear combination, basis, dimension, etc.) to symmetric quasivector spaces. Let us also mention that symmetric quasivector spaces with finite basis are isomorphic to a canonic space similar I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 246–253, 2004. © Springer-Verlag Berlin Heidelberg 2004
On the Approximation of Centered Zonotopes in the Plane
247
to [4]. These results can be used for computations with zonotopes as then we practically work in a vector space. Computing with centered zonotopes is especially simple and instructive [1]. In the present work we demonstrated some properties of centered zonotopes in the plane. In particular, an approximation problem related to zonotopes has been formulated and solved by means of a numerical procedure and a MATLAB program. Our procedure allows us to present approximately any centered zonotope in the plane by means of a class of zonotopes over a given basis of centered segments.
2
Quasivector Spaces
By we denote the set of reals; we use the same notation for the linearly ordered field of reals For any integer denote by the set of all where The set forms a vector space under addition and multiplication by scalars. Every abelian monoid with cancellation law induces an abelian group where consists of all pairs (A, B) factorized by the congruence relation ~: (A, B) ~ (C, D) iff A + D = B + C, for all. Addition in is (A, B) + (C, D) = (A + C, B + D). The null element of is the class (Z, Z), we have (Z, Z) ~ (0,0). The opposite element to is opp(A,B) = (B,A). All elements of admitting the form (A, 0) are called proper and the remaining are improper. The opposite of a proper element is improper; opp(A, 0) = (0, A). Definition 1. Let be an abelian monoid with cancellation law. Assume that a mapping (multiplication by scalars) is defined on satisfying: i) ii) iii) iv) if The algebraic system is called a quasilinear space over Every quasilinear space Multiplication by scalars by means of:
can be embedded into a group is naturally extended from to
In the sequel we shall call quasilinear spaces of group structure, such as quasivector spaces, and denote their elements by lower case roman letters, e. g. Definition 2. [4] A quasivector space (over denoted abelian group with a mapping (multiplication by scalars) such that for if
is an
248
Svetoslav Markov and Dalcidio Claudio
Proposition 1. [3] Let be a quasilinear space over and be the induced abelian group. Let be multiplication by scalars defined by (1). Then is a quasivector space over Let a be an element of a quasivector space The operator is called negation; in the literature it is usually denoted We write note that may not generally hold. From we obtain that is We shall use the notation the latter operator is called dualization or conjugation. The relations imply shortly Thus, the symbolic notation can be used instead of and, for we can write resp. We also note that some vector space concepts, such as subspace, sum and direct sum are trivially extended to quasivector spaces [4]. Some rules for calculation in quasivector spaces are summarized in [4]. Example 1. For any integer the set of all with whenever forms a quasivector space over under the operations
This quasivector space is denoted by Negation in is the same as identity while the opposite operator is the same as conjugation:
The direct sum the quasivector space
of the
vector space is a quasivector space.
and
Example 2. The system of all convex bodies [7] in a real Euclidean vector space with addition: is an abelian monoid with cancellation law having as a neutral element the origin “0” of The system where is multiplication by real scalars defined by: is a quasilinear space (of monoid structure), that is the following four relations are satisfied: i) ii) iii) iv) if The monoid induces a group of generalized convex bodies According to Proposition 2.1. the space where is defined by (1) is a quasivector space [3]. A centrally symmetric convex body with center at the origin will be called centered convex body, cf. [7], p. 383. Centered convex bodies do not change under multiplication by –1.
Definition 3. Q is a quasivector space. An element with called linear. An element with ¬ is called origin symmetric.
is
On the Approximation of Centered Zonotopes in the Plane
249
Proposition 2. Assume that Q is a quasivector space. The subsets of linear and symmetric elements resp. form subspaces of The subspace is a vector space called the vector (linear) subspace of The space of centered elements is called the symmetric centered subspace of For symmetric elements the following relations are equivalent: The following theorem shows the important roles of symmetric quasivector spaces. Theorem 1. [4] For every quasivector space specifically, for every we have and Symmetric Quasivector Spaces. Let space over Define the operation
where
if
if
we have
More with unique
be a symmetric quasivector by
and
Theorem 2. [3], [4] Let be a symmetric quasivector space over Then with “·” defined by (5), is a vector space over
to
Note that for that is
centered, the element resp.
is the opposite
Linear Combinations. Assume that is a symmetric quasivector space and is the associated vector space from Theorem 2. We may transfer vector space concepts from such as linear combination, linear dependence, basis etc., to the original symmetric quasivector space For example, let be finitely many elements of The familiar linear combination in the induced vector space can be rewritten using (5) as
Thus (6) is a linear combination of in the symmetric quasivector space Similarly, the concepts of spanned subspace, linear (in)dependency, linear mapping, basis, dimension, etc. are defined, and the theory of vector spaces can be reformulated in [3,4]. Theorem 3. [4] Any symmetric quasivector space over elements, is isomorphic to
with a basis of
Svetoslav Markov and Dalcidio Claudio
250
3
Computation with Centered Zonotopes in the Plane
Using the theory of symmetric quasivector spaces that we briefly outlined in the previous section, we consider next a class of special convex bodies, namely centered zonotopes in the plane. Every unit vector in the plane defines a centered segment with endpoints
For scalar
denote is:
and
Multiplication of a unit centered segment
by a
More generally, multiplication of a centered segment by a scalar gives Note that more generally, (whereas, for comparison, we have, of course, Minkowski addition of colinear centered segments is To present Minkowski addition of noncolinear centered segments, assume and denote The points
define two noncolinear centered segments The Minis a centered quadrangle (parallelepiped) P with vertices where The perimeter of is and the area of P is kowski sum
Assume that we are given fixed centered unit vectors in cyclic anticlockwise order, such that A system
of centered unit vectors satisfying will be further called regular, the same notion will be used for the induced system of centered unit segments In particular, the system with is regular; for this system we have For segments segments
the vectors
induce the centered The positive combination of the
On the Approximation of Centered Zonotopes in the Plane
is a centered zonotope with 8], where
vertices:
251
[7,
The vertices given by (8) are lying in cyclic anticlockwise order in a half-plane between the vectors and Two centered zonotopes over same system are added by
Thus given a fixed regular system of centered unit segments the set of all zonotopes of the form is closed under Minkowski addition and multiplication by scalars and forms a quasilinear space (of monoid structure) [3,4]. Consider two zonotopes where are two distinct regular systems of centered unit segments. From the expression for the sum: we see that the vertices of B + C can be restored using (8), where is the union of the two sets (after proper ordering). Clearly the number of vertices of B + C equals (generally) the sum of the numbers of vertices of B and C, resp. If we want to use a fixed presentation of the zonotopes of the form (7), then we need to present (approximately) all zonotopes using one and the same system of centered unit segments.
4
An Approximation Problem
The Problem. Assume that a regular system of centered unit segments is given, which will be considered as basic. Assume that is a regular system of unit centered segments, distinct from the given system We want to approximate a given zonotope of the form by means of zonotopes from the class so that The Algorithm. Given are unit vectors that and nonnegative numbers
such defining
Svetoslav Markov and Dalcidio Claudio
252
a zonotope the zonotope
We want to find suitable values is an outer approximation of
We present the vector
where and the coefficients
such that that is
as
are the nearest basic unit vectors enclosing with are some nonnegative coeficients. Clearly, relations (9) define in a unique way. We note that if some of the equalities takes place, then one of the coefficients will be equal
to zero. Relations (9) imply the inclusions Subsequently we obtain
with some that can be effectively computed. It follows from (10) that the zonotope is an outer approximation of the zonotope From (10) one can compute the vertices of the zonotope by means of (8). It can be shown that the above algorithm produces an optimal approximation as regard the Hausdorff/integral metric. Roughly specking this follows from the fact that every single segment has been optimally approximated. Note that the area of the zonotope is This can be used to compute the area of the zonotope Relations (9), (10) define an algorithm leading to the construction of an outer approximation of Such an algorithm using a uniform mesh of centered segments in of the form has been realized in MATLAB. It has been demonstrated that when the number of mesh points increases, the zonotope approaches the original zonotope in Hausdorff sense. The Method of Support Functions. Let us compare our method with a corresponding method based on support functions. The support function of a set is this function is well defined by its values on the unit circle C. For we can easily calculate for any zonotope A. Let first A be a centered unit segment with We have Similarly we can write down the support function of the zonotope where We have The above approximation problem can be stated in terms of support functions as follows. Given some we want to approximate from above in the interval the function by means of a function of the class that is we need to find the in so that and approximates
On the Approximation of Centered Zonotopes in the Plane
253
Concluding Remarks From the theory of quasivector spaces it follows that we can compute with zonotopes as we do in vector spaces unless the zonotopes belong to the same space, that is to be presented by the same basic system. This makes it important to be able to approximately present any zonotope by a zonotope from a chosen basic class. We construct a simple algorithm for this purpose which can be effectively implemented in a software environment. Our approach is alternative to the approach based on support functions, extensively used in the literature on convex bodies and seems to be more direct and more simple. From the formulation of the approximation problem in terms of support functions we see that the latter formulation is in no way easier than the direct formulation in terms of zonotopes. We thus conclude that in some cases direct computation with zonotopes may be preferable that related computations based on support functions. Due to their simple presentation zonotopes can be used instead of more traditional convex bodies like boxes, parallelepipeds, ellipsoids etc.
Acknowledgments This work was supported in part by Bulgarian NSF grants I-903/99, MM1104/01, and the Swiss NSF Co-operation Agreement No. 7IP 65642 (Program SCOPES).
References 1. Akyildiz, A., Claudio, D., S., Markov: On the Linear Combinations of Symmetric Segments, Mathematics and Education of Mathematics, 2002 (Eds. E. Kelevedzhiev, P. Boyvalenkov), Institute of Mathematics and Informatics, Bulg. Acad. of Sci., Sofia, 2000, 321–326. 2. Kracht, M., G. Schröder: Eine Einfürung in die Theorie der quasilinearen Räume mit Anwendung auf die in der Intervallrechnung auftretenden Räume, Math.physik. Semesterberichte XX (1973) 226–242. 3. Markov, S.: On the Algebraic Properties of Convex Bodies and Some Applications, J. of Convex Analysis 7 (2000) 129–166. 4. Markov, S.: On Quasilinear Spaces of Convex Bodies and Intervals, Journal of Computational and Applied Mathematics, to appear. 5. Ratschek, H., G. Schröder: Über den quasilinearen Raum, in: Ber. Math.-Statist. Sekt. 65 (Forschungszentr. Graz, 1976) 1–23. 6. Ratschek, H., G. Schröder: Representation of Semigroups as Systems of Compact Convex Sets, Proc. Amer. Math. Soc. 65 (1977) 24–28. 7. Schneider, R.: Convex Bodies: The Brunn-Minkowski Theory, Cambridge Univ. Press, 1993. 8. Ziegler, G.: Lectures on Polytopes, Springer, 1994.
On Range Evaluation of Polynomials by Applying Interval Arithmetic Shinya Miyajima and Masahide Kashiwagi 61-404 Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan {shin,kashi}@kashi.info.waseda.ac.jp
Abstract. Interval arithmetic is able to be applied in the case that we evaluate the ranges of polynomials. When we evaluate the ranges of polynomials by applying the interval arithmetic, the problem that interval widths of the ranges increase extremely exists. Horner’s method is widely known as the evaluating method which mitigates this problem. The purpose of this paper is to propose the new methods which are able to mitigate this problem more efficiently than the Horner’s method. And in this paper, we show and compare the efficiencies of the each new method by the results of some numerical examples.
1 Introduction Interval arithmetic, e.g. [1], is able to be applied in the case that we evaluate the ranges of polynomials. When we evaluate the ranges of polynomials by applying the interval arithmetic, the problem that interval widths of the ranges increase extremely exists. Horner’s method is widely known as the evaluating method which mitigates this problem (which supply ranges whose interval widths are narrow). The purpose of this paper is to propose the seven new methods which are able to supply ranges whose interval widths are narrower than the interval width of ranges which the Horner’s method supplies. And in this paper, we show and compare the efficiencies of the each new method by the results of some numerical examples.
2 2.1
Conventional Methods Standard Method
In this method, range of polynomial of degree is calculated by the interval arithmetic without any contrivances. Interval widths of ranges increase extremely when this method is applied.
2.2
Horner’s Method
In this method, formula (1) is transformed as follows:
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 254–261, 2004. © Springer-Verlag Berlin Heidelberg 2004
On Range Evaluation of Polynomials by Applying Interval Arithmetic
3
255
New Methods
In this section, we propose seven new methods. Here, let the polynomial of degree be or and let the domain be
3.1
New Method 1
Let
be the center of the domain. Namely,
By introducing the
the
formula (1) is transformed as follows: When the formula (1) is transformed into the formula (3), following two efficiencies are expected. (efficiency 1) Generally, for a domain and its center following formula is satisfied: Therefore, following formula is satisfied: Note that (efficiency 2) By introducing
and radius of domain
are the interval widths of
and
is able to be described as follows:
Therefore, for an even number is satisfied:
following formula
In this way, negative part is able to be eliminated.
3.2
New Method 2
In this method, by introducing formed as follows:
the formula (3) is trans-
Note that and are the maximum even number and odd number which do not exceed And these and are described as follows:
By transforming the formula (3) into the formula (8) , Horner’s method is able to be applied maintaining most of two efficiencies of the new method 1. And this method is able to be applied in smaller time than the new method 1 because of application of Horner’s method.
256
3.3
Shinya Miyajima and Masahide Kashiwagi
New Method 3
Introducing
let the formula (3) be as follows:
In this method, the formula (11) is transformed as follows:
Note that
The range of the formula (12) is obtained by calculating the maximum value and the minimum value of each parenthesised formula (quadratic interval bounded function) successively. Concrete step of this method is as follows: Step1. Calculate
Step2. Let
and
which satisfies the formula (14).
be as follows:
And calculate following
and
Step3. The range is
3.4
New Method 4
In this method, the formula (11) is transformed as follows:
On Range Evaluation of Polynomials by Applying Interval Arithmetic
257
Note that
The range of the formula (16) is obtained by calculating the maximum value and the minimum value of each parenthesised formula (cubic interval bounded function) successively. Concrete step of this method is similar to that of the new method 3 mostly.
3.5
New Method 5
Let the quotient and the surplus when is divided by its derivative (degree: be (degree: 1) and (degree: And introducing these and following formula is satisfied:
Here, when we put the which satisfies into formula is satisfied by substituting for the formula (17) because
following
By this formula, the following formula is satisfied:
Therefore, the range of is able to be included even if the upper bound and the lower bound of the range of are nominated instead of the extreme values of in Accordingly, we have to calculate the upper bound and the lower bound of the range of The upper bound and the lower bound of the range of are able to be calculated by repeating the above mentioned operation and lowering the degree of until extreme values are able to be calculated easily. Concrete step of this method is as follows: Step1. Calculate and and nominate them for the upper bound and the lower bound of the range of Step2. Calculate the surplus when is divided by its derivative And calculate and and nominate them for the upper bound and the lower bound of the range of Step3. Repeat the step2 until becomes primary function or quadratic function. Step4. When becomes primary function or quadratic function, calculate the extreme values of in And calculate and and nominate them for the upper bound and the lower bound of the range of Step5. In the obtained candidates, the upper bound and the lower bound of the range of are the maximum value and the minimum value.
258
3.6
Shinya Miyajima and Masahide Kashiwagi
New Method 6
Introducing follows:
and its derivative
Here, when we put the
let
whichsatisfies
formula is satisfied by substituting
(degree:
into
be as
following
for the formula (20) because
By this formula, the following formula is satisfied:
Therefore, the range of is able to be included even if the upper bound and the lower bound of the range of are nominated instead of the extreme values of in Accordingly, we have to calculate the upper bound and the lower bound of the range of The upper bound and the lower bound of the range of are able to be calculated by repeating the above mentioned operation and lowering the degree of until extreme values are able to be calculated easily. Concrete step of this method is similar to that of the new method 5 mostly.
3.7
New Method 7
In this method, well narrow intervals which include satisfying are calculated by bisection method. Concrete step of this method is as follows: Step1. Calculate and and nominate them for the upper bound and the lower bound of the range of Step2. Let be the derivative of order of (i) In the case that Step2-1. Calculate and nominate it for the upper bound and the lower bound of the range of Step2-2. Let the intervals:
be the initial in-
tervals of the bisection method in the derivative of order operate the bisection method proper times and calculate and
And
the well narrow intervals which include the only one
solution of Step2-3. Calculate the upper bound and the lower bound of the range of which the standard method is applied for the intervals and and nominate them for the upper bound and the lower bound of the range of
On Range Evaluation of Polynomials by Applying Interval Arithmetic
259
Step2-4. Let the intervals: be the initial intervals of the bisection method in the derivative of order And operate the bisection method proper times and calculate the well narrow intervals which includes the only one solution of and calculate the upper bound and the lower bound of the range of which the standard method is applied for these intervals and nominate them for the upper bound and the lower bound of the range of Step2-5. Repeat the operation similar to step2-4 for the derivative of order and calculate the candidates of the upper bound and the lower bound of the range of (ii) In the case that Step2-1. Let the interval be the initial interval of the bisection method in the derivative of order And operate the bisection method proper times and calculate the well narrow interval which includes the only one solution of Step2-2. Calculate the upper bound and the lower bound of the range of which the standard method is applied for the intervals and nominate them for the upper bound and the lower bound of the range of Step2-3. Let the intervals: be the initial intervals of the bisection method in the derivative of order And operate the bisection method proper times and calculate the well narrow intervals which includes the only one solution of and calculate the upper bound and the lower bound of the range of which the standard method is applied for these intervals and nominate them for the upper bound and the lower bound of the range of Step2-4. Repeat the operation similar to step2-3 for the derivative of order and calculate the candidates of the upper bound and the lower bound of the range of Step3. In the obtained candidates, the upper bound and the lower bound of the range of are the maximum value and the minimum value.
4 Evaluation of the New Method In this section, we show and compare the efficiencies of the each new method by the results of some numerical examples. Here, computer environment is CPU: PentiumII 350MHz, memory: 128MB, OS: Free BSD 2.2.8 and compiler: g++ 2.7.2.1.
4.1
Numerical Examples
First, for the formula the formula (23) which the strict ranges is able to be calculated easily, the range, the interval width and the calculation time when
260
Shinya Miyajima and Masahide Kashiwagi
the each method are applied, the strict range and its interval width are shown in the table 1.
Note that the repetition time of the bisection method in the new method 7 is 10. Next, for the formula (24), polynomials of degree 10, the range, the interval width and the calculation time when the each new method are applied are shown in the table 2.
Note that the repetition time of the bisection method in the new method 7 is 30.
4.2
On the Result of the Numerical Example
By the numerical examples described in the section 4.1, the fact that the seven new methods are able to supply ranges whose interval widths are narrower than the range which Horner’s method supplies is confirmed although these new methods need longer calculating times than the Horner’s method. The new method 2
On Range Evaluation of Polynomials by Applying Interval Arithmetic
261
is superior to the new method 1 in both of the interval width and the calculating time. The new method 3 supplies the ranges whose interval width are narrower than that of the new method 2, but the new method 3 needs longer calculating time than the new method 2. Superiority of the new method 4 and the other new methods is different in numerical examples. The new method 6 possesses the same character. Generally, the new method 5 is superior to the new method 6 in both of the interval width and the calculating time. The new method 7 supplies the range whose interval width is narrowest in all methods, but this method needs extremely long calculating time.
5
Conclusion
In this paper, we proposed the seven new methods which are able to supply the ranges whose interval widths are narrower than the interval width of the ranges which the Homer’s method supplies. And we showed and compared the efficiencies of the each new method by the results of some numerical examples.
References 1. A. Neumaier: “Interval methods for systems of equations”, Cambrige University
Press (1990)
Sharp Bounds for Strains and Stresses in Uncertain Mechanical Models* Evgenia D. Popova1, Maria Datcheva2,3, Roumen Iankov 2, and Tom Schanz3 1
Inst. of Mathematics & Informatics, Bulgarian Academy of Sciences 2 Institute of Mechanics, Bulgarian Academy of Sciences Acad. G. Bonchev str., block 8 1, block 42 , BG-1113 Sofia, Bulgaria
[email protected], {iankovr,datchevam}@yahoo.com 3
Bauhaus-Universität Weimar, Germany
[email protected]
Abstract. Many mechanical systems, involving interval uncertainties and modelled by finite element method, can be described by parameter dependent systems of linear interval equations and response quantities depending on the system solution. A newly developed hybrid interval approach for solving parametric interval linear systems is applied to a 2D engineering model with interval parameters and the results are compared to other interval methods. A new technique providing very sharp bounds for the response quantities, such as element strains and stresses, is developed. The sources for overestimation when dealing with interval computations are demonstrated on a numerical plain strain example.
1 Introduction All engineering design problems involve imprecision, approximation, or uncertainty to varying degrees [1,3]. In particular, mathematical models in environmental geomechanics cover a broad class of problems involving deformation of geomaterials. Since soil, rock and clay materials are natural ones, there is uncertainty in the material properties [6]. When the information about an uncertain parameter in form of a preference or probability function is not available or not sufficient then the interval analysis can be used most conveniently [3]. Many mechanical systems, modelled by finite element method (FEM), can be described by parameter dependent systems of linear equations. If some of the parameters are uncertain but bounded, the problem can be transformed into a parametric interval linear system which should be solved appropriately to bound the mechanical system response. This technique is usually called Interval Finite Element Method. The efforts for developing suitable interval FE methods started at mid nineties and attract considerable attention [1]. Contrary to structural mechanics, where significant effort is devoted to solving FE models involving interval uncertainties, this paper makes a first attempt to consider a 2D model with interval parameters related to soil physics and soil mechanics, applied to *
This work was supported by the NATO CLG 979541 and the Bulgarian National Science Fund under grants No. I-903/99, MM-1301/03.
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 262–269, 2004. © Springer-Verlag Berlin Heidelberg 2004
Sharp Bounds for Strains and Stresses in Uncertain Mechanical Models
263
engineering. Since safety is an issue in environmental geomechanics, the goal is to describe the response of the system under a worst case scenario of uncertain parameters varying within prescribed bounds. Like in FE analysis of most engineering systems [1,3], the problem considered here can be stated as follows: Given a system of linear equations, that describe the response of a discretized structural mechanical model, in the form
and a response vector where is the global stiffness matrix of the system, is the global load vector (which might depend on uncertain parameters), U is the nodal displacement vector of the total discrete model, and is the vector of uncertain input parameters which are described as intervals where denotes the lower bound and denotes the upper bound for the corresponding parameter value. R is the vector of response quantities which can be expressed in terms of nodal displacements, U, of the model. In this work we consider response quantities: element strains, and element stresses,
where is the element parameter vector which is, generally, a subvector of the global parameter vector is the subvector of element nodal displacements, amd matrices B, D are defined in Sec. 3. The problem is to find the range of each response quantity due to the uncertainty present in the input parameters. A key issue is how to avoid the overestimation effect caused by the dependency problem in interval analysis. The solution of parametric linear interval equations (1) and subsequent estimation of the response parameters (3), (4) is discussed in this work. We extend the hybrid approach for solving parametric interval linear systems [2] to the subsequent bounding of the response quantities – element strains and stresses.
2 2.1
Interval Methods Solving Parametric Interval Linear Systems
Linear algebraic systems (1) usually involve complicated dependencies between the parameters involved in the stiffness matrix and the load vector In this paper we assume that these dependencies are affine-linear, that is
264
Evgenija Popova et al.
where dimension of (1). When the values from given intervals
are numerical vectors, with being the parameters take arbitrary the solution of (1) is a set
called parametric solution set (PSS). The PSS is a subset of, and has much smaller volume than the corresponding non-parametric solution set The simplest example of dependencies is when the matrix is symmetric. Since the solution sets have a complicated structure which is difficult to find, we look for the interval hull whenever is a nonempty bounded subset of or for an interval enclosure [U] of There are many works devoted to interval treatment of uncertain mechanical systems [1]. A main reason for some conservative results that have been obtained is that the interval linear systems (1), involving more dependencies than in a symmetric matrix, are solved by methods for nonparametric interval systems (or for symmetric matrices). Although designed quite long ago, the only available iterative method [5] for guaranteed enclosure of the PSS seems to be not known to the application scientists, or at least not applied. The parametric Rump’s method is a fixed-point method applying residual iteration and accounting for all the dependencies between parameters. As it will be shown in Sec. 3, this parametric method produces very sharp bounds for narrow intervals but tends to overestimate the PSS hull with increasing the interval tolerances and the number of parameters. To overcome this deficiency, we designed a new hybrid approach for sharp PSS enclosures, that combines parametric residual iteration and exact bounds based on monotonicity properties [2]. Finding very sharp bounds (or exact in exact arithmetic) for the PSS is based on the monotonicity properties of the analytic solution For large real-life problems, the computer aided proof of the monotonicity of with respect to each parameter is based on taking partial derivatives on (1) [4]. This leads to a parametric interval linear system
where
and
is a PSS enclosure. Let for fixed
Define numerical vectors
Then the exact bounds of the PSS,
and
componentwise
Sharp Bounds for Strains and Stresses in Uncertain Mechanical Models
265
can be obtained by solving at most point linear systems. This approach is extended in the next subsection for sharp bounding of the response quantities.
2.2
Bounding the Response Quantities
A naive interval approach in finding the range of the response quantities R(U), such as element strains (3) and element stresses (4), under the uncertainties in input parameters is to replace the exact hull (or its enclosure) [U], found by some method, into the expressions (3), (4) and to perform all interval operations. Since the nonzero components of depend on the uncertain parameters, the range computation of implicitly involves the dependency problem implying a huge overestimation of the exact range, as will be shown in Sec. 3. To get sharp range estimations, we use the monotonicity properties of the responses. Taking partial derivatives on (3), (4) we obtain
where and
is a subvector of the solution of (5), is taken from (7). Let for fixed are the dimensions of respectively.
is the solution of (3), where
Note, that the sets differ between themselves and for every fixed Define the numerical vectors for every and for every analogously to (6). Then the exact bounds of are obtained as Note, that a rigorous very sharp enclosure can be obtained if instead of we take the corresponding components of the rigorous interval enclosures of the solution to the corresponding point systems (1). For fixed
However, the above computation of the range is rigorous only in exact arithmetic. A rigorous and very sharp range enclosure of in floating-point arithmetic should use rigorous interval enclosures of the solution to the corresponding point linear systems (1) and computations with validated interval arithmetic.
266
3
Evgenija Popova et al.
Numerical Example
We consider an elastic material model based on the following assumptions: small strain theory is applied to describe the deformation in material; the latter is deformed elastic; material properties are isotropic; the temperature, creep and time dependent effects are not taken into account; the Elastic moduli in the material properties are considered to be uncertain and varying within prescribed bounds. For example, the prediction of soil behavior due to environmental changes, experimental procedure and measurement errors of such properties is a typical situation. Let V be a representative volume and S be the boundary of this volume. where is a boundary part with prescribed displacement, and is a boundary part with prescribed stresses. The governing equations are as follows:
where are the displacements, is a strain tensor, is the body force, is the stress tensor, is the elastic tensor, and are the prescribed displacements and distributed load resp., is the outward normal direction and are the corresponding directional cosines. The equilibrium equation (9), relating the stress vector to the body force and the boundary force specified at the boundary of V, is formulated in terms of the unknown displacement vector Using the principle of virtual work, the general equilibrium statement can be written in a variational form [7] as:
for a virtual displacement vector The resulting system of equations is created using mathematical formulations for FEM based on variational technique. A typical FE in the 2D plane strain case is defined by nodes Let the displacement at any point within the element be approximated by linear shape functions and nodal unknown displacement With displacements, known at all points within an element, the strains and stresses at any point of the FE can be determined by nodal unknowns. These relationships can be written in matrix notations [8] as:
where N is a matrix of shape functions, L is a matrix of a suitable differential operators, is the elastic matrix involving Elastic modulus and Poisson’s
Sharp Bounds for Strains and Stresses in Uncertain Mechanical Models
267
Fig. 1. 2D plane strain FEM model with 4 finite elements, where and each FE has a different material property
ratio, denotes a parameter (or parameter vector), related to the material properties, to be considered as uncertain in the subsequent analysis. Substituting Eqs.(11) and (12) into eq. (10) and taking into account that eq. (10) is valid for any virtual displacement, for an element we obtain a system of parametric linear equations
where is the element stiffness matrix which is function of the uncertain material properties, is the element load vector. It could also depend of some uncertain parameters. The final parametric global stiffness matrix and global load vector based on FEM matrices (13) is obtained by using an element-by-element technique [8]. We consider a 2D plain strain example presented on Fig. 1. The model is discretized with four triangular FEs and the kinematic boundary conditions are presented on the figure. The corresponding material and geometrical data are also given therein. It is assumed that in each FE the Young’s modulus, is an independent uncertain parameter varying within an interval. Three different parametric interval linear systems (1) are considered that involve: a) four uncertain parameters – Young’s modulus for each FE of the model is uncertain and varies independently in an interval b) two uncertain parameters varying in and varying in c) one parameter – assuming that the four FEs of the model have same Elastic modulus, E, varying in an interval where is the degree of uncertainty measured in % from the corresponding nominal value. The three parametric systems are solved for different values of the tolerance and 0.1%.
268
Evgenija Popova et al.
A Mathematica package IntervalComputations‘LinearSystems’ [2], providing a variety of functions for solving parametric and nonparametric interval linear systems in validated interval arithmetic, and supporting the hybrid interval approach, is used for the computations. Due to the boundary conditions, only the 6th, 8th, 10th, and 12th degree of freedom (DOF) of are nonzero. We compare the results, obtained by the parametric iteration method, to the exact interval hull of the displacements. The overestimation of Rump’s enclosure is measured in % as where is the exact hull of the corresponding parametric solution set, [U] is the interval enclosure, obtained by the parametric iteration method, and is the width of an interval, defined as The percentage by which the iterative solution enclosure overestimates the exact hull of the corresponding parametric solution set is presented in Table 1. The computationally efficient iterative procedure gives sharp enclosures for small number of parameters and thin intervals for the uncertainties. However, the overestimation grows with increasing the number of parameters or the width of the intervals. For the model with 4 uncertain parameters, a straightforward interval evaluation of (3) and (4) was done basing on interval estimations for the element displacements. Two interval estimations for the element displacements were used: interval enclosure, obtained by the parametric iteration method, and the exact interval hull. The results for the element strains are presented in Table 2. The results for the first component of the strains are not given since the component
Sharp Bounds for Strains and Stresses in Uncertain Mechanical Models
269
is zero. A huge overestimation of the strains range is demonstrated, which is due to the parameter dependencies between the components of The dependency, accumulated in the interval enclosure cannot be taken into account by the interval computation It is remarkable, that the difference in the interval estimations of U (by Rump’s method and the exact hull) is obscured by the inherited dependencies between the displacement components. The naive range estimation of the element stressess showed similar results. Note that the expression (4) involves additional dependencies due to the parameters in the matrix Accounting for the dependencies in reduces the overestimation by 30–50%. Since the considered model is small, the exact interval hulls and the exact ranges were computed by symbolic-algebraic methods in the environment of Mathematica. Our hybrid approach produced very sharp bounds for both the solution set of the displacements system and the ranges of the response parameters. The difference to the corresponding exact values was within
4
Conclusions
The interval analysis methods, presented in this paper, can be applied to all engineering problems whose behavior is governed by parametric interval linear systems and response quantities of type (1), (2). The hybrid interval approach, proposed in Sec. 2, is based on numerical proof of monotonicity properties and on a fast iteration method for enclosing PSS. By using validated interval computations, this approach has the additional property that, owing to an automatic error control mechanism, every computed result is guaranteed to be correct. The Mathematica package [2] should be extended by suitable functions automating the computations of the response quantities. Further research toward improving the computational efficiency of the method would allow large industrial applications.
References 1. Muhanna R.; Mullen R.: Short Review for the Latest Development of Interval FEM. 2002. http://www.gtrep.gatech.edu/~rmuhanna/IFEM-review.pdf 2. Popova, E.: Parametric Interval Linear Solver. Numerical Algorithms, submitted. 3. Rao S. S.; Berke L.: Analisys of Uncerttain Structural Systems Using Interval Analysis. AIAA Journal, 35 (1997) 4, 727-735. 4. Rohn, J.: Interval Linear Equations with Dependent Coefficients. Reliable Computing mailing list, 2002. http://www.ms.mff.cuni.cz/~rohn/letter/letter.ps 5. Rump, S.: Verification Methods for Dense and Sparse Systems of Equations. In: Herzberger, J.(ed.): Topics in Validated Computations. Elsevier Science B. V. (1994) 63-135. 6. Schanz T.: Zur Modellierung des mechanischen Verhaltens von Reibungsmaterialien. Habilitationsschrift, Institut fuer Geotechnik, Stuttgart, 1998. 7. Washizu K.: Variational Methods in Elasticity and Plasticity. Pergamon Press, New York, 1982. 8. Zienkiewicz O. C.: The Finite Element Method in Engineering Science. McGrawHill, London, 1971.
One Approximation to the Reachable Sets of Strongly Convex Differential Inclusions Nedka Pulova1 and Vladimir Pulov2 1
Department of Mathematics, Technical University-Varna 2 Department of Physics, Technical University-Varna Studentska str. 1, Varna 9010, Bulgaria
[email protected]
Abstract. This paper deals with a set-valued version of the two-stage Runge-Kutta method with coefficients satisfying certain conditions. Applying it to a differential inclusion which right-hand side is a strongly convex set-valued map, we obtain third order local accuracy of the approximate reachable sets.
1 Introduction We consider a differential inclusion
where is a set-valued map An absolutely continuous function which satisfies (1) for a. is called solution of (1). Different discretization methods for differential inclusions are available [1,2]. Most of the approaches are directly inspired from similar schemes for differential equations. In this paper we consider a set-valued version of the explicit two-stage Runge-Kutta method
where are real coefficients, We introduce some notations and definitions. Consider the differential inclusion (1) with F defined in a neighborhood of a compact convex set The reachable set is the set of all points for which there is a solution of (1) on starting from such that The discrete-time reachable set is defined as
I. Lirkov et al. (Eds.): LSSC 2003, LNCS 2907, pp. 270–275, 2004. © Springer-Verlag Berlin Heidelberg 2004
One Approximation to the Reachable Sets
271
The Hausdorff distance between two convex and compact sets is given by
where
stands for the support function of the compact set is the scalar product in The set is called strongly convex, if there exists
If Y is strongly convex, and such that
such that
imply
then there is a unique element We shall also use the notations
Following the paper of Veliov [3], where under the assumptions (see below) A1–A3 (resp. A1–A3 and B) an estimate (resp. for the local error of the Heun scheme is obtained, we consider the Runge-Kutta method (2) with coefficients satisfying the conditions
Without any additional suppositions we achieved the same order of accuracy of the approximate reachable set (3). A1. F is compact valued and there is convex for every and A2. in in A3.
is
is differentiable with respect to and is Lipschitz continuous and is Lipschitz continuous in uniformly is compact,
B.
2
such that
and
is Lipschitz continuous in
for every uniformly in
Main Result
Theorem. Let the assumptions A1–A3 and the conditions (4) hold. Then for a given compact set and there exists a constant such that for every holds, then
and in (5) can be replaced with
If, in addition, the assumption B
272
Nedka Pulova and Vladimir Pulov
We need the following two lemmas. Lemma 1. Let A1–A3 hold. Then given exists a constant such that
for every
and
there
and
Lemma 2. Let A1–A3 and (4) be fulfilled. Then for a given compact set and there exists a constant such that
for every holds, then
and in (6) can be replaced with
If, in addition, the assumption B
Proof (of Lemma 1). The assumptions A1–A3 imply Lipschitz continuity of F, hence the set is bounded. Let be fixed. There exist such that
where
Denote
Obviously and are convex sets. Since the proof of Lemma 1 is obtained by estimating in succession
and
One Approximation to the Reachable Sets
Proof (of Lemma 2). Assumption A2 implies that given there is such that the equation
has a unique solution on satisfying Using successively we obtain
for every
273
and
and
Taking into account that, under the special choice (4) of the coefficients, it follows the estimate
we arrive at
274
Nedka Pulova and Vladimir Pulov
It is proven in [3] (Lemma 4, formula (16)), that on the assumptions A2 and A3, for every compact and there exists such that
for every and In the remainder of the proof we use the following trapezoidal formula for numerical integration of
uniformly in and The latter is obtained in a standard way by linearly interpolating at the points and considering the (resp. Lipschitz) continuity of As it is seen from (see [3])
(since
is a strongly convex set, and thus implying uniqueness of the corresponding continuity properties of come as a consequence of the continuous differentiability of A2, and another result proven in [3] (Lemma 3 in [3]): The mapping is Lipschitz continuous in uniformly in and continuous in uniformly in when and
continuity means that there is a constant L such that for every and as above.) Thus, by applying Lemma 3 from [3] (resp. Lemma 3 from [3] in combination with the assumption B) we complete the proof. Proof (of the Theorem). It follows from Lemma 2 that
Making use of Lemma 1 and the estimate (see [3], Lemma 2, formula (12))
One Approximation to the Reachable Sets
275
by utilizing the triangular inequality
we complete the proof.
References 1. Dontchev, A. and Lempio, F.: Difference methods for differential inclusions: A survey. SIAM Review 34 (1992) 263–294 2. Lempio, F. and Veliov V.: Discrete approximations of differential inclusions. Bayreuter Mathematische Schriften, Heft 54 (1998) 149–232 3. Veliov, V.: Second order discrete approximations to
